Changeset 9054:7fa8073a3557 in orange


Ignore:
Timestamp:
10/04/11 18:08:56 (3 years ago)
Author:
ales_erjavec <ales.erjavec@…>
Branch:
default
Convert:
d3c078f2b7f03adbadb1b004b23d4e284d8e5ea9
Message:

Added MultiClassSVMLearner and documentation for LinearSVMLearner,
Made changed to the module documentation to more clearly differentiate between LibSVM and LIBLINEAR learners.

Location:
orange/Orange/classification/svm
Files:
2 edited

Legend:

Unmodified
Added
Removed
  • orange/Orange/classification/svm/__init__.py

    r9025 r9054  
    88********************************* 
    99 
    10 This module wraps the `LibSVM library 
    11 <http://www.csie.ntu.edu.tw/~cjlin/libsvm/>`_, a library for `support vector 
    12 machines <http://en.wikipedia.org/wiki/Support_vector_machine>`_ (SVM). SVM  
    13 learners from LibSVM behave like ordinary Orange learners and can be 
    14 used as Python objects for training, classification and evaluation. The 
    15 implementation supports Python-based kernels, that can be plugged-in the 
    16 LibSVM. 
     10This is a module for `Support Vector Machine`_ (SVM) classification. It 
     11exposes the underlying `LibSVM`_ and `LIBLINEAR`_ library in a standard 
     12Orange Learner/Classifier interface. 
     13 
     14Choosing the right learner 
     15========================== 
     16 
     17Choose an SVM learner suitable for the problem. 
     18:obj:`SVMLearner` is a general SVM learner. :obj:`SVMLearnerEasy` will 
     19help with the data normalization and parameter tuning. Learn with a fast 
     20:obj:`LinearSVMLearner` on data sets with a large number of features.  
    1721 
    1822.. note:: SVM can perform poorly on some data sets. Choose the parameters  
    1923          carefully. In cases of low classification accuracy, try scaling the  
    20           data and try with different parameters. :obj:`SVMLearnerEasy` class does this  
    21           automatically (it is similar to the `svm-easy.py` script in the LibSVM  
    22           distribution). 
     24          data and experiment with different parameters. 
     25          :obj:`SVMLearnerEasy` class does this automatically (it is similar 
     26          to the `svm-easy.py` script in the LibSVM distribution). 
     27 
    2328           
    24 SVM learners 
    25 ============ 
    26  
    27 Choose an SVM learner suitable for the problem. :obj:`SVMLearner` is a  
    28 general SVM learner. Use :obj:`SVMLearnerSparse` to learn from the  
    29 meta attributes. :obj:`SVMLearnerEasy` helps with 
    30 the data normalization and parameter tuning. Learn with a fast  
    31 :obj:`LinearLearner` on data sets with a large number of features. 
     29SVM learners (from `LibSVM`_) 
     30============================= 
     31 
     32The most basic :class:`SVMLearner` implements the standard `LibSVM`_ learner 
     33(if you have used the `svm-train` command-line tool this will be familiar). 
     34It supports four built-in kernel types (Linear, Polynomial, RBF and Sigmoid). 
     35Additionally kernel functions defined in Python can be used instead.  
     36 
     37.. note:: For learning from ordinary :class:`Orange.data.Table` use the 
     38    :class:`SVMLearner`. For learning from sparse dataset (i.e. 
     39    data in `basket` format) use the :class:`SVMLearnerSparse` class. 
     40 
     41.. autoclass:: Orange.classification.svm.SVMLearner 
     42    :members: 
     43 
     44.. autoclass:: Orange.classification.svm.SVMLearnerSparse 
     45    :members: 
     46    :show-inheritance: 
     47     
     48.. autoclass:: Orange.classification.svm.SVMLearnerEasy 
     49    :members: 
     50    :show-inheritance: 
    3251 
    3352The next example shows how to use SVM learners and that :obj:`SVMLearnerEasy`  
    3453with automatic data preprocessing and parameter tuning  
    35 outperforms :obj:`SVMLearner` with the default :obj:`~SVMLearner.nu` and :obj:`~SVMLearner.gamma`:  
     54outperforms :obj:`SVMLearner` with the default :obj:`~SVMLearner.nu` and :obj:`~SVMLearner.gamma`:   
    3655     
    3756.. literalinclude:: code/svm-easy.py 
    3857 
    39 .. autoclass:: Orange.classification.svm.SVMLearner 
    40    :members: 
     58 
    4159    
    42 .. autoclass:: Orange.classification.svm.SVMLearnerSparse 
    43    :members: 
     60Linear SVM learners (from `LIBLINEAR`_) 
     61======================================= 
     62 
     63The :class:`LinearSVMLearner` learner is more suitable for large scale 
     64problems as it is significantly faster then :class:`SVMLearner` and its 
     65subclasses. A down side is it only supports a linear kernel (as the name 
     66suggests) and does not support probability estimation for the 
     67classifications. Furthermore a Multi-class SVM learner 
     68:class:`MultiClassSVMLearner` is provided. 
    4469    
    45 .. autoclass:: Orange.classification.svm.SVMLearnerEasy 
     70.. autoclass:: Orange.classification.svm.LinearSVMLearner 
    4671   :members: 
    4772    
    48 .. autoclass:: Orange.classification.svm.LinearLearner 
     73.. autoclass:: Orange.classification.svm.MultiClassSVMLearner 
    4974   :members: 
    5075    
     76    
     77SVM Based feature selection and scoring 
     78======================================= 
     79 
     80.. autoclass:: Orange.classification.svm.RFE 
     81 
     82.. autoclass:: Orange.classification.svm.Score_SVMWeights 
     83    :show-inheritance: 
     84  
     85  
    5186Utility functions 
    52 ----------------- 
     87================= 
     88 
     89Some utility functions which are used in this module but might also 
     90be of use to you. 
    5391 
    5492.. automethod:: Orange.classification.svm.max_nu 
     
    5896.. automethod:: Orange.classification.svm.table_to_svm_format 
    5997 
    60 The following example shows how to get linear SVM weights:  
    61  
     98The following example shows how to get linear SVM weights: 
     99     
    62100.. literalinclude:: code/svm-linear-weights.py     
    63101 
    64 SVM-derived feature weights 
    65 --------------------------- 
    66  
    67 .. autoclass:: Orange.classification.svm.Score_SVMWeights 
    68    :members: 
    69102 
    70103.. _kernel-wrapper: 
     
    73106=============== 
    74107 
    75 Kernel wrappers are used to build custom kernels. All wrapper constructors take one 
    76 or more Python functions (`wrapped` attribute) to wrap. The function must be a 
    77 positive definite kernel, taking two floating point parameters of type double, and return a  
    78 floating point number.   
     108Kernel wrappers are helper classes used to build custom kernels for use 
     109with :class:`SVMLearner` and subclasses. All wrapper constructors take 
     110one or more Python functions (`wrapped` attribute) to wrap. The  
     111function must be a positive definite kernel, taking two arguments of  
     112type :class:`Orange.data.Instance` and return a float. 
    79113 
    80114.. autoclass:: Orange.classification.svm.kernels.KernelWrapper 
     
    102136   :members: 
    103137 
    104 .. autoclass:: Orange.classification.svm.kernels.BagOfWords 
    105    :members: 
    106  
    107138Example: 
    108139 
    109140.. literalinclude:: code/svm-custom-kernel.py 
    110  
    111  
    112  
    113 SVM-based recursive feature elimination 
    114 ======================================= 
    115  
    116 .. autoclass:: Orange.classification.svm.RFE 
    117    :members: 
    118141 
    119142 
     
    125148.. _vehicle.tab: code/vehicle.tab 
    126149 
     150.. _`Support Vector Machine`: http://en.wikipedia.org/wiki/Support_vector_machine 
     151.. _`LibSVM`: http://www.csie.ntu.edu.tw/~cjlin/libsvm/ 
     152.. _`LIBLINEAR`: http://www.csie.ntu.edu.tw/~cjlin/liblinear/ 
     153 
    127154""" 
    128155 
     
    145172                        SVMClassifier, \ 
    146173                        SVMClassifierSparse 
    147  
    148 # ORANGE Support Vector Machines 
    149 # This module was written by Ales Erjavec 
    150 # and supersedes an earlier one written by Alex Jakulin (jakulin@acm.org), 
    151 # based on: Chih-Chung Chang and Chih-Jen Lin's 
    152 # LIBSVM : a library for support vector machines 
    153 # (http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.ps.gz) 
    154  
    155 #from Orange.misc import _orange__new__ 
    156  
     174                         
     175from Orange.preprocess import Preprocessor_impute, \ 
     176                              Preprocessor_continuize, \ 
     177                              Preprocessor_preprocessorList, \ 
     178                              DomainContinuizer 
     179 
     180from Orange.misc import _orange__new__ 
     181     
    157182def _orange__new__(base=Orange.core.Learner): 
    158183    """Return an orange 'schizofrenic' __new__ class method. 
     
    168193    from functools import wraps 
    169194    @wraps(base.__new__) 
    170     def _orange__new_wrapped(cls, data=None, **kwargs): 
     195    def _orange__new_wrapped(cls, data=None, weight_id=None, **kwargs): 
    171196        self = base.__new__(cls, **kwargs) 
    172197        if data: 
    173198            self.__init__(**kwargs) 
    174             return self.__call__(data) 
     199            return self.__call__(data, weight_id) 
    175200        else: 
    176201            return self 
     
    178203 
    179204def max_nu(data): 
    180     """Return the maximum nu parameter for Nu_SVC support vector learning 
    181      for the given data.  
    182      
    183     :param data: data with continuous features 
     205    """Return the maximum nu parameter for Nu_SVC support vector learning  
     206    for the given data table.  
     207     
     208    :param data: Data with discrete class variable 
    184209    :type data: Orange.data.Table 
    185210     
     
    197222     
    198223class SVMLearner(_SVMLearner): 
    199     """:param svm_type: defines the SVM type (can be C_SVC, Nu_SVC  
     224    """ 
     225    :param svm_type: defines the SVM type (can be C_SVC, Nu_SVC  
    200226        (default), OneClass, Epsilon_SVR, Nu_SVR) 
    201227    :type svm_type: SVMLearner.SVMType 
     
    203229        (can be kernels.RBF (default), kernels.Linear, kernels.Polynomial,  
    204230        kernels.Sigmoid, kernels.Custom) 
    205     :type kernel_type: kernel function, see :ref:`kernel-wrapper` 
     231    :type kernel_type: SVMLearner.Kernel 
    206232    :param degree: kernel parameter (for Polynomial) (default 3) 
    207233    :type degree: int 
     
    212238    :type coef0: int 
    213239    :param kernel_func: function that will be called if `kernel_type` is 
    214         `Custom`. It must accept two :obj:`Orange.data.Instance` arguments and 
    215         return a distance between the instances. 
     240        `kernels.Custom`. It must accept two :obj:`Orange.data.Instance` 
     241        arguments and return a float (see :ref:`kernel-wrapper` for some 
     242        examples). 
    216243    :type kernel_func: callable function 
    217     :param C: C parameter for C_SVC, Epsilon_SVR, Nu_SVR 
     244    :param C: C parameter for C_SVC, Epsilon_SVR and Nu_SVR 
    218245    :type C: float 
    219246    :param nu: Nu parameter for Nu_SVC, Nu_SVR and OneClass (default 0.5) 
     
    221248    :param p: epsilon in loss-function for Epsilon_SVR 
    222249    :type p: float 
    223     :param cache_size: cache memory size in MB (default 100) 
     250    :param cache_size: cache memory size in MB (default 200) 
    224251    :type cache_size: int 
    225252    :param eps: tolerance of termination criterion (default 0.001) 
     
    231258        (default True) 
    232259    :type shrinking: bool 
    233     :param weights: a list of class weights 
    234     :type weights: list 
     260    :param weight: a list of class weights 
     261    :type weight: list 
    235262     
    236263    Example: 
    237264     
    238265        >>> import Orange 
     266        >>> from Orange.classification import svm 
     267        >>> from Orange.evaluation import testing, scoring 
    239268        >>> table = Orange.data.Table("vehicle.tab") 
    240         >>> svm = Orange.classification.svm.SVMLearner() 
    241         >>> results = Orange.evaluation.testing.cross_validation([svm], table, folds=5) 
    242         >>> print Orange.evaluation.scoring.CA(results) 
     269        >>> learner = svm.SVMLearner() 
     270        >>> results = testing.cross_validation([learner], table, folds=5) 
     271        >>> print scoring.CA(results) 
    243272     
    244273    """ 
     
    281310 
    282311    def __call__(self, data, weight=0): 
    283         """Construct a SVM classifier. 
     312        """Construct a SVM classifier 
    284313         
    285314        :param table: data with continuous features 
    286315        :type table: Orange.data.Table 
     316         
    287317        :param weight: refer to `LibSVM documentation  
    288318            <http://http://www.csie.ntu.edu.tw/~cjlin/libsvm/>`_ 
     
    291321         
    292322        examples = Orange.core.Preprocessor_dropMissingClasses(data) 
     323        class_var = examples.domain.class_var 
    293324        if len(examples) == 0: 
    294325            raise ValueError("Example table is without any defined classes") 
     326         
     327        # Fix the svm_type parameter if we have a class_var/svm_type mismatch 
    295328        if self.svm_type in [0,1] and \ 
    296         examples.domain.classVar.varType!=Orange.data.Type.Discrete: 
    297             self.svm_type+=3 
     329            isinstance(class_var, Orange.data.variable.Continuous): 
     330            self.svm_type += 3 
    298331            #raise AttributeError, "Cannot learn a discrete classifier from non descrete class data. Use EPSILON_SVR or NU_SVR for regression" 
    299332        if self.svm_type in [3,4] and \ 
    300         examples.domain.classVar.varType==Orange.data.Type.Discrete: 
    301             self.svm_type-=3 
     333            isinstance(class_var, Orange.data.variable.Discrete): 
     334            self.svm_type -= 3 
    302335            #raise AttributeError, "Cannot do regression on descrete class data. Use C_SVC or NU_SVC for classification" 
    303         if self.kernel_type==4 and not self.kernel_func: 
    304             raise AttributeError, "Custom kernel function not supplied" 
    305         ################################################## 
    306 #        if self.kernel_type==4:     #There is a bug in svm. For some unknown reason only the probability model works with custom kernels 
    307 #            self.probability=True 
    308         ################################################## 
     336        if self.kernel_type == kernels.Custom and not self.kernel_func: 
     337            raise ValueError("Custom kernel function not supplied") 
     338         
    309339        nu = self.nu 
    310         if self.svm_type == SVMLearner.Nu_SVC: #is nu feasibile 
     340        if self.svm_type == SVMLearner.Nu_SVC: #is nu feasible 
    311341            max_nu= self.max_nu(examples) 
    312342            if self.nu > max_nu: 
     
    329359            data = self._normalize(data) 
    330360            svm = self.learner(data) 
    331 #            if self.: 
    332 #                return SVMClassifierWrapper(svm) 
    333 #            else: 
    334361            return SVMClassifierWrapper(svm) 
    335362        return self.learner(data) 
     
    338365    def tune_parameters(self, data, parameters=None, folds=5, verbose=0,  
    339366                       progress_callback=None): 
    340         """Tune parameters on given data using  
     367        """Tune the ``parameters`` on given ``data`` using  
    341368        cross validation. 
    342369         
     
    357384            >>> svm = Orange.classification.svm.SVMLearner() 
    358385            >>> svm.tune_parameters(table, parameters=["gamma"], folds=3) 
    359              
     386                     
    360387        """ 
    361388         
     
    389416    def _normalize(self, data): 
    390417        dc = Orange.core.DomainContinuizer() 
    391         dc.classTreatment = Orange.core.DomainContinuizer.Ignore 
    392         dc.continuousTreatment = Orange.core.DomainContinuizer.NormalizeBySpan 
    393         dc.multinomialTreatment = Orange.core.DomainContinuizer.NValues 
     418        dc.class_treatment = Orange.core.DomainContinuizer.Ignore 
     419        dc.continuous_treatment = Orange.core.DomainContinuizer.NormalizeBySpan 
     420        dc.multinomial_treatment = Orange.core.DomainContinuizer.NValues 
    394421        newdomain = dc(data) 
    395422        return data.translate(newdomain) 
     
    417444    def class_distribution(self, example): 
    418445        example = Orange.data.Instance(self.wrapped.domain, example) 
    419         return self.wrapped.classDistribution(example) 
     446        return self.wrapped.class_distribution(example) 
    420447     
    421448    def get_decision_values(self, example): 
    422449        example = Orange.data.Instance(self.wrapped.domain, example) 
    423         return self.wrapped.getDecisionValues(example) 
     450        return self.wrapped.get_decision_values(example) 
    424451     
    425452    def get_model(self): 
    426         return self.wrapped.getModel() 
     453        return self.wrapped.get_model() 
    427454     
    428455    def __reduce__(self): 
     
    439466class SVMLearnerSparse(SVMLearner): 
    440467     
    441     """A :class:`SVMLearner` that learns from   
     468    """A :class:`SVMLearner` that learns from 
    442469    meta attributes. 
    443  
     470     
    444471    Meta attributes do not need to be registered with the data set domain, or  
    445472    present in all the instances. Use this for large  
     
    458485    :func:`SVMLearner.tune_parameters`. It is similar to the easy.py script in  
    459486    the LibSVM package. 
    460  
     487     
    461488    """ 
    462489     
     
    508535        SVMLearnerSparse.__init__(self, **kwds) 
    509536 
    510 class LinearLearner(Orange.core.LinearLearner): 
    511     """A fast learner (LinearLearner) with a default solver type 
    512     ``L2Loss_SVM_Dual``. 
    513     """ 
    514      
    515     def __new__(cls, data=None, weightId=0, **kwargs): 
    516         self = Orange.core.LinearLearner.__new__(cls, **kwargs) 
    517         if data: 
    518             self.__init__(**kwargs) 
    519             return self.__call__(data, weightId) 
    520         else: 
    521             return self 
    522          
    523     def __init__(self, **kwargs): 
    524         if "solver_type" not in kwargs: 
    525             #The default in Orange.core.LinearLearner is L2_LR 
    526             kwargs["solver_type"] = Orange.core.LinearLearner.L2Loss_SVM_Dual 
     537def default_preprocessor(): 
     538    # Construct and return a default preprocessor for use by 
     539    # Orange.core.LinearLearner learner. 
     540    impute = Preprocessor_impute() 
     541    cont = Preprocessor_continuize(multinomialTreatment= 
     542                                   DomainContinuizer.AsOrdinal) 
     543    preproc = Preprocessor_preprocessorList(preprocessors= 
     544                                            [impute, cont]) 
     545    return preproc 
     546 
     547class LinearSVMLearner(Orange.core.LinearLearner): 
     548    """Train a linear SVM model.""" 
     549     
     550    L2R_L2LOSS_DUAL = Orange.core.LinearLearner.L2R_L2Loss_SVC_Dual 
     551    L2R_L2LOSS = Orange.core.LinearLearner.L2R_L2Loss_SVC  
     552    L2R_L1LOSS_DUAL = Orange.core.LinearLearner.L2R_L1Loss_SVC_Dual 
     553    L2R_L1LOSS_DUAL = Orange.core.LinearLearner.L2R_L2Loss_SVC_Dual 
     554    L1R_L2LOSS = Orange.core.LinearLearner.L1R_L2Loss_SVC 
     555     
     556    __new__ = _orange__new__(base=Orange.core.LinearLearner) 
     557         
     558    def __init__(self, solver_type=L2R_L2LOSS_DUAL, C=1.0, eps=0.01, **kwargs): 
     559        """ 
     560        :param solver_type: Can be one of class constants: 
     561         
     562            - L2R_L2LOSS_DUAL 
     563            - L2R_L2LOSS  
     564            - L2R_L1LOSS_DUAL 
     565            - L2R_L1LOSS 
     566            - L1R_L2LOSS 
     567         
     568        :param C: Regularization parameter (default 1.0) 
     569        :type C: float   
     570         
     571        :param eps: Stopping criteria (default 0.01) 
     572        :type eps: float 
     573          
     574        """ 
     575        self.solver_type = solver_type 
     576        self.eps = eps 
     577        self.C = C 
    527578        for name, val in kwargs.items(): 
    528579            setattr(self, name, val) 
     580        if self.solver_type not in [self.L2R_L2LOSS_DUAL, self.L2R_L2LOSS, 
     581                self.L2R_L1LOSS_DUAL, self.L2R_L1LOSS_DUAL, self.L1R_L2LOSS]: 
     582            pass 
     583#            raise ValueError("Invalid solver_type parameter.") 
     584         
     585        self.preproc = default_preprocessor() 
     586             
     587    def __call__(self, instances, weight_id=None): 
     588        instances = self.preproc(instances) 
     589        classifier = super(LinearSVMLearner, self).__call__(instances, weight_id) 
     590        return classifier 
     591         
     592LinearLearner = LinearSVMLearner 
     593 
     594class MultiClassSVMLearner(Orange.core.LinearLearner): 
     595    """ Multi-class SVM (Crammer and Singer) from the `LIBLINEAR`_ library. 
     596    """ 
     597    __new__ = _orange__new__(base=Orange.core.LinearLearner) 
     598         
     599    def __init__(self, C=1.0, eps=0.01, **kwargs): 
     600        """\ 
     601        :param C: Regularization parameter (default 1.0) 
     602        :type C: float   
     603         
     604        :param eps: Stopping criteria (default 0.01) 
     605        :type eps: float 
     606         
     607        """ 
     608        self.C = C 
     609        self.eps = eps 
     610        for name, val in kwargs.items(): 
     611            setattr(self, name, val) 
     612             
     613        self.solver_type = self.MCSVM_CS 
     614        self.preproc = default_preprocessor() 
     615         
     616    def __call__(self, instances, weight_id=None): 
     617        instances = self.preproc(instances) 
     618        classifier = super(MultiClassSVMLearner, self).__call__(instances, weight_id) 
     619        return classifier 
     620 
     621#TODO: Unified way to get attr weights for linear SVMs. 
    529622 
    530623def get_linear_svm_weights(classifier, sum=True): 
    531624    """Extract attribute weights from the linear SVM classifier. 
    532625     
    533     For multi class classification the weights are square-summed over all binary  
    534     one vs. one classifiers. If obj:`sum` is False, the reported weights are a 
    535     seqeunce: class1 vs class2, class1 vs class3 ... class2 vs class3 ... . 
     626    For multi class classification the weights are square-summed over all 
     627    binary one vs. one classifiers unles obj:`sum` is False, in which case 
     628    the return value is a list of weights for each individual binary 
     629    classifier (in the order of [class1 vs class2, class1 vs class3 ... class2 
     630    vs class3 ...]). 
    536631         
    537632    """ 
     
    546641        return float(val) if not val.isSpecial() else 0.0  
    547642             
    548     SVs=classifier.supportVectors 
    549     weights=[] 
    550     classes=classifier.supportVectors.domain.classVar.values 
    551     classSV=dict([(value, filter(lambda sv: sv.getclass()==value, \ 
    552                                  classifier.supportVectors)) \ 
    553                                  for value in classes]) 
    554     svRanges=[(0, classifier.nSV[0])] 
    555     for n in classifier.nSV[1:]: 
    556         svRanges.append((svRanges[-1][1], svRanges[-1][1]+n)) 
    557     for i in range(len(classes)-1): 
     643    SVs=classifier.support_vectors 
     644    weights = [] 
     645     
     646    class_var = SVs.domain.class_var 
     647    if classifier.svm_type in [SVMLearner.C_SVC, SVMLearner.Nu_SVC]: 
     648        classes = class_var.values 
     649    else: 
     650        classes = [""] 
     651    if len(classes) > 1: 
     652        sv_ranges = [(0, classifier.nSV[0])] 
     653        for n in classifier.nSV[1:]: 
     654            sv_ranges.append((sv_ranges[-1][1], sv_ranges[-1][1]+n)) 
     655    else: 
     656        sv_ranges = [(0, len(SVs))] 
     657         
     658    for i in range(len(classes) - 1): 
    558659        for j in range(i+1, len(classes)): 
    559             w={} 
    560             coefInd=j-1 
    561             for svInd in apply(range, svRanges[i]): 
     660            w = {} 
     661            coef_ind = j - 1 
     662            for sv_ind in range(*sv_ranges[i]): 
    562663                attributes = SVs.domain.attributes + \ 
    563                 SVs[svInd].getmetas(False, Orange.data.variable.Variable).keys() 
     664                SVs[sv_ind].getmetas(False, Orange.data.variable.Variable).keys() 
     665                for attr in attributes: 
     666                    if attr.varType == Orange.data.Type.Continuous: 
     667                        update_weights(w, attr, to_float(SVs[sv_ind][attr]), \ 
     668                                       classifier.coef[coef_ind][sv_ind]) 
     669            coef_ind=i 
     670            for sv_ind in range(*sv_ranges[j]): 
     671                attributes = SVs.domain.attributes + \ 
     672                SVs[sv_ind].getmetas(False, Orange.data.variable.Variable).keys() 
    564673                for attr in attributes: 
    565674                    if attr.varType==Orange.data.Type.Continuous: 
    566                         update_weights(w, attr, to_float(SVs[svInd][attr]), \ 
    567                                       classifier.coef[coefInd][svInd]) 
    568             coefInd=i 
    569             for svInd in apply(range, svRanges[j]): 
    570                 attributes = SVs.domain.attributes + \ 
    571                 SVs[svInd].getmetas(False, Orange.data.variable.Variable).keys() 
    572                 for attr in attributes: 
    573                     if attr.varType==Orange.data.Type.Continuous: 
    574                         update_weights(w, attr, to_float(SVs[svInd][attr]), \ 
    575                                       classifier.coef[coefInd][svInd]) 
     675                        update_weights(w, attr, to_float(SVs[sv_ind][attr]), \ 
     676                                       classifier.coef[coef_ind][sv_ind]) 
    576677            weights.append(w) 
    577678             
     
    580681         
    581682        for w in weights: 
    582             for attr, wAttr in w.items(): 
    583                 scores[attr] += wAttr**2 
     683            for attr, w_attr in w.items(): 
     684                scores[attr] += w_attr**2 
    584685        for key in scores: 
    585686            scores[key] = math.sqrt(scores[key]) 
     
    593694    sum=0 
    594695    for attr, w in weights.items(): 
    595         sum+=float(example[attr])*w 
     696        sum += float(example[attr]) * w 
    596697    return sum 
    597698         
     
    599700 
    600701class Score_SVMWeights(Orange.feature.scoring.Score): 
    601      
    602     """Base: :obj:`Orange.feature.scoring.Score` 
    603      
    604     Score feature by training a linear SVM classifier, using a squared sum of  
     702    """Score feature by training a linear SVM classifier, using a squared sum of  
    605703    weights (of each binary classifier) as the returned score. 
    606704         
     
    625723     
    626724    def __init__(self, learner=None, **kwargs): 
    627         """:param learner: Learner used for weight estimation  
     725        """ 
     726        :param learner: Learner used for weight estimation  
    628727            (default LinearLearner(solver_type=L2Loss_SVM_Dual)) 
    629728        :type learner: Orange.core.Learner  
     
    661760     
    662761        >>> rfe = RFE(SVMLearner(kernel_type=kernels.Linear, \ 
    663 normalization=False)) # normalization=False does not change the domain  
     762normalization=False)) # normalization=False -> do not change the domain  
    664763        >>> data_with_removed_features = rfe(table, 5) # table with 5 best attributes 
    665764         
     
    675774        A score is a step number at which the attribute 
    676775        was removed from the recursive evaluation. 
     776         
    677777        """ 
    678778        iter = 1 
     
    753853        for i, attr in enumerate(attrs): 
    754854            if not ex[attr].isSpecial(): 
    755                 file.write(" "+str(i+1)+":"+str(ex[attr])) 
     855                file.write(" "+str(i+1)+":"+str(float(ex[attr]))) 
    756856        file.write("\n") 
    757857       
  • orange/Orange/classification/svm/kernels.py

    r9013 r9054  
    1515     
    1616    :param wrapped: a function to wrap 
    17     :type wrapped: function(double, double) 
     17    :type wrapped: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    1818     
    1919    """ 
     
    3030     
    3131    :param wrapped1:  a function to wrap 
    32     :type wrapped1: function(double, double) 
     32    :type wrapped1: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    3333    :param wrapped2:  a function to wrap 
    34     :type wrapped2: function(double, double) 
     34    :type wrapped2: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    3535     
    3636    """ 
     
    4646     
    4747    :param wrapped: a function to wrap 
    48     :type wrapped: function(double, double) 
     48    :type wrapped: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    4949    :param gamma: the gamma of the RBF 
    5050    :type gamma: double 
     
    6969     
    7070    :param wrapped: a function to wrap 
    71     :type wrapped: function(double, double) 
    72     :param degree: degree of the polinomial 
     71    :type wrapped: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
     72    :param degree: degree of the polynomial 
    7373    :type degree: double 
    7474     
     
    113113     
    114114    :param wrapped1:  a function to wrap 
    115     :type wrapped1: function(double, double) 
     115    :type wrapped1: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    116116    :param wrapped2:  a function to wrap 
    117     :type wrapped2: function(double, double) 
     117    :type wrapped2: function(:class:`Orange.data.Instance`, :class:`Orange.data.Instance`) 
    118118    :param l: coefficient 
    119119    :type l: double 
     
    138138         
    139139        """ 
    140         s=set(example1.getmetas().keys()+example2.getmetas().keys()) 
    141         sum=0 
    142         getmeta=lambda e: e.hasmeta(key) and float(e[key]) or 0.0 
    143         for key in s: 
    144             sum+=pow(getmeta(example2)-getmeta(example1), 2) 
    145         return pow(sum, 0.5) 
    146  
    147 class BagOfWords(object): 
    148     def __call__(self, example1, example2): 
    149         """Computes a BOW kernel function: 
    150           
    151         :math:`\sum_{i=1}^n example1_i * example2_i` 
    152          
    153         using the examples meta attributes (need to be floats). 
    154          
    155         """ 
    156140        s = set(example1.getmetas().keys()) & set(example2.getmetas().keys()) 
    157141        sum = 0 
Note: See TracChangeset for help on using the changeset viewer.