Changeset 7519:d4bffd0de24a in orange


Ignore:
Timestamp:
02/04/11 20:04:45 (3 years ago)
Author:
jzbontar <jure.zbontar@…>
Branch:
default
Convert:
0529d2e1a6ff170f56bed75fc679b27e5ea32bbb
Message:

finished documenting module testing

File:
1 edited

Legend:

Unmodified
Added
Removed
  • orange/Orange/evaluation/testing.py

    r7514 r7519  
    183183.. autoclass:: ExperimentResults 
    184184    :members: 
     185 
     186References 
     187========== 
     188 
     189Salzberg, S. L. (1997). On comparing classifiers: Pitfalls to avoid 
     190and a recommended approach. Data Mining and Knowledge Discovery 1, 
     191pages 317-328. 
    185192 
    186193""" 
     
    267274 
    268275class ExperimentResults(object): 
    269     """ ExperimentResults stores results of one or more repetitions of 
     276    """ 
     277    ``ExperimentResults`` stores results of one or more repetitions of 
    270278    some test (cross validation, repeated sampling...) under the same 
    271279    circumstances. 
     280 
     281    .. attribute:: results 
     282 
     283        A list of instances of TestedExample, one for each example in 
     284        the dataset. 
     285 
     286    .. attribute:: classifiers 
     287 
     288        A list of classifiers, one element for each repetition (eg 
     289        fold). Each element is a list of classifiers, one for each 
     290        learner. This field is used only if storing is enabled by 
     291        ``storeClassifiers=1``. 
     292 
     293    .. attribute:: numberOfIterations 
     294 
     295        Number of iterations. This can be the number of folds 
     296        (in cross validation) or the number of repetitions of some 
     297        test. ``TestedExample``'s attribute ``iterationNumber`` should 
     298        be in range ``[0, numberOfIterations-1]``. 
     299 
     300    .. attribute:: numberOfLearners 
     301 
     302        Number of learners. Lengths of lists classes and probabilities 
     303        in each :obj:`TestedExample` should equal ``numberOfLearners``. 
     304 
     305    .. attribute:: loaded 
     306 
     307        If the experimental method supports caching and there are no 
     308        obstacles for caching (such as unknown random seeds), this is a 
     309        list of boolean values. Each element corresponds to a classifier 
     310        and tells whether the experimental results for that classifier 
     311        were computed or loaded from the cache. 
     312 
     313    .. attribute:: weights 
     314 
     315        A flag telling whether the results are weighted. If ``False``, 
     316        weights are still present in ``TestedExamples``, but they are 
     317        all ``1.0``. Clear this flag, if your experimental procedure 
     318        ran on weighted testing examples but you would like to ignore 
     319        the weights in statistics. 
    272320 
    273321    """ 
     
    306354                 
    307355    def saveToFiles(self, learners, filename): 
     356        """ 
     357        Saves and load testing results. ``learners`` is a list of learners and 
     358        ``filename`` is a template for the filename. The attribute loaded is 
     359        initialized so that it contains 1's for the learners whose data 
     360        was loaded and 0's for learners which need to be tested. The 
     361        function returns 1 if all the files were found and loaded, 
     362        and 0 otherwise. 
     363 
     364        The data is saved in a separate file for each classifier. The 
     365        file is a binary pickle file containing a list of tuples 
     366        ``((x.actualClass, x.iterationNumber), (x.classes[i], 
     367        x.probabilities[i]))`` where ``x`` is a :obj:`TestedExample` 
     368        and ``i`` is the index of a learner. 
     369 
     370        The file resides in the directory ``./cache``. Its name consists 
     371        of a template, given by a caller. The filename should contain 
     372        a %s which is replaced by name, shortDescription, description, 
     373        func_doc or func_name (in that order) attribute of the learner 
     374        (this gets extracted by orngMisc.getobjectname). If a learner 
     375        has none of these attributes, its class name is used. 
     376 
     377        Filename should include enough data to make sure that it 
     378        indeed contains the right experimental results. The function 
     379        :obj:`learningCurve`, for example, forms the name of the file 
     380        from a string ``"{learningCurve}"``, the proportion of learning 
     381        examples, random seeds for cross-validation and learning set 
     382        selection, a list of preprocessors' names and a checksum for 
     383        examples. Of course you can outsmart this, but it should suffice 
     384        in most cases. 
     385 
     386        """ 
     387 
    308388        for i in range(len(learners)): 
    309389            if self.loaded[i]: 
Note: See TracChangeset for help on using the changeset viewer.