Changeset 7519:d4bffd0de24a in orange

02/04/11 20:04:45 (3 years ago)
jzbontar <jure.zbontar@…>

finished documenting module testing

1 edited


  • orange/Orange/evaluation/

    r7514 r7519  
    183183.. autoclass:: ExperimentResults 
    184184    :members: 
     189Salzberg, S. L. (1997). On comparing classifiers: Pitfalls to avoid 
     190and a recommended approach. Data Mining and Knowledge Discovery 1, 
     191pages 317-328. 
    268275class ExperimentResults(object): 
    269     """ ExperimentResults stores results of one or more repetitions of 
     276    """ 
     277    ``ExperimentResults`` stores results of one or more repetitions of 
    270278    some test (cross validation, repeated sampling...) under the same 
    271279    circumstances. 
     281    .. attribute:: results 
     283        A list of instances of TestedExample, one for each example in 
     284        the dataset. 
     286    .. attribute:: classifiers 
     288        A list of classifiers, one element for each repetition (eg 
     289        fold). Each element is a list of classifiers, one for each 
     290        learner. This field is used only if storing is enabled by 
     291        ``storeClassifiers=1``. 
     293    .. attribute:: numberOfIterations 
     295        Number of iterations. This can be the number of folds 
     296        (in cross validation) or the number of repetitions of some 
     297        test. ``TestedExample``'s attribute ``iterationNumber`` should 
     298        be in range ``[0, numberOfIterations-1]``. 
     300    .. attribute:: numberOfLearners 
     302        Number of learners. Lengths of lists classes and probabilities 
     303        in each :obj:`TestedExample` should equal ``numberOfLearners``. 
     305    .. attribute:: loaded 
     307        If the experimental method supports caching and there are no 
     308        obstacles for caching (such as unknown random seeds), this is a 
     309        list of boolean values. Each element corresponds to a classifier 
     310        and tells whether the experimental results for that classifier 
     311        were computed or loaded from the cache. 
     313    .. attribute:: weights 
     315        A flag telling whether the results are weighted. If ``False``, 
     316        weights are still present in ``TestedExamples``, but they are 
     317        all ``1.0``. Clear this flag, if your experimental procedure 
     318        ran on weighted testing examples but you would like to ignore 
     319        the weights in statistics. 
    273321    """ 
    307355    def saveToFiles(self, learners, filename): 
     356        """ 
     357        Saves and load testing results. ``learners`` is a list of learners and 
     358        ``filename`` is a template for the filename. The attribute loaded is 
     359        initialized so that it contains 1's for the learners whose data 
     360        was loaded and 0's for learners which need to be tested. The 
     361        function returns 1 if all the files were found and loaded, 
     362        and 0 otherwise. 
     364        The data is saved in a separate file for each classifier. The 
     365        file is a binary pickle file containing a list of tuples 
     366        ``((x.actualClass, x.iterationNumber), (x.classes[i], 
     367        x.probabilities[i]))`` where ``x`` is a :obj:`TestedExample` 
     368        and ``i`` is the index of a learner. 
     370        The file resides in the directory ``./cache``. Its name consists 
     371        of a template, given by a caller. The filename should contain 
     372        a %s which is replaced by name, shortDescription, description, 
     373        func_doc or func_name (in that order) attribute of the learner 
     374        (this gets extracted by orngMisc.getobjectname). If a learner 
     375        has none of these attributes, its class name is used. 
     377        Filename should include enough data to make sure that it 
     378        indeed contains the right experimental results. The function 
     379        :obj:`learningCurve`, for example, forms the name of the file 
     380        from a string ``"{learningCurve}"``, the proportion of learning 
     381        examples, random seeds for cross-validation and learning set 
     382        selection, a list of preprocessors' names and a checksum for 
     383        examples. Of course you can outsmart this, but it should suffice 
     384        in most cases. 
     386        """ 
    308388        for i in range(len(learners)): 
    309389            if self.loaded[i]: 
Note: See TracChangeset for help on using the changeset viewer.