Changeset 10192:2e2fcc44248e in orange


Ignore:
Timestamp:
02/13/12 14:55:02 (2 years ago)
Author:
anzeh <anze.staric@…>
Branch:
default
rebase_source:
7442ed36b118f0b29e46383c2280858e7d61475e
Message:

Updated documentation for evaluation.testing.

Files:
1 added
2 edited

Legend:

Unmodified
Added
Removed
  • Orange/evaluation/testing.py

    r10189 r10192  
    561561 
    562562        :param learners: list of learners to be tested 
    563         :param learn_set: a dataset used for learning 
     563        :param learn_set: a dataset used for evaluation 
    564564        :param test_set: a dataset used for evaluation 
    565565        :param proportions: proportions of train data to be used 
  • docs/reference/rst/Orange.evaluation.testing.rst

    r10179 r10192  
    1212:obj:`~Orange.evaluation.scoring` functions to evaluate model. 
    1313 
     14.. literalinclude:: code/testing-example.py 
    1415 
     16Different evaluation techniques are implemented as instance methods of 
     17:obj:`Evaluation` class. For ease of use, an instance of this class in 
     18created at module loading time and instance methods are exposed as functions 
     19with the same name in Orange.evaluation.testing namespace. 
    1520 
    16 Your scripts will thus basically conduct experiments using methods of 
    17 :obj:`Evaluation` class and functions in  :obj:`Orange.evaluation.testing`, 
    18 covered on this page and then evaluate the results by functions in 
    19 :obj:`Orange.evaluation.scoring`. For those interested in writing their own 
    20 statistical measures of the quality of models, 
    21 description of :obj:`TestedExample` and :obj:`ExperimentResults` are 
    22 available at the end of this page. 
     21.. autoclass:: Evaluation 
     22   :members: 
    2323 
    24 Example scripts in this section suppose that the data is loaded and a 
    25 list of learning algorithms is prepared. 
     24.. autoclass:: ExperimentResults 
     25    :members: 
    2626 
    27 part of :download:`testing-test.py <code/testing-test.py>` 
     27.. autoclass:: TestedExample 
     28    :members: 
    2829 
    29 .. literalinclude:: code/testing-test.py 
    30     :start-after: import random 
    31     :end-before: def printResults(res) 
     30Generating random numbers 
     31========================= 
    3232 
    33 After testing is done, classification accuracies can be computed and 
    34 printed by the following function. 
     33Many evaluation 
    3534 
    36 .. literalinclude:: code/testing-test.py 
    37     :pyobject: printResults 
    38  
    39 Common Arguments 
    40 ================ 
    41  
    42 Many function in this module use a set of common arguments, which we define here. 
    43  
    44 *learners* 
    45     A list of learning algorithms. These can be either pure Orange objects 
    46     (such as :obj:`Orange.classification.bayes.NaiveLearner`) or Python 
    47     classes or functions written in pure Python (anything that can be 
    48     called with the same arguments and results as Orange's classifiers 
    49     and performs similar function). 
    50  
    51 *examples, learnset, testset* 
    52     Examples, given as an :obj:`Orange.data.Table` (some functions need an undivided 
    53     set of examples while others need examples that are already split 
    54     into two sets). If examples are weighted, pass them as a tuple 
    55     ``(examples, weightID)``. Weights are respected by learning and testing, 
    56     but not by sampling. When selecting 10% of examples, this means 10% 
    57     by number, not by weights. There is also no guarantee that sums 
    58     of example weights will be (at least roughly) equal for folds in 
    59     cross validation. 
    60  
    61 *strat* 
     35*stratified* 
    6236    Tells whether to stratify the random selections. Its default value is 
    6337    :obj:`orange.StratifiedIfPossible` which stratifies selections 
    6438    if the class variable is discrete and has no unknown values. 
    6539 
    66 *randseed (obsolete: indicesrandseed), random_generator* 
    67     Random seed (``randseed``) or random generator (``random_generator``) for 
    68     random selection of examples. If omitted, random seed of 0 is used and 
    69     the same test will always select the same examples from the example 
    70     set. There are various slightly different ways to randomize it. 
     40*random_generator* 
     41    If evaluation method relies on randomness, parameter ``random_generator`` 
     42    can be used to either provide a random seed or an instance of 
     43    :obj:`~Orange.misc.Random` which will be used to generate random numbers. 
    7144 
    72     * 
    73       Set ``random_generator`` to :obj:`orange.globalRandom`. The function's 
    74       selection will depend upon Orange's global random generator that 
    75       is reset (with random seed 0) when Orange is imported. The Script's 
    76       output will therefore depend upon what you did after Orange was 
    77       first imported in the current Python session. :: 
     45    By default, a new instance of random generator is constructed for each 
     46    call of the method with random seed 0. 
    7847 
    79           res = Orange.evaluation.testing.proportion_test(learners, data, 0.7, 
    80               random_generator=orange.globalRandom) 
     48    If you use more than one method that is based on randomness, 
     49    you can construct an instance of :obj:`~Orange.misc.Random` and pass it 
     50    to all of them. This way, all methods will use different random numbers, 
     51    but this numbers will be the same for each run of the script. 
    8152 
    82     * 
    83       Construct a new :obj:`Orange.misc.Random`. The code below, 
    84       for instance, will produce different results in each iteration, 
    85       but overall the same results each time it's run. 
    86  
    87       .. literalinclude:: code/testing-test.py 
    88         :start-after: but the same each time the script is run 
    89         :end-before: # End 
    90  
    91     * 
    92       Set the random seed (argument ``randseed``) to a random 
    93       number. Python has a global random generator that is reset when 
    94       Python is loaded, using the current system time for a seed. With this, 
    95       results will be (in general) different each time the script is run. 
    96  
    97  
    98       .. literalinclude:: code/testing-test.py 
    99         :start-after: proportionsTest that will give different results each time it is run 
    100         :end-before: # End 
    101  
    102  
    103       The same module also provides random generators as object, so 
    104       that you can have independent local random generators in case you 
    105       need them. 
     53    For truly random number, set seed to a random number generated with 
     54    python random generator. Since python's random generator is reset each 
     55    time python is loaded with current system time as seed, 
     56    results of the script will be different each time you run it. 
    10657 
    10758*preprocessors* 
     
    12576        res = Orange.evaluation.testing.proportion_test(learners, data, 0.7, 100, pps = [("L", classnoise)]) 
    12677 
    127 *proportions* 
    128     Gives the proportions of learning examples at which the tests are 
    129     to be made, where applicable. The default is ``[0.1, 0.2, ..., 1.0]``. 
    130  
    13178*store_classifiers (keyword argument)* 
    13279    If this flag is set, the testing procedure will store the constructed 
     
    14289        100, store_classifiers=1) 
    14390 
    144 Sampling and Testing Functions 
    145 ============================== 
    146  
    147 .. autoclass:: Evaluation 
    148    :members: 
    149  
    150 Classes 
    151 ======= 
    15291 
    15392Knowing classes :obj:`TestedExample` that stores results of testing 
     
    15897provided by Orange. If not, you can skip the remainder of this page. 
    15998 
    160 .. autoclass:: TestedExample 
    161     :members: 
    16299 
    163 .. autoclass:: ExperimentResults 
    164     :members: 
    165100 
    166101References 
Note: See TracChangeset for help on using the changeset viewer.