source: orange/docs/reference/rst/Orange.tuning.rst @ 10712:428fb1432e9e

Revision 10712:428fb1432e9e, 2.7 KB checked in by anze <anze.staric@…>, 2 years ago (diff)

Fixes to documentation of Orange.tuning.

Line 
1.. py:currentmodule:: Orange.classification.majority
2
3###############################
4Tuning (``tuning``)
5###############################
6
7.. automodule:: Orange.tuning
8
9.. index:: tuning
10
11Wrappers for Tuning Parameters and Thresholds
12
13Classes for two very useful purposes: tuning learning algorithm's parameters
14using internal validation and tuning the threshold for classification into
15positive class.
16
17*****************
18Tuning parameters
19*****************
20
21Two classes support tuning parameters.
22:obj:`~Tune1Parameter` for fitting a single parameter and
23:obj:`~TuneMParameters` fitting multiple parameters at once,
24trying all possible combinations. When called with data and, optionally, id
25of meta attribute with weights, they find the optimal setting of arguments
26using cross validation. The classes can also be used as ordinary learning
27algorithms - they are in fact derived from
28:obj:`~Orange.classification.Learner`.
29
30Both classes have a common parent, :obj:`~TuneParameters`,
31and a few common attributes.
32
33.. autoclass:: TuneParameters
34   :members:
35
36.. autoclass:: Tune1Parameter
37   :members:
38
39.. autoclass:: TuneMParameters
40   :members:
41
42**************************
43Setting Optimal Thresholds
44**************************
45
46Some models may perform well in terms of AUC which measures the ability to
47distinguish between instances of two classes, but have low classifications
48accuracies. The reason may be in the threshold: in binary problems, classifiers
49usually classify into the more probable class, while sometimes, when class
50distributions are highly skewed, a modified threshold would give better
51accuracies. Here are two classes that can help.
52
53.. autoclass:: ThresholdLearner
54   :members:
55
56.. autoclass:: ThresholdClassifier
57   :members:
58
59Examples
60========
61
62This is how you use the learner.
63
64part of :download:`optimization-thresholding1.py <code/optimization-thresholding1.py>`
65
66.. literalinclude:: code/optimization-thresholding1.py
67
68The output::
69
70    W/out threshold adjustement: 0.633
71    With adjusted thredhold: 0.659
72    With threshold at 0.80: 0.449
73
74part of :download:`optimization-thresholding2.py <code/optimization-thresholding2.py>`
75
76.. literalinclude:: code/optimization-thresholding2.py
77
78The script first divides the data into training and testing subsets. It trains
79a naive Bayesian classifier and than wraps it into
80:obj:`~ThresholdClassifiers` with thresholds of .2, .5 and
81.8. The three models are tested on the left-out data, and we compute the
82confusion matrices from the results. The printout::
83
84    0.20: TP 60.000, TN 1.000
85    0.50: TP 42.000, TN 24.000
86    0.80: TP 2.000, TN 43.000
87
88shows how the varying threshold changes the balance between the number of true
89positives and negatives.
90
91.. autoclass:: PreprocessedLearner
92   :members:
Note: See TracBrowser for help on using the repository browser.