Changeset 8170:bb0c3f6df90a in orange


Ignore:
Timestamp:
08/16/11 11:13:05 (3 years ago)
Author:
anze <anze.staric@…>
Branch:
default
Convert:
7c094930632c5693b7182d7a664826d2ecbc735d
Message:

Added bayes back to the documentation.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • orange/doc/Orange/rst/Orange.classification.bayes.rst

    r7546 r8170  
    11.. automodule:: Orange.classification.bayes 
     2 
     3.. index:: naive Bayes classifier 
     4 
     5.. index:: 
     6   single: classification; naive Bayes classifier 
     7 
     8********************************** 
     9Naive Bayes classifier (``bayes``) 
     10********************************** 
     11 
     12The most primitive Bayesian classifier is :obj:`NaiveLearner`. 
     13`Naive Bayes classification algorithm <http://en.wikipedia.org/wiki/Naive_Bayes_classifier>`_ 
     14estimates conditional probabilities from training data and uses them 
     15for classification of new data instances. The algorithm learns very fast if all features 
     16in the training data set are discrete. If a number of features are continues, though, the 
     17algorithm runs slower due to time spent to estimate continuous conditional distributions. 
     18 
     19The following example demonstrates a straightforward invocation of 
     20this algorithm (`bayes-run.py`_, uses `titanic.tab`_): 
     21 
     22.. literalinclude:: code/bayes-run.py 
     23   :lines: 7- 
     24 
     25.. index:: Naive Bayesian Learner 
     26.. autoclass:: Orange.classification.bayes.NaiveLearner 
     27   :members: 
     28   :show-inheritance: 
     29 
     30.. autoclass:: Orange.classification.bayes.NaiveClassifier 
     31   :members: 
     32   :show-inheritance: 
     33 
     34 
     35Examples 
     36======== 
     37 
     38:obj:`NaiveLearner` can estimate probabilities using relative frequencies or 
     39m-estimate (`bayes-mestimate.py`_, uses `lenses.tab`_): 
     40 
     41.. literalinclude:: code/bayes-mestimate.py 
     42    :lines: 7- 
     43 
     44Observing conditional probabilities in an m-estimate based classifier shows a 
     45shift towards the second class - as compared to probabilities above, where 
     46relative frequencies were used. Note that the change in error estimation did 
     47not have any effect on apriori probabilities 
     48(`bayes-thresholdAdjustment.py`_, uses `adult-sample.tab`_): 
     49 
     50.. literalinclude:: code/bayes-thresholdAdjustment.py 
     51    :lines: 7- 
     52 
     53Setting adjustThreshold parameter can sometimes improve the results. Those are 
     54the classification accuracies of 10-fold cross-validation of a normal naive 
     55bayesian classifier, and one with an adjusted threshold:: 
     56 
     57    [0.7901746265516516, 0.8280138859667578] 
     58 
     59Probabilities for continuous features are estimated with \ 
     60:class:`ProbabilityEstimatorConstructor_loess`. 
     61(`bayes-plot-iris.py`_, uses `iris.tab`_): 
     62 
     63.. literalinclude:: code/bayes-plot-iris.py 
     64    :lines: 4- 
     65 
     66.. image:: code/bayes-iris.png 
     67   :scale: 50 % 
     68 
     69If petal lengths are shorter, the most probable class is "setosa". Irises with 
     70middle petal lengths belong to "versicolor", while longer petal lengths indicate 
     71for "virginica". Critical values where the decision would change are at about 
     725.4 and 6.3. 
     73 
     74 
     75.. _bayes-run.py: code/bayes-run.py 
     76.. _bayes-thresholdAdjustment.py: code/bayes-thresholdAdjustment.py 
     77.. _bayes-mestimate.py: code/bayes-mestimate.py 
     78.. _bayes-plot-iris.py: code/bayes-plot-iris.py 
     79.. _adult-sample.tab: code/adult-sample.tab 
     80.. _iris.tab: code/iris.tab 
     81.. _titanic.tab: code/iris.tab 
     82.. _lenses.tab: code/lenses.tab 
     83 
     84Implementation details 
     85====================== 
     86 
     87The following two classes are implemented in C++ (*bayes.cpp*). They are not 
     88intended to be used directly. Here we provide implementation details for those 
     89interested. 
     90 
     91Orange.core.BayesLearner 
     92------------------------ 
     93Fields estimatorConstructor, conditionalEstimatorConstructor and 
     94conditionalEstimatorConstructorContinuous are empty (None) by default. 
     95 
     96If estimatorConstructor is left undefined, p(C) will be estimated by relative 
     97frequencies of examples (see ProbabilityEstimatorConstructor_relative). 
     98When conditionalEstimatorConstructor is left undefined, it will use the same 
     99constructor as for estimating unconditional probabilities (estimatorConstructor 
     100is used as an estimator in ConditionalProbabilityEstimatorConstructor_ByRows). 
     101That is, by default, both will use relative frequencies. But when 
     102estimatorConstructor is set to, for instance, estimate probabilities by 
     103m-estimate with m=2.0, the same estimator will be used for estimation of 
     104conditional probabilities, too. 
     105P(c|vi) for continuous attributes are, by default, estimated with loess (a 
     106variant of locally weighted linear regression), using 
     107ConditionalProbabilityEstimatorConstructor_loess. 
     108The learner first constructs an estimator for p(C). It tries to get a 
     109precomputed distribution of probabilities; if the estimator is capable of 
     110returning it, the distribution is stored in the classifier's field distribution 
     111and the just constructed estimator is disposed. Otherwise, the estimator is 
     112stored in the classifier's field estimator, while the distribution is left 
     113empty. 
     114 
     115The same is then done for conditional probabilities. Different constructors are 
     116used for discrete and continuous attributes. If the constructed estimator can 
     117return all conditional probabilities in form of Contingency, the contingency is 
     118stored and the estimator disposed. If not, the estimator is stored. If there 
     119are no contingencies when the learning is finished, the resulting classifier's 
     120conditionalDistributions is None. Alternatively, if all probabilities are 
     121stored as contingencies, the conditionalEstimators fields is None. 
     122 
     123Field normalizePredictions is copied to the resulting classifier. 
     124 
     125Orange.core.BayesClassifier 
     126--------------------------- 
     127Class NaiveClassifier represents a naive bayesian classifier. Probability of 
     128class C, knowing that values of features :math:`F_1, F_2, ..., F_n` are 
     129:math:`v_1, v_2, ..., v_n`, is computed as :math:`p(C|v_1, v_2, ..., v_n) = \ 
     130p(C) \\cdot \\frac{p(C|v_1)}{p(C)} \\cdot \\frac{p(C|v_2)}{p(C)} \\cdot ... \ 
     131\\cdot \\frac{p(C|v_n)}{p(C)}`. 
     132 
     133Note that when relative frequencies are used to estimate probabilities, the 
     134more usual formula (with factors of form :math:`\\frac{p(v_i|C)}{p(v_i)}`) and 
     135the above formula are exactly equivalent (without any additional assumptions of 
     136independency, as one could think at a first glance). The difference becomes 
     137important when using other ways to estimate probabilities, like, for instance, 
     138m-estimate. In this case, the above formula is much more appropriate. 
     139 
     140When computing the formula, probabilities p(C) are read from distribution, which 
     141is of type Distribution, and stores a (normalized) probability of each class. 
     142When distribution is None, BayesClassifier calls estimator to assess the 
     143probability. The former method is faster and is actually used by all existing 
     144methods of probability estimation. The latter is more flexible. 
     145 
     146Conditional probabilities are computed similarly. Field conditionalDistribution 
     147is of type DomainContingency which is basically a list of instances of 
     148Contingency, one for each attribute; the outer variable of the contingency is 
     149the attribute and the inner is the class. Contingency can be seen as a list of 
     150normalized probability distributions. For attributes for which there is no 
     151contingency in conditionalDistribution a corresponding estimator in 
     152conditionalEstimators is used. The estimator is given the attribute value and 
     153returns distributions of classes. 
     154 
     155If neither, nor pre-computed contingency nor conditional estimator exist, the 
     156attribute is ignored without issuing any warning. The attribute is also ignored 
     157if its value is undefined; this cannot be overriden by estimators. 
     158 
     159Any field (distribution, estimator, conditionalDistributions, 
     160conditionalEstimators) can be None. For instance, BayesLearner normally 
     161constructs a classifier which has either distribution or estimator defined. 
     162While it is not an error to have both, only distribution will be used in that 
     163case. As for the other two fields, they can be both defined and used 
     164complementarily; the elements which are missing in one are defined in the 
     165other. However, if there is no need for estimators, BayesLearner will not 
     166construct an empty list; it will not construct a list at all, but leave the 
     167field conditionalEstimators empty. 
     168 
     169If you only need probabilities of individual class call BayesClassifier's 
     170method p(class, example) to compute the probability of this class only. Note 
     171that this probability will not be normalized and will thus, in general, not 
     172equal the probability returned by the call operator. 
Note: See TracChangeset for help on using the changeset viewer.