# source:orange/Orange/doc/reference/ProbabilityEstimation.htm@9671:a7b056375472

Revision 9671:a7b056375472, 14.5 KB checked in by anze <anze.staric@…>, 2 years ago (diff)

Moved orange to Orange (part 2)

Line
1<html>
6
7<BODY>
8<h1>Classes for Probability Estimation</h1>
9<index name="probability estimation">
10
11<P>Probability estimators are general objects that compute probabilities of values.</P>
12
13<P>There are two branches of probability estimators; those for unconditional and those for conditional probabilities. They are suited for <a href="BayesLearner.htm">naive Bayesian classification</a>, where the former are used to compute p(C) and the latter for p(C|v), where C is a class and v is an attribute value.</P>
14
15<P>Since probability estimation is usually based on the data - learning examples, that is - the whole setup is done in orange way. As for learning, where you use a learner to construct a classifier, in probability estimation there are estimator constructors whose purpose is to construct probability estimators.</P>
16
17<P>This page is divided into three sections. The first describes the basic classes, the second contains classes that are abstract or only support "real" estimators - you would seldom use these directly. The last section contains estimators and constructors that you would most often use. If you are not interested in too many details, you might want to skip the first two sections.</P>
18
19<P>You will seldom use those classes by yourself; most often you'll need them as components for <CODE>BayesLearner</CODE>.</P>
20
21<hr>
22
23<H2>Basic classes</H2>
24
25<P>There are four basic abstract objects that serve as roots of the hierarchy: <CODE>ProbabilityEstimator</CODE>, <CODE>ProbabilityEstimatorConstructor</CODE>, <CODE>ConditionalProbabilityEstimator</CODE> and <CODE>ConditionalProbabilityEstimatorConstructor</CODE>.<P>
26
27<H3>ProbabilityEstimator</H3>
28<index name="classes/ProbabilityEstimator">
29
30<P class=section>Attributes</P>
31<DL class=attributes>
32<DT>supportsDiscrete</DT>
33<DD>Tells whether the estimator can handle discrete attributes.</DD>
34<DT>supportsContinuous</DT>
35<DD>Tells whether the estimator can handle continuous attributes.</DD>
36</DL>
37
38<P class=section>Methods</P>
39<DL class=attributes>
40<DL>
41<DT>__call__([Value])</DT>
42<DD>
43<P>The class defines a single method - a call operator. When given a value, it returns a probability of the value (as float).</P>
44
45<P>When the value is omitted, the object attempts to return a distribution of probabilities for all values (as <CODE>Distribution</CODE>). The result can be <CODE>DiscDistribution</CODE> for discrete, <CODE>ContDistribution</CODE> for continuous attributes or an instance of some other class derived from <CODE>Distribution</CODE>. Note that it indeed makes sense to return continuous distribution. Although probabilities are stored point-wise (as something similar to Python's map, where keys are attribute values and items are probabilities), <CODE>ContDistribution</CODE> can compute probabilities between the recorded values by interpolation.</P>
46
47<P>The estimator does not necessarily support returning precomputed probabilities in form of <CODE>Distribution</CODE>; in this case, it simply returns <CODE>None</CODE>.</P>
48</DD>
49</DL>
50
51<H3>ProbabilityEstimatorConstructor</H3>
52<index name="classes/ProbabilityEstimatorConstructor">
53
54<P>This is an abstract class; derived classes define call operators that return different probability estimators. The class is call-constructible (i.e., if called with appropriate parameters, the constructor returns a probability estimator, not a probability estimator constructor).<P>
55
56<P>The call operator can accept an already computed distribution of classes or a list of examples or both.</P>
57
58<P class=section>Methods</P>
59<DL class=attributes>
60<DT>__call__([distribution[, apriori]], [examples[, weightID]])</DT>
61<DD> If distribution is given, it can be followed by apriori class distribution. Similarly, examples can be followed by with the ID of meta attribute with example weights. (Hint: if you want to have examples and a priori distribution, but don't have distribution ready, just pass None for distribution.) When both, distribution and examples are given, it is up to constructor to decide what to use.</DD>
62</DL>
63
64<H3>ConditionalProbabilityEstimator</H3>
65<index name="classes/ConditionalProbabilityEstimator">
66
67<p>As a counterpart of <CODE>ProbabilityEstimator</CODE>, this estimator can return conditional probabilities.</p>
68
69<P class=section>Methods</P>
70<DL class=attributes>
71<DT>__call__([[Value,] ConditionValue])</DT>
72<DD>
73<P>When given two values, it returns a probability of p(Value|Condition) (as float). When given only one value, it is interpreted as condition; the estimator returns a <CODE>Distribution</CODE> with probabilities p(v|Condition) for each possible value v. When called without arguments, it returns a <CODE>Contingency</CODE> matrix containing probabilities p(v|c) for each possible value and condition; condition is used as outer variable.</P>
74
75<P>If estimator cannot return precomputed distributions and/or contingencies, it returns <CODE>None</CODE>.</P>
76</DD>
77</DL>
78
79<H3>ConditionalProbabilityEstimatorConstructor</H3>
80<index name="classes/ConditionalProbabilityEstimator">
81
82<P>A counterpart of <CODE>ProbabilityEstimatorConstructor</CODE>. It has similar arguments, except that the first argument is not a <CODE>Distribution</CODE> but <CODE>Contingency</CODE>.</P>
83
84<hr>
85
86<H2>Abstract and supporting classes</H2>
87
88<P>There are several abstract classes that simplify the actual classes for probability estimation.</P>
89
90<H3>ProbabilityEstimator_FromDistribution</H3>
91<index name="classes/ProbabilityEstimator_FromDistribution">
92
93<P class=section>Attributes</P>
94<DL class=attributes>
95<DT>probabilities</DT>
96<DD>Stores a precomputed list of probabilities.</DD>
97</DL>
98
99<P>There are many estimator constructors that compute probabilities of classes from frequencies of classes or from list of examples. Probabilities are stored as <CODE>Distribution</CODE>, and <CODE>ProbabilityEstimator_FromDistribution</CODE> is returned. This is done for estimators that use relative frequencies, Laplace's estimation, m-estimation and even estimators that compute continuous distributions.</P>
100
101<P>When asked about probability of certain value, the estimator returns a corresponding element of <CODE>probabilities</CODE>. Note that when distribution is continuous, linear interpolation between two points is used to compute the probability. When asked for a complete distribution, it returns a copy of <CODE>probabilities</CODE>.</P>
102
103<H3>ConditionalProbabilityEstimator_FromDistribution</H3>
104<index name="classes/ConditionalProbabilityEstimator_FromDistribution">
105
106<P class=section>Attributes</P>
107<DL class=attributes>
108<DT>probabilities</DT>
109<DD>stores a precomputed list of probabilities</DD>
110</DL>
111
112<P>This counterpart of <CODE>ProbabilityEstimator_FromDistribution</CODE> stores conditional probabilities in <CODE>Contingency</CODE>.</P>
113
114<H3>ConditionalProbabilityEstimator_ByRows</H3>
115<index name="classes/ConditionalProbabilityEstimator_ByRows">
116
117<P class=section>Attributes</P>
118<DL class=attributes>
119<DT>estimatorList</DT>
120<DD>A list of estimators; one for each value of <CODE>Condition</CODE>.</DD>
121</DL>
122
123<P>This conditional probability estimator has different estimators for different values of conditional attribute. For instance, when used for computing p(c|A) in naive Bayesian classifier, it would have an estimator for each possible value of attribute A. This does not mean that the estimators were constructed by different constructors, i.e. using different probability estimation methods. This class is normally used when we only have a probability estimator constructor for unconditional probabilities but need to construct a conditional probability estimator; the constructor is used to construct estimators for subsets of original example set and the resulting estimators are stored in <CODE>ConditionalProbabilityEstimator_ByRows</CODE>.</P>
124
125<A name="ConditionalProbabilityEstimatorConstructor_ByRows"></A>
126<H3>ConditionalProbabilityEstimatorConstructor_ByRows</H3>
127<index name="classes/ConditionalProbabilityEstimatorConstructor_ByRows">
128
129
130<P class=section>Attributes</P>
131<DL class=attributes>
132<DT>estimatorConstructor</DT>
133<DD>An unconditional probability estimator constructor.</DD>
134</DL>
135
136<P>This class computes a conditional probability estimator using an unconditional probability estimator constructor. The result can be of type <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE> or <CODE>ConditionalProbabilityEstimator_ByRows</CODE>, depending on the type of constructor.</P>
137
138<P>The class first computes contingency matrix if it hasn't been computed already. Then it calls <CODE>estimatorConstructor</CODE> for each value of condition attribute. If all constructed estimators can return distribution of probabilities for all classes (usually either all or none can), the <CODE>Distribution</CODE>s are put in a contingency, and <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE> is constructed and returned. If constructed estimators are not capable of returning distribution of probabilities, a <CODE>ConditionalProbabilityEstimator_ByRows</CODE> is constructed and the estimators are stored in its <CODE>estimatorList</CODE>.
139
140<hr>
141
142<H2>Concrete probability estimators and constructors</H2>
143
144<A name="ProbabilityEstimatorConstructor_relative">
145<H3>ProbabilityEstimatorConstructor_relative</H3>
146<index name="classes/ProbabilityEstimatorConstructor_relative">
147<index name="probability estimation+relative frequency">
148
149<P>Computes relative frequencies of classes, puts it into a Distribution and returns it as <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P>
150
151<H3>ProbabilityEstimatorConstructor_Laplace</H3>
152<index name="classes/ProbabilityEstimatorConstructor_Laplace">
153<index name="probability estimation+Laplace estimate">
154
155<P>Uses Laplace estimation to compute probabilities from frequencies of classes.</P>
156
157<CENTER>
158p(c) = (Nc+1) / (N+n)
159</CENTER>
160<P>where Nc is number of occurences of an event (e.g. number of examples in class c), N is the total number of events (examples) and n is the number of different events (classes).</P>
161
162<P>The resulting estimator is again of type <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P>
163
164<a name="mestimate"></a>
165<H3>ProbabilityEstimatorConstructor_m</H3>
166<index name="classes/ProbabilityEstimatorConstructor_m">
167<index name="probability estimation+m-estimate">
168
169<P class=section>Attributes</P>
170<DL class=attributes>
171<DT>m</DT>
172<DD>Parameter for m-estimation</DD>
173</DL>
174
175Uses m-estimation to compute probabilities from frequencies of classes.
176
177<CENTER>
178p(c) = (Nc+m*ap(c)) / (N+m)
179</CENTER>
180<P>where Nc is number of occurences of an event (e.g. number of examples in class c), N is the total number of events (examples) and ap(c) is the apriori probability of event (class) c.</P>
181
182<P>The resulting estimator is of type <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P>
183
184
185<H3>ProbabilityEstimatorConstructor_kernel</H3>
186<index name="classes/ProbabilityEstimatorConstructor_kernel">
187<index name="probability estimation/kernel">
188
189<P class=section>Attributes</P>
190<DL class=attributes>
191<DT>minImpact</DT>
192<DD>A requested minimal weight of a point (default: 0.01); points with lower weights won't be taken into account</DD>
193<DT>smoothing</DT>
194<DD>Smoothing factor (default: 1.144)</DD>
195<DT>nPoints</DT>
196<DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD>
197</DL>
198
199<P>Useful for continuous distributions, this constructor computes probabilities for certain number of points using <a href="http://www.stat.lsa.umich.edu/~kshedden/Courses/Stat606/Notes/interpolation.pdf">Gaussian kernels</a>. The resulting point-wise continuous distribution is stored as </CODE>ContDistribution</CODE> and returned in <CODE>TProbabilityEstimator_FromDistribution</CODE>.</P>
200
201<P>The points at which probabilities are computed are determined like this.
202Probabilities are always computed at all points that are present in the data (i.e. the existing values of the continuous attribute). If <CODE>nPoints</CODE> is positive and greater than the number of existing data points, additional points are inserted between the existing points to achieve the required number of points. Approximately equal number of new points is inserted between each adjacent existing points. If <CODE>nPoints</CODE> is negative, <CODE>-nPoints</CODE> points will be inserted between each pair of data points.</P>
203
204<H3>ProbabilityEstimatorConstructor_loess</H3>
205<index name="classes/ProbabilityEstimatorConstructor_loess">
206<index name="probability estimation+loess">
207
208<P class=section>Attributes</P>
209<DL class=attributes>
210<DT>windowProportion</DT>
211<DD>a proportion of points in a windows</DD>
212<DT>nPoints</DT>
213<DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD>
214</DL>
215
216<P>This method of probability estimation is similar to <CODE>ProbabilityEstimatorConstructor_kernel</CODE>. They both return a curve computed at certain number of points and the points are determined by the same procedure. They differ, however, at the method for estimating the probabilities.</P>
217
218<P>To estimate probability at point <EM>x</EM>, <CODE>ProbabilityEstimatorConstructor_loess</CODE> examines a window containing a prescribed proportion of original data points. The window is as simetric as possible; the number of points to the left of <EM>x</EM> might differ from the number to the right, but the leftmost point is approximately as far from <EM>x</EM> as the rightmost. Let us denote the width of the windows, e.g. the distance to the farther of the two edge points, by <EM>h</EM>.</P>
219
220<P>Points are weighted by bi-cubic weight function; a weight of point at <EM>x'</EM> is (1-|<EM>t</EM>|<SUP>3</SUP>)<SUP>3</SUP>, where <EM>t</EM> is (<EM>x</EM>-<EM>x'</EM>)/<EM>h</EM>.</P>
221
222<P>Probability at point <EM>x</EM> is then computed as weighted local regression of probabilities for points in the window.</P>
223
224<A name="ConditionalProbabilityEstimatorConstructor_loess">
225<H3>ConditionalProbabilityEstimatorConstructor_loess</H3>
226<index name="classes/ConditionalProbabilityEstimatorConstructor_loess">
227
228<P class=section>Attributes</P>
229<DL class=attributes>
230<DT>windowProportion</DT>
231<DD>a proportion of points in a windows</DD>
232<DT>nPoints</DT>
233<DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD>
234</DL>
235
236<p>Constructs similar estimator as <CODE>ProbabilityEstimatorConstructor_loess</CODE>, except that it computes conditional probabilites. The result is of type <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE>.</p>
237
238</BODY>
Note: See TracBrowser for help on using the repository browser.