#
source:
orange/Orange/doc/reference/ProbabilityEstimation.htm
@
9671:a7b056375472

Revision 9671:a7b056375472, 14.5 KB checked in by anze <anze.staric@…>, 2 years ago (diff) |
---|

Line | |
---|---|

1 | <html> |

2 | <HEAD> |

3 | <LINK REL=StyleSheet HREF="../style.css" TYPE="text/css"> |

4 | <LINK REL=StyleSheet HREF="style-print.css" TYPE="text/css" MEDIA=print> |

5 | </HEAD> |

6 | |

7 | <BODY> |

8 | <h1>Classes for Probability Estimation</h1> |

9 | <index name="probability estimation"> |

10 | |

11 | <P>Probability estimators are general objects that compute probabilities of values.</P> |

12 | |

13 | <P>There are two branches of probability estimators; those for unconditional and those for conditional probabilities. They are suited for <a href="BayesLearner.htm">naive Bayesian classification</a>, where the former are used to compute p(C) and the latter for p(C|v), where C is a class and v is an attribute value.</P> |

14 | |

15 | <P>Since probability estimation is usually based on the data - learning examples, that is - the whole setup is done in orange way. As for learning, where you use a learner to construct a classifier, in probability estimation there are estimator constructors whose purpose is to construct probability estimators.</P> |

16 | |

17 | <P>This page is divided into three sections. The first describes the basic classes, the second contains classes that are abstract or only support "real" estimators - you would seldom use these directly. The last section contains estimators and constructors that you would most often use. If you are not interested in too many details, you might want to skip the first two sections.</P> |

18 | |

19 | <P>You will seldom use those classes by yourself; most often you'll need them as components for <CODE>BayesLearner</CODE>.</P> |

20 | |

21 | <hr> |

22 | |

23 | <H2>Basic classes</H2> |

24 | |

25 | <P>There are four basic abstract objects that serve as roots of the hierarchy: <CODE>ProbabilityEstimator</CODE>, <CODE>ProbabilityEstimatorConstructor</CODE>, <CODE>ConditionalProbabilityEstimator</CODE> and <CODE>ConditionalProbabilityEstimatorConstructor</CODE>.<P> |

26 | |

27 | <H3>ProbabilityEstimator</H3> |

28 | <index name="classes/ProbabilityEstimator"> |

29 | |

30 | <P class=section>Attributes</P> |

31 | <DL class=attributes> |

32 | <DT>supportsDiscrete</DT> |

33 | <DD>Tells whether the estimator can handle discrete attributes.</DD> |

34 | <DT>supportsContinuous</DT> |

35 | <DD>Tells whether the estimator can handle continuous attributes.</DD> |

36 | </DL> |

37 | |

38 | <P class=section>Methods</P> |

39 | <DL class=attributes> |

40 | <DL> |

41 | <DT>__call__([Value])</DT> |

42 | <DD> |

43 | <P>The class defines a single method - a call operator. When given a value, it returns a probability of the value (as float).</P> |

44 | |

45 | <P>When the value is omitted, the object attempts to return a distribution of probabilities for all values (as <CODE>Distribution</CODE>). The result can be <CODE>DiscDistribution</CODE> for discrete, <CODE>ContDistribution</CODE> for continuous attributes or an instance of some other class derived from <CODE>Distribution</CODE>. Note that it indeed makes sense to return continuous distribution. Although probabilities are stored point-wise (as something similar to Python's map, where keys are attribute values and items are probabilities), <CODE>ContDistribution</CODE> can compute probabilities between the recorded values by interpolation.</P> |

46 | |

47 | <P>The estimator does not necessarily support returning precomputed probabilities in form of <CODE>Distribution</CODE>; in this case, it simply returns <CODE>None</CODE>.</P> |

48 | </DD> |

49 | </DL> |

50 | |

51 | <H3>ProbabilityEstimatorConstructor</H3> |

52 | <index name="classes/ProbabilityEstimatorConstructor"> |

53 | |

54 | <P>This is an abstract class; derived classes define call operators that return different probability estimators. The class is call-constructible (i.e., if called with appropriate parameters, the constructor returns a probability estimator, not a probability estimator constructor).<P> |

55 | |

56 | <P>The call operator can accept an already computed distribution of classes or a list of examples or both.</P> |

57 | |

58 | <P class=section>Methods</P> |

59 | <DL class=attributes> |

60 | <DT>__call__([distribution[, apriori]], [examples[, weightID]])</DT> |

61 | <DD> If distribution is given, it can be followed by apriori class distribution. Similarly, examples can be followed by with the ID of meta attribute with example weights. (Hint: if you want to have examples and a priori distribution, but don't have distribution ready, just pass None for distribution.) When both, distribution and examples are given, it is up to constructor to decide what to use.</DD> |

62 | </DL> |

63 | |

64 | <H3>ConditionalProbabilityEstimator</H3> |

65 | <index name="classes/ConditionalProbabilityEstimator"> |

66 | |

67 | <p>As a counterpart of <CODE>ProbabilityEstimator</CODE>, this estimator can return conditional probabilities.</p> |

68 | |

69 | <P class=section>Methods</P> |

70 | <DL class=attributes> |

71 | <DT>__call__([[Value,] ConditionValue])</DT> |

72 | <DD> |

73 | <P>When given two values, it returns a probability of p(Value|Condition) (as float). When given only one value, it is interpreted as condition; the estimator returns a <CODE>Distribution</CODE> with probabilities p(v|Condition) for each possible value v. When called without arguments, it returns a <CODE>Contingency</CODE> matrix containing probabilities p(v|c) for each possible value and condition; condition is used as outer variable.</P> |

74 | |

75 | <P>If estimator cannot return precomputed distributions and/or contingencies, it returns <CODE>None</CODE>.</P> |

76 | </DD> |

77 | </DL> |

78 | |

79 | <H3>ConditionalProbabilityEstimatorConstructor</H3> |

80 | <index name="classes/ConditionalProbabilityEstimator"> |

81 | |

82 | <P>A counterpart of <CODE>ProbabilityEstimatorConstructor</CODE>. It has similar arguments, except that the first argument is not a <CODE>Distribution</CODE> but <CODE>Contingency</CODE>.</P> |

83 | |

84 | <hr> |

85 | |

86 | <H2>Abstract and supporting classes</H2> |

87 | |

88 | <P>There are several abstract classes that simplify the actual classes for probability estimation.</P> |

89 | |

90 | <H3>ProbabilityEstimator_FromDistribution</H3> |

91 | <index name="classes/ProbabilityEstimator_FromDistribution"> |

92 | |

93 | <P class=section>Attributes</P> |

94 | <DL class=attributes> |

95 | <DT>probabilities</DT> |

96 | <DD>Stores a precomputed list of probabilities.</DD> |

97 | </DL> |

98 | |

99 | <P>There are many estimator constructors that compute probabilities of classes from frequencies of classes or from list of examples. Probabilities are stored as <CODE>Distribution</CODE>, and <CODE>ProbabilityEstimator_FromDistribution</CODE> is returned. This is done for estimators that use relative frequencies, Laplace's estimation, m-estimation and even estimators that compute continuous distributions.</P> |

100 | |

101 | <P>When asked about probability of certain value, the estimator returns a corresponding element of <CODE>probabilities</CODE>. Note that when distribution is continuous, linear interpolation between two points is used to compute the probability. When asked for a complete distribution, it returns a copy of <CODE>probabilities</CODE>.</P> |

102 | |

103 | <H3>ConditionalProbabilityEstimator_FromDistribution</H3> |

104 | <index name="classes/ConditionalProbabilityEstimator_FromDistribution"> |

105 | |

106 | <P class=section>Attributes</P> |

107 | <DL class=attributes> |

108 | <DT>probabilities</DT> |

109 | <DD>stores a precomputed list of probabilities</DD> |

110 | </DL> |

111 | |

112 | <P>This counterpart of <CODE>ProbabilityEstimator_FromDistribution</CODE> stores conditional probabilities in <CODE>Contingency</CODE>.</P> |

113 | |

114 | <H3>ConditionalProbabilityEstimator_ByRows</H3> |

115 | <index name="classes/ConditionalProbabilityEstimator_ByRows"> |

116 | |

117 | <P class=section>Attributes</P> |

118 | <DL class=attributes> |

119 | <DT>estimatorList</DT> |

120 | <DD>A list of estimators; one for each value of <CODE>Condition</CODE>.</DD> |

121 | </DL> |

122 | |

123 | <P>This conditional probability estimator has different estimators for different values of conditional attribute. For instance, when used for computing p(c|A) in naive Bayesian classifier, it would have an estimator for each possible value of attribute A. This does not mean that the estimators were constructed by different constructors, i.e. using different probability estimation methods. This class is normally used when we only have a probability estimator constructor for unconditional probabilities but need to construct a conditional probability estimator; the constructor is used to construct estimators for subsets of original example set and the resulting estimators are stored in <CODE>ConditionalProbabilityEstimator_ByRows</CODE>.</P> |

124 | |

125 | <A name="ConditionalProbabilityEstimatorConstructor_ByRows"></A> |

126 | <H3>ConditionalProbabilityEstimatorConstructor_ByRows</H3> |

127 | <index name="classes/ConditionalProbabilityEstimatorConstructor_ByRows"> |

128 | |

129 | |

130 | <P class=section>Attributes</P> |

131 | <DL class=attributes> |

132 | <DT>estimatorConstructor</DT> |

133 | <DD>An unconditional probability estimator constructor.</DD> |

134 | </DL> |

135 | |

136 | <P>This class computes a conditional probability estimator using an unconditional probability estimator constructor. The result can be of type <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE> or <CODE>ConditionalProbabilityEstimator_ByRows</CODE>, depending on the type of constructor.</P> |

137 | |

138 | <P>The class first computes contingency matrix if it hasn't been computed already. Then it calls <CODE>estimatorConstructor</CODE> for each value of condition attribute. If all constructed estimators can return distribution of probabilities for all classes (usually either all or none can), the <CODE>Distribution</CODE>s are put in a contingency, and <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE> is constructed and returned. If constructed estimators are not capable of returning distribution of probabilities, a <CODE>ConditionalProbabilityEstimator_ByRows</CODE> is constructed and the estimators are stored in its <CODE>estimatorList</CODE>. |

139 | |

140 | <hr> |

141 | |

142 | <H2>Concrete probability estimators and constructors</H2> |

143 | |

144 | <A name="ProbabilityEstimatorConstructor_relative"> |

145 | <H3>ProbabilityEstimatorConstructor_relative</H3> |

146 | <index name="classes/ProbabilityEstimatorConstructor_relative"> |

147 | <index name="probability estimation+relative frequency"> |

148 | |

149 | <P>Computes relative frequencies of classes, puts it into a Distribution and returns it as <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P> |

150 | |

151 | <H3>ProbabilityEstimatorConstructor_Laplace</H3> |

152 | <index name="classes/ProbabilityEstimatorConstructor_Laplace"> |

153 | <index name="probability estimation+Laplace estimate"> |

154 | |

155 | <P>Uses Laplace estimation to compute probabilities from frequencies of classes.</P> |

156 | |

157 | <CENTER> |

158 | p(c) = (Nc+1) / (N+n) |

159 | </CENTER> |

160 | <P>where Nc is number of occurences of an event (e.g. number of examples in class c), N is the total number of events (examples) and n is the number of different events (classes).</P> |

161 | |

162 | <P>The resulting estimator is again of type <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P> |

163 | |

164 | <a name="mestimate"></a> |

165 | <H3>ProbabilityEstimatorConstructor_m</H3> |

166 | <index name="classes/ProbabilityEstimatorConstructor_m"> |

167 | <index name="probability estimation+m-estimate"> |

168 | |

169 | <P class=section>Attributes</P> |

170 | <DL class=attributes> |

171 | <DT>m</DT> |

172 | <DD>Parameter for m-estimation</DD> |

173 | </DL> |

174 | |

175 | Uses m-estimation to compute probabilities from frequencies of classes. |

176 | |

177 | <CENTER> |

178 | p(c) = (Nc+m*ap(c)) / (N+m) |

179 | </CENTER> |

180 | <P>where Nc is number of occurences of an event (e.g. number of examples in class c), N is the total number of events (examples) and ap(c) is the apriori probability of event (class) c.</P> |

181 | |

182 | <P>The resulting estimator is of type <CODE>ProbabilityEstimator_FromDistribution</CODE>.</P> |

183 | |

184 | |

185 | <H3>ProbabilityEstimatorConstructor_kernel</H3> |

186 | <index name="classes/ProbabilityEstimatorConstructor_kernel"> |

187 | <index name="probability estimation/kernel"> |

188 | |

189 | <P class=section>Attributes</P> |

190 | <DL class=attributes> |

191 | <DT>minImpact</DT> |

192 | <DD>A requested minimal weight of a point (default: 0.01); points with lower weights won't be taken into account</DD> |

193 | <DT>smoothing</DT> |

194 | <DD>Smoothing factor (default: 1.144)</DD> |

195 | <DT>nPoints</DT> |

196 | <DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD> |

197 | </DL> |

198 | |

199 | <P>Useful for continuous distributions, this constructor computes probabilities for certain number of points using <a href="http://www.stat.lsa.umich.edu/~kshedden/Courses/Stat606/Notes/interpolation.pdf">Gaussian kernels</a>. The resulting point-wise continuous distribution is stored as </CODE>ContDistribution</CODE> and returned in <CODE>TProbabilityEstimator_FromDistribution</CODE>.</P> |

200 | |

201 | <P>The points at which probabilities are computed are determined like this. |

202 | Probabilities are always computed at all points that are present in the data (i.e. the existing values of the continuous attribute). If <CODE>nPoints</CODE> is positive and greater than the number of existing data points, additional points are inserted between the existing points to achieve the required number of points. Approximately equal number of new points is inserted between each adjacent existing points. If <CODE>nPoints</CODE> is negative, <CODE>-nPoints</CODE> points will be inserted between each pair of data points.</P> |

203 | |

204 | <H3>ProbabilityEstimatorConstructor_loess</H3> |

205 | <index name="classes/ProbabilityEstimatorConstructor_loess"> |

206 | <index name="probability estimation+loess"> |

207 | |

208 | <P class=section>Attributes</P> |

209 | <DL class=attributes> |

210 | <DT>windowProportion</DT> |

211 | <DD>a proportion of points in a windows</DD> |

212 | <DT>nPoints</DT> |

213 | <DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD> |

214 | </DL> |

215 | |

216 | <P>This method of probability estimation is similar to <CODE>ProbabilityEstimatorConstructor_kernel</CODE>. They both return a curve computed at certain number of points and the points are determined by the same procedure. They differ, however, at the method for estimating the probabilities.</P> |

217 | |

218 | <P>To estimate probability at point <EM>x</EM>, <CODE>ProbabilityEstimatorConstructor_loess</CODE> examines a window containing a prescribed proportion of original data points. The window is as simetric as possible; the number of points to the left of <EM>x</EM> might differ from the number to the right, but the leftmost point is approximately as far from <EM>x</EM> as the rightmost. Let us denote the width of the windows, e.g. the distance to the farther of the two edge points, by <EM>h</EM>.</P> |

219 | |

220 | <P>Points are weighted by bi-cubic weight function; a weight of point at <EM>x'</EM> is (1-|<EM>t</EM>|<SUP>3</SUP>)<SUP>3</SUP>, where <EM>t</EM> is (<EM>x</EM>-<EM>x'</EM>)/<EM>h</EM>.</P> |

221 | |

222 | <P>Probability at point <EM>x</EM> is then computed as weighted local regression of probabilities for points in the window.</P> |

223 | |

224 | <A name="ConditionalProbabilityEstimatorConstructor_loess"> |

225 | <H3>ConditionalProbabilityEstimatorConstructor_loess</H3> |

226 | <index name="classes/ConditionalProbabilityEstimatorConstructor_loess"> |

227 | |

228 | <P class=section>Attributes</P> |

229 | <DL class=attributes> |

230 | <DT>windowProportion</DT> |

231 | <DD>a proportion of points in a windows</DD> |

232 | <DT>nPoints</DT> |

233 | <DD>Number of points for the interpolating curve. If negative, say -3 (default), 3 points will be inserted between each data points.</DD> |

234 | </DL> |

235 | |

236 | <p>Constructs similar estimator as <CODE>ProbabilityEstimatorConstructor_loess</CODE>, except that it computes conditional probabilites. The result is of type <CODE>ConditionalProbabilityEstimator_FromDistribution</CODE>.</p> |

237 | |

238 | </BODY> |

**Note:**See TracBrowser for help on using the repository browser.