Index: docs/rst/Orange.evaluation.reliability.rst
===================================================================
 docs/rst/Orange.evaluation.reliability.rst (revision 53)
+++ docs/rst/Orange.evaluation.reliability.rst (revision 54)
@@ 15,14 +15,13 @@
Reliability assessment aims to predict reliabilities of individual
predictions. Most of implemented algorithms for regression described in
[Bosnic2008]_ and in [Pevec2011]_ for classification.
+predictions. Most of the implemented algorithms for regression are described in
+[Bosnic2008]_; the algorithms for classification are described in [Pevec2011]_.
We can use reliability estimation with any Orange learners. The following example:
+We can use reliability estimation with any prediction method. The following example:
* Constructs reliability estimators (implemented in this module),
 * The :obj:`Learner` wrapper combines a regular learner, here a :obj:`~Orange.classification.knn.kNNLearner`, with reliability estimators.
 * Obtains prediction probabilities from the constructed classifier
 (:obj:`Orange.classification.Classifier.GetBoth` option). The resulting
 probabilities have an additional attribute, :obj:`reliability_estimate`,
+ * The :obj:`Learner` wrapper combines a a prediction method (learner), here a :obj:`~Orange.classification.knn.kNNLearner`, with reliability estimators.
+ * Obtains prediction probabilities, which have an additional attribute,
+ :obj:`reliability_estimate`,
that contains a list of :class:`Orange.evaluation.reliability.Estimate`.
@@ 30,5 +29,5 @@
:lines: 7
We could also evaluate more examples. The next example prints reliability estimates
+The next example prints reliability estimates
for first 10 instances (with crossvalidation):
@@ 117,5 +116,5 @@
A dictionary that that maps reliability estimation
 method IDs (integerss) to method names (strings).
+ method IDs (integers) to method names (strings).
.. autoclass:: Estimate
Index: orangecontrib/reliability/__init__.py
===================================================================
 orangecontrib/reliability/__init__.py (revision 53)
+++ orangecontrib/reliability/__init__.py (revision 54)
@@ 52,11 +52,9 @@
def get_pearson_r(res):
"""
 :param res: results of evaluation, done using learners,
 wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+ :param res: Evaluation results with :obj:`reliability_estimate`.
:type res: :class:`Orange.evaluation.testing.ExperimentResults`
 Return Pearson's coefficient between the prediction error and each of the
 used reliability estimates. Also, return the pvalue of each of
 the coefficients.
+ Pearson's coefficients between the prediction error and
+ reliability estimates with pvalues.
"""
prediction_error = get_prediction_error_list(res)
@@ 76,11 +74,9 @@
def get_spearman_r(res):
"""
 :param res: results of evaluation, done using learners,
 wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+ :param res: Evaluation results with :obj:`reliability_estimate`.
:type res: :class:`Orange.evaluation.testing.ExperimentResults`
 Return Spearman's coefficient between the prediction error and each of the
 used reliability estimates. Also, return the pvalue of each of
 the coefficients.
+ Spearman's coefficients between the prediction error and
+ reliability estimates with pvalues.
"""
prediction_error = get_prediction_error_list(res)
@@ 100,10 +96,9 @@
def get_pearson_r_by_iterations(res):
"""
 :param res: results of evaluation, done using learners,
 wrapped into :class:`Orange.evaluation.reliability.Classifier`.
+ :param res: Evaluation results with :obj:`reliability_estimate`.
:type res: :class:`Orange.evaluation.testing.ExperimentResults`
 Return average Pearson's coefficient over all folds between prediction error
 and each of the used estimates.
+ Pearson's coefficients between prediction error
+ and reliability estimates averaged over all folds.
"""
results_by_fold = Orange.evaluation.scoring.split_by_iterations(res)
@@ 112,4 +107,5 @@
number_of_folds = len(results_by_fold)
results = [0 for _ in xrange(number_of_estimates)]
+ M
sig = [0 for _ in xrange(number_of_estimates)]
method_list = [0 for _ in xrange(number_of_estimates)]
@@ 199,23 +195,22 @@
class Estimate:
"""
 Reliability estimate. Contains attributes that describe the results of
 reliability estimation.
+ Describes a reliability estimate.
.. attribute:: estimate
 A numerical reliability estimate.
+ Value of reliability.
.. attribute:: signed_or_absolute
 Determines whether the method used gives a signed or absolute result.
+ Determines whether the method returned a signed or absolute result.
Has a value of either :obj:`SIGNED` or :obj:`ABSOLUTE`.
.. attribute:: method
 An integer ID of reliability estimation method used.
+ An integer ID of the reliability estimation method used.
.. attribute:: method_name
 Name (string) of reliability estimation method used.
+ Name (string) of the reliability estimation method used.
"""
@@ 281,11 +276,10 @@
:rtype: :class:`Orange.evaluation.reliability.SensitivityAnalysisClassifier`
 To estimate the reliability of prediction for a given instance,
 the learning set is extended with that instance with the label changes to
+ The learning set is extended with that instancem, where the label is changed to
:math:`K + \epsilon (l_{max}  l_{min})` (:math:`K` is the initial prediction,
:math:`\epsilon` a sensitivity parameter, and :math:`l_{min}` and
 :math:`l_{max}` the lower and upper bounds of labels on training data)
+ :math:`l_{max}` the lower and upper bounds of labels on training data).
Results for multiple values of :math:`\epsilon` are combined
 into SAvar and SAbias. SAbias can be used either in a signed or absolute form.
+ into SAvar and SAbias. SAbias has a signed or absolute form.
:math:`SAvar = \\frac{\sum_{\epsilon \in E}(K_{\epsilon}  K_{\epsilon})}{E}`
@@ 386,8 +380,8 @@
"""
 :param m: Number of bagging models to be used with BAGV estimate
+ :param m: Number of bagged models. Default: 50.
:type m: int
 :param for instances: Optional. If test instances
+ :param for_instances: Optional. If test instances
are given as a parameter, this class can compute their reliabilities
on the fly, which saves memory.
@@ 397,9 +391,5 @@
:rtype: :class:`Orange.evaluation.reliability.BaggingVarianceClassifier`

 :math:`m` different bagging models are used to estimate
 the value of dependent variable for a given instance. For regression,
 the variance of predictions is a reliability
 estimate:
+ For regression, BAGV is the variance of predictions:
:math:`BAGV = \\frac{1}{m} \sum_{i=1}^{m} (K_i  K)^2`, where
@@ 407,10 +397,9 @@
predictions of individual models.
 For classification, 1 minus the average Euclidean distance between class
 probability distributions predicted by the model, and distributions
 predicted by the individual bagged models, is the BAGV reliability
 measure. For classification, a greater value implies a better
 prediction.

+ For classification, BAGV is 1 minus the average Euclidean
+ distance between class probability distributions predicted by the
+ model, and distributions predicted by the individual bagged model;
+ a greater value implies a better prediction.
+
This reliability measure can run out of memory if individual classifiers themselves
use a lot of memory; it needs :math:`m` times memory
@@ 502,5 +491,5 @@
:type distance: function
 :param distance_weighted: For classification,
+ :param distance_weighted: Relevant only for classification;
use an average distance between distributions, weighted by :math:`e^{d}`,
where :math:`d` is the distance between predicted instance and the
@@ 594,5 +583,5 @@
:rtype: :class:`Orange.evaluation.reliability.CNeighboursClassifier`
 For regression, CNK is defined a difference
+ For regression, CNK is a difference
between average label of its nearest neighbours and the prediction. CNK
can be either signed or absolute. A greater value implies greater prediction error.
@@ 1070,10 +1059,9 @@
class Learner:
"""
 Adds reliability estimation to any learner: multiple reliability estimation
 algorithms can be used simultaneously.
 This learner can be used as any other learner,
+ Adds reliability estimation to any prediction method.
+ This class can be used as any other Orange learner,
but returns the classifier wrapped into an instance of
:class:`Orange.evaluation.reliability.Classifier`.

+
:param box_learner: Learner to wrap into a reliability estimation
classifier.
@@ 1106,7 +1094,7 @@
def __call__(self, instances, weight=None, **kwds):
 """Learn from the given table of data instances.
+ """Construct a classifier.
 :param instances: Data to learn from.
+ :param instances: Learning data.
:type instances: Orange.data.Table
:param weight: Id of meta attribute with weights of instances
@@ 1149,5 +1137,5 @@
def __call__(self, instance, result_type=Orange.core.GetValue):
"""
 Classify and estimate reliability of estimation for a new instance.
+ Classify and estimate reliability for a new instance.
When :obj:`result_type` is set to
:obj:`Orange.classification.Classifier.GetBoth` or