Changeset 13:f98f0417ceb6 in orange-reliability

Timestamp:

07/11/12 20:58:41 (11 months ago)

Author:

Matija Polajnar <matija.polajnar@…>

Branch:

default

Message:

Documentation of reference method for classification.

Files:

• _reliability/__init__.py (modified) (1 diff)
• docs/rst/Orange.evaluation.reliability.rst (modified) (3 diffs)

_reliability/__init__.py

@@ -383 +383 @@
 
 class ReferenceExpectedError:
-
+    """
+
+    :rtype: :class:`Orange.evaluation.reliability.ReferenceExpectedErrorClassifier`
+
+    Reference reliability estimation method for classification as used in Evaluating Reliability of Single
+    Classifications of Neural Networks, Darko Pevec, 2011.
+
+    :math:`O_{ref} = 2 (\hat y - \hat y ^2) = 2 \hat y (1-\hat y)`
+
+    where :math:`\hat y` is the estimated probability of the predicted class.
+
+    Note that for this method, in contrast with all others, a greater estimate means lower reliability (greater
+    expected error).
+
+    """
     def __init__(self, name="reference"):
         self.name = name

docs/rst/Orange.evaluation.reliability.rst

@@ -10 +10 @@
 ##########################################################
 
-*************************************
-Reliability Estimation for Regression
-*************************************
+********************************************************
+Reliability Estimation for Regression and Classification
+********************************************************
 
 Reliability assessment statistically predicts reliability of single
@@ -49 +49 @@
 ===================
 
-For regression, all the described measures can be used. Classification domains
-are supported by the following methods: BAGV, LCV, CNK and DENS.
+For regression, all the described measures can be used, except for the :math:`O_{ref}`. Classification domains
+are supported by the following methods: BAGV, LCV, CNK and DENS, :math:`O_{ref}`.
 
 Sensitivity Analysis (SAvar and SAbias)
@@ -87 +87 @@
 
 .. autoclass:: ParzenWindowDensityBased
+
+Reference Estimate for Classification (:math:`O_{ref}`)
+-------------------------------------------------------
+
+.. autoclass:: ReferenceExpectedError
 
 Reliability estimation wrappers