Changeset 7757:c694f77187ab in orange


Ignore:
Timestamp:
03/16/11 17:14:27 (3 years ago)
Author:
jzbontar <jure.zbontar@…>
Branch:
default
Convert:
eff0c0b3fd8855325c8663b2d327726bbab1cb29
Message:

Mereged Logistic regression documentation.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • orange/Orange/classification/logreg.py

    r7747 r7757  
    1616features, is also supported. 
    1717 
     18Logistic regression is a popular classification method that comes 
     19from statistics. The model is described by a linear combination of 
     20coefficients, 
     21 
     22.. math:: 
     23     
     24    F = \\beta_0 + \\beta_1*X_1 + \\beta_2*X_2 + ... + \\beta_k*X_k 
     25 
     26and the probability (p) of a class value is  computed as: 
     27 
     28.. math:: 
     29 
     30    p = \\frac{\exp(F)}{1 + \exp(F)} 
     31 
     32 
     33.. class :: LogRegClassifier 
     34 
     35    :obj:`LogRegClassifier` stores estimated values of regression 
     36    coefficients and their significances, and uses them to predict 
     37    classes and class probabilities using the equations described above. 
     38 
     39    .. attribute :: beta 
     40 
     41        Estimated regression coefficients. 
     42 
     43    .. attribute :: beta_se 
     44 
     45        Estimated standard errors for regression coefficients. 
     46 
     47    .. attribute :: wald_Z 
     48 
     49        Wald Z statistics for beta coefficients. Wald Z is computed 
     50        as beta/beta_se. 
     51 
     52    .. attribute :: P 
     53 
     54        List of P-values for beta coefficients, that is, the probability 
     55        that beta coefficients differ from 0.0. The probability is 
     56        computed from squared Wald Z statistics that is distributed with 
     57        Chi-Square distribution. 
     58 
     59    .. attribute :: likelihood 
     60 
     61        The probability of the sample (ie. learning examples) observed on 
     62        the basis of the derived model, as a function of the regression 
     63        parameters. 
     64 
     65    .. attribute :: fitStatus 
     66 
     67        Tells how the model fitting ended - either regularly 
     68        (:obj:`LogRegFitter.OK`), or it was interrupted due to one of beta 
     69        coefficients escaping towards infinity (:obj:`LogRegFitter.Infinity`) 
     70        or since the values didn't converge (:obj:`LogRegFitter.Divergence`). The 
     71        value tells about the classifier's "reliability"; the classifier 
     72        itself is useful in either case. 
    1873 
    1974.. autoclass:: LogRegLearner 
     75 
     76.. class:: LogRegFitter 
     77 
     78    :obj:`LogRegFitter` is the abstract base class for logistic fitters. It 
     79    defines the form of call operator and the constants denoting its 
     80    (un)success: 
     81 
     82    .. attribute:: OK 
     83 
     84        Fitter succeeded to converge to the optimal fit. 
     85 
     86    .. attribute:: Infinity 
     87 
     88        Fitter failed due to one or more beta coefficients escaping towards infinity. 
     89 
     90    .. attribute:: Divergence 
     91 
     92        Beta coefficients failed to converge, but none of beta coefficients escaped. 
     93 
     94    .. attribute:: Constant 
     95 
     96        There is a constant attribute that causes the matrix to be singular. 
     97 
     98    .. attribute:: Singularity 
     99 
     100        The matrix is singular. 
     101 
     102 
     103    .. method:: __call__(examples, weightID) 
     104 
     105        Performs the fitting. There can be two different cases: either 
     106        the fitting succeeded to find a set of beta coefficients (although 
     107        possibly with difficulties) or the fitting failed altogether. The 
     108        two cases return different results. 
     109 
     110        `(status, beta, beta_se, likelihood)` 
     111            The fitter managed to fit the model. The first element of 
     112            the tuple, result, tells about the problems occurred; it can 
     113            be either :obj:`OK`, :obj:`Infinity` or :obj:`Divergence`. In 
     114            the latter cases, returned values may still be useful for 
     115            making predictions, but it's recommended that you inspect 
     116            the coefficients and their errors and make your decision 
     117            whether to use the model or not. 
     118 
     119        `(status, attribute)` 
     120            The fitter failed and the returned attribute is responsible 
     121            for it. The type of failure is reported in status, which 
     122            can be either :obj:`Constant` or :obj:`Singularity`. 
     123 
     124        The proper way of calling the fitter is to expect and handle all 
     125        the situations described. For instance, if fitter is an instance 
     126        of some fitter and examples contain a set of suitable examples, 
     127        a script should look like this:: 
     128 
     129            res = fitter(examples) 
     130            if res[0] in [fitter.OK, fitter.Infinity, fitter.Divergence]: 
     131               status, beta, beta_se, likelihood = res 
     132               < proceed by doing something with what you got > 
     133            else: 
     134               status, attr = res 
     135               < remove the attribute or complain to the user or ... > 
     136 
     137 
     138.. class :: LogRegFitter_Cholesky 
     139 
     140    :obj:`LogRegFitter_Cholesky` is the sole fitter available at the 
     141    moment. It is a C++ translation of `Alan Miller's logistic regression 
     142    code <http://users.bigpond.net.au/amiller/>`_. It uses Newton-Raphson 
     143    algorithm to iteratively minimize least squares error computed from 
     144    learning examples. 
     145 
     146 
    20147.. autoclass:: StepWiseFSS 
    21148.. autofunction:: dump 
    22149 
     150 
     151 
    23152Examples 
    24 ======== 
     153-------- 
    25154 
    26155The first example shows a very simple induction of a logistic regression 
     
    142271""" 
    143272 
    144 #from Orange.core import LogRegLearner, LogRegClassifier, LogRegFitter, LogRegFitter_Cholesky 
     273from Orange.core import LogRegLearner, LogRegClassifier, LogRegFitter, LogRegFitter_Cholesky 
    145274 
    146275import Orange 
Note: See TracChangeset for help on using the changeset viewer.