Changeset 7944:e4b6c5bda633 in orange


Ignore:
Timestamp:
05/26/11 12:13:16 (3 years ago)
Author:
mocnik <mocnik@…>
Branch:
default
Convert:
8147848132e28d3fb4d4e36b633d1275521cfb38
Message:

Updating formulas in documentation.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • orange/Orange/evaluation/reliability.py

    r7923 r7944  
    2929 
    3030To estimate the reliabilty for given example we extend the learning set  
    31 with given example and labeling it with K + e(l\ :sub:`max` \ - l\ :sub:`min` \), 
    32 where K denotes the initial prediction, e is sensitivity parameter and 
    33 l\ :sub:`min` \ and l\ :sub:`max` \ denote lower and the upper bound of 
     31with given example and labeling it with :math:`K + \epsilon (l_{max} - l_{min})`, 
     32where K denotes the initial prediction, :math:`\epsilon` is sensitivity parameter and 
     33:math:`l_{min}` and :math:`l_{max}` denote lower and the upper bound of 
    3434the learning examples. After computing different sensitivity predictions 
    3535using different values of e, the prediction are combined into SAvar and SAbias. 
    3636SAbias can be used as signed estimate or as absolute value of SAbias.  
    3737 
     38:math:`SAvar = \\frac{\sum_{\epsilon \in E}(K_{\epsilon} - K_{-\epsilon})}{|E|}` 
     39 
     40:math:`SAbias = \\frac{\sum_{\epsilon \in E} (K_{\epsilon} - K ) + (K_{-\epsilon} - K)}{2 |E|}` 
     41 
    3842Variance of bagged models (BAGV) 
    3943-------------------------------- 
    4044 
    4145We construct m different bagging models of the original chosen learner and use 
    42 those predictions of given example to calculate the variance, which we use as 
     46those predictions (:math:`K_i, i = 1, ..., m`) of given example to calculate the variance, which we use as 
    4347reliability estimator. 
     48 
     49:math:`BAGV = \\frac{1}{m} \sum_{i=1}^{m} (K_i - K)^2` 
     50 
     51where 
     52 
     53:math:`K = \\frac{\sum_{i=1}^{m} K_i}{m}` 
    4454 
    4555Local cross validation reliability estimate (LCV) 
     
    5161weighted absolute prediction error. 
    5262 
     631. define the set of k nearest neighours :math:`N = { (x_1, x_1),..., (x_k, c_k)}` 
     642. FOR EACH :math:`(x_i, c_i) \in N` 
     65 
     66  2.1. generare model M on :math:`N \\backslash (x_i, c_i)` 
     67   
     68  2.2. for :math:`(x_i, c_i)` compute LOO prediction :math:`K_i` 
     69   
     70  2.3. for :math:`(x_i, c_i)` compute LOO error :math:`E_i = | C_i - K_i |` 
     71   
     723. :math:`LCV(x) = \\frac{ \sum_{(x_i, c_i) \in N} d(x_i, x) * E_i }{ \sum_{(x_i, c_i) \in N} d(x_i, x) }` 
     73 
    5374Local modeling of prediction error (CNK) 
    5475---------------------------------------- 
     
    5778average label of the nearest neighbours and the examples prediction. CNK can 
    5879be used as a signed estimate or only as absolute value.  
     80 
     81:math:`CNK = \\frac{\sum_{i=1}^{k}C_i}{k} - K` 
     82 
     83Where k denotes number of neighbors, C :sub:`i` denotes neighbours' labels and 
     84K denotes the example's prediction. 
    5985 
    6086Bagging variance c-neighbours (BVCK) 
     
    278304        :param weight: Id of meta attribute with weights of instances 
    279305        :type weight: integer 
    280         :rtype: :class:`estimator.Classifier` 
     306        :rtype: :class:`Orange.evaluation.reliability.Classifier` 
    281307        """ 
    282308        return Classifier(examples, self.learner, self.e, self.m, self.cnk_k, 
     
    440466            SAvar /= len(self.e) 
    441467            SAbias /= 2*len(self.e) 
    442             probabilities.reliability_estimate.append( (SAvar, ABSOLUTE, SAVAR_ABSOLUTE) ) 
    443             probabilities.reliability_estimate.append( (SAbias, SIGNED, SABIAS_SIGNED) ) 
    444             probabilities.reliability_estimate.append( (SAbias, ABSOLUTE, SABIAS_ABSOLUTE) ) 
     468            probabilities.reliability_estimate.append( Estimate(SAvar, ABSOLUTE, SAVAR_ABSOLUTE) ) 
     469            probabilities.reliability_estimate.append( Estimate(SAbias, SIGNED, SABIAS_SIGNED) ) 
     470            probabilities.reliability_estimate.append( Estimate(SAbias, ABSOLUTE, SABIAS_ABSOLUTE) ) 
    445471         
    446472         
     
    462488             
    463489            if self.use[DO_BAGV]: 
    464                 probabilities.reliability_estimate.append( (BAGV, ABSOLUTE, BAGV_ABSOLUTE) ) 
     490                probabilities.reliability_estimate.append( Estimate(BAGV, ABSOLUTE, BAGV_ABSOLUTE) ) 
    465491         
    466492        # For each of the classifiers 
     
    491517             
    492518            if self.use[DO_CNK]: 
    493                 probabilities.reliability_estimate.append( (CNK, SIGNED, CNK_SIGNED) ) 
    494                 probabilities.reliability_estimate.append( (CNK, ABSOLUTE, CNK_ABSOLUTE) ) 
     519                probabilities.reliability_estimate.append( Estimate(CNK, SIGNED, CNK_SIGNED) ) 
     520                probabilities.reliability_estimate.append( Estimate(CNK, ABSOLUTE, CNK_ABSOLUTE) ) 
    495521         
    496522        # Calculate local cross-validation reliability estimate 
     
    524550            LCV = LCVer / LCVdi if LCVdi != 0 else 0 
    525551             
    526             probabilities.reliability_estimate.append( (LCV, ABSOLUTE, LCV_ABSOLUTE) ) 
     552            probabilities.reliability_estimate.append( Estimate(LCV, ABSOLUTE, LCV_ABSOLUTE) ) 
    527553         
    528554        # BVCK 
     
    534560            BVCK = (BAGV + abs(CNK)) / 2 
    535561             
    536             probabilities.reliability_estimate.append( (BVCK, ABSOLUTE, BVCK_ABSOLUTE) ) 
     562            probabilities.reliability_estimate.append( Estimate(BVCK, ABSOLUTE, BVCK_ABSOLUTE) ) 
    537563         
    538564        # Mahalanobis distance to 3 closest neighbours 
     
    544570            mahalanobis_distance = sum(ex[self.mid].value for ex in self.nnm(example, 3)) 
    545571             
    546             probabilities.reliability_estimate.append( (mahalanobis_distance, ABSOLUTE, MAHAL_ABSOLUTE) ) 
     572            probabilities.reliability_estimate.append( Estimate(mahalanobis_distance, ABSOLUTE, MAHAL_ABSOLUTE) ) 
    547573             
    548574        #probabilities.reliability_estimate = [SAvar, SAbias, BAGV, CNK, LCV, BVCK, mahalanobis_distance] 
     
    551577            method = [SAvar, SAbias, SAbias, BAGV, CNK, CNK, LCV, BVCK, mahalanobis_distance] 
    552578            ICV = method[self.icv_method] 
    553             probabilities.reliability_estimate.append( (ICV, self.icv_signed_or_absolute, 10, self.icv_method)) 
     579            probabilities.reliability_estimate.append( Estimate(ICV, self.icv_signed_or_absolute, 10, self.icv_method)) 
    554580         
    555581        if result_type == Orange.core.GetValue: 
     
    561587 
    562588def get_reliability_estimation_list(res, i): 
    563     return [result.probabilities[0].reliability_estimate[i][0] for result in res.results], res.results[0].probabilities[0].reliability_estimate[i][1], res.results[0].probabilities[0].reliability_estimate[i][2] 
     589    return [result.probabilities[0].reliability_estimate[i].estimate for result in res.results], res.results[0].probabilities[0].reliability_estimate[i].signed_or_absolute, res.results[0].probabilities[0].reliability_estimate[i].method 
    564590 
    565591def get_prediction_error_list(res): 
     
    628654    return statc.betai (df * 0.5, 0.5, df/(df + t*t)) 
    629655 
     656class Estimate: 
     657    def __init__(self, estimate, signed_or_absolute, method, icv_method = -1): 
     658        self.estimate = estimate 
     659        self.signed_or_absolute = signed_or_absolute 
     660        self.method = method 
     661        self.method_name = METHOD_NAME[method] 
     662        self.icv_method = icv_method 
     663        self.icv_method_name = METHOD_NAME[icv_method] if icv_method != -1 else "" 
Note: See TracChangeset for help on using the changeset viewer.