Ignore:
Timestamp:
03/15/12 21:05:43 (2 years ago)
Author:
blaz <blaz.zupan@…>
Branch:
default
Message:

Added stacking (ensemble method).

File:
1 edited

Legend:

Unmodified
Added
Removed
  • docs/reference/rst/Orange.ensemble.rst

    r10539 r10540  
    4747Example 
    4848======= 
    49 Let us try boosting and bagging on Lymphography data set and use TreeLearner 
    50 with post-pruning as a base learner. For testing, we use 10-fold cross 
    51 validation and observe classification accuracy. 
    52  
    53 :download:`ensemble.py <code/ensemble.py>` 
     49 
     50The following script fits classification models by boosting and 
     51bagging on Lymphography data set with TreeLearner and post-pruning as 
     52a base learner. Classification accuracy of the methods is estimated by 
     5310-fold cross validation (:download:`ensemble.py <code/ensemble.py>`): 
    5454 
    5555.. literalinclude:: code/ensemble.py 
    5656  :lines: 7- 
    5757 
    58 Running this script, we may get something like:: 
     58Running this script demonstrates some benefit of boosting and bagging 
     59over the baseline learner:: 
    5960 
    6061    Classification Accuracy: 
     
    6364        bagged tree: 0.790 
    6465 
     66******** 
     67Stacking 
     68******** 
     69 
     70.. index:: stacking 
     71.. index:: 
     72   single: ensemble; stacking 
     73 
     74 
     75.. autoclass:: Orange.ensemble.stacking.StackedClassificationLearner 
     76  :members: 
     77  :show-inheritance: 
     78 
     79.. autoclass:: Orange.ensemble.stacking.StackedClassifier 
     80   :members: 
     81   :show-inheritance: 
     82 
     83Example 
     84======= 
     85 
     86Stacking often produces classifiers that are more predictive than 
     87individual classifiers in the ensemble. This effect is illustrated by 
     88a script that combines four different classification 
     89algorithms (:download:`ensemble-stacking.py <code/ensemble-stacking.py>`): 
     90 
     91.. literalinclude:: code/ensemble-stacking.py 
     92  :lines: 3- 
     93 
     94The benefits of stacking on this particular data set are 
     95substantial (numbers show classification accuracy):: 
     96 
     97   stacking: 0.934 
     98      bayes: 0.858 
     99       tree: 0.688 
     100         lr: 0.764 
     101        knn: 0.830 
    65102 
    66103************* 
     
    173210---------- 
    174211 
    175 * L Breiman. Bagging Predictors. `Technical report No. 421 \ 
    176     <http://www.stat.berkeley.edu/tech-reports/421.ps.Z>`_. University of \ 
    177     California, Berkeley, 1994. 
    178 * Y Freund, RE Schapire. `Experiments with a New Boosting Algorithm \ 
    179     <http://citeseer.ist.psu.edu/freund96experiments.html>`_. Machine \ 
    180     Learning: Proceedings of the Thirteenth International Conference (ICML'96), 1996. 
    181 * JR Quinlan. `Boosting, bagging, and C4.5 \ 
    182     <http://www.rulequest.com/Personal/q.aaai96.ps>`_ . In Proc. of 13th \ 
    183     National Conference on Artificial Intelligence (AAAI'96). pp. 725-730, 1996.  
    184 * L Brieman. `Random Forests \ 
    185     <http://www.springerlink.com/content/u0p06167n6173512/>`_.\ 
    186     Machine Learning, 45, 5-32, 2001.  
    187 * M Robnik-Sikonja. `Improving Random Forests \ 
    188     <http://lkm.fri.uni-lj.si/rmarko/papers/robnik04-ecml.pdf>`_. In \ 
    189     Proc. of European Conference on Machine Learning (ECML 2004),\ 
    190     pp. 359-370, 2004. 
    191 """ 
     212* L Breiman. Bagging Predictors. `Technical report No. 421 
     213  <http://www.stat.berkeley.edu/tech-reports/421.ps.Z>`_. University 
     214  of California, Berkeley, 1994. 
     215* Y Freund, RE Schapire. `Experiments with a New Boosting Algorithm 
     216  <http://citeseer.ist.psu.edu/freund96experiments.html>`_. Machine 
     217  Learning: Proceedings of the Thirteenth International Conference 
     218  (ICML'96), 1996.  
     219* JR Quinlan. `Boosting, bagging, and C4.5 
     220  <http://www.rulequest.com/Personal/q.aaai96.ps>`_ . In Proc. of 13th 
     221  National Conference on Artificial Intelligence 
     222  (AAAI'96). pp. 725-730, 1996. 
     223* L Brieman. `Random Forests 
     224  <http://www.springerlink.com/content/u0p06167n6173512/>`_. Machine 
     225  Learning, 45, 5-32, 2001. 
     226* M Robnik-Sikonja. `Improving Random Forests 
     227  <http://lkm.fri.uni-lj.si/rmarko/papers/robnik04-ecml.pdf>`_. In 
     228  Proc. of European Conference on Machine Learning (ECML 2004), 
     229  pp. 359-370, 2004. 
    192230 
    193231.. automodule:: Orange.ensemble 
Note: See TracChangeset for help on using the changeset viewer.