source: orange/docs/tutorial/rst/ensembles.rst @ 9385:fd37d2ce5541

Revision 9385:fd37d2ce5541, 2.4 KB checked in by mitar, 2 years ago (diff)

Cleaned up tutorial.

Line 
1.. index:: ensembles
2.. index:: 
3   single: ensembles; bagging
4.. index:: 
5   single: ensembles; boosting
6
7Ensemble learners
8=================
9
10Building ensemble classifiers in Orange is simple and easy. Starting
11from learners/classifiers that can predict probabilities and, if
12needed, use example weights, ensembles are actually wrappers that can
13aggregate predictions from a list of constructed classifiers. These
14wrappers behave exactly like other Orange learners/classifiers. We
15will here first show how to use a module for bagging and boosting that
16is included in Orange distribution (:py:mod:`Orange.ensemble` module), and
17then, for a somehow more advanced example build our own ensemble
18learner. Using this module, using it is very easy: you have to define
19a learner, give it to bagger or booster, which in turn returns a new
20(boosted or bagged) learner. Here goes an example (:download:`ensemble3.py <code/ensemble3.py>`,
21uses :download:`promoters.tab <code/promoters.tab>`)::
22
23   import orange, orngTest, orngStat, orngEnsemble
24   data = orange.ExampleTable("promoters")
25   
26   majority = orange.MajorityLearner()
27   majority.name = "default"
28   knn = orange.kNNLearner(k=11)
29   knn.name = "k-NN (k=11)"
30   
31   bagged_knn = orngEnsemble.BaggedLearner(knn, t=10)
32   bagged_knn.name = "bagged k-NN"
33   boosted_knn = orngEnsemble.BoostedLearner(knn, t=10)
34   boosted_knn.name = "boosted k-NN"
35   
36   learners = [majority, knn, bagged_knn, boosted_knn]
37   results = orngTest.crossValidation(learners, data, folds=10)
38   print "        Learner   CA     Brier Score"
39   for i in range(len(learners)):
40       print ("%15s:  %5.3f  %5.3f") % (learners[i].name,
41           orngStat.CA(results)[i], orngStat.BrierScore(results)[i])
42
43Most of the code is used for defining and naming objects that learn,
44and the last piece of code is to report evaluation results. Notice
45that to bag or boost a learner, it takes only a single line of code
46(like, ``bagged_knn = orngEnsemble.BaggedLearner(knn, t=10)``)!
47Parameter ``t`` in bagging and boosting refers to number of
48classifiers that will be used for voting (or, if you like better,
49number of iterations by boosting/bagging). Depending on your random
50generator, you may get something like::
51
52           Learner   CA     Brier Score
53           default:  0.473  0.501
54       k-NN (k=11):  0.859  0.240
55       bagged k-NN:  0.813  0.257
56      boosted k-NN:  0.830  0.244
57
58
Note: See TracBrowser for help on using the repository browser.