# source:orange/docs/tutorial/rst/ensembles.rst@9994:1073e0304a87

Revision 9994:1073e0304a87, 2.3 KB checked in by Matija Polajnar <matija.polajnar@…>, 2 years ago (diff)

Remove links from documentation to datasets. Remove datasets reference directory.

Line
1.. index:: ensembles
2.. index::
3   single: ensembles; bagging
4.. index::
5   single: ensembles; boosting
6
7Ensemble learners
8=================
9
10Building ensemble classifiers in Orange is simple and easy. Starting
11from learners/classifiers that can predict probabilities and, if
12needed, use example weights, ensembles are actually wrappers that can
13aggregate predictions from a list of constructed classifiers. These
14wrappers behave exactly like other Orange learners/classifiers. We
15will here first show how to use a module for bagging and boosting that
16is included in Orange distribution (:py:mod:`Orange.ensemble` module), and
17then, for a somehow more advanced example build our own ensemble
18learner. Using this module, using it is very easy: you have to define
19a learner, give it to bagger or booster, which in turn returns a new
21
22   import orange, orngTest, orngStat, orngEnsemble
23   data = orange.ExampleTable("promoters")
24
25   majority = orange.MajorityLearner()
26   majority.name = "default"
27   knn = orange.kNNLearner(k=11)
28   knn.name = "k-NN (k=11)"
29
30   bagged_knn = orngEnsemble.BaggedLearner(knn, t=10)
31   bagged_knn.name = "bagged k-NN"
32   boosted_knn = orngEnsemble.BoostedLearner(knn, t=10)
33   boosted_knn.name = "boosted k-NN"
34
35   learners = [majority, knn, bagged_knn, boosted_knn]
36   results = orngTest.crossValidation(learners, data, folds=10)
37   print "        Learner   CA     Brier Score"
38   for i in range(len(learners)):
39       print ("%15s:  %5.3f  %5.3f") % (learners[i].name,
40           orngStat.CA(results)[i], orngStat.BrierScore(results)[i])
41
42Most of the code is used for defining and naming objects that learn,
43and the last piece of code is to report evaluation results. Notice
44that to bag or boost a learner, it takes only a single line of code
45(like, ``bagged_knn = orngEnsemble.BaggedLearner(knn, t=10)``)!
46Parameter ``t`` in bagging and boosting refers to number of
47classifiers that will be used for voting (or, if you like better,
48number of iterations by boosting/bagging). Depending on your random
49generator, you may get something like::
50
51           Learner   CA     Brier Score
52           default:  0.473  0.501
53       k-NN (k=11):  0.859  0.240
54       bagged k-NN:  0.813  0.257
55      boosted k-NN:  0.830  0.244
56
57
Note: See TracBrowser for help on using the repository browser.