source: orange/docs/reference/rst/Orange.regression.lasso.rst @ 10859:08a0a35c1687

Revision 10859:08a0a35c1687, 2.4 KB checked in by Lan Zagar <lan.zagar@…>, 2 years ago (diff)

Reimplemented lasso. Breaks compatibility.

It now uses a proximal gradient method for optimization instead of using scipy.optimize (see #1118).
The formulation is slightly different so there are new parameters (mainly lasso_lambda instead of t/s).
Improved some other things as well.

Line 
1############################
2Lasso regression (``lasso``)
3############################
4
5.. automodule:: Orange.regression.lasso
6
7.. index:: regression
8
9.. _`Lasso regression. Regression shrinkage and selection via the lasso`:
10    http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf
11
12
13`The lasso <http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf>`_
14(least absolute shrinkage and selection operator) is a regularized
15version of least squares regression.
16It minimizes the sum of squared errors while also penalizing the
17:math:`L_1` norm (sum of absolute values) of the coefficients.
18
19Concretely, the function that is minimized in Orange is:
20
21.. math:: \frac{1}{n}\|Xw - y\|_2^2 + \frac{\lambda}{m} \|w\|_1
22
23Where :math:`X` is a :math:`n \times m` data matrix, :math:`y` the vector
24of class values and :math:`w` the regression coefficients to be estimated.
25
26.. autoclass:: LassoRegressionLearner
27    :members:
28    :show-inheritance:
29
30.. autoclass:: LassoRegression
31    :members:
32    :show-inheritance:
33
34Utility functions
35-----------------
36
37.. autofunction:: get_bootstrap_sample
38
39.. autofunction:: permute_responses
40
41
42========
43Examples
44========
45
46To fit the regression parameters on housing data set use the following code:
47
48.. literalinclude:: code/lasso-example.py
49   :lines: 9,10,11
50
51To predict values of the response for the first five instances:
52
53.. literalinclude:: code/lasso-example.py
54   :lines: 15,16
55
56Output::
57
58    Actual: 24.00, predicted: 30.45
59    Actual: 21.60, predicted: 25.60
60    Actual: 34.70, predicted: 31.48
61    Actual: 33.40, predicted: 30.18
62    Actual: 36.20, predicted: 29.59
63
64To see the fitted regression coefficients, print the model:
65
66.. literalinclude:: code/lasso-example.py
67   :lines: 19
68
69Output::
70
71      Variable  Coeff Est  Std Error          p
72     Intercept     22.533
73          CRIM     -0.023      0.024      0.050     .
74          CHAS      1.970      1.331      0.040     *
75           NOX     -4.226      2.944      0.010     *
76            RM      4.270      0.934      0.000   ***
77           DIS     -0.373      0.170      0.010     *
78       PTRATIO     -0.798      0.117      0.000   ***
79             B      0.007      0.003      0.020     *
80         LSTAT     -0.519      0.102      0.000   ***
81    Signif. codes:  0 *** 0.001 ** 0.01 * 0.05 . 0.1 empty 1
82
83    For 5 variables the regression coefficient equals 0:
84    ZN, INDUS, AGE, RAD, TAX
85
86Note that some of the regression coefficients are equal to 0.   
87
Note: See TracBrowser for help on using the repository browser.