#
source:
orange/docs/reference/rst/Orange.regression.lasso.rst
@
10859:08a0a35c1687

Revision 10859:08a0a35c1687, 2.4 KB checked in by Lan Zagar <lan.zagar@…>, 2 years ago (diff) |
---|

Line | |
---|---|

1 | ############################ |

2 | Lasso regression (``lasso``) |

3 | ############################ |

4 | |

5 | .. automodule:: Orange.regression.lasso |

6 | |

7 | .. index:: regression |

8 | |

9 | .. _`Lasso regression. Regression shrinkage and selection via the lasso`: |

10 | http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf |

11 | |

12 | |

13 | `The lasso <http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf>`_ |

14 | (least absolute shrinkage and selection operator) is a regularized |

15 | version of least squares regression. |

16 | It minimizes the sum of squared errors while also penalizing the |

17 | :math:`L_1` norm (sum of absolute values) of the coefficients. |

18 | |

19 | Concretely, the function that is minimized in Orange is: |

20 | |

21 | .. math:: \frac{1}{n}\|Xw - y\|_2^2 + \frac{\lambda}{m} \|w\|_1 |

22 | |

23 | Where :math:`X` is a :math:`n \times m` data matrix, :math:`y` the vector |

24 | of class values and :math:`w` the regression coefficients to be estimated. |

25 | |

26 | .. autoclass:: LassoRegressionLearner |

27 | :members: |

28 | :show-inheritance: |

29 | |

30 | .. autoclass:: LassoRegression |

31 | :members: |

32 | :show-inheritance: |

33 | |

34 | Utility functions |

35 | ----------------- |

36 | |

37 | .. autofunction:: get_bootstrap_sample |

38 | |

39 | .. autofunction:: permute_responses |

40 | |

41 | |

42 | ======== |

43 | Examples |

44 | ======== |

45 | |

46 | To fit the regression parameters on housing data set use the following code: |

47 | |

48 | .. literalinclude:: code/lasso-example.py |

49 | :lines: 9,10,11 |

50 | |

51 | To predict values of the response for the first five instances: |

52 | |

53 | .. literalinclude:: code/lasso-example.py |

54 | :lines: 15,16 |

55 | |

56 | Output:: |

57 | |

58 | Actual: 24.00, predicted: 30.45 |

59 | Actual: 21.60, predicted: 25.60 |

60 | Actual: 34.70, predicted: 31.48 |

61 | Actual: 33.40, predicted: 30.18 |

62 | Actual: 36.20, predicted: 29.59 |

63 | |

64 | To see the fitted regression coefficients, print the model: |

65 | |

66 | .. literalinclude:: code/lasso-example.py |

67 | :lines: 19 |

68 | |

69 | Output:: |

70 | |

71 | Variable Coeff Est Std Error p |

72 | Intercept 22.533 |

73 | CRIM -0.023 0.024 0.050 . |

74 | CHAS 1.970 1.331 0.040 * |

75 | NOX -4.226 2.944 0.010 * |

76 | RM 4.270 0.934 0.000 *** |

77 | DIS -0.373 0.170 0.010 * |

78 | PTRATIO -0.798 0.117 0.000 *** |

79 | B 0.007 0.003 0.020 * |

80 | LSTAT -0.519 0.102 0.000 *** |

81 | Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 empty 1 |

82 | |

83 | For 5 variables the regression coefficient equals 0: |

84 | ZN, INDUS, AGE, RAD, TAX |

85 | |

86 | Note that some of the regression coefficients are equal to 0. |

87 |

**Note:**See TracBrowser for help on using the repository browser.