Partial least sqaures regression (PLS)

Partial least squares regression is a statistical method for simultaneous prediction of multiple response variables. Orange’s implementation is based on Scikit learn python implementation.

The following code shows how to fit a PLS regression model on a multi-target data set.

import Orange
data ="")
learner = Orange.regression.pls.PLSRegressionLearner()
classifier = learner(data)
class Orange.regression.pls.PLSRegressionLearner(n_comp=2, deflation_mode=regression, mode=PLS, algorithm=nipals, max_iter=500, imputer=None, continuizer=None, **kwds)

Fit the partial least squares regression model, i.e. learn the regression parameters. The implementation is based on Scikit learn python implementation

The class is derived from Orange.regression.base.BaseRegressionLearner that is used for preprocessing the data (continuization and imputation) before fitting the regression parameters

__call__(table, weight_id=None, x_vars=None, y_vars=None)
  • table ( – data instances.
  • y_vars (x_vars,) – List of input and response variables (Orange.feature.Continuous or Orange.feature.Discrete). If None (default) it is assumed that the data domain provides information which variables are reponses and which are not. If data has class_var defined in its domain, a single-target regression learner is constructed. Otherwise a multi-target learner predicting response variables defined by class_vars is constructed.
__init__(n_comp=2, deflation_mode=regression, mode=PLS, algorithm=nipals, max_iter=500, imputer=None, continuizer=None, **kwds)

number of components to keep (default: 2)


“canonical” or “regression” (default)


“CCA” or “PLS” (default)


The algorithm for estimating the weights: “nipals” or “svd” (default)

fit(X, Y)

Fit all unknown parameters, i.e. weights, scores, loadings (for x and y) and regression coefficients. Return a dict with all of the parameters.

class Orange.regression.pls.PLSRegression(domain=None, multitarget=False, coefs=None, sigma_x=None, sigma_y=None, mu_x=None, mu_y=None, x_vars=None, y_vars=None, **kwargs)

Predict values of the response variables based on the values of independent variables.

Basic notations: n - number of data instances p - number of independent variables q - number of reponse variables


A n x n_comp numpy array of x-scores


A n x n_comp numpy array of y-scores


A p x n_comp numpy array of x-weights


A q x n_comp numpy array of y-weights


A p x n_comp numpy array of x-loadings


A q x n_comp numpy array of y-loading


A p x q numpy array coefficients of the linear model: Y = X coefs + E


Predictor variables


Response variables

__call__(instance, result_type=0)
Parameters:instance ( – data instance for which the value of the response variable will be predicted

Pretty-prints the coefficient of the PLS regression model.

Utility functions


Normalize a matrix column-wise: subtract the means and divide by standard deviations. Returns the standardized matrix, sample mean and standard deviation

Parameters:X (numpy.array) – data matrix
Orange.regression.pls.nipals_xy(X, Y, mode=PLS, max_iter=500, tol=1e-06)

NIPALS algorithm; returns the first left and rigth singular vectors of X’Y.

  • Y (X,) – data matrix
  • mode (string) – possible values “PLS” (default) or “CCA”
  • max_iter (int) – maximal number of iterations (default: 500)
  • tol (a not negative float) – tolerance parameter; if norm of difference between two successive left singular vectors is less than tol, iteration is stopped
Orange.regression.pls.svd_xy(X, Y)

Return the first left and right singular vectors of X’Y.

Parameters:Y (X,) – data matrix


The following code predicts the values of output variables for the first two instances in data.

print "Prediction for the first 2 data instances: \n" 
for d in data[:2]:
    print "Actual    ", d.get_classes()
    print "Predicted ", classifier(d)
Actual     [<orange.Value 'Y1'='0.490'>, <orange.Value 'Y2'='1.237'>, <orange.Value 'Y3'='1.808'>, <orange.Value 'Y4'='0.422'>]
Predicted  [<orange.Value 'Y1'='0.613'>, <orange.Value 'Y2'='0.826'>, <orange.Value 'Y3'='1.084'>, <orange.Value 'Y4'='0.534'>]

Actual     [<orange.Value 'Y1'='0.167'>, <orange.Value 'Y2'='-0.664'>, <orange.Value 'Y3'='-1.378'>, <orange.Value 'Y4'='0.589'>]
Predicted  [<orange.Value 'Y1'='0.058'>, <orange.Value 'Y2'='-0.706'>, <orange.Value 'Y3'='-1.420'>, <orange.Value 'Y4'='0.599'>]

To see the coefficient of the model, print the model:

print 'Regression coefficients:\n', classifier    
Regression coefficients:
                   Y1           Y2           Y3           Y4
      X1        0.714        2.153        3.590       -0.078 
      X2       -0.238       -2.500       -4.797       -0.036 
      X3        0.230       -0.314       -0.880       -0.060 

Note that coefficients are stored in a matrix since the model predicts values of multiple outputs.