Orange Forum • View topic - Definition of RMSE

Definition of RMSE

A place to ask questions about methods in Orange and how they are used and other general support.

Definition of RMSE

Postby sreastman » Wed Mar 13, 2013 14:12

I was looking at the defintion of RMSE under the documentation for Orange.evaluation.scoring.

It defines it as the square root of SS sub R over n where SS sub R is the sum of the squares of the differences between each predicted value and the avarage value. All other defintions I have ever encountered would use what you call SS sub E instead of SS sub R, where SS sub E is the sum of the square the of differences between each actual and predicted value.

Is this a typo or is there a reason for it?

Also, there seems to be two different divisors in common use. Some people use n, the number of points. Others use, n - 2. Coould you also comment on this?

Thanks--Steve

Re: Definition of RMSE

Postby Ales » Fri Mar 15, 2013 12:07

sreastman wrote:Is this a typo or is there a reason for it?
It appears to be a typo.
sreastman wrote:Also, there seems to be two different divisors in common use. Some people use n, the number of points. Others use, n - 2. Coould you also comment on this?
The degrees of freedom correction is usually employed when evaluating a model on the training set to account for the learned model parameters.

The intended use of the Orange.evaluate/scoring module is to test the models on separate test data set (where the correction is not needed). It also works across different models at the same time (for model comparison), so a single degree of freedom can not be used as different models can learn different number of parameters.

Never the less maybe a 'dof' parameter could be added to the MSE,RMSE, ... functions, to be supplied by the user (since Orange.evaluate.learn_and_test_on_train_data does exist).


Return to Questions & Support



cron