Orange Forum • View topic - LIBLINEAR vs. LinearSVM

LIBLINEAR vs. LinearSVM

A place to ask questions about methods in Orange and how they are used and other general support.

LIBLINEAR vs. LinearSVM

Postby re832003 » Wed Mar 06, 2013 5:22

This is my code for using the Linear SVM classifier. It's working fine but I'm kind of suspicious about the results. They differ considerably from the LIBSVM results. I want to check my code: I'm either doing something wrong here or with LIBSVM. Are there any reported differences between the results of the two implementations?

Many thanks

Code: Select all
import Orange
import Orange.evaluation.scoring
from Orange.classification import svm

learners = [svm.LinearSVMLearner(name="L_SVM")]
data = Orange.data.Table("1234567_new.tab")   

res = Orange.evaluation.testing.cross_validation(learners, data, folds=5) # training procedure

# cofusition matrix of the first learner
cm = Orange.evaluation.scoring.confusion_matrices(res)[0]

Accuracy = Orange.evaluation.scoring.CA(cm)
Precision = Orange.evaluation.scoring.Precision(cm)
Recall = Orange.evaluation.scoring.Recall(cm)
F_measure = Orange.evaluation.scoring.F1(cm)

print "Accuracy", Accuracy, "Precision", Precision, "Recall", Recall, "F_measure", F_measure


Re: LIBLINEAR vs. LinearSVM

Postby Ales » Thu Mar 07, 2013 10:45

re832003 wrote:Are there any reported differences between the results of the two implementations?
Yes. They use different regularization parameters and/or methods (depending on which 'solver_type'/'svm_type' you choose), as well as different multi-class classification strategies (one vs. rest/one vs. one).

Re: LIBLINEAR vs. LinearSVM

Postby re832003 » Thu Mar 07, 2013 10:57

Many thanks for the reply. I used LinearSVM with the default parameters in your documentation.

Is there another place in which I can read more about the LinearSVM parameters so that I can document it properly?

Thanks,
Rania

Re: LIBLINEAR vs. LinearSVM

Postby Ales » Thu Mar 07, 2013 11:09

re832003 wrote:Is there another place in which I can read more about the LinearSVM parameters
See the original documentation of LIBLINEAR and LibSVM.

Re: LIBLINEAR vs. LinearSVM

Postby re832003 » Thu Mar 07, 2013 11:26

I'll explain myself more:

For the linearSVM wrapper from Orange, the paramters are

(1) solver_type=L2R_L2Loss_SVC_Dual << the default for LIBLINEAR
(2) C=1 << the default for LIBLINEAR
(3) eps=0.01 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR
(4) bias=1 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR
(5) normalization=True << This is the scaling part and in LIBLINEAR there are three possible options -1,1 | 0,1 | -1,0 and I tried them all.


So I know the parameters for Orange wrapper and I used the same parameters - except for the normalization one - when I used LIBLINEAR and I still get this difference. That's why I was asking for an explanation if possible.

Thanks

Re: LIBLINEAR vs. LinearSVM

Postby Ales » Thu Mar 07, 2013 13:48

Sorry, I completely misunderstood your problem/question.

Can you provide some more details about your data?
Does it have discrete features? In this case LinearSVMLearner will automatically expand every n-valued discrete feature into n new continuous indicator features (this definitely needs documenting).

Re: LIBLINEAR vs. LinearSVM

Postby re832003 » Thu Mar 07, 2013 15:45

Yes, all my features are discrete, even for the last feature (lemma), you can consider those values dummy numbers (I hope this is not a problem). I have two classes only {0,1}, 2076 instances and 5 features.

Here is a sample:

Modal POS-1 POS_token POS+1 Lemma
d d d d d
class
0 PR$ JJ RB 8:70
1 CC V V 8:1
1 NN V NN 8:1
1 RB V NNP 8:1
1 NG V NN 8:1
1 OBJP V CM 8:1
1 NG V V 8:1
1 CC V NN 8:1
1 BEG V NN 8:1
1 SBJP V V 8:1
1 BEG V NN 8:1
1 BEG V V 8:1
1 SBJP V NN 8:1

Re: LIBLINEAR vs. LinearSVM

Postby Ales » Fri Mar 08, 2013 11:50

re832003 wrote:Yes, all my features are discrete
How did you convert the dataset to LIBLINEAR format for testing?

If you do not want LinearSVMLearner to expand the discrete feature them you need to continuize the domain before learning (you can use DomainContinuizer using the 'AsOrdinal' flag).

Maybe it should be a good idea to add a 'multinomial_treatment' parameter to LinearSVMLearner?

Re: LIBLINEAR vs. LinearSVM

Postby re832003 » Sat Mar 09, 2013 1:24

re832003 wrote:If you do not want LinearSVMLearner to expand the discrete feature then you need to continuize the domain before learning (you can use DomainContinuizer using the 'AsOrdinal' flag).


Actually, I think this expansion really helps with my data. I get 6% increase when using your linearSVM instead of the pure implementation of LIBLINEAR.

I just want to know the details so I can document that as accurately as possible because I'm planning to publish my work very soon and I'll document your package and this expansion method to account for the performance difference I'm getting.

re832003 wrote:How did you convert the dataset to LIBLINEAR format for testing?


I just converted every discrete feature to a dummy number.

I'm not a machine learning person so I'm not sure I get all the math/stat details here. I googled "continuization" and found out that it is one way of kernel equating. Is that correct?

Re: LIBLINEAR vs. LinearSVM

Postby Ales » Mon Mar 11, 2013 12:41

re832003 wrote:I googled "continuization" and found out that it is one way of kernel equating. Is that correct?
I don't think so.
I think there is no standard naming convention for this.

Basically what the LinearSVMLearner does is it replaces a m-valued discrete feature (where m >= 3) with m new continuous features where for each example only one of this new features takes the value of 1.0 while the rest are 0.0, the 1.0 indicating which of the discrete values was present in the original column.

Note that this is exactly as A practical guide to SVM classification ('Data preprocessing' section) recommends to treat discrete (categorical) features (the guide is from the same authors as LIBLINEAR).

Re: LIBLINEAR vs. LinearSVM

Postby re832003 » Fri Mar 15, 2013 11:44

Many thanks

Rania


Return to Questions & Support