## LIBLINEAR vs. LinearSVM

11 posts
• Page

**1**of**1**### LIBLINEAR vs. LinearSVM

This is my code for using the Linear SVM classifier. It's working fine but I'm kind of suspicious about the results. They differ considerably from the LIBSVM results. I want to check my code: I'm either doing something wrong here or with LIBSVM. Are there any reported differences between the results of the two implementations?

Many thanks

Many thanks

- Code: Select all
`import Orange`

import Orange.evaluation.scoring

from Orange.classification import svm

learners = [svm.LinearSVMLearner(name="L_SVM")]

data = Orange.data.Table("1234567_new.tab")

res = Orange.evaluation.testing.cross_validation(learners, data, folds=5) # training procedure

# cofusition matrix of the first learner

cm = Orange.evaluation.scoring.confusion_matrices(res)[0]

Accuracy = Orange.evaluation.scoring.CA(cm)

Precision = Orange.evaluation.scoring.Precision(cm)

Recall = Orange.evaluation.scoring.Recall(cm)

F_measure = Orange.evaluation.scoring.F1(cm)

print "Accuracy", Accuracy, "Precision", Precision, "Recall", Recall, "F_measure", F_measure

### Re: LIBLINEAR vs. LinearSVM

Yes. They use different regularization parameters and/or methods (depending on which 'solver_type'/'svm_type' you choose), as well as different multi-class classification strategies (one vs. rest/one vs. one).re832003 wrote:Are there any reported differences between the results of the two implementations?

### Re: LIBLINEAR vs. LinearSVM

Many thanks for the reply. I used LinearSVM with the default parameters in your documentation.

Is there another place in which I can read more about the LinearSVM parameters so that I can document it properly?

Thanks,

Rania

Is there another place in which I can read more about the LinearSVM parameters so that I can document it properly?

Thanks,

Rania

### Re: LIBLINEAR vs. LinearSVM

See the original documentation of LIBLINEAR and LibSVM.re832003 wrote:Is there another place in which I can read more about the LinearSVM parameters

### Re: LIBLINEAR vs. LinearSVM

I'll explain myself more:

For the linearSVM wrapper from Orange, the paramters are

(1) solver_type=L2R_L2Loss_SVC_Dual << the default for LIBLINEAR

(2) C=1 << the default for LIBLINEAR

(3) eps=0.01 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR

(4) bias=1 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR

(5) normalization=True << This is the scaling part and in LIBLINEAR there are three possible options -1,1 | 0,1 | -1,0 and I tried them all.

So I know the parameters for Orange wrapper and I used the same parameters - except for the normalization one - when I used LIBLINEAR and I still get this difference. That's why I was asking for an explanation if possible.

Thanks

For the linearSVM wrapper from Orange, the paramters are

(1) solver_type=L2R_L2Loss_SVC_Dual << the default for LIBLINEAR

(2) C=1 << the default for LIBLINEAR

(3) eps=0.01 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR

(4) bias=1 << not the default for LIBLINEAR but I set it when I tried LIBLINEAR

(5) normalization=True << This is the scaling part and in LIBLINEAR there are three possible options -1,1 | 0,1 | -1,0 and I tried them all.

So I know the parameters for Orange wrapper and I used the same parameters - except for the normalization one - when I used LIBLINEAR and I still get this difference. That's why I was asking for an explanation if possible.

Thanks

### Re: LIBLINEAR vs. LinearSVM

Sorry, I completely misunderstood your problem/question.

Can you provide some more details about your data?

Does it have discrete features? In this case LinearSVMLearner will automatically expand every n-valued discrete feature into n new continuous indicator features (this definitely needs documenting).

Can you provide some more details about your data?

Does it have discrete features? In this case LinearSVMLearner will automatically expand every n-valued discrete feature into n new continuous indicator features (this definitely needs documenting).

### Re: LIBLINEAR vs. LinearSVM

Yes, all my features are discrete, even for the last feature (lemma), you can consider those values dummy numbers (I hope this is not a problem). I have two classes only {0,1}, 2076 instances and 5 features.

Here is a sample:

Modal POS-1 POS_token POS+1 Lemma

d d d d d

class

0 PR$ JJ RB 8:70

1 CC V V 8:1

1 NN V NN 8:1

1 RB V NNP 8:1

1 NG V NN 8:1

1 OBJP V CM 8:1

1 NG V V 8:1

1 CC V NN 8:1

1 BEG V NN 8:1

1 SBJP V V 8:1

1 BEG V NN 8:1

1 BEG V V 8:1

1 SBJP V NN 8:1

Here is a sample:

Modal POS-1 POS_token POS+1 Lemma

d d d d d

class

0 PR$ JJ RB 8:70

1 CC V V 8:1

1 NN V NN 8:1

1 RB V NNP 8:1

1 NG V NN 8:1

1 OBJP V CM 8:1

1 NG V V 8:1

1 CC V NN 8:1

1 BEG V NN 8:1

1 SBJP V V 8:1

1 BEG V NN 8:1

1 BEG V V 8:1

1 SBJP V NN 8:1

### Re: LIBLINEAR vs. LinearSVM

How did you convert the dataset to LIBLINEAR format for testing?re832003 wrote:Yes, all my features are discrete

If you do not want LinearSVMLearner to expand the discrete feature them you need to continuize the domain before learning (you can use DomainContinuizer using the 'AsOrdinal' flag).

Maybe it should be a good idea to add a 'multinomial_treatment' parameter to LinearSVMLearner?

### Re: LIBLINEAR vs. LinearSVM

re832003 wrote:If you do not want LinearSVMLearner to expand the discrete feature then you need to continuize the domain before learning (you can use DomainContinuizer using the 'AsOrdinal' flag).

Actually, I think this expansion really helps with my data. I get 6% increase when using your linearSVM instead of the pure implementation of LIBLINEAR.

I just want to know the details so I can document that as accurately as possible because I'm planning to publish my work very soon and I'll document your package and this expansion method to account for the performance difference I'm getting.

re832003 wrote:How did you convert the dataset to LIBLINEAR format for testing?

I just converted every discrete feature to a dummy number.

I'm not a machine learning person so I'm not sure I get all the math/stat details here. I googled "continuization" and found out that it is one way of kernel equating. Is that correct?

### Re: LIBLINEAR vs. LinearSVM

I don't think so.re832003 wrote:I googled "continuization" and found out that it is one way of kernel equating. Is that correct?

I think there is no standard naming convention for this.

Basically what the LinearSVMLearner does is it replaces a m-valued discrete feature (where m >= 3) with m new continuous features where for each example only one of this new features takes the value of 1.0 while the rest are 0.0, the 1.0 indicating which of the discrete values was present in the original column.

Note that this is exactly as A practical guide to SVM classification ('Data preprocessing' section) recommends to treat discrete (categorical) features (the guide is from the same authors as LIBLINEAR).

11 posts
• Page

**1**of**1**