## Inconsistent SVM results

**Forum rules**

Archived/read-only, please use our ticketing system for reporting bugs and their discussion.

3 posts
• Page

**1**of**1**### Inconsistent SVM results

There seems to be a bug in orngSVM which can be illustrated with the following example:

Simply: try two SVM-learners with the exact same parameters on the same data, and report the results.

This code outputs (first one without learners.reverse(), second on with learners.reverse()):

This is quite strange, since this tells me that the performance of the SVMs is dependent on the order in which they are evaluated. As far as I understand SVMs there is no stochastic element in the learning/classification process (but please correct me if I am wrong). Note that I am assuming that the folds on which both learners are tested are the same, which is stated in the Orange documentation (http://www.ailab.si/orange/doc/modules/orngTest.htm).

Is this a bug, or am I missing something?

- Code: Select all
`# Load example dataset`

data = orange.ExampleTable("iris.tab")

learners = []

# Create learner:

svm_1 = orngSVM.SVMLearner(kernel_type=0, C = 512, normalization=True, name="SVM-1" )

learners.append(svm_1)

svm_2 = orngSVM.SVMLearner(kernel_type=0, C = 512, normalization=True, name="SVM-2" )

learners.append(svm_2)

# Now, do cross-validation on data:

results = orngTest.crossValidation(learners, data, folds=10)

##Uncomment to reverse order of learners:

#learners.reverse()

#output the results

print "Learner CA IS Brier AUC"

for i in range(len(learners)):

print "%-8s %5.3f %5.3f %5.3f %5.3f" % (learners[i].name, \

orngStat.CA(results)[i], orngStat.IS(results)[i],

orngStat.BrierScore(results)[i], orngStat.AUC(results)[i])

Simply: try two SVM-learners with the exact same parameters on the same data, and report the results.

This code outputs (first one without learners.reverse(), second on with learners.reverse()):

- Code: Select all
`Learner CA IS Brier AUC`

SVM-1 0.960 1.429 0.049 0.997

SVM-2 0.967 1.431 0.047 0.997

Learner CA IS Brier AUC

SVM-2 0.960 1.429 0.049 0.997

SVM-1 0.967 1.431 0.047 0.997

This is quite strange, since this tells me that the performance of the SVMs is dependent on the order in which they are evaluated. As far as I understand SVMs there is no stochastic element in the learning/classification process (but please correct me if I am wrong). Note that I am assuming that the folds on which both learners are tested are the same, which is stated in the Orange documentation (http://www.ailab.si/orange/doc/modules/orngTest.htm).

Is this a bug, or am I missing something?

- Code: Select all
`svm_1 = orngSVM.SVMLearner(kernel_type=0, C = 512, probability=False, normalization=True, name="SVM-1" )`

learners.append(svm_1)

svm_2 = orngSVM.SVMLearner(kernel_type=0, C = 512, probability=False, normalization=True, name="SVM-2" )

learners.append(svm_2)

3 posts
• Page

**1**of**1**