Orange Forum • View topic - How to use custom kernel functions?

How to use custom kernel functions?

A place to ask questions about methods in Orange and how they are used and other general support.

How to use custom kernel functions?

Postby binford3000 » Wed Jul 21, 2010 17:49

Hi,

Id like to try out "fractional norms" [1]. So I coded a distance function:
Code: Select all
    def fracDist(self, e1, e2):
        sum = 0
        for attr1, attr2 in zip(e1,e2):
            if e1.varType == 3 or e2.varType == 3:
                continue
            else:
                sum = sum + pow((attr1.value - attr2.value),0.1)

        return math.pow(sum,10)


And used the custom SVM:

Code: Select all
svm = orange.SVMLearner()
kernelWrap = orngSVM.RBFKernelWrapper(self.fracDist)
kernelWrap.gamma = gamma
svm.svm_type = orange.SVMLearner.Custom
svm.kernelFunc = kernelWrap
svm.nu = nu
classifier = svm(self.histograms)

(...)
       
c = classifier(inputHistograms[i])


However, when I classify an example i get:
Code: Select all
21.07.10 18:36:22 - Classified as: #RNGE


I have no idea if the way I try to use my custom kernel is correct? Any hints what Im doing wrong? If I use a standard SVM (like the NU_SVM) without my custom distance function it works.

[1]http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.23.7409&rep=rep1&type=pdf

Postby Ales » Thu Jul 22, 2010 10:09

replace
Code: Select all
svm.svm_type = orange.SVMLearner.Custom
with
Code: Select all
svm.kernel_type = orange.SVMLearner.Custom

Postby binford3000 » Fri Jul 23, 2010 12:27

Thanks, that worked. However I get very bad classification results, so Im still doing sth wrong. Probably my assumptions about how the nu-SVM works in orange are wrong. I assum that orange uses...

    * a RBF-Kernel
    * the euclidean norm inside the kernel

If I set the RBF-Kernel and the euclidean distance explicitly in the code, I get very bad classification results:
Code: Select all
        svm = orange.SVMLearner()
        kernelWrap = orngSVM.RBFKernelWrapper(orange.ExamplesDistanceConstructor_Euclidean(inputHistograms), gamma=0.5)
        svm.svm_type = orange.SVMLearner.Nu_SVC
        svm.kernel_type = orange.SVMLearner.Custom
        svm.kernelFunc = kernelWrap
        svm.nu = nu
        svm.gamma = gamma


If I use the defaults like this...
Code: Select all
        svm = orange.SVMLearner()
        svm.svm_type = orange.SVMLearner.Nu_SVC
        svm.nu = nu
        svm.gamma = gamma


the classification rate is very good. However, I expected that both code snippets give the same classification results. Where's my error in reasoning?

Postby Ales » Fri Jul 23, 2010 16:08

1. by default orange.ExamplesDistanceConstructor_Euclidean builds a 'normalized' distance measure (pass normalize=False to override this)
Code: Select all
orange.ExamplesDistanceConstructor_Euclidean(inputHistograms, normalize=False)


2. it also handlers missing values differently

3. orngSVM.RBFKernelWrapper used gamma differently then LibSVM, (libSVM uses exp(-gamma*(dot(e1, e2)**2) notation while RBF wrapper used exp(-dot(...)**2/gamma) ). I have changed the wrapper to use LibSVM's notation. Update orange from SVN or wait for tomorrows snapshot (and pass gamma=1/gamma to RBFKernelWrapper constructor in the mean time)


Return to Questions & Support