Orange Forum • View topic - Test learners and confusion matrix

Test learners and confusion matrix

A place to ask questions about methods in Orange and how they are used and other general support.

Test learners and confusion matrix

Postby x0160768 » Mon May 13, 2013 21:55

Hi there,

I'm hoping someone can help with an orange project I am trying to make. I am trying to understand the tool better and I've been playing with the iris dataset. I've created a test learner project with 3 classifiers and a confusion matrix to view the results. I want to take it one level further though and see what the impact of using different sample sizes of the iris data (and other data later) has on the accuracy of the tool. However I can't seem to use the sampling widget and a confusion matrix. Can anyone help me out? Thanks.

Re: Test learners and confusion matrix

Postby Ales » Tue May 14, 2013 9:31

x0160768 wrote: However I can't seem to use the sampling widget and a confusion matrix.
Just put the "Data sampler' widget between "File" and "Test Learners" and observe how the confusion matrix changes as you change the sample size.

Re: Test learners and confusion matrix

Postby x0160768 » Tue May 14, 2013 9:49

Thanks for the response. I did try that but I wanted to then compare the model predictors in the confusion matrix with the full data set to test its effectiveness. Currently the confusion matrix is testing the model predictors against the sampled data set.

Does that make sense? Do I need to do something different?

Thanks for your help. You are a life-saver :)

Re: Test learners and confusion matrix

Postby x0160768 » Tue May 14, 2013 15:58

Just as a follow up to this, I discovered that you can use sampling as part of the Test Learner widget settings. However, if I use random sampling the Confusion Matrix reports an error stating that:

"Output for results from 'proportion test' is not supported".

Not sure what I am doing wrong here?

Re: Test learners and confusion matrix

Postby x0160768 » Tue May 28, 2013 17:20

Can anyone help with this? I'm really not sure what I am doing wrong. It's driving me nuts!

Re: Test learners and confusion matrix

Postby Ales » Wed May 29, 2013 9:38

x0160768 wrote:but I wanted to then compare the model predictors in the confusion matrix with the full data set to test its effectiveness
You can use the "Separate Test Data" input into the Test Learner widget to pass the full dataset for testing (make sure to select the "Test on test data" sampling option).

Re: Test learners and confusion matrix

Postby x0160768 » Wed May 29, 2013 18:03

Ok. I tried that but now I can't seem to select a 'target class' in the 'test learner' widget :?

Re: Test learners and confusion matrix

Postby x0160768 » Thu May 30, 2013 21:10

Any ideas?

Re: Test learners and confusion matrix

Postby Ales » Fri May 31, 2013 9:18

x0160768 wrote:Any ideas?
No. It works for me.

What exactly happens with the target class selection? Are the class values displayed in the "Target class" box or is the list empty? Or disables (grayed out)?

Re: Test learners and confusion matrix

Postby x0160768 » Fri May 31, 2013 11:29

It is not greyed out as such. It's just that the drop down box is empty.
I'll attach some screenshots so you can see what I mean:

Schema - http://img855.imageshack.us/img855/5687/irisschema.jpg
Signal connector - http://img692.imageshack.us/img692/7089 ... onnect.jpg
Test learner without seperate data signal - http://img825.imageshack.us/img825/9927 ... rnerwi.jpg
Test learner with seperate data signal - http://img855.imageshack.us/img855/9927 ... rnerwi.jpg

Re: Test learners and confusion matrix

Postby Ales » Fri May 31, 2013 12:08

I see what the problem is (you probably misunderstood my post about "Seperate Test Data").

You have to input two signals into the Test Learners.
One is the training data set (in your case this wold be the "Data Sampler - 60% sample"). This will be used to train the classifiers in the Test Learner.

The second input is the test data set (in your case the full table as output by "File - iris data").

Note that the signals must be connected at the same time (first connect the train data set then the test data set). Also remember to select "Test on test data" option in the "Test Learners" widget.

Re: Test learners and confusion matrix

Postby x0160768 » Fri May 31, 2013 12:39

Amazing. Thanks ever so much! You are a life saver :)


Return to Questions & Support