Orange Forum • View topic - (patch) new metric: AUC single

(patch) new metric: AUC single

Discussions about new ideas and features you would like to see in Orange.
(Archived/read-only, please use our ticketing system for your wishes and their discussion.)
Forum rules
(Archived/read-only, please use our ticketing system for your wishes and their discussion.)

(patch) new metric: AUC single

Postby bshanks » Tue May 11, 2010 5:23

Here's a patch to add a new metric that uses "AUC_single" (one against all), in addition to the current one called AUC that is always AUC_multi on multiclass:


--- OWTestLearners.py~ 2010-04-15 20:18:53.000000000 -0700
+++ OWTestLearners.py 2010-05-10 20:49:06.000000000 -0700
@@ -48,7 +48,8 @@
('Classification accuracy', 'CA', 'CA(res)', True),
('Sensitivity', 'Sens', 'sens(cm)', True, True),
('Specificity', 'Spec', 'spec(cm)', True, True),
- ('Area under ROC curve', 'AUC', 'AUC(res)', True),
+ ('Area under ROC curve (multi)', 'AUC', 'AUC(res)', True),
+ ('Area under ROC curve (single)', 'AUCs', 'AUC_single(res, targetClass)', True),
('Information score', 'IS', 'IS(res)', False),
('F-measure', 'F1', 'F1(cm)', False, True),
('Precision', 'Prec', 'precision(cm)', False, True),
@@ -275,6 +276,7 @@

# computation of results (res, and cm if classification)
pb = None
+ targetClass = self.targetClass
if self.resampling==0:
pb = OWGUI.ProgressBar(self, iterations=self.nFolds)
res = orngTest.crossValidation(learners, self.data, folds=self.nFolds,
@@ -496,6 +498,10 @@

def changedTarget(self):
self.recomputeCM()
+ if self.data and self.learners:
+ self.score([l.id for l in self.learners.values()])
+ self.paintscores()
+

def scoresForTargetClass(self, targetClass):
self.targetClass = targetClass

Return to Wish List



cron