Changeset 9141:6f6034fc6eed in orange


Ignore:
Timestamp:
10/28/11 18:18:19 (2 years ago)
Author:
ales_erjavec <ales.erjavec@…>
Branch:
default
Convert:
cb4ca9f9def699d3293cff45200f1277b4b8a3df
Message:

Fixed calculation of deviance when the sum2 is a small negative number (happens due to a limited precision).

File:
1 edited

Legend:

Unmodified
Added
Removed
  • source/orange/basstat.cpp

    r6784 r9141  
    9494  if (!holdRecomputation && (n>0)) { 
    9595    avg = sum/n; 
    96     dev = sqrt(sum2/n - avg*avg); 
     96    dev = sqrt(std::max(sum2/n - avg*avg, 0.0f)); 
    9797  } 
    9898 
     
    106106{ if (n>0) { 
    107107    avg = sum/n; 
    108     dev = sqrt(sum2/n - avg*avg); 
     108    dev = sqrt(std::max(sum2/n - avg*avg, 0.0f)); 
    109109  } 
    110110  else 
Note: See TracChangeset for help on using the changeset viewer.