Orange Forum • View topic - Entropy of attribute

Entropy of attribute

A place to ask questions about methods in Orange and how they are used and other general support.

Entropy of attribute

Postby aonlazio » Sat Apr 11, 2009 21:36

HI, orangers.

Is there any method to measure the entropy of an attribute?
Thanks in advance


Postby GuyHoozdis » Wed Apr 15, 2009 17:52

Sorry, nobody has replied to your question. Unfortunately I don't think there is a simple answer to your question. You can look here for one application of entropy that Orange uses:

In other places you will find that phrase "information gain" is used instead of the term "entropy", but the calculations are similar. I think strictly speaking "Information gain" means that you use a log-base of 2 in your entropy calculation. Otherwise, you must know the number-of-states possible for the data you are calculating the entropy over.

If you are looking for a code example there is a python project unrelated to Orange or Data Mining called "pefile". You can find one implementation of entropy there and adapt it to your specific needs.

Return to Questions & Support