If you do not feel comfortable with how entropy is applied to measure inequality, you are not alone: Many people have difficulties to accept that inequality is "order" and that equality is "disorder". Using the terms "order" and "disorder" is a popular but not the best way to explain entropy. One man's order is another man's disorder, which leads to the second point: Is equality good or bad? Is order good or bad? Is inequality good or bad? is disorder good or bad?

But the main reason for the confusion is a simple mistake: An entropy measure like Theil's index is not an entropy, it is a redundancy. A redundancy is actual entropy of a system deducted from the possible maximum entropy of that system. Therefore, redundancy yields a high value for inequal distribution, whereas entropy is high for even distribution.

Calling Theil's measure an "entropy" even confused Amartya Sen. From Amartya Sen's "On Economic Inequality" I learned a lot about inequality measures. But entropy seems not do go down too well with him (1973) and his co-author James E. Foster (1997). When describing the "interesting" "Theil entropy" (chapter 2.11), Sen sees a contradiction between entropy being a measure of "disorder" in thermodynamics and entropy being a measure for "equality". If you assume that equality is "order" and thus a antonym for "disorder", then you may believe - Sen even calls it a "fact" - that the Theil coefficient is computed from an "arbitrary formula". However, there is no contradiction: As you know by now, the Theil index is a redundancy, not an entropy. That is the answer to Sen's objection.

Sen and Foster had another complaint. They didn't think, that Theil's index really yields to "intuition". It may help to remember, that the Theil index (I prefer to call it Theil redundancy) is 0% for a 50%:50% distribution (equality) and close to 100% for an equivalent to the (in)famous 80%:20% distribution. And how much does the Gini index yield to intuition? ( )

Very good! ( )