Don't Compare Averages

We point out that for two sets of measurements, the sign of the difference of their averages is not necessarily maintained by a non-linear monotone transformation of the individual measurements. We show that the inclusion of error bars is no safeguard against this phenomenon. We give a theorem, however, that limits the amount of “reversal” that can occur; as a by-product we get two non-standard one-sided tail estimates for arbitrary random variables which may be of independent interest. Our findings suggest that in the, as we point out, not infrequent situation where more than one cost measure makes sense, there is no alternative other than to explicitly compare averages for each of them, much unlike what is common practice.

• Type: Conference
• Year: 2005
• Where: WEA
• Authors: Holger Bast, Ingmar Weber
• Keywords: Algorithms, Arbitrary Random Variables, Non-linear Monotone Transformation, Non-standard One-sided Tail, WEA 2005