← Back to arXiv
arXivProbabilityarXiv:2611.06987

Validity of the Behrens--Fisher solution

The Behrens-Fisher problem is a classic statistics puzzle: how do you compare the average values of two groups when you do not know, and cannot assume, that the two groups have the same variability? A solution proposed nearly a century ago using a technique called fiducial inference gives a confidence interval for the difference between two group means. Numerical experiments by Robinson in 1976 suggested this interval was "conservative," meaning it actually contains the true value at least as often as advertised, which is a good property. However, nobody had formally proved this was always the case.

The paper provides that missing proof, and extends it beyond just two groups to any weighted combination of means from multiple groups. The core of the argument is a new mathematical inequality the authors call a "plug-in inequality." Roughly speaking, it shows that a certain statistical bound continues to work correctly even after you replace unknown quantities with their observed estimates, as long as the underlying measurement errors satisfy a condition related to how concentrated, or tightly clustered, their distribution is. Normal (bell-curve) distributions, which are the standard assumption in this setting, satisfy this condition.

The practical upshot is that researchers using the Behrens-Fisher interval can now be confident it has a formal guarantee: the stated confidence level is never overstated. If the interval claims 95% coverage, the true coverage is at least 95%, regardless of how different the variability of the groups might be. This closes a long-standing theoretical gap and validates a procedure that has been used in applied statistics for decades.

Read original →