[Paper 171] The Fence around Least Squares Is Linearity, Not Normality

When told that nothing has been assumed, look for the assumption that was not written down.

Paper 157 wrote that the two in least squares is an assumed error law. This paper stands at the other fence: Gauss answered the same question twice, and the second answer of 1821 assumes no distribution at all. And best is still fenced, by linearity rather than by normality. NO NEW MATHEMATICAL THEOREM IS CLAIMED: the Gauss-Markov theorem, the Cramer-Rao bound, M-estimation and generalised least squares are all standard. Nor is least squares criticised, since what is shown is how far its best extends. On the overlap with Paper 157: that paper showed the exponent to be an assumed error law, and this one shows that the argument assuming no error law is fenced too. Two fences around one theorem, and nothing there is retracted.

First, Gauss is one person with two theorems. The Theoria Motus of 1809 assumes that the arithmetic mean is best and deduces that the error law is normal, an argument that places in its assumptions what one wants as its conclusion. Gauss himself replaced it in the Theoria Combinationis of 1821, where the new argument demands only three things: zero mean, equal variances and no correlation. One person with two theorems is a recurring type in this corpus, as with Yamabe, Hopf, Noether and Nash, and here too the two are not a staircase of strength: the assumptions differ, and so does how each of them breaks.

Second, Gauss-Markov was verified directly. With a sample of eleven, generating two hundred thousand random weight vectors summing to one and comparing the sum of their squares, not one fell below the value 0.0909090909 attained by the uniform weights, the best being 0.0919952051 and the worst 0.2864219851. No distribution was used anywhere in that section: all that was needed was equal variances and no correlation, and the normal distribution appeared nowhere.

Third, and this is the core. Let the errors follow a t distribution with five degrees of freedom, whose variance is finite, so that Gauss-Markov holds in full. With a sample of eleven over two hundred thousand trials, the mean square error is 0.15233893 for the mean, that is least squares, 0.15945434 for the median, 0.12773773 for the trimmed mean and 0.12848074 for the Huber estimator, giving ratios to the mean of 1.046708, 0.838510 and 0.843387. The trimmed mean and the Huber estimator beat the mean, and Gauss-Markov is not violated, since the mean is still best among linear estimators. The fence is linearity rather than normality, and it was in place before any distribution was mentioned.

Fourth, yet nonlinear therefore better is false. In the same setting the median loses, at 1.046708 times. Leaving the linear class is necessary and not sufficient: being nonlinear buys the possibility of winning, not the win. Repeating the comparison with normal errors reverses the outcome, the mean winning with the median at 1.503425 times and the trimmed mean at 1.116649. What normality does is promote the mean from best among linear estimators to best among all unbiased ones, which is the Cramer-Rao bound. There are two bests, and only one of them needs an error law.

Fifth, unbiasedness and efficiency have distinct roots. Making the variances unequal, the i-th equal to i for i from one to eleven, the plain mean has variance 0.5454545455 and the inverse-variance weighting has 0.3311392768, a ratio of 1.647206. Yet the measured biases are 0.000952 and 0.000498, so both are unbiased. Unbiasedness and efficiency are separate properties, and what moves the second is the spread of the variances. The equal-variance clause of Gauss-Markov is not decoration: break it and the conclusion breaks, but unbiasedness does not.

Finally, the two of the square never reaches the answer. Minimising the sum of squared deviations gives minus twice the sum of the deviations equal to zero, and the two divides out. Multiplying the square by one, two, seven or a hundred leaves the estimate at 0.700000000000, agreeing with the plain mean. The two of squares is not inside the estimator; what is inside is the exponent, which Paper 157 treated, and they are two different twos. A census: least variance among linear unbiased estimators is Gauss-Markov, assuming no distribution and fenced by linearity; least among all unbiased is Cramer-Rao, assuming normality and fenced by the error law; maximum likelihood gives one estimator per error law; robust estimation wins on heavy tails by being nonlinear. The sentence that least squares is best can be read four ways, and until it says which, it has not yet become checkable, the same shape as what Paper 164 wrote about probability.

On the making of this work: The ideas and content of this work stem from the author’s own considerations. Assistance from an AI (a large language model) was used for structuring, English translation, and checking the algebra. Any remaining errors or misinterpretations are solely the author’s. Feedback and corrections are sincerely appreciated.

⚠️ Scope ── what is not claimed

a census of established standard facts (Layer 2). What is not shown: any new mathematical theorem, none is claimed; any criticism of least squares, none is offered; any retraction of Paper 157, none is made. What is added: verifying Gauss-Markov directly by random search, showing with the median that nonlinear therefore better is false, and setting out in numbers that unbiasedness and efficiency can hold and fail independently. What remains: there is no argument that assumes nothing. Gauss dropped the assumption of an error law in 1821 and in its place laid down linearity, and simply did not write that he had. When told that nothing has been assumed, what to look for is not the absence of an assumption but the assumption that was not written down.

Reference

Yamagishi, Y. (2026). The Fence around Least Squares Is Linearity, Not Normality [Paper 171]. Zenodo.
DOI: 10.5281/zenodo.22094287
ORCID: 0009-0006-8587-3450