In a letter to Bessel dated Feb 28 1839, Gauss explains why he decided to drop the metaphysics of maximum likelihood in favour of the method of least squares...

and apologizes for using square loss function, which is used only for its obvious conveniences. He says he would take other choices where appropriate....