Learn in the textbookRegression, Correlation, and Spurious Correlation
DefinitionThe least squares method (最小二乗法) fixes a line or other formula so that the sum of the squared vertical gaps between each data point and the line (the residuals, 残差) is as small as possible.
ExplanationA residual is the actual value minus the value the line gives, so it is positive when the point lies above the line and negative when it lies below. Adding residuals as they are lets the positives and negatives cancel out, so each one is squared to make it zero or more before summing. Squaring makes large gaps count far more heavily, which means a point lying far from the rest tends to pull the line toward itself.
AnalogyIt is like choosing one shared bus stop for several houses: the farther a house is, the more heavily its inconvenience counts, and you pick the spot where the total is lowest.