The least squares method (最小二乗法) fixes a line or other formula so that the sum of the squared vertical gaps between each data point and the line (the residuals, 残差) is as small as possible.
Data point
Residual
Squared residual
Point P
+3
9
Point Q
-1
1
Point R
-2
4
Total
0 (the gaps cancel out)
14 (the size of the gaps remains)
A residual is the actual value minus the value the line gives, so it is positive when the point lies above the line and negative when it lies below. Adding residuals as they are lets the positives and negatives cancel out, so each one is squared to make it zero or more before summing. Squaring makes large gaps count far more heavily, which means a point lying far from the rest tends to pull the line toward itself.
AnalogyIt is like choosing one shared bus stop for several houses: the farther a house is, the more heavily its inconvenience counts, and you pick the spot where the total is lowest.
Exam points
Mix-upSimply adding up the residuals (残差) can give 0 as positives and negatives cancel, so it says nothing about how well the line fits. What the least squares method compares is the sum of the squared residuals.
Regression analysis (回帰分析) expresses the relationship between variables as a formula so that the value of one variable can be explained or predicted from the values of one or more explanatory variables.
Collect data
Fit a line by least squares
Regression equation y = a + bx
Estimate y from a new x
Plug an x value you have not yet observed into the formula to estimate y.
4 / 4
The variable used as the clue for prediction is the explanatory variable (説明変数), and the one being predicted is the target variable (目的変数). With a single explanatory variable and a straight-line relationship (y = a + bx) it is simple regression analysis (単回帰分析), and with several explanatory variables it is multiple regression analysis (重回帰分析).
AnalogyIt is like turning your past records of walking steps and calories burned into a quick reference chart, then using tomorrow's planned step count to estimate the calories you will burn.
How to rememberThe name comes from Galton's study showing that even when parents are unusually tall or short, their children's heights tend to "regress" toward the average.
Exam points
ExamExpressing the relationship between two quantities as a straight-line (linear) formula and estimating one from the other is regression analysis.
Mix-upPrincipal component analysis (主成分分析) condenses many variables into a few overall indicators to summarize the information, a different aim from regression analysis, which builds a formula to predict values.
In correlation and causation (相関と因果), correlation means two quantities rise and fall together, while causation means one brings about the other, and a correlation does not by itself prove causation.
A chart that puts two quantities on the horizontal and vertical axes and plots the data as dots is a scatter diagram (散布図); dots rising to the right show a positive correlation (正の相関), and dots falling to the right show a negative correlation (負の相関). The direction and strength are given by the correlation coefficient (相関係数), which ranges from -1 to +1, with values near 1 in absolute terms meaning a strong relationship and values near 0 a weak one. To claim causation, it is not enough that the two move together; you must confirm that changing one actually changes the other.
AnalogyTwo people who ride the same train every morning always show up at the station together, yet neither one is bringing the other.
How to rememberThe Japanese 相関 starts with a character meaning "mutual", while 因果 literally reads "cause and fruit", so the word itself separates the cause from the result.
Exam points
ExamA scatter diagram (散布図) whose dots fall to the right shows a negative correlation (負の相関): as one quantity increases, the other tends to decrease.
ExamThe closer the correlation coefficient (相関係数) is to +1, the stronger the tendency for a large value of one to go with a large value of the other.
Mix-upA negative correlation does not mean a weak relationship. A correlation coefficient of -0.9 indicates a stronger relationship than +0.3.
A spurious correlation (擬似相関) is when two quantities with no direct causal link appear correlated because a third factor (第三の要因) affects both of them.
The third factor is also called a confounding factor (交絡因子), and because the two quantities move together whenever it changes, the data can show a strong correlation. Since changing one of them leaves the other unchanged, a measure chosen on the strength of the correlation alone will have no effect.
AnalogyIn summer, electric fans and shaved ice both sell well, but the shaved ice is not selling the fans; the heat is driving both.
How to rememberThe first character of 擬似相関 means "imitating", the same one used for onomatopoeia (擬音), and the English word spurious likewise means "false".
Exam points
Mix-upWhen you compare only data in which the third factor is at the same level, a spurious correlation disappears or weakens. A relationship with direct causation remains even under that comparison.
Correlation and Causation vs. Spurious Correlation
Aspect
Correlation and Causation
Spurious Correlation
Key differenceWhat the relationship really is
With causation, one directly drives the other
No direct link; a third factor drives both
When comparing data with the third factor at the same level
The causal relationship remains
The correlation disappears or weakens
Key points
The least squares method fixes a formula so that the sum of squared residuals is as small as possible. The line in regression analysis is usually found this way.
Regression analysis builds a formula that explains or predicts the target variable from the explanatory variable.
The correlation coefficient takes values from -1 to +1; its sign gives the direction of the relationship and its absolute value gives the strength.
A correlation alone does not establish causation. When a third factor drives both quantities, a spurious correlation appears even without any direct link.