Precision and bias (精度と偏り) are separate views of measurement quality: precision is how small the spread of values is, and bias is how far they shift one way from the true value.
Example measurements (true value 100)
Mean
Precision
Bias
99, 100, 101
100
High
Small
90, 100, 110
100
Low
Small
109, 110, 111
110
High
Large
100, 110, 120
110
Low
Large
Precision is how closely repeated measurements of the same thing agree, and it rises as the spread narrows. Bias is how far the center of the measurements sits to one side of the true value, and it also goes by バイアス or systematic error (系統誤差).
AnalogyWhen arrows land in a tight cluster on a target, precision is high. How far that cluster sits from the bullseye is the bias.
How to rememberThe kanji 偏 appears in the Japanese words for picky eating and prejudice and means leaning to one side. The kanji 精 appears in the word for exactness and suggests fine, even detail.
Exam points
Mix-upMeasurements that agree closely are not necessarily correct. If they all shift one way, precision (精度) stays high while bias (偏り) is also large.
Statistical bias (統計的バイアス) is the umbrella term for systematic skew in which the way data is collected or measured pushes results away from the true value in one direction.
The whole you want to know about
Choose whom to study
Obtain the values
Total the results
A shift from either stage carries straight through to the totals.
3 / 3
The whole group you want to know about is the population (母集団), and the part you actually examine is the sample (標本). When a sample is used to infer the population, any shift caused by the survey method is statistical bias, and selection bias (選択バイアス) and information bias (情報バイアス), for example, are kinds of it.
AnalogyA clock running five minutes fast is off in the same direction every time you look. Checking it more often brings you no closer to the right time.
How to rememberThe English word bias originally meant a slant. A ball on a tilted table rolls to the same side however many times you release it.
Exam points
Mix-upRandom scatter cancels out in the average as data grows, but statistical bias (統計的バイアス) stays for as long as the method stays the same.
Selection bias (選択バイアス) is a statistical bias in which a skewed way of choosing the sample leaves it unrepresentative of the population, which distorts the result.
For example, treating a web survey of your own site's members as the trend for the whole country leaves out everyone who is not a member. Random sampling (無作為抽出), which gives everyone in the population an equal chance of being chosen, is the basic way to reduce this skew.
AnalogyIt is like tasting only the top of an unstirred pot and deciding how the whole pot tastes.
How to rememberA famous story tells of studying bullet holes only on military planes that made it back, while overlooking those shot down (survivorship bias, 生存者バイアス).
Exam points
Mix-upIf the cause is how the sample was chosen, it is selection bias (選択バイアス); if the cause is the values obtained from those chosen, it is information bias (情報バイアス).
Information bias (情報バイアス) is a statistical bias in which the values obtained while measuring or answering are themselves off from the true value.
For example, a leading question such as “What do you think of this handy new feature?” draws more favorable answers. An uncalibrated instrument or a respondent's faulty memory can also shift the values.
AnalogyA warped mirror distorts everyone it reflects. Changing who stands in front of it does not help; you will not see a true image until the mirror is fixed.
Exam points
Mix-upInformation bias (情報バイアス) does not come only from respondents lying. Even in machine measurements where nobody answers anything, an instrument's quirks can shift the values.
Selection Bias vs. Information Bias
Aspect
Selection Bias
Information Bias
Key differenceStage where the skew enters
Choosing the sample
Measuring or answering
What shifts
Who is in the sample
The values obtained
How to reduce it
Use random sampling
Word questions neutrally and calibrate instruments
Cognitive bias (認知バイアス) is a skew in which thinking habits such as assumptions and preconceptions bend judgment and interpretation away from what is rational.
Example
What happens
Confirmation bias
Gathering only information that fits your view and playing down the opposite
Normalcy bias
Assuming that even an emergency is nothing serious
Hindsight bias
Feeling, once the outcome is known, that you could have predicted it from the start
It works unconsciously, so people rarely notice that their own judgment is off. When reading analysis results, looking for data that supports the opposite conclusion and checking with several people reduces its effect.
AnalogyThrough tinted glasses, the same scenery looks a different color. The person wearing them gets used to the tint and stops noticing it.
Exam points
Mix-upEven data collected without bias can lead to a warped conclusion through the reader's assumptions. Better data quality alone cannot prevent cognitive bias (認知バイアス).
Statistical Bias vs. Cognitive Bias
Aspect
Statistical Bias
Cognitive Bias
Key differenceCause
How data is collected or measured
Human thinking habits
What gets warped
The collected data and result values
Judgment and interpretation
How to reduce it
Rework the design of collection and measurement
Look for opposing evidence and check with several people
Key points
Assess precision and bias separately, since a measurement can be good on only one of them.
Statistical bias comes from the method, so the fix starts with the survey design.
If the sample does not represent the population, even the most accurate measuring cannot reveal the whole picture.
Leading questions and uncalibrated instruments push the values themselves away from the true value.
Cognitive bias works unconsciously, so deliberately seeking opposing evidence reduces its effect.