Learn in the textbookThe Framework of Hypothesis Testing
DefinitionThe significance level (有意水準) is the probability that marks the cutoff for rejecting the null hypothesis, and it is set before the test is run.
ExplanationValues such as 5% or 1% are common, and when the probability of the sample result or a more extreme one under a true null hypothesis drops below this value, the null hypothesis is rejected as “hard to put down to chance”. The significance level also fixes how much probability of rejecting a null hypothesis that is in fact true you are willing to accept. The smaller you set it, the stricter the condition for rejection becomes.
AnalogyIt is like a pass line drawn in advance: anything rarer than this will not be treated as chance. Moving the line after seeing the result lets the judgment bend to suit you.
Memory hook有意 means “having a meaning that chance cannot explain away”. A result below the cutoff is called a “significant difference” (有意な差).