Does a p-value of 0.05 mean there is a 5% chance the null hypothesis is true?

No. The p-value tells you the probability of getting your data assuming the null hypothesis is definitely true. It makes no claims about the probability of the hypothesis itself.

What happens if the p-value is high?

If the p-value is high (greater than your $\alpha$ level), it means your data is perfectly normal under the null hypothesis. You 'fail to reject' the null hypothesis because you don't have enough evidence to prove otherwise.

Can a p-value be exactly 0?

No. A p-value can be extremely small (like 0.000001), meaning the result is incredibly rare under the null hypothesis, but it never reaches absolute zero because there is always a tiny theoretical chance of any random outcome.

What is a p-value, in plain terms?

A p-value is a probability. It measures how likely it is to see the results you just got (or something even more extreme) if there is actually no real effect or difference in the population you are studying. This baseline assumption of 'no effect' is called the null hypothesis. If your p-value is very small, it tells you that your data would be highly unusual if the null hypothesis were true. This gives you statistical evidence to reject the idea that your results are just due to random chance, suggesting that something real is actually happening.

The Courtroom Analogy

Think of a criminal trial. The defendant is assumed innocent until proven guilty. In statistics, the null hypothesis is the 'innocent' assumption: it states that nothing special is going on, and any patterns in your data are just random luck. The p-value is the evidence presented by the prosecutor. If the evidence (the data) is extremely unlikely to occur if the defendant were truly innocent, the jury rejects the assumption of innocence. A low p-value means the data looks too suspicious to be just random chance.

The Formal Definition

In strict statistical terms, a p-value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct. It is calculated using the sampling distribution of a chosen test statistic. You compare the p-value to a threshold called the alpha level (often set at $\alpha = 0.05$ ). If the p-value is less than or equal to $\alpha$ , you reject the null hypothesis. If it is greater, you fail to reject the null hypothesis.

Where Students Slip Up

The most common mistake is thinking the p-value is 'the probability that the null hypothesis is true.' It is not. The p-value assumes the null hypothesis is 100% true from the very beginning. It only measures the probability of your data, given that assumption. It evaluates the strangeness of your sample, not the ultimate truth of the hypothesis.

Worked through

You want to test if a coin is fair. Your null hypothesis is that the coin is perfectly fair (50% chance of heads). You flip the coin 10 times and get 9 heads. A statistical test gives you a p-value of 0.021. What does this mean, and what should you conclude at an $\alpha = 0.05$ level?

First, interpret the p-value: If the coin were truly fair, there is only a 2.1% chance (0.021) of getting 9 or more heads (or 9 or more tails) in 10 flips. Next, compare it to your threshold. Since $0.021 < 0.05$ , the result is statistically significant. Because seeing 9 heads is so unlikely for a fair coin, you reject the null hypothesis. You have strong evidence to conclude that the coin is not fair.

Questions students ask

Ask about this topic

Explain at

Where this comes from: OpenStax Introductory Statistics, Chapter 9: Hypothesis Testing with One Sample · Khan Academy: Significance Tests (Hypothesis Testing)