Choose the sample design before calculating a p value

Separate individual variation, uncertainty of a mean, matched differences and independent-group comparisons.

Original worked example
Record	Before	After	After - before
R01	10	11	1
R02	20	18	-2
R03	30	33	3
R04	40	42	2

A margin needs its center

A mean interval combines an estimated center with a margin on either side. The sample standard deviation describes individual readings; its division by the square root of the count describes uncertainty of the mean under the stated model.

For an unknown population spread, this course uses a two-sided Student interval with sample spread and n-1 degrees of freedom. Alpha 0.05 gives a 95 percent procedure. CONFIDENCE.T supplies the margin; add and subtract it from the sample mean.

Long-run coverage describes repeated intervals from the assumed sampling process. It does not make 95 percent of individual future readings fall inside this mean interval or assign a posterior probability to a fixed population mean.

=CONFIDENCE.T(D2,STDEV.S(A2:A21),COUNT(A2:A21))

Matched records carry their own baseline

Two measurements of the same identified record form a pair. Calculate after minus before on each row. The variation of those differences controls the standard error of mean change; either source column by itself answers another question.

Sorting one column alone changes the pairs. A before/after example also does not prove that an intervention caused the observed change. The model assumes independent records and approximately normal within-record differences.

=T.TEST(B2:B21,C2:C21,2,1)

Independent groups need separate variance contributions

Two independent groups can have different counts and spreads. Welch inference keeps each sample variance divided by its own count, then combines these contributions in the squared standard error.

The resulting degrees of freedom usually have a fractional part. The two-sided T.TEST type=3 calculation retains this unequal-variance model. A pooled type=2 calculation introduces an additional equal-variance assumption. Manually passing fractional degrees of freedom to a function that truncates them can change the tail probability.

=T.TEST(A2:A21,B2:B21,2,3)

Direction, tail evidence and practical size are different

The signed mean difference and signed t statistic show direction. A two-sided p value concerns outcomes at least as extreme in absolute magnitude under the specified zero-difference model. It is not the probability that the null statement is true.

Comparing p with a prespecified alpha gives a threshold decision within that model. A large p does not establish equality, practical equivalence or adequate sample size. A small p does not quantify a useful effect or establish valid sampling assumptions.

The exercises keep a bounded numeric domain, positive relevant spread and complete typed source values. Unsupported sources and numerically ambiguous threshold comparisons withhold grading. The app does not infer random sampling, independence or population shape from these short examples.

Check the idea

A two-sided comparison has p greater than the prespecified alpha. Which conclusion follows?

Try the idea

References

Examples and explanations are original. Source links describe the related Excel features; native Excel workbook testing is separate from browser learning checks.