Tests of Proportions

Introduction

• Tests of proportion are widely used in medicine and public health research.¹
  • They allow researchers to assess treatment effectiveness, compare risks between populations, and evaluate changes in outcomes over time.
• Many clinically relevant outcomes are binary; therefore, tests of proportion are among the most used statistical tools in medical research.^2,3
• An understanding of tests of proportions enables clinicians and researchers to critically evaluate the literature, interpret findings, and make informed decisions based on data.

Statistical Inference

• Statistical inference is the process of using data from a sample to make conclusions about a larger population.⁴
• A proportion is a population parameter when it refers to the true fraction of individuals in a population with a specific characteristic.³
  • Examples include the proportion of patients who experience a specific anesthetic complication.

Binary Outcomes

• A proportion arises when an outcome can be classified as binary.³
• The sample proportion is calculated as the number of observed successes divided by the total sample size.³
• This sample proportion serves as an estimate of the population proportion, which is typically unknown.⁵
  • Larger samples tend to produce more stable estimates, whereas smaller samples show greater variability.⁶
• This sampling variability underlies the need for statistical testing, which helps determine whether an observed proportion reflects a true population characteristic or is attributable to random error.⁶

What is a Test of Proportion?

• A test of proportion is a hypothesis test used to evaluate claims about population proportions.³
  • Consider a study comparing postoperative nausea and vomiting between patients receiving total intravenous anesthesia and those receiving inhalational anesthesia.
  • The outcome of interest (i.e., whether a patient experiences postoperative nausea and vomiting [PONV]) is binary (yes/no).
  • A test of proportions may be used to compare the proportion of patients who develop PONV between groups and to determine whether the observed difference exceeds what would be expected by chance alone.⁷
• A two-sided test is used when the goal is to determine whether two proportions differ, without assuming that one is higher or lower.^3,7
  • For example, an anesthesiologist may compare the proportion of patients who experience postoperative nausea and vomiting between two anesthetic techniques to assess whether a difference exists.
• A one-sided test is used when the researcher is interested in a difference in a specific direction.^3,7
  • For instance, a study may test whether the proportion of patients who develop postoperative nausea is lower with total intravenous anesthesia than with inhalational anesthesia.

Assumptions and Limitations

• Tests of proportion rely on several assumptions, including independence of observations and adequate sample size.^6,8
• When sample sizes are sufficiently large, the sampling distribution of the sample proportion can be approximated by a normal distribution.³
  • Under the normal approximation, tests of proportions are commonly performed using a Z-test, which compares the observed proportion or the difference in proportions between groups to the value expected under the null hypothesis.⁷
    • When sample sizes are small, the normal approximation may be inappropriate and exact methods such as Fisher’s exact test are preferred.⁹
  • The resulting Z statistic is then used to compute a corresponding P-value, which indicates the probability of obtaining a result at least as extreme as the observed one under the null hypothesis.⁷
• A common pitfall is confusing statistical significance with clinical significance, as statistical significance is highly dependent on sample size.¹⁰
  • In large studies, very small and clinically inconsequential differences may appear statistically significant.¹⁰