Sampling the Hard-to-Reach: Examining the Efficacy of Response-Rate Adjusted Stratified Sampling Selection

Across projects, the Center for Survey Research team has made a concerted effort to reach people who are usually underrepresented in surveys, like certain ethnic or racial groups, younger people, or those with lower incomes. This is important to ensure our survey data, and the resulting conclusions, are representative and reflective of the full population of interest, not just those most likely to respond. 

While decisions at every stage of the research process that impact response, our team examined the impact of varying address-based sample designs to improve representation. Specifically, we applied a sampling design we call Response-Rate Adjusted Stratified Sampling Selection (RRASSS light), which stratifies the sample based on household response propensity using the U.S. Census Bureau American Community Survey Data Self-Response Rate (SRR) score. Then, we mailed more surveys to the lower expected response areas. Using data from the Reston Community Survey, we compared this sampling method with two other designs—proportional sampling (i.e., not over- or under-sampling any specific groups), and a more robust version of the RRASSS technique. 

In general, SRR scores were highly effective at predicting response rate ratios. Of the three sampling techniques, we found RRASSS light resulted in the highest effective sample size after weighting. While robust RRASSS method led to better representation from younger people under 40, Asian respondents, and those of multiple or other races. The increase was minimal compared to two other sampling methods and may increase costs, however, most efforts to reach hard-to-reach respondents would have associated costs. Survey researchers may consider these results when conducting future surveys to increase representation in the sample. 

We presented this work at the American Association for Public Opinion Research (AAPOR) Annual Conference in St. Louis, MO in May 2025. For the full presentation, download the file below.