How does the public learn about cancer and cancer prevention?

Challenge

Knowing more about how Americans get their information about cancer, their knowledge, attitudes, and use of cancer-related information, and their cancer prevention behavior helps the National Cancer Institute (NCI) choose the most effective communication strategies, messages, and channels to understand, reach, and motivate various populations. NCI initiated the Health Information National Trends Survey (HINTS) and awarded the HINTS contract to Westat in 2001 to research these important questions. HINTS is focused on monitoring changes in the rapidly evolving fields of health communications and health IT.

Solution

Collaboration is key in this work, and our instrument design specialists work in tandem with NCI to ensure that the questions asked of survey respondents can be both clearly understood and yield reliable, meaningful responses. All new survey questions are tested before being added to a HINTS instrument. We conduct cognitive testing with English-speaking respondents, as well as populations targeted by specific rounds of HINTS (i.e., cancer survivors, caregivers, tobacco users, and rural residents).

Our methodologists create mailing and web materials to encourage survey response; periodically, Westat conducts focus groups to elicit feedback about what messages would inspire a respondent to complete the survey. The focus groups have included English- and Spanish-speaking respondents.

Since 2003, for each round of data collection, Westat’s statisticians have developed a single-stage, stratified national sample of non-institutionalized adults living in the U.S. In 2011, Westat shifted from telephone interviews to collecting data by mail using a standard mailing protocol. Paper surveys were made available in English and Spanish. Starting in 2019, we have offered respondents the option of a web survey.

In early 2024, Westat fielded HINTS 7 with a target of 7,000 completed surveys. Special topics such as discrimination experiences, clinical health care experiences, the use of cannabis, climate change, and belief in false health messages were included in this round. In addition, we included an experiment with a commitment statement at the beginning of the survey with the hopes that it would improve data quality. Half of respondents were randomized to receive this statement that asked them to make a commitment to provide complete and accurate information. 

Maximizing Response Rates

To increase response rates, reduce bias, and obtain the highest quality data, Westat runs methodology experiments. These have focused on incentive levels, number and method of telephone calls, respondent selection methods, order and contents of mailings, cover appearance, grid item formatting, and activities to encourage response from Spanish-speaking respondents. In addition, Westat used a split-sample method (half telephone, half mail) when the survey was transitioned from telephone to mail, allowing analysts to monitor trends across the time series, regardless of interview mode.

In 2019, Westat conducted a similar dual-mode experiment (mail and web) to test whether offering respondents the web option affected response rates or data quality. This dual-mode pilot included an additional incentive for responding via the web and interventions for prompting web respondents with incomplete or low-quality responses. The 2019 pilot was successful, demonstrating that we could get a good response using a dual mode. Also, the web experiment brought in people who were young, low-educated, and healthier so the overall result was data that were more representative of the population.

To increase minority participation, we oversample in high-minority areas and the most recent HINTS data collection included an experiment offering an additional incentive to respondents from the high-minority strata. While respondents from non-minority strata were offered an additional incentive for web completion, the respondents from the high-minority strata were offered an incentive to complete in any mode. Westat statisticians also oversample for respondents in rural areas, as people living in these areas are less likely to have easy access to health care. To facilitate analysis for specific regions of the country, Westat has added codes to the dataset after collection that identify, for example, respondents from Appalachia or the Mississippi Delta in addition to other rural areas.

Delivering a Cleaned, Weighted Dataset

For each round of HINTS, Westat statisticians have created full-sample weights and replicate weights for all survey respondents. The full-sample weights are used to calculate population and subpopulation estimates. Replicate weights are used to compute standard errors for these estimates. The procedures include calculating household-level weights, adjusting for household nonresponse, calculating person-level initial weights, and calibrating the person-level weights to population counts.

Additionally, the Westat team delivers a cleaned and weighted dataset with a codebook/data dictionary that defines all variables in the questionnaire for each round of data collection. These items are accompanied by an annotated instrument and a methodology report describing that particular round.

Including Cancer Survivors

In one round, we worked with NCI’s Surveillance, Epidemiology, and End Results (SEER) program to identify cancer survivors through central cancer registries. This provided a larger sample size of cancer survivors than a typical HINTS survey administration and allowed NCI to better identify the communication needs of this population. SEER variables were added to the HINTS-SEER dataset to provide additional information about the respondent’s experience.

HINTS panel

In early 2022, Westat began recruiting for the HINTS Panel to build a probability-based web survey group of respondents agreeing to be surveyed regularly. The panel is designed to provide NCI with supplemental heath communication data between the regular HINTS data collections and enable the agency to survey the same respondents over time. HINTS Panel recruitment is ongoing.

Results

The data generated by HINTS have deepened NCI’s insight about Americans’ knowledge, prevention behaviors, and attitudes about cancer and the degree to which they understand cancer prevention messages. This insight enables NCI and practitioners to fine-tune and target their messaging to encourage greater awareness of cancer prevention methods.

HINTS data can be found in over 1,000 peer-reviewed articles, and the data have been the focus of a number of presentations at both health conferences such as the Society for Behavioral Health as well as at methodology conferences such as the American Association for Public Opinion Research.