Skip to main content

Texting in Mixed-Mode Surveys

Research Brief
Close up of two people walking side by side as they text

July 2024

This brief highlights NORC’s recent research on text messaging as a contact mode.

Using multiple modes of contact increases participation and representation in surveys over a single contact mode. Text messaging has emerged as a new mode for contacting survey participants, but little is known about how best to combine texting with other contact modes, such as mail or phone, to improve response rates and data quality. For the past two years, NORC methodologists have been researching the effectiveness of text messaging as a contact mode, including experiments in NORC surveys to better understand the impact and benefits of texting. Our work has focused on using text messages to encourage people to respond to a web survey. 

Text messages increased response to the survey.

Text messages can be an effective way to encourage response to a web survey. We examined the impact of both text invites and text reminders and found that both were effective. Text messages were useful for inviting people to participate in the next wave of a longitudinal survey, although the effects diminished quickly until other contacts were sent. Text reminders can be especially helpful, as initial outreach via mail or another mode provides background about the survey and helps build legitimacy and trust. Early text reminders reinforce prior contacts and encourage people to respond more quickly, both in fewer days and sooner in the field period, saving the cost of additional outreach.

“Texting is becoming an important mode in survey outreach and recruitment and will play a key role in our mixed-mode survey contact strategies.”

Senior Vice President, Methodology & Quantitative Social Sciences

“Texting is becoming an important mode in survey outreach and recruitment and will play a key role in our mixed-mode survey contact strategies.”

Switching contact modes can help bring in additional responses.

Many studies over the past few decades have found that mixed-mode contact strategies are successful in part because some people who do not respond when contacted via one mode may be responsive to another contact mode. Our research suggests this is true for text messages as well. In one survey, sending a text reminder improved response over sending another reminder in the same mode as previous contacts. In this study, the text message outperformed sending a second postcard after prior outreach via mail. Sending the texts was also cheaper than printing and mailing costs for the postcard.

People who respond to the text are more likely to complete via smartphone.

Many respondents will complete a web survey on their smartphones, while others will use a tablet, laptop, or desktop computer. The number of people who complete on a smartphone can vary by survey features (e.g., length, complexity, etc.) and respondent characteristics (e.g., age, home interest access, etc.). We found that using text messages to encourage response to web surveys can increase this even further, with more web respondents choosing to complete the survey on their smartphones. The text message can include a direct link to the survey, making it easier for people to take it from their smartphones. Regardless of whether texting is part of your contact strategy, designing web surveys so they can be completed on a smartphone is critical, given how many people choose to take web surveys on their phones.

Next steps for text messaging and survey outreach

Our research suggests that text messages can be an important contact mode in mixed-mode survey designs. Further research should continue to explore the efficacy of text messaging for survey outreach, including using texting with additional survey populations, various sequences of mail, text, email, and other outreach modes, and the text messages’ branding and content. As text messaging for survey outreach continues to grow, it will be important to understand how to best integrate texting with other contact modes and how these design choices impact response in different survey populations and study designs.


Related Publications and Presentations

Christian, L.M., Sun, H., Slowinski, Z., Hansen, C. and McRoy, M. (2024). When to Text? How the Timing of Text Message Contacts in Mixed-Mode Surveys Impacts Response, Journal of Survey Statistics and Methodology, 12(3): 674-696. doi.org/10.1093/jssam/smae014. Currently available free access.

McRoy, M., Christian, L.M., Slowinski, Z. and Hansen, C. (Forthcoming). Testing Whether Text and Email Contacts Improve Response in a Large ABS Mixed-Mode Study. Presentation for the Joint Statistical Meetings annual conference.

McRoy, M., Christian, L.M., English, N. and Alvarez, E. (2024). The Effects of Late Reminder Text Messages on Completion Rate and Respondent Characteristics. Presentation for the American Association for Public Opinion Research annual conference. 

Christian, L.M., Hansen, C., McRoy, M. and Slowinski, Z. (2024). Texting in Mixed-Mode Studies: Results from Recent Research and Experimentation. Presentation for the Joint Program in Survey Methodology and Michigan Program in Survey and Data Science Seminar Series. 

Johnson, K. and Scott, C. (2023). Optimizing Address-Based Sample Outreach: Findings from a Late-Stage Contacting Experiment. Presentation for the Midwest Association for Public Opinion Research annual conference. 

Christian, L.M., Sun, H., Slowinski, Z., Hansen, C., McRoy, M., Sterrett, D., Alvarez, E. and Stripp, S. (2023). The Effects of Sequencing and Time of Text Messages on Completion Rate and Data Quality Measures. Presentation for the American Association for Public Opinion Research annual conference. 

Citation

Christian, L., Slowinski, Z., McRoy, M. and Hansen, C. (2024, July 17). Texting in Mixed-Mode Surveys. NORC at the University of Chicago. Retrieved from https://www.norc.org.


Tags

Research Divisions

Departments, Centers & Programs


Explore NORC Research Science Projects

Analyzing Parent Narratives to Create Parent Gauge™

Helping Head Start build a tool to assess parent, family, and community engagement

Client:

National Head Start Association, Ford Foundation, Rainin Foundation, Region V Head Start Association

America in One Room

A “deliberative polling” experiment to bridge American partisanship

Client:

Stanford University