Survey Stuff: Validating Surveys in Low- and Middle-Income Countries

Surveys ask us to answer all sorts of questions. Sometimes they ask us about facts such as our age, income, racial or gender identity. Other times they ask us about opinions such as what we think about human rights, democracy, specific issues, etc. And then again surveys might ask us about behaviors — did we do action A or action B, why we did what we did, when we did these things and so on and so forth. But in asking these things, surveys often assume that respondents understand all the terms and words used in the question exactly the way we, the researcher, want them to understand the question. But do they?

To assess if respondents actually understand the question the way they are meant to, survey researchers use a technique called cognitive testing. This type of testing which is used to establish construct validity, is a one-on-one, in-depth interview with the respondent that digs deep into how they understand the question, how they are answering the question, and attempts to identify areas of misunderstandings and inability to respond.

While these interviewing techniques are used often in the United States and Europe, trying to use this technique presents a variety of challenges in the developing world. Key among these challenges is the lack of usage of this technique outside the western world, and hence the lack of interviewers experienced in this type of interviewing technique.

Through two USAID-funded projects in which we conducted cognitive interviews in Asia, Africa and Latin America. Based on this experience, we developed a set of modified best practices for conducting cognitive interviews in low- and middle-income countries (LMIC), where this technique is not prevalent. My colleague, Julie de Jong, and I presented our recommendations for adapting best practices to the LMIC context at the 2025 American Association of Public Opinion Research’s (AAPOR) Annual Conference in St. Louis a couple of weeks ago. The following chart provides a brief overview of adaptations to best practices in cognitive testing.

To see our full presentation, please click here.

Please contact either of us if you have questions or would like to chat more about cognitive testing, especially in LMICs.