Against Campus Food-Insecurity Surveys

Or: How self-selection bias corrupts social science.

Food insecurity has been put forward as an obstacle for many students in higher education. During the pandemic, the federal government responded to this issue and other hunger-related problems with an emergency subsidy that broadened both the benefits and the eligibility pool of SNAP, the Supplemental Nutrition Assistance Program. Having benefited college students and others for several years, the emergency fund expired after February of this year.

Interviewed by NPR, Brian Montes, a student at Portland State University, related how he had personally benefited from the enhanced SNAP funding. The subsidy, Montes assured listeners, had allowed him to be free from the anxiety of wondering where his next meal would come from. Yet exactly how dire are the needs of students like Montes? Do “many” American college students really feel “terrified” and “ashamed” due to food insecurity?

To answer this question, we look first to Utah. A recent report from a consortium of universities and nonprofits concluded that almost 40 percent of college students in the Beehive State are food insecure. Specifically, almost 18 percent report having “low” food security, while 21.1 percent report having “very low” food security.

Nationally, according to a report from the Center for Community College Student Engagement (CCCSE), 29 percent of community college students are food insecure.

Those negatively affected by food insecurity are more likely to participate in the survey. This skews the data.Both the Utah and the CCCSE reports used questions from the USDA’s U.S. Household Food Security Module to measure food insecurity. According to the USDA’s definition, food insecurity is the “limited or uncertain availability of nutritionally adequate and safe foods, or [the] limited or uncertain ability to acquire acceptable foods in socially acceptable ways.” Insinuated by the two reports, then, is that a significant portion of Utahn and community college students (and, presumably, a similar fraction of their national four-year-college peers) are deprived of food and are subsequently suffering physically, socially, and academically.

However, a closer look at the two surveys, along with the methods in which they were dispersed to participants, calls into question the accuracy of the data.

The designers of the Utah study distributed their survey online through Qualtrics, while CCCSE used both a paper-and-pencil survey and an online one. Unfortunately, such methods of collecting data—namely, via the voluntary participation of self-selected respondents—introduces bias into the process and renders any resulting generalizations about the broader student population largely untrustworthy. This “self-selection bias” occurs when participants voluntarily opt in to a research study, thus creating “severely biased estimates.” To begin with, participants are “more likely” to respond to questionnaires “if they see items which interest them.” In other words, those negatively affected by food insecurity are more likely to participate in the survey. This skews the data by making food insecurity look like more of a problem than it is.

The second strike against the surveys has to do with the USDA questions themselves. For instance, both the inability to afford “balanced meals” and the condition of eating “less than you [feel] you should” are both entirely subjective. Many college students who are by no means food insecure could self-apply those descriptions. [Editor’s note: I was hungry for four straight years. It’s called being a male, college-age human.) Other, similar statements in the rest of the survey are equally applicable across the income spectrum, leading one to question the validity of the USDA’s definitions themselves.

It is worth noting that questioning these data is in no way meant to diminish the hardships of college students who are truly in dire need of food on a consistent basis. On the contrary, it is crucial that we accurately grasp the extent of the problem, so that we may more effectively help those who need aid. By inflating the problem frivolously, however, we are obscuring the truth and delaying effective solutions.

Natalia Mayorga recently graduated with a bachelor’s in psychology from UNC-Chapel Hill and is a Martin Center intern.