AI can “poison” online surveys, states Dartmouth professor
Online survey research — a core method of data collection in many fields — is facing a serious threat from large language models, according to research published in the Proceedings of the National Academy of Sciences. Sean Westwood, an associate professor of government at Dartmouth and director of the Polarization Research Lab, developed an AI tool he calls an “autonomous synthetic respondent” that can answer survey questions and “demonstrated a near-flawless ability to bypass the full range” of state-of-the-art bot-detection methods.
The AI system evaded detection 99.8% of the time, the paper reports.
“We can no longer trust that survey responses are coming from real people,” Westwood said in a news release. “With survey data tainted by bots, AI can poison the entire knowledge ecosystem.”
According to a release, survey researchers “typically rely on attention-check questions, behavioral flags and response-pattern analysis to identify inattentive participants or automated bots. Westwood said those safeguards are now ineffective after his AI agent bypassed all standard detection techniques, including methods outlined in prominent papers — one of them specifically designed to spot AI-generated responses. The agent also defeated “reverse shibboleth” questions meant to reveal nonhuman behavior by posing tasks that are easy for an AI model to complete but nearly impossible for a human.”
