Writing survey questions for research studies is a very tricky task that must be done carefully in order to not bias the results. Not only must one be careful in the wording of the questions in order to not bias the survey taker or lead them toward a certain response, even the order of the questions must be taken into account. Researchers have long known about the existence of what is called the “order effect” where the order that questions in a survey are asked actually effects the response. For example, a 1997 Gallup poll asked whether respondents considered Bill Clinton and Al Gore honest and trustworthy. There were two versions of this survey, one with the question about Bill Clinton first and the other with the question about Al Gore first. Among those that took the version with the question about Clinton first, 49% said that both Clinton and Gore were trustworthy and 28% said that neither were trustworthy; the rest said only one of the two were trustworthy. The results were different with the other version of the survey as 56% of those who got asked about Gore first said that both men were trustworthy while 21% said neither were trustworthy.
Researchers normally use standard probability theory to explain human reasoning but this theory is unable to explain the order effect. Thus, researchers have mostly viewed the order effect as a source of noise in the data that they try to control for by varying the order that questions are asked to different survey takers.