The classic decomposition of the question-answering process has four components: 1) comprehension of the item, 2) retrieval of relevant information, 3) use of that information to make required judgments and 4) selection and reporting of an answer (Tourangeau, Rips and Rasinski, 2000). However, many authors state that 'Agree-Disagree' (AD) questions require an extra effort by respondents to answer the question (e.g. Carpenter and Just, 1975; Clark and Clark, 1977; Trabasso, Rollins, and Shaughnessy, 1971). The response process would be composed in the following way: (1) the respondents must read the stem and understand its literal meaning; (2) they must look deeper into the question to discern the underlying dimension of interest to the researcher; (3) having identified this dimension, respondents must then place themselves on the scale of interest; then, (4) they must translate this judgement into the AD response options appropriately, depending upon the valence of the stem. Although Saris et al. (2010) have shown that the distinction between AD and 'Item-Specific' (IS) requests is a very important one, meaning that the quality is affected greatly depending whether an AD or an IS request is used, the current version of SQP does not distinguish explicitly between 'Agree-Disagree' (AD) requests and other requests.Furthermore, Revilla, Saris and Krosnick (2013) have also shown that for AD scales the quality decreases when the number of scale points increases, whereas in general the quality increases with the number of points. For SQP users, this means that:
SQP probably tends to overestimate the quality of questions using an AD request.
Users should not trust the predicted effect of the number of scale points on the quality for AD requests.For instance, if a 5-point AD scale is evaluated and the 'Potential improvements' tool suggests that increasing the number of points to 11 would improve the quality, then users should not do that, users should stick to the 5-points (Revilla, Saris and Krosnick, 2013) or even better, switch to an IS scale (Saris et al., 2010).In addition, whenever users are interested in evaluating such questions, as the one presented below, it is recommend that they code them as 'Complex concepts'.
Request for an answer: Please rate if you agree or disagree with the following statement: a high income is important for well-being.
1. Strongly disagree
3. Neither disagree nor agree
5. Strongly agreeIndeed, for the question above, while the concept 'Importance' is a simple concept, the use of AD as the format of the question increases the complexity of the concept being evaluated, which in this case would be: 'An agreement regarding the importance of income for well being', a complex concept. Users should identify such questions using the option 'Complex concepts â Other' in the 'Concept' characteristic.