The Effect of the Number of Scale Points in Measuring Product Perceptions
Introduction
Lists of characteristics are often used by researchers to assess consumers' perceptions of a product or concept. For example, respondents may be asked whether or to what extent they agree that the characteristics describe the product or concept, or how much they like various attributes of the product. Prior to data collection, the researcher must decide what type or form of scale should be used. Scales may vary not only in the intent (e.g., performance, satisfaction, description, agreement) and the semantic descriptions of the scale points, but also in the number of scale points. Two experiments were conducted focusing on the latter issue, with particular reference to agreement scales. This paper summarizes the results of these experiments. Specifically, the research issue addressed is whether the conclusions about product or concept differences are affected by the number of points used in an agree/disagree scale.
Experiment 1
An analgesic concept study was conducted among 3,000 respondents via Market Facts' Consumer Mail Panel. Five concepts, assessed monadically, were presented for evaluation using either a 5-point or a 2 point scale, for a total of ten questionnaire versions.
The following descriptions were used for each scale:
- Disagree Strongly
- Disagree Somewhat
- Neither Agree/Disagree vs. 1. Disagree
2. Agree - Agree Somewhat
- Agree Strongly
The top-two box percentages from the 5-point scale (Agree Somewhat or Agree Strongly) and the percent "Agree" from the 2-point scale were compared using bar charts and perceptual maps (biplots). (Refer to Research on Research Number 29 for a description of biplots.) The bar charts in Figure 1A are a representative subset of the 15 attributes on which the five concepts were rated. The similarity in the overall patterns of the bars across concepts for a given attribute is of particular interest. The relative concept differences are very similar for the 5-point and the 2-point scales, although the absolute magnitude of agreement varies.
To further illustrate this point, a biplot is provided in Figure 1B. The map reveals five pairs of brand points. Each pair represents ratings of one concept, based on both the 5-point and the 2-point scale. The small distance between the two points for a concept indicates that the concept was perceived similarly relative to the others on both scales. Therefore, the conclusions concerning concept differences and more importantly the marketing decisions to be made would not change as a function of scale change.
Experiment 2
In this experiment, approximately 400 respondents were asked to describe their perceptions of 8 beer brands using a list of characteristics. Both a 7-point agreement scale and a checklist were included in the study. Top-two box percentages from the 7-point scale and the percent "checked" were again compared via bar charts and a biplot. The bar charts in Figure 2A are a representative subset of the 16 attributes on which each brand was rated. The results presented in Figure 2A are similar to those found in the first experiment. Specifically, for a given attribute, the patterns of response across brands are quite similar for both scale types.
The biplot for this experiment is displayed in Figure 2B. Once again, in interpreting the biplot, the closer brand points are, the more similarly they are perceived. The proximity of the two points for a given brand of beer indicates that regardless of the scale used, the brand's position is quite stable relative to the other brands. When this analysis was replicated using top three in place of top-two box percentages for the 7-point scale, the results obtained were virtually identical to those shown in Figure 2B.
Discussion
The experiments presented above show that the relative positions of concepts may not depend on the scale used. However, the absolute level of agreement may differ. Concerning results from the first experiment, in general, the percentage of agreement from the 2-point scale was larger than the top-two box percentages from the 5-point scale. This higher level of agreement may be attributable to the fact that a random and essentially unknown fraction of respondents who were neutral, indicating they "neither agree nor disagree" (i.e. a "3" from the 5-point scale) would have agreed had they been restricted to a 2-point scale.
Considering results from the second experiment, the "4" on the 7-point scale could be considered the neutral point, just as "3" was taken as neutral on the 5-point scale, As such, the top-three box percentages should be compared to those obtained from the checklist. However, the relationship between the absolute level of responses is not consistent with that found above: the level of agreement from the top-three boxes of the 7-point scale was higher. This suggests a rather complex interaction between respondents and scales. Possibly, response to a checklist yields different reactions than to a 2-point scale. Another possibility is that a "4" may not be a neutral point but rather negative, which coincides with the notion that respondents tend not to use or consider all the scale points. In general then, it is quite difficult to compare absolute percentages across different scales. (This aspect requires further investigation before more stable recommendations can be made.)
The main advantage of using fewer scale points is simplicity; the ease and speed with which a respondent can answer a list of statements is greatly increased as the number of scale points is decreased. This may be quite beneficial in studies where respondents are asked to evaluate large numbers of statements or attributes. Further, the 2-point scale is generally easier for the respondent to understand and respond to, increasing the speed with which telephone surveys can be completed. In general, a 2-point scale is preferred to a checklist due to the inability to differentiate between a response of "no" and a non-response with the use of checklists.
A drawback of the reduced scale is that the restriction in the number of points may compromise the ability to discriminate among respondents. The ability to measure variability is decreased as the number of scale points is restricted (e.g., the finer gradation of a 5-point scale allows more discrimination than a 2-point scale).
The consistency of differences across various scales was also found in an earlier study, reported in Research on Research Number 28, in which respondents rated the importance of various benefits of hand and body lotions. Even though that study focused on ordering attributes in terms of average importance rather than ordering products in terms of agreement percentages, comparable results were obtained: changing the scale did not affect the ordering of attributes.
Summary
Reducing the number of scale points may lead to higher measured levels of agreement, but has little effect on the relative product or concept positions. That is, the same interpretation or conclusion is reached using either of the scales. If the objective is to order the products or concepts, 5-point or 7-point scales appear to offer little advantage over a 2-point scale or checklist. Therefore, the 2-point scale may be sufficient and worthy of consideration for this purpose.
