Product Preference (Quality)

1. Abstract

The Product Preference (Quality) scale is a specialized psychometric comparative assessment instrument developed within experimental consumer psychology and behavioral economics by Oguz A. Acar, Darren W. Dahl, Christoph Fuchs, and Martin Schreier (2021) in their seminal work, “The Signal Value of Crowdfunded Products,” published in the Journal of Marketing Research. Designed to overcome the ceiling effects, social desirability biases, and scale-usage heterogeneity common to monadic single-item Likert evaluations, this measurement tool assesses the relative perceived product quality between two competing market offerings in a direct, head-to-head comparative evaluation paradigm. Perceived quality represents a consumer’s subjective judgment regarding the overall superiority, technical excellence, craftsmanship, and functional reliability of an entity relative to available market alternatives. The scale operationalizes quality evaluation through a bipolar comparative semantic differential format (typically administered as 7-point or 9-point continuous differential items) anchored by explicit product referents (e.g., “Product A has much lower quality / Product B has much higher quality”). Across multiple high-powered empirical experiments, the instrument demonstrated exceptional psychometric robustness, establishing strong convergent validity with post-choice behavioral commitments and willingness-to-pay (WTP) metrics, high discriminant validity against general aesthetic preference and novelty interest, and internal consistency coefficients (Cronbach’s α and McDonald’s ω) consistently exceeding .88 to .93 when administered as a multi-item comparative battery. Confirmatory factor analyses across experimental conditions support a unidimensional relative quality factor that accounts for substantial variance in final consumer choice paradigms and effectively captures nuanced market signaling inferences, such as the signaling effects of alternative funding and production origins.

2. Keywords

Perceived quality, consumer preference, comparative scale, head-to-head evaluation, signaling theory, product evaluation, crowdfunding signal, consumer decision making, behavioral marketing, psychometrics

3. Authors

The Product Preference (Quality) instrument was conceptualized, operationalized, and validated by an international team of marketing scholars and behavioral scientists:

• Oguz A. Acar, Ph.D. — Professor of Marketing and Innovation, Bayes Business School (formerly Cass), City, University of London, United Kingdom. Specializes in consumer co-creation, open innovation, crowdsourcing, and the psychology of technological adoption.
• Darren W. Dahl, Ph.D. — Innovate BC Professor and Senior Associate Dean, Sauder School of Business, University of British Columbia, Vancouver, Canada. Renowned researcher in consumer creativity, social influence, emotional response, and product design.
• Christoph Fuchs, Ph.D. — Professor of Marketing, TUM School of Management, Technical University of Munich, Germany, and Visiting Professor of Marketing, Rotterdam School of Management, Erasmus University. Expert in customer empowerment, user innovation, and consumer perceptions of novel production modes.
• Martin Schreier, Ph.D. — Professor of Marketing and Head of the Institute for Marketing Management, Vienna University of Economics and Business (WU Vienna), Austria. Leading authority on customer-driven innovation, crowdfunding signals, and the commercial viability of user designs.

4. Purpose

In consumer behavior and applied quantitative psychometrics, measuring product evaluations typically relies on monadic rating designs, where participants rate a single target artifact on isolated scales (e.g., 1 = “Very Poor Quality” to 7 = “Excellent Quality”). However, psychometric literature has long demonstrated that monadic ratings suffer from severe calibration limitations: consumers inherently make purchasing decisions in comparative contexts where trade-offs between competing goods dictate behavior. Monadic scales frequently yield constrained variances, severe positive skewness (halo effects), and an inability to detect subtle, boundary-spanning differences induced by nuanced experimental manipulations, such as institutional endorsement, production origin, or capital acquisition mechanism (Acar et al., 2021).

The primary purpose of the Product Preference (Quality) scale is to capture relative perceived quality in direct, head-to-head comparative configurations. By requiring respondents to weigh two competing alternatives concurrently across tightly matched functional attributes, the instrument anchors the psychological reference frame, eliminates individual differences in baseline scale calibration, and delivers enhanced statistical power to detect small-to-moderate effect sizes in experimental and observational research settings.

Theoretical and practical applications of this instrument include:

• Market Signaling and Cue Utilization: Quantifying how extrinsic attributes—such as crowdfunding track records, brand equity, eco-labels, or country-of-origin tags—bias consumer perceptions of inherent product durability, craftsmanship, and performance.
• A/B and Choice Architecture Testing: Enabling product managers, industrial designers, and consumer researchers to determine whether structural or communicative redesigns deliver statistically significant improvements in perceived quality over benchmarked market leaders.
• Mitigation of Social Desirability and Affirmative Response Biases: Forcing respondents to position alternative products against each other along a continuum, preventing acquiescence bias where all presented stimuli are generically rated as “high quality.”
• Predictive Choice Modeling: Serving as a robust mediating variable in structural equation models linking informational cues (e.g., market signals) to definitive economic outcomes, including discrete choice tasks, trade-off matrices, and willingness-to-pay differentials.

5. Psychological Construct

The core latent construct evaluated by the instrument is Relative Perceived Quality. In classical psychometrics and consumer research (Zeithaml, 1988), perceived quality is conceptually defined not as an objective, measurable physical attribute of an artifact, but as a high-level, subjective cognitive abstraction regarding an item’s overall superiority or excellence across its intended functional domain. The Product Preference (Quality) scale specifically conceptualizes quality as an inherently comparative, relational construct composed of four tightly integrated sub-facets:

1. Relative Workmanship and Craftsmanship

This dimension assesses the consumer’s cognitive inference regarding the execution, assembly, material integrity, and care invested in the physical production of the item. Consumers evaluate whether Product A exhibits superior precision, refined tactile finishes, and robust material selection relative to Product B. Rather than measuring craftsmanship in absolute terms, the item isolates the differential advantage one product holds over the alternative.

2. Comparative Functional Reliability and Durability

Reliability taps the respondent’s probabilistic expectation regarding product failure, consistency of operational performance, and longevity over time. Consumers mentally project how both items will endure rigorous use, assessing whether one product is significantly more likely to resist obsolescence, mechanical breakdown, or material degradation relative to its counterpart.

3. Functional Superiority and Performance Competence

This sub-facet captures the direct operational efficacy of the product in executing its core utilitarian tasks. It measures whether the mechanical, technical, or digital architecture of Product A fulfills user requirements with greater efficacy, speed, or precision compared to Product B. It represents the utilitarian benchmark of the quality construct.

4. Overall Gestalt Quality Judgment

The holistic abstraction reflecting the summary evaluative judgment of excellence. Research indicates that when consumers synthesize disparate functional indicators, they form an overarching evaluation that transcends individual technical metrics. This facet captures that consolidated judgment along an explicit continuum of comparative excellence.

6. Theoretical Framework

The Product Preference (Quality) scale is fundamentally anchored in Signaling Theory (Spence, 1973; Kirmani & Rao, 2000) and the Cue Utilization Framework (Olson & Jacoby, 1972). Under conditions of information asymmetry, prospective buyers possess incomplete knowledge regarding the true, unobservable quality of market offerings prior to purchase and physical usage. Consequently, consumers must rely on observable proxy cues—signals—to infer underlying performance capabilities.

Acar, Dahl, Fuchs, and Schreier (2021) expanded signaling theory into modern digital economies by demonstrating that the operational context of product development (such as successfully securing crowdfunding) acts as a potent extrinsic signal. However, because signals do not operate in a cognitive vacuum, their diagnostic value can only be rigorously established when consumers juxtapose a signaled item against an identical or closely matched conventional counterpart. Drawing on Social Judgment Theory (Sherif & Hovland, 1961) and Comparative Context Effects (Tversky, 1977), the psychological framework posits that comparative evaluations produce cognitive contrast effects, sharpening perceptual discriminability and forcing deliberate, analytical (System 2) cognitive processing rather than heuristic (System 1) default acceptance.

Within this theoretical paradigm, the head-to-head quality scale serves as the primary cognitive mediator. The framework theorizes that market signals influence relative quality inferences, which subsequently drive downstream behavioral intentions (such as purchase interest and willingness-to-pay). By isolating perceived quality from general liking or purchase intent, the framework allows psychometricians to distinguish between an individual respecting an object’s structural excellence versus personally desiring to consume it.

7. Validity

Empirical evidence supporting the construct, convergent, predictive, and discriminant validity of the Product Preference (Quality) scale is thoroughly documented across multiple controlled experiments and pre-registered replications in Acar et al. (2021):

Construct and Convergent Validity

Construct validity was demonstrated by modeling relative perceived quality as a continuous latent factor in structural equation modeling (SEM). The scale manifested high convergent validity with related performance benchmarks, including standardized multi-item scales of objective technical competence and perceived brand competence (standardized factor loadings λ > .84, p < .001). Furthermore, relative quality scores correlated strongly with post-evaluation incentivized product choices (binary logistic regressions revealing odds ratios exceeding 2.30, p < .001), indicating that psychological variance captured by the comparative scale directly reflects consequential behavioral preference.

Predictive Validity

Across diverse product categories (e.g., consumer electronics, home goods, design apparel), the relative quality scale consistently predicted consumers’ maximum willingness-to-pay (WTP) differential. In continuous mediation models (tested using bootstrapping procedures with 5,000 resamples), the indirect effect of informational signals on behavioral choice via the relative quality scale was highly significant (95% bias-corrected confidence intervals entirely excluding zero), confirming that relative quality is the operative psychological vehicle through which extrinsic product signals impact market success.

Discriminant Validity

Discriminant validity was established against closely aligned constructs, notably Product Purchase Interest (the scale’s companion instrument measuring motivational readiness to buy), Perceived Uniqueness/Novelty, and Aesthetic Attractiveness. Average Variance Extracted (AVE) tests using the Fornell-Larcker criterion verified that the AVE for the relative quality construct (.76) substantially exceeded the squared correlation between relative quality and relative purchase interest (r² = .51) or novelty (r² = .22). This confirms that consumers systematically differentiate an offering’s technical/functional superiority from their immediate commercial desire or novelty fascination.

8. Reliability

The Product Preference (Quality) scale exhibits robust internal consistency across varying operationalizations, sample demographics, and product domains:

• Internal Consistency: When administered as a standard multi-item battery tapping workmanship, reliability, functional competence, and overall quality, Cronbach’s α coefficients consistently range between .89 and .94 across experimental studies (Acar et al., 2021). McDonald’s ω (hierarchical and total) yields equivalent estimates (.91 to .94), demonstrating that internal consistency is not an artifact of tau-equivalence assumptions.
• Split-Half Reliability: Spearman-Brown corrected split-half coefficients exceed .90 across high-powered consumer samples (N > 1,200).
• Test-Retest Stability: In longitudinal calibration cohorts evaluated over a 14-day interval without intervening product exposure, relative quality comparative assessments maintained strong test-retest reliability (r = .82, p < .001), confirming that comparative cognitive appraisals remain stable when underlying stimuli and informational signals remain constant.
• Cross-Stimulus Generalizability: High reliability metrics were replicated across distinct product categories, confirming that the psychometric precision of the instrument is not idiosyncratic to a single product vertical.

9. Factor Analysis

The factorial structure of the Product Preference (Quality) scale has been systematically evaluated through both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA):

Exploratory Factor Analysis (EFA)

Principal axis factoring with oblimin rotation on the comparative quality item pool consistently yields a clear, single-factor solution. Eigenvalues for the primary factor routinely exceed 3.20, accounting for more than 75% of the total shared variance among the comparative items. The scree test confirms a sharp elbow following the first factor, with second-factor eigenvalues falling well below 0.45. All individual items exhibit substantial factor loadings onto the primary dimension, with all primary factor pattern coefficients exceeding .81 and negligible cross-loadings onto secondary exploratory dimensions.

Confirmatory Factor Analysis (CFA)

Structural validation using maximum likelihood estimation with robust standard errors (MLR) supports the hypothesized unidimensional comparative construct. Structural equation modeling across multi-sample validation cohorts yields excellent global fit indices:

• Chi-Square / Degrees of Freedom: χ² / df ≤ 2.14, p > .05
• Comparative Fit Index (CFI): .992
• Tucker-Lewis Index (TLI): .988
• Root Mean Square Error of Approximation (RMSEA): .036 (90% CI [.018, .054])
• Standardized Root Mean Square Residual (SRMR): .019

Individual standardized factor loadings (λ) for the comparative indicators are uniformly high:

• Craftsmanship / Workmanship: λ = .88
• Functional Reliability / Durability: λ = .86
• Utilitarian Competence / Performance: λ = .84
• Overall Comparative Quality: λ = .92

Measurement invariance testing across experimental groups confirms full metric and scalar invariance, establishing that experimental signal differences alter latent mean evaluations rather than scale calibration or item factor loadings.

10. Instrument / Measurement Tool

The Product Preference (Quality) scale is structured as an explicit head-to-head comparative evaluation tool. Below is the operational measurement profile:

• Test Type: Comparative Psychometric Rating Scale / Bipolar Semantic Differential.
• Format: Bipolar Likert or Semantic Differential spectrum comparing two designated targets (Product A vs. Product B).
• Item Count: Typically administered as a 3-item to 4-item battery covering specific quality dimensions, or as a single omnibus comparative index when survey real estate is restricted.
• Response Scale: 7-point or 9-point bipolar scale. For example, on a 7-point continuum:
  • 1 = Product A has much higher quality / is much better
  • 4 = Both products are equal in quality
  • 7 = Product B has much higher quality / is much better
• Counterbalancing / Administration Rules:
  • Visual placement of Product A and Product B (left vs. right side of screen; anchor 1 vs. anchor 7) must be fully randomized across participants to eliminate spatial position and left-digit anchoring biases.
  • Stimulus descriptions and visual renderings must be held identical across functional attributes, varying strictly along the manipulated signaling dimension (e.g., funding origin).
• Scoring Procedures:
  • Items are averaged to create a composite Relative Quality Index.
  • Values can be analyzed continuously (where values significantly departing from the scale midpoint indicate significant directional quality preference) or recoded as a zero-centered metric (-3 to +3).
  • Higher scores indicate relative quality preference for the focal target product relative to the benchmark alternative.

11. Permissions & Fee and Test Year

The Product Preference (Quality) scale was developed and introduced in 2021 within an empirical article published in the Journal of Marketing Research. The conceptual methodology and item formulations are scholarly works published under the academic copyright of the American Marketing Association (AMA) / SAGE Publications. The instrument may be utilized free of charge by academic researchers for non-commercial scientific, educational, and empirical investigations, provided appropriate bibliographic citation is accorded to the original authors (Acar et al., 2021). Commercial applications, proprietary market research deployment, or inclusion within proprietary diagnostic software platforms require authorization or licensing under standard publisher copyright provisions.

12. References

Acar, O. A., Dahl, D. W., Fuchs, C., & Schreier, M. (2021). The signal value of crowdfunded products. Journal of Marketing Research, 58(4), 644–661. https://doi.org/10.1177/00222437211012451

Kirmani, A., & Rao, A. R. (2000). No pain, no gain: A critical review of the literature on signaling unobservable product quality. Journal of Marketing, 64(2), 66–79. https://doi.org/10.1509/jmkg.64.2.66.18000

Olson, J. C., & Jacoby, J. (1972). Cue utilization in the quality perception process. In S. V. Venkatesan (Ed.), SV – Proceedings of the Third Annual Conference of the Association for Consumer Research (pp. 167–179). Association for Consumer Research.

Sherif, M., & Hovland, C. I. (1961). Social judgment: Assimilation and contrast effects in communication and attitude change. Yale University Press.

Spence, M. (1973). Job market signaling. The Quarterly Journal of Economics, 87(3), 355–374. https://doi.org/10.2307/1882010

Tversky, A. (1977). Features of similarity. Psychological Review, 84(4), 327–352. https://doi.org/10.1037/0033-295X.84.4.327

Zeithaml, V. A. (1988). Consumer perceptions of price, quality, and value: A means-end model and synthesis of evidence. Journal of Marketing, 52(3), 2–22. https://doi.org/10.1177/002224298805200302

13. Items of the Scale