Abstract
The AI Healthcare Preference Scale (Kerstan, Bienefeld, & Grote, 2024) is a psychometric instrument designed to evaluate individuals’ behavioral and evaluative preferences regarding the allocation of clinical tasks between artificial intelligence (AI) systems and human physicians. Developed in response to the accelerated integration of algorithmic diagnostic and therapeutic tools into contemporary medicine, the scale quantifies the degree to which health consumers prefer human-led, collaborative, or autonomous machine-led care. Grounded theoretically in the classic levels of automation (LOA) taxonomies developed by Endsley and Kaber (1999), the instrument comprises six items administered via an electronic questionnaire. Each item is scored on a 5-point bipolar continuum ranging from 1 (only physician) to 5 (only AI), with intermediate anchors denoting variable degrees of joint human-machine agency.
Psychometric evaluation among adult participants demonstrated robust internal consistency, yielding a Cronbach’s alpha of .75. Exploratory factor analysis (EFA) demonstrated an essentially unidimensional structural organization, with all items loading decisively onto a single underlying factor of AI preference. Subsequent confirmatory factor analysis (CFA) incorporated within a broader nomological network confirmed the scale’s distinctiveness, achieving acceptable goodness-of-fit indices (χ² = 1008.46, df = 547, p < .001, χ²/df = 1.84, CFI = 0.91, TLI = 0.90, RMSEA = 0.04, SRMR = 0.06). The AI Healthcare Preference Scale serves as a standardized empirical diagnostic tool for researchers, health policy analysts, and organizational psychologists investigating patient acceptance, algorithm aversion, and the transition toward hybrid clinical decision architectures.
Keywords
Artificial Intelligence, Healthcare Preferences, Levels of Automation, Human-AI Interaction, Algorithm Aversion, Medical Decision-Making, Patient Acceptance, Psychometrics, Clinical Trust, Digital Health
Authors
The AI Healthcare Preference Scale was developed by researchers at the ETH Zurich (Swiss Federal Institute of Technology), based in Zurich, Switzerland, within the Department of Management, Technology, and Economics (D-MTEC), specifically the Chair of Work and Organizational Psychology:
- Sophie Kerstan, M.Sc. — Department of Management, Technology, and Economics, ETH Zurich. ORCID: ORCID Registry. Email: [email protected]
- Nadine Bienefeld, Ph.D. — Department of Management, Technology, and Economics, ETH Zurich. ORCID: ORCID Registry.
- Prof. Dr. Gudela Grote — Professor of Work and Organizational Psychology, Department of Management, Technology, and Economics, ETH Zurich. ORCID: ORCID Registry.
Correspondence Address: Sophie Kerstan, ETH Zurich, Department of Management, Technology, and Economics, Work and Organizational Psychology, Weinbergstrasse 56/58, 8092 Zurich, Switzerland. Email: [email protected].
Purpose
The deployment of machine learning algorithms, computer vision, and large language models across medical specialties has precipitated a paradigm shift in healthcare delivery. Despite demonstrable algorithmic parity or superiority in diagnostic imaging, oncological triage, and risk stratification, implementation frequently falters due to socio-psychological resistance, colloquially termed algorithm aversion. The primary objective of the AI Healthcare Preference Scale is to provide an empirical assessment of user inclinations regarding who should execute critical healthcare tasks: a human physician, an autonomous algorithm, or an integrated hybrid collaborative team.
Historically, research evaluating technological readiness in medicine relied on generic instruments, such as the Technology Acceptance Model (TAM) or general trust scales. However, these tools frequently obscure the nuanced reality of contemporary clinical workflows, where automation is not a binary present-or-absent condition but exists across a dynamic spectrum of shared responsibility. Kerstan, Bienefeld, and Grote (2024) developed this instrument to quantify the comparative agency that end-users are willing to surrender to automated systems across distinct stages of the clinical pathway.
From an applied and translational perspective, the scale offers multiple strategic applications:
- Health Policy and Implementation: Identifying demographic or psychographic cohorts that harbor systemic distrust or, conversely, excessive complacency toward medical automation, thereby enabling tailored communication and governance frameworks.
- Clinical Workflow Design: Helping health system designers understand which operational domains (e.g., automated administrative synthesis versus definitive invasive interventions) patients feel comfortable delegating to autonomous platforms.
- Theoretical Research: Facilitating the testing of complex causal models linking comparative trust associations, health literacy, subjective algorithmic literacy, and perceived risk to ultimate technological utilization.
Psychological Construct
The focal psychological construct captured by the instrument is Artificial Intelligence Healthcare Preference. Rather than capturing a general personality trait, this construct represents a domain-specific, evaluative attitudinal disposition that reflects an individual’s preferred allocation of cognitive, diagnostic, and prescriptive authority between a human medical professional and an automated computational system.
This construct is characterized by three core psychometric facets embedded within its unidimensional operationalization:
- Comparative Cognitive Authority: The degree to which an individual perceives algorithmic reasoning as sufficiently competent, objective, and valid relative to human clinical intuition. Individuals scoring high on this dimension exhibit minimal epistemic skepticism toward data-driven diagnostic pipelines.
- Tolerance of Algorithmic Agency: The psychological readiness to permit an autonomous agent to execute decisions without explicit, real-time human verification. This involves managing the perceived existential and physical vulnerability inherent in medical vulnerability without the reassurance of human interpersonal warmth.
- Collaborative Hybrid Allocation: The structural preference for intermediate levels of automation, wherein the human physician maintains ultimate supervisory control (human-in-the-loop or human-on-the-loop) while leveraging algorithmic precision for analytical, computational, or predictive tasks.
At the low end of the spectrum (scale anchors 1–2), the construct reflects pronounced human physician preference, rooted in interpersonal trust, beliefs in human empathy, and fear of algorithmic inflexibility. At the midpoint (anchor 3), the construct reflects an egalitarian integration of both human and artificial competencies. At the high end (scale anchors 4–5), it reflects technocentric optimization, wherein the respondent prioritizes algorithmic consistency, speed, and evidence-based precision over human practitioner involvement.
Theoretical Framework
The AI Healthcare Preference Scale is conceptually anchored in the classic engineering psychology framework of Levels of Automation (LOA) formulated by Endsley and Kaber (1999), alongside subsequent elaborations by Parasuraman, Sheridan, and Wickens (2000). The LOA paradigm posits that automation cannot be treated as a monolithic, all-or-nothing phenomenon; rather, task execution within complex socio-technical systems is distributed along a continuum ranging from purely manual human execution to complete computer autonomy.
According to Endsley and Kaber (1999), system operations can be broken down into four structural stages: (1) information acquisition, (2) information analysis, (3) decision selection, and (4) action implementation. Within each stage, varying levels of agency can be assigned to either the human or the machine. Kerstan et al. (2024) translated these engineering principles into the healthcare delivery context by establishing a five-point behavioral response architecture:
- Level 1 (Manual Operation): The human physician independently executes all clinical deliberations and interventions without automated aid.
- Level 2 (Physician-Centric Collaboration): The physician maintains dominant decision agency, utilizing automated algorithms strictly for marginal informational support.
- Level 3 (Shared Cooperative Agency): The human physician and AI system function as interdependent collaborative partners, co-generating hypotheses and treatment avenues.
- Level 4 (AI-Centric Oversight): The AI system generates primary diagnoses and treatment selections, with the human physician performing passive or supervisory confirmation.
- Level 5 (Full Algorithmic Autonomy): The AI operates as an autonomous medical agent, completing clinical assessments and enacting medical directives directly.
By mapping scale responses to this structured hierarchy, the scale bridges human-computer interaction (HCI) theory with social-cognitive theories of trust, risk perception, and decision science.
Validity
Empirical validation of the AI Healthcare Preference Scale was executed by Kerstan, Bienefeld, and Grote (2024) across adult samples recruited via the Prolific research platform. The authors assessed multiple aspects of validity:
- Construct Validity: The scale exhibited strong structural coherence with its theoretical foundation. In a comprehensive structural equation modeling (SEM) framework, the scale demonstrated clear convergence with hypothesized psychological drivers. Preference for AI was positively associated with perceived benefits of AI in medicine and comparative trust in automated algorithms, while showing negative associations with perceived risk and comparative trust in human physicians.
- Convergent and Nomological Validity: In the comprehensive structural model (χ² = 1008.46, df = 547, p < .001), AI preference integrated systematically with adjacent socio-cognitive variables. Specifically, comparative trust associations directly predicted perceived benefits and risks, which sequentially predicted overall healthcare preference, establishing clear nomological network validity.
- Discriminant Validity: The latent factor representing AI Healthcare Preference was psychometrically distinguishable from related constructs, including general technology readiness, subjective algorithmic knowledge, and general health-seeking behaviors. All factor indicators exhibited statistically significant, substantial standardized factor loadings on their designated latent variable, confirming distinctiveness.
Reliability
The AI Healthcare Preference Scale demonstrates robust internal consistency. Across its initial validation sample of adult participants in the United States, Kerstan et al. (2024) observed a Cronbach’s alpha of .75. For a brief, six-item instrument measuring an evolving attitudinal construct, a coefficient of .75 indicates sound internal consistency, comfortably exceeding the standard .70 threshold recommended for academic research instruments (Nunnally & Bernstein, 1994).
Item-total correlations across the six items remained uniform, demonstrating that each item reliably taps into the core latent dimension without excessive item redundancy. The modest length of the scale balances measurement precision with minimal participant burden, mitigating survey fatigue in extended online experimental protocols.
Factor Analysis
The structural dimensionality of the scale was systematically explored using both exploratory and confirmatory analytic frameworks:
- Exploratory Factor Analysis (EFA): Initial unconstrained exploratory factor extraction demonstrated that all six items loaded onto a single common factor. Scree plot inspection and eigenvalue analyses supported a unidimensional model, with no secondary factors reaching statistical retention criteria. All items exhibited substantial factor loadings, confirming that preferences across diverse clinical tasks are mediated by an underlying general preference orientation.
- Confirmatory Factor Analysis (CFA): To verify the scale’s structural integrity within a multi-construct system, Kerstan et al. (2024) performed a combined confirmatory factor analysis incorporating four other measurement scales (yielding a five-factor overall model). The evaluation demonstrated acceptable to good model fit across standard indices:
- Chi-Square Metric: χ² = 1008.46, df = 547, p < .001
- Normed Chi-Square: χ²/df = 1.84 (well below the conservative threshold of 2.00 or 3.00)
- Comparative Fit Index (CFI): 0.91
- Tucker-Lewis Index (TLI): 0.90
- Root Mean Square Error of Approximation (RMSEA): 0.04 (90% CI indicating exceptional fit below the .05 benchmark)
- Standardized Root Mean Square Residual (SRMR): 0.06 (well within the acceptable cutoff of < .08)
All six items contributed significantly to their designated latent variable (p < .001), corroborating the structural validity and dimensional parsimony of the measure.
Instrument / Measurement Tool
The technical parameters and administration profile of the scale are structured as follows:
- Instrument Name: AI Healthcare Preference Scale
- Instrument Type: Self-report rating scale / behavioral preference questionnaire
- Test Format: Web-based electronic questionnaire
- Number of Items: 6 items
- Response Continuum: 5-point bipolar Likert-type scale
- Scale Anchors:
- 1 = Only physician
- 2 = Physician with AI assistance
- 3 = Physician and AI equally
- 4 = AI with physician supervision
- 5 = Only AI
- Target Population: General adult population (≥ 18 years of age)
- Administration Duration: Approximately 2 to 3 minutes
- Scoring Protocol: Individual item responses are scored numerically from 1 to 5. An overall preference index is derived by computing the mean or composite sum across all six items. Higher aggregate scores indicate a pronounced orientation toward algorithmic automation and autonomous AI in medical tasks, whereas lower scores reflect strong preferences for human-centric clinical care.
Permissions & Fee and Test Year
The AI Healthcare Preference Scale was published in 2024. The instrument was developed under the auspices of academic research conducted at ETH Zurich and published in the peer-reviewed journal Risk Analysis (Society for Risk Analysis / John Wiley & Sons, Inc.).
Researchers intending to use the scale for non-commercial academic research, pedagogical purposes, or public policy analysis should refer to the original publication (https://doi.org/10.1111/risa.14216) or contact the corresponding author, Sophie Kerstan ([email protected]), for formal permission and to verify any applicable institutional repository conditions. Commercial applications, clinical software integrations, or proprietary deployments require explicit licensing permission from the copyright holders.
References
- Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
- Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155–1170. https://doi.org/10.1287/mnsc.2016.2643
- Endsley, M. R., & Kaber, D. B. (1999). Level of automation effects on human performance and workload in dynamic control tasks. Ergonomics, 42(3), 462–492. https://doi.org/10.1080/001401399185595
- Kerstan, S., Bienefeld, N., & Grote, G. (2024). Choosing human over AI doctors? How comparative trust associations and knowledge relate to risk and benefit perceptions of AI in healthcare. Risk Analysis, 44(4), 939–957. https://doi.org/10.1111/risa.14216
- Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
- Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354