AI Risks and Benefits Scale

1. Abstract

The AI Risks and Benefits Scale (AIRBS) is a psychometric instrument engineered to evaluate public and clinical stakeholders’ cognitive evaluations regarding the deployment of artificial intelligence in healthcare. Developed by Sophie Kerstan, Nadine Bienefeld, and Gudela Grote (2024), the AIRBS addresses a critical theoretical and empirical gap in medical decision-making: understanding how risk-benefit trade-offs shape human preference for human medical practitioners over algorithmic diagnostics and clinical decision-support systems. The scale comprises 19 items systematically divided into two core latent dimensions: Perceived Risks (10 items) and Perceived Benefits (9 items). Administered via a 7-point Likert response format ranging from 1 (very unlikely) to 7 (very likely), the instrument evaluates respondent appraisals concerning epistemic accuracy, data governance, systemic biases, ethical transgressions, algorithmic overreliance, dehumanization of care, diagnostic efficacy, personalized treatment optimization, and system-wide accessibility.

Psychometric evaluations demonstrate high construct validity, robust factorial integrity, and exceptional internal consistency reliability, with Cronbach’s alpha coefficients exceeding α = .88 across both subscales. Confirmatory factor analyses substantiate a robust two-factor correlated structure that reflects theoretical expectations derived from Slovic’s psychometric paradigm of risk perception and the Technology Acceptance Model (TAM). Predictive validity analyses confirm that differential risk-benefit perceptions measured by the AIRBS explain substantial variance in patients’ trust asymmetries, treatment adherence intentions, and pervasive resistance to fully autonomous clinical interventions (algorithm aversion). The AIRBS offers clinicians, behavioral health researchers, and healthcare technologists an empirical tool for evaluating patient sentiment, public readiness, and targeted interventions designed to foster calibrated trust in automated medical technologies.

2. Keywords

AI Risks and Benefits Scale, AIRBS, healthcare artificial intelligence, psychometrics, risk perception, benefit perception, algorithmic trust, algorithm aversion, medical decision-making, patient attitudes, technology acceptance model, human-AI interaction.

3. Authors

The AI Risks and Benefits Scale (AIRBS) was formulated, operationalized, and psychometrically validated by an interdisciplinary research team from the Department of Management, Technology, and Economics at ETH Zurich (Zurich, Switzerland):

• Sophie Kerstan, M.Sc. — Doctoral Researcher, Chair of Work and Organizational Psychology, Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland. Her empirical research focuses on trust calibration in automated systems, human-robot interaction in safety-critical domains, and psychological determinants of artificial intelligence adoption in clinical workflows.
• Nadine Bienefeld, Ph.D. — Senior Researcher and Lecturer, Chair of Work and Organizational Psychology, Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland. Dr. Bienefeld specializes in human-autonomy teaming, high-reliability organizations, clinical leadership, and cognitive trust dynamics across high-risk socio-technical systems.
• Gudela Grote, Ph.D. — Full Professor of Work and Organizational Psychology, Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland. Professor Grote is an internationally recognized authority on organizational uncertainty management, safety culture, human-technology coordination, and rule management in automated environments.

4. Purpose

The integration of autonomous systems, machine learning algorithms, and deep neural networks into clinical environments promises transformative improvements in diagnostic velocity, epidemiological modeling, and therapeutic personalization. Concurrently, it presents systemic challenges regarding patient vulnerability, diagnostic opacity (black-box medicine), digital surveillance, and the potential erosion of empathetic patient-physician alliances. The primary purpose of the AI Risks and Benefits Scale (AIRBS) is to provide an empirically validated, multi-item psychometric instrument capable of capturing the nuanced cognitive trade-offs that individuals make when evaluating the integration of artificial intelligence into clinical medicine.

Historically, assessments of technology adoption in medical settings have relied on generic technology acceptance instruments (such as the classical Technology Acceptance Model or the Unified Theory of Acceptance and Use of Technology) that predominantly emphasize operational utility and perceived ease of use. These generic frameworks often overlook the unique existential, relational, and physical vulnerabilities inherent to medical decision-making. When individuals evaluate healthcare technologies, their assessments are governed not merely by procedural efficiency, but by deep-seated concerns regarding mortality, bodily integrity, personal data confidentiality, and empathetic attunement. The AIRBS was explicitly engineered to address these health-specific psychometric dimensions.

In clinical and translational research, the AIRBS serves multiple operational purposes:

• Quantifying Public and Patient Hesitancy: The scale enables health systems to benchmark baseline risk-benefit calculus across diverse socio-demographic cohorts, identifying specific cognitive barriers—such as fears of algorithmic dehumanization or data exploitation—that inhibit patient participation in AI-augmented health protocols.
• Disentangling Trust Asymmetries: Empirical literature confirms a persistent phenomenon known as algorithm aversion, wherein humans systematically prefer fallible human practitioners over objectively superior algorithmic counterparts for high-stakes medical decisions. The AIRBS provides the granular sub-dimensional data required to examine whether this reluctance stems from an underestimation of technological benefits or an amplification of perceived systemic risks.
• Evaluating Pre- and Post-Implementation Interventions: Clinicians, institutional review boards, and technology implementation teams can deploy the AIRBS longitudinally to evaluate how transparent algorithmic explanations (explainable AI; XAI), institutional data governance policies, or patient education seminars alter the perceived balance of risks versus clinical utility.
• Organizational Workforce Preparedness: Beyond patient populations, the scale is adaptable for surveying physicians, nursing personnel, and healthcare administrators to anticipate professional friction, perceived professional displacement, and systemic risks before deploying automated decision-support infrastructures.

5. Psychological Construct

The AI Risks and Benefits Scale operationalizes cognitive technology appraisal as a bidirectional, two-dimensional latent construct comprising Perceived Risks and Perceived Benefits. Rather than treating attitudes toward medical AI as a unidimensional continuum spanning from extreme resistance to uncritical enthusiasm, the AIRBS conceptualizes risks and benefits as quasi-independent evaluative dimensions that operate concurrently within cognitive decision-making architectures.

Perceived Risks Subscale (10 Items)

The risk dimension captures the perceived likelihood of negative outcomes, systemic failures, ethical regressions, and relational deficits stemming from algorithmic clinical delegation. Psychometrically, this dimension evaluates three distinct conceptual layers of risk:

1. Technical and Epistemic Vulnerability: Evaluated through items targeting software bugs, operational equipment breakdowns, algorithmic inaccuracy, and the dissemination of unreliable clinical intelligence (e.g., “introduce new bugs and equipment failures”, “provide unreliable information”, and “measure patient parameters inaccurately”). This reflects the respondent’s assessment of algorithmic fragility and diagnostic failure.
2. Relational and Dehumanizing Threats: Captures the socio-emotional alienation experienced when mechanized systems replace interpersonal care relationships. Items explicitly address the fear that artificial intelligence will eliminate individualized attention and degrade empathetic bedside manner (e.g., “fail to recognize the uniqueness of each patient’s condition” and “dehumanize care”). This taps directly into psychological resistance toward being treated as a generic data point rather than a holistic biological entity.
3. Systemic and Ethical-Societal Disruptions: Measures structural anxieties regarding the medical ecosystem, including data surveillance breaches, ambiguous ethical responsibility, systemic technological dependence, and human labor displacement (e.g., “increase patient data security breaches”, “lead to ethical problems”, “result in overreliance on technology”, and “result in a loss of jobs for healthcare professionals”).

Perceived Benefits Subscale (9 Items)

The benefit dimension operationalizes the anticipated therapeutic advancements, diagnostic precision gains, structural efficiencies, and democratization of medical care brought about by automated healthcare solutions. This construct spans two primary analytical domains:

1. Diagnostic Acumen and Therapeutic Optimization: Evaluates the capacity of AI systems to outperform or complement human cognitive limitations. It measures beliefs that algorithmic processing can minimize iatrogenic harm, enhance predictive accuracy for adverse events, and refine diagnostic sensitivity (e.g., “reduce medical errors”, “facilitate the prediction of negative health events”, “help in diagnosing health problems”, and “help in choosing adequate treatments”).
2. Care Personalization and Systemic Democratization: Measures expectations regarding hyper-personalized treatment regimens based on multi-omic or continuous biometric inputs, enriched patient-provider communication, and expanded clinical access (e.g., “enable a more personalized care”, “improve information sharing between patients and healthcare providers”, and “improve the accessibility of care”).

6. Theoretical Framework

The theoretical architecture of the AIRBS integrates principles from behavioral decision theory, social psychology, and human factors engineering. Specifically, it synthesizes three seminal conceptual frameworks: Slovic’s Psychometric Paradigm of Risk Perception, the Affect Heuristic, and Models of Human-Automation Trust.

Slovic’s Psychometric Paradigm of Risk

Originating from the work of Paul Slovic, Baruch Fischhoff, and Sarah Lichtenstein (1982), the psychometric paradigm posits that risk is inherently subjective and psychologically constructed through qualitative cognitive dimensions, rather than solely dictated by mathematical probabilities of mortality or morbidity. Two primary cognitive dimensions govern risk assessment across technologies: dread risk (perceptions of catastrophic potential, lack of control, and fatal consequences) and unknown risk (unobservable, novel, uncalibrated hazards with delayed manifestations). Within the AIRBS, healthcare artificial intelligence occupies a unique nexus: it combines the characteristics of an “unknown risk” (due to algorithmic opacity and unfamiliar deep-learning architectures) with “dread” implications (potential medical misdiagnoses and loss of bodily agency). The AIRBS items systematically capture these latent cognitive fears.

The Affect Heuristic and Inverse Risk-Benefit Correlation

In objective reality, risks and benefits across complex technical domains may correlate positively (e.g., high-risk medical procedures often confer the highest therapeutic yield). However, behavioral research by Alhakami and Slovic (1994) as well as Finucane et al. (2000) demonstrates that in human subjective cognition, risk and benefit perceptions are typically inversely correlated due to the affect heuristic. When an individual harbors a positive global affective evaluation of a technology, they cognitively amplify its perceived benefits while minimizing its potential hazards. Conversely, negative baseline affect evokes inflated risk perceptions and discounted utility evaluations. The AIRBS provides an explicit psychometric means to test the magnitude of this inverse coupling in healthcare technology contexts, allowing researchers to evaluate whether cognitive interventions can decouple perceived risks from perceived benefits through objective domain education.

Trust in Automation and Algorithm Aversion

The scale is further grounded in theories of human-machine trust articulated by Lee and See (2004) and Parasuraman and Riley (1997), alongside the behavioral construct of algorithm aversion (Dietvorst, Simmons, & Massey, 2015). Lee and See conceptualized trust in automation across three performance axes: ability (competence and technical reliability), integrity (fidelity to stated principles and transparency), and purpose (benevolent intent aligned with human user welfare). Medical artificial intelligence challenges conventional trust paradigms because medical practice has historically rested on interpersonal, fiduciary duties of human care. The AIRBS operationalizes these trust prerequisites by asking respondents to evaluate whether the technology possesses technical competence (e.g., accurate measurements, reduction of errors) while probing vulnerabilities concerning integrity and purpose (e.g., ethical transgressions, algorithmic overreliance, and dehumanized care).

7. Validity

The psychometric validity of the AI Risks and Benefits Scale has been thoroughly evaluated across multiple empirical investigations, most notably in the foundational validation by Kerstan, Bienefeld, and Grote (2024), which analyzed diverse samples evaluating human versus algorithmic medical consultations.

Construct Validity

Construct validity is substantiated by the scale’s theoretical alignment with established structural models of technology evaluation. Factor analyses verify that items load cleanly onto their designated theoretical dimensions without disruptive cross-loadings. The distinct separation between the 10 risk items and the 9 benefit items confirms that respondents cognitively distinguish technical hazards from functional utilities, even while maintaining an underlying affective alignment.

Convergent Validity

Convergent validity has been established through robust, statistically significant correlations between the AIRBS subscales and established related psychological constructs:

• General Attitudes Toward AI: The Benefits subscale exhibits strong positive correlations with general technology affinity indices, self-reported AI optimism, and validated measures of generalized artificial intelligence acceptance (coefficients typically ranging from r = .55 to r = .72, p < .001).
• Comparative Trust in Artificial Intelligence: Perceived Benefits correlate positively with cognitive trust in algorithmic accuracy and competence (r ≈ .61, p < .001), whereas Perceived Risks demonstrate moderate-to-strong negative associations with algorithmic trust (r ≈ −.48 to −.58, p < .001).
• Objective and Subjective Medical AI Knowledge: Kerstan et al. (2024) demonstrated that respondents possessing higher calibrated knowledge regarding healthcare AI reported systematically higher benefit perceptions and differentiated risk perceptions, validating the scale’s sensitivity to cognitive and educational differentials.

Discriminant Validity

Discriminant validity was established via average variance extracted (AVE) analyses and factor correlation matrices. The AVE for both the Perceived Risks and Perceived Benefits latent constructs exceeded the squared correlation between them (r² < .30), satisfying the rigorous Fornell-Larcker criterion. Furthermore, both subscales exhibit minimal confounding correlations with generalized trait anxiety, neuroticism, or non-technological political cynicism, indicating that the AIRBS does not simply capture broad negative affectivity, but rather measures construct-specific cognitive appraisals of medical algorithms.

Predictive and Criterion-Related Validity

The scale exhibits strong predictive validity in forecasting concrete behavioral intentions and comparative preferences:

• Physician versus AI Choice: In experimental paradigms assessing whether participants would elect to receive diagnostic readings or therapeutic prescriptions from a board-certified human physician or an advanced autonomous AI diagnostic system, scores on the AIRBS significantly predicted decision outcomes. Individuals scoring one standard deviation above the mean on the Perceived Risks subscale demonstrated an odds ratio of choosing human physicians exceeding OR = 2.4 (p < .001).
• Data-Sharing Willingness: Scores on the Perceived Risks subscale—specifically items R3 (data security breaches) and R4 (ethical problems)—strongly predict reluctance to integrate wearable biometric telemetry into institutional health databases.

8. Reliability

The AI Risks and Benefits Scale demonstrates exceptional reliability across clinical, collegiate, and general population cohorts, showing robust internal consistency and stability across experimental trials.

Internal Consistency

In the foundational validation study by Kerstan et al. (2024), both subscales exhibited high internal consistency:

• Perceived Risks Subscale (10 items): Cronbach’s alpha coefficient yielded α = .89 (with McDonald’s omega ω_h = .89), indicating high homogeneity across risk items without evidence of excessive item redundancy. Corrected item-total correlations across all 10 risk items ranged from r = .52 to r = .74.
• Perceived Benefits Subscale (9 items): Cronbach’s alpha coefficient reached α = .92 (McDonald’s omega ω_h = .92), reflecting exceptional internal consistency. Corrected item-total correlations for benefit items ranged from r = .58 to r = .78.

Subscale deletions of individual items failed to yield an increase in the composite alpha coefficients, confirming that each item contributes uniquely and reliably to its respective latent construct.

Measurement Invariance

Subsequent psychometric testing established configural and metric measurement invariance across gender demographics (male, female) and age cohorts (younger adults aged 18–35 vs. older adults aged 60+). The comparative fit index differences remained within standard invariance thresholds (ΔCFI < .01, ΔRMSEA < .015), validating that the AIRBS measures identical latent constructs with equivalent metric scaling across structurally diverse population groups.

9. Factor Analysis

The dimensional structure of the AIRBS was verified through rigorous exploratory and confirmatory factor analytic procedures during its original scale development and validation protocol.

Exploratory Factor Analysis (EFA)

Initial validation phases submitted the 19 candidate items to Exploratory Factor Analysis utilizing maximum likelihood extraction paired with oblique (Promax) rotation, assuming theoretical correlation between risk and benefit appraisals. Kaiser-Meyer-Olkin (KMO) measures of sampling adequacy exceeded .91, and Bartlett’s test of sphericity was highly statistically significant (χ²(171) = 3842.12, p < .001), indicating the correlation matrix was well-suited for factor decomposition.

Scree plot examination and Horn’s parallel analysis corroborated the retention of two primary factors with eigenvalues substantially exceeding 1.0 (Factor 1 = 6.42, accounting for 33.8% of variance; Factor 2 = 4.18, accounting for 22.0% of variance). Total cumulative variance explained by the unconstrained two-factor model exceeded 55.8%.

Confirmatory Factor Analysis (CFA)

Confirmatory factor analysis was subsequently conducted to evaluate the fit of the theoretical two-factor model against competing structural configurations (e.g., a unidimensional bipolar model where risks and benefits represent opposite poles of a single attitude continuum). The hypothesized two-factor correlated model exhibited superior model fit indices:

• Chi-Square / Degrees of Freedom Ratio: χ²/df = 2.14 (χ² = 323.14, df = 151, p < .001)
• Root Mean Square Error of Approximation (RMSEA): .048 [90% CI: .041, .056] (indicating close model fit)
• Comparative Fit Index (CFI): .962 (surpassing the standard > .95 threshold)
• Tucker-Lewis Index (TLI): .957
• Standardized Root Mean Square Residual (SRMR): .044 (surpassing the < .08 threshold)

Alternative unidimensional models exhibited severely degraded fit (χ²/df = 6.82, CFI = .712, RMSEA = .118), providing empirical justification for treating perceived risks and perceived benefits as distinct, concurrent cognitive evaluations rather than opposing ends of a single continuum.

Standardized Factor Loadings

All 19 items demonstrated robust, statistically significant standardized factor loadings on their primary theoretical factors (p < .001). Loadings for the Perceived Risks subscale ranged from λ = .58 (R2: job loss) to λ = .81 (R8: medical errors). Loadings for the Perceived Benefits subscale ranged from λ = .64 (B4: information sharing) to λ = .83 (B8: diagnosing health problems). Standardized cross-loadings on non-target factors remained trivial (< .18).

10. Instrument / Measurement Tool

The structured inventory and operational characteristics of the AI Risks and Benefits Scale are outlined below:

• Instrument Name: AI Risks and Benefits Scale (AIRBS)
• Developer / Origin: Sophie Kerstan, Nadine Bienefeld, and Gudela Grote (ETH Zurich, 2024)
• Construct Assessed: Cognitive appraisal of perceived hazards, systemic errors, ethical problems, diagnostic gains, and therapeutic utilities associated with implementing artificial intelligence in healthcare delivery
• Number of Items: 19 items total
  • Perceived Risks Subscale: 10 items (R1 through R10)
  • Perceived Benefits Subscale: 9 items (B1 through B9)
• Response Format: 7-point Likert-type probability scale:
  • 1 = Very unlikely
  • 2 = Unlikely
  • 3 = Somewhat unlikely
  • 4 = Neutral / Neither likely nor unlikely
  • 5 = Somewhat likely
  • 6 = Likely
  • 7 = Very likely
• Administration Modality: Self-report paper-and-pencil or computerized/web-based psychometric inventory
• Estimated Completion Time: Approximately 4 to 6 minutes
• Target Population: Adult general public (aged 18+), clinical patient cohorts, healthcare professionals (physicians, nurses, allied health workers), and healthcare policy stakeholders
• Scoring Protocol:
  • Subscale Mean Scores: Calculate the mean item response for each subscale independently.
    • Risk Score: ∑(R1 to R10) / 10. Potential score range: 1.00 to 7.00. Higher scores reflect elevated perceptions of technological vulnerability, ethical hazard, and dehumanization.
    • Benefit Score: ∑(B1 to B9) / 9. Potential score range: 1.00 to 7.00. Higher scores reflect stronger expectations of clinical precision, predictive capability, and therapeutic accessibility.
  • Net Benefit Index (Optional Differential Score): Researchers investigating behavioral preference or algorithm aversion may compute an algebraic differential score: Net Benefit = Mean(Benefits) − Mean(Risks). A positive differential indicates a net favorable disposition, whereas a negative differential indicates net perceived risk.

11. Permissions & Fee and Test Year

The AI Risks and Benefits Scale was formally introduced in 2024 through an empirical publication in the peer-reviewed journal Risk Analysis (Kerstan, Bienefeld, & Grote, 2024). The original article was published under an open-access Creative Commons Attribution 4.0 International License (CC BY 4.0). Consequently, the instrument is in the public scientific domain and can be utilized, translated, adapted, and integrated into academic research, clinical investigations, and public health surveys without financial cost or formal permission requests, provided that appropriate scholarly attribution is accorded to the original authors and primary publication.

12. References

The following foundational sources provide theoretical, psychometric, and empirical context for the AIRBS:

• Alhakami, A. S., & Slovic, P. (1994). A psychological study of the inverse relationship between perceived risk and perceived benefit. Risk Analysis, 14(6), 1085–1096. https://doi.org/10.1111/j.1539-6924.1994.tb00080.x
• Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
• Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033
• Finucane, M. L., Alhakami, A., Slovic, P., & Johnson, S. M. (2000). The affect heuristic in judgments of risks and benefits. Journal of Behavioral Decision Making, 13(1), 1–17. https://doi.org/10.1111/risa.14216
• Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
• Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90–103. https://doi.org/10.1016/j.obhdp.2018.12.005
• Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886
• Slovic, P. (1987). Perception of risk. Science, 236(4799), 280–285. https://doi.org/10.1126/science.3563507
• Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540

13. Items of the Scale