Abstract
The Ambiguous Decisions Task (ADT; van Mulukom et al., 2024) is a psychometric and experimental assessment tool developed to measure metacognitive trust in thinking styles. Unlike conventional self-report inventories that assess subjective dispositional preferences (such as the Rational-Experiential Inventory) or performance tasks that assess cognitive ability through veridical problem-solving (such as the Cognitive Reflection Test), the ADT assesses an individual’s evaluative confidence, epistemic endorsement, and social-metacognitive validation of analytical versus intuitive decision-making strategies when facing uncertain, real-world dilemmas. Comprising 12 carefully structured ambiguous scenarios, the task presents respondents with two contrasting agents: one employing an explicit, systematic, data-driven analytical strategy, and another utilizing an experiential, rapid, gut-level intuitive strategy. Participants evaluate both approaches independently using a continuous horizontal visual analogue rating scale ranging from 0 (“Not at all”) to 100 (“Completely”), calibrated in 10-point increments.
The ADT yields two functionally independent subscale scores: Analytical Decisions and Intuitive Decisions. Psychometric evaluation across cross-national adult cohorts spanning the United States, Canada, and Brazil demonstrates robust psychometric properties. Exploratory factor analyses using common principal axis factor analysis confirm that both subscales possess robust unidimensional structural validity without cross-factor contamination, accounting for 36.9% of the variance in analytical evaluations (Eigenvalue = 4.42; factor loadings = 0.40 to 0.69) and 42.2% of the variance in intuitive evaluations (Eigenvalue = 5.06; factor loadings = 0.44 to 0.72). Both subscales display high internal consistency, with Cronbach’s alpha coefficients of α = .83 for the Analytical Decisions subscale and α = .87 for the Intuitive Decisions subscale. The instrument demonstrates robust convergent and discriminant validity, exhibiting convergent associations with established first-order cognitive metrics while demonstrating that metacognitive trust operates as a distinct psychological layer in cognitive processing, belief formation, and behavioral decision science.
Keywords
Ambiguous Decisions Task, Metacognitive Trust, Thinking Styles, Dual-Process Theory, Intuitive Decisions, Analytical Decisions, Cognitive Reflection, Decision-Making Under Ambiguity, Epistemic Cognition, Psychometrics
Authors
The Ambiguous Decisions Task was developed by an international consortium of researchers specializing in experimental psychology, cognitive science, and social-epistemic cognition:
- Valerie van Mulukom, Ph.D. — Centre for Trust, Peace and Social Relations, Brain, Belief & Behaviour Lab, Coventry University, Coventry, United Kingdom. (ORCID: 0000-0002-8611-3700; Email: [email protected]).
- Adam Baimel, Ph.D. — Center for Psychological Research, Department of Psychology, Oxford Brookes University, Oxford, United Kingdom.
- Everton Maraldi, Ph.D. — Graduate Program in Social Psychology, Pontifical Catholic University of São Paulo (PUC-SP), São Paulo, Brazil.
- Miguel Farias, D.Phil. — Centre for Trust, Peace and Social Relations, Brain, Belief & Behaviour Lab, Coventry University, Coventry, United Kingdom. (ORCID: 0000-0003-4939-5026).
Correspondence Address: Dr. Valerie van Mulukom, Centre for Trust, Peace and Social Relations, Brain, Belief & Behaviour Lab, Innovation Village IV5, Cheetah Road, Coventry, CV1 2TL, United Kingdom.
Purpose
The primary purpose of the Ambiguous Decisions Task (ADT) is to assess an individual’s metacognitive trust in analytical and intuitive thinking styles in decision-making contexts characterized by profound epistemic ambiguity. For decades, cognitive and social psychology have measured human rationality and intuition using either performance-based tasks, such as the Cognitive Reflection Test (CRT; Frederick, 2005), or self-report dispositional scales, such as the Rational Experiential Inventory (REI; Epstein et al., 1996). However, both methodological approaches suffer from critical conceptual and psychometric limitations that the ADT was explicitly designed to overcome.
First, performance-based measures like the CRT assess an individual’s ability to suppress an initial intuitive, incorrect response in favor of an analytically derived correct answer. While highly effective at evaluating object-level cognitive inhibition and numerical reasoning, these tasks are inherently normative: they prescribe a single logically correct answer and equate intuition with cognitive failure or bias. Consequently, they fail to measure scenarios in which intuition might be ecologically adaptive, or situations where analytical rumination yields suboptimal outcomes due to missing or unquantifiable information. Second, self-report inventories like the REI measure self-concept, self-efficacy, and personal enjoyment of thinking (e.g., “I enjoy intellectual challenges” or “I trust my initial feelings”). Such scales conflate an individual’s perceived identity or personality traits with their actual endorsement of thinking styles when observing or solving dynamic problems.
The ADT bridges this empirical gap by presenting ambiguous scenarios that lack a single objectively verifiable “right” or “wrong” answer. In each scenario, one character uses an explicit analytical process (gathering data, systematically comparing metrics, calculating probabilities), while another character uses an intuitive process (relying on gut feelings, tacit experience, holistic impressions). By asking participants to rate their support for both approaches independently, the ADT yields distinct, non-normative indices of metacognitive trust. Crucially, the task does not force a zero-sum trade-off between rationality and intuition; instead, it acknowledges that a decision-maker may simultaneously place high epistemic trust in both styles, place low trust in both, or favor one over the other depending on their metacognitive orientation.
In research contexts, the ADT serves as a sensitive tool for investigating epistemic vigilance, cognitive flexibility, susceptibility to misinformation, ideological dogmatism, conspiracy endorsement, and scientific skepticism. In organizational and clinical contexts, the instrument provides clinicians, executive coaches, and behavioral economists with an objective lens to evaluate how individuals navigate uncertainty, manage ambiguity-induced anxiety, and allocate cognitive confidence across complex life domains.
Psychological Construct
The core construct captured by the ADT is Metacognitive Trust in Thinking Styles. In cognitive science, metacognition refers to higher-order cognitive processes involved in monitoring, evaluating, and regulating one’s own or others’ cognitive activities (Flavell, 1979; Koriat, 2007). When extended to decision-making styles, metacognitive trust represents the degree of subjective confidence, credibility, and normative endorsement an individual assigns to analytical versus intuitive cognitive operations as reliable pathways to truth and optimal decision-making.
The construct is partitioned into two orthogonal, functionally independent subscales:
1. Analytical Decisions (Analytical Metacognitive Trust)
The Analytical Decisions dimension measures the extent to which an individual endorses deliberate, systematic, rule-based, and evidence-driven cognitive processes when resolving ambiguity. Individuals scoring high on this dimension believe that objective data, logical decomposition, computational scrutiny, and systematic comparisons are the most legitimate and trustworthy tools for resolving real-world uncertainty, even when information is incomplete. They endorse characters who create spreadsheets, consult statistical reports, cross-reference objective criteria, and methodically weigh pros and cons. A low score on this subscale does not necessarily indicate hostility toward logic, but rather an awareness of the diminishing returns or paralysis associated with over-analysis in unpredictable contexts.
2. Intuitive Decisions (Intuitive Metacognitive Trust)
The Intuitive Decisions dimension captures the extent to which an individual validates tacit, experiential, associative, and gut-level feelings as reliable guides under conditions of uncertainty. Individuals scoring high on this dimension trust rapid, holistic pattern-matching, somatic markers, and subconscious heuristics. They endorse characters who rely on instinctive hunches, emotional comfort, immediate resonance, and lived experience rather than waiting for exhaustive external data. Crucially, in the ADT framework, intuitive trust is not conceptualized as irrationality or cognitive laziness; rather, it reflects epistemic faith in heuristic processing, evolutionary adaptive instincts, and implicit learning.
The conceptual independence of these two dimensions is a defining feature of the ADT. Rather than positioning analytical and intuitive processing on opposite poles of a single continuum, the ADT allows for four distinct epistemic profiles:
- Dual-Trust Endorsers (High Analytical / High Intuitive): Individuals who demonstrate high cognitive flexibility, recognizing the ecological validity of both systematic calculation and intuitive wisdom depending on situational affordances.
- Rational Purists (High Analytical / Low Intuitive): Individuals who exclusively trust algorithmic, formal, and quantifiable evidence, actively dismissing intuition as noisy, biased, or irrational.
- Intuitive Purists (Low Analytical / High Intuitive): Individuals who prioritize subjective resonance, visceral authenticity, and lived experience, exhibiting profound skepticism toward bureaucratic, statistical, or hyper-rational approaches.
- Skeptical Disengaged (Low Analytical / Low Intuitive): Individuals who demonstrate generalized epistemic doubt, finding neither analytical rigor nor gut instinct sufficiently compelling to resolve fundamental ambiguities.
Theoretical Framework
The Ambiguous Decisions Task is grounded primarily in Dual-Process Theories of Cognition (Evans & Stanovich, 2013; Kahneman, 2011) and Seymour Epstein’s Cognitive-Experiential Self-Theory (CEST) (Epstein et al., 1996). Dual-process theories posit that human judgment arises from the dynamic interplay between two distinct modes of information processing: Type 1 (Intuitive/Experiential), which is autonomous, fast, context-dependent, and computationally undemanding; and Type 2 (Analytical/Rational), which is deliberative, slow, rule-governed, capacity-limited, and requires working memory.
While classical dual-process models often framed Type 1 processing as error-prone and Type 2 processing as normative, modern iterations (e.g., Evans & Stanovich, 2013; Thompson et al., 2011) emphasize that both systems have evolved to solve distinct environmental problems. Type 1 heuristics can yield optimal outcomes under severe time constraints or complex naturalistic conditions (as demonstrated in naturalistic decision-making; Klein, 1998; Gigerenzer & Gaissmaier, 2011), whereas Type 2 cognitive decoupling is necessary for formal logic, counterfactual simulation, and probabilistic computation.
The ADT extends dual-process theory by integrating Valerie Thompson’s Metacognitive Dual-Process Framework (Thompson et al., 2011). Thompson argues that the initiation and persistence of Type 2 analytical thinking is moderated by a metacognitive evaluation: specifically, the Feeling of Rightness (FOR) triggered by initial Type 1 outputs. If an intuitive output carries a high FOR, analytical intervention is aborted; if the FOR is low, analytical reflection is recruited. The ADT operationalizes this metacognitive tier not as a fleeting internal cue during arithmetic problem-solving, but as a stable, generalizable epistemic stance. When an individual observes third-party decision-makers navigating complex real-world dilemmas, their evaluation reflects their underlying normative beliefs about which cognitive mode is epistemically superior in ambiguous contexts.
Furthermore, the task incorporates insights from Epistemic Rationality and Metacognitive Monitoring (Stanovich, 2011). Stanovich distinguishes between the algorithmic mind (individual differences in cognitive capacity and fluid intelligence) and the reflective mind (rational thinking dispositions and mindware). The ADT operates squarely within the domain of the reflective mind, evaluating how individuals epistemically validate cognitive strategies without relying on IQ or numeric computational capability.
Validity
The psychometric validation of the Ambiguous Decisions Task conducted by van Mulukom et al. (2024) establishes robust construct, convergent, and discriminant validity across diverse demographic and international cohorts.
Construct and Factorial Validity
Construct validity was established by testing whether the 12 scenarios accurately operationalized the theoretical distinction between analytical and intuitive metacognitive evaluations. Exploratory Factor Analysis (Principal Axis Factoring) revealed that responses to the analytical characters loaded exclusively onto an analytical factor, while responses to the intuitive characters loaded onto a distinct intuitive factor. Crucially, the subscales demonstrated orthogonality or modest inverse correlations depending on the context, confirming that trusting analysis does not necessitate an equal rejection of intuition, thereby validating the two-dimensional construct.
Convergent Validity
Convergent validity was demonstrated through systematic correlations with established cognitive style inventories:
- Cognitive Reflection Test (CRT): The Analytical Decisions subscale exhibited significant positive correlations with CRT performance, confirming that individuals with higher objective cognitive reflection also place greater metacognitive trust in analytical decision strategies. Conversely, the Intuitive Decisions subscale demonstrated negative correlations with CRT scores.
- Rational-Experiential Inventory (REI): Metacognitive trust in analytical decisions correlated positively with the Need for Cognition (Rational Ability and Rational Engagement) subscales of the REI. Metacognitive trust in intuitive decisions correlated significantly and positively with Faith in Intuition (Experiential Ability and Experiential Engagement).
Discriminant Validity
Crucially, van Mulukom et al. (2024) demonstrated that the ADT captures unique variance that is not accounted for by first-order personality or cognitive ability measures. While the REI measures self-reported disposition (e.g., “I like solving puzzles”), regression models indicated that the ADT predicts scenario-specific epistemic endorsements even after controlling for baseline REI scores. Furthermore, the task displayed discriminant validity from general social desirability response bias, as the ambiguous nature of the dilemmas eliminates clear socially sanctioned “correct” answers.
Cross-Cultural and Demographic Generalizability
Validity was corroborated across diverse samples from the United States, Canada, and Brazil. Despite notable cultural differences in baseline cognitive styles and intuitive trust between Western, Educated, Industrialized, Rich, and Democratic (WEIRD) populations and South American cohorts, the structural integrity of the two subscales remained consistent, demonstrating cross-cultural metric invariance.
Reliability
The Ambiguous Decisions Task exhibits high internal consistency and psychometric reliability across repeated testing cohorts (van Mulukom et al., 2024):
- Analytical Decisions Subscale: The 12 items assessing analytical decision-making yielded a Cronbach’s alpha of α = .83. Item-total correlations ranged from moderate to high, and iterative item-elimination analyses confirmed that removing any single scenario did not yield an increase in overall reliability, demonstrating that all 12 scenarios contribute meaningfully to the composite score.
- Intuitive Decisions Subscale: The 12 items assessing intuitive decision-making yielded a Cronbach’s alpha of α = .87. Similar to the analytical subscale, no single item degradation was observed upon item deletion, demonstrating high consistency in how participants evaluate intuitive agents across varying contextual themes.
The split-half reliability and composite reliability coefficients similarly exceed the standard psychometric threshold of .80, indicating that the ADT possesses sufficient measurement precision for use in both basic experimental research and individual-differences assessments.
Factor Analysis
To investigate the latent dimensional structure of the Ambiguous Decisions Task, van Mulukom et al. (2024) conducted exploratory factor analyses using common Principal Axis Factoring (PAF). Because participants provided two ratings per scenario (one for the analytical character and one for the intuitive character), factor analyses were conducted separately on the analytical evaluations and intuitive evaluations to evaluate the unidimensionality of each subscale.
Analytical Decisions Factor Structure
For the 12 analytical decision items:
- The first unrotated factor extracted had an Eigenvalue of 4.42, accounting for 36.9% of the total variance.
- Standardized factor loadings across the 12 scenarios ranged from 0.40 to 0.69, with all items exhibiting significant positive loadings on the primary latent dimension.
- Examination of the scree plot revealed an elbow after the first factor, with subsequent eigenvalues dropping substantially below 1.0, confirming an unforced unidimensional structure for analytical metacognitive trust.
Intuitive Decisions Factor Structure
For the 12 intuitive decision items:
- The first unrotated factor extracted had an Eigenvalue of 5.06, accounting for 42.2% of the total variance.
- Standardized factor loadings across the 12 scenarios ranged from 0.44 to 0.72, demonstrating that diverse ambiguous scenarios (ranging from career shifts to medical decisions) consistently load onto a single latent construct of intuitive endorsement.
- Scree plot inspection demonstrated a clean tapering after the primary factor, confirming that intuitive evaluations reflect a unified, unidimensional metacognitive judgment.
When analytical and intuitive ratings were modeled simultaneously in a two-factor exploratory model, cross-loadings between analytical and intuitive items were negligible (< .15), providing definitive empirical evidence that analytical trust and intuitive trust operate as distinct, non-overlapping psychological dimensions rather than mutually exclusive extremes of a single axis.
Instrument / Measurement Tool
The Ambiguous Decisions Task is administered as a structured, computerized behavioral-evaluative task. The structural parameters of the instrument are detailed below:
- Test Type: Situational Judgment / Behavioral Scenario Task (Metacognitive Thinking Style Assessment).
- Administration Method: Electronic / Computerized (optimized for Qualtrics, Gorilla Experiment Builder, Pavlovia, and modern web browsers).
- Target Population: Adults (18 years and older); applicable across general, student, and cross-national populations.
- Estimated Completion Time: Approximately 8 to 12 minutes.
- Number of Scenarios: 12 standardized real-world vignettes depicting high-stakes decisions under conditions of irreducible ambiguity.
- Number of Evaluative Items: 24 total ratings (12 scenarios × 2 decision-making strategies evaluated independently).
- Response Format: Continuous horizontal visual analogue scale (slider bar) anchored from 0 (“Not at all”) to 100 (“Completely”), displaying numerical increments of 10 for participant orientation.
- Subscale Partitioning:
- Analytical Decisions Score: Arithmetic mean (or sum) of the 12 ratings evaluating characters who utilized analytical, data-driven, and systematic decision strategies (score range: 0–100).
- Intuitive Decisions Score: Arithmetic mean (or sum) of the 12 ratings evaluating characters who utilized intuitive, heuristic, and gut-feeling decision strategies (score range: 0–100).
- Scoring and Interpretation Guidelines:
- Independent Scoring: Analytical and intuitive scores must be calculated independently; they should not be collapsed into a single difference score unless an explicit relative-preference index is required for specific experimental hypotheses.
- High Analytical Score (> 65): Reflects strong metacognitive confidence in deliberative, systematic, evidence-based methods when facing uncertainty.
- High Intuitive Score (> 65): Reflects strong metacognitive confidence in affective heuristics, gut feelings, and tacit pattern recognition under uncertainty.
- Interaction Analysis: Researchers frequently compute interaction or quadrant groupings (e.g., Dual Endorsers vs. Rational Purists vs. Intuitive Purists vs. Skeptics) to examine how epistemic trust moderates real-world outcomes such as misinformation acceptance or scientific literacy.
Permissions & Fee and Test Year
The Ambiguous Decisions Task was officially published in 2024 in Consciousness and Cognition by Dr. Valerie van Mulukom and colleagues. In accordance with open-science principles, the task was developed for open academic, non-commercial use. Researchers wishing to utilize the ADT in empirical studies, grant-funded investigations, or non-profit academic research may do so freely, provided appropriate scholarly attribution is cited.
Supplementary materials, procedural protocols, and scoring scripts are cataloged via the Open Science Framework (OSF) repository. Researchers interested in commercial implementations, large-scale organizational testing, or translation rights are encouraged to contact the primary author, Dr. Valerie van Mulukom (Centre for Trust, Peace and Social Relations, Coventry University; [email protected]).
References
- Epstein, S., Pacini, R., Denes-Raj, V., & Heier, H. (1996). Individual differences in intuitive-experiential and analytical-rational thinking styles. Journal of Personality and Social Psychology, 71(2), 390–405. https://doi.org/10.1037/0022-3514.71.2.390
- Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223–241. https://doi.org/10.1177/1745691612460685
- Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
- Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25–42. https://doi.org/10.1257/089533005775196732
- Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62, 451–482. https://doi.org/10.1146/annurev-psych-120709-145346
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
- Klein, G. (1998). Sources of Power: How People Make Decisions. MIT Press.
- Koriat, A. (2007). Metacognition and consciousness. In P. D. Zelazo, M. Moscovitch, & E. Thompson (Eds.), The Cambridge Handbook of Consciousness (pp. 289–325). Cambridge University Press. https://doi.org/10.1017/CBO9780511816789.012
- Stanovich, K. E. (2011). Rationality and the Reflective Mind. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195341140.001.0001
- Thompson, V. A., Prowse Turner, J. A., & Pennycook, G. (2011). Intuition, reason, and metacognition. Cognitive Psychology, 63(3), 107–140. https://doi.org/10.1016/j.concog.2011.06.001
- van Mulukom, V., Baimel, A., Maraldi, E., & Farias, M. (2024). Measuring metacognitive trust in thinking styles: Development and validation of the Ambiguous Decisions Task. Consciousness and Cognition, 118, 103689. https://doi.org/10.1016/j.concog.2024.103689