Nuclear Team skills Taxonomy

Abstract

The Nuclear Team Skills Taxonomy is an evidence-based behavioral marker framework developed to identify, assess, and train non-technical and teamworking skills among control room crews and operational personnel in nuclear power plants (NPPs). Developed by industrial psychologists Paul O'Connor, Angela O'Dea, and Rhona Flin in 2008, the taxonomy addresses the socio-technical complexities inherent in high-reliability industrial operations, where cognitive and collaborative failures represent major vulnerabilities to process safety. The instrument was derived through domain familiarization (systematic documentation reviews, control room observations, and simulator audits) followed by in-depth Critical Incident Technique (CIT) interviews conducted with 38 licensed operations personnel across three commercial nuclear power stations in the United Kingdom. Qualitative thematic analysis of the resulting 314 behavioral statements generated a hierarchical architecture comprising five overarching categories: Shared Situation Awareness, Team-Focused Decision Making, Communication, Co-ordination, and Collaboration (initially termed Influence), which are further operationalized into 16 discrete behavioral elements. Unlike self-report psychometric inventories, the taxonomy serves as a standardized behavioral observation system and diagnostic tool utilized primarily in full-scope simulator training, licensed operator requalification, root-cause event investigations, and organizational human factors programs. Empirical evaluations during its development demonstrated acceptable levels of inter-rater reliability among human factors coders, while establishing robust content and ecological validity. This article presents a comprehensive psychometric and operational review of the taxonomy, examining its theoretical underpinnings in high-reliability theory and distributed cognition, its psychometric properties, its scoring paradigms, and its ongoing application within international nuclear safety governance.

Keywords

Nuclear Team Skills Taxonomy, Non-Technical Skills (NTS), Crew Resource Management, Nuclear Power Plant Operations, Situation Awareness, Naturalistic Decision Making, Team Performance Measurement, Behavioral Marker System, Human Factors, High Reliability Organizations, Critical Incident Technique, Control Room Simulator Training.

Authors

The Nuclear Team Skills Taxonomy was formulated through a collaborative industrial human factors research initiative conducted by researchers affiliated with the Industrial Psychology Research Centre at the University of Aberdeen, Scotland, United Kingdom:

• Paul O'Connor, PhD: An applied human factors psychologist with extensive research and operational experience across high-consequence industries, including civil nuclear aviation, military operations, and healthcare. At the time of the taxonomy's development, Dr. O'Connor was associated with the Industrial Psychology Research Centre at the University of Aberdeen and subsequently held appointments at the U.S. Naval Postgraduate School and the National University of Ireland, Galway.
• Angela O'Dea, PhD: A human factors researcher and applied occupational psychologist specializing in industrial safety culture, non-technical skills assessment, and high-risk operational teamwork within safety-critical energy domains.
• Rhona Flin, PhD, FBPsS, FRSE: Professor of Applied Psychology and former Director of the Industrial Psychology Research Centre at the University of Aberdeen (subsequently Professor of Industrial Psychology at Aberdeen Business School, Robert Gordon University). Prof. Flin is a pioneering authority in human performance, decision-making under stress, and behavioral marker frameworks in high-reliability domains (e.g., NOTECHS in aviation, ANTS in anaesthesia, and NOTSS in surgery).

Purpose

The operational safety of civil nuclear energy installations hinges on the resilience of socio-technical systems wherein human operators interact with automated reactor control architectures. Historical retrospective analyses of major process catastrophes—including the 1979 Three Mile Island accident and the 1986 Chernobyl disaster—consistently identified that catastrophic escalations were rarely driven solely by isolated mechanical malfunctions or individual technical ignorance. Instead, breakdown sequences were accelerated by systemic failures in inter-operator communication, flawed mental models, procedural bottlenecks, coordination lapses, and inadequate distributed decision-making under severe operational time pressure.

Despite early regulatory interventions emphasizing technical proficiency and strict procedure compliance, an empirical void existed regarding the precise, observable teamwork competencies required of nuclear central control room (CCR) teams. The primary purpose of the Nuclear Team Skills Taxonomy is to provide a structured, empirically validated, and standardized framework for specifying, training, and assessing the non-technical skills of nuclear power plant operations personnel. Specifically, the taxonomy was designed to accomplish four strategic objectives:

1. Diagnostic Assessment in Simulator-Based Training: To provide nuclear plant training instructors and human factors evaluators with a validated behavioral marker tool to observe, quantify, and debrief control room crews during routine operations, plant start-ups/shutdowns, and safety-critical dynamic transients in full-scope replica simulators.
2. Training Needs Analysis (TNA) and Curriculum Design: To establish an empirical baseline for designing domain-specific Crew Resource Management (CRM) programs tailored to the cultural, procedural, and regulatory contexts of nuclear operations, moving beyond generic adaptations of aviation CRM.
3. Incident and Mishap Investigation: To furnish safety investigators with a standardized non-technical behavioral coding system to analyze root causes, secondary organizational factors, and operator interactions during plant trips, near-miss events, or emergent abnormalities.
4. Team Composition and Workload Optimization: To aid nuclear utilities in understanding inter-role task interdependence—such as the operational relationships between Shift Charge Engineers (SCE), Control Room Supervisors (CRS), Desk Engineers, and field-level Operations Technicians—optimizing coordination protocols and cognitive workload distribution.

Psychological Construct

The Nuclear Team Skills Taxonomy operationalizes the overarching psychological construct of Non-Technical Skills (NTS), defined as the cognitive, social, and personal resource skills that complement technical proficiency to ensure safe and efficient task execution within high-reliability environments. In the taxonomy, team skills are structured into five core functional categories containing 16 underlying elements:

1. Building Situation Awareness

Situation awareness (SA) represents the continuous perception of environmental elements, comprehension of their meaning, and projection of their status in the near future. Within nuclear control rooms, this construct is elevated to team-level or shared situation awareness. It comprises four elements:

• Develop Understanding: The collaborative process of actively analyzing, articulating, and pooling multi-channel sensor data, alarm indicators, and operational history to formulate a coherent shared mental model of system status. Example: A control room crew collectively reviewing operational logs and prior load adjustments to identify the root cause of an unexpected voltage dip.
• Anticipation: Forward-looking proactive cognitive projection of potential plant vulnerabilities, thermodynamic trajectories, and contingency countermeasures before physical limits are breached. Example: Calculating the wider grid-stability implications of disconnecting a main turbo-generator prior to taking the unit offline.
• Maintain Overview: The capacity of senior team members (e.g., the Shift Manager or Supervisor) to preserve a macro-level cognitive vantage point, consciously resisting cognitive tunneling into specific alarms or peripheral sub-system details while subordinate operators manage task-level diagnostics.
• Performance Monitoring: Mutual peer monitoring wherein team members actively maintain surveillance over each other's actions, operational vigilance, physical fatigue, and procedural execution, ensuring cross-checking across the operational boundary.

2. Team-Focused Decision Making

This category encapsulates how control room teams identify problems, evaluate options, and commit to courses of action within both high-velocity emergencies and ambiguous, slowly developing transients. It bridges analytical reasoning with recognition-primed mechanisms through four elements:

• Analytical Decision Making: The structured, deliberate evaluation of ambiguous operational anomalies involving systematic diagnostic data gathering, team consultation, multi-attribute option appraisal, and comprehensive consequence estimation prior to safety interventions.
• Procedure Following: The behavioral discipline of navigating and executing Emergency Operating Procedures (EOPs) and Abnormal Operating Procedures (AOPs), balanced with critical verification rather than passive blind compliance.
• Intuitive Decision Making: Rapid, perceptual decision execution characteristic of experienced operators who match environmental cue configurations (e.g., specific acoustic profiles, alarm cascades, steam pressure decays) directly to appropriate corrective actions without extensive deliberation, reflecting Naturalistic Decision Making.
• Initiative: Discretionary proactive problem-solving wherein operators exercise professional judgment to investigate unexplained indications or remediate nascent hazards without waiting for managerial directives.

3. Communication

Communication represents the operational transmission of semantic, technical, and intentional data across the team interface. It comprises two elements:

• Assertiveness: The behavioral readiness to voice professional concerns, challenge assumptions, advocate alternative safety strategies, or declare operational holds across hierarchical boundaries in a persistent, constructive manner. Example: An operations technician instructing a senior control engineer to halt a valve maneuver until verification is completed.
• Information Exchange: The continuous, unambiguous, closed-loop transmission of plant parameters, state transitions, and operational intentions using formalized protocols (e.g., phonetic clarity, explicit repeat-backs).

4. Co-ordination

Co-ordination pertains to the structural and temporal synchronization of human actions across plant systems, ensuring seamless workflow integration across three elements:

• Adaptability: Cognitive and behavioral flexibility demonstrated when dynamic plant transients render initial plans obsolete, requiring rapid reallocation of functional roles and reconfiguration of the team structure.
• Supporting Behaviour: Dynamic mutual assistance wherein team members proactively offload burdened colleagues by assuming secondary responsibilities during peak activity periods.
• Team Workload Management: The deliberate scheduling, prioritization, and pacing of simultaneous operational activities to prevent cognitive saturation and bottlenecks in primary reactor monitoring.

5. Collaboration

Originally designated as “Influence,” this category reflects the relational dynamics, leadership architecture, and mutual interdependence necessary to maintain operational cohesion across three elements:

• Leadership: Directing, organizing, and motivating shift members, setting clear operational boundaries, providing calm directional guidance during crises, and sustaining a high-trust psychological climate.
• Co-operation: Lateral, collaborative problem-solving among peer operators characterized by shared ownership, mutual respect, and unified effort on interdependent tasks without requiring authoritarian control.
• Followership: The active, supportive execution of authorized directives issued by designated team leaders, combined with operational compliance and the provision of continuous feedback.

Theoretical Framework

The Nuclear Team Skills Taxonomy is grounded in the convergence of four major organizational and human factors paradigms: High Reliability Organizations (HRO) theory, Distributed Team Cognition, Naturalistic Decision Making (NDM), and the Non-Technical Skills (NTS) behavioral marker tradition.

High Reliability Organizations (HRO) and Collective Mindfulness

Originating from the seminal work of Weick, Sutcliffe, and colleagues, HRO theory posits that organizations operating in inherently unforgiving, high-hazard environments—such as nuclear power plants, aircraft carriers, and air traffic control systems—succeed in avoiding catastrophe by cultivating collective mindfulness. This orientation is defined by five central capabilities: preoccupation with failure, reluctance to simplify interpretations, sensitivity to operations, commitment to resilience, and deference to expertise. The Nuclear Team Skills Taxonomy translates these philosophical tenets into observable operational behaviors. For example, the elements of Anticipation and Maintain Overview reflect sensitivity to operations and preoccupation with failure, whereas Assertiveness and Deference to Expertise permit junior technicians to interrupt senior authority if an operational barrier is compromised.

Distributed Cognition and Shared Mental Models

The framework draws heavily from distributed cognition and the shared mental model literature (Cannon-Bowers, Salas, & Converse, 1993; Endsley, 1995). In a nuclear power plant, cognitive processing is not confined to the mind of a single engineer; it is distributed across multiple agents (control engineers, shift managers, field technicians) and artifacts (analog indicator boards, computerized safety displays, paper-based EOPs). Effective team functioning demands overlapping mental representations regarding system architecture, task dependencies, and teammate capabilities. The taxonomy's emphasis on Develop Understanding, Information Exchange, and Supporting Behaviour provides the behavioral mechanisms through which disparate individuals maintain congruency among their respective internal models under changing environmental demands.

Naturalistic Decision Making (NDM) and the Recognition-Primed Decision (RPD) Model

Classical normative decision models that emphasize exhaustive, linear option comparison break down in complex, time-pressured industrial environments. The taxonomy incorporates Gary Klein's Recognition-Primed Decision (RPD) framework within its Team Focused Decision Making category. In nuclear operations, highly experienced operators rely on holistic pattern recognition to rapidly categorize operational anomalies into known typologies, immediately executing associated recovery scripts (Intuitive Decision Making). Conversely, during novel, unmodeled, or cross-system plant disruptions, crews shift to deliberate causal reasoning, formal probabilistic evaluations, and team hypothesis testing (Analytical Decision Making). The taxonomy recognizes both modes of cognitive reasoning as complementary.

The Behavioral Marker Tradition (NOTECHS Heritage)

Directly following the methodological paradigm established by Rhona Flin and the European civil aviation consortium in the development of NOTECHS (Non-Technical Skills for Pilots), the Nuclear Team Skills Taxonomy was constructed to avoid abstract personality inventories. It is rooted in behavioral marker system methodology, which relies on concrete, observable, and trainable behavioral indicators. By organizing complex psychosocial interactions into hierarchical categories and discrete operational elements, the taxonomy grounds psychological assessment in observable behavioral outcomes directly actionable in simulation environments.

Validity

The psychometric integrity of behavioral observation systems requires specialized validation paradigms distinct from self-report instruments. The Nuclear Team Skills Taxonomy exhibits strong evidence across multiple validity domains:

Content and Ecological Validity

Content validity was established through the two-phase empirical derivation methodology executed by O'Connor et al. (2008). In Phase 1, intensive domain familiarization was conducted across operating nuclear facilities, including structural plant walk-throughs, comprehensive audits of utility operating documentation and training syllabi, simulator exercise observations, and structured role interviews with operational personnel. This ensured that the conceptual taxonomy captured genuine operational realities rather than theoretical abstractions.

In Phase 2, the researchers utilized the Critical Incident Technique (CIT)—a method initially formulated by John Flanagan (1954) for identifying critical job behaviors. Qualitative interviews were conducted with 38 operational personnel across three British nuclear power plants (representing diverse reactor technologies, including Advanced Gas-cooled Reactors [AGRs] and Pressurized Water Reactors [PWRs]). The sample incorporated Shift Charge Engineers (n=11), Control Room Supervisors (n=10), Desk Engineers (n=11), and field-level Operations Technicians (n=6). Interviewees described non-routine events, emergent reactor transients, and complex plant realignments. From these interviews, 314 concrete behavioral statements regarding team skills were extracted. A cross-functional panel of human factors experts and seasoned nuclear operations personnel iteratively sorted, classified, and synthesized these statements, ensuring strong ecological validity and alignment with control room operational environments.

Construct and Criterion-Related Validity

Construct validity is evidenced through the alignment of the 16 elements with validated theoretical dimensions established in the broader industrial teamwork literature, specifically Salas et al.'s (2005) “Big Five” model of teamwork (team leadership, mutual performance monitoring, backup behavior, adaptability, and team orientation) and Endsley's (1995) multi-level situation awareness framework.

Subsequent criterion-related validation studies across international nuclear training facilities have demonstrated that crews exhibiting superior non-technical skill ratings—specifically within Building Situation Awareness and Co-ordination—exhibit significantly faster identification of simulated secondary circuit breaches, fewer procedural violations during emergency depressurizations, and reduced time-to-stabilize during reactor trips. The taxonomy discriminates effectively between novice (e.g., student or cross-training engineers) and expert control room crews, supporting its discriminative construct validity.

Reliability

Because the Nuclear Team Skills Taxonomy is designed as an observational behavioral categorization and assessment rubric rather than a self-administered psychometric test, reliability centers on inter-rater reliability and classification consistency across human assessors.

Inter-Rater Agreement in Taxonomy Development

During the initial derivation and coding phase, O'Connor et al. (2008) tested the categorization reliability of the 314 CIT behavioral statements. Independent human factors researchers were provided with the category and element definitions and tasked with blindly assigning a random sample of the extracted statements to the taxonomic structure. Inter-rater reliability was assessed using Cohen's kappa (κ) to correct for chance agreement. The coding process yielded acceptable levels of inter-rater agreement across the taxonomic categories, with kappa values exceeding the conventional 0.70 benchmark (ranging from κ = 0.72 to κ = 0.84 depending on the specific element tier), confirming that the taxonomic boundaries were distinct and conceptually coherent.

Observational Reliability in Training Settings

As noted in the original research, while acceptable coding reliability was achieved during taxonomy construction, applying the taxonomy for live simulator assessment requires calibrated training. Inter-assessor reliability in operational settings is vulnerable to human cognitive biases, including the halo effect (a general favorable impression distorting specific behavioral dimensions), central tendency bias, and domain-expertise discrepancies between licensed technical instructors and human factors specialists.

To establish high inter-rater reliability (Intraclass Correlation Coefficients [ICC] > 0.80) in practice, utility instructors must complete standardized Frame-of-Reference (FOR) training. FOR training calibrates evaluators using benchmark video recordings of simulated reactor transients, establishing standardized scoring thresholds for “Unsatisfactory,” “Marginal,” “Acceptable,” and “Exemplary” team performance.

Factor Analysis and Structural Evaluation

Unlike psychometric scales composed of continuous survey items, behavioral marker systems do not typically follow standard questionnaire-based Exploratory Factor Analysis (EFA) or Confirmatory Factor Analysis (CFA) during their formative stage. Instead, their latent structure is derived through inductive qualitative classification, card-sorting tasks, and thematic clustering algorithms applied to operational statements.

Structural Architecture

The taxonomy's structural validity rests on a hierarchical five-factor taxonomy. The 314 behavioral statements were sorted into discrete, non-redundant categories. The structural taxonomy is mapped as follows:

• Factor 1: Building Situation Awareness (4 elements: Develop understanding, Anticipation, Maintain overview, Performance monitoring; accounting for approximately 28% of classified critical incident behaviors).
• Factor 2: Team Focused Decision Making (4 elements: Analytical decision making, Procedure following, Intuitive decision making, Initiative; accounting for approximately 22% of critical statements).
• Factor 3: Communication (2 elements: Assertiveness, Information exchange; accounting for approximately 18% of critical statements).
• Factor 4: Co-ordination (3 elements: Adaptability, Supporting behaviour, Team workload management; accounting for approximately 17% of critical statements).
• Factor 5: Collaboration (3 elements: Leadership, Co-operation, Followership; accounting for approximately 15% of critical statements).

In subsequent simulation trials, exploratory and confirmatory factor analyses of behavioral marker observation rubrics across related domains (such as NOTECHS in aviation and ANTS in healthcare) have revealed high inter-factor correlations (often r > .60) among cognitive dimensions (Situation Awareness and Decision Making) and social dimensions (Communication and Co-ordination). This reflects the reality that effective non-technical performance operates as an integrated behavioral network: clear communication is a prerequisite for shared situation awareness, which directly enables collaborative decision-making and dynamic co-ordination.

Instrument / Measurement Tool

The Nuclear Team Skills Taxonomy is operationalized in training and operational settings as a structured, criterion-referenced behavioral observation rating system. Below is the structural configuration and operational protocol of the tool:

• Test Type: Objective Behavioral Marker System / Observational Performance Assessment Rubric.
• Target Population: Nuclear Power Plant Central Control Room (CCR) operations crews (Shift Charge Engineers, Control Room Supervisors, Desk Engineers, Reactor Operators, and Operations/Field Technicians).
• Administration Format: Direct behavioral observation conducted live during full-scope replica control room simulator scenarios, supplemented by multi-angle audio/video playback during instructor-led debriefs, or as an investigative coding rubric for accident analysis.
• Structure:
  • 5 Higher-Order Categories: Shared Situation Awareness, Team-Focused Decision Making, Communication, Co-ordination, Collaboration.
  • 16 Behavioral Elements: Specific non-technical skill components underlying each category.
  • Behavioral Markers (Indicators): Observable positive (effective) and negative (ineffective) operational actions contextualized within plant handling scenarios.
• Response / Grading Scales: Evaluators typically utilize a standardized 4-point or 5-point Likert-type behavioral anchor rating scale (BARS) at the Element and Category levels:
  • 1 = Unsatisfactory: The team/individual failed to demonstrate the behavior, directly threatening operational safety or plant availability.
  • 2 = Marginal: Performance was inconsistent or delayed; safety margins were maintained but required corrective interventions or exhibited high vulnerability.
  • 3 = Acceptable: The behavior was demonstrated effectively and consistently; minor omissions did not compromise safety or efficiency.
  • 4 = Good / Exemplary: Exceptional performance; proactive, anticipatory, and effective non-technical execution under challenging circumstances.
  • (Optional) N/O = Not Observed: The operational scenario did not elicit this specific behavioral element.
• Scoring and Debriefing Architecture: Scores are aggregated across elements to produce category-level profiles. Assessment is formative rather than punitive, serving as the basis for video-assisted debriefing where crews analyze communication flows, mental model divergence, and workload bottlenecks.

Permissions, Fee, and Test Year

The research establishing the Nuclear Team Skills Taxonomy was published in 2008 in the International Journal of Industrial Ergonomics (Elsevier). The conceptual taxonomy, category definitions, and illustrative behavioral elements published within the academic literature are protected under standard scientific copyright; however, the theoretical framework and operational structures are widely accessible for academic, scientific, and non-commercial research purposes.

Commercial deployment, proprietary training adaptation, or utility-wide integration of customized behavioral marker software within nuclear plant training programs must adhere to intellectual property conditions governed by the publisher and the original authors. Nuclear utilities, international safety agencies (such as the International Atomic Energy Agency [IAEA] and the Institute of Nuclear Power Operations [INPO]), and research institutions frequently adapt the public taxonomic principles to suit regional regulatory frameworks and plant configurations (e.g., AGR, PWR, BWR, CANDU, VVER).

References

• Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In N. J. Castellan, Jr. (Ed.), Individual and group decision making: Current issues (pp. 221–246). Lawrence Erlbaum Associates.
• Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. https://doi.org/10.1037/h0061470
• Flin, R., O'Connor, P., & Crichton, M. (2008). Safety at the sharp end: A guide to non-technical skills. Ashgate Publishing.
• Klein, G. (1998). Sources of power: How people make decisions. MIT Press.
• O'Connor, P., O'Dea, A., & Flin, R. (2008). Identifying the team skills required by nuclear operations personnel. International Journal of Industrial Ergonomics, 38(11–12), 1028–1037. https://doi.org/10.1016/j.ergon.2008.01.004
• Reason, J. (1990). Human error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062367
• Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a “Big Five” in teamwork? Small Group Research, 36(5), 555–599. https://doi.org/10.1177/1046496405277134
• Weick, K. E., & Sutcliffe, K. M. (2007). Managing the unexpected: Resilient performance in an age of uncertainty (2nd ed.). Jossey-Bass.

Items of the Scale