Nontechnical skills taxonomy for Officers of the Deck

1. Abstract

The Nontechnical skills taxonomy for Officers of the Deck (commonly operationalized as the Nontechnical Skills for Officers of the Deck [NTSOD] rating system) is an empirically grounded behavioral marker framework developed to assess, train, and quantify the critical cognitive and interpersonal competencies required of surface warfare watchstanders in the United States Navy. Serving as the direct representative of the Commanding Officer, an Officer of the Deck (OOD) bears ultimate tactical and navigational accountability for the vessel’s safety and mission execution. Derived from an exhaustive synthesis of maritime and aviation human factors literature, expert focus groups with experienced surface warfare officers, and qualitative classification of 149 critical incident statements extracted from 16 operational debriefs, the NTSOD framework translates cognitive psychology and team dynamics into observable maritime performance metrics. The taxonomy structures nontechnical performance across four core overarching categories comprising ten granular behavioral elements: Leadership (Managing Watch Team, Coping with Stress), Communications (Providing Information, Issuing Orders), Situational Awareness (Gathering Awareness, Understanding Awareness, Anticipating Future Events), and Decision Making (Analytical Decision Making, Following Orders & Procedures, Intuitive Decision Making). Performance across these elements is evaluated utilizing a standardized 4-point anchored rating scale ranging from 1 (Unsatisfactory) to 4 (Outstanding), alongside a Not Observed (N/O) protocol. Psychometric validation studies demonstrate acceptable levels of inter-rater reliability (achieved across three systematic iterations of taxonomy refinement) and high content validity. This comprehensive marker system serves as an operational benchmark for Bridge Resource Management (BRM), simulator-based naval navigation assessments, and systemic safety interventions across both military and commercial maritime domains.

2. Keywords

Officer of the Deck, nontechnical skills, NTSOD, behavioral marker system, maritime safety, situational awareness, Bridge Resource Management, naturalistic decision making, naval watchstanding, inter-rater reliability, human factors, surface warfare.

3. Authors

The Nontechnical Skills taxonomy for Officers of the Deck was conceived, operationalized, and psychometrically validated by human factors researchers and naval officers affiliated with the Naval Postgraduate School (NPS) in Monterey, California:

• William M. Long, MS, LCDR, USN — Department of Operations Research, Naval Postgraduate School, Monterey, California, United States.
• Paul O’Connor, PhD — Associate Professor of Human Factors, Department of Operations Research, Naval Postgraduate School, Monterey, California, United States (subsequently affiliated with the School of Medicine, University of Galway, Ireland).
• Michael E. McCauley, PhD — Research Professor of Operations Research, Department of Operations Research, Naval Postgraduate School, Monterey, California, United States.

4. Purpose

The primary purpose of the Nontechnical skills taxonomy for Officers of the Deck is to provide a structured, objective, and psychometrically robust behavioral marker system designed to observe, evaluate, and debrief nontechnical performance on the navigational bridge of military surface vessels. Historically, naval bridge watchstanding evaluation has disproportionately emphasized technical qualifications, navigation rules of the road (COLREGs), radar interpretation, maneuvering characteristics, and mechanical propulsion oversight. However, formal maritime accident investigations conducted by the U.S. Navy and the National Transportation Safety Board (NTSB) consistently demonstrate that catastrophic navigation errors, groundings, and bridge collisions rarely stem from purely technical equipment failures; rather, they overwhelmingly originate from human nontechnical breakdowns, including degraded situational awareness, dysfunctional bridge communications, command hierarchy intimidation, and inadequate stress inoculation.

Recognizing that nontechnical skills (defined as the cognitive, social, and personal resource skills that complement technical skills to contribute to safe and efficient task performance) are critical determinants of maritime survivability, the NTSOD was engineered to fulfill three distinct operational and psychological objectives:

1. Standardized Diagnostic Assessment: To replace idiosyncratic, subjective instructor evaluations with an explicit behavioral rubric that anchors performance against observable, observable maritime actions rather than vague personality impressions or hindsight bias.
2. Bridge Resource Management (BRM) Instruction: To furnish military and civilian maritime training institutions (such as the Surface Warfare Officers School [SWOS]) with a diagnostic taxonomy capable of pinpointing granular developmental deficits during high-fidelity bridge simulator evolutions, man-overboard drills, restricted water transits, and formation steaming.
3. Systemic Error Mitigation: To operationalize high reliability organizational practices within naval watch teams, ensuring that watchstanders cultivate psychological safety, maintain multi-layered situational models, and balance procedural compliance with adaptive, recognition-primed decision execution under combat and high-density shipping stress.

Beyond military surface warfare, the NTSOD framework holds substantial utility for commercial maritime operations, merchant marine training academies, oil tanker navigation crews, and autonomous vessel supervisory control architectures where human monitoring and supervisory override demand heightened nontechnical vigilance.

5. Psychological Construct

The psychological construct underpinning the NTSOD framework conceptualizes nontechnical competence not as an immutable personality trait, but as a dynamic constellation of observable cognitive, communicative, and managerial behaviors executed under operational constraints. The taxonomy partitions these behaviors into four overarching categories, encompassing ten distinct behavioral elements:

Leadership

Leadership on the naval bridge encompasses social coordination, resource allocation, authority maintenance, and emotional equilibrium during normal steaming and acute crises. It is partitioned into two core elements:

• Managing Watch Team: The capacity of the OOD to organize, supervise, and direct the bridge team (including the Junior Officer of the Deck [JOOD], Conning Officer, Quartermaster of the Watch [QMOW], and Boatswain’s Mate of the Watch [BMOW]). This involves delegating responsibilities clearly, avoiding cognitive overload across watchstanders, fostering an atmosphere where subordinates feel empowered to challenge erroneous navigation assumptions, and maintaining watch team cohesion.
• Coping with Stress: The cognitive and emotional self-regulation required to function effectively amidst physiological fatigue, combat alarms, emergency casualties, high traffic density, or adverse meteorological conditions. Watchstanders demonstrating superior stress coping maintain calm vocal tone, prevent tunnel vision (cognitive narrowing), and resist emotional outbursts that paralyze watch team communication.

Communications

Communications within the NTSOD framework are viewed through the lens of closed-loop information processing, psychological transmission accuracy, and hierarchical clarity. It consists of two behavioral elements:

• Providing Information: Proactive, unambiguous transmission of critical tactical, navigational, and environmental data to all relevant entities (including the Commanding Officer, Tactical Action Officer [TAO], radar watchstanders, and nearby vessels). This requires concise phrasing, timely updates during evolving crossing situations, and explicit verification that messages have been acknowledged and comprehended.
• Issuing Orders: Direct, firm, and doctrinally standard delivery of helm, engine, and course commands. Effective execution demands authoritative cadence, standardized standard naval maritime phrasing, rapid correction of erroneous read-backs, and the elimination of ambiguity in rudder angle or engine order executions.

Situational Awareness

Situational awareness within the maritime watch environment is conceptualized using dynamic mental modeling across three cognitive phases:

• Gathering Awareness: Active, continuous scanning of visual sightlines, radar repeaters, electronic chart displays (VMS/ECDIS), depth sounders, and lookout reports. It involves systematically cross-checking automated instrumentation with raw physical observations to prevent automation complacency.
• Understanding Awareness: Synthesizing disparate environmental inputs into an accurate, cohesive comprehension of the current tactical situation. This includes understanding the relative motion of target vessels, the impact of crosswinds and tidal currents on vessel drift, and the operational limitations imposed by navigation boundaries.
• Anticipating Future Events: The highest cognitive level of situational awareness, requiring forward projection of spatial-temporal positions. Watchstanders must anticipate closest point of approach (CPA), forecast future traffic bottlenecks, pre-plan collision avoidance maneuvers well before entering safety domains, and project weather pattern impacts on ship handling.

Decision Making

The decision-making construct captures the OOD’s ability to select and implement courses of action under uncertainty, temporal pressure, and risk. It comprises three interconnected elements:

• Analytical Decision Making: Deliberate, methodical evaluation of alternative navigational routes or tactical plans when time permits. This involves calculating turn radiuses, reviewing navigational charts, weighing fuel efficiency against transit timelines, and evaluating formal contingency options.
• Following Orders & Procedures: Strict fidelity to standard operating procedures (SOPs), the International Regulations for Preventing Collisions at Sea (COLREGs), and the Commanding Officer’s Standing Orders. It involves verifying parameter compliance (e.g., maximum allowable CPA boundaries) while recognizing the formal procedural boundaries within which the OOD operates.
• Intuitive Decision Making: Rapid, recognition-primed execution of decisive corrective action during sudden emergencies or high-stakes, time-compressed anomalies (e.g., steering casualty in a narrow channel or sudden appearance of an unlit craft). Watchstanders synthesize domain experience to identify familiar patterns and instantly deploy workable solutions without extensive comparative deliberation.

6. Theoretical Framework

The development and operationalization of the NTSOD taxonomy are anchored in three converging theoretical paradigms in applied cognitive psychology and industrial human factors: the European Civil Aviation Non-Technical Skills (NOTECHS) framework, Endsley’s three-level model of Situation Awareness, and Klein’s Recognition-Primed Decision (RPD) model.

The NOTECHS Paradigm and Crew Resource Management

Following the widespread adoption of Crew Resource Management (CRM) in commercial aviation, human factors researchers developed NOTECHS to assess European airline pilots across four dimensions: Co-operation, Leadership and Managerial Skills, Situation Awareness, and Decision Making. The NTSOD taxonomy directly adapts this behavioral marker architecture to the maritime domain. In naval contexts, Bridge Resource Management (BRM) mirrors CRM by seeking to mitigate single-point human failures on the ship bridge. The foundational assumption of the NOTECHS paradigm is that nontechnical competencies can be decomposed into an observable hierarchy of broad categories, sub-elements, and concrete behavioral markers (both exemplary and poor), thereby removing subjective assessor bias and providing actionable formative feedback.

Endsley’s Cognitive Model of Situation Awareness

The situational awareness dimension of the NTSOD is directly mapped onto Mica Endsley’s tripartite cognitive framework:

• Level 1 SA (Perception): Operationalized in the NTSOD as Gathering Awareness, involving the sensory detection of cues (vessel running lights, radar contacts, sound signals).
• Level 2 SA (Comprehension): Operationalized as Understanding Awareness, representing the integration of sensory cues with domain knowledge to determine the operational meaning of contacts (e.g., determining whether a contact represents a crossing or head-on risk).
• Level 3 SA (Projection): Operationalized as Anticipating Future Events, representing the temporal extrapolation of current vessel dynamics to forecast future states and avoid hazardous navigational conditions.

By disaggregating situational awareness into these three cognitive strata, the NTSOD enables instructors to pinpoint precisely where an OOD’s mental model deteriorated during critical bridge evolutions.

Naturalistic Decision Making and Recognition-Primed Decision (RPD) Theory

Naval surface warfare operations routinely occur in high-stress, ambiguous, and time-compressed environments that violate the foundational assumptions of classical normative decision theory (such as expected utility maximization). Accordingly, the NTSOD integrates Gary Klein’s Naturalistic Decision Making (NDM) perspective. While analytical decision making is preserved for pre-transit planning and deliberative maneuvers, the inclusion of Intuitive Decision Making directly reflects Klein’s RPD model. Expert OODs rely on extensive perceptual repertoires to rapidly recognize situational archetypes, conduct mental simulation of a single plausible course of action, and execute rapid maritime maneuvers without comparative evaluation of alternatives.

7. Validity

The validation process for the NTSOD behavioral marker system followed rigorous qualitative and psychometric workflows designed to establish comprehensive content, construct, and face validity within military surface warfare populations.

Content and Face Validity

Initial content formulation commenced with an exhaustive review of 17 nontechnical skill categories derived from civil aviation (NOTECHS), offshore drilling crew evaluations, medical anesthesia markers (ANTS), and commercial maritime bridge literature. To ensure high ecological validity in naval surface environments, an expert focus group comprising four qualified Surface Warfare Officers (SWOs) with extensive OOD watchstanding experience synthesized these candidate categories into a preliminary five-category, multi-element taxonomy tailored specifically to warship bridge dynamics.

Subsequently, content validity was empirically challenged and enriched using the Critical Incident Technique (CIT). In-depth, semi-structured interviews were conducted with 16 qualified OODs, eliciting detailed narratives of challenging, safety-critical, or high-risk watchstanding evolutions. Qualitative transcript analysis generated 149 discrete behavioral statements exemplifying positive and negative nontechnical performance. These statements were subjected to deductive classification against the emerging taxonomy. The comprehensive mapping of these operational incident statements confirmed that the behavioral markers adequately represented the full spectrum of actual bridge dilemmas, confirming robust content and face validity.

Construct and Structural Validity

Through three successive iterations of classification and semantic refinement, the taxonomy was streamlined from its original five categories into four highly coherent, non-overlapping functional domains: Leadership, Communications, Situational Awareness, and Decision Making. Redundant behavioral elements were consolidated, and operational definitions were harmonized to align with formal naval bridge doctrine and Surface Warfare Officers School operational vernacular. The resulting structural clarity demonstrated clear construct boundaries, ensuring that interpersonal coordination (Communications and Leadership) remained conceptually distinguishable from internal cognitive processes (Situational Awareness and Decision Making).

8. Reliability

Establishing satisfactory inter-rater reliability is the most critical psychometric hurdle for any behavioral marker rating tool, given that observer ratings in complex operational simulations are inherently susceptible to halo effects, leniency bias, and varying assessor experience levels.

Inter-Rater Reliability Calibration

During the developmental phase of the NTSOD taxonomy, inter-rater reliability was formally assessed by having two independent human factors experts code the 149 critical incident statements collected from operational OODs. In initial trials, divergent interpretations of borderline statements (e.g., distinguishing between communicating tactical intent and gathering situational awareness) revealed inadequate inter-coder concordance.

To resolve this, the authors implemented three systematic rounds of taxonomical revision and rater calibration:

• Iteration 1: Preliminary mapping resulted in modest agreement due to overlap between team leadership behaviors and communication protocols.
• Iteration 2: Operational definitions and behavioral markers were refined, adding explicit boundary criteria to distinguish between issuing orders (Communication) and managing the watch team (Leadership). Inter-rater agreement improved markedly.
• Iteration 3: Final structural adjustments yielded acceptable and robust levels of inter-rater reliability (achieving inter-coder agreement exceeding accepted Cohen’s kappa thresholds of κ > .70 across categories). Both raters demonstrated high concordance in categorizing complex multi-faceted behavioral incidents into discrete taxonomic elements.

Subsequent operational implementation of the NTSOD rating form in high-fidelity warship simulator environments highlighted that rater training (often termed Frame-of-Reference [FOR] training) is vital to maintaining inter-rater consistency. When naval instructors are systematically trained on the behavioral anchors and objective criteria of the 4-point rating scale, the system yields reliable, reproducible assessments of bridge watchstander proficiency.

9. Factor Analysis

Because the NTSOD taxonomy was constructed via inductive and deductive qualitative taxonomical engineering (combining the critical incident technique, focus groups, and iterative content coding) rather than large-scale questionnaire administration, traditional large-sample Exploratory Factor Analysis (EFA) or Confirmatory Factor Analysis (CFA) was not the primary vehicle of initial structural derivation. This methodology mirrors the development of seminal nontechnical marker systems, such as aviation NOTECHS and the Non-Technical Skills for Surgeons (NOTSS) system.

Nevertheless, the structural integrity of the 4-factor higher-order model (Leadership, Communications, Situational Awareness, Decision Making) has been subjected to conceptual factor mapping and qualitative structural validation:

• Latent Factor Distinctiveness: Qualitative factor separation is demonstrated by the low cross-classification of behavioral statements across orthogonal axes. Social/interpersonal performance (Factor 1: Leadership; Factor 2: Communications) separates cleanly from cognitive/individual processing (Factor 3: Situational Awareness; Factor 4: Decision Making).
• Hierarchical Element Loadings: In empirical simulation evaluations using analogous marker systems in maritime and high-reliability operations, individual behavioral elements exhibit strong latent factor saturation. For example, Gathering, Understanding, and Anticipating load heavily onto a unified higher-order Situational Awareness construct, while Managing Watch Team and Coping with Stress load cleanly onto Leadership.
• Covariance and Inter-Factor Dynamics: Consistent with cognitive human factors theory, high-order correlations are routinely observed between Situational Awareness and Decision Making. This dynamic reflects the reality that sound intuitive or analytical decisions are fundamentally contingent upon the maintenance of an accurate, up-to-date mental model of the navigational environment.

10. Instrument / Measurement Tool

The Nontechnical Skills for Officers of the Deck (NTSOD) is operationalized as an observer-based behavioral rating form designed for use by qualified naval instructors, bridge trainers, and commanding officers during live watch evolutions or high-fidelity bridge simulator training scenarios.

Tool Structure and Architecture

• Administrative Metadata: Records essential contextual parameters, including Ship Name/Hull Number, Trainee Name/Rank, Watch Designation (e.g., 0800-1200), Date, Trainer/Evaluator Name, and Operational Evolution (e.g., Underway Replenishment, Low Visibility Navigation, Transit of Straits, Open Ocean Steaming).
• Assessment Dimensions: Features 4 core behavioral categories encompassing 10 granular behavioral elements.
• Dual-Level Scoring Architecture: Instructors provide a numeric performance rating for each individual Element, which culminates in an overarching rating for the overarching Category, supplemented by narrative operational notes.
• Rating Scale Format: Evaluated on a 4-point behavioral anchor scale, with an additional option for unobserved actions:
  • 1 – Unsatisfactory: Watchstander performance is deficient, unsafe, or non-compliant with standard watch routines; could endanger ship and crew without considerable, immediate instructor intervention or correction.
  • 2 – Marginal: Watchstander exhibits inconsistent nontechnical competency; demonstrates basic procedural familiarity but requires frequent correction, exhibiting clear areas where improvement is needed.
  • 3 – Satisfactory: Watchstander performs at an acceptable, professional operational level; maintains safety and communication integrity, though minor room for refinement remains.
  • 4 – Outstanding: Watchstander performs at a consistently high, exemplary level under routine and crisis conditions; actively models optimal bridge resource management, proactive communication, and superior foresight.
  • N/O – Not Observed: The specific behavioral element was not relevant, required, or observable during the specific operational watch window or simulator scenario.
• Qualitative Debriefing Module: Structured qualitative field spaces within each category facilitate real-time behavioral note-taking, enabling instructors to record specific positive and negative operational markers to underpin structured post-watch debriefings.

11. Permissions & Fee and Test Year

The Nontechnical Skills for Officers of the Deck (NTSOD) rating system was developed in 2010 and published in the peer-reviewed literature in 2011 under the auspices of the Naval Postgraduate School (NPS) and the United States Navy. As a technical report and scholarly project produced by personnel of the U.S. Federal Government and academic researchers at a public military institution, the fundamental taxonomy and user guide exist within the public domain.

The instrument is freely accessible for scholarly research, military training, civilian maritime education, and clinical human factors safety applications without commercial licensing fees. Researchers and maritime organizations seeking to integrate or adapt the NTSOD rating form are expected to maintain formal academic attribution by citing the primary Naval Postgraduate School technical monograph (Long et al., 2010) and its accompanying peer-reviewed journal publication (O’Connor & Long, 2011). Organizations intending to integrate the taxonomy into proprietary commercial simulator software or automated evaluation packages should review standard institutional intellectual property guidelines via the Naval Postgraduate School’s Office of Research.

12. References

• Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
• Flin, R., Martin, L., Goeters, K.-M., Hörmann, H.-J., Amalberti, R., Valot, C., & Nijhuis, H. (2003). Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Human Factors and Aerospace Safety, 3(2), 95–117.
• Klein, G. A. (1998). Sources of power: How people make decisions. MIT Press.
• Long, W. M., O’Connor, P., & McCauley, M. E. (2010). Nontechnical skills for Officers of the Deck (NTSOD) rating form: A user’s guide (Technical Report NPS-OR-10-009). Naval Postgraduate School. https://core.ac.uk/download/pdf/36712390.pdf
• O’Connor, P., & Long, W. M. (2011). The development of a prototype behavioural marker system for U.S. Navy Officers of the Deck. Safety Science, 49(10), 1381–1387. https://doi.org/10.1016/j.ssci.2011.05.006
• Reason, J. (1990). Human error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062367
• Ystensen, O. M., Kongsvik, T., & Oltedal, H. (2023). Non-technical skills in bridge resource management: A systematic review of training and evaluation paradigms. WMU Journal of Maritime Affairs, 22(1), 45–71. https://doi.org/10.1007/s13437-022-00293-x

13. Items of the Scale