Abstract
The Operating Theatre Human Factors Questionnaire (OTHFQ) is a specialized psychometric and evaluative instrument designed to assess non-technical skills, safety attitudes, and human factors knowledge among surgical personnel. Developed by Paul O’Connor, Ivan Keogh, and Stephen Ryan, the OTHFQ was adapted from high-reliability aviation frameworks—specifically the Naval Aviator Human Factors Questionnaire—to address non-technical failures in perioperative medicine. The instrument comprises two distinct operational components: a 25-item attitude inventory and an 8-item objective knowledge assessment, alongside standard demographic indices. The attitude section measures four primary subscales: My Stress, Stress of Others, Communication, and Command Responsibility, supplemented by three contextual items assessing perceived psychological safety (junior staff apprehension to speak up) and the institutional execution of pre- and post-operative surgical briefings. The objective knowledge section evaluates comprehension of core human factors concepts, including situation awareness, stress-performance relationships, decision-making, and communication strategies during acute crises. Psychometric investigations across operating theatre personnel demonstrate subscale internal consistency estimates (Cronbach’s alpha) ranging from 0.41 to 0.66, aligning with established cross-industry Crew Resource Management (CRM) attitudinal surveys. The OTHFQ serves as an empirical instrument for diagnostic safety culture audits, surgical training curriculum design, and the institutional evaluation of surgical team coordination.
Keywords
Operating Theatre Human Factors Questionnaire, human factors, Crew Resource Management, surgical safety, non-technical skills, situational awareness, command responsibility, team communication, psychological safety, patient safety.
Authors
The Operating Theatre Human Factors Questionnaire was developed and validated by an interdisciplinary team of researchers and clinical practitioners specializing in human performance, psychology, and otolaryngology:
- Paul O’Connor, PhD: Specialist in human performance, human factors psychology, and high-reliability operations. Affiliated with the Department of General Practice, National University of Ireland, Galway, Ireland, and formerly associated with the Naval Postgraduate School and the U.S. Navy CRM research divisions.
- Ivan Keogh, MD, FRCSI: Consultant Otolaryngologist / Head and Neck Surgeon and clinical researcher affiliated with the Department of Surgery, University Hospital Galway and the National University of Ireland, Galway, Ireland.
- Stephen Ryan, MRCSI: Surgical specialist and clinical investigator affiliated with the Department of Otorhinolaryngology, University Hospital Galway, Ireland.
Purpose
The primary purpose of the Operating Theatre Human Factors Questionnaire (OTHFQ) is to provide a standardized, empirically grounded metric to measure perioperative clinicians’ attitudes toward non-technical skills (NTS) and quantify baseline knowledge regarding human performance limitations. While technical expertise, manual dexterity, and biomedical knowledge have historically defined surgical training, epidemiological analyses across global healthcare systems reveal that between 50% and 80% of adverse surgical events and perioperative sentinel occurrences stem from latent non-technical failures. These failures include poor communication, hierarchical intimidation, mismanaged workload, fatigue, and inadequate situational assessment (World Health Organization, 2009).
The OTHFQ fulfills three primary diagnostic and pedagogical roles:
- Baseline Cultural and Attitudinal Assessment: The tool quantifies shared operating room norms, illuminating differences between professional subcultures (e.g., attending surgeons vs. resident trainees, surgical staff vs. scrub nurses and anesthesiologists).
- Identification of Latent Safety Vulnerabilities: By evaluating perceptions of hierarchy, steep authority gradients, and the willingness of junior team members to challenge superiors during emergent threats, the scale identifies communication barriers that impair team coordination.
- Curriculum Design and Training Evaluation: The instrument facilitates pre- and post-intervention evaluations for surgical Crew Resource Management (CRM) workshops, crisis resource management simulation courses, and Non-Technical Skills for Surgeons (NOTSS) curricula.
Psychological Construct
The OTHFQ operationalizes human factors attitudes and theoretical concepts across four subscales, three contextual climate markers, and an objective knowledge dimension:
1. My Stress
This subscale captures personal stress vulnerability and self-awareness regarding physiological and psychological limitations. Drawing on foundational human performance models, it assesses whether clinicians acknowledge that acute stress, physical fatigue, and personal problems degrade cognitive bandwidth, decision-making quality, and manual coordination. High scores represent realistic self-assessment and rejection of the myth of clinical invulnerability, whereas low scores reflect hazardous attitudes characterized by denial of human cognitive limitations under emergency pressure.
2. Stress of Others
This dimension evaluates interpersonal awareness, social monitoring, and supportive interventions within the multidisciplinary surgical team. It assesses the degree to which an individual believes team members should actively observe colleagues for manifestations of acute fatigue, cognitive overload, or personal distress, and take proactive steps to mitigate these risks. It reflects a collective perspective on safety, viewing error detection as a shared responsibility rather than an individual burden.
3. Communication
The communication construct operationalizes information exchange, explicit verbalization of clinical intent, task delegation, and mutual workload management. It measures attitudes toward formal pre-operative briefings and post-operative debriefings, the explicit statement of procedural milestones, and transparency regarding personal workload limits. High scores indicate an understanding of closed-loop communication, collaborative mental models, and adaptive coordination in the operating room.
4. Command Responsibility
This subscale addresses authority gradients, leadership philosophy, and power distance within the operating theatre. It explores attitudes regarding whether leadership requires rigid authoritarian dominance or collaborative management. The dimension evaluates how authority should be exercised during routine versus emergent scenarios, whether junior members are expected to defer completely to senior staff, and whether challenge and advocacy are recognized as essential safety behaviors.
5. Contextual Safety Markers
Complementing the core subscales, three independent items assess organizational climate:
- Junior Staff Psychological Safety: Evaluates the perceived frequency with which junior personnel hesitate or fear to express professional disagreement with senior colleagues.
- Pre-Operative Briefing Frequency: Measures the perceived regularity of comprehensive safety briefings prior to surgical incision.
- Post-Operative Debriefing Frequency: Assesses the institutional adoption of reflective reviews following procedures to analyze team coordination and process vulnerabilities.
6. Human Factors Knowledge
Unlike purely subjective attitude inventories, the OTHFQ incorporates an objective knowledge metric evaluating the retention of core principles of human performance. This subscale measures understanding of the prevalence of systemic latent errors, cognitive breakdown etiologies under high workload, verbal assertiveness taxonomies (passive vs. assertive vs. aggressive), naturalistic decision-making paradigms in time-pressured settings, sleep hygiene, and the Yerkes-Dodson inverted-U stress-performance relationship.
Theoretical Framework
The OTHFQ is grounded in several foundational paradigms from industrial and organizational psychology, cognitive ergonomics, and high-reliability operations.
Latent Conditions → Active Failures → Defenses Pierced → Adverse Event
Figure: Conceptual flow of Reason’s Swiss Cheese Model operationalized in perioperative systems.
1. Crew Resource Management (CRM) and Threat and Error Management (TEM)
Originating from NASA and aviation psychology (Helmreich et al., 1999), CRM theoretical models assert that catastrophic system failures rarely stem from isolated technical incompetence. Instead, they arise from breakdowns in team interaction, loss of situation awareness, and uncoordinated responses to environmental stressors. The Threat and Error Management (TEM) framework conceptualizes the operating room as a dynamic risk environment where clinicians must detect external threats (e.g., anatomical anomalies, equipment malfunction) and trap internal human errors (e.g., miscommunication, dosage calculation slips) before clinical outcomes are compromised.
2. Reason’s Swiss Cheese Model of System Accidents
James Reason’s (1990) system accident model distinguishes between active failures (unsafe acts committed by practitioners at the sharp end) and latent conditions (flawed organizational hierarchies, normalization of deviance, inadequate briefing policies). The OTHFQ evaluates whether operating theatre personnel view errors through an individualistic, punitive framework or understand the interaction between organizational latent conditions, communication barriers, and downstream human failures.
3. Psychological Safety and Steep Authority Gradients
Drawing on Amy Edmondson’s work on team psychological safety, the OTHFQ operationalizes the interpersonal risks associated with speaking up in professional hierarchies. In high-power-distance surgical environments, steep authority gradients often prevent trainees, scrub nurses, and junior residents from voicing safety concerns, resulting in preventable adverse events. The OTHFQ measures the degree to which practitioners view cross-hierarchical questioning as an essential safety mechanism rather than insubordination.
Validity
Empirical assessment of the OTHFQ demonstrates adequate construct, content, and convergent validity across healthcare and high-reliability operational settings:
- Content Validity: Content adaptation of the original Naval Aviator Human Factors Questionnaire was conducted by an expert panel of consultant surgeons, otolaryngologists, and aviation human factors psychologists. Aviation-specific terminology was adjusted to reflect perioperative equivalents (e.g., changing “aircraft commander” to “senior person” or “team member in charge”, and “aircrew” to “theatre team member”). This process preserved construct representation while achieving clinical face validity.
- Construct and Comparative Validity: O’Connor, Keogh, and Ryan (2012) established construct validity through a comparative benchmark study matching Irish surgical cohorts against operational U.S. Naval aviators. The analysis revealed significant divergence in command responsibility and stress perception profiles. Surgeons reported substantially higher rates of junior staff fear of speaking up (with 48% of surgeons reporting that junior staff were frequently or sometimes afraid to express disagreement, compared to 15.6% among military aviators). Surgeons also exhibited significantly higher rates of clinical invulnerability, reflecting predictable cultural differences across the two domains and confirming the scale’s sensitivity to organizational subcultures.
- Convergent and Criterion Validity: Dimensions of the OTHFQ demonstrate structural alignment with established non-technical assessment systems, including the Safety Attitudes Questionnaire (SAQ), the Non-Technical Skills for Surgeons (NOTSS) behavior marker system, and the Operating Room Management Attitudes Questionnaire (ORMAQ). Elevated scores on the OTHFQ communication and command subscales correlate with structured briefing implementation and higher perceived institutional safety culture.
Reliability
The psychometric evaluation of internal consistency across the OTHFQ attitude dimensions yields coefficients that reflect the broad bandwidth and brief item sets characteristic of multi-faceted safety culture metrics:
- Internal Consistency: In initial surgical validation cohorts, Cronbach’s alpha coefficients across the primary attitude subscales ranged from 0.41 to 0.66:
- My Stress: α ≈ 0.61 – 0.66
- Stress of Others: α ≈ 0.52 – 0.58
- Communication: α ≈ 0.41 – 0.49
- Command Responsibility: α ≈ 0.48 – 0.56
- Psychometric Context: While these internal consistency coefficients fall below conventional thresholds for high-stakes individual diagnostics (α ≥ 0.80), they are comparable to results documented in foundational aviation and surgical CRM attitude inventories (such as the Cockpit Management Attitudes Questionnaire [CMAQ] and the Flight Management Attitudes Questionnaire [FMAQ], where alphas typically range from 0.45 to 0.65). These modest values stem from the heterogeneous facets captured within brief 3- to 6-item subscales and the complex, multi-component nature of safety culture constructs.
- Objective Knowledge Reliability: The 8-item multiple-choice human factors knowledge section functions as a criterion-referenced mastery test, evaluated via difficulty indices (item facility) and point-biserial discrimination coefficients rather than uniform tau-equivalent models of internal consistency.
Factor Analysis
The factor architecture of the OTHFQ attitude scale was derived from the theoretical structure of the Naval Aviator Human Factors Questionnaire and confirmed via exploratory factor analysis (EFA) using principal axis factoring with orthogonal (Varimax) and oblique (Promax) rotations:
- Dimensional Structure: Factor analytic investigations confirm a four-factor solution accounting for the theoretical distribution of the 22 core attitude statements. The factors represent:
- Factor 1: My Stress (Items assessing individual performance degradation under emergency pressure, fatigue recognition, and personal distress awareness).
- Factor 2: Stress of Others (Items indexing active peer monitoring, sensitivity to colleague strain, and obligatory disclosure of personal impairment).
- Factor 3: Communication & Briefing (Items reflecting the importance of pre-session briefs, structured post-case debriefs, workload verbalization, and non-technical skills parity with technical proficiency).
- Factor 4: Command Responsibility & Hierarchy (Items capturing authority steepness, whether juniors should challenge superiors during routine vs. emergency events, and delegation models).
- Factor Loadings: Salient item loadings across primary factor dimensions consistently exceed the 0.35 cutoff, with minimal secondary cross-loadings. Residual variances align with the documented multidimensionality of cross-hierarchical communication and professional attitudes in acute healthcare domains.
Instrument / Measurement Tool
The Operating Theatre Human Factors Questionnaire comprises 25 attitudinal items, 8 multiple-choice knowledge items, and a demographic background inventory:
- Tool Specifications:
- Administration Format: Paper-and-pencil self-report or computer-delivered survey.
- Time Required: Approximately 10–15 minutes.
- Target Respondents: Attending surgeons, surgical fellows, surgical residents, nurse anesthetists, scrub nurses, circulating nurses, and operating department practitioners (ODPs).
- Structural Composition:
- Part 1: Core Attitude Scale (Items 1–22): 5-point Likert response format: Disagree Strongly (A/1), Disagree Slightly (B/2), Neutral (C/3), Agree Slightly (D/4), Agree Strongly (E/5).
- Part 2: Climate & Practice Frequency Markers (Items 23–25): 5-point Likert response format: Very frequently (A), Frequently (B), Sometimes (C), Seldom (D), Very seldom (E).
- Part 3: Human Factors Knowledge Test (Questions 1–8): Multiple-choice and categorization items evaluating foundational concepts in human error, situational awareness, communication styles, crisis decision-making, sleep physiology, and stress effects.
- Part 4: Demographic Background Information: Professional role (Surgeon, Nurse, Anesthetist, Other), professional grade/seniority, and total years of operating room clinical experience.
- Scoring and Directionality:
- Attitude subscale scores are generated by calculating the mean or sum of items within each respective domain.
- Reverse coding is applied to authoritarian, hierarchical, or clinical invulnerability items (e.g., Item 2, “My decision making ability is as good in emergencies as it is in routine situations,” and Items 17, 20, 21, and 22 regarding junior deference) such that higher overall numerical scores consistently indicate safer, non-hierarchical, human-factors-congruent attitudes.
- Knowledge items are scored dichotomously (0 = Incorrect, 1 = Correct) and aggregated to produce a total knowledge mastery score (range: 0–8 points).
Permissions & Fee and Test Year
The Operating Theatre Human Factors Questionnaire was adapted and published in 2011/2012 by Paul O’Connor, Ivan Keogh, and Stephen Ryan following initial comparative research conducted through the National University of Ireland, Galway. The instrument is listed in public research indices, including the Measurement Instrument Database for the Social Sciences (MIDSS). It is accessible for non-commercial academic, clinical, educational, and institutional research purposes without licensing fees, provided formal academic attribution is maintained. Commercial use, integration into proprietary healthcare software platforms, or modified deployment in enterprise assessment suites requires explicit written authorization from the primary authors.
References
- Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383. https://doi.org/10.2307/2666999
- Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of Crew Resource Management training in commercial aviation. The International Journal of Aviation Psychology, 9(1), 19–32. https://doi.org/10.1207/s15327108ijap0901_2
- O’Connor, P., Keogh, I., & Ryan, S. (2012). A comparison of the teamwork attitudes and knowledge of Irish surgeons and U.S. Naval aviators. The Surgeon, 10(1), 23–28. https://doi.org/10.1016/j.surge.2011.01.007
- Reason, J. (1990). Human error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062367
- World Health Organization. (2009). WHO guidelines for safe surgery 2009: Safe surgery saves lives. World Health Organization. https://apps.who.int/iris/handle/10665/44185
- Yule, S., Flin, R., Paterson-Brown, S., & Maran, N. (2006). Development of a rating system for surgeon’s non-technical skills. Medical Education, 40(11), 1098–1104. https://doi.org/10.1111/j.1365-2929.2006.02610.x
Items of the Scale
Operating Theatre Human Factors Questionnaire
Part 1: Attitude Scale
Please answer the following items by using the response scale below in writing your response beside each item:
- A: Disagree Strongly
- B: Disagree Slightly
- C: Neutral
- D: Agree Slightly
- E: Agree Strongly
- I let other team members know when my workload is becoming (or is about to become) excessive.
- My decision making ability is as good in emergencies as it is in routine situations.
- I am more likely to make judgment errors in an emergency.
- A regular debriefing of procedures and decisions after a theatre session or shift is an important part of developing and maintaining effective team co-ordination.
- In critical situations, I rely on my superiors to tell me what to do.
- I am less effective when stressed or fatigued.
- If I perceive a problem with the management of a patient, I will speak up, regardless of who might be affected.
- The pre-session team briefing is important for safety and for effective team management.
- Team members should monitor each other for signs of stress or tiredness.
- Personal problems can adversely affect my performance.
- Team members should feel obligated to mention their own psychological stress or physical problems to other theatre personnel before or during a shift or assignment.
- Good communication and team coordination are as important as technical proficiency for patient safety.
- Effective team coordination requires team members to consider the personal work styles of others.
- Team members should alert others to their actual, or potential, work overload.
- The specific roles and responsibilities of team members in an emergency are identified during the pre-operation brief.
- Team members should be aware of, and sensitive to, the personal problems of other team members.
- Junior theatre team members should not question the decisions made by senior personnel in emergencies.
- The senior person, if available, should take over and make all decisions in life threatening emergencies.
- The team member in charge should verbalize plans for procedures or actions and should be sure that the information is understood and acknowledged by others.
- Team members should not question the decisions or actions of senior staff except when they threaten patient safety.
- There are no circumstances where a junior team member should assume control of patient management.
- Junior team members should not question the decisions made by senior personnel during routine situations.
Part 2: Climate and Practice Frequency Items
- How frequently are junior personnel afraid to express disagreement with more senior personnel?
- Very frequently
- Frequently
- Sometimes
- Seldom
- Very seldom
- How frequently are adequate pre-operation team briefs conducted?
- Very frequently
- Frequently
- Sometimes
- Seldom
- Very seldom
- How frequently are adequate post-operation team briefs conducted?
- Very frequently
- Frequently
- Sometimes
- Seldom
- Very seldom
Part 3: Human Factors Knowledge Test
Please answer the following to the best of your ability by circling or filling in your response:
- What percentage of sentinel events in healthcare are attributed to human error?
- 0-10%
- 20-30%
- 50-60%
- 80-90%
- What is the most common reason skilled personnel fail to obtain a good understanding of what is happening in a high workload situation?
- Data/information is unavailable
- Information/data is difficult to detect
- Memory loss
- Failure to monitor or observe
- You are working in a potentially high-risk situation with a more senior person. Identify whether each of the statements below is PASSIVE, which is ASSERTIVE, and which is AGGRESSIVE:
- “I think that perhaps this may not be the best thing to do.” [ ____________________ ]
- “There may be a better way to do this task. In my opinion we should do ….” [ ____________________ ]
- “There is no way we should do this.” [ ____________________ ]
- During an operation, something unexpected happens. There is no procedure available. You have seconds to make a decision to attempt to prevent a bad outcome, what is the best thing to do?
- Consider all of the possible options, and select the best.
- React the best you can to the situation based upon your experience.
- Do nothing.
- During a time limited, abnormal, situation what is the most effective team communication strategy?
- The team should talk a lot about what is happening in an attempt to solve a problem.
- Use the minimum amount of communication necessary in an attempt to solve the problem.
- Don’t talk. This distracts from thinking about the problem.
- The following are all characteristics of an effective pre-operation brief EXCEPT:
- Assigning of roles and responsibilities
- Rapid information dissemination
- Professional
- Involve input from all members of the team
- The normal need for sleep to maximize performance in a 24 hour period is:
- 1-3 hours
- 4-6 hours
- 7-9 hours
- 10-12 hours
- Which of the following statement is true regarding the relationship between stress and performance:
- Performance is optimized when an individual is experiencing no or a very little amount of stress.
- Performance is optimized when an individual is experiencing a moderate amount of stress.
- Performance is optimized when an individual is experiencing an excessive amount of stress.
- Stress has no effect on an individual’s performance.
Part 4: Background Information
- Position (Please circle the appropriate response):
Nurse Surgeon Anesthetist Other (please specify) ____________________ - What is your grade? ____________________
- How much experience do you have working in the Operating Room? __________ years