Feedback Instrument for Rescue Force Development – Tabletop Exercises (FIRE-PL)

Abstract

The Feedback Instrument for Rescue Force Development – Tabletop Exercises (German: Feedback-Instrument zur Rettungskräfte-Entwicklung – Planübung, abbreviated as FIRE-PL) is a specialized, psychometrically validated evaluation scale designed to measure the perceived educational quality and instructional effectiveness of tabletop simulation exercises (Planübungen) within incident command and leadership training for emergency rescue services. Originally developed through an institutional collaboration between the Institute of the North Rhine-Westphalia Fire Brigade (Institut der Feuerwehr Nordrhein-Westfalen; IdF NRW) and the Department of Organizational and Business Psychology at the University of Münster (WWU Münster), the instrument functions primarily as a modular extension to the overarching Feedback Instrument for Rescue Force Development (FIRE core scale; Schulte & Thielsch, 2019; Schulte et al., 2019). The FIRE-PL comprises four positively worded items that capture critical facets of simulation-based learning: perceived individual knowledge acquisition, instructor feedback utility, deliberate application of newly acquired command knowledge, and the appropriateness of the tactical exercise difficulty.

Administered using a 7-point Likert response format ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), along with an explicit opt-out category (“nicht sinnvoll beantwortbar” / not meaningfully answerable), the FIRE-PL requires approximately one minute of completion time, establishing exceptional procedural economy. The psychometric architecture of the instrument was established across three empirical validation studies encompassing over 400 career and volunteer fire service officers. Confirmatory factor analysis (CFA) employing robust maximum likelihood estimation (MLR) substantiated an essentially unidimensional factor structure with excellent global fit indices (χ²(2) = 4.72, p = .094, CFI = .99, TLI = .98, RMSEA = .07, SRMR = .02). Reliability assessments confirmed high internal consistency across independent samples, yielding Cronbach’s α values between .81 and .86 and McDonald’s hierarchical omega (ω) coefficients between .82 and .86, reflecting a congeneric measurement model. Construct validity was corroborated by substantial convergent correlations with core leadership training dimensions—specifically instructor behavior (r = .53), competence acquisition (r = .40), and transfer preparation (r = .41)—alongside divergent dissociation from transient mood states (r = .20) and cognitive pre-entry test scores (r = -.10, n.s.). Criterion validity was verified through pronounced associations with overall seminar satisfaction (r = .49) and comprehensive course grade assignment (r_s = .32). Consequently, the FIRE-PL constitutes a brief, robust, and theoretically grounded diagnostic instrument for quality assurance, didactic benchmarking, and formative evaluation in high-stakes emergency services training environments.

Keywords

FIRE-PL, tabletop exercises, Planübung, rescue force development, fire service leadership training, incident command simulation, educational evaluation, psychometrics, Kirkpatrick training evaluation, formative assessment

Authors

The Feedback Instrument for Rescue Force Development – Tabletop Exercises (FIRE-PL) was conceptualized, operationalized, and psychometrically validated by a specialized research consortium bridging academic applied psychology and professional civil protection education:

• Meinald T. Thielsch, Ph.D. — Professor and Senior Researcher at the Department of Psychology, Organizational and Business Psychology (OWMS), University of Münster (Westfälische Wilhelms-Universität Münster), Münster, Germany. His research focuses on organizational diagnostics, personnel training, digital user experience, and quality assurance within civil defense and emergency command organizations.
• Carolin Schulte, M.Sc. — Organizational Psychologist and Research Associate at the Department of Psychology, University of Münster, Münster, Germany. Co-lead in the development of the FIRE diagnostic battery for emergency command personnel.
• Institute of the North Rhine-Westphalia Fire Brigade (IdF NRW) — Institut der Feuerwehr Nordrhein-Westfalen, Münster, Germany. The largest state-level public firefighting and civil emergency defense training academy in Germany, responsible for educating unit leaders, battalion chiefs, and incident command staff across the state of North Rhine-Westphalia.

Institutional correspondence regarding the FIRE assessment battery can be directed to the Organizational and Business Psychology research group at the University of Münster (Project URL: https://www.uni-muenster.de/OWMS/bfo/projekte/fire/).

Purpose

In high-reliability organizations (HROs) such as municipal fire departments, regional rescue services, disaster management agencies, and civil defense forces, frontline leadership personnel operate under volatile, uncertain, complex, and ambiguous (VUCA) operating conditions. Incident commanders (e.g., Gruppenführer [crew commanders], Zugführer [platoon commanders], and higher-echelon command staff) are legally and operationally tasked with rapid, safety-critical decision-making under severe temporal pressure, acute threat, sensory overload, and information deficits. The structural backbone of incident command pedagogy in European fire services is governed by doctrinal frameworks such as the German Fire Service Regulation 100 (Feuerwehr-Dienstvorschrift 100: Führung und Leitung im Einsatz; FwDV 100, 1999). A pivotal instructional methodology utilized to cultivate command competence without the prohibitive resource expenditure and physical hazards of full-scale field operations is the tabletop exercise (German: Planübung).

Despite the ubiquitous implementation of tabletop simulations across international emergency services academies, educational quality management within this domain historically relied upon idiosyncratic, non-standardized participant feedback or generic higher education course evaluations. Such general-purpose teaching evaluation questionnaires suffer from substantial construct misalignment: they fail to capture the specific didactic mechanisms unique to synthetic simulation environments, such as exploratory reconnaissance on scaled terrain, cyclical command-decision formulation, verbalization of tactical situational pictures, instructor feedback loops, and calibrated task complexity. The FIRE-PL was deliberately engineered to resolve this diagnostic deficit.

The primary applied and scientific purposes of the FIRE-PL are multi-fold:

• Modular Didactic Diagnostics: To serve as an empirical adjunct module to the comprehensive FIRE core evaluation battery (Schulte & Thielsch, 2019), allowing academies to selectively measure the pedagogical quality of tabletop exercises whenever that specific method is deployed in a training curriculum.
• Formative Instructional Optimization: To furnish fire service instructors and simulation facilitators with highly targeted, actionable diagnostic feedback regarding whether their simulated emergency scenarios were perceived as excessively simplistic, excessively challenging, rich in cognitive knowledge gain, or supported by constructive debriefing and formative critique.
• Standardized Benchmarking: To enable horizontal comparisons between different instructional formats (e.g., classical sand-table models, 1:87 scale architectural dioramas, and emerging 2D/3D virtual tabletop simulations) as well as longitudinal quality tracking across recurrent cohorts of incident command courses.
• Evaluation Economy in High-Tempo Curricula: To minimize survey fatigue among adult professional learners by delivering a robust psychometric profile through a four-item instrument that requires merely 60 seconds to complete, thereby preserving instructional contact hours while maintaining scientific rigor.

Psychological Construct

The psychological construct operationalized by the FIRE-PL is the Perceived Quality of Tabletop Simulation Exercises (Perzipierte Lehrqualität von Planübungen) within high-stakes vocational command development. Unlike basic declarative classroom instruction, simulation-based training relies on experiential, social-cognitive, and metacognitive learning processes. The construct is conceived as an integrated, unidimensional pedagogical outcome reflecting how effectively a tabletop exercise bridges didactic knowledge acquisition, real-time command application, calibrated cognitive load, and feedback-driven behavioral adjustment.

The instrument operationalizes this construct across four interconnected didactic indicators, each reflected by an authentic manifest item:

1. Subjective Learning Gain (Subjektiver Lerngewinn)

Operationalized via Item 1 (“Ich habe bei den Planübungen viel gelernt” [I learned a lot during the tabletop exercises]), this facet measures the participant’s retrospective appraisal of individual cognitive enhancement. Rooted in classical course evaluation research (e.g., Hirschfeld & Thielsch, 2009), perceived learning gain reflects the subjective acquisition of novel tactical heuristics, operational schema expansion, and enhanced situational awareness. Rather than measuring rote memorization, this indicator captures whether the simulated emergency scenarios successfully enriched the commander’s internal repertoire of tactical responses.

2. Formative Instructor Feedback Utility (Nützlichkeit des Dozentenfeedbacks)

Operationalized via Item 2 (“Die Dozenten gaben mir nützliches Feedback zu meinen Leistungen in den Planübungen” [The instructors gave me useful feedback on my performance in the tabletop exercises]), this facet gauges the instructional quality of the facilitator’s debriefing. In tabletop simulation pedagogy, the instructor serves not merely as a lecturer, but as an active interlocutor, role-player (e.g., simulating dispatchers, adjacent operational units, or commanding officers), and post-exercise evaluator. Immediate, actionable, and psychologically safe corrective feedback is widely recognized as the single strongest pedagogical determinant of simulation efficacy (Hattie & Timperley, 2007; Salas et al., 2008).

3. Knowledge Application and Transfer of Learning (Wissenstransfer und Wissensanwendung)

Operationalized via Item 3 (“In den Planübungen konnte ich das neuerworbene Wissen anwenden” [In the tabletop exercises, I was able to apply the newly acquired knowledge]), this indicator evaluates experiential enactment. Theoretical command principles—such as assessing building construction collapse hazards, hazardous materials isolation zones, and resource calculation equations—remain abstract unless operationalized in an applied context. Item 3 captures whether the tabletop scenario afforded authentic opportunities for deliberate proceduralization: translating declarative doctrine into concrete command directives under dynamic simulated conditions.

4. Appropriateness of Cognitive Challenge / Task Difficulty (Angemessenheit des Schwierigkeitsgrades)

Operationalized via Item 4 (“Der Schwierigkeitsgrad der Planübungen war angemessen” [The difficulty level of the tabletop exercises was appropriate]), this facet measures instructional calibration against the learner’s zone of proximal development (Vygotsky, 1978) and cognitive load thresholds (Sweller, 2011). If a tabletop tactical scenario is trivial, experienced firefighters disengage; conversely, if the scenario introduces overwhelming compounding emergencies prematurely, cognitive overload ensues, paralyzing decision-making and impairing structural learning. An appropriate difficulty level signifies optimal balance: challenging the commander’s analytical capacities without inducing cognitive collapse.

Theoretical Framework

The conceptual architecture of the FIRE-PL is grounded in the intersection of cognitive psychology, adult instructional design, and naturalistic decision-making models specific to incident command.

1. Naturalistic Decision Making (NDM) and Recognition-Primed Decision (RPD) Theory

Classical decision theory presumes that decision-makers comprehensively list all viable alternatives, weight probabilities, and compute utility functions. In contrast, Gary Klein’s Recognition-Primed Decision (RPD) model demonstrates that experienced incident commanders under severe temporal constraints rely on perceptual pattern recognition. When confronted with an evolving blaze or structural collapse, commanders assess situational cues, instantly categorize the situation based on mental prototypes, mentally simulate a prospective course of action, and execute it if no fatal flaws are visualized.

Tabletop exercises function as artificial cognitive engines designed to expand an officer’s catalog of mental prototypes. In the German fire service model, exercises utilize physical 1:87 scale architectural dioramas (representing industrial complexes, rail catastrophes, residential multi-story fires, or hazardous materials spills) or high-resolution simulated maps. According to NDM theory, the pedagogical value of such tabletop simulations depends entirely on whether the commander is forced to extract environmental cues, verbally formulate a continuous “situational picture” (Lagebild), and mentally simulate operational interventions.

2. The Three-Phase Command Cycle: Feuerwehr-Dienstvorschrift 100 (FwDV 100)

The structural execution of the Planübung is rigorously anchored in the German standardized command sequence (Führungsvorgang; FwDV 100, 1999; IdF NRW, 2021), a continuous three-phase cyclic heuristic comprising:

1. Reconnaissance (Erkundung): The exercising commander confronts the abstracted 1:87 model and interrogates the instructor to extract vital diagnostic data (smoke propagation, wind direction, building occupancy, hydrant network availability). The commander must verbalize every observation to establish situational consensus across the unit.
2. Planning and Tactical Assessment (Planung): The commander synthesizes the reconnaissance data, conducts a hazard analysis (Gefahrenmatrix), weighs alternative courses of action, and selects an operational strategy. In tabletop pedagogy, this is often executed as an interactive tactical discourse before the cohort.
3. Issuance of Orders (Befehlsausgabe): The commander formulates unambiguous, standardized tactical commands directed to simulated subordinate units (e.g., designating staging areas, water supply lines, rescue assignments, and ventilation tactics).

Following this sequence, the instructor conducts an extensive post-exercise debriefing (Planbesprechung). The FIRE-PL was constructed specifically to capture whether learners perceive this structural sequence as fostering authentic tactical synthesis rather than detached theoretical exercise.

3. Cognitive Load Theory and Experiential Learning

From an educational psychology perspective, the FIRE-PL draws upon Cognitive Load Theory (Sweller, 2011) and Kolb’s Experiential Learning Theory. Full-scale field exercises (Einsatzübungen) introduce intense physical stress, acoustic noise, thermal load, and logistical overhead. Tabletop exercises isolate the cognitive and tactical dimensions of incident command, stripping away peripheral physical noise to allow focused deliberate practice on decision logic. The FIRE-PL monitors whether the tabletop design maintains germane cognitive load—maximizing the cognitive resources allocated to learning schemata—while preventing extraneous load stemming from confusing exercise design or unbalanced tactical difficulty.

Validity

The validity of the FIRE-PL was comprehensively evaluated through content, construct (convergent and discriminant), and criterion-related validation protocols across multiple independent cohorts at the IdF NRW.

1. Content Validity

Initial content validation was executed in Study I (late 2015) involving an expert panel of N = 33 individuals, comprising 7 veteran fire service instructors (86% male; mean age M = 38.9, SD = 6.0) and 26 officer candidates enrolled in group leader (Gruppenführer) qualification courses (96% male; mean age M = 30.5, SD = 6.9). Panelists rated each prospective item regarding semantic clarity, unambiguous phrasing, and domain relevance. Over 90% of all participants judged the tabletop items as “important” or “highly important” for training quality, and less than 3% identified any phrasing ambiguities. Consequently, all four items were retained in their original theoretical formulation without requiring semantic revision.

2. Convergent Construct Validity

Convergent validity was evaluated in Study III (N = 288 fire service commanders) by correlating the FIRE-PL score with validated subscales of the FIRE core questionnaire (Schulte & Thielsch, 2019). Because tabletop exercises fundamentally depend upon facilitator coaching and aim to bridge classroom theory with practical command capability, substantial positive correlations were hypothesized with the process dimension of instructor behavior and the outcome dimensions of competence acquisition and transfer preparation. Empirical correlations confirmed these hypotheses:

• Instructor Behavior (Dozentenverhalten): r = .53, p < .001 (n = 281), reflecting the pivotal role of instructor feedback, scaffolding, and tactical critique during tabletop debriefings.
• Competence Acquisition (Kompetenzerwerb): r = .40, p < .001 (n = 283), demonstrating that positive appraisals of tabletop simulations align with general perceived learning gains across the leadership curriculum.
• Preparation and Transfer (Vorbereitung & Transfer): r = .41, p < .001 (n = 282), indicating that participants who rate tabletop exercises highly perceive themselves as significantly better prepared to execute command duties in real-world emergencies.

3. Discriminant Construct Validity and Control for Biases

Discriminant validity was verified against constructs theoretically distant from tabletop instructional quality, as well as known evaluation bias variables:

• Group Dynamics (Gruppe): The correlation between the FIRE-PL and the FIRE core subscale measuring peer group climate was markedly lower (r = .25, p < .001, n = 286) than correlations with instructional and transfer scales, confirming that the tool specifically measures tabletop simulation quality rather than general cohort social cohesion.
• Participant Mood State (Stimmung): Evaluator mood was assessed using Jäger’s (2004) five-point smiley affective scale. Mood exhibited only a modest correlation with FIRE-PL ratings (r = .20, p < .001, n = 282), demonstrating that ratings reflect objective didactic quality rather than subjective emotional valence.
• Pre-Entry Cognitive Examination Score (Punktzahl Eingangsprüfung): Correlation with objective cognitive pre-test scores was negligible and non-significant (r = -.10, p = .21, n = 158), verifying that evaluations of tabletop exercises are unbiased by baseline academic ability.
• Career Firefighter Status (Erfahrung in der Berufsfeuerwehr): Dichotomous comparison between professional career firefighters and volunteer personnel revealed no significant relationship (r = -.04, p = .50, n = 283), confirming that the instrument functions equivalently across professional and voluntary branches of civil protection.

4. Criterion-Related Validity

Criterion validity was established against overarching educational evaluation criteria:

• Overall Course Satisfaction (Gesamtzufriedenheit): Measured via an established four-item composite satisfaction scale (Gläßer et al., 2002), yielding a strong correlation of r = .49, p < .001 (n = 280).
• Overall Course Academic Grade (Benotung des Seminars): Participants assigned an overall academic school grade (German grading scale 1–6, reverse-coded so higher values indicate superior performance). Spearman rank correlation demonstrated a robust association: r_s = .32, p < .001 (n = 284).

Reliability

The reliability of the FIRE-PL has been extensively scrutinized across independent training cohorts utilizing classical and modern psychometric metrics. Because factor analytic investigations indicated that manifest item loadings and error variances are not strictly identical, the underlying measurement model satisfies congeneric rather than essentially tau-equivalent assumptions. Consequently, while Cronbach’s alpha (α) is reported for historical comparability, McDonald’s hierarchical omega (ω) represents the unbiased, psychometrically superior indicator of internal consistency.

Empirical Reliability Findings

Reliability analyses conducted in Study II (N = 155) and Study III (N = 288) demonstrated consistently high internal consistency:

• Study II Sample (N = 155): Cronbach’s α = .86; McDonald’s ω = .86. A formal chi-square difference test comparing a congeneric model against an essentially tau-equivalent model indicated significant superiority for the congeneric specification (Δχ²(3) = 9.38, p = .025).
• Study III Sample (N = 288): Cronbach’s α = .81; McDonald’s ω = .82. The chi-square difference test again confirmed congeneric superiority over tau-equivalence (Δχ²(3) = 14.53, p = .002).

Item Discrimination and Manifest Metrics

Across the validation samples, corrected item-total correlations (Trennschärfe) ranged from .61 to .84, well above the conventional psychometric threshold of .30, demonstrating that every item contributes meaningfully to the common core construct:

• Item 1 (Learning Gain): Mean M = 5.85, SD = 0.86, Corrected Item-Total Correlation r_it = .61, Skewness = -0.80, Kurtosis = 1.11.
• Item 2 (Instructor Feedback): Mean M = 5.79, SD = 1.05, Corrected Item-Total Correlation r_it = .64, Skewness = -1.40, Kurtosis = 3.23.
• Item 3 (Knowledge Application): Mean M = 5.93, SD = 0.81, Corrected Item-Total Correlation r_it = .84, Skewness = -0.92, Kurtosis = 1.30.
• Item 4 (Appropriate Difficulty): Mean M = 5.89, SD = 0.84, Corrected Item-Total Correlation r_it = .73, Skewness = -1.43, Kurtosis = 4.28.

The high item discrimination parameters, especially for Item 3 (.84) and Item 4 (.73), demonstrate that the operationalization of applied knowledge transfer and calibrated task challenge forms a remarkably cohesive measurement core.

Factor Analysis

The factorial validity and latent dimensionality of the FIRE-PL were investigated through sequential exploratory and confirmatory factor analyses using RStudio (version 1.2.5001) with the psych (Revelle, 2021) and lavaan (Rosseel, 2012) statistical packages.

1. Exploratory Factor Analysis (Study II)

In Study II, an initial pool of teaching method evaluation items—encompassing self-directed learning (EVA), group work (G), tabletop exercises (PL), and full-scale operational field exercises (E)—was administered to N = 155 course participants. An oblimin-rotated principal component analysis / exploratory factor analysis (EFA) was performed. The four tabletop exercise items clustered unequivocally on a distinct latent factor (F2), exhibiting exceptionally strong standardized factor loadings:

• Item PL_1: Factor loading λ = .70 (communality h² = .61)
• Item PL_2: Factor loading λ = .85 (communality h² = .73)
• Item PL_3: Factor loading λ = .84 (communality h² = .79)
• Item PL_4: Factor loading λ = .88 (communality h² = .73)

None of the tabletop items displayed cross-loadings greater than .20 on adjacent factors representing group work or self-directed learning, establishing pristine structural independence from other instructional modalities.

2. Confirmatory Factor Analysis (Study III)

In Study III, the hypothesized single-factor structure was formally evaluated using Confirmatory Factor Analysis (CFA) on a fresh validation cohort of N = 288 fire service commanders. Given mild negative skewness and leptokurtic distribution typical of high-quality vocational education evaluations, robust maximum likelihood estimation (MLR) was implemented. The unidimensional model demonstrated outstanding goodness-of-fit based on the strict methodological standards articulated by Hu and Bentler (1999):

• Robust Chi-Square: χ²(2) = 4.72, p = .094
• Comparative Fit Index (CFI): .99
• Tucker-Lewis Index (TLI): .98
• Root Mean Square Error of Approximation (RMSEA): .07, 90% CI [.00, .15]
• Standardized Root Mean Square Residual (SRMR): .02

All standardized latent factor loadings in the CFA were uniform, robust, and highly statistically significant (λ ≥ .60, p < .001). The convergence across both exploratory and confirmatory modeling firmly substantiates the theoretical presumption of an essentially unidimensional construct.

Instrument / Measurement Tool

The FIRE-PL is structured as follows:

• Test Type: Standardized self-report evaluation questionnaire / modular educational diagnostic scale.
• Target Population: Emergency rescue forces, incident commanders, fire officers, disaster management personnel, and command staff undergoing tabletop simulation training.
• Administration Format: Standard Paper & Pencil survey administration or adaptive computerized/online survey administration.
• Item Count: 4 manifest items (all positively keyed; no reverse scoring required).
• Estimated Completion Time: Approximately 1 minute (60 seconds).
• Authentic Response Scale: Siebenstufiges Antwortformat mit den Optionen 1 = stimme gar nicht zu, 2 = stimme nicht zu, 3 = stimme eher nicht zu, 4 = neutral, 5 = stimme eher zu, 6 = stimme zu, 7 = stimme vollkommen zu. Ein zusätzliches Feld bietet die Möglichkeit anzugeben, dass das jeweilige Item nicht sinnvoll beantwortbar sei.
• Standard Administration Instructions (Stand-Alone Module):
  

  “Liebe/r Lehrgangsteilnehmer/in, wir danken Ihnen, dass Sie an der Befragung teilnehmen! Indem Sie diese Planübung bewerten, helfen Sie uns, die Qualität der Lehre in der Rettungskräfteausbildung zu beurteilen und gegebenenfalls zu verbessern. Bitte geben Sie an, wie sehr Sie den untenstehenden Aussagen zustimmen. Kreuzen Sie für jede Aussage das Kästchen an, das den Grad Ihrer Zustimmung am besten wiedergibt. Es gibt bei dieser Befragung keine richtigen oder falschen Antworten. Vielmehr interessieren wir uns für Ihre ganz persönliche Meinung. Bitte beachten Sie: Machen Sie hinter jeder Aussage jeweils nur ein Kreuz in einem der vorgesehenen Kästchen. Bitte lassen Sie keine Aussagen aus. Wenn eine Aussage für Sie nicht sinnvoll beantwortbar ist, können Sie uns dies durch ein Kreuz in dem entsprechenden Kästchen mitteilen. Wenn Sie ein Kreuz ändern möchten, malen Sie das falsch markierte Kästchen vollständig aus und machen ein neues Kreuz an der gewünschten Stelle. Ihre Teilnahme an der Befragung ist freiwillig. Zudem erfolgt die Befragung selbstverständlich anonym. Die Ergebnisse werden nur in gesammelter Form, das heißt beispielsweise in Form von Mittelwerten zurückgemeldet. Ein Rückschluss auf Ihre Person ist damit ausgeschlossen.”

  (Note: When appended as a supplementary module to the broader FIRE core survey, this standalone instruction is omitted to streamline testing).

• Scoring and Computational Rules:
  1. Point Allocation: Quantitative response options are scored numerically: 1 = 1 point, 2 = 2 points, 3 = 3 points, 4 = 4 points, 5 = 5 points, 6 = 6 points, 7 = 7 points. Responses marking “nicht sinnvoll beantwortbar” (not meaningfully answerable) receive no points and are coded as missing data for that individual.
  2. Missing Data Protocol: In paper administrations, any questionnaire with two or more omitted tabletop items is excluded from analysis. In online surveys, mandatory item response parameters are recommended. High frequencies of missing responses across a cohort suggest a qualitative mismatch between the questionnaire and the specific instructional format.
  3. Individual Item Means: For each item, compute the unweighted arithmetic mean across all responding participants (sum of item scores divided by the number of valid respondents).
  4. Composite Scale Mean: Calculate the overall FIRE-PL scale mean by summing the four unweighted item means and dividing by 4:
     
     FIRE-PL Scale Score = (Mean Item 1 + Mean Item 2 + Mean Item 3 + Mean Item 4) / 4
• Anonymity and Threshold Guidelines: To safeguard respondent anonymity in hierarchical paramilitary organizations, individual demographic identifiers should not be collected alongside ratings. Furthermore, statistical aggregation should only be calculated if at least N = 8 completed forms are available (or in small seminars of 10–14 individuals, a minimum participation rate of 50%; Thielsch & Weltzin, 2013).
• Normative Reference Values: In a combined pooled sample of fire service commanders (N = 443; 95.0% male, mean age M = 33.72, SD = 6.88, mean operational experience 14.85 years), the overall benchmark mean was M = 5.85 (SD = 0.75, Skewness = -1.13, Kurtosis = 3.35). Scale values approaching 6.00 to 7.00 reflect high instructional quality, whereas mean values dropping below 5.00 signify substantial didactic deficits warranting pedagogical review.

Permissions & Fee and Test Year

The Feedback Instrument for Rescue Force Development – Tabletop Exercises was formally standardized and validated between 2015 and 2017, with complete psychometric documentation published in 2019 and 2020. The instrument is classified as an open-access psychometric measurement tool for academic, non-commercial educational, and organizational quality assurance purposes. No licensing fees, per-administration royalties, or institutional purchase charges are required for training academies, fire departments, or civil protection organizations utilizing the scale for internal course evaluation.

Researchers and training administrators are permitted to administer the items in paper-and-pencil or online software environments, provided that authorship is acknowledged and proper scientific citation is maintained. Detailed documentation and access to related modules (such as FIRE-B for basic training, FIRE-E for field exercises, and modules for command post exercises) are maintained via the University of Münster Organizational and Business Psychology project platform (https://www.uni-muenster.de/OWMS/bfo/projekte/fire/). Commercial redistribution or inclusion in proprietary paid software suites requires formal authorization from the copyright holders.

References

• Careless, L. (2007). The utility of tabletop exercises in emergency management: A systematic evaluation. Journal of Homeland Security and Emergency Management, 4(2), 1–18. https://doi.org/10.2202/1547-7355.1275
• Chi, M. T., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471–533. https://doi.org/10.1207/s15516709cog2504_1
• Federal Emergency Management Agency. (2020). Homeland Security Exercise and Evaluation Program (HSEEP). U.S. Department of Homeland Security. https://www.fema.gov/emergency-managers/national-preparedness/exercises/hseep
• Feuerwehr-Dienstvorschrift 100. (1999). Führung und Leitung im Einsatz – FwDV 100. Ausschuss Feuerwehrangelegenheiten, Katastrophenschutz und zivile Verteidigung (AFKzV). Kohlhammer.
• Gläßer, D., Hofer, M., & Schilling, E. (2002). Messung von Zufriedenheit mit Weiterbildungsseminaren. Zeitschrift für Pädagogische Psychologie, 16(2), 105–115. https://doi.org/10.1024//1010-0652.16.2.105
• Hagemann, V. (2011). Beanspruchung und Leistung in High-Reliability-Organisationen: Trainingsansätze zur Bewältigung komplexer Einsatzlagen. Verlag Dr. Kovač.
• Hannah, S. T., Uhl-Bien, M., Avolio, B. J., & Cavarretta, F. L. (2009). A framework for examining leadership in extreme contexts. The Leadership Quarterly, 20(6), 897–919. https://doi.org/10.1016/j.leaqua.2009.09.006
• Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
• Hirschfeld, G., & Thielsch, M. T. (2009). Lehrveranstaltungsevaluation an Hochschulen: Zur Konstruktvalidität studentischer Urteile. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 41(4), 193–203. https://doi.org/10.1026/0049-8637.41.4.193
• Holloway, R. A. (2007). Tabletop exercises: An economical tool for incident command training. Fire Engineering, 160(8), 75–82.
• Hu, L. t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
• Institut der Feuerwehr Nordrhein-Westfalen. (2021). Planübung / Planbesprechung in der Zugführerausbildung (Lernunterlage B2-305). IdF NRW.
• Jäger, R. S. (2004). Von der Beobachtung zur Notengebung: Ein praxisorientiertes Handbuch zur Leistungsbeurteilung. Empirische Pädagogik.
• Niemann, F. J., & Thielsch, M. T. (2020). Evaluation von Grundausbildungslehrgängen im Rettungswesen mit dem FIRE-B. Notfall + Rettungsmedizin, 23(7), 530–538. https://doi.org/10.1007/s10049-019-00662-7
• Revelle, W. (2021). psych: Procedures for Psychological, Psychometric, and Personality Research (R package version 2.1.9). Northwestern University. https://CRAN.R-project.org/package=psych
• Röseler, S., Schulte, C., & Thielsch, M. T. (2020). Feedback-Instrument zur Rettungskräfte-Entwicklung – Einsatzübung (FIRE-E): Ein Instrument zur Evaluation praktischer Einsatzübungen. Zeitschrift für Arbeits- und Organisationspsychologie, 64(3), 185–197. https://doi.org/10.1026/0932-4089/a000325
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
• Salas, E., DiazGranados, D., Klein, C., Guzmán, E., Burke, C. S., Stagl, K. C., Goodwin, G. F., & Halpin, S. M. (2008). Does team training improve team performance? A meta-analysis. Human Factors, 50(6), 903–933. https://doi.org/10.1518/001872008X375009
• Schulte, C., & Thielsch, M. T. (2019). Evaluation von Führungskräftelehrgängen bei der Feuerwehr: Entwicklung und Validierung des Feedback-Instruments zur Rettungskräfte-Entwicklung (FIRE). Zeitschrift für Arbeits- und Organisationspsychologie, 63(3), 133–148. https://doi.org/10.1026/0932-4089/a000298
• Schulte, C., Röseler, S., & Thielsch, M. T. (2019). Feedback-Instrument zur Rettungskräfte-Entwicklung (FIRE): Manual und Skalendokumentation. Westfälische Wilhelms-Universität Münster. https://doi.org/10.17877/DE290R-20229
• Sjöberg, M., Wallenius, C., & Larsson, G. (2006). Leadership in military operations: Overcoming emotional challenges. Military Psychology, 18(sup1), S85–S101. https://doi.org/10.1207/s15327876mp1803s_7
• Smith, T. D., & Dyal, M. A. (2016). Examining the relationship between safety climate and firefighter injury. Journal of Safety Research, 59, 39–45. https://doi.org/10.1016/j.jsr.2016.10.001
• Spiel, C. (2001). Qualitätssicherung und Qualitätsmanagement an Schulen: Ein theoretischer Rahmen. Zeitschrift für Pädagogische Psychologie, 15(3/4), 127–134. https://doi.org/10.1024//1010-0652.15.34.127
• Sweller, J. (2011). Cognitive load theory. In J. P. Mestre & B. H. Ross (Eds.), The Psychology of Learning and Motivation: Cognition in Education (Vol. 55, pp. 37–76). Academic Press. https://doi.org/10.1016/B978-0-12-387693-5.00002-8
• Thielsch, M. T., & Hadzihalilovic, D. (2020). Evaluation von Stabsrahmenübungen im Katastrophenschutz: Anpassung und Validierung der FIRE-Stabsbatterie. Bevölkerungsschutz, 4, 28–32.
• Thielsch, M. T., & Weltzin, S. (2013). Lehrevaluation an Hochschulen: Praxisleitfaden für Dozierende und Qualitätsmanager. Universität Münster.
• Useem, M., Cook, J. R., & Sutton, L. (2005). Developing leaders for decision-making under catastrophic conditions: The Wildland Fire Service model. MIT Sloan Management Review, 47(1), 46–59.
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
• Young, K. M., & Cooper, C. L. (1995). Occupational stress in the ambulance service: A review. Stress Medicine, 11(4), 253–264. https://doi.org/10.1002/smi.2460110406

Items of the Scale