Feedback Tool for Rescue Force Development – Basic Training (FIRE-B)

1. Abstract

The Feedback Tool for Rescue Force Development – Basic Training (known in German as the Feedback-Instrument zur Rettungskräfte-Entwicklung – Basisausbildung, or FIRE-B) is a scientifically validated, multidimensional survey inventory engineered to assess the perceived pedagogical and organizational quality of foundational firefighter training programs at municipal, county, and district levels. Originating from an academic-applied collaboration between the Department of Organizational and Business Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster) and the Fire Service Institute of North Rhine-Westphalia (Institut der Feuerwehr Nordrhein-Westfalen; IdF NRW), the FIRE-B evaluates foundational courses for both career fire brigades (e.g., Brandmeisteranwärter) and voluntary fire services (e.g., Truppmann and Truppführer certifications).

The conceptual framework of the FIRE-B emerged from a rigorous multi-stage developmental program comprising qualitative critical incident analyses and comparative adaptations from university teaching evaluations and emergency service leadership assessment tools. Psychometrically established across two large-scale independent empirical investigations (total N = 708 emergency responders), the core instrument consists of 29 primary items structured across six distinct, correlated first-order dimensions: (1) Structure & Didactics (Struktur & Didaktik; 6 items), (2) Support & Encouragement (Unterstützung & Förderung; 5 items), (3) Group Dynamics & Cohesion (Gruppe; 3 items), (4) Practical Training Realism (Praxis; 3 items), (5) Materials & Facilities (Material & Räumlichkeiten; 5 items), and (6) Competence Acquisition (Kompetenzerwerb; 7 items). Four supplementary diagnostic items (evaluating instructional pace, global learning gain, traditional academic grading, and qualitative commentary) provide actionable institutional intelligence.

Scale responses are captured via a 7-point Likert scale ranging from 1 ("completely disagree") to 7 ("completely agree"), supplemented by a dedicated non-applicable response option. Psychometric analyses demonstrate robust internal consistency across all subscales, with Cronbach’s alpha ($lpha$) and McDonald’s omega ($\omega$) values consistently meeting or exceeding .76. Confirmatory factor analyses support the postulated six-factor structural taxonomy, while extensive validation demonstrates robust content, construct, convergent, and criterion validity against objective and self-reported performance markers. The FIRE-B represents an empirically validated benchmark for quality assurance and pedagogical continuous improvement in high-risk emergency responder instruction.

2. Keywords

FIRE-B, firefighter training evaluation, rescue forces, emergency responder education, psychometrics, training quality assessment, instructional didactics, competence acquisition, organizational psychology, vocational training evaluation.

3. Authors

The development, empirical validation, and psychometric operationalization of the FIRE-B were executed by research psychologists and emergency training specialists affiliated with the University of Münster and the IdF NRW:

• Prof. Dr. Meinald T. Thielsch — Professor of Organizational and Business Psychology, Institute of Psychology, University of Münster, Germany. Academic lead on emergency management evaluations, training diagnostics, and human-computer/workplace psychometrics.
• Henrike Streppel, M.Sc. — Department of Psychology, University of Münster, Germany. Co-investigator on item generation, dimensional structuring, and foundational training quality criteria (Streppel, 2018).
• Lennart Kläpker, M.Sc. — Department of Psychology, University of Münster, Germany. Lead analyst on construct validation, structural equation modeling, and scale refinement (Kläpker, 2018).
• Fabian Schulte, M.Sc. — Department of Psychology, University of Münster, Germany. Co-creator of the foundational leadership evaluation instrument FIRE (Feedback-Instrument zur Rettungskräfte-Entwicklung; Schulte & Thielsch, in press).
• Institut der Feuerwehr Nordrhein-Westfalen (IdF NRW) — The central fire service training and research facility for North Rhine-Westphalia, providing institutional collaboration, subject-matter expertise, and research access.

4. Purpose

Emergency response personnel operate in high-reliability, time-pressured environments characterized by extreme physical hazard, environmental volatility, cognitive overload, and acute psychological stress. In statutory emergency frameworks—such as those codified under the German Fire Service Regulations (e.g., Feuerwehr-Dienstvorschriften FwDV 2 and FwDV 3) and federal state legislation like the Gesetz über den Brandschutz, die Hilfeleistung und den Katastrophenschutz (BHKG NRW)—firefighters completing their entry-level credentials are immediately expected to carry out frontline search, rescue, suppression, and technical extrication operations under direct command. Despite prescriptive curriculum catalogs stipulating mandatory instructional hours and thematic topics, local executing bodies retain wide didactic and methodological autonomy. Historically, training quality at municipal and district training centers remained largely unmeasured by validated psychometric procedures.

The fundamental purpose of the FIRE-B is to resolve this diagnostic void by supplying a scientifically constructed, psychometrically grounded instrument tailored specifically to entry-level emergency services education. Distinct from general course evaluation questionnaires designed for traditional university lecture halls, the FIRE-B captures the dual classroom-practical nature of high-consequence vocational training. The scale functions along three primary applied vectors:

• Diagnostic Pedagogical Quality Assurance: The instrument affords course directors, instructional cadres, and municipal fire chiefs granular diagnostic visibility into didactic delivery, instructor competence, resource sufficiency, and curriculum pacing. It identifies specific training deficits prior to operational failure in the field.
• Evidence-Based Instructional Feedback: It provides instructors with non-punitive, standardized, and aggregate evaluations from recruits, distinguishing between instructional support, interpersonal climate, and technical-didactic clarity, thereby guiding faculty development initiatives.
• Longitudinal and Benchmarked Research: The standardized structure enables municipal organizations, fire schools, and organizational psychologists to monitor longitudinal training evolutions, contrast contrasting educational frameworks (e.g., modular versus intensive courses, blended versus purely physical learning), and evaluate systemic curriculum reforms across voluntary and professional career tracks.

5. Psychological Construct

The construct assessed by the FIRE-B is the multidimensional perceived quality of basic emergency responder training. Rooted in training transfer frameworks, organizational behavior, and educational psychology, training quality is operationalized not as a monolithic score, but as a dynamic six-dimensional system capturing the holistic learning ecosystem:

1. Structure & Didactics (Struktur & Didaktik)

Comprising 6 items (Items 1–6), this dimension gauges the cognitive organization, clarity, topical breadth, and pedagogical adaptability of the curriculum. It operationalizes the instructor's capacity to decompose complex fireground physics, hydraulics, and tactical operational models into intelligible, sequenced learning modules. Furthermore, it measures the instructional staff’s didactic adaptability—their ability to flexibly tailor content to varying participant baseline proficiencies—and evaluates the subject-matter mastery exhibited by the training staff.

2. Support & Encouragement (Unterstützung & Förderung)

Capturing the socio-emotional climate and interpersonal instructor behavior through 5 items (Items 7–11), this scale evaluates the instructional cadre's student-centered engagement. It examines whether trainers display open receptivity toward student challenges, demonstrate active commitment to individual learner success, foster intrinsic motivation, and project psychological calm and composure under stress. Crucially for emergency responders, it measures the instructor's ability to contextualize didactic theory using authentic operational deployments (Einsatzerfahrungen).

3. Group Dynamics & Cohesion (Gruppe)

Measured via 3 items (Items 12–14), this subscale assesses the lateral learning environment, collective peer engagement, and social cohesion among trainees. In high-reliability emergency operations, operational safety rests heavily on team trust and dynamic coordination. This dimension captures whether course peers actively participate, maintain strong collective motivation, and develop reciprocal solidarity and team cohesion (Zusammenhalt) during rigorous instructional phases.

4. Practical Training Realism (Praxis)

Reflecting the experiential core of tactical emergency operations via 3 items (Items 15–17), this facet captures the physical fidelity and instructional scaffolding of tactical drills. It measures the ecological validity and operational realism of physical exercises (e.g., live-fire drills, heavy extrication simulations, self-contained breathing apparatus confidence courses), the adequacy of opportunities for repetitive physical skill rehearsal, and the rigor of structured pedagogical debriefings (after-action reviews).

5. Materials & Facilities (Material & Räumlichkeiten)

Operationalized across 5 items (Items 18–22), this dimension evaluates the socio-technical and infrastructural foundation of the training facility. It measures the quantitative availability and operational quality of classrooms, drill yards, simulation buildings, audiovisual media, instructional textbooks, frontline fire apparatus (engines, ladder trucks), specialized rescue tooling (hydraulic spreaders, thermal imaging cameras), and authentic consumable practice objects (e.g., training manikins, vehicle chassis for extrication practice).

6. Competence Acquisition (Kompetenzerwerb)

Forming the primary self-reported outcome criterion across 7 items (Items 23–29), this scale measures the recruit's perceived growth across multidimensional operational domains. It explicitly evaluates heightened hazard recognition and tactical reaction capacities, operational mastery of technical apparatus, perceived mission readiness for real-world frontline calls, expanded self-awareness regarding personal physical and psychological stress boundaries, operational self-efficacy, enhanced teamwork competencies, and broad theoretical knowledge consolidation.

6. Theoretical Framework

The architecture of the FIRE-B is embedded in the intersection of Kirkpatrick's Four-Level Training Evaluation Model, Cognitive Load Theory, and the sociotechnical conceptualization of High-Reliability Organizations (HROs):

Kirkpatrick's Hierarchical Evaluation Taxonomy

Donald Kirkpatrick's foundational model conceptualizes training effectiveness across four progressive tiers: Level 1 (Reaction), Level 2 (Learning), Level 3 (Behavior), and Level 4 (Results). Traditional course evaluations frequently conflate Level 1 satisfaction ("smile sheets") with genuine training effectiveness. The FIRE-B methodologically stratifies these layers. While Dimensions 1, 2, 4, and 5 assess Level 1 environmental and instructional design elements, Dimension 6 (Kompetenzerwerb) directly taps into subjective Level 2 learning outcomes (knowledge retention, cognitive self-efficacy) and perceived precursors to Level 3 transfer (confidence to perform on upcoming frontline deployments).

Cognitive Load and Dual-Processing in Tactical Training

Emergency responder training mandates that novices transition from cognitively demanding, effortful analytical processing to rapid, automated pattern matching under life-threatening stressors. According to Sweller’s Cognitive Load Theory, poorly structured didactic material or chaotic exercise pacing induces extraneous cognitive load, impeding germane schema acquisition. The subscales Struktur & Didaktik and the diagnostic pacing item (Item 30: "Das Tempo der Stoffvermittlung war zu hoch für mich") directly capture whether instructional pacing avoids cognitive overload, enabling foundational schema construction regarding equipment operation and hazard identification.

Sociotechnical Systems and High-Reliability Organizing

In high-reliability frameworks (Weick & Sutcliffe), organizations operating in hazardous settings maintain safety through collective mindfulness, deference to expertise, and redundancy. These qualities must be internalized during entry-level socialization. The FIRE-B explicitly includes Gruppe and Unterstützung & Förderung to operationalize the psychological safety, shared mental models, and supportive feedback loops necessary for recruits to admit errors, ask questions, and build resilient peer bonds before encountering hazardous real-world scenes.

7. Validity

The FIRE-B underwent rigorous empirical validation through a three-phase research program yielding robust content, construct, and criterion-related validity evidence:

Content Validity

Content validity was established in Study I via a qualitative exploratory study of subject-matter experts (N = 21, including career and volunteer recruits, certified instructors, and fire school directors). Utilizing the Critical Incident Technique (CIT; Flanagan, 1954) combined with open-ended exploratory probes regarding effective training characteristics, researchers conducted structured qualitative content analyses. This yielded eight fundamental training categories (Didactics, Motivation & Engagement, Personality, Contents & Methods, Structure & Organization, Materials & Premises, Group, and Goal Attainment), directly informing the initial 51-item pool. Subject-matter panels affirmed that every core operational aspect of basic responder training was adequately represented.

Construct and Structural Validity

Construct validity was demonstrated across Study II (N = 257) and Study III (N = 451). In Study II, exploratory factor analyses (EFA) on the preliminary 51-item pool guided psychometric pruning down to 30 items across six theoretical factors. In Study III, confirmatory factor analyses (CFA) cross-validated the refined 29-item instrument on an independent cohort of career (Brandmeister) and voluntary (Truppmann/Truppführer) candidates. The hypothesized six-factor model exhibited superior structural fit compared to competing unidimensional and hierarchical models, confirming that the six dimensions represent distinct yet correlated facets of training quality.

Convergent and Criterion-Related Validity

Convergent validity was evaluated by correlating the FIRE-B subscales with established academic teaching evaluation tools adapted from higher education (e.g., FEVOR, TRIL, MFE scales) and leadership emergency evaluation scales (FIRE; Schulte & Thielsch). Subscales displayed strong, theoretically aligned bivariate correlations with global training evaluations:

• Overall learning success (Item 31: "Ich habe in der Ausbildung sehr viel gelernt") correlated strongly with Structure & Didactics (r > .60) and Competence Acquisition (r > .70).
• The summative academic school grade (Item 32) demonstrated robust negative correlations with all six FIRE-B subscales (as lower numerical grades indicate superior performance in the German grading system: 1 = very good, 6 = fail), with the strongest associations observed for Struktur & Didaktik and Kompetenzerwerb (typically $|r| = .50 – .68$).
• Discriminant validity was evidenced by low-to-moderate intercorrelations between external environmental variables (such as Material & Räumlichkeiten) and interpersonal processes (such as Gruppe), proving that recruits accurately differentiate between infrastructure quality and group cohesion.

8. Reliability

Across validation samples, the FIRE-B exhibited exemplary internal consistency metrics. Psychometric assessments conducted during Study II (N = 257) and confirmed in Study III (N = 451) revealed scale reliability estimates well exceeding conventional psychometric thresholds for applied organizational diagnostics ($lpha, \omega ge .70$):

• Cronbach's Alpha ($lpha$): All six subscales consistently demonstrated internal consistencies of $lpha ge .76$, with core dimensions such as Struktur & Didaktik and Kompetenzerwerb frequently exceeding $lpha = .85$.
• McDonald's Omega ($\omega$): Given the known limitations of Cronbach's alpha regarding tau-equivalence assumptions, composite reliability was verified using McDonald's omega, with all six dimensions demonstrating $\omega ge .76$.
• Item-Total Correlations: Corrected item-total correlations ($r_{it}$) across the 29 primary items were uniformly high, consistently exceeding .40 and demonstrating that each item contributes meaningfully to its assigned theoretical dimension without excessive redundancy.
• Standard Error of Measurement: Low standard errors of measurement across all dimensions substantiate the scale's suitability for aggregate organizational benchmarking and course-level decision making.

9. Factor Analysis

The dimensionality of the FIRE-B was evaluated through a sequential exploratory and confirmatory factor analytic strategy across two independent responder samples:

Exploratory Factor Analysis (Study II)

In Study II (N = 257; 229 males, 26 females, 2 unspecified; mean age 25.75, SD = 6.48; 37% career fire service, 63% volunteer fire service), the preliminary 51-item instrument was subjected to an exploratory factor analysis (EFA; Principal Axis Factoring with oblique rotation [Promax], reflecting anticipated latent correlations among instructional facets). Missing values (0.5% of total cells, occurring primarily due to the non-applicable response option) were imputed via the Expectation-Maximization (EM) algorithm.

Eigenvalue scree tests, parallel analysis, and factor interpretability converged on a clear six-factor architecture. Items demonstrating cross-loadings ($> .30$ on non-target factors), low primary factor loadings ($< .40$), or poor item distributions were pruned. This resulted in an optimized 30-item operational version showing clean primary loadings and meaningful theoretical coherence.

Confirmatory Factor Analysis (Study III)

In Study III (N = 451; 414 males, 37 females; mean age 34.02, SD = 9.92; encompassing career recruit training [24%], volunteer Truppmann [19%], volunteer Truppführer [33%], and other district-level courses [24%]), the factor structure was tested using maximum likelihood structural equation modeling. One item was eliminated due to marginal statistical contribution, producing the final 29-item primary inventory across six latent dimensions.

The postulated six-factor model exhibited solid goodness-of-fit indices: the comparative fit index (CFI) and Tucker-Lewis index (TLI) approached or surpassed .90/.95 thresholds, with acceptable root mean square error of approximation (RMSEA $le .06$) and standardized root mean square residual (SRMR $le .06$). Standardized factor loadings across all 29 items loaded substantially on their respective target factors (predominantly ranging from .60 to .88, all $p < .001$), supporting the multidimensional construct definition.

10. Instrument / Measurement Tool

The FIRE-B is a standardized, self-administered survey inventory available in digital (web-based) and paper-and-pencil formats.

Structural Configuration

• Core Item Inventory: 29 standardized declarative statements across six subscales.
  • Struktur & Didaktik: Items 1 to 6 (6 items)
  • Unterstützung & Förderung: Items 7 to 11 (5 items)
  • Gruppe: Items 12 to 14 (3 items)
  • Praxis: Items 15 to 17 (3 items)
  • Material & Räumlichkeiten: Items 18 to 22 (5 items)
  • Kompetenzerwerb: Items 23 to 29 (7 items)
• Optional Diagnostic Module: 4 supplementary items.
  • Item 30 (Instructional Pacing): "Das Tempo der Stoffvermittlung war zu hoch für mich." (7-point Likert scale)
  • Item 31 (Global Learning): "Ich habe in der Ausbildung sehr viel gelernt." (7-point Likert scale)
  • Item 32 (Summative Academic Grade): "Ich würde der Ausbildung insgesamt folgende Schulnote geben." (6-point grading scale)
  • Item 33 (Qualitative Feedback): Open-ended qualitative feedback box for textual commendations, criticisms, and structural improvements regarding apparatus, facilities, practice objects, and learning materials.

Response Formats

• Items 1–31: Siebenstufiges Antwortformat mit den Optionen 1 = stimme gar nicht zu, 2 = stimme nicht zu, 3 = stimme eher nicht zu, 4 = neutral, 5 = stimme eher zu, 6 = stimme zu, 7 = stimme vollkommen zu. Ein zusätzliches Feld bietet die Möglichkeit anzugeben, dass das jeweilige Item nicht sinnvoll zu beantworten sei.
• Item 32: Sechsstufiges Antwortformat mit den Optionen 1 = sehr gut, 2 = gut, 3 = befriedigend, 4 = ausreichend, 5 = mangelhaft und 6 = ungenügend.
• Item 33: Offenes Antwortformat (unconstrained free-text commentary).

Scoring and Aggregation Instructions

• Individual Scale Means: Subscale scores are computed by summing the numerical ratings of the completed items within that subscale and dividing by the number of answered items. Non-applicable responses (nicht sinnvoll zu beantworten) are treated as missing data and excluded from mean calculations.
• Course-Level Aggregation: Subscale means are aggregated across all trainees within a course cohort to yield the overall training evaluation profile. Higher scores reflect more favorable evaluations of course quality and higher self-reported competence acquisition.
• Missing Data and Quality Safeguards: In digital administrations, forced-choice validation is advised for Items 1–32. In paper administrations, protocols with three or more missing items across the core scale should be excluded from final psychometric aggregation.
• Aggregation Thresholds: To safeguard anonymity and statistical stability, group-level analyses should only be calculated when a minimum of 8 completed questionnaires are available (or at least a 50% response rate in small cohorts where total enrollment is under 15).

11. Permissions & Fee and Test Year

The FIRE-B was developed and finalized between 2017 and 2018 through research conducted at the University of Münster in collaboration with the Institut der Feuerwehr NRW. The empirical foundation was documented in master's theses and research monographs by Streppel (2018) and Kläpker (2018), building upon the leadership evaluation instrument FIRE (Schulte & Thielsch).

The instrument is published as an open-access scientific diagnostic tool for public, municipal, and non-commercial educational and research applications within public safety organizations, voluntary fire brigades, professional fire departments, and emergency service academies. No licensing fees or royalties are charged for non-commercial institutional evaluations, provided proper scientific attribution is maintained. Commercial assessment distributors or proprietary consulting vendors wishing to incorporate the instrument into paid commercial platforms must seek explicit permission from the original developers at the University of Münster.

12. References

• Babiel, J. (2016). Evaluation von Führungskräftetrainings der Feuerwehr: Entwicklung eines Fragebogens [Unpublished bachelor's thesis]. University of Münster, Germany.
• Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. https://doi.org/10.1037/h0061470
• Gediga, G., Hamborg, K.-C., & Willumeit, K. (2000). Das Kieler Evaluationsmodell (KIEL). In M. Krämer (Ed.), Beiträge zur Hochschuldidaktik. DGPs.
• Gläßer, D., Pohlenz, P., Rindermann, H., & Wild, E. (2002). Trierer Inventar zur Lehrevaluation (TRIL). Universität Trier.
• Grötemeier, I., & Thielsch, M. T. (2010a). Münsteraner Fragebogen zur Evaluation von Zusatzangeboten – Großgruppen (MFE-ZGr). Universität Münster.
• Grötemeier, I., & Thielsch, M. T. (2010b). Münsteraner Fragebogen zur Evaluation von Hausarbeiten (MFE-ZHa). Universität Münster.
• Hirschfeld, G., & Thielsch, M. T. (2009). Münsteraner Fragebogen zur Evaluation von Vorlesungen (MFE-V). Universität Münster.
• Kläpker, L. (2018). Konstruktion und Validierung des Feedback-Instruments zur Rettungskräfte-Entwicklung – Basisausbildung (FIRE-B) [Unpublished master's thesis]. Department of Psychology, University of Münster, Germany.
• Olde lütke Beverborg, M. (2017). Evaluation von Führungskräftetrainings der Feuerwehr: Dozentenverhalten [Unpublished bachelor's thesis]. University of Münster, Germany.
• Rindermann, H., & Amelang, M. (1994). Das Heidelberger Inventar zur Lehrveranstaltungs-Evaluation (HILVE). Empirische Pädagogik, 8(2), 183–208.
• Robertz

  , A. (2015). Evaluation von Führungskräftetrainings: Pilotierung eines Fragebogens [Unpublished master's thesis]. University of Münster, Germany.

• Schulte, F., & Thielsch, M. T. (in press). Das Feedback-Instrument zur Rettungskräfte-Entwicklung (FIRE). Zeitschrift für Arbeits- und Organisationspsychologie.
• Staufenbiel, T. (2000). Fragebogen zur Evaluation von universitären Lehrveranstaltungen (FEVOR). Diagnostica, 46(4), 169–181. https://doi.org/10.1026//0012-1924.46.4.169
• Streppel, H. (2018). Evaluation der Basisausbildung bei der Feuerwehr: Entwicklung eines Fragebogens [Unpublished master's thesis]. Department of Psychology, University of Münster, Germany.
• Taber, N., Plumb, D., & Jolemore, S. (2008). "Pretending to be safe": Firefighters' learning of risk and its dynamic relation to workplace culture. Journal of Workplace Learning, 20(6), 397–411. https://doi.org/10.1108/13665620810893510
• Thielsch, M. T., Busjan, R., & Frerichs, B. (2018). Feedback-Instrument zur Prüfungsevaluation im Rettungswesen (FIRE-P). Universität Münster.
• Thielsch, M. T., & Hirschfeld, G. (2012). Münsteraner Fragebogen zur Evaluation von Seminaren (MFE-Sr). Zeitschrift für Hochschulentwicklung, 7(3), 1–12. https://doi.org/10.3217/zfhe-7-03/01

13. Items of the Scale