1. Abstract
The Feedback Instrument for Rescue Force Development – Seminars and One-day Events (known by its German acronym FIRE-ST: Feedback-Instrument zur Rettungskräfte-Entwicklung – Seminare und Tagesveranstaltungen) is a standardized psychometric evaluation instrument engineered to evaluate instructional quality, learning processes, and self-reported learning outcomes in emergency service professional development. Adapted from the foundational FIRE instrument (Schulte & Thielsch, 2019)—which originally targeted multi-week residential leadership academies—the FIRE-ST is streamlined specifically for short-format educational interventions ranging from one-day seminars to three-day workshops. The instrument comprises 23 items structured across two distinct hierarchical evaluation planes: instructional process dimensions, which capture Trainer Behavior (Dozentenverhalten, 6 items), Level of Challenge / Overextension (Anforderungsniveau, 3 items), and Structure (Struktur, 3 items); and instructional outcome dimensions, which evaluate Competence Acquisition (Kompetenzerwerb, 5 items) and Training Transfer (Transfer, 3 items). In addition, three optional single items gauge global academic grades, program recommendation intent, and open-ended qualitative feedback. Responses for the primary 20 items are recorded on a 7-point Likert scale alongside an explicit option to indicate that an item cannot be meaningfully evaluated. Psychometric validation was conducted on a sample of $N = 247$ emergency commanders and firefighters attending leadership and technical seminars at the Institute of the North Rhine-Westphalia Fire Brigade (Institut der Feuerwehr Nordrhein-Westfalen; IdF NRW). Confirmatory factor analyses demonstrated robust structural validity across both the process model ($\chi^2(51) = 72.40, p = .026, ext{CFI} = .99, ext{TLI} = .98, ext{RMSEA} = .04$) and the outcome model ($\chi^2(17) = 36.05, p = .005, ext{CFI} = .97, ext{TLI} = .95, ext{RMSEA} = .09$). Internal consistency is well-supported across subscales, with Cronbach’s $lpha$ ranging from $.74$ to $.89$ and McDonald’s $\omega_H$ ranging from $.76$ to $.89$. The FIRE-ST provides an empirically grounded, rapid-administration framework for quality assurance, personnel development, and operational safety training in high-reliability organizations.
2. Keywords
FIRE-ST, training evaluation, emergency services, rescue force development, instructional quality, firefighter leadership, Kirkpatrick model, competence acquisition, training transfer, high-reliability organizations, psychometrics
3. Authors
The FIRE-ST was conceptualized, operationalized, and psychometrically validated through an academic–practitioner partnership between the Department of Organizational and Business Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster) and the Institute of the North Rhine-Westphalia Fire Brigade (Institut der Feuerwehr Nordrhein-Westfalen – IdF NRW):
- Meinald T. Thielsch, PhD – Associate Professor of Organizational and Business Psychology, Department of Psychology, University of Münster, Münster, Germany. Lead investigator specializing in operational evaluation, user experience, and training assessment in emergency management systems.
- Carsten Schulte, MSc – Organizational Psychologist and Research Scientist, Department of Psychology, University of Münster, Münster, Germany. Primary architect of the original FIRE measurement system.
- Cooperating Institution: Institut der Feuerwehr Nordrhein-Westfalen (IdF NRW), Münster, Germany. The largest state-level fire service academy in Germany, serving as the validation setting for emergency management personnel development.
4. Purpose
Emergency response personnel operate in dynamic, high-stakes environments characterized by time compression, high ambiguity, severe physical risk, and life-or-death decision-making. Operating as High-Reliability Organizations (HROs), emergency service systems—such as municipal fire departments, disaster relief agencies, and technical rescue teams—must guarantee that training programs produce measurable competencies and reliable behavioral transfer under operational stress. Incident commanders, group leaders, and technical specialists participate regularly in continuing professional education to master advanced tactical protocols, disaster psychology, chemical defense, and inter-agency coordination.
Despite the vital importance of this training, the tools historically used to evaluate emergency service instruction have suffered from significant methodological limitations. Training institutions frequently rely on ad-hoc, unvalidated customer-satisfaction forms (often termed “smile sheets”). These unstandardized forms confound affective entertainment with actual instructional efficacy, fail to differentiate cognitive load from pedagogical clarity, and ignore the structural distinction between classroom process characteristics and proximal learning outcomes.
While the original Feedback Instrument for Rescue Force Development (FIRE; Schulte & Thielsch, 2019) resolved these deficits for extensive multi-week residential leadership certifications, its structural assumptions do not align directly with short-format seminars and one-day events. Multi-week academies involve small cohort cohorts, extensive peer collaboration, and iterative scenario-based tactical field exercises. In contrast, short seminars (spanning 1 to 3 days) often accommodate up to 120 attendees, rely predominantly on instructor-centered didactic delivery, and feature concentrated thematic presentations. Under these condensed conditions, peer-group dynamics diminish as an instructional factor, while the didactic clarity, cognitive pacing, and communicative competence of the individual trainer become paramount.
The FIRE-ST was engineered to resolve this applied psychometric gap. Its purpose is to deliver a concise, psychometrically sound, and diagnostically differentiated instrument capable of assessing both instructional delivery (processes) and self-assessed capability growth (outcomes) in 3 to 5 minutes of administration time. In applied contexts, it serves four central functions:
- Instructor-Level Formative Feedback: Providing individual trainers with standardized benchmarks on clarity, responsiveness, motivational capacity, and subject mastery, allowing targeted pedagogical refinement.
- Curricular and Cognitive Load Calibration: Diagnosing whether an intensive technical curriculum overextends participants through excessive pacing, dense slide presentations, or conceptual difficulty.
- Outcome and Transfer Readiness Monitoring: Quantifying self-efficacy gains in situational assessment, critical decision-making, and stress resilience, while assessing whether participants can translate course materials into operational field behaviors.
- Organizational Quality Assurance: Establishing institutional norms across diverse seminar topics, enabling training directors to benchmark different teaching modalities, identify underperforming seminars, and recognize instructional excellence.
5. Psychological Construct
The FIRE-ST operationalizes instructional quality in short-format emergency service training as a multi-dimensional construct organized into two correlated, conceptually distinct domains: Learning Processes (instructional inputs and classroom environment) and Learning Outcomes (proximal competence acquisition and perceived transfer readiness).
5.1. Instructional Process Dimensions
The process domain reflects the participant’s direct experience of teaching methods, content organization, and cognitive pacing during the seminar:
- Trainer Behavior (Dozentenverhalten, Items 1–6): This subscale measures the pedagogical and communicative competence of the instructor. Rooted in research on teacher clarity, instructional warmth, and classroom climate, it assesses the trainer’s ability to synthesize complex tactical topics concisely, deliver actionable feedback, stimulate engagement, display commitment to participant comprehension, communicate clearly, and demonstrate expertise in the subject matter.
- Level of Challenge / Overextension (Anforderungsniveau, Items 7–9): This subscale captures subjective cognitive overextension, conceptualized through Cognitive Load Theory. Rather than assessing simple difficulty, the items detect counterproductive overextension in curricular volume, instructional pacing, and intellectual complexity. Because high scores reflect excessive instructional speed and intellectual strain, lower scores are desirable for this dimension.
- Structure (Struktur, Items 10–12): This subscale quantifies the organizational coherence and conceptual transparency of the instructional design. It evaluates whether the seminar followed a clear sequence, whether the thematic outline remained intelligible throughout, and whether the course provided an orienting overview of the topic. Structural clarity minimizes extraneous cognitive load, facilitating schema integration.
5.2. Instructional Outcome Dimensions
The outcome domain evaluates the proximal changes in cognitive capabilities and operational readiness attributed directly to course participation:
- Competence Acquisition (Kompetenzerwerb, Items 13–17): Drawing on naturalistic decision-making models in emergency management, this dimension measures subjective gains in critical command competencies. The items operationalize five core capabilities required in crisis leadership: early hazard detection, rapid decision-making in critical scenarios, awareness of personal performance limits, emotional stability under stress, and the delivery of unambiguous operational commands.
- Training Transfer (Transfer, Items 18–20): Grounded in Baldwin and Ford’s transfer framework, this scale assesses the perceived generalizability of seminar training to real-world command roles. It assesses command readiness for upcoming incident deployments, personal confidence derived from tactical or simulation exercises, and the perceived transferability of learned principles to the respondent’s home department. Because some one-day seminars consist entirely of theoretical lectures, this subscale features a conditional gate: it is completed only when seminars incorporate practical tabletop, command-board, or simulation components.
5.3. Supplementary Evaluative Items
To facilitate administrative reporting, three non-psychometric items supplement the validated core scales: Item 21 provides a traditional German school grade summary metric (1 = very good to 6 = insufficient); Item 22 captures behavioral recommendation intent via a dichotomous (yes/no) metric; and Item 23 offers an open-ended qualitative field for text-based participant commentary.
6. Theoretical Framework
The architecture of the FIRE-ST is grounded in Kirkpatrick’s Four-Level Training Evaluation Model, augmented by modern instructional design and organizational psychology paradigms (Alliger et al., 1997; Blanchard & Thacker, 2013).
6.1. The Adapted Kirkpatrick Evaluation Paradigm
Kirkpatrick’s classical taxonomy stratifies training evaluations into four ascending tiers:
- Level 1: Reaction (affective appraisals and perceived utility of the training).
- Level 2: Learning (acquisition of knowledge, skills, capabilities, and attitudes).
- Level 3: Behavior (transfer of learning into everyday job performance).
- Level 4: Results (organizational-level operational and safety impacts).
Methodologists emphasize that initial evaluations must rigorously establish reactions and proximal learning before meaningful inferences can be drawn regarding distal behavioral transfer and organizational outcomes (Kirkpatrick & Kirkpatrick, 2006). Alliger et al. (1997) revised Kirkpatrick’s Level 1 by differentiating between affective reactions (“Did I enjoy the seminar?”) and utility reactions (“Is this training relevant to my command performance?”). The FIRE-ST incorporates utility reactions within its process scales (e.g., Structure and Trainer Feedback) and measures self-assessed Level 2 Learning through the Competence Acquisition scale.
6.2. The Process–Outcome Dualism
Following the structural formulation established in the broader FIRE framework (Schulte & Thielsch, 2019), the FIRE-ST relies on a fundamental distinction between teaching processes and learning outcomes. This conceptual dualism is illustrated below:
| Evaluation Plane | FIRE-ST Subscale | Underlying Pedagogical Theory | Primary Focus |
|---|---|---|---|
| Learning Processes (Instructional Inputs) |
Trainer Behavior (Items 1–6) | Teacher Clarity & Behavioral Feedback Theories | Pedagogical delivery, subject mastery, participant engagement |
| Level of Challenge (Items 7–9) | Cognitive Load Theory (Sweller, 1988) | Pacing, volume, cognitive overextension (extrinsic load) | |
| Structure (Items 10–12) | Advance Organizer & Schema Construction Theories | Conceptual flow, structural clarity, syllabus transparency | |
| Learning Outcomes (Proximal Yields) |
Competence Acquisition (Items 13–17) | Naturalistic Decision Making (NDM; Klein, 1993) | Hazard appraisal, rapid decision-making, stress regulation |
| Transfer (Items 18–20) | Model of Training Transfer (Baldwin & Ford, 1988) | Deployment readiness, command confidence, local applicability |
By decoupling the evaluation of instructional processes from learning outcomes, the FIRE-ST avoids the confounding effects common to single-score satisfaction instruments. For example, a charismatic instructor might receive high marks for presentation style while presenting an overloaded, unstructured syllabus that yields poor competence acquisition. Conversely, an analytically demanding seminar led by an understated instructor may produce substantial gains in hazard assessment capabilities. The dual-model architecture of the FIRE-ST disentangles these instructional dynamics.
7. Validity
The validity of the FIRE-ST was evaluated across multiple empirical dimensions using data collected from emergency service commanders and firefighters at the IdF NRW.
7.1. Content and Curricular Validity
Content validity was established through adaptation from the extensively validated parent FIRE instrument (Schulte & Thielsch, 2019; Schulte et al., 2019). The adaptation process systematically accounted for the structural characteristics of short seminars:
- Removal of Cohort Dynamics: The Group (Gruppe) subscale from the original FIRE was eliminated, as peer socialization and group dynamics play a minimal role in short seminars compared to multi-week residential academies.
- Expansion of Trainer Items: To capture the instructor-centric nature of short lectures, the Trainer Behavior scale was expanded to six items by incorporating two previously omitted items regarding communicative clarity and subject-matter expertise.
- Lexical Adaptation: All item wordings were adjusted from “academy course” (Lehrgang) to “seminar / educational event” (Seminar / Lehrveranstaltung) and shifted from team-teaching references to individual instructors.
7.2. Construct and Factorial Validity
Structural construct validity was evaluated using Confirmatory Factor Analysis (CFA) using the lavaan package in R with Robust Maximum Likelihood (MLR) estimation. Factorial integrity was examined across both the Process Model and the Outcome Model:
- Process Measurement Model ($N = 230$): A three-factor specification (Trainer Behavior, Level of Challenge, Structure) demonstrated a close fit to the empirical data: $\chi^2(51) = 72.40, p = .026$, normed $\chi^2/df = 1.42$, Comparative Fit Index $( ext{CFI}) = .99$, Tucker-Lewis Index $( ext{TLI}) = .98$, Root Mean Square Error of Approximation $( ext{RMSEA}) = .04$ ($90%\text{ CI } [.02, .06]$), and Standardized Root Mean Square Residual $( ext{SRMR}) = .04$. All standardized factor loadings were statistically significant ($p < .001$) and exceeded $.50$.
- Outcome Measurement Model ($N = 148$): A two-factor specification (Competence Acquisition and Transfer) was tested among respondents who participated in seminars with practical scenario exercises. The baseline model yielded an acceptable fit ($\chi^2(19) = 58.54, p < .001, ext{CFI} = .93, ext{TLI} = .90, ext{RMSEA} = .12, ext{SRMR} = .05$). Inspection of modification indices indicated correlated residuals between Item 13 and Item 14, as well as between Item 14 and Item 16. These correlated error variances reflect shared method and conceptual variance tied to acute stress regulation and operational crisis management. Specifying these residual covariances improved model fit significantly ($\Delta \chi^2(2) = 22.49, p < .001$), resulting in an adjusted model with strong fit: $\chi^2(17) = 36.05, p = .005$, $\chi^2/df = 2.12$, $ ext{CFI} = .97$,$ ext{TLI} = .95$,$ ext{RMSEA} = .09$ ($90%\text{ CI } [.05, .12]$), and $ ext{SRMR} = .05$.
7.3. Convergent and Criterion-Related Validity
Evidence for criterion validity is demonstrated through the systematic pattern of correlations between the FIRE-ST core dimensions and the supplementary evaluative criteria (Academic Grade and Recommendation Intent):
- Instructional process dimensions (Trainer Behavior, Structure) correlate positively with overall seminar grades and participant recommendation intent.
- Level of Challenge (Overextension) behaves as an inverse indicator when cognitive pacing exceeds comprehension thresholds, showing negative associations with structural satisfaction and overall ratings.
- Competence Acquisition and perceived Transfer Readiness correlate significantly with overall training satisfaction, confirming that participants evaluate seminar utility through the lens of operational readiness.
8. Reliability
The FIRE-ST demonstrates robust internal consistency across all subscales. Because four of the five subscales represent congeneric rather than tau-equivalent measurement models, both Cronbach’s $lpha$ and McDonald’s $\omega_H$ (hierarchical omega / composite reliability) were computed. McDonald’s $\omega_H$ provides the less biased estimate of true internal consistency under congeneric assumptions.
| Subscale | Number of Items | Cronbach’s $lpha$ | McDonald’s $\omega_H$ | Measurement Model |
|---|---|---|---|---|
| Trainer Behavior (Dozentenverhalten) | 6 | .89 | .89 | Essentially Tau-Equivalent |
| Level of Challenge (Anforderungsniveau) | 3 | .87 | .88 | Congeneric |
| Structure (Struktur) | 3 | .78 | .79 | Congeneric |
| Competence Acquisition (Kompetenzerwerb) | 5 | .86 | .87 | Congeneric |
| Transfer (Transfer) | 3 | .74 | .76 | Congeneric |
8.1. Item Discrimination and Distributional Parameters
Corrected item-total correlations ($r_{it}$) across the instrument fall within favorable psychometric thresholds. As detailed in the validation data ($N = 247$):
- Trainer Behavior: Corrected item-total correlations range from $r_{it} = .62$ to $.77$, with mean scores between $M = 5.95$ ($SD = 0.92$) and $M = 6.59$ ($SD = 0.68$), reflecting generally high instructional appreciation.
- Level of Challenge: Items exhibit item discriminations between $r_{it} = .73$ and $.79$. Means range from $M = 1.61$ ($SD = 0.94$) to $M = 1.88$ ($SD = 1.22$). Strong positive skewness ($2.20$ to $2.67$) indicates that significant cognitive overextension was relatively rare in the validation sample.
- Structure: Item discriminations range from $r_{it} = .51$ to $.68$, with item means exceeding $6.10$.
- Competence Acquisition: Item-total correlations span $r_{it} = .59$ to $.77$, with mean item scores between $5.30$ and $5.82$, demonstrating balanced variance in perceived capability acquisition.
- Transfer: Item discriminations range between $r_{it} = .48$ and $.62$. Item 20 (“I can transfer what I learned to my home station”) demonstrated a high mean ($M = 6.08, SD = 0.90$), whereas Item 19 (“practical exercises provided confidence”) showed higher variance ($M = 5.60, SD = 1.18$).
9. Factor Analysis
The structural validation of the FIRE-ST employed Structural Equation Modeling (SEM) and confirmatory factor analytic procedures in R. The decision to evaluate separate Process and Outcome measurement models reflects the theoretical premise that instructional methods and learning outcomes operate as distinct conceptual systems.
9.1. Confirmatory Factor Analysis: Process Model
The process model specifies three latent constructs: Trainer Behavior ($\xi_1$, 6 indicators), Level of Challenge ($\xi_2$, 3 indicators), and Structure ($\xi_3$, 3 indicators). The input covariance matrix was derived from respondents with complete responses on all 12 process items ($N = 230$ after removing 17 respondents who endorsed “not answerable”).
| Item | Latent Factor | Std. Loading ($lambda$) | Standard Error ($SE$) | $p$-value |
|---|---|---|---|---|
| Item 1 (Concise summary) | Trainer Behavior | .79 | .041 | < .001 |
| Item 2 (Useful feedback) | Trainer Behavior | .78 | .043 | < .001 |
| Item 3 (Motivation to participate) | Trainer Behavior | .68 | .051 | < .001 |
| Item 4 (Interest in learning success) | Trainer Behavior | .84 | .035 | < .001 |
| Item 5 (Linguistic clarity) | Trainer Behavior | .76 | .045 | < .001 |
| Item 6 (Expert subject knowledge) | Trainer Behavior | .69 | .053 | < .001 |
| Item 7 (Volume overextension) | Level of Challenge | .79 | .042 | < .001 |
| Item 8 (Pacing too rapid) | Level of Challenge | .88 | .031 | < .001 |
| Item 9 (Content too difficult) | Level of Challenge | .84 | .037 | < .001 |
| Item 10 (Clear course structure) | Structure | .83 | .044 | < .001 |
| Item 11 (Comprehensible outline) | Structure | .80 | .048 | < .001 |
| Item 12 (Broad thematic overview) | Structure | .57 | .067 | < .001 |
9.2. Confirmatory Factor Analysis: Outcome Model
The outcome model evaluates Competence Acquisition ($\eta_1$, 5 indicators) and Transfer ($\eta_2$, 3 indicators). The analysis was conducted on participants who attended seminars with practical exercises ($N = 148$). The initial model yielded $\chi^2(19) = 58.54$, $ ext{CFI} = .93$,$ ext{TLI} = .90$,$ ext{RMSEA} = .12$, and$ ext{SRMR} = .05$. Modification indices indicated correlated residuals among Items 13, 14, and 16. Modeling these specific error covariances yielded improved fit:$chi^2(17) = 36.05, p = .005$,$chi^2/df = 2.12$,$ ext{CFI} = .97$,$ ext{TLI} = .95$,$ ext{RMSEA} = .09$,$ ext{SRMR} = .05$.
| Item | Latent Factor | Std. Loading ($lambda$) | Standard Error ($SE$) | $p$-value |
|---|---|---|---|---|
| Item 13 (Early hazard detection) | Competence Acquisition | .68 | .062 | < .001 |
| Item 14 (Critical decision-making) | Competence Acquisition | .84 | .044 | < .001 |
| Item 15 (Personal limits awareness) | Competence Acquisition | .74 | .053 | < .001 |
| Item 16 (Composure under stress) | Competence Acquisition | .87 | .040 | < .001 |
| Item 17 (Clear, correct commands) | Competence Acquisition | .76 | .051 | < .001 |
| Item 18 (Readiness for command) | Transfer | .79 | .061 | < .001 |
| Item 19 (Confidence from exercises) | Transfer | .75 | .064 | < .001 |
| Item 20 (Transfer to home agency) | Transfer | .58 | .080 | < .001 |
10. Instrument / Measurement Tool
- Instrument Designation: Feedback Instrument for Rescue Force Development – Seminars and One-day Events (FIRE-ST; German: Feedback-Instrument zur Rettungskräfte-Entwicklung – Seminare und Tagesveranstaltungen).
- Target Respondent Group: Fire service commanders, volunteer/career firefighters, and emergency management personnel participating in short training events (1–3 days).
- Item Count: 23 total items (20 core psychometric indicators organized into 5 subscales, plus 3 supplementary evaluative items).
- Administration Format: Standardized self-report inventory, administered via paper-and-pencil or digital survey platforms at the conclusion of training.
- Estimated Completion Time: 3 to 5 minutes.
- Core Subscale Architecture:
- Trainer Behavior (Dozentenverhalten): 6 items (Items 1–6).
- Level of Challenge / Overextension (Anforderungsniveau): 3 items (Items 7–9).
- Structure (Struktur): 3 items (Items 10–12).
- Competence Acquisition (Kompetenzerwerb): 5 items (Items 13–17).
- Transfer (Transfer): 3 items (Items 18–20; administered conditionally if practical scenario exercises were included).
- Supplementary Single Items:
- Global Rating 1 (Globalurteil 1): Item 21 (Academic overall grade).
- Global Rating 2 (Globalurteil 2): Item 22 (Recommendation intent).
- Qualitative Feedback: Item 23 (Open-ended commentary for praise, critique, or suggestions).
- Core Response Scale: 7-point Likert scale (1 = stimme gar nicht zu / strongly disagree, 2 = stimme nicht zu / disagree, 3 = stimme eher nicht zu / somewhat disagree, 4 = neutral, 5 = stimme eher zu / somewhat agree, 6 = stimme zu / agree, 7 = stimme vollkommen zu / strongly agree). Includes a separate option: nicht sinnvoll beantwortbar (“cannot be answered meaningfully / not applicable”).
- Scoring and Computational Rules:
- Point Allocation: Responses are scored 1 to 7. Endorsement of “cannot be answered meaningfully” is treated as missing data ($NA$) and excluded from individual item calculations.
- Directionality: Higher scores reflect favorable evaluations across Trainer Behavior, Structure, Competence Acquisition, and Transfer. For the Level of Challenge subscale, lower scores reflect appropriate pacing, while higher scores indicate participant overextension. For aggregated reporting, Level of Challenge items may be reverse-coded ($x_{inv} = 8 – x$) so that higher scores consistently denote positive instructional outcomes.
- Data Screening: Participants omitting three or more core items should be excluded from aggregate analyses. High proportions of missing or “not answerable” ratings suggest poor curricular fit with the evaluation form.
- Scale Means: Subscale scores are calculated as the unweighted mean of valid item responses: $M_{scale} = \frac{\sum_{i=1}^{k} M_{item_i}}{k}$.
- Global Grade Analysis: Because German school grades represent an ordinal scale (1 = sehr gut to 6 = ungenügend), the median or frequency distribution should be reported rather than the arithmetic mean.
- Confidentiality and Anonymity Threshold: To preserve anonymity, individual evaluations must not collect identifiable demographic variables. Aggregate reporting should be restricted to cohorts with at least $N = 8$ completed questionnaires (or a response rate of $ge 50%$ in courses with 9 to 14 attendees). Open-ended comments in paper-and-pencil administrations should be transcribed by a neutral third party to prevent handwriting identification.
11. Permissions & Fee and Test Year
The FIRE-ST was developed in 2018 and published in 2020 by Carsten Schulte and Meinald T. Thielsch in cooperation with the Institut der Feuerwehr Nordrhein-Westfalen (IdF NRW). The instrument is placed in the public domain for research, academic, and organizational quality improvement purposes under appropriate citation. Commercial redistribution or incorporation into proprietary commercial learning platforms without author consent is prohibited. Training institutions and researchers can access documentation through the University of Münster Organizational and Business Psychology project repository (FIRE Project Portal).
12. References
- Alliger, G. M., Tannenbaum, S. I., Bennett, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358. https://doi.org/10.1111/j.1744-6570.1997.tb00911.x
- Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105. https://doi.org/10.1111/j.1744-6570.1988.tb00632.x
- Blanchard, P. N., & Thacker, J. W. (2013). Effective training: Systems, strategies, and practices (5th ed.). Pearson.
- Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resources development (pp. 301–319). McGraw-Hill.
- Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publishers.
- Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G. A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138–147). Ablex Publishing.
- Noe, R. A. (2017). Employee training and development (7th ed.). McGraw-Hill Education.
- Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
- Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
- Schulte, C., & Thielsch, M. T. (2019). Feedback-Instrument zur Rettungskräfte-Entwicklung (FIRE). In Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). GESIS. https://doi.org/10.6102/zis269
- Schulte, C., Thielsch, M. T., & Kläpker, L. (2019). Evaluation in der Feuerwehr: Das Feedback-Instrument zur Rettungskräfte-Entwicklung (FIRE). BrandSchutz/Deutsche Feuerwehr-Zeitung, 73(4), 282–286.
- Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
- Tamkin, P., Yarnall, J., & Kerrin, M. (2002). Kirkpatrick and beyond: A review of models of training evaluation (Report 392). Institute for Employment Studies.
- Thielsch, M. T., Busjan, L., & Frerichs, B. (2018). FIRE-P: Feedback-Instrument zur Rettungskräfte-Entwicklung – Prüfungen. Universität Münster.
- Thielsch, M. T., Kläpker, L., & Streppel, D. (2019). FIRE-B: Feedback-Instrument zur Rettungskräfte-Entwicklung – Basisausbildung. Universität Münster.
- Thielsch, M. T., & Weltzin, C. (2013). Evaluation: Verfahren, Fallstricke und Praxistipps für Großgruppen. In M. T. Thielsch & T. Brandenburg (Eds.), Praxis der Wirtschaftspsychologie: Themenfelder und Fallbeispiele (pp. 177–196). MV-Verlag.
13. Items of the Scale
Instruktion
Liebe/r Lehrgangsteilnehmer/in,
wir schätzen es sehr, dass Sie an unserer Befragung teilnehmen. Indem Sie das Seminar hinsichtlich verschiedener Aspekte bewerten, helfen Sie uns, die Qualität der Lehre zu beurteilen und gegebenenfalls zu verbessern. Bitte kreuzen Sie hierzu bei jeder der folgenden Aussagen das Kästchen an, das den Grad Ihrer Zustimmung am besten beschreibt. Es gibt bei dieser Befragung keine richtigen oder falschen Antworten. Vielmehr interessieren wir uns für Ihre ganz persönliche Meinung.
Bitte beachten Sie:
- Machen Sie hinter jeder Aussage jeweils nur ein Kreuz in einem der dafür vorgesehenen Kästchen (und niemals zwischen oder neben den Kästchen).
- Bitte lassen Sie keine Aussagen aus. Wenn eine Aussage für Sie nicht sinnvoll beantwortbar ist, können Sie uns dies durch ein Kreuz in dem entsprechenden Kästchen mitteilen.
- Aus Gründen der besseren Lesbarkeit wird an einigen Stellen nur die männliche Form (Dozent/Teilnehmer) verwendet. Die weibliche Form ist dabei jeweils miteingeschlossen.
- Ihre Teilnahme an der Befragung ist freiwillig und anonym.
Bitte teilen Sie uns bei den folgenden Aussagen mit, wie sehr Sie diesen zustimmen:
Antwortformat (Items 1–20)
7-stufiges Antwortformat mit den Optionen:
- 1 = stimme gar nicht zu
- 2 = stimme nicht zu
- 3 = stimme eher nicht zu
- 4 = neutral
- 5 = stimme eher zu
- 6 = stimme zu
- 7 = stimme vollkommen zu
- [Zusätzliches Feld]: nicht sinnvoll beantwortbar
Skala: Dozentenverhalten
- Der Dozent / Die Dozentin fasste schwierige Sachverhalte prägnant zusammen.
- Der Dozent / Die Dozentin gab nützliches Feedback.
- Der Dozent / Die Dozentin motivierte mich, mich einzubringen.
- Der Dozent / Die Dozentin war am Lernerfolg der Teilnehmer interessiert.
- Der sprachliche Ausdruck des Dozenten / der Dozentin war verständlich.
- Der Dozent / Die Dozentin kannte sich mit den Inhalten der Lehre sehr gut aus.
Skala: Anforderungsniveau
- Der Umfang der zu lernenden Inhalte hat mich überfordert.
- Das Tempo der Stoffvermittlung war zu hoch für mich.
- Die Inhalte der Lehrveranstaltung waren zu schwierig für mich.
Skala: Struktur
- Ich finde, die Lehrveranstaltung war klar strukturiert.
- Ich konnte im Verlauf der Lehrveranstaltung die Gliederung immer nachvollziehen.
- Ich finde, die Lehrveranstaltung gab einen guten Überblick über das Themengebiet.
Skala: Kompetenzerwerb
- Durch meine Teilnahme am Seminar gelingt es mir jetzt besser, Gefahrenlagen frühzeitig zu erkennen.
- Durch meine Teilnahme am Seminar fällt es mir jetzt leichter, Entscheidungen in kritischen Situationen zu treffen.
- Durch meine Teilnahme am Seminar kenne ich meine persönlichen Grenzen besser.
- Durch meine Teilnahme am Seminar traue ich mir besser zu, in stressigen Situationen ruhig zu bleiben.
- Durch meine Teilnahme am Seminar kann ich korrektere und verständlichere Anweisungen geben.
Skala: Transfer
[Instruktion für Items 18–20: Bitte beantworten Sie den folgenden Teil nur, wenn Ihr Seminar praktische Anteile wie z.B. Einsatz- oder Planübungen enthielt.]
- Ich fühle mich auf meinen nächsten Einsatz, den ich leiten werde, sehr gut vorbereitet.
- Durch die praktischen Übungen im Seminar habe ich die nötige Sicherheit gewonnen, einen Einsatz zu leiten.
- Ich kann das Gelernte auf meine Heimatstelle übertragen.
Optionale Items: Globalurteil und Feedback
- Ich würde dem Seminar folgende Schulnote geben:
Antwortformat (Sechsstufiges Format):
1 = sehr gut, 2 = gut, 3 = befriedigend, 4 = ausreichend, 5 = mangelhaft, 6 = ungenügend - Ich würde dieses Seminar weiterempfehlen.
Antwortformat (Dichotom):
[ ] Ja [ ] Nein - Platz für weitere Anmerkungen (Hier können Sie noch Lob, Kritik oder Anregungen zur Ausbildung äußern):
Antwortformat: Offenes Textfeld