Evaluation of Fire Service Command Unit Trainings

Meinald T. Thielsch; Dzenita Hadzihalilovic

doi:10.1007/s13753-020-00279-6

Article Contents

Article Navigation > International Journal of Disaster Risk Science > 2020 > 11(3): 300-315

Meinald T. Thielsch, Dzenita Hadzihalilovic. Evaluation of Fire Service Command Unit Trainings[J]. International Journal of Disaster Risk Science, 2020, 11(3): 300-315. doi: 10.1007/s13753-020-00279-6

Citation:

Meinald T. Thielsch, Dzenita Hadzihalilovic. Evaluation of Fire Service Command Unit Trainings[J]. International Journal of Disaster Risk Science, 2020, 11(3): 300-315. doi: 10.1007/s13753-020-00279-6

Citation:

PDF

Evaluation of Fire Service Command Unit Trainings

doi: 10.1007/s13753-020-00279-6

Department of Psychology, University of Münster, 48149 Münster, Germany

Funds:

This study was supported by the State Fire Service Institute NRW with non-monetary resources. The authors particularly thank Patrick Fuchs, Christoph Lamers, and Yannick Ngatchou of the State Fire Service Institute NRW for their helpful support during the evaluation project. Furthermore, we thank Guido Hertel for his valuable feedback on the study as well as Celeste Brennecka for her very helpful comments on earlier versions of this manuscript.

Available Online: 2021-04-26

Abstract

Abstract

The lack of routine and training of command units and emergency managers is among the main causes of suboptimal decisions and could lead to serious consequences. To ensure optimal standards of emergency management training, specific and valid evaluation tools are needed—but are lacking. Thus, the present study's purpose is to validate instruments for the evaluation of tactical and strategic leader trainings, in particular command unit trainings, based on survey data of n=288 German Command Unit members. Resulting questionnaires were named “FIRE-CU” (Feedback Instrument for Rescue forces Education – Command Unit) and “FIRE-CPX” (Feedback Instrument for Rescue forces Education – Command Post eXercise scale). Results of confirmatory factor analyses show a good fit for the postulated four-dimensional structure of process scales in the FIRE-CU (trainer's behavior, structure, overextension, group), for the two-dimensional structure of outcome scales in the FIRE-CU (self-rated competence, transfer), and for the one-dimensional structure of the FIRE-CPX. Further, strong evidence is found for reliability as well as for convergent, divergent, and concurrent validity of both the FIRE-CU and FIRE-CPX. Implications for research and practical application are also discussed to enable broad applicability in various educational programs for public security and crisis management.
- Command post exercise,
- Command staff,
- Crisis management,
- Emergency management,
- Rescue forces,
- Vocational training evaluation

FullText(HTML)

References(61)

References

Alliger, G.M., S.I. Tannenbaum, W. Bennett, H. Traver, and A. Shotland. 1997. A meta-analysis of the relations among training criteria. Personnel Psychology 50(2): 341–358.

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 2014. Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Arthur, W., W. Bennett, P.S. Edens, and S.T. Bell. 2003. Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology 88(2): 234–245.

Badke-Schaub, P., G. Hofinger, and K. Lauche (eds.). 2012. Human factors. Berlin: Springer.

Baran, B.E., and C.W. Scott. 2010. Organizing ambiguity: A grounded theory of leadership and sensemaking within dangerous contexts. Military Psychology 22(sup1): S42–S69.

Berlin, J.M., and E.D. Carlström. 2014. Collaboration exercises – The lack of collaborative benefits. International Journal of Disaster Risk Science 5(3): 192–205.

Blanchard, P.N., and J.W. Thacker. 2013. Effective training: Systems, strategies, and practices, 5th edn. Boston: Pearson.

Boon, A., E. Raes, E. Kyndt, and F. Dochy. 2013. Team learning beliefs and behaviours in response teams. European Journal of Training and Development 37(4): 357–379.

Cannon-Bowers, J.A., E. Salas, and S. Converse. 2013. Shared mental models in expert team decision making. In Individual and group decision making: Current issues, ed. N.J. Castellan, 221–246. Hoboken: Taylor and Francis.

Clark, L.A., and D. Watson. 1995. Constructing validity: Basic issues in objective scale development. Psychological Assessment 7(3): 309–319.

Dixon, D.P., M. Weeks, R. Boland, and S. Perelli. 2017. Making sense when it matters most: An exploratory study of leadership in extremis. Journal of Leadership & Organizational Studies 24(3): 294–317.

Evers, A. 2001. The revised Dutch rating system for test quality. International Journal of Testing 1(2): 155–182.

Feuerwehr-Dienstvorschrift (FwDV) 2. 2012. Fire service regulations 2: Training of volunteer fire service (Ausbildung der Freiwilligen Feuerwehr). http://www.bbk.bund.de/SharedDocs/Downloads/BBK/DE/FIS/DownloadsRechtundVorschriften/Volltext_Fw_Dv/FwDV_2_Stand_Jan2012.pdf?__blob=publicationFile. Accessed 8 May 2020.

Gagne, P., and G.R. Hancock. 2006. Measurement model quality, sample size, and solution propriety in confirmatory factor models. Multivariate Behavioral Research 41(1): 65–83.

Grohmann, A., and S. Kauffeld. 2013. Evaluating training programs: Development and correlates of the questionnaire for professional training evaluation. International Journal of Training and Development 17(2): 135–155.

Grunnan, T., and H. Fridheim. 2017. Planning and conducting crisis management exercises for decision-making: The do’s and don’ts. EURO Journal on Decision Processes 5(1–4): 79–95.

Hagemann, V., A. Kluge, and J. Greve. 2012. Measuring the effects of team resource management training for the fire service. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 56(1): 2442–2446.

Hagemann, V., A. Kluge, and S. Ritzmann. 2012. Flexibility under complexity: Work contexts, task profiles and team processes of high responsibility teams. Employee Relations 34(3): 322–338.

Harrell Jr, F.E. 2018. Hmisc: Harrell miscellaneous (Version 4.1–1). https://CRAN.R-project.org/package=Hmisc. Accessed 8 May 2020.

Haus, M., C. Adler, M. Hagl, M. Maragkos, and S. Duschek. 2016. Stress and stress management in European crisis managers. International Journal of Emergency Services 5(1): 66–81.

Heath, R. 1998. Dealing with the complete crisis – The crisis management shell structure. Safety Science 30(1–2): 139–150.

Hofinger, G., and R. Heimann (eds.). 2016. Handbook staff work (Handbuch Stabsarbeit). Berlin: Springer.

Hu, L.-T., and P.M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6(1): 1–55.

Irwing, P., and D.J. Hughes. 2018. Test development. In The Wiley handbook of psychometric testing, ed. P. Irwing, T. Booth, and D.J. Hughes, 3–48. Chichester, UK: John Wiley and Sons.

Jäger, R. 2004. Construction of a rating scale with smilies as symbolic marks (Konstruktion einer Ratingskala mit Smilies als symbolische Marken). Diagnostica 50(1): 31–38.

Kaplan, D., and R. Pruim. 2018. ggformula. https://CRAN.R-project.org/package=ggformula. Accessed 8 May 2020.

Karsten, A. 2012. Leading an operational-tactical unit: The tasks of the head of a command unit at D level in civil protection (Leiten eines operativ-taktischen Stabes: Die Aufgaben der Leiterin/des Leiters eines operativ-taktischen Stabes der Führungsstufe D im Bevölkerungsschutz). Bevölkerungsschutz 3: 16–19.

Kirkpatrick, D.L. 1979. Techniques for evaluating training programs. Training and Development Journal 33: 78–92.

Kirkpatrick, D.L., and J.D. Kirkpatrick. 2006. Evaluating training programs: The four levels, 3rd edn. San Francisco: Berrett-Koehler.

Kraiger, K., J.K. Ford, and E. Salas. 1993. Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. The Journal of Applied Psychology 78(2): 311–328.

Lamers, C. 2016. Command units in civil protection: History, analysis and suggestions for optimisation (Stabsarbeit im Bevölkerungsschutz: Historie, Analyse und Vorschläge zur Optimierung). Edewecht: S+K Verlagsgesellschaft Stumpf+Kossendey mbH.

Mohammed, S., L. Ferzandi, and K. Hamilton. 2010. Metaphor no more: A 15-year review of the team mental model construct. Journal of Management 36(4): 876–910.

Nunnally, J.C. 1978. Psychometric theory, 2nd edn. New York: McGraw-Hill.

Paechter, M., B. Maier, and D. Macher. 2011. Evaluation of university teaching by means of assessments of subjective competence acquisition (Evaluation universitärer Lehre mittels Einschätzungen des subjektiven Kompetenzerwerbs). Psychologie in Erziehung und Unterricht 58(2): Article 128.

R Core Team. 2018. The R project for statistical computing. http://www.R-project.org/. Accessed 2 May 2020.

Revelle, W. 2018. Psych: Procedures for psychological, psychometric, and personality research. https://CRAN.R-project.org/package=psych. Accessed 2 May 2020.

Rindermann, H. 2001. Student assessment of courses – Object of research and implications (Die studentische Beurteilung von Lehrveranstaltungen – Forschungsgegenstand und Implikationen). In Evaluation of university teaching: Between quality management and self purpose (Evaluation universitärer Lehre: Zwischen Qualitätsmanagement und Selbstzweck), ed. C. Spiel, 61–88. Münster: Waxmann.

Rindermann, H., and M. Amelang. 1994. The Heidelberg Inventory for Course Evaluation (HILVE). Instruction manual (Das Heidelberger Inventar zur Lehrveranstaltungs-Evaluation (HILVE). Handanweisung). Heidelberg: Asanger.

Ritzmann, S., V. Hagemann, and A. Kluge. 2014. The Training Evaluation Inventory (TEI) – Evaluation of training design and measurement of training outcomes for predicting training success. Vocations and Learning 7(1): 41–73.

Röseler, S., H. Thölking, M. Hagel, and M.T. Thielsch. 2020. Feedback Instrument for Rescue forces Education – Mission exercises (FIRE-E) (Feedback-Instrument zur Rettungskräfte-Entwicklung – Einsatzübungen). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis282.

Rosseel, Y. 2012. Lavaan: An R package for structural equation modeling. Journal of Statistical Software 48(2): 1–36.

Salas, E., and J.A. Cannon-Bowers. 2001. The science of training: A decade of progress. Annual Review of Psychology 52: 471–499.

Schaub, H. 2016. Command units in the German army (Militärische Stäbe in der Bundeswehr). In Handbook staff work (Handbuch Stabsarbeit), ed. G. Hofinger, and R. Heimann, 33–38. Berlin: Springer.

Schermelleh-Engel, K., H. Moosbrugger, and H. Müller. 2003. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research 8(2): 23–74.

Schönbrodt, F.D., and M. Perugini. 2013. At what sample size do correlations stabilize? Journal of Research in Personality 47(5): 609–612.

Schulte, N., and M.T. Thielsch. 2019. Evaluation of firefighter leadership trainings. International Journal of Emergency Services 8(1): 34–49.

Scriven, M. 1999. The nature of evaluation part I: Relation to psychology. Practical Assessment, Research & Evaluation 6(11). https://doi.org/10.7275/egax-6010.

Sjoberg, M., C. Wallenius, and G. Larsson. 2011. Leadership in complex, stressful rescue operations: A quantitative test of a qualitatively developed model. Disaster Prevention and Management: An International Journal 20(2): 199–212.

Sommer, M., and O. Njå. 2011. Learning amongst Norwegian fire-fighters. Journal of Workplace Learning 23(7): 435–455.

Thielsch, M.T., J.N. Busjan, and K. Frerichs. 2018. Feedback Instrument for Rescue forces Education – Examinations (FIRE-P) (Feedback-Instrument zur Rettungskräfte-Entwicklung – Prüfungen). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). http://doi.org/10.6102/zis260.

Thielsch, M.T., L. Kläpker, and L. Streppel. 2019. Feedback Instrument for Rescue forces Education – Basic Training (FIRE-B) (Feedback-Instrument zur Rettungskräfte-Entwicklung – Basisausbildung). Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis266.

Thieme, U., and G. Hofinger, G. 2012. Command units and “Standing staffs” in the police: Security through professionalisation (Stabsarbeit und»Ständige Stäbe«bei der Polizei: Sicherheit durch Professionalisierung). In Human factors, ed. P. Badke-Schaub, G. Hofinger, and K. Lauche, 275–291. Berlin: Springer.

Thompson, B. 2004. Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.

Trizano-Hermosilla, I., and J.M. Alvarado. 2016. Best alternatives to Cronbach’s alpha reliability in realistic conditions: Congeneric and asymmetrical measurements. Frontiers in Psychology 7: Article 769.

Useem, M., J. Cook, and L. Sutton. 2005. Developing leaders for decision making under stress: Wildland firefighters in the south canyon fire and its aftermath. Academy of Management Learning and Education 4(4): 461–485.

Uttl, B., C.A. White, and D.W. Gonzalez. 2017. Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation 54: 22–42.

Van Buuren, S., and K. Groothuis-Oudshoorn. 2011. Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software 45(3): 1–67.

Van den Bossche, P., W.H. Gijselaers, M. Segers, and P.A. Kirschner. 2006. Social and cognitive factors driving teamwork in collaborative learning environments. Small Group Research 37(5): 490–521.

Wickham, H. 2011. Plyr: The Split-Apply-Combine Strategy for data analysis. Journal of Statistical Software 40(1): 1–29.

Wroblewski, A., and A. Leitner. 2009. Between scientific standards and claims to efficiency: Expert interviews in programme evaluation. In Research methods series. Interviewing experts, ed. A. Bogner, B. Littig, and W. Menz, 235–251. Basingstoke: Palgrave Macmillan.

Wybo, J.L., and K.M. Kowalski. 1998. Command centers and emergency management support. Safety Science 30(1–2): 131–138.

Relative Articles

Supplements(0)

Cited By

Proportional views