Kirkpatrick Level 3 (Behavior)

Advancing the public health workforce to achieve organizational excellence

Home

Emergency Preparedness Evaluation Repository

Introduction and Methods Overview

Kirkpatrick Level 3 evaluations are intended to assess the degree to which participants apply to their daily job objectives and skills learned during training (Kirkpatrick, 1996). Specific to disaster preparedness and response, Kirkpatrick Level 3 evaluations are intended to assess behavior change in the field setting. Operational definitions of behavior change and field setting are provided for consideration when Level 3 evaluation is considered:

Behavior change specifically refers to the transfer of knowledge, skills, and abilities as defined within the Public Health Preparedness and Response Core Competency Model from the training environment (e.g., online classroom) to the field setting.

The field setting is any applied environment. The continuum includes:

Work behaviors in everyday venues

Behaviors that emerge during exercises

Behaviors that emerge during participation in a real event

Kirkpatrick Level 3 evaluation of behavior change in the field setting is based on having a sound knowledge of the training objectives, content, techniques used in the training, and findings from prior Level 1 and Level 2 evaluations. A sound understanding of research methods, particularly in the area of tests and measurements, is necessary. Thus, a tactical plan for measuring behavior change in the field setting is critical and should consider several issues when developing a Level 3 evaluation strategy (Hodges, 2002):

The purpose of the assessment
Who will be reviewing the data and making related decisions
The sources of information that will be needed
How the data will be analyzed
How many people should be included in the assessment
The preferred data collection method
Skill and resource limitations

In consideration of these issues, the Preparedness and Emergency Response Learning Centers Evaluation Working Group (WG) developed a table of possible evaluation approaches and sample questions to assess training impact at Level 3. Additionally, the Level 3 sub-WG developed guiding principles for the evaluator to consider when designing Kirkpatrick Level 3 evaluations.

Back | Next