Evaluation considers the success of training and is necessary for identifying elements of training processes and training programs that were beneficial and areas for improvement. Literature in this section addresses planning for evaluation, conducting evaluation, and reporting the results of evaluation. Two types of evaluation are highlighted: process evaluation and outcome evaluation. Process evaluation focuses on how well the elements of the training were conducted, while outcome evaluation focuses on whether the training elicited the desired results.
General
- Basarab, Dave. (2011 March). Forecasting the value of training. Performance Improvement, 50(3), 22-27. Available from: http://onlinelibrary.wiley.com/doi/10.1002/pfi.20203/abstract.
- Basarab, Dave. (2011). Predictive evaluation: Ensuring training delivers business and organizational results. San Francisco, CA: Berett-Koehler Publishers, Inc.
- Burkhart, Jennifer. (1996). Evaluating workplace education program effectiveness. Available from: http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED399435.
- Eseryel, Deniz. (2002). Approaches to evaluation of training: Theory and practice. Educational Technology & Society, 5(2), 93-98. Available at: http://www.ifets.info/journals/5_2/eseryel.html.
- Freeman, Michael K. (2003 Winter). Assessing and evaluating continuing education programs: Why and how? Journal of Continuing Higher Education, 51(1), 23-30. Available from: http://www.tandfonline.com/doi/abs/10.1080/07377366.2003.10401211?journalCode=ujch20#.VP8VNMt0zcs.
- Kirwan, Cyril, & Birchall, David. (2006 December). Transfer of learning from management development programmes: Testing the Holton model. International Journal of Training and Development, 10(4), 252-268. Available from: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=947000.
- Leviton, Laura C., Kettel Khan, Laura, Rog, Debra, Dawkins, Nicola, & Cotton, David. (2010 April). Evaluability assessment to improve public health policies, programs, and practices. Annual Review of Public Health, 31, 213-233. Available from: http://www.annualreviews.org/doi/abs/10.1146/annurev.publhealth.012809.103625.
- Patton, Michael Quinn. (2000). Utilization-focused evaluation. In D. L. Stufflebeam, G. F. Madaus, & T. Kellaghan (Eds.), Evaluation models. Boston: Kluwer Academic Publishers.
- Phillips, P. P. (Ed.). (2010). ASTD handbook of measuring and evaluating training. Alexandria, VA: ASTD Press. Available from: https://www.td.org/Publications/Books/ASTD-Handbook-of-Measuring-and-Evaluating-Training.
- Pineda, Pilar. (2010). Evaluation of training in organisations: A proposal for an integrated model. Journal of European Industrial Training, 34(7), 673-693. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/03090591011070789.
- Robertson, M. K., Umble, K. E., & Cervero, R. M. (2003). Impact studies in continuing education for health professions: Update. Journal of Continuing Education in the Health Professions, 23, 146-156. Available from: http://onlinelibrary.wiley.com/doi/10.1002/chp.1340230305/abstract.
- The value of evaluation. (2009). Alexandria, VA: ASTD Press. Available from: http://store.astd.org/Default.aspx?tabid=167&ProductId=20770.
Process Evaluation
- Linnan, Laura, & Steckler, Allan. (2002). Process evaluation for public health interventions and research: An overview. Hoboken, NJ: Jossey-Bass. Available at: http://media.johnwiley.com.au/product_data/excerpt/66/07879597/0787959766.pdf.
- Saunders, Ruth P., Evans, Martin, & Joshi, Praphul. (2005 April). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice, 6(2), 134-147. Available from: http://hpp.sagepub.com/content/6/2/134.short?rss=1&ssource=mfc.
Outcome Evaluation
- Bates, Reid. (2004). A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning, 27, 341–347. Available at: https://aetcnec.ucsf.edu/sites/aetcnec.ucsf.edu/files/resources/A%20critical%20analysis%20of%20evaluation%20practice.pdf.
- Brown, Robert E., & Reed, Celeste Sturdevant. (2002 March). An integral approach to evaluating outcome evaluation training. American Journal of Evaluation, 23(1), 1-17. Available from: http://aje.sagepub.com/content/23/1/1.short.
- Burkett, H. (2005). ROI on a shoestring: Strategies for resource-constrained environments: Measuring more with less (Pt. I). Industrial and Commercial Training, 37(1), 10-17. Available from: http://www.ingentaconnect.com/content/mcb/037/2005/00000037/00000001/art00002?crawler=true.
- Burkett, H. (2005). ROI on a shoestring: Evaluation strategies for resource-constrained environments or ROI on a shoestring: Measuring more with less (Pt. II). Industrial and Commercial Training, 37(2), 97-105. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/00197850510584250.
- Burrow, Jim, & Berardinelli, Paula. (2003). Systematic performance improvement – Refining the space between learning and results. Journal of Workplace Learning, 15(1), 6-13. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/13665620310458776.
- Centers for Disease Control and Prevention. (1999 September). Framework for program evaluation in public health. Morbidity and Mortality Weekly Report. 48(RR-11). Available at: http://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf.
- Chevalier, Roger. (2004 April). The link between learning and performance. Performance Improvement, 43(4), 40-44. Available from: http://onlinelibrary.wiley.com/doi/10.1002/pfi.4140430410/abstract.
- Curran, V. R., & Fleet, L. (2005). A review of evaluation outcomes of web-based continuing medical education. Medical Education, 39(6), 561-567. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2929.2005.02173.x/abstract.
- Dreisinger, M., Leet, T. L., Baker, E. A., Gillespie, K. N., Haas, B., & Brownson, R. C. (2008 March-April). Improving the public health workforce: Evaluation of a training course to enhance evidence-based decision making. Journal of Public Health Management and Practice, 14(2), 138-143. Available from: http://www.ncbi.nlm.nih.gov/pubmed/18287919.
- Elliott, Michael, Dawson, Ray, & Edwards, Janet. (2009). Providing demonstrable return-on-investment for organisational learning and training. Journal of European Industrial Training, 33(7), 657-670. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/03090590910985408.
- Flanagan, Mary Jane. (2010 December). Measuring the effectiveness of L&D. Training Journal, 62-66.
- Fleck, E., & Fyffe, T. (1997 January). Changing nursing practice through continuing education: A tool for evaluation. Journal of Nursing Management, 5(1), 37-41. Available from: http://www.ncbi.nlm.nih.gov/pubmed/9146202.
- Galloway, Dominique L. (2005 April). Evaluating distance delivery and e-learning: Is Kirkpatrick's model relevant? Performance Improvement, 44(4), 21-27. Available at: http://aprender.ead.unb.br/pluginfile.php/66872/mod_folder/content/0/Em%20Ingl%C3%AAs/Galloway_2005_Evaluating_Distance_Delivery_and_E-Learning.pdf?forcedownload=1.
- Gardênia Abbad, Jairo, Borges-Andrade, Eduardo, & Henriques Sallorenzo, Lúcia. (2004). Self-assessment of training impact at work: Validation of a measurement scale. Revista Interamericana de Psicología/Interamerican Journal of Psychology, 38(2), 277-284. Available at: http://www.psicorip.org/Resumos/PerP/RIP/RIP036a0/RIP03829.pdf.
- Green, M. L., Reddy, S. G., & Holmboe, E. (2009 Fall). Teaching and evaluating point of care learning with an Internet-based clinical-question portfolio. The Journal of Continuing Education in the Health Professions, 29(4), 209-219. Available from: http://onlinelibrary.wiley.com/doi/10.1002/chp.20039/abstract. Note: Also relevant for Design.
- Hicks, C., & Hennessy, D. (2001 January). An alternative technique for evaluating the effectiveness of continuing professional development courses for health care professionals: A pilot study with practice nurses. Journal of Nursing Management, 9(1), 39-49. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2834.2001.00203.x/abstract.
- Hites, L. S., Sass, M. M., D'Ambrosio, L., Brown, L. M., Wendelboe, A. M., Peters, K. E. & Sobelson, R. K. (2014). The preparedness and emergency response learning centers: Advancing standardized evaluation of public health Preparedness and Response Trainings. Journal of Public Health Management and Practice, 20(Suppl. 5), S17-S23. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25072484.
- Hodges, T. K. (2002). Linking learning and performance: A practical guide to measuring learning and on-the-job application. Boston, MA: Butterworth-Heinemann.
- Kirkpatrick, Donald L. (1998). Another look at evaluating training programs: Fifty articles from 'Training & Development' and 'Technical Training' magazines cover the essentials of evaluation and return-on-investment. Alexandria, VA: American Society for Training & Development. Available from: http://www.voced.edu.au/content/ngv6445.
- Kirkpatrick, Donald L. (1977 November). Evaluating training programs: Evidence vs. proof. Training and Development Journal, 31(11), 9-12. Available from: http://eric.ed.gov/?id=EJ169223.
- Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). San Francisco: Berrett-Koehler Publishers. Available from: http://www.amazon.com/Evaluating-Training-Programs-Levels-Edition/dp/1576753484.
- Koo, Denise, & Miner, Kathleen. (2010 April). Outcome-based workforce development and education in public health. Annual Review of Public Health, 31, 253-269. Available from: http://www.annualreviews.org/doi/abs/10.1146/annurev.publhealth.012809.103705?url_ver=Z39.88-2003&rfr_dat=cr_pub%3Dpubmed&rfr_id=ori%3Arid%3Acrossref.org&journalCode=publhealth. Note: Also relevant for Design.
- Kraiger, Kurt, Ford, J. Kevin, & Salas, Eduardo. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78(2), 311-328. Available at: http://www.researchgate.net/profile/Kurt_Kraiger/publication/201381982_Application_of_cognitive_skill-based_and_affective_theories_of_learning_outcomes_to_new_methods_of_training_evaluation/links/0912f510954e870ca3000000.pdf.
- Liebermann, Susanne, & Hoffmann, Stefan. (2008 June). The impact of practical relevance on training transfer: Evidence from a service quality training program for German bank clerks. International Journal of Training and Development, 12(2), 74-86. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2419.2008.00296.x/abstract. Note: Also relevant for Deliver.
- Mainor, A., Leeman, J., Sommers, J., Heiser, C., Gonzales, C., Farris, R.P., Ammerman, A. (2014). A systematic approach to evaluating public health training: The obesity prevention in public health course. Journal of Public Health Management and Practice, 20(6), 647-653. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24402433. Note: Also relevant for Design.
- Menix, K. D. (2007 September-October). Evaluation of learning and program effectiveness. Journal of Continuing Education in Nursing, 38(5), 201-208, quiz 209-210. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17907664.
- Murdock, J. E., & Neafsey, P. J. (1995 July-August). Self-efficacy measurements: An approach for predicting practice outcomes in continuing education? Journal of Continuing Education in Nursing, 26(4), 158-165. Available from: http://www.ncbi.nlm.nih.gov/pubmed/7601961.
- Olmstead, T., Carroll, K. M., Canning-Ball, M., & Martino, S. (2011 July 1). Cost and cost-effectiveness of three strategies for training clinicians in motivational interviewing. Drug and Alcohol Dependence, 116(1-3), 195-202. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3105154/.
- Olson, D. K., Scheller, A. & Wey, A. (2014). Using gaming simulation to evaluate bioterrorism and emergency readiness training. Journal of Public Health Management and Practice, 20(Suppl. 5), S52-S60. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848272/.
- Parry, Scott B. (1997). Evaluating the impact of training: A collection of tools and techniques. Alexandria, VA: American Society for Training & Development. Available from: http://books.google.com/books/about/Evaluating_the_Impact_of_Training.html?id=pepqM4sMpGcC.
- Parry, Scott B. (1996 May). Measuring training's ROI. Training & Development, 50(5), 72-77. Available from: https://www.questia.com/magazine/1G1-18441767/measuring-training-s-roi.
- Peak, D., & Berge, Z. L. (2006). Evaluation and eLearning. Turkish Online Journal of Distance Education, 7(1), 124-131. Available from: https://tojde.anadolu.edu.tr/tojde21/pdf/article_11.pdf.
- Peddecord, K. M., Holsclaw, P., Jacobson, I.G., Kwizera, L., Rose, K., Gersberg, R., & Macias-Reynolds, V. (2007). Nationwide satellite training for public health professionals: Web-based follow-up. Journal of Continuing Education in the Health Professions, 27(2), 111-117. Available from: http://onlinelibrary.wiley.com/doi/10.1002/chp.109/abstract.
- Phillips, Jack J. (1996 April). How much is the training worth? Training and Development, 50(4), 20-24. Available from: https://www.questia.com/magazine/1G1-18434799/how-much-is-the-training-worth.
- Phillips, Jack J. (1996 March). Was it the training? Training and Development, 50(3), 28-32.
- Phillips, Jack J. (1996 February). ROI: The search for best practices. Training and Development, 50(2), 42-47. Available from: https://www.questia.com/magazine/1G1-18156945/roi-the-search-for-best-practices.
- Phillips, J. J. (1983). Handbook of training evaluation and measurement methods. Houston: Gulf Publishing. Available from http://books.google.com/books/about/Handbook_of_training_evaluation_and_meas.html?id=zz0rx2zyuA4C.
- Phillips, J. J., & Phillips, P. P. (2012). 10 steps to successful business alignment. Alexandria, VA: ASTD Press. Available from: https://www.td.org/Publications/Books/10-Steps-to-Successful-Business-Alignment. Note: Also relevant for Design.
- Phillips, J. J., & Phillips, P. P. (2008). Beyond learning objectives: Develop measurable objectives that link to the bottom line. Alexandria, VA: ASTD Press. Available from: https://www.td.org/Publications/Books/Beyond-Learning-Objectives. Note: Also relevant for Design.
- Phillips, J. J. & Phillips, P. P. (1998 July-August). The return-on-investment (ROI) process: Issues and trends. Educational Technology, 38(4), 7-14. Available at: http://www.fasset.org.za/downloads/SDF4_ROI_Training.doc.
- Phillips, P. P. (2011 November 8). Business alignment: The path to meaningful data and a positive ROI. Note: Also relevant for Design.
- Phillips, P. P., & Phillips, J. J. (2007). The value of learning: How organizations capture value and ROI. San Francisco: Pfeiffer. Available from: http://www.wiley.com/WileyCDA/WileyTitle/productCd-0787985325.html.
- Phillips, P. P., & Phillips, J. J. (2005). Return on investment (ROI) basics. Alexandria, VA: ASTD Press. Available from: http://www.amazon.com/Return-Investment-Basics-ASTD-Training/dp/1562864068.
- Pinkerton, S. E. (2004). The financial return on education programs. Journal of Continuing Education in Nursing, 35(6), 244-245. Available from: http://www.ncbi.nlm.nih.gov/pubmed/15584674.
- Renger, R. & Granillo, B. (2014). Lessons learned in testing the feasibility of evaluating transfer of training to an operations setting. Journal of Public Health Management and Practice, 20(Suppl. 5), S30-S36. Available at: http://mwperlc.arizona.edu/sites/mwperlc.arizona.edu/files/Lessons_Learned_in_Testing_the_Feasibility_of.6.pdf.
- Riley, W., Parsons, W., McCoy, K., Burns, D., Anderson, D., Lee, S., & Sainfort, F. (2009 October). Introducing quality improvement methods into local public health departments: Structured evaluation of a statewide pilot project. Health Services Research, 44(5, Pt. 2), 1863-1879. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2758410/.
- Roszkowski, Michael J., & Soven, Margot. (2010). Did you learn something useful today? An analysis of how perceived utility relates to perceived learning and their predictiveness of satisfaction with training. Performance Improvement Quarterly, 23(2), 71-91. Available from: http://onlinelibrary.wiley.com/doi/10.1002/piq.20082/abstract. Note: Also relevant for Deliver.
- Sharp, L. K., & Lipsky, M. S. (2002 Spring). Continuing medical education and attitudes of health care providers toward treating diabetes. Journal of Continuing Education in the Health Professions, 22(2), 103-112. Available from: http://onlinelibrary.wiley.com/doi/10.1002/chp.1340220206/abstract.
- Shrock, S. A., & Coscarelli, W. C. C. (2007). Criterion-referenced test development: Technical and legal guidelines for corporate training (3rd ed.). Alexandria, VA: International Society of Performance Improvement. Available from: http://www.wiley.com/WileyCDA/WileyTitle/productCd-047041040X.html.
- Steensma, Herman, & Groeneveld, Karin. (2010). Evaluating a training using the "Four Levels Model". Journal of Workplace Learning, 22(5), 319-331. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/13665621011053226.
- Strother, Judith. (2002 April). An assessment of the effectiveness of e-learning in corporate training programs. International Review of Research in Open and Distance Learning, 3(1). Available from: http://www.irrodl.org/index.php/irrodl/article/viewArticle/83/160.
- Tamkin, P., Yarnall, J., & Kerrin, M. (2002). Kirkpatrick and beyond: A review of models of training evaluation. IES Report. Grantham, UK: Grantham Book Services. Available from: http://eric.ed.gov/?id=ED470777.
- Tian, J., Atkinson, N. L., Portnoy, B., & Gold, R. S. (2007 Winter). A systematic review of evaluation in formal continuing medical education. Journal of Continuing Education in the Health Professions, 27(1), 16-27. Available from: http://onlinelibrary.wiley.com/doi/10.1002/chp.89/abstract.
- United States Office of Personnel Management. (2011 January). Training evaluation field guide: Demonstrating the value of training at every level. Available at: https://www.opm.gov/policy-data-oversight/training-and-development/reference-materials/training_evaluation.pdf.
- Wang, Greg. (2003 January-February). Valuing learning: The measurement journey. Educational Technology, 43(1), 32-37. Available from: http://eric.ed.gov/?id=EJ664973.
- Westcott-Abudi, Gina. (2008). Using return on investment to evaluate project management training. In 2008 PMI Global Congress Proceedings. Denver, CO. Available at: http://www.ginaabudi.com/articles/using-return-on-investment-to-evaluate-project-management-training/.
- Woltring, Carol, Constantine, Wendy, & Schwarte, Liz. (2003 March/April). Does leadership training make a difference? The CDC/UC Public Health Leadership Institute: 1991-1999. Journal of Public Health Management and Practice, 9(2), 103-122. Available at: http://www.researchandevaluationsystems.com/reports/5_public_health_mgmt_and_practice.pdf.