Background: In this article we analyzed the extent of the usage of Theories of Change (TOCs) and causal pathways in the evaluation of immunization programs to identify the challenges to generating evidence on how interventions improve immunization.
Methods: We analyzed the use of the TOC in impact evaluations (IEs) of immunization interventions published after 2010, and its associated articles. The review includes studies from Evidence Gap Map and Yale review that were conducted in May and March of 2020, respectively. We synthesized data on six domains using NVIVO — program theory, context, assumptions, usage of TOC, use in evaluation, and description causal pathways.
Results: Our review included 47 large-scale and 45 small-to medium-scale interventions. Of the included studies, 19% used a TOC, 56% described a causal pathway or used a conceptual diagram with varying degrees of detail, and 25% of the IEs did not provide any information on how their intervention was expected to affect change. Only 19 of the 92 IEs explicitly outlined any assumptions associated with the implementation of the interventions. Forty studies measured the outputs or intermediate outcomes leading to improved immunization coverage.
Conclusion: Future implementers and evaluators need to develop clear TOCs that are based on established theory and have clearly articulated the underlying assumptions. Large-scale health system strengthening initiatives implemented by governments, also need to build TOCs and integrate them into their results frameworks. Additionally, there is a need to combine both impact and process evaluations to understand the how context affects the causal pathways.
BT - Journal of Health, Population and Nutrition DO - 10.1186/s41043-024-00615-2 IS - 1 LA - ENG M3 - Article N2 -Background: In this article we analyzed the extent of the usage of Theories of Change (TOCs) and causal pathways in the evaluation of immunization programs to identify the challenges to generating evidence on how interventions improve immunization.
Methods: We analyzed the use of the TOC in impact evaluations (IEs) of immunization interventions published after 2010, and its associated articles. The review includes studies from Evidence Gap Map and Yale review that were conducted in May and March of 2020, respectively. We synthesized data on six domains using NVIVO — program theory, context, assumptions, usage of TOC, use in evaluation, and description causal pathways.
Results: Our review included 47 large-scale and 45 small-to medium-scale interventions. Of the included studies, 19% used a TOC, 56% described a causal pathway or used a conceptual diagram with varying degrees of detail, and 25% of the IEs did not provide any information on how their intervention was expected to affect change. Only 19 of the 92 IEs explicitly outlined any assumptions associated with the implementation of the interventions. Forty studies measured the outputs or intermediate outcomes leading to improved immunization coverage.
Conclusion: Future implementers and evaluators need to develop clear TOCs that are based on established theory and have clearly articulated the underlying assumptions. Large-scale health system strengthening initiatives implemented by governments, also need to build TOCs and integrate them into their results frameworks. Additionally, there is a need to combine both impact and process evaluations to understand the how context affects the causal pathways.
PB - Springer Science and Business Media LLC PY - 2024 SP - 1 EP - 15 T2 - Journal of Health, Population and Nutrition TI - Analyzing the usage of theories of change for routine immunization programs -- a review of impact evaluations from LMICs UR - https://jhpn.biomedcentral.com/counter/pdf/10.1186/s41043-024-00615-2.pdf VL - 43 SN - 2072-1315 ER -