Skip to main content

Comparative Effectiveness of Early Childhood Interventions (public)

Research reportView only

Loading... AlertsChat

September 12, 2025

What are the comparative effect sizes of early childhood educational interventions across different age groups and learning domains?

Early childhood educational interventions demonstrate varying effectiveness across domains, producing effect sizes of 0.10-1.45 standard deviations for cognitive outcomes, 0.18-1.22 for mathematics, 0.10-0.44 for language (up to 1.10 for literacy), and 0.19-0.56 for socioemotional and motor development, with comparable benefits observed across preschool and infant age groups.

Abstract

Early childhood educational interventions yield effect sizes that vary by learning domain, intervention modality, and age group. Studies targeting cognitive outcomes report gains ranging from 0.10 to 1.45 standard deviations—with intensive, multi-year programs (e.g., Ramey and Ramey, 2023) achieving the largest benefits—while language-focused programs generally produce improvements of 0.10 to 0.44 standard deviations, with some literacy interventions reaching 1.10. Mathematics interventions typically exhibit effects from 0.18 to 1.22 standard deviations, especially when curriculum-based and structured (as in Yıldız et al., 2025 and Clements et al., 2011). Socioemotional and motor development interventions tend to yield modest gains (approximately 0.19–0.56), a range further supported by evaluations of responsive caregiving and physical activity programs. Most studies concentrate on preschool-aged children, though interventions for infants show comparable benefits. Greater intervention intensity, duration, and multi-component design are associated with larger effects, according to the reported findings.

Methods

We analyzed 40 sources from an initial pool of 997, using 8 screening criteria. Each paper was reviewed for 6 key aspects that mattered most to the research question. More on methods

Papers identified with Elicit search

n = 997

Papers screened using: Population Age Range, Quantitative Data Availability, Study Design Rigor, Educational Setting, Learning Domain Focus, Typical Development Population, Intervention Component, Educational vs. Medical Focus

n = 997

Papers screened out

n = 957

Papers included for extraction

n = 40

Press enter or space to select a node.You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.

Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.

Paper search

Using your research question “What are the comparative effect sizes of early childhood educational interventions across different age groups and learning domains?”, we searched across over 126 million academic papers from the Semantic Scholar corpus. We retrieved the 997 papers most relevant to the query.

Screening

We screened in sources based on their abstracts that met these criteria:

• Population Age Range: Does the study focus on children aged 0-8 years (early childhood period)?
• Quantitative Data Availability: Does the study report quantitative effect sizes or provide sufficient quantitative data to calculate effect sizes?
• Study Design Rigor: Is the study a randomized controlled trial, quasi-experimental study, or controlled study with comparison groups?
• Educational Setting: Is the intervention implemented in early childhood educational settings (e.g., preschools, kindergartens, childcare centers, or home-based educational programs)?
• Learning Domain Focus: Does the intervention target specific learning domains (cognitive, language, literacy, numeracy, social-emotional, or motor skill development)?
• Typical Development Population: Does the study include typically developing children (rather than focusing solely on children with diagnosed developmental disabilities or special needs)?
• Intervention Component: Does the study include an intervention component (rather than being purely observational)?
• Educational vs. Medical Focus: Is the intervention primarily educational in nature (rather than primarily focused on medical or therapeutic treatments)?

We considered all screening questions together and made a holistic judgement about whether to screen in each paper.

Data extraction

We asked a large language model to extract each data column below from each paper. We gave the model the extraction instructions shown below for each column.

• Study Design Type:

Identify the specific type of study design used. Look in the methods section for precise description. Categorize as:

• Randomized Controlled Trial (RCT)
• Quasi-experimental study
• Comparative study
• Longitudinal study

If multiple design elements are present, list all that apply. If uncertain, note “unclear” and provide a brief explanation of why.

• Intervention Type and Components:

Comprehensively list all components of the educational intervention:

• Primary pedagogical approach (e.g., direct instruction, inquiry-based)
• Specific learning domains targeted
• Additional support services (if any)
• Delivery method (group size, individual/group)

Be as detailed as possible. If multiple intervention components exist, list each separately. Include frequency, duration, and specific content of interventions where mentioned.

• Participant Demographics:

Extract the following participant details:

• Age range of children
• Socioeconomic status (if reported)
• Risk factors or special circumstances
• Total sample size
• Gender distribution (if reported)

Use exact numbers or percentages from the study. If a range is given, record the full range. If any demographic information is not reported, clearly mark as “NR” (not reported).

• Outcome Domains and Measurement:

List all primary outcome domains measured:

• Cognitive development
• Language development
• Motor development
• Socioemotional development
• Academic achievement
• Other specific domains

For each domain:

• Note specific measurement tools/instruments used
• Record effect sizes if provided
• Indicate statistical significance
• Note follow-up time points for measurements

If multiple measurements exist for a domain, list all with their respective details.

• Long-term Outcomes:

Identify and extract any long-term outcomes measured beyond immediate child development:

• Educational attainment
• Employment
• Social relationships
• Health indicators
• Economic outcomes

Record:

• Specific outcome measured

• Age/time point of measurement

• Effect size or key findings

• Statistical significance (if reported)

• Geographical and Contextual Factors:

Extract:

• Country or region of study
• Income level classification (low, middle, high-income)
• Urban/rural setting
• Specific contextual challenges or environmental factors

If multiple contexts are represented, list each separately. Use the study’s own classification or standard international definitions for income levels.

Results

Characteristics of Included Studies

Study

Intervention Type

Age Group

Learning Domain

Study Design

Full text retrieved

Love et al., 2013

Direct coaching, video modeling, self-reflection, resource/referral services

Prenatal to 5 years

Cognitive, language, attention, behavior, health, parenting, mental health, employment

Randomized controlled trial (RCT), Longitudinal, Comparative

Yes

Meghir et al., 2023

Early stimulation (home-based), enhanced preschool (center-based)

7-16 months to ~4 years

Cognition, language, executive function, school readiness

Randomized controlled trial (RCT), Longitudinal

No

Olive et al., 2023

Physical activity intervention (AEL)

3-5 years

Executive function, expressive vocabulary, motor skills

Randomized controlled trial (RCT)

Yes

Camilli et al., 2010

Direct instruction, inquiry-based, teacher/small-group

Pre-Kindergarten

Cognitive, social, school progress

Comparative, Quasi-experimental, Randomized controlled trial (RCT)

No

Campbell et al., 2001

Full-time, high-quality educational child care

Infancy to 21 years

Cognitive, academic (reading, math)

Randomized controlled trial (RCT), Longitudinal

No

Campbell and Ramey, 1994

Infant/preschool/primary school educational treatment

Infancy to 12 years

Cognitive, academic achievement

Randomized controlled trial (RCT), Longitudinal

Yes

Reynolds et al., 2001

Comprehensive education, family, health services

3-9 years, followed to 20

Academic, social, educational attainment, crime

Longitudinal, Quasi-experimental, Comparative

Yes

Blewitt et al., 2018

Universal curriculum-based social-emotional learning (SEL)

2-6 years

Social, emotional, behavioral, early learning

Randomized controlled trial (RCT), Quasi-experimental

Yes

Jeong et al., 2021

Parenting interventions (responsive caregiving, stimulation, etc.)

Prenatal to 3 years

Cognitive, language, motor, socioemotional, behavior, attachment

Randomized controlled trial (RCT)

Yes

Cahoon et al., 2023

Home-based literacy/math interventions

3.07–5.32 years

Literacy, mathematics

Randomized controlled trial (RCT), Quasi-experimental

Yes

toof

Pageof

View 30 more rows

Intervention Types:

• Curriculum-based interventions:13 studies (including literacy, math, social-emotional learning, spatial skills, etc.).
• Center-based early childhood education/preschool programs:10 studies.
• Home-based or parenting interventions:7 studies.
• Multi-component interventions (education plus health, nutrition, or social services):6 studies.
• Physical or motor skill interventions:3 studies.
• Technology-based interventions (educational apps, computer-supported learning):3 studies.
• Policy or large-scale program evaluations:3 studies.

Age Groups:

• 25 studies targeted preschool-aged children (3-5/6 years).
• 15 studies included prenatal or infancy (0-2 years).
• 5 studies included early primary ages (6-8 years).
• 7 studies followed participants into adolescence or adulthood.
• We didn’t find mention of the age group in 7 studies.

Learning Domains:

• Language and literacy outcomes: 22 studies.
• Cognitive or academic outcomes: 20 studies.
• Math outcomes: 14 studies.
• Social-emotional or behavioral outcomes: 15 studies.
• Motor or physical development outcomes: 5 studies.
• Health outcomes: 4 studies.
• Economic or employment outcomes: 5 studies.

Study Designs:

• 26 studies used randomized controlled trial (RCT) designs.
• 14 studies used quasi-experimental designs.
• 10 studies used comparative designs.
• 17 studies included longitudinal follow-up.
• 6 studies were systematic reviews or meta-analyses.

Effects

Effect Sizes by Age Group

Study

Age Group

Learning Domain

Effect Size Range

Number of Studies

Love et al., 2013

1-5 years

Cognitive, language, socioemotional

0.10–0.20 (cognitive/language), 0.31–0.38 (engagement)

1

Meghir et al., 2023

7-48 months

IQ, school readiness

0.13–0.24 standard deviations

1

Olive et al., 2023

3-5 years

Executive function, vocabulary

Cohen’s d=0.24–0.29

1

Camilli et al., 2010

Pre-Kindergarten

Cognitive, social

Largest for cognitive, no mention found

1

Campbell et al., 2001

3-21 years

Cognitive, academic

Moderate-large, no mention found

1

Campbell and Ramey, 1994

Infancy-12 years

Cognitive, academic

Maintained advantage, no mention found

1

Reynolds et al., 2001

3-20 years

Educational, social

11.2% increase in high school completion, 8.2% decrease in arrest

1

Blewitt et al., 2018

2-6 years

Socioemotional, cognitive

Cohen’s d=0.18–0.54

1

Jeong et al., 2021

0-3 years

Cognitive, language, motor, socioemotional

Standardized mean difference (SMD)=0.19–0.56

1

Cahoon et al., 2023

3-5 years

Literacy, math

Cohen’s d=0.10 (literacy), 0.18 (math)

1

toof

Pageof

View 30 more rows

Learning Domains Targeted:

• Mathematics: 13 studies.
• Cognitive outcomes: 12 studies.
• Language outcomes: 11 studies.
• Socioemotional or emotional outcomes: 6 studies.
• Academic or educational outcomes: 5 studies.
• Literacy and reading outcomes: 5 studies (3 literacy, 5 reading; some overlap).
• Other domains: executive function (2 studies), motor/physical/fine motor skills (4 studies), health (2 studies), social (3 studies), behavior (2 studies), school readiness (1 study), problem-solving/STEM (1 study), spatial skills (1 study), phonological/alphabet (4 studies), pre-literacy (1 study).

Effect Size Reporting:

• Numeric effect size estimates (e.g., standard deviation, Cohen’s d, Hedges’ g, standardized mean difference, correlation coefficient, or percentage) were found in 26 studies.
• 13 studies reported only qualitative descriptors of effect (e.g., “significant”, “modest”, “moderate-large”).
• Of the 26 studies with numeric effect sizes, 2 reported only percentage or non-standard deviation outcomes (e.g., percentage increase in school completion).
• All included studies reported at least a qualitative or numeric effect size.

Age Groups:

• 16 studies targeted preschool or pre-Kindergarten children (3-5/6 years).
• 6 studies targeted infants (0-2 years).
• 3 studies targeted early childhood (0-6/8 years).
• 8 studies included school-age children (6+ years), with some spanning into adolescence or adulthood.
• We didn’t find mention of the age group in 7 studies.

Effect Sizes by Learning Domain

Cognitive Development:

• Effect sizes reported in the included studies range from 0.10–1.45 standard deviations, with larger effects observed in intensive, multi-year interventions (e.g., as reported by Ramey and Ramey, 2023).
• Meta-analyses included in this review (Jeong et al., 2021; Blewitt et al., 2018) reported pooled standardized mean differences of 0.18–0.32 for cognitive and early learning outcomes.

Language Development:

• Effect sizes typically ranged from 0.10–0.44 standard deviations, with some studies reporting larger effects for emergent literacy (Barnett et al., 2018: 1.10).
• Computer-supported and storybook-based interventions yielded small to moderate effects (Verhoeven et al., 2020: 0.28; Carolin and Fardzadeh, 2018: 0.21–0.27).

Mathematics:

• Effect sizes ranged from 0.18–1.22 standard deviations, with the largest effects in structured, curriculum-based interventions (Yıldız et al., 2025; Clements et al., 2011).
• Meta-analyses (Wang et al., 2016; Malofeeva, 2005) reported moderate effects (0.43–0.68; 0.47).

Socioemotional Development:

• Effect sizes were generally small to moderate (Blewitt et al., 2018: Cohen’s d=0.19–0.54; Jeong et al., 2021: standardized mean difference=0.19).
• Classroom-wide and universal social-emotional learning interventions showed consistent, though modest, benefits.

Motor Development:

• Physical activity and fine motor interventions yielded moderate effects (Grady et al., 2024: standardized mean difference=0.19–0.54; Strooband et al., 2020: moderate).

Academic Achievement:

• Reading and math achievement effect sizes ranged from 0.10–0.60 standard deviations, with larger effects in high-intensity or multi-year programs.
• Long-term follow-up studies (Reynolds et al., 2001; Campbell and Ramey, 1995) reported sustained academic benefits.

Other Domains:

• Executive function, school readiness, spatial skills, and problem-solving showed moderate to large effects in targeted interventions (Meghir et al., 2023; Yang et al., 2020; Burns et al., 2025).

Methodological Factors Influencing Effect Sizes

• Study Design:In the included studies, randomized controlled trials and large-scale longitudinal studies tended to report more robust and reliable effect sizes, while quasi-experimental and comparative studies appeared more susceptible to bias.
• Sample Size:Larger studies and meta-analyses provided more precise estimates, though heterogeneity remained a concern.
• Intervention Intensity and Duration:More intensive, longer-duration, and multi-component interventions were associated with larger and more sustained effects.
• Measurement Tools:Use of standardized, validated instruments enhanced comparability; studies using researcher-developed measures often reported larger effects.
• Fidelity and Implementation:Higher fidelity and professional development for educators were associated with greater effects (as reported by Carolin and Fardzadeh, 2018).
• Contextual Moderators:Socioeconomic status, country income level, and risk status of participants moderated effect sizes, with disadvantaged and low- and middle-income country populations often showing larger gains.

Generalizability and Study Context

• Geographical Diversity:The included studies spanned high-, middle-, and low-income countries, with interventions adapted to local contexts.
• Population Diversity:Most studies focused on low-income or high-risk populations, which may limit generalizability to more advantaged groups.
• Setting:Interventions were delivered in a variety of settings, including center-based, home-based, community, and digital environments. The context of delivery influenced both feasibility and effectiveness, as reported in several studies.
• Long-term Outcomes:Sustained benefits were more likely in intensive, multi-year interventions, but evidence was limited by the small number of long-term follow-up studies.
• Adverse Effects:Few studies reported adverse effects. One study (Love et al., 2013) noted increased parent-reported aggression with formal program participation. We didn’t find mention of other adverse effects in the available full texts or abstracts.

Limitations:

• Heterogeneity in intervention types, outcome measures, and reporting limited direct comparison across studies.
• Some studies lacked detailed reporting of effect sizes, statistical significance, or confidence intervals.
• Risk of bias was present in studies with small samples, non-randomized designs, or low fidelity.
• Generalizability was limited by the focus on high-risk and low-income populations in many studies.

References

Gregory Camilli, Sadako Vargas, S. Ryan, W. Barnett\ (2010).Meta-Analysis of the Effects of Early Education Interventions on Cognitive and Social Development. Teachers College Record

F. Campbell, E. Pungello, S. Miller-Johnson, M. Burchinal, C. Ramey\ (2001).The development of cognitive and academic abilities: growth curves from an early childhood educational experiment. Developmental Psychology

F. Campbell, C. Ramey\ (1994).Effects of early intervention on intellectual and academic achievement: a follow-up study of children from low-income families. Child Development

A. Reynolds, Judy A Temple, Dylan L. Robertson, Emily A. Mann\ (2001).Long-term effects of an early childhood intervention on educational achievement and juvenile arrest: A 15-year follow-up of low-income children in public schools. Journal of the American Medical Association (JAMA)

C. Blewitt, M. Fuller‐Tyszkiewicz, A. Nolan, Heidi J Bergmeier, D. Vicary, and 4 more\ (2018).Social and Emotional Learning Associated With Universal Curriculum-Based Interventions in Early Childhood Education and Care Centers. JAMA Network Open

Joshua Jeong, E. Franchett, C. V. Ramos de Oliveira, K. Rehmani, A. Yousafzai\ (2021).Parenting interventions to promote early child development in the first three years of life: A global systematic review and meta-analysis. PLoS Medicine

Abigail Cahoon, Carolina Jiménez Lira, Nancy Estévez Pérez, Elia Verónica Benavides Pando, Yanet Campver García, and 2 more\ (2023).Meta-Analyses and Narrative Review of Home-Based Interventions to Improve Literacy and Mathematics Outcomes for Children Between the Ages of 3 and 5 Years Old. Review of Educational Research

S. Ramey, Craig T. Ramey\ (2023).Early Childhood Education that Promotes Lifelong Learning, Health, and Social Well-being: The Abecedarian Project and its Replications. Medical Research Archives

F. Campbell, C. Ramey\ (1995).Cognitive and School Outcomes for High-Risk African-American Students at Middle Adolescence: Positive Effects of Early Intervention

A. Reynolds, Judy A Temple\ (1998).Extended early childhood intervention and school achievement: age thirteen findings from the Chicago Longitudinal Study. Child Development

D. Clements, J. Sarama, M. E. Spitler, Alissa A. Lange, Christopher B. Wolfe\ (2011).Mathematics Learned by Young Children in an Intervention Based on Learning Trajectories: A Large-Scale Cluster Randomized Trial

Noreen Yazejian, D. Bryant, S. Hans, Diane M. Horm, L. St Clair, and 2 more\ (2017).Child and Parenting Outcomes After 1 Year of Educare. Child Development

A. Grady, Rebecca Lorch, Luke Giles, Hannah Lamont, Amy Anderson, and 6 more\ (2024).The impact of early childhood education and care‐based interventions on child physical activity, anthropometrics, fundamental movement skills, cognitive functioning, and social–emotional wellbeing: A systematic review and meta‐analysis. Obesity Reviews

C. Ramey, F. Campbell, M. Burchinal, M. Skinner, D. Gardner, and 1 more\ (2000).Persistent Effects of Early Childhood Education on High-Risk Children and Their Mothers

Li Luo, B. Reichow, Patricia A. Snyder, J. Harrington, Joy C. Polignano\ (2020).Systematic Review and Meta-Analysis of Classroom-Wide Social–Emotional Interventions for Preschool Children. Topics in Early Childhood Special Education

Aubrey Wang, Janine M. Firmender, Joshua R. Power, James P. Byrnes\ (2016).Understanding the Program Effectiveness of Early Mathematics Interventions for Prekindergarten and Kindergarten Environments: A Meta-Analytic Review

Marit Carolin, Ebad Fardzadeh\ (2018).University of Southern Denmark The Effectiveness of a Large-Scale Language and Preliteracy Intervention The SPELL Randomized Controlled Trial in Denmark

Weipeng Yang, Haidan Liu, N. Chen, Peng Xu, Xunyi Lin\ (2020).Is Early Spatial Skills Training Effective? A Meta-Analysis. Frontiers in Psychology

A. Reynolds, Judy A Temple, Suh-Ruu Ou, Dylan L. Robertson, J. Mersky, and 2 more\ (2007).Effects of a school-based, early childhood intervention on adult health and well-being: a 19-year follow-up of low-income families. Archives of Pediatrics & Adolescent Medicine

L. Verhoeven, M. Voeten, E. V. Setten, E. Segers\ (2020).Computer-supported early literacy intervention effects in preschool and kindergarten: A meta-analysis. Educational Research and Reviews

Howard Goldstein, Lindsey A Peters-Sanders, Keri M. Madsen, Jeffrey M. Williams, J. Drobisz, and 4 more\ (2023).Efficacy of a Supplemental Small-Group Early Literacy Intervention Implemented by Early Childhood Educators. American Journal of Speech-Language Pathology

Karel F B Strooband, M. Rosnay, A. Okely, Sanne L. C. Veldman\ (2020).Systematic Review and Meta-Analyses: Motor Skill Interventions to Improve Fine Motor Development in Children Aged Birth to 6 Years. Journal of Developmental and Behavioral Pediatrics

J. Love, Rachel Chazan-Cohen, H. Raikes, J. Brooks-Gunn\ (2013).What makes a difference: Early Head Start evaluation findings in a developmental context. Monographs of the Society for Research in Child Development

Noreen Yazejian, D. Bryant, Laura J. Kuhn, Margaret R. Burchinal, Diane M. Horm, and 3 more\ (2020).The Educare intervention: Outcomes at age 3. Early Childhood Research Quarterly

Ensar Yıldız, Özge Koca, Şenel Elaldı\ (2025).Effectiveness of Early Intervention Programs in Developing Early Mathematical Skills: A meta-Analysis. Kuramsal Eğitimbilim

A. Feller, Todd Grindal, Luke W. Miratrix, Lindsay C. Page\ (2016).Compared to What? Variation in the Impacts of Early Childhood Education by Alternative Care-Type Settings

C. Meghir, O. Attanasio, Pamela Jervis, Monimalika Day, Prerna Makkar, and 6 more\ (2023).Early Stimulation and Enhanced Preschool: A Randomized Trial. Pediatrics

A. Holla, Magdalena Bendini, Lelys Dinarte, Iva Trako\ (2021).Is Investment in Preprimary Education Too Low? Lessons from (Quasi) Experimental Evidence across Countries. Policy Research Working Papers

Samantha Burns, Sumayya Saleem, Evelyn McMullen, O. Falenchuk, Linda A. White, and 2 more\ (2025).A systematic review and meta‐analysis of approaches to teaching problem‐solving skills in early childhood education and care settings: A focus on science, technology, engineering and mathematics activities. Revista de educación

F. Campbell, C. Ramey\ (1989).Preschool vs. School-Age Intervention for Disadvantaged Children: Where Should We Put Our Efforts?.

James S. Kim, Josh Gilbert, Qun Yu, Ch. Gale\ (2021).Measures Matter: A Meta-Analysis of the Effects of Educational Apps on Preschool to Grade 3 Children’s Literacy and Math Skills. AERA Open

W. Barnett, Kwanghee Jung, A. Friedman-Krauss, E. Frede, M. Nores, and 3 more\ (2018).State Prekindergarten Effects on Early Learning at Kindergarten Entry: An Analysis of Eight State Programs

C. Lonigan, David J. Purpura, S. B. Wilson, P. Walker, Jeanine Clancy-Menchetti\ (2013).Evaluating the components of an emergent literacy intervention for preschool children at risk for reading difficulties. Journal of Experimental Child Psychology

Helen F. Ladd, C. Muschkin, K. Dodge\ (2014).From Birth to School: Early Childhood Initiatives and Third‐Grade Outcomes in North Carolina

Italo Lopez Garcia, U. Saya, Jill E. Luoto\ (2021).Cost-effectiveness and economic returns of group-based parenting interventions to promote early childhood development: Results from a randomized controlled trial in rural Kenya. PLoS Medicine

A. J. Dowd, Ivelina I Borisova, Ali Amente, Alene Yenew\ (2016).Realizing Capabilities in Ethiopia: Maximizing Early Childhood Investment for Impact and Equity

Elena V. Malofeeva\ (2005).Meta -analysis of mathematics instruction with young children

Suzanne M. Winter, D. Sass\ (2011).Healthy & Ready to Learn: Examining the Efficacy of an Early Approach to Obesity Prevention and School Readiness

Corey A. DeAngelis, Heidi Holmes Erickson, Gary W. Ritter\ (2018).What’s the state of the evidence on pre-K programmes in the United States? A systematic review. Educause Review

L. Olive, R. Telford, E. Westrupp, R. Telford\ (2023).Physical activity intervention improves executive function and language development during early childhood: The active early learning cluster randomized controlled trial. Child Development

Download BIBDownload RISDownload TXT

Report

Status

Gather sources

997 sources found

Details

Screen sources

40 sources included

Details

Extract data

240 data points extracted

Details

Generate report

Save PDF

BIBLaTeX, ZoteroRISZotero, MendeleyTXTAPA BibliographyPDFPDFDOCXMicrosoft Word

Chat

Got some follow-up questions?

Sign up or sign in to chat with this report.

Back

Cost-effectiveness and economic returns of group-based parenting interventions to promote early childhood development: Results from a randomized controlled trial in rural Kenya

Italo Lopez Garcia, U. Saya, Jill E. Luoto

PLoS Medicine·

2021·

26 citations

SourceDOI

Plain textPDFFigures

Searching for PDF

Unable to find PDF from source

Back

Study Design Type

Randomized Controlled Trial (RCT)

Intervention Type and Components

- Primary pedagogical approach: Group-based parenting intervention - Specific learning domains targeted: Cognitive, receptive language, socioemotional development - Additional support services: Training for CHVs, small gifts for attendance - Delivery method: Group-only model with 16 fortnightly group sessions; Mixed-delivery model with 12 group sessions and 4 home visits - Frequency: Fortnightly group sessions; Review sessions every fourth session - Duration: 8 months - Specific content: Cognitive and language development measured using Bayley Scales; Socioemotional development measured with Wolke Scale

Participant Demographics

- Age range of children: 6 to 24 months - Socioeconomic status: NR - Risk factors or special circumstances: NR - Total sample size: 1,152 at baseline; 1,070 at endline - Gender distribution: NR

Outcome Domains and Measurement

- Cognitive development: Measured using Bayley Scales of Infant Development, Third Edition; Effect size: Group-only: 0.52 SD (p = 0.001), Mixed-delivery: 0.34 SD (p = 0.021); Follow-up: Endline survey from August 5 to October 31, 2019 - Language development (Receptive): Measured using Bayley Scales of Infant Development, Third Edition; Effect size: Group-only: 0.42 SD (p = 0.017), Mixed-delivery: 0.20 SD (p = 0.17); Follow-up: Endline survey from August 5 to October 31, 2019 - Socioemotional development: Measured using Wolke scale; Effect size: Group-only: 0.23 SD (p = 0.024), Mixed-delivery: 0.22 SD (p = 0.011); Follow-up: Endline survey from August 5 to October 31, 2019

Long-term Outcomes

- Educational attainment: Increased schooling by 0.93 years in the group-only arm and by 0.61 years in the mixed-delivery arm; p < 0.0001 - Economic outcomes: Increased annual long-term wages by 20.6% in the group-only arm and by 13.5% in the mixed-delivery arm; p < 0.0001

Geographical and Contextual Factors

- Country or region of study: Kenya, specifically rural western Kenya - Income level classification: Low- and middle-income country (LMIC) - Urban/rural setting: Rural - Specific contextual challenges or environmental factors: Impoverished rural settings

Background Early childhood development (ECD) programs can help address disadvantages for the 43% of children under 5 in low- and middle-income countries (LMICs) experiencing compromised development. However, very few studies from LMIC settings include information on their program’s cost-effectiveness or potential returns to investment. We estimated the cost-effectiveness, benefit–cost ratios (BCRs), and returns on investment (ROIs) for 2 effective group-based delivery models of an ECD parenting intervention that utilized Kenya’s network of local community health volunteers (CHVs). Methods and findings Between October 1 and November 12, 2018, 1,152 mothers with children aged 6 to 24 months were surveyed from 60 villages in rural western Kenya. After baseline, villages were randomly assigned to one of 3 intervention arms: a group-only delivery model with 16 fortnightly sessions, a mixed-delivery model combining 12 group sessions with 4 home visits, and a control group. At endline (August 5 to October 31, 2019), 1,070 children were retained and assessed for primary outcomes including cognitive and receptive language development (with the Bayley Scales of Infant Development, Third Edition) and socioemotional development (with the Wolke scale). Children in the 2 intervention arms showed better developmental outcomes than children in the control arm, although the group-only delivery model generally had larger effects on children. Total program costs included provider’s implementation costs collected during the intervention period using financial reports from the local nongovernmental organization (NGO) implementer, as well as societal costs such as opportunity costs to mothers and delivery agents. We combined program impacts with these total costs to estimate incremental cost-effectiveness ratios (ICERs), as well as BCRs and the program’s ROI for the government based on predictions of future lifetime wages and societal costs. Total costs per child were US$140 in the group-only arm and US$145 in the mixed-delivery arm. Because of higher intention-to-treat (ITT) impacts at marginally lower costs, the group-only model was the most cost-effective across all child outcomes. Focusing on child cognition in this arm, we estimated an ICER of a 0.37 standard deviation (SD) improvement in cognition per US$100 invested, a BCR of 15.5, and an ROI of 127%. A limitation of our study is that our estimated BCR and ROI necessarily make assumptions about the discount rate, income tax rates, and predictions of intervention impacts on future wages and schooling. We examine the sensitivity of our results to these assumptions. Conclusions To the best of our knowledge, this study is the first economic evaluation of an effective ECD parenting intervention targeted to young children in sub-Saharan Africa (SSA) and the first to adopt a societal perspective in calculating cost-effectiveness that accounts for opportunity costs to delivery agents and program participants. Our cost-effectiveness and benefit–cost estimates are higher than most of the limited number of prior studies from LMIC settings providing information about costs. Our results represent a strong case for scaling similar interventions in impoverished rural settings, and, under reasonable assumptions about the future, demonstrate that the private and social returns of such investments are likely to largely outweigh their costs. Trial registration This trial is registered at ClinicalTrials.gov, NCT03548558, June 7, 2018. American Economic Association RCT Registry trial AEARCTR-0002913.

What did the researchers do and find?

• We calculate the total costs, cost-effectiveness, and compare the benefits versus costs for an ECD parenting intervention recently tested across 60 villages in rural western Kenya involving nearly 1,200 mothers and their young children.

• The program compared 2 different delivery models, one featuring only group visits within villages and a second featuring a combination of group and home visits. Both were delivered by local community health volunteers (CHVs) over 8 months.

• We estimate the opportunity costs for the CHVs and participants in our program under both delivery models and predict the future benefits and costs stemming from participation, such as increases in lifetime wages and schooling costs.

• We find that children experienced large improvements in their cognitive development at very reasonable costs under both delivery models, but the group-only model was the most cost-effective.

• Under reasonable assumptions about the future, we find that long-term program benefits could outweigh its associated costs by a factor of 15, implying a large potential return on investment (ROI) for a government willing to scale this program.

Introduction

Scientific evidence confirms that the first 3 years of life are when risk factors such as inadequate parental stimulation exert the greatest harm on child development and when effective interventions can have the greatest benefit [1,2]. Numerous studies have demonstrated the effectiveness of parenting interventions that promote responsive stimulation to improve early childhood development (ECD) in low-and middle-income country (LMIC) settings among children under 3 years of age [3,4]. However, very few studies include accompanying economic evaluations to provide information about a program's cost-effectiveness or the potential economic returns to scaling these interventions in resource-limited settings [5,6]. For example, while recent meta-analyses have identified more than 40 studies evaluating early childhood stimulation interventions targeted to young children in LMICs [4,7], another recent metaanalysis of economic evaluations of ECD parenting interventions and programs identified just 5 studies that provided information about program costs [8]. Institutions such as the World Health Organization (WHO), United Nations International Children's Emergency Fund (UAU : PleasenotethatUNICEFhasbeendefinedasUnitedNationsInternationalChildrensEmergencyFund NICEF), and the World Bank now advocate filling this gap via collection and sharing of program cost data from evaluations of ECD interventions in LMIC settings [9]. Data on program costs can enable different types of economic evaluations of ECD interventions. Cost-effectiveness analysis (CEA) compares the relative costs of a program to its generated impacts on outcomes and is one of the most important indicators used to inform scalability [10]. However, existing CEAs of ECD interventions from LMICs are more readily available for nutrition or health interventions or for preschool programs focused on older ages, with less attention paid to responsive parenting interventions for young children [4,8,11]. Moreover, to date, the very few parenting interventions including a CEA adopt a provider's costing perspective, thereby excluding any societal costs such as opportunity costs to participants and delivery agents. This risks creating perverse incentives for governments to offload such costs onto users and goes against recommended best practice [12]. Notably, no CEA studies are available from sub-Saharan Africa (SSA), where the coverage of ECD programs for young children is lowest, and supporting evidence to expand coverage of responsive parenting interventions is arguably most needed [10].

A more comprehensive economic evaluation may also include a benefit-cost analysis (BCA), which compares the monetized expected program benefits relative to its costs, and can better inform policy on the potential benefits to society of scaling a program. Yet BCAs are even more rare than CEAs in the evaluation of ECD parenting programs in LMICs [13], likely because it can be difficult to estimate long-term benefits without making a number of assumptions (or waiting many years to measure those benefits). A recent systematic review identified only 3 evaluations of ECD parenting programs focused on young children from LMICs that provided benefit-cost ratios (BCRs) [8]. Again, none are from SSA.

In addition to a lack of economic evaluations of ECD parenting programs, there has been growing interest in understanding the delivery format that is most scalable to reach young children before age 3 in LMICs [14,15]. ECD parenting programs that have been shown to be effective are typically delivered in one of 2 formats: individual home visits or group sessions held in the village or at local health clinics. Home visits have the advantage of offering personalized coaching, demonstration, and feedback, but are time and labor intensive and hence expensive to deliver at scale. Group sessions are less labor intensive and may be relatively less expensive to deliver, can encourage peer support, and can help modify group norms for child rearing, but may be weak to overcome personal barriers to behavioral change [3]. Knowing the cost-effectiveness of competing delivery models can help policymakers select the model that is most scalable for the local context.

The aims of this study are to perform an economic evaluation of an effective ECD parenting intervention targeted to families with children under age 3 in SSA that can contribute new insights about the relative cost-effectiveness of 2 different delivery models and to estimate their potential BCRs and returns to investment.

Background of research trial

The intervention, named Msingi Bora ("Good Foundation" in Swahili), was a group-based parenting intervention implemented among rural Kenyan households with children aged 6 to 24 months at baseline that significantly improved child cognitive, receptive language, and socioemotional development among treated families [15]. The cluster randomized controlled effectiveness trial of Msingi Bora was conducted across 60 rural villages in western Kenya in the subcounties of East and South Rachuonyo within Homa Bay county and Sabatia subcounty within Vihiga county. The trial featured 3 study arms, each 20 villages in size: (1) a group-only delivery model with 16 fortnightly group sessions; (2) a mixed-delivery model combining 12 group sessions with 4 home visits; and (3) a control group that received no program under the status quo. Eligible participants within villages were mothers or other female primary caregivers aged 15 and over with a child between 6 and 24 months at recruitment without signs of severe mental or physical impairment. All participants provided written informed consent at the time of data collection. At baseline (October 1 to November 12, 2018), we collected household sociodemographic information and measures of child developmental outcomes from 1,152 eligible households. The 60 villages were then randomly assigned to one of the 3 study arms. The intervention lasted from mid-November 2018 to mid-July 2019 and was immediately followed by an endline survey (August 5 to October 31, 2019) among 1,070 retained households. The primary endpoints were child cognitive and language development as measured using the Bayley Scales of Infant Development, Third Edition [16] and child socioemotional development measured with the Wolke Scale [17]. Ethics approval was obtained from the ethics committee at Maseno University in Kisumu, Kenya and RAND; the trial is registered with ClinicalTrials.gov NCT03548558, and the study protocol has been published [18].

Delivery agents were drawn from the existing cadre of local community health volunteers (CHVs), which is a part-time and voluntary position under Kenya's Ministry of Health, tasked with improving community health through home visits. For our study, CHVs received a total of 16 days of training, with 8 days before the program started to cover the first half of sessions (sessions 1 to 8) and another 8 days at midline to cover the second half of sessions (sessions 9 to 16). Each training included 5 days in the city of Kisumu to introduce the material and curriculum, followed by 3 days of supervised practice in the rural subcounties. From session 4 onwards, CHVs received monthly 1-day refresher trainings in their subcounties to prepare them for that month's upcoming sessions.

Mothers and children were invited to attend every session and received a small gift for attendance (e.g., small bar of soap or bag of milk). Every fourth session served as a review session, for which households in the group-only arm continued with group meetings, while households in the mixed arm received individual home visits. During these home visits, participants received identical messages to those in the group reviews, but the focus was personalized on that family. The remaining 12 non-review sessions were delivered in group meetings in both intervention arms. Group sessions took place in local community centers or churches and lasted a median of 90 minutes, while the median home visit lasted an hour. The median travel time for CHVs to group meetings was 20 minutes each way, while the median travel time to each home visit was also 20 minutes. Over the 8-month program, mothers in the group-only arm averaged 64% attendance versus 74% in the mixed-delivery arm (Table 1 ), where this difference was driven only by higher attendance (88%) to the 4 review sessions that were delivered as home visits in the mixed-delivery arm (and thus, mothers did not have to travel to these sessions) [19].

The local nongovernmental organization (NGO) Safe Water and AIDS Project (SWAP) oversaw the study's implementation. Three of SWAP's staff from the central office in Kisumu provided oversight of the project and became lead trainers of CHVs under a Training of Trainers (ToT) model. They also coordinated the work of 3 subcounty supervisors who were hired by the research project to observe and monitor each group session and at least 1 home visit from each review session in the assigned villages. Subcounty supervisors were assisted by 1 or 2 "mentor" CHVs who had been previously trained in a piloting phase to deliver the program across 6 villages not included in the main trial. In the group-only arm, 89% of the non-review and 91% of the review sessions were supervised. In the mixed arm, 87% of the group nonreview sessions and 8% of the 1,531 attempted home visits were supervised (Table 1 ). Supervisors rated CHVs using monitoring forms with a checklist of items related to fidelity and quality of delivery. CHVs were provided supervisor feedback immediately after each session and filled out self-evaluation forms after each session.

Outcomes

The impact analysis of Msingi Bora was by intention to treat (ITT) among the 1,070 children assessed at endline [15]. We estimated adjusted multivariate linear regressions of each agestandardized outcome on treatment assignment relative to the control group. Preplanned adjustments included child age, sex and birth order, maternal education, household wealth, baseline outcomes (if available,) and subcounty strata fixed effects. We found that both the group-only and the mixed-delivery models significantly improved child cognitive and socioemotional development relative to the control group, and the group-only model also showed statistically significant improvements in child receptive language. The ITT effect sizes of improvements in the group-only arm were 0.52 standard deviation (SD; p = 0.001) in cognition, 0.42 SD (p = 0.017) in receptive language, and 0.23 SD (p = 0.024) in socioemotional development. In the mixed-delivery arm, the ITT effect sizes were 0.34 SD (p = 0.021) in cognition, 0.20 SD (p = 0.17) in receptive language, and 0.22 SD (p = 0.011) in socioemotional development (Table 1 ) [15]. Although the mixed-delivery arm had smaller effect sizes on child cognition and receptive language, differences across intervention arms were only statistically significant based on 1-sided tests at the 10% level.

Economic evaluation

This study is reported as per the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guideline (S1 Checklist) [20]. We estimate incremental cost-effectiveness ratios (ICERs), BCRs, and return on investment (ROI) for the 2 delivery models tested in the Msingi Bora trial. The CEA follows a preanalysis plan laid out in our study protocol [18] to calculate the incremental ITT impacts on child developmental outcomes divided by the total present period costs of an intervention (either group only or mixed delivery), where both are expressed relative to the control group. Following recent guidelines for cost-effectiveness analyses [21,22], we present ICERs in terms of SD improvements in child developmental outcomes per US$100 investment. In addition to the preplanned CEA, we also estimate the expected BCRs and ROIs of our program to help fill this gap in the literature evaluating ECD interventions and programs in LMICs.

All costs and benefits in local currency included in our analyses were converted at 101.8 Kenyan Shillings = US$1 and reported in 2020 prices, adjusted for inflation.

Costing perspective

For the calculation of cost-effectiveness, we adopt a societal perspective that combines the implementation costs a health service provider would incur if the program were to be scaled using local CHVs with present period societal costs to the household and the community. These societal costs include the opportunity costs of mothers and CHVs to travel to and attend sessions, as well as rental costs for venues to host the sessions if operating at scale (although we did not pay for venues in our trial). We use accounting cost methods for the provider costs and economic costing methods to incorporate the opportunity costs of participation for mothers and CHVs. For the calculation of BCRs and ROIs, we additionally incorporate the predicted future costs and benefits stemming from gains in cognitive skills following participation in the program. Future costs include the costs of increased schooling attendance among participants in a scaled version of the program [23], and future benefits include the expected gains in lifetime wages. Research costs (e.g., piloting, qualitative and quantitative data collection, and cost of researchers conducting the trainings) were excluded.

Costs

Provider costs relevant to implementing the Msingi Bora program were collected during the 8-month intervention period using quantitative financial data reported by SWAP. For these costs, we used a step-down accounting costing methodology using actual program costs rather than budgeted costs [24,25] and where costs were divided into direct and indirect implementation costs depending on whether they were related to specific implementation activities or not. All program-related cost data were monitored on a regular basis during the start-up and intervention period by SWAP's financial manager. We used economic costing methods to estimate opportunity costs for CHV and mother participants. Travel times to conduct sessions and session duration both for home visits and group meetings were based on CHV self-assessment forms collected after every session. We used supervisor monitoring forms to impute supervision time-related costs.

Unit costs by intervention arm are presented in Table 2 . Provider costs. We present the costs that were required to implement the intervention "as it happened," which include additional costs incurred due to unexpected delays in fieldwork, as well as cost inefficiencies in practice (e.g., having more than 1 supervisor at a given session). The only exception is that we account for the higher time use costs for CHVs delivering home visits for the 4 review sessions in the mixed-delivery arm using estimates of the opportunity cost of their time. For the sake of comparison, in S2 Text, we present an alternative "best case" costing scenario based on our calculations of program costs without those delays and inefficiencies that better reflect how the program was intended to be implemented and thus how much it would arguably cost to replicate it in similar settings taking advantage of potential economies of scale. Results from this comparison are included in a sensitivity analysis.

Direct program costs. Direct costs included salaries and stipends for personnel (CHVs, mentor CHVs, subcounty supervisors, and SWAP staff), training, travel and accommodation for training, and any food and supplements necessary for the implementation of the program.

Personnel

We include the flat monthly stipends paid to CHVs and mentor CHVs, as well as salary costs for 3 subcounty supervisors and 4 SWAP staff members involved in day-to-day activities of implementation, supervision, and training, prorated to their percentage time commitment to the project. These base salary costs were equally shared among the 2 intervention arms. For the 4 review sessions that were delivered through home visits in the mixed-delivery arm, we add to these salary costs the time use costs that CHVs, mentor CHVs, and subcounty supervisors spent delivering or supervising these sessions using an estimated hourly wage rate and the time taken to conduct these activities (Table 1 ). These time use costs were larger in the mixeddelivery arm because home visits are more time intensive than group meetings. To estimate the opportunity costs of CHV's time, we use the 2015 to 2016 Kenyan Integrated Household Budget Survey (KIHBS), which provides individual data on wages at the county level. We calculate the average implied hourly wage for all workers who work at least 20 hours per week in some outside paid activity in the study counties of Homa Bay and Vihiga, updated to 2020 prices. We used session monitoring data to impute supervision costs for mentors and supervisors as appropriate.

Travel and accommodation

These costs include transportation, meals, and lodging for CHVs, mentor CHVs, and subcounty supervisors to attend the 2 centralized 5-day training sessions in Kisumu, transportation costs for supervisors to supervise sessions, and transportation and meal costs for mentor CHVs and CHVs to attend the monthly refresher trainings hosted in the respective subcounties. These costs again differ by delivery arm due to the travel time necessary to conduct home visit review sessions in the mixed-delivery arm, again priced at local wage rates.

Supplements and incentives

We include milk, soap, or eggs given to participants during sessions, as well as a locally published picture book given to all children in session 14.

Indirect program costs. These costs included start-up costs such as the cost of tablets used for session monitoring and an online data transfer platform, administrative costs such as the design and printing of manuals for CHVs used during training, mobile phone credit for CHVs, mentor CHVs, and supervisors, and the printing of T-shirts for CHVs and certificates for participants who completed the program.

Societal costs. In addition to the provider costs above, we include the following societal costs.

Mothers' opportunity cost of time

Mothers who participate in Msingi Bora pay opportunity costs of their time stemming from attending sessions, as well as enacting the new recommended behaviors with their children at home. To price this time, we again use the 2015 to 2016 KIHBS to estimate an implied hourly wage for all mothers based on the average wage rate for females in the 2 study counties, regardless of their employment status, and updated to 2020 prices (Table 2 ). We combine average attendance, session duration, and travel times from Table 1 to estimate the time mothers spent attending sessions, and then in response to a reviewer recommendation, we add to this an estimated 1 additional hour per week in stimulation practices over 3 years of the child's life to reflect time opportunity costs associated with maternal behavioral changes. This implicitly assumes no opportunity costs for behavioral changes after 3 years, but aligns with Kenya's 2018 policy of universal free preschool beginning at age 3 [26]. Moreover, our curriculum emphasized that mothers can stimulate their children while doing their daily chores such as speaking to the child and providing the child with a playing object and did not ask for more than 10 minutes per day of direct stimulation activities.

Venues While our program enjoyed the free use of community centers or churches for the groupbased sessions, under a scaled version of Msingi Bora, we estimate that venues would no longer be free and thus include a daily rental fee per venue per group session, for which there are 12 in the mixed-delivery arm and 16 in the group-only arm (Table 2 ).

CEA

We calculate incremental cost-effectiveness separately for each delivery model as the ratio of that intervention model's ITT impact on each child outcome divided by that model's total present period costs, where we use a societal perspective and present results in 2020 US$. This allows us to interpret our results in terms of SD improvement in a given outcome per US$100 investment by intervention arm. Following standard recommendations for reporting of costeffectiveness [12], we also present ICERs based on provider costs, although we only use these to facilitate comparisons with existing estimates. Standard errors (SEs) for our estimates are recovered from simple linear transformations of SEs of the intervention impacts, which we assume to be normally distributed random variables.

BCA

To estimate BCRs and ROIs of our program in the long term, we additionally consider future costs and benefits that would result from increased cognitive abilities among participants in a scaled version of our program. Future costs include additional schooling costs due to increased attendance, and future benefits include net gains in lifetime wages.

Schooling costs. Improvements in child cognition resulting from the intervention may increase schooling attendance and associated costs such as additional infrastructure, tuition payments, and teacher salaries. To predict these future schooling costs, we first predict the intervention impacts on schooling attendance, which we obtain as the product of our intervention impacts on cognition with an estimate for the additional years of schooling associated with an increase in cognitive abilities. We estimate this schooling return to cognition using data from the Kenya Life Panel Survey (KLPS) rounds 1 to 4 (2003 to 2005, 2007 to 2009, 2011 to 2014, and 2018 to 2019) [27,28], which includes longitudinal data on cognitive tests, schooling, earnings, and household sociodemographics for a representative sample of more than 5,000 individuals participating in the Primary School Deworming Project (PSDP) between 1997 and 2001 in nearby Busia District. Using multivariate linear regressions of years of schooling on an age-standardized measure of cognition, we estimate that a one SD increase in cognition is associated with an increase of 1.79 (SE = 0.12, p < 0.0001) years of schooling after adjusting for covariates including parental education, survey wave and month of interview, a female indicator variable, and baseline 1998 school grade fixed effects (more details in Table A in S1 Text). Combined with our ITT impacts on cognition (Table 1 ), we predict that participation in our program would increase schooling by 0.93 years in the group-only arm and by 0.61 years in the mixed-delivery arm. We estimate final schooling costs for each delivery model by multiplying these predicted intervention impacts on schooling with the public cost of an additional grade per child per year, which we assume to be 15% of the basic wage for low-skilled workers from KIHBS wage data, following Nandi and colleagues [29].

Gains in lifetime wages. To predict the expected gains in lifetime wages from our intervention, we need 2 ingredients: (a) data on wage profiles by age of children as adults for our setting; and (b) the predicted intervention impacts on wages. We obtain potential wage profiles by age of children as adults from a representative sample of individuals aged 16 to 64 again using the 2015 to 2016 KIHBS. We restrict the sample to at least halftime workers (working at least 20 hours per week in any paid occupation) and calculate the life stream of average earnings by age expressed in 2020 prices, as well as the sum of discounted earnings to present values adjusting for expected survival probabilities using age life tables from Kenya [30]. To predict the intervention impacts on wages, we combine our ITT impacts on cognition with an estimate for the wage return to increased cognitive abilities, which we estimate with KLPS data. Using multivariate linear regressions of log wages on an age-standardized measure of cognition, we predict that a one SD increase in cognition is associated with a 39.7% (SE 0.058, p < 0.0001) increase in annual wages after controlling for the same covariates as the schooling regressions (more details in S1 Text). Combined with the intervention effects on cognition reported in Table 1 , we estimate that the intervention would increase annual long-term wages by 20.6% in the group-only arm and by 13.5% in the mixed-delivery arm. Net gains in lifetime earnings by arm are the product of the sum of discounted earnings, and these predicted intervention effects on adult wages.

Discounting

For our BCA, we follow standard practice to translate the long-term costs and benefits of our program into present discounted value (PDV) terms using an annual discount rate to reflect their delayed nature. We assume an annual discount rate of 5%, which is in line with previous benefit-cost analyses of ECD programs from LMIC settings [31][32][33]. However, in a sensitivity analysis, we examine how our BCR and ROI estimates would change under different assumptions about the discount rate.

Benchmark BCR and ROI calculations. We use our estimates of the full societal benefits and costs that includes these future schooling costs and wage increases to estimate both a BCR and the expected ROI. The BCR compares the child's expected increase in discounted lifetime earnings resulting from early cognitive gains following participation in Msingi Bora versus its average costs per child, also expressed in PDV terms. The ROI is the difference in additional expected tax revenues due to the increased PDV of lifetime earnings from the intervention relative to the average costs per child, expressed in terms of a percentage change. For calculating ROI, we assume a 14.6% average tax rate, again based on KIHBS earnings data updated to 2020 prices and matched to Kenya's marginal tax rate schedule from 2020 [34]. In a sensitivity analysis, we explore alternative average tax rates. Positive ROIs indicate that public revenues are greater than the costs of scaling the program.

In order to derive SEs for our estimated BCRs and ROIs, we perform 1,000 Monte Carlo simulations that allow for uncertainty in our estimates of wage and schooling returns to cognitive abilities, assuming that all these parameters are normally distributed random variables.

Sensitivity analyses. We conduct 2 types of sensitivity analyses. We first test for the robustness of our BCRs and ROIs by systematically varying one parameter at a time while holding all others constant. This allows us to identify what "extreme values" are needed for our BCR to be less than 1 and our ROI less than 0, which, in turn, would make scaling Msingi Bora a poor societal investment. We vary the discount rates, total costs per child, average tax rate, cognitive wage returns, and the public cost of an additional grade per child per year to find these extreme values. In addition to these parameters, we test for the degree of fade-out in impacts in the longer-term that our program could sustain while still maintaining positive net benefits and ROIs, which is motivated by a recent systematic review of ECD parenting trials from LMICs that finds that early intervention impacts on children's developmental outcomes tend to fade over time [35].

In addition, in Table B in S2 Text, we estimate by how much the resulting ICERs, BCRs, and ROIs would change if we allow for both socioemotional and cognitive abilities to influence lifetime wages and if we use costs based on our "best case" scenario for provider costs that allows for some implementation efficiencies and economies of scale. For this analysis, we assume that a one SD increase in socioemotional skills leads to a 6.4% increase in wages as based on a recent study on the returns to socioemotional skills for Kenya [36].

Program costs

The total costs for the program were US$56,171 for the group-only arm and US$58,103 for the mixed-delivery arm (Table 3 ), which translate into a total cost per child of US$140 in the group-only arm and US$145 in the mixed-delivery arm. Provider costs comprised the bulk of these total costs, amounting to US$47,423 in the group-only arm (US$119 per child) and US $48,670 in the mixed-delivery arm (US$122 per child). The only difference in these costs across the 2 models of delivery comes from the time use costs associated with the review sessions that account for CHVs spending an average of 18.5 additional hours delivering home visit review sessions relative to group review sessions. Travel and accommodation were by far the highest direct provider costs, largely as a result of the 2 centralized full board trainings. The small difference in these costs across the 2 delivery models stem from the extra travel time required for home visits, which we estimate at an additional 8.5 hours per CHV per review session in the mixed-delivery arm.

The present period societal costs were US$8,748 in the group-only arm (US$22 per child) and US$9,433 in the mixed-delivery arm (US$24 per child) (Table 3 ). This small difference stems from a few countervailing factors. On the one hand, the group-only arm entails 4 extra group review sessions that require greater venue costs, as well as greater time for mothers to travel and attend group sessions, which lasted 30 minutes longer than home visits on average. On the other hand, only mothers who attend sessions can enact the recommended behavioral changes, which implies greater mother opportunity costs on average due to the higher attendance at the 4 home visit review sessions in this arm.

Cost-effectiveness

Table 4 reports the ICERs for each delivery model and considers costs under both provider and societal perspectives as reported in Table 3 . Higher ratios imply a higher impact on child outcomes per US$100 invested. ICER estimates for the group-only arm are generally greater than those for the mixed-delivery arm due to the higher estimated impacts on child outcomes at slightly lower costs. For the group-only delivery model under a societal perspective, an additional US$100 investment is estimated to lead to a 0.37 SD improvement in child cognition, 0.30 SD improvement in receptive language, and 0.16 SD improvement in socioemotional development. Under the mixed-delivery model, the corresponding estimates are improvements of 0.23 SD, 0.14 SD, and 0.15 SD, respectively. ICERs under a provider perspective are slightly larger as societal costs are excluded from the denominator.

BCRs and ROIs

We estimate gains in lifetime wages from participation in the program of US$2,729 per child in the group-only arm and US$1,784 in the mixed-delivery arm (Table 5 ), a difference primarily due to the lower intervention impacts on cognition in the mixed-delivery arm. We estimate future schooling costs of US$35 in the group-only arm and US$23 in the mixed-delivery arm, implying total long-term costs per child of US$176 and US$168, respectively. As a result, Table 5 shows that the benefits of Msingi Bora are estimated to substantially outweigh the associated costs under both delivery models, with a BCR of 15.5 and an ROI of 127% for the group-only delivery model and a corresponding BCR of 10.6 and an ROI of 55% for the mixed-delivery model.

Sensitivity analysis

Table 6 shows the calculated values of the key input parameters needed to reach an estimated BCR less than 1 or an estimated ROI less than 0, holding all other values constant. These "extreme values" identify at which point scaling Msingi Bora would become a poor societal investment. For example, to obtain a BCR less than 1 for the group-only delivery model, we would need a discount rate of 17% (as opposed to 5%), holding all other parameters constant at assumed values as listed in column 1. Similarly, for predicted program costs to outweigh benefits under both the group-only delivery and mixed-delivery models (implying a BCR less than 1), our early intervention impacts on cognition would need to nearly completely fade out, from 0.52 SD to 0.03 SD in the long term. Finally, scaling up the group-only version of Msingi Bora would have a negative ROI for the government if the average tax rate was 6.4%, which is less than half our calculated average tax rate.

In Table B in S2 Text, we simulate new estimates for ICERs, BCRs, and ROIs under the alternative "best case" cost scenario that assumes a more efficient scaled version of the program's implementation, as well as that allows for socioemotional skills to further influence lifetime wages beyond the impacts of cognition. In the group-only arm, these adjustments increase our ICER to a 0.55 SD improvement in cognition per US$100 investment, with an associated BCR of 22.4 and an ROI of 227%.

Discussion

We conducted an economic evaluation of 2 potentially scalable and effective group-based ECD parenting interventions targeted to mothers with young children in rural Kenya. We found that the total costs per child were slightly smaller for the group-only model (US$140) than for the mixed-delivery model (US$145) that substituted periodic home visits for group sessions. Coupled with the fact that effect sizes in child outcomes were consistently higher for the group-only delivery model (although not statistically significantly so at traditional levels), these cost estimates imply that the group-only model was also the most cost-effective, with effect sizes per US$100 additional investment of 0.37 SD in cognition, 0.30 SD in receptive language, and 0.16 SD in socioemotional outcomes. We also found that the long-term economic returns of scaling Msingi Bora could be extremely high if our predicted long-term program impacts hold. Focusing on cognitive outcomes, we find a BCR of 15.5 for the group-only delivery model and 10.6 for the mixed-delivery model, while the average ROIs for a government scaling the program are 127% and 55%, respectively. We would also require very extreme values of our input parameters to have negative ROIs and BCRs less than 1. For example, program costs would outweigh benefits under the group-only model with a discount rate of 17% or if our early impacts in cognition almost entirely fade out in the longer term from 0.52 SD to 0.03 SD.

The small difference in costs across delivery models of US$5 per child is perhaps surprising. However, it is worth remembering that in 12 of 16 sessions (75%), the delivery model was identical across intervention arms, so the only difference in costs comes from the 4 review sessions. In addition, even though CHVs in the mixed-delivery arm spent an average of 27 additional hours traveling to and delivering each review session as home visits, when priced at current local wage rates, the additional hours spent did not translate into large differences in monetary costs.

Our group-only delivery model remains highly cost-effective when we compare our ICERs based on a health provider's perspective to those we calculate for 2 recent effectiveness studies from Pakistan and India reporting unit costs for similar parenting interventions targeted to children aged 0 to 3 from LMICs and focused on similar outcomes of children's cognitive and language development [13,14]. We consider only provider costs simply because no earlier studies accounted for societal costs in their analyses, and we adjust all costs to 2020 US$ in purchasing power parity (PPP) terms to facilitate comparisons. Under this criteria, for Msingi Bora, we estimate an ICER of 0.176 SD improvement in cognition per PAU : Pleasecheckandconfirmifa PP$100 investment. For the Lady Health Worker (LHW) Program from Pakistan, we estimate a lower ICER for cognitive outcomes of 0.148 SD improvement in cognition in similar 2020 US$PPP terms, based on a reported monthly cost per child of US$4.15 (in 2012 US$) in the responsive stimulation arm [13], which was the study arm with the largest impacts on cognition [37], and assuming that each child was treated for 24 months. However, for language outcomes, their program had a relatively larger ICER of 0.17 SD improvement per PPP$100 compared to our study's estimated ICER of 0.14 SD improvement in language per PPP$100. It is worth noting that their program costs are at least partially lowered by the fact that their interventions were integrated within the existing infrastructure of government health services with already highly trained personnel, costs that were excluded from their analysis under a provider costing perspective. We imagine that our training costs, which are roughly one-third of our total provider costs, may be more realistic in similar settings where existing infrastructure and personnel are not on par with Pakistan's LHW program [4]. Furthermore, their study does not account for private opportunity costs to participants or delivery agents during their 33-month program as we do.

Another recent effectiveness study conducted in Odisha, India compared group meetings versus individual home visits over 2 years and similarly found group visits to be much more cost-effective for a relatively similar sample to ours [14]. This study also did not account for private opportunity costs of participants or delivery agents. The total provider costs for group meetings were US$76 per child for the 2-year intervention, which translates to US$265 in 2020 PPP terms, implying a cost-effectiveness ratio of 0.106 SD improvement in cognition per PPP$100 investment and 0.114 SD in language. However, these reported costs per child are based on the assumed number of children that a scaled version of this intervention would be able to cover if facilitators worked full time, which is roughly 9 times higher than the actual number of children invited to the intervention. If we similarly assume that each CHV can work full time to conduct 8 group sessions per week, a scaled version of Msingi Bora using only group visits would cost PPP$89 per child using a societal perspective, where we a full-time CHV wage again based on local wage rates. At these assumed costs, we would estimate an ICER of 0.58 SD improvement in cognition per PPP$100 investment and 0.47 SD in language, which again makes our program much more cost-effective despite the inclusion of societal costs, which in a scaled version of Msingi Bora we estimate would account for 61% of the total cost per child. Besides having higher impacts on cognitive and language outcomes, other reasons our program is more cost-effective than the one conducted in India include that our group meetings were nearly twice as large as theirs (they invited 7 to 8 children while our median attendance was 13 children at group meetings), and while 23% of their total costs were allocated to purchase toys and learning materials, our program only used play materials freely available in the home. Msingi Bora's estimated BCRs are also comparatively high with the very limited number of related interventions reporting similar estimates (Table 7 ) including 3 home visitation programs in Latin America [32], an intervention delivered through health center meetings and home visits in Jamaica and other Caribbean countries [38], and a combined day care and home visitation program in Nicaragua [39], all of which used a lower discount rate (3%) than we did in our study (5%). Our BCRs are also high when compared to high-profile home visitation programs evaluated in the United States such as the Nurse-Family Partnership Program, for which recent evaluations report BCRs in the range 1.5 to 5 [40,41]. Furthermore, our BCRs and ROIs only consider the increase in discounted lifetime earnings as benefits of program participation and ignore any potential long-term impacts on other health and social outcomes, and, as such, can be interpreted as likely lower bounds on the true potential societal returns (under the assumption that the program impacts do not fade out in the long term).

Our study has numerous strengths related to methods and the economic evaluation itself. To the best of our knowledge, our study is the first economic evaluation of an ECD parenting Strengths of our study include adoption of a societal perspective for our CEA that accounts for private opportunity costs to participants and delivery agents, the combination of cost-effectiveness and BCAs to assess outcomes, and the inclusion of sensitivity analyses showing the values at which the program generates negative returns. All of these complement each other to provide policymakers a more comprehensive portrait of private and social costs as well as expected economic returns of our program. Limitations of our analysis include that our BCA necessarily relies on different assumptions of how program impacts translate into long-term benefits and costs using external data sources and that we assume that our early impacts can be sustained in the long term. Future work could address these limitations, but obviously necessitate the passage of time. We also do not account for varying levels of schooling quality across public or private schools that may present different economic opportunities for children enrolled in Msingi Bora, and we do not account for patterns of formal and informal employment or internal migration that may influence the profiles of wages by age we used to estimate benefits. We also examine the economic benefits of enhanced human capital only in terms of increased wages, but we do not include other outcomes that could potentially improve such as labor market participation, share of workers in the informal sector, health, life expectancy, or prosocial behaviors. We may similarly be missing some societal costs in addition to those included.

As discussed above, we believe the BCR outcomes can be interpreted as the lower bound of the true societal returns as long as our early impacts are sustained, but if there are large costs incurred to society as a result of intervention-related improvements, leaving these out may modify our BCR and ROI estimates.

Our findings highlight the need for careful cost analyses to accompany evaluations of ECD parenting programs in LMICs to help inform governments and policymakers on the potential returns to society and to the government of investing in such programs. The growing recognition worldwide about the importance of a child's early years and the potential promise of responsive parenting interventions to improve children's outcomes are not likely to affect policy decision-making without accompanying information about the cost-effectiveness of such investments. We also believe that cost-effectiveness analyses of ECD parenting programs should account for the private opportunity costs to participants and delivery agents for both ethical and practical reasons: Excluding these costs could create perverse incentives for governments to offload program costs onto users and delivery agents, who are often vulnerable and poor citizens whose burden of time should not be assumed to be free. We hope our results can motivate similar analyses from other ECD parenting programs from LMIC settings.

Our results show that an integrated child development intervention featuring exclusively group sessions delivered by paraprofessional community health workers can be very cost-effective. We find that a mixed-delivery model that substitutes periodic home visits for group sessions comes at a slightly higher cost, albeit with a lower benefit, and thus is less cost-effective for our setting. Our results represent a strong case for exploring pathways to scale similar interventions in low-resource rural settings where many children and families could potentially benefit from ECD parenting programming, and investments could pay significant dividends over time.

availability

The KLPS data files used in this manuscript are available from the Harvard Dataverse database (https://dataverse. harvard.edu/dataverse/KLPS). The KIHBS data is publicly available here: http://statistics.knbs.or.ke/ nada/index.php/catalog/88. All relevant data from the Msingi Bora trial are within the manuscript and its Supporting Information files.

Acknowledgements

AcknowledgmentsWe thank Alie Eleveld, Alex Mwaki, Aloyce Odhiambo, Edith Alu, Ronald Ochieng, Nicholas Ogendo, the CHVs, and the entire team at the Safe Water and AIDS Project (SWAP) in Kenya for invaluable support in carrying out this study.We thank Michael Walker, Michelle Layvant, and the team at UC Berkeley for help generating the estimates for impacts on wages and schooling using KLPS data.WAU : PleasenotethatPLOSMedicinedoesnotallowrecognitionofreviewersor e extend our gratitude to Frances Aboud and Rebecca Kilburn for providing feedback on earlier versions of this draft.

Funding

This work is supported by the Eunice Kennedy Shriver National Institute of Child Health

Loading PDF...

StripeM-Inner

Paper sources

Abstract screening pilot

Abstract screening results

Extraction pilot

Extraction results

Research report

Modify setup