Discussion and implications of evaluation findings

Parts of this chapter have been reproduced from the published papers by Cavallaro et al.,1 Cavallaro et al.2 and Cavallaro et al.101 These are open-access articles distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) licence, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given and an indication of whether changes were made. See: http://creativecommons.org/licenses/by/4.0/.

Chapter outline

This chapter brings together the findings from the three objectives of this study in order to provide an understanding of the context in which the FNP has been delivered over the past decade in England and to summarise our findings on the real-world implementation of the programme and the factors that may influence effectiveness.

For each objective, we summarise the main results, compare these with previous literature and discuss the implications of our findings. Where relevant, we report initial findings from a qualitative analysis of the FNP (see Qualitative analysis). We then explore the strengths and limitations of the study as a whole. Lastly, we suggest recommendations for policy and practice and further research.

Objective 1 – which groups of adolescent mothers receive Family Nurse Partnership across local authorities?

While eligibility criteria suggest all first-time adolescent mothers are eligible for the FNP, the service has been rationed to ~ 25–30% of adolescent mothers since the rollout in 2010, as sufficient funding for all was not made available.102 To inform commissioning and targeting of the FNP, we quantified the variation in enrolment rates across 122 FNP sites in England, comparing maternal vulnerability indicators between eligible mothers who were and were not enrolled in FNP and evaluating site characteristics associated with enrolment.

Summary of main results

This study fills an important evidence gap in coverage and targeting of intensive home-visiting services such as FNP within England. We show the most vulnerable mothers are being targeted for FNP, especially the youngest adolescents and those with prior contact with children’s social care. However, our findings indicate insufficient commissioning of FNP relative to local needs: only 23% of eligible mothers in FNP catchment areas were enrolled (2010–7), and areas with the greatest numbers of adolescent mothers had lower enrolment rates. Overall, 26% of eligible mothers aged 13–19 with vulnerability markers associated with adverse outcomes were enrolled, including 52% of those aged 13–15, 44% of those ever recorded as being Looked After and 40% of those with prior mental health- or adversity-related hospital admissions. These groups represent mothers and infants at significantly greater risk of low birthweight, unplanned hospital admissions for injury and infant mortality.89 We did not find any differences in the populations of mothers in areas that did and did not commission FNP.

Comparison with existing evidence

To our knowledge, this is the first study examining enrolment in a targeted intensive home-visiting programme for expectant mothers. Two previous studies have shown high variation in acceptance rates for universal home visiting between sites, with higher acceptance rates among higher-risk mothers.103,104 In a similar targeted home-visiting intervention for vulnerable families in Canada (Families First), unenrolled mothers were more vulnerable than enrolled mothers – contrary to our findings – illustrating that it can be difficult to enrol the most vulnerable groups.105

Interpretation

Budget cuts since the inception of the FNP in England in 2007 mean that the programme, costing approximately £3000 per client per year, has been offered to a more select group of mothers over time.80,106 This study demonstrates FNP places are not commissioned proportionately to the number of adolescent mothers within local areas, contrary to the stated aims of prioritising ‘areas with […] the highest numbers of eligible population’.107 Pregnant adolescents living in areas with many adolescent mothers are less likely to receive support than those in areas with few adolescent mothers, with important implications for equity.

Young age is the main eligibility criterion for FNP in England, based on ease of identification, associations with social adversity, disrupted education and employment13,61 and other factors contributing to poor health outcomes among their children.5,6,62 Unlike some other countries, FNP eligibility criteria in England do not include low educational level or unemployment,27,35,64 which is informed by evidence of higher effectiveness in socioeconomically deprived groups in the US.32,36,38 In our analysis, only half of mothers aged 13–15 living in areas in which FNP was commissioned were enrolled. Given strict caseload limits (maximum 25 mothers per Family Nurse), many sites in England have explicit policies of prioritising younger adolescent mothers. Overall, 74% of eligible mothers with vulnerability markers were not enrolled in FNP, indicating opportunities for improving policy and commissioning to address vulnerable mothers’ needs, with important and unexplained variation in who is offered intensive services across England.

There are several explanations and implications of this. First, FNP teams may prioritise mothers with additional vulnerabilities not captured in administrative data we used (homelessness or community mental health service use; Box 1), but it is likely that family nurses do not have the full picture of family vulnerability when prioritising services, and some vulnerabilities (such as family violence) may be disclosed only after a trusting relationship is built with their family nurse or health visitor.108–110 This underscores the need for effective universal services which can identify those at greatest need for intensive services.

Second, there is uncertainty regarding which vulnerable mothers are most likely to accept and benefit from intensive support. Given insufficient funding to offer the programme to all eligible mothers, family nurses and referring providers need standardised, real-time information on vulnerabilities for all mothers to support decision-making and better target the FNP, alongside effective communication between health visitors and family nurses. Ongoing work on defining vulnerabilities by the FNP National Unit will support sites to determine priority criteria for their local context.

Third, vulnerable mothers may have higher refusal rates. Although FNP’s fidelity target is to enrol 75% of mothers offered FNP, aggregate site data suggest not all sites meet this target. We were unable to determine whether under-representation of some groups, for example, South Asian adolescent mothers, was because they were less likely to be offered a place to accept it or both. FNP teams report that most mothers who decline feel socially well-supported, although some decliners are especially vulnerable (e.g. involved with social care services).111 Last, more vulnerable mothers may be unknown to midwifery services due to presenting to a booking appointment after 28 weeks (or because of moves between LAs in pregnancy). Individual-level information on who is approached and who accepts would help inform strategies to increase uptake among especially vulnerable mothers.

Objective 2 – what is the effect of Family Nurse Partnership enrolment on maternal and child outcomes?

For this objective, we aimed to determine the effect of the FNP on maternal and child outcomes, including identifying which families benefit the most from the FNP.

Summary of main results

Cohort description

We identified a number of pre-pregnancy maternal risk factors for indicators of child maltreatment, including younger maternal age, living in more deprived areas, having a history of adversity- or mental health-related hospital admissions, having later first contact with antenatal services, having been referred to social care services as a child, having SEN provision, being persistently absent from school or excluded and having lower educational attainment. Our findings add to previous research that describes the association between having a history of adversity- or mental health-related hospital admissions and increased hospital admissions in infancy by linking additional maternal data on education and social care risk factors and identifying novel risk factors that are important for first-time adolescent mothers.89 Some of the most important risk factors for child maltreatment in this population are maternal histories of being in care or being excluded from school.

Mothers enrolled in the FNP were much more likely to have these risk factors than those who were not enrolled, and FNP enrolment therefore works as a composite indicator of vulnerability.

Indicators of child maltreatment

There was no difference in CiN or CLA according to enrolment in the FNP, but there was weak evidence that FNP was associated with a reduction in the percentage of children with a CPP (RR 0.84, 95% CI 0.71 to 1.00; p-value = 0.049). We saw an increased risk of unplanned admissions for maltreatment/injury in the 2 years after birth in children born to mothers enrolled in FNP (RR 1.15, 95% CI 1.07 to 1.24, compared to children born to mothers not enrolled in FNP), which did not persist at 7 years after birth.

Other child health/development/education outcomes

We found that FNP was associated with an increase in the number of children with ≥ 1 unplanned admission for any diagnosis up to age 2 and the number of children with ≥ 1 A&E attendance by age 2 and 7. We also found evidence of a small increase in children achieving a Good Level of Development (school readiness) at age 5 in the FNP group (RR 1.05, 95% CI 1.01 to 1.09; p-value = 0.031). There was no evidence that this advantage persisted at Key Stage 1, nor was there any evidence of a difference between groups in SEN provision. FNP was also associated with an increase in the percentage of children registered for FSM (RR 1.09, 95% CI 1.04 to 1.14).

Maternal outcomes

Mothers who were enrolled in the FNP were less likely to have a rapid repeat pregnancy resulting in a subsequent birth within 18 months of the index birth (RR 0.83, 95% CI 0.88 to 0.97). This association varied by region, with strongest associations seen in the South-East, South-West and London. There was no evidence of a difference between groups in the numbers of mothers obtaining 5 A*–C equivalents, including Maths and English at GCSE level or in school enrolment following birth.

Mothers enrolled in the FNP were more likely to have an unplanned admission to hospital for diagnoses relating to adversity (substance misuse, self-harm or violence) or mental health conditions than those who were not enrolled. This association persisted until 7 years following delivery, though the size of the association decreased over time.

Comparison with existing evidence

We were able to use data for almost all eligible mothers ever enrolled in the FNP since 2010 in England, excluding 1360 mothers who did not agree for their data to be used for research. Although we cannot know what the characteristics of the declining mothers are, the high coverage of our cohort means that our results are likely highly generalisable to the eligible population. We were able to link 99% of FNP mothers to a health record, and 98% of these to their baby’s health record.

Linked education data were available for 83% of mothers aged 13–19 (who were born between 1991 and 2006). These linkage rates are similar to those of other studies linking with NPD: 83% of Millennium Cohort Study children (born 2000–1) and 85% of Case Register Interactive Search Child and Adolescent Mental Health children (CRIS CAMHS, children referred between 2008 and 2013) were linked to the NPD.112 In the study, mothers who could not be linked to NPD were more likely to be older, less likely to be white, more likely to be living in the most deprived areas and less likely to have had hospital admissions in the 2 years prior to 20 weeks of pregnancy. Similar differences in the characteristics of linked and unlinked individuals have been observed in another evaluation of linkage between HES and NPD, which found that the most deprived pupils and those in ethnic minority groups were less likely to link.71 Such differential linkage is widespread and represents a problem of inequality, whereby particular groups and ethnicities are excluded from research based on analyses of linked administrative data or are less likely to link to information on risk factors or outcomes, meaning they are more likely to have their needs underestimated.113

Numbers of children classified as CiN in our cohort (22%) was similar to those observed in the Building Blocks trial, which found that 21.5% of children were ever classified as CiN. Since we cannot identify children who were in contact with children’s social care services only before school entry (due to the way in which identifiers are assigned in the DfE data), these estimates likely underestimate true numbers of referrals. Other studies have shown that a large proportion of CiN occurs in preschool children: based on information from a Freedom of Information request, Bilson et al. estimated that 14% of children born between 2009 and 2010 were CiN before their fifth birthday.114 Based on model estimates using data from the NPD, Jay et al. estimate that 25% of children are ever classed as CiN before age 16.115

Rates of CPPs were lower in our cohort (4.5% overall and 5.1% in the FNP group) than in the Building Blocks trial (6.8% in the FNP arm and 6.6% in the usual care arm), and rates of CLA were also lower (2.0% in our cohort overall and 2.6% in the FNP group compared with 3.6% in the FNP arm and 3.3% in the usual care arm of the trial). Numbers of CLA in our study population were higher than those observed in the general population, including children born to older mothers. For example, a study of the cumulative incidence of CLA using NPD data found that 1.5% of children born in 2004–6 were ever Looked After by age 6, with increasing rates over time.78 This may be due to the inclusion of all children recorded as CLA in our data, irrespective of the primary need code.

Our findings of no difference in CiN or CLA between groups mirror those of the Building Blocks 2–6 evaluation, which also found no difference in the timing or first referral, duration or primary need for CiN.24 The Building Blocks evaluation did find that CLA in the FNP arm were in care for a shorter period of time compared with children in the usual care arm. We did not evaluate duration of periods of care in this study.

In the study, we found that 75% of children had an unplanned hospital admission or A&E attendance in the 2 years following birth (37% had an unplanned hospital admission and 72% had an A&E attendance). This was comparable to rates seen in the Building Blocks trial, where 81% of the FNP arm and 77% of the usual care arm had an unplanned hospital admission or A&E attendance.99

The proportion of children achieving a Good Level of Development (school readiness) in our cohort was 58%, which was the same as reported in the FNP arm of the Building Blocks trial.24 Percentages of children reaching the expected standard at Key Stage 1 for Reading and Maths were also similar (65% and 63% compared with 65% and 62%, respectively). Key Stage 1 Writing results were less comparable, due in part to increases in rates of children reaching the expected levels over time, as described in detail in the Building Blocks 2–6 report.24

Our finding of a reduction in rapid repeat pregnancies in mothers enrolled in the FNP differs to findings from the Building Blocks trial, which observed no evidence of an effect on subsequent pregnancies within 24 months of the first birth. This may in part be due to differences in definitions: we only captured pregnancies that resulted in a live birth (as abortion data were not available and early miscarriages are not well captured in hospital data). We chose the 18-month time period based on evidence that shorter birth-to-pregnancy intervals are associated with the highest risks of adverse outcomes for women and babies and after discussions with our Patient and Public Involvement (PPI) group, who felt that a second birth within 2 years was a positive event for many young mothers.84 Nevertheless, 17% of mothers had a second child within 24 months in our cohort, compared with 21% of mothers having a repeat pregnancy within 24 months (based on inpatient data only) in the Building Blocks trial.

There are differences in the population of mothers and their social circumstances when enrolling in versions of the FNP that have been implemented in different countries.22 These differences in underlying risks of maltreatment and in how it is detected and responded to may determine the likelihood that families might benefit from the programme in a particular setting. While there may be benefits in some settings, our findings support previous results in England of no impact on child maltreatment outcomes, and there is no evidence to suggest that benefits from the FNP will appear as children get older.

Interpretation

We found no evidence of an association between FNP and the majority of the outcomes that we evaluated. Our findings are likely to be affected by residual confounding, whereby mothers enrolled in FNP were at higher risk of adverse outcomes than those who were not enrolled, even when matching on pre-pregnancy health, education and social care characteristics. Where differences between groups were observed, interpretation of outcomes captured in administrative data is challenging and complex.24 For example, our finding of an increased risk of unplanned admissions for injury/maltreatment in the FNP group could represent more injuries in this group but could also reflect more appropriate care-seeking, particularly for minor injuries. Family nurses encourage mothers to have their child seen by a clinician when necessary, and this is particularly likely to be apparent during the 2-year period in which family nurses are visiting families. We also saw increased rates of A&E attendances, outpatient attendances and unplanned admissions for any reason, which may represent appropriate care-seeking behaviour by parents. However, findings from the Elmira trial of the NFP showed a substantial reduction in visits to the physician for injuries or ingestions and in visits to the emergency department associated with the intervention, which, combined with other findings, was interpreted as reflecting a reduction in underlying maltreatment.25 Therefore, the increased risk of unplanned admissions for injury/maltreatment in the FNP group in the study could genuinely represent an underlying increased risk of maltreatment. This, in turn, might be due to residual unmeasured confounding since mothers enrolled in FNP had more indicators of adversity than mothers who were not enrolled.

Our finding of weak evidence of a reduction in the number of CPPs associated with FNP (and similar directions of non-significant associations for CiN and CLA) reinforces the complexity of understanding the mechanisms underlying these outcomes. Firstly, this finding should be interpreted with some caution given the borderline p-value (0.049) in the context of evaluating a large number of outcomes. Secondly, others have argued that given the ethical, clinical and legal mandate that family nurses have to ensure that children are protected through linkages with appropriate services, increased rates of social care contacts might well be expected in mothers enrolled in the FNP, reflecting earlier and more comprehensive surveillance.29,116 Family nurses are in a unique position to identify early problems and start processes for safeguarding, which may explain the increased rates of discharge to social services at birth. Indeed, families in the FNP arm of the Building Blocks trial had higher rates of safeguarding procedures noted in the primary care record (13.6%) than families in the usual care arm (8.0%).99 However, the Building Blocks 2–6 trial found no evidence of surveillance bias when evaluating thresholds for referrals to social care: rates of children who needed no further assessment after being referred were similar between arms.24 In the study, we measured CiN and CLA in school-age children, who would not have still been seen by the family nurse and who would therefore be less likely to have been affected by surveillance bias.

We observed a higher prevalence of preterm births and low birthweight babies in the FNP group compared with mothers who were not enrolled. These are important birth outcomes, as they are associated with a disproportionate amount of healthcare use (due in part to an increased risk of chronic disease), childhood mortality and poor short- and long-term health and educational outcomes.117,118 However, the inclusion of low birthweight as one of the primary outcomes of the Building Blocks trial was questioned, as it was not an outcome that the NFP claimed to affect and not one for which previous replicated effects had been observed.22,23 Home visiting occurs too late to be able to influence preconception health, which is an important predictor of adverse birth outcomes.119 Indeed, there is little evidence in general for the effect of home visiting on pregnancy outcomes, including from the NFP.43,120,121 The apparent increased risk of all-cause hospital admissions that we observed in the FNP group may therefore be influenced in part by the higher prevalence of adverse birth outcomes in this group, which are associated with increased unplanned hospital admissions in the first years of life.122

Our finding of an association between FNP enrolment and a small increase in the number of children achieving a Good Level of Development (school readiness) at age 5 supports similar findings from the Building Blocks 2–6 trial and fewer maternally reported developmental concerns at age 2 in the FNP arm of the original Building Blocks trial.24 The Good Level of Development is not particularly sensitive to fine changes as a measure of school readiness, and so the underlying association may be even more substantial.123 Building Blocks 2–6 also found an effect of FNP on Key Stage 1 outcomes, which was not replicated in this study.

Evidence suggests that a large number of families who meet the criteria for FSM do not apply for the benefit.124 Our finding that mothers enrolled in the FNP were more likely to have children registered for FSM may therefore indicate that family nurses are able to help mothers access the support that is available to them (which includes needing to claim the underlying welfare benefits that form the basis for FSM eligibility) and to overcome recognised barriers such as inadequate information about the claiming process. FSM eligibility is a robust indicator of social disadvantage, and we show that almost half of children born to adolescent mothers are registered (compared with 14% of children nationally who were in school in 2015–6).125,126 Given the associations between socioeconomic disadvantage, adolescent motherhood and adverse child outcomes, strategies to address the root causes of social disadvantage are also required in order to reduce inequalities and improve outcomes for mothers and their children.

Despite controlling for higher rates of pre-pregnancy unplanned admissions to hospital for diagnoses relating to adversity (substance misuse, self-harm or violence) or mental health conditions in mothers enrolled in FNP, we found that this group continued to have higher rates of these admissions, and A&E attendances, in the 2 and 7 years after giving birth. Part of this may be due to increased self-reflection and self-efficacy as a result of the programme: family nurses work with mothers to explore the trauma that many of them have experienced in the past and to become more insightful about their own needs. Cuts to mental health services have made making appropriate referrals difficult, which may lead to mothers ending up in A&E. We could only measure the most severe problems, as we only identified cases that resulted in a hospital admission record. We did not have data on primary care, which could have provided more information on outcomes that were not severe enough to result in a hospital admission. In our cohort as a whole, 6% of mothers had an unplanned admission to hospital for adversity-related diagnoses, and 8% of mothers had an unplanned admission for mental health-related diagnoses in the 7 years following delivery. Capturing measures of domestic violence is challenging, as mothers may choose not to report violence for fear that a child may be taken away or only make disclosures after a trusting relationship is built with their family nurse or health visitor.108–110 We are therefore likely to have underestimated the extent of these conditions, meaning that there are many more mothers who are in need of continued support for their health after birth. Others have argued the importance of taking a long-term view of the health of vulnerable mothers, and our findings highlight that vulnerable mothers as well as their children continue to need support for many years after giving birth.127

Of the younger mothers in our cohort who had not completed their education before becoming pregnant, 10% were able to go on to achieve 5 A*–C grades at GCSE level. While these rates are lower than national figures (around 60%), this still represents a substantial achievement for young mothers who are balancing the responsibilities of caring for a new baby and studying for their exams. However, these findings highlight the structural barriers and challenges that adolescent mothers will face as they become young adults and move into the world of employment. More needs to be done to halt this cycle of poverty and income inequality and to improve the life chances of young mothers.5

Our finding of a reduction in the number of subsequent deliveries within 18 months should be interpreted cautiously. As we could not measure abortions or miscarriages in this study, we do not know how many mothers in each group became pregnant but chose, or were unable, to continue their pregnancy. However, birth spacing is key for both promoting maternal education and engagement in the workforce and preventing harm to the mother and child, and we show that it may be possible for the programme to influence birth spacing and therefore the maternal life course. This is an important finding: national surveys in England have shown that only 12% of pregnancies in mothers aged 16–19 are planned, highlighting the need for primary prevention strategies to avoid unwanted pregnancies.128 Previous research has also shown that the effects of adolescent motherhood persist for previous adolescent mothers giving birth again in their 20s, meaning that strategies to improve the life changes of reproductive choices for young women about the timing of their pregnancies could have lasting effects.12

Objective 3 – what contextual and programme factors influence the effect of Family Nurse Partnership?

For this objective, we aimed to identify contextual and programme factors that might influence FNP programme effect.

Summary of main results and comparison with previous evidence

Overall, 58% of mothers enrolled in the FNP giving birth between April 2010 and January 2018 completed the programme, and 42% left early. Compared with targets, attrition was slightly lower in pregnancy (8%) but higher in infancy (23%) and toddlerhood (19%). These attrition rates are also higher than were observed in the Building Blocks trial (4% in pregnancy, 10% in infancy and 8% in toddlerhood), possibly reflecting a trial effect whereby FNP nurses involved in the trial might have felt more expectation to retain mothers in the programme. Alternatively, this could reflect changes in practice over time and increased flexibility in recent years, allowing mothers to leave the programme early.

The mean number of visits received by mothers across the programme was 36 (median 38). This is similar to that reported in Building Blocks (mean = 39 visits). Across the programme stages, the median number of visits was 10 in pregnancy, 10 in infancy and 14 in toddlerhood, compared with 10, 19 and 13, respectively, in Building Blocks. Overall, mothers enrolled in FNP received an average of 42 hours of contact time with a family nurse throughout the programme.

Approximately 60% of mothers met the fidelity targets for the number of completed visits in pregnancy, 65% in infancy and 60% in toddlerhood. This was slightly higher than fidelity targets reported in the Buildings Blocks trial, which reported 58% in pregnancy, 53% in infancy and 44% in toddlerhood. This may be due to differences in the way that the expected number of visits was calculated in each study, taking into account the time period in which a mother was enrolled (and therefore the number of visits for which she could possibly receive).

Overall, only 28% of mothers met the fidelity targets across all stages of the programme. This could reflect an unrealistic time commitment for mothers or practical problems in arranging visits to suit both mothers and nurses (who have maximum caseloads of 25 mothers). Following the publication of the Building Blocks trial and qualitative research suggesting that mothers wanted more flexibility within the programme, the FNP National Unit made the decision to allow greater flexibility in the appropriate number of visits per mother, which is now decided locally (‘dialling up’ or ‘dialling down’ according to a nurse-client assessment of need), along with the optimal time for graduation.129,130 It is not yet clear how these recent changes to the programme will impact longer-term outcomes: more information on why a mother leaves the programme is needed to evaluate these changes.

We show that the most vulnerable and youngest mothers tended to have greater numbers of visits and were more likely to meet the fidelity targets than those who did not have these risk factors. After adjusting for these characteristics, mothers who met the target for the number of expected visits in pregnancy were less likely to have a subsequent birth within 18 months than those who did not meet the target; children born to mothers who met the target in infancy and toddlerhood were more likely to have unplanned hospital admissions for maltreatment/injury up to age 2 than those who did not meet the targets in these stages.

Interpretation

Our finding that the most vulnerable and youngest mothers received a greater dosage of visits and were more likely to meet fidelity targets than those who did not have these risk factors suggests that even before the introduction of the Accelerated Design and Programme Testing (ADAPT) sites, family nurses were making decisions about where to prioritise their time and resources and were able to provide more intensive support to those they felt most needed it. These findings make evaluating the impact of dosage of the FNP problematic since the most vulnerable mothers receive more visits but are also independently more likely to experience worse outcomes. However, our findings for unplanned hospital admissions for maltreatment/injury and subsequent births mirror those of Objective 2 comparing mothers enrolled or not enrolled in FNP: mothers who had more visits were more likely to have children with unplanned hospital admissions for maltreatment/injury up to age 2 but less likely to have a subsequent pregnancy within 18 months. For a Good Level of Development and maternal hospital admissions, we found no evidence of a difference in outcomes for mothers who met the fidelity targets and those who did not. An experimental design, combined with more detailed information on why mothers leave the programme, could be used to further evaluate the impact of the number of visits on outcomes. This may be especially relevant in the context of limited resources and previous evidence suggesting that ‘less is more’ for interventions aiming to enhance parental sensitivity and promote attachment.131

Findings from the qualitative analysis emphasise that the length of the programme is important to both mothers and Family Nurses (Box 2).

Strengths and limitations

Strengths

A major strength of this study was the use of a population-based cohort of all first-time adolescent mothers giving birth in English NHS hospitals, linked to routine education and social care records. This provided us with a large sample size for analysing subgroups and increased statistical power for detecting small differences. Linking 99% of FNP mothers to HES gave us a highly generalisable cohort of FNP mothers enrolled between April 2010 and March 2019. Using propensity scores derived from observed data on maternal histories recorded in health, education and social care data, we were able to create closely matched groups for > 95% of the FNP mothers in our cohort. The unmatched mothers were captured in a sensitivity analysis using matching with replacement and in the multivariable regression analysis.

Another strength was our use of objective measures of indicators of maltreatment and child development. However, these blunt measures could be supplemented by multimethod approaches in order to give a more nuanced interpretation of outcomes, as is being done for the evaluation of the German Pro Kind intervention, which will use a combination of interviews, observations and administrative data.132 We also spent a significant amount of time talking to FNP teams, including observing a home visit, in order to understand the perspectives of family nurses and the mothers and families they represent. This enhanced our ability to interpret the results we observed in the data.

Limitations

The potential sources of bias and the steps we took to try to minimise these are summarised in Table 18 and the following sections.

Indication bias due to unmeasured confounders

The main limitation of our analyses was that the administrative data we used did not capture information on all relevant maternal characteristics associated with enrolment, and there were important differences in the characteristics of mothers enrolled in FNP and those who were not, meaning that mothers enrolled in FNP were more vulnerable than the comparison group. For some outcomes, including a Good Level of Development, the findings in the propensity-score-matched cohort reversed those of the unadjusted comparison: while the crude analysis of this outcome showed lower rates in mothers enrolled in FNP, the adjusted comparisons showed higher rates. This suggests that the propensity score matching (and multivariable regression analyses) was able to account for some confounders. However, the higher rates of low birthweight and maternal unplanned admissions after birth in the FNP arm could indicate that residual confounding remains.

Although we used a large number of pre-enrolment maternal characteristics from health, education and social care data for our propensity score matching, it was only possible to evaluate the balance between propensity score-matched groups in terms of these measured characteristics. For measured characteristics, we can only know if groups were balanced on the presence of a characteristic (e.g. a history of hospital admissions for adversity) but not on the nature or severity of that characteristic. This means that although the groups looked comparable in terms of measured characteristics, we likely did not fully account for important unmeasured differences between those that did and did not receive FNP: a phenomenon known as the treatment paradox.133 For example, some important vulnerabilities (such as family violence) may not be disclosed until a trusting relationship has been built with providers and may not be captured in administrative data at all.109,110,134 Our approach was restricted by data that were systematically collected by services and the quality of recording of that data.

Characterising usual care provided to mothers not enrolled in Family Nurse Partnership

Usual care for adolescent mothers differs substantially between LAs and over time (including varying numbers of health visitor contacts and additional services for adolescent mothers).135,136 Health visitors are under-resourced, which makes it likely that not all adolescent mothers are receiving the standard of care that should be expected.65 Although national data on health visiting are collected, this is not yet well completed nor disaggregated by maternal age.137–139 We did not have any information on midwifery care and therefore were unable to identify areas in which mothers received care from specialist teenage pregnancy midwives. Bespoke data collection was not feasible within the timeframe of this study; we were therefore unable to include a quantitative measure of usual care in our models, limiting the precision of our intervention effect estimates.

We did not have access to information on additional services that mothers might have accessed as a result of interactions with the family nurse, including, for example, housing services. If mothers not enrolled in FNP had access to high levels of support from other teams in the community, it may be difficult to identify an incremental effect on outcomes for those enrolled in FNP.23

Misclassification bias (defining our intervention and comparison groups)

In order to make comparisons with mothers who had not been enrolled in FNP but who were eligible, we needed to define the population of adolescent mothers who would have been eligible for the FNP but who were not enrolled due to living in an area in which the FNP was not offered at the time of their pregnancy. If information on recruitment dates was inaccurate, misclassification bias could occur if mothers were categorised as being eligible for the FNP when they were not or vice versa. Site activity dates and geography were key to defining these populations, but this information was not readily available. FNP sites merged and split over time, site boundaries moved and sites discontinued or joined the FNP at different times.

We used extensive data quality checks for LA-specific site activity dates and geographic coverage with the FNP National Unit to minimise misclassification of unenrolled eligible mothers. However, we may have overestimated unenrolled eligible mothers and therefore underestimated some enrolment rates. Conversely, birth ascertainment in HES (97% of total births in English hospitals according to the ONS)69 means some mothers may have been wrongly excluded from the denominator. Neither under- nor over-ascertainment of unenrolled eligible mothers is likely to have biased the analysis of predictors of enrolment, since they should not be associated with risk factors.

Missing data on gestational age at booking appointment (32%) and birth (13%) required us to make some assumptions to determine eligibility for the FNP. Some mothers with missing gestational age at first antenatal appointment may have booked after 28 weeks but were retained in our cohort (2.6% were excluded due to known booking appointment after 28 weeks vs. 7% expected based on primary care records – S Syed, personal communication). We cannot know whether the 0.5% of children who were discharged to social services at birth were reunited with their birth mothers; inclusion of these children in the analysis of outcomes in the 2 and 7 years following birth might have led to a small bias in either direction (as these babies may not have been exposed to their birth mothers during follow-up).

Consent bias due to lack of information on mothers who declined the intervention

It was not possible to identify eligible mothers who were offered enrolment but declined. As a result, these mothers were included in the comparison of mothers who had never been enrolled in the FNP. This could lead to consent bias: if mothers who declined were more vulnerable than those who accepted, it might lead us to underestimate the intervention effect. FNP sites had limited aggregate information on these mothers. Some sites reported that although a small number were particularly vulnerable mothers (e.g. involved with social care services), the majority of mothers who declined were those with strong social support.

Missing data due to linkage error

Eight-three per cent of mothers aged 13–19 in our cohort were linked to NPD. Some unlinked mothers would genuinely not have been captured in NPD due to attending an independent school or a school in a different country. Other mothers may not have been linked due to inconsistencies between identifiers recorded in both data sources. HES records for a small number of FNP mothers who did not link to HES would mistakenly have been treated as belonging to the comparison group, which again may lead to misclassification bias. The extent to which these missed links lead to bias depends on how the unlinked records are dealt with in analysis: in this study, all mothers/babies were retained in the analysis (but we could only measure educational outcomes for those who linked).140 Determining the potential direction of bias is complex, particularly when successive linkages are performed (such as FNP data linked to health data, then to educational data). We identified groups of mothers who were more at risk of linkage bias or missing data by comparing the characteristics of FNP mothers who did and did not link with HES and NPD records (see Appendix 2, Tables 24 and 29).

Data approval and access delays

It took 4 years from data applications being submitted to the final linked data set being available for analysis (see Appendix 10). Lengthy application processes and delays in receiving administrative data have been widely documented elsewhere.141–144 Cross-sectoral data linkage adds other delays, including in data providers sending identifier information to trusted third parties for linkage and in migrating data to a single trusted research environment. These delays impeded analysis time: linked data were finally available 1 month before our initial grant endpoint, an insufficient period within which to deliver results based on extensive administrative data cleaning, assessment of linkage quality, construction of study cohorts and optimisation of quasi-experimental approaches. This was exacerbated by problems with the secure server on which the data were hosted, which meant that there were many days when the data were not accessible.

Other limitations

We did not consider costs in this study. The Building Blocks trial found that the FNP costed on average an additional £1812 (95% CI −£2814 to £5557) per mother compared to usual care and that there were only minor differences across trial arms, with largest drivers of costs being hospital inpatient stays.80 Costs associated with the involvement of other professionals interacting with family nurses, education, social care and primary care costs have never been considered.

In our analysis of the number and duration of visits, we did not account for the content or other factors (e.g. continuity of care) that might influence the effectiveness of the programme. We were unable to report on mortality due to discrepancies in recording of deaths between data sources.

Our PPI was limited by difficulties in engaging FNP graduates. Although the participants who attended our focus group were enthusiastic about remaining involved, the practicalities of keeping in touch with young mothers who have changing priorities over time meant that we struggled to retain mothers through the course of the study. Difficulty in engaging FNP graduates was also described in the Building Blocks 2–6 trial.24

We designed the study to describe a wide range of outcomes, due in part to the fact that the primary outcomes specified in the original Building Blocks trial were challenged and to enable us to compare the findings from the study with other evaluations of the FNP.22,23,116 A limitation of this approach is that the large number of comparisons made could generate spurious results since 1 in 20 findings will be by chance when using 95% CIs. Therefore, the few statistically significant findings that we observed out of the many that were evaluated should be considered in the context of patterns of effect (rather than relying on p-values) and the extent to which they align with previous studies.

Patient and public involvement

In the process of designing the study, we engaged with two groups of mothers (some who had participated in FNP, some who had not). We discussed the use of administrative data for research (including concepts that mothers were not previously familiar with, such as de-identification and data linkage), linkage of health and education data without explicit consent and the use of these data specifically for evaluating the FNP. Workshop participants were supportive of the proposed study and fed back that linking health and education data for mothers and their children was a good idea. Participants strongly agreed with sharing their data so that services could be improved and future mothers could benefit and wanted to know how their data had been used to benefit others. Mothers were strongly supportive of taking into account maternal education and area, and wider family support for the FNP, in order to understand whether the programme worked.145 We incorporated these opinions into the study plan by planning to evaluate whether the linkage of FNP differs by area and by maternal risk factors.

This Study Steering Committee included a former adolescent mother, who was consulted throughout the study period. When it was not possible for her to attend meetings, we organised separate meetings to present the same findings and discuss progress. Towards the end of the study, our lay member withdrew from being involved in the study due to changes in family circumstances. Our committee also initially included one mother who had completed the FNP but later dropped out. In order to increase the level of involvement in the study, we contacted FNP sites via the FNP National Unit to recruit mothers participating in FNP part way through the study and approached two additional mothers (one current and one former FNP participant), who both agreed to join the study. However, communication was challenging as it was difficult to find a time that fitted their schedule. The former FNP participant eventually contacted us to say she no longer had the time to be involved, and we stopped receiving replies from the current FNP participant (whose baby was very young), reflecting the challenges of effective PPI with this target population.

We ran a training day for parents interested in learning more about administrative data research in October 2019 at Great Ormond Street Hospital, which was well attended by parents. On the training day, we gave an overview of what is meant by administrative data research and data linkage, with examples from specific data sets, including those used in this study (HES). We discussed the appropriateness of linking data on mothers and children for purposes such as this study. Participants were strongly supportive of the data being used in this way and fed back that they found the training extremely useful and interesting.

Parent participants will help coproduce information on the study for the FNP and institutional websites, help with interpretation of results and identify the most appropriate methods of dissemination.

Equality, diversity and inclusion

This study included all mothers and their children who were ever enrolled in FNP and was therefore representative of the population receiving the intervention. However, we do not have information on the characteristics of mothers who are not approached for or decline to enrol in FNP, and we do not know the extent to which FNP is inclusive of all mothers who stand to benefit. We aimed to fill gaps in evidence on whether the effect of FNP varied across different subgroups and present results of the intervention effect stratified by ethnicity, age and deprivation.

Our research team did not include those from groups who are generally under-represented in our field of study, but there was a range of experience and expertise across the research team. We provided development opportunities for more junior members of the team, including providing opportunities for teaching and supervision experience and supporting a promotion application. We attempted to include members of the public who were representative of the eligible population of adolescent mothers across a geographic range in England, although this proved to be challenging in practice.

Conclusions

Our findings support previous evaluations of FNP in England, which show no impact on child maltreatment outcomes and no evidence to suggest that further benefits will appear as children get older but are in contrast to qualitative research describing the positive impacts of the programme. There remains uncertainty about whether intensive home visiting is effective for reducing child maltreatment because of the potential for bias relating to the selection of the most vulnerable mothers into FNP on the basis of unmeasured characteristics (which could mask beneficial effects), the lack of information about usual care for adolescent mothers not enrolled in FNP (which could introduce bias to the null) and the challenges of interpreting outcomes captured in administrative data. The findings in this report should therefore be interpreted with three main considerations.

First, we could only control for the fairly crude maternal risk factors associated with enrolment in FNP that are captured in administrative data. The increased rates of maternal unplanned admissions post pregnancy for adversity and mental health-related diagnoses in the FNP group could indicate that there may be additional residual confounding due to bias by indication, even after propensity score matching. Given that we know that mothers enrolled in FNP had more indicators of vulnerability at enrolment than the comparison group, such residual confounding could have limited our ability to detect beneficial effects of the programme. The weak evidence for small improvements in school readiness as measured by a Good Level of Development in the EYFSP and reductions in the number of rapid repeat pregnancies may therefore reflect larger positive effects of the programme. However, these outcomes should still be considered in the context of being the only ‘positive’ effects amongst the many outcomes that were evaluated. The importance of these small effects should be examined alongside other population-level interventions aiming to improve child health and development outcomes.

Second, the increased rates of unplanned admissions and A&E attendances in the mother and child associated with FNP may reflect appropriate care-seeking as a result of advice and support from family nurses rather than residual confounding. Babies born to mothers enrolled in the FNP were more likely to be born preterm and with low birthweight; these are health conditions which are influenced by pre-conception health, and babies with these conditions would be expected to have appropriately increased rates of hospital contacts throughout childhood. Increased rates of admissions in the mother could also be interpreted as demonstrating that family nurses can have a long-term effect on maternal health care-seeking behaviours.

Third, FNP might have positive effects on a range of other outcomes that are not captured in administrative data (Box 3). This study was not able to identify effects on changes in self-reported parental mental health, sense of self, well-being, confidence, behaviour and parent–child engagement and interaction, including for fathers. Nuances in behaviour change are difficult to measure: for example, a mother may not give up smoking completely but may change how she smokes by not smoking in the home and not allowing others to smoke in the home. Existing qualitative work demonstrates overwhelming support for the programme from mothers who have been enrolled and from family nurses who can see the changes and impact that the intensive service has provided for the families they have worked with.99,130 Mothers often stay in touch with their family nurses after graduation and can reflect on how the programme has helped them transition, become confident as a parent, and be a more responsive parent.130 There have been no dissenting voices as to the overall efficacy of the programme, and each mother reported it had been helpful in some way – either ‘life changing’ or serving to consolidate existing skills and qualities.99 However, there is a lack of conclusive, systematic evidence supporting interventions for preventing child maltreatment more generally.146–149 Evidence is stronger for benefits at later follow-up: ‘sleeper’ effects may mean that benefits of interventions need time to emerge as parents become more practiced and confident in positive parenting strategies over time.150 Expecting to detect effects of home visiting that starts in pregnancy on birth outcomes and on relatively insensitive child development measures may also be unreasonable in the context of the social disadvantage, discrimination and other challenges that adolescent mothers face before, during and after pregnancy.151

Despite reductions in adolescent pregnancies over recent decades, there remains a significant population of young and vulnerable mothers in England who need intensive support. Currently, the majority of these mothers are not receiving support from FNP, as it is not offered in all areas, and is only offered to around one in four mothers in areas in which it is commissioned. There is strong support for FNP locally, and FNP practitioners report that mothers participating in the programme develop more reflective parenting and awareness of their child’s needs. Without better evidence, removing support for young mothers could be harmful, especially in the context of increasing social disadvantage and widespread health visitor shortages that are already putting pressure on other services.152 However, there remains uncertainty about which elements of intensive interventions are most effective, for whom and when and whether it is better to commission highly intensive services for a small portion of the target population or to extend and enhance universal services to better support all adolescent mothers.

Implications for policy and practice

Delivering interventions to high-risk families

To reduce social inequalities, effective interventions need to be available to all those who meet eligibility criteria. In 2010–7, most adolescent mothers at the highest risk of adverse outcomes were not receiving FNP, and we have few data on the support that they did receive during and after pregnancy. Eligible adolescent mothers living in areas with many births to adolescent mothers were least likely to receive FNP support. Moreover, adolescent mothers with similar vulnerabilities in areas where FNP was not commissioned were unlikely to have had access to equivalent intensive support. Our recommendations for delivering interventions to high-risk families are:

• Commissioning of interventions to adolescent mothers should aim to provide adequate support to meet the needs of all adolescent mothers (not just a subset of them) through increased provision of intensive services in line with local needs and the availability of other local services. For example, commissioning could be linked to the expected number of adolescents giving birth in an area; local priorities and other factors influencing local commissioning should be routinely recorded so that different commissioning strategies can be evaluated.
• Information about the maternal characteristics associated with outcomes should inform targeting in the context of insufficient commissioning of the programme.
• Other groups of mothers who have been shown to be at equally high risk for adverse child outcomes to adolescent mothers, such as those living in deprived areas and those with a history of unplanned hospital admissions for adversity or mental health conditions, should also be considered for intensive support (as they are in FNP ADAPT sites).60,89
• Decisions about the appropriate level of care for each family (including ‘dialling up and dialling down’ the intensity of support) should be based on continuous evaluations of who is most likely to benefit, supported by a more complete recording of vulnerabilities antenatally and real-time linkage of routine health and social care data.89,153 This could add to primary prevention strategies aiming to improve reproductive choices for women about the timing of their pregnancies.

Improving data to support quasi-experimental research studies

There remains uncertainty about whether intensive home visiting is effective for reducing child maltreatment because of the potential for bias relating to the selection of the most vulnerable mothers into FNP on the basis of unmeasured characteristics (which could mask beneficial effects) and the lack of information about usual care for adolescent mothers not enrolled in FNP (which could introduce bias to the null). Understanding variations in usual care provision among both mothers enrolled in FNP and controls is also necessary to better estimate the incremental effect of FNP and account for any unexpected variation in usual care during the evaluation period. Such information would allow more nuanced interpretation of results, including, for example, if the programme worked better in one local area than another. Our recommendations for improving the available data to support evaluations of complex interventions using administrative data are:

• More complete recording of the characteristics that are used to prioritise mothers for enrolment in each site (including start and end dates of these prioritisation characteristics) and other important maternal characteristics for adjustment purposes.
• Detailed recording of programme metadata, including site activity dates and geography and criteria being used by FNP teams in order to correctly define eligible groups of mothers who were and were not enrolled or eligible for the intervention. These programme characteristics should be available in an easily searchable central repository.
• Individual-level or aggregate data on characteristics of all mothers-to-be offered enrolment and those who declined versus those who accepted enrolment.
• Improved, high-quality data on usual care, that is, community health contacts of all eligible women at the individual level (including, e.g. public health or teenage pregnancy midwife services, number and type of health visitor contacts, number of children’s centres). These data could be captured in the Community Services Dataset, but the completeness of this data set is variable over time and across areas.137,139

Recommendations for research

Development of the intervention

• More research is needed to understand which elements of intensive interventions are most effective, for whom and when and to help inform decisions about whether it is better to commission highly intensive services for a small portion of the target population or to extend and enhance universal services to better support all adolescent mothers. For example, research could evaluate the effectiveness of enhanced midwifery support in pregnancy delivered by specialist teenage pregnancy midwives.
• Research is needed to understand the spillover effects of the programme on non-FNP mothers.154 While the FNP may enhance other services (e.g. as trained family nurses take on other roles) by dissemination of trauma-informed approaches and understanding of the teenage brain, qualitative data also suggest that professionals are concerned about the licensed nature of FNP and not being able to share freely with wider health visiting colleagues or use it to change practice across services.111 We do not yet know if FNP diverts resources away from the usual care that an adolescent mother should receive or if there are any other unintended harms of the programme.155
• There is ongoing work in the ADAPT sites to tailor the programme more closely to individual mothers and families. Further robust evaluation is needed of modifications to the FNP, including the changes included in the ADAPT sites and including outcomes for mothers aged 20–24 (who we could not assess in this study due to small numbers enrolled with sufficient follow-up).
• Better understanding is needed of how the programme and outcomes are affected by different modes of delivery, such as those that were implemented during the COVID-19 lockdown.156 An evaluation of the FNP during lockdown in Scotland found that all survey respondents reported continuing home visits during lockdown, though some mothers were not comfortable receiving home visits. Mothers reported invaluable support and felt their family nurses provided stability, advice and care, particularly those who became socially isolated.

Improvement of delivery, including targeting

• Research is needed to find ways to support achieving continuity of care and to support learning and wider sharing with health visiting colleagues, which would allow FNP to change practice across the service.
• Availability of more accurate individual- or household-level deprivation measures (including those available through Unique Property Reference Numbers) would improve our ability to create closely matched groups, provide enhanced information on the social support available within a household and help more effective targeting and follow-up of the many young mothers experiencing socioeconomic disadvantage.157

Measuring the right outcomes

• We have reported initial findings from a qualitative study of the experiences of parents and practitioners involved in FNP, and further findings from this work will inform our understanding of the mechanisms of effect of FNP and for which outcomes we are most likely to see effects. This is particularly important in the context of quasi-experimental research studies, where bias and/or data limitations are likely to influence findings.
• Better measures of changes from enrolment in maternally reported well-being, confidence, mental health, parent–child interaction and child behaviour would allow us to understand more nuanced effects of the programme not routinely captured in administrative data.
• Future research should also consider outcomes for fathers, who have not been considered to date in evaluations of FNP and who are hard to capture in administrative data.158
• Research is needed to understand whether FNP can mediate outcomes for those with adverse birth conditions (e.g. preterm birth and low birthweight) and whether there are any residual (‘sleeper’) effects for subsequent children.
• More research is needed to understand whether the FNP has an effect on mortality for mothers and children, rates of abortions and the reasons for hospital admissions, including malnutrition.
• Future studies should quantify the total costs of FNP compared to usual care, including non-contact time of family nurses and additional resources used by FNP mothers and their children.

Design of future studies

• A randomised controlled trial is needed to further evaluate the impact of the number of FNP visits or contacts on outcomes, including self-reported outcomes and other measures of child development, for example, Language Environment Analysis (LENA).159 As a trial will also have its challenges, including attrition, recruitment and standardisation of usual care, experimental designs should also have a strong qualitative component to extend the contextual explanation of variation, for example, across localities.
• Other ways to analyse the data and avoid excluding high-risk mothers from analysis could be explored, for example, using target trial emulation.160
• Linkage to primary care and the Maternity Services Dataset could enable investigation of a wider range of risk factors.
• Routine linkage of education, social care and health data should be used to enable more efficient evaluations of early interventions.161 Some organisations have called for the collection of a unique identifier in data for all children’s services to enable services to work in a more joined-up way, which could be used to facilitate research in the future.162,163