Synthesis (work package 4)

Introduction

The aims of the research reported here were to:

• identify, characterise and explore the potential benefits and challenges associated with DF in general practice in England
• design a framework for future evaluations of the effectiveness and cost effectiveness of DF interventions.

This chapter sets out the synthesis of the four major sources of evidence that we have produced in order to meet those objectives and which are presented in the preceding chapters: review of literature (WP1), survey of GP practices (WP2A), survey of patients (WP2B) and the qualitative exploration (WP3) that combined case studies in GP practices with stakeholder interviews. In this study, we are using Weiss’s approach to theory-based evaluation as our theoretical framework.70 After describing our approach to the synthesis of the findings from all parts of the study, and the role of PPIE within that, the themes identified from the research are specified and a matrix is presented summarising the corresponding data. Following the approach defined by Weiss we use our findings to create a programme theory, which specifies the mechanism of change, and an implementation theory, which describes how DF is carried out and the associated barriers and enablers. We conclude the chapter by proposing a framework for future evaluations of approaches to DF in primary care.

Synthesis methods

We have applied a pragmatic, iterative and cumulative approach to synthesising the findings from all WPs and to developing the programme and implementation theories. The underlying approach has been to triangulate what we have found in each WP, summarising the results in a triangulation matrix,180 which has evolved as the study progressed and as different WPs’ evidence became available. The analyses within, and findings of, each WP have been detailed in earlier chapters. Those findings were brought together in a series of three workshop meetings of researchers and the PAG as each WP in turn was completed. Two team members who work in GP practices (one GP and one administrative) participated in all three synthesis workshops. Thus, all three workshops had PAG members and primary care staff inputs in addition to researcher inputs. Box 7 briefly summarises the PAG contributions.

In November 2021, the findings of the survey of practices (WP2A) were combined and compared with those from the literature review (WP1) in an online workshop with researchers from both WPs (nine researchers in all) and three PAG members. Briefing material had been circulated to all participants in advance of the workshop, which included plenty of time for clarification and further explanation as well as for identification of themes and discussion of the extent to which different evidence sources reinforced, added to, or differed from each other in their implications.

In July 2022 the emerging findings from the ethnographic case studies at GP practices and the stakeholder interviews (WP3) were analysed alongside the previous synthesis of WP1 and WP2A (practice survey). The participants at this online workshop were 12 researchers, including from all 3 WPs and 3 PAG members. The approach to this workshop mirrored the previous synthesis workshop, with pre-workshop briefing and time for clarification and further explanation of the material from all three WPs being considered, as well as for the synthesis work itself.

The final synthesis workshop, adding the findings of the patient survey (WP2B) and including the fully finalised WP3 findings, took place at an in-person meeting of the research team in London in September 2022, including 15 researchers and 3 PAG members. The approach to the third workshop was the same as for the two prior workshops. The following paragraphs describe the themes that were agreed after detailed discussion to represent the sum of triangulated findings from all WPs.

The synthesis process concluded in early December 2022 following an online discussion with national and regional stakeholders – from NHS Digital, NHS England and from NHS Devon Integrated Care Board (ICB) – at which the overall synthesis results were discussed, probed and, as a consequence, further clarified.

Themes after triangulation

The process of triangulating our findings from the different WPs has enabled us to build a detailed and balanced understanding of a range of approaches to DF, and patients’ experiences of it, in NHS primary care in England. After the cumulative synthesis of the findings from all WPs, the research team identified three scene-setting themes from the evidence, which relate to the context in which DF takes place (when it happens), and eight themes related to types of DF, their implementation and effectiveness.

Scene-setting themes (Table 18)

1. Value and purpose of digital services. The usefulness of DF depends on the value of the digital services whose use might be facilitated.
2. Digital is conflated with other routes to accessing care. Patients desire access to care, and digital is only one type of route to accessing NHS primary care services, along with face to face and telephone.
3. COVID. The onset of the pandemic, and of the response of primary care to the pandemic, was associated with a shift away from patients attending GP premises and towards remote transactions. Combined, these had a major impact on the need for, and provision of, DF.

Digital facilitation themes (Table 19)

1. Defining and identifying DF can be difficult. There is lack of clarity among some practice staff and patients about what comprises DF.
2. Types of DF. It can be provided actively (e.g. practice staff recommending to patients that they use an online service and telling them how to do so) or passively (e.g. a poster in the waiting room, a recorded telephone message played to callers to the practice number); and DF can be reactive (helping patients when they ask for assistance) or proactive (without waiting to be asked).
3. Supporting initial sign-up or sustained use over time. When an online service is being introduced or rolled out, facilitation may focus on supporting patients to register for a service. But once they are registered, they may need support (e.g. problem solving) to continue using the service.
4. Who delivers DF. There is no clear consensus over whose responsibility it is and who should be expected to fund the resources needed to provide DF.
5. Enablers of DF, including the usability, or ‘user-friendliness’ of digital platforms.
6. Barriers to DF, including staff attitudes towards digital services and stereotyped assumptions about the capabilities of some patients.
7. Digital access and DF for different subgroups in the population. There is evidence in the literature that some groups are relatively disadvantaged when using online services and they might benefit from targeted DF efforts to support them.
8. Effectiveness. Is DF effective at supporting patients to access and use online services; and which types of facilitation are more effective than others.

Mapping of findings to theoretical framework

We used Weiss’s theory-based evaluation as our theoretical framework.70 Weiss distinguishes between ‘programme theory’, which specifies the mechanism of change, and ‘implementation theory’, which describes how the intervention is carried out. We used the theory-based evaluation approach to understand how, and in what ways, different models of DF bring benefits and challenges to general practice and patients. We did this by drawing on the findings of the evidence synthesis, surveys and case studies to develop the programme theory and the implementation theory.

To develop the ‘programme theory’ we used a realist approach to describe provision of DF in terms of:

• context
• the flow of activities that comprise the intervention
• the theory and assumptions underlying the intervention, including the intended benefits/outcomes.

The ‘implementation theory’ explored moderating factors which influenced the extent to which the process and outcomes were achieved, including factors acting as barriers and facilitators to practices offering DF or to different groups of patients using them.

Programme theory

Context

Digital facilitation needs to be understood within the context of current NHS policy, which seeks to encourage greater use of digital services across the NHS, and with technology viewed by policy-makers as addressing some of the demands facing health care,181 including streamlining of service delivery, improved triage systems and improved access for patients.3 Our focused ethnography found, however, that at the practice level there was a lack of shared understanding about what digital services should achieve, and that the need for and value of digital services was not always clear to practices themselves. This was also true for patients who sometimes saw methods of digital access to primary care services as a barrier to receiving care rather than as a method for expanding or improving their access to care. Supporting the uptake and use of digital services by patients is challenging when the value of those services is unclear to practices and patients.

Within primary care, practices are mandated to provide online access to services such as appointment booking, prescription ordering and access to medical records. However, practices individually determine which platforms or providers they will use to deliver them, and if and how they will support patients to use them. General practices in England have long been incentivised to introduce online consultation platforms, with particular commercial platforms being market leaders.182 Provision of an online route of access to patients is now part of the GP contract in England.183 These platforms have been promoted to general practice as tools for demand management while also fulfilling the need to provide patients with another access option. While practices can and do collaborate through various organisations such as primary care networks (PCNs) and ICBs, the responsibility for providing digital services and supporting patients to use those services rests with individual practices or groups of practices. The result is a patchwork of systems and approaches shaped according to the resources available to and choices made by practices in a context where commercial suppliers and other NHS organisations, such as local ICBs, may be influencing the decisions made. This might prioritise various (sometimes competing) aims including managing demand, enabling access or improving patient experiences of care. On the ground, this looks like a complex and varied collection of digital services at each practice that might change as the needs of the practice change or new providers and services are developed.

Digital facilitation should also be understood within the context of the pandemic. Our survey of GP practices showed that the pandemic led to the rapid introduction of remote ways of accessing and delivering primary care including online booking of appointments, online prescription ordering and remote consultation. Implemented alongside reductions in face-to-face services such as in-person appointment booking and repeat prescription requests, the rapid introduction of digital services during the pandemic accelerated the digital agenda set out by NHSE181 that patients will use digital services to access care, regardless of their ability to do so.12

Our focused ethnography found that some practices that had provided proactive DF to support patients prior to the pandemic stopped those activities during the pandemic due to pandemic-related pressures on the practice or because the facilitation required face-to-face activities that were unsuitable given the social distancing requirements, for example, by using tablet computers to sign patients up to digital services as they sat in the waiting room. These facilitation activities were not resumed after the pandemic, and in some practices it was perceived that the need for support had passed. In others the capacity to do this under COVID restrictions was limited.

The pandemic also placed further strain on a primary care system that was already facing high levels of demand from a growing and ageing population. The national vaccination programme during the pandemic relied on GP facilities (among others) and personnel to deliver much of the work, drawing extensively on primary care resources. The primary care workforce is also facing declining numbers of GPs and primary care nurses, with many practices struggling to recruit and retain staff.5,184 A recurring finding in our research was how competing demands on staff time make it challenging to provide DF.

Flow of activities comprising digital facilitation

Digital facilitation can take a wide variety of forms, encompassing active forms of facilitation such as practice champions, training and workshops, and passive forms such as informational leaflets, text messages and recorded messages on practice phone lines. Facilitation is reactive when it is responding to direct patient requests for support, and proactive when practices provide support before issues arise. We found that in practice, most facilitation efforts are reactive, and that many could be further described as ‘ad hoc’ in that they were seemingly applied anew each time an issue arose, relying on the digital skills of practice staff to respond to the immediate needs of the patient. This reflects the tendency of general practices (and patients) to conflate DF with the digital services themselves, and therefore to see facilitation efforts as troubleshooting activities to enable access to the services rather than as part of a wider effort to enable participation in a digitised, and digitally prioritised, healthcare economy.

Determining the ownership of and the responsibility for developing and delivering DF are not straightforward. In practice, most DF is delivered by general practice reception and administrative staff. This is consistent with reactive approaches to facilitation that view it as troubleshooting or enabling access to services, in that patient facing roles are given responsibility for helping patients resolve issues with access. Proactive DF requires planning and resources to develop and deliver it, that may be beyond practices’ capabilities. Our research found that practices and patients saw a role for ‘other parts’ of the NHS (e.g. CCGs, NHSE), in addition to a role for individual practices, in supporting patients to use digital services. Practices noted the need for wider patient education efforts about the benefits of booking appointments online, for example.

The release of the NHS App, which gives patients in England access to many digital healthcare services including appointment booking, prescription ordering and access to medical records through their smartphone, has added to the confusion over who has responsibility for supporting patients to use digital services. Practices were not able to support patients in their use of the NHS App as it sat outside of the practice with separate digital support via an NHS helpline and web page. Patients tended to view the NHS App as just another way to access the practice’s services and therefore turned to the practice for help. This contributed to confusion over who has responsibility for supporting patients with using services.

Theory and assumptions underlying digital facilitation

Digital facilitation has the potential to support the move to digital primary care services. It could do so through direct and indirect pathways. Digital facilitation directly supports the move to digital services when patients are helped to access or use specific digital services to meet their current or future healthcare needs, for example, when they are helped to order a prescription through an online platform. Indirect support occurs when facilitation helps patients to become more confident users of digital services through, for example, enhancing their digital literacy. Both pathways have the potential to lead to more people using digital services more often when accessing primary care, in theory leading to benefits such as efficiencies for practices and enhanced access for patients.

In our research we found that the pathways linking DF to anticipated beneficial outcomes are more complex than the version theorised above and that tensions over the concept and goals of DF contribute to this complexity. Firstly, our focused ethnographic case studies found that practices and patients conflated digital services with access to an appointment. This means that although DF is theoretically valuable to practices, they may not see themselves as providing facilitation to digital services more generally, but instead as just facilitating access to an appointment. Patients see themselves as seeking help with accessing practice services rather than as receiving support with digital services. It is unsurprising then that most DF provided by practices is ad hoc and reactive, resolving immediate issues of patient access rather than seeking to build patients’ capacity to access digital services more broadly.

Which raises the question: what does it mean for DF to be effective? The move to online primary care provision assumes that it can help address some of the pressures facing primary care while improving patient choice, convenience and ease of access.3 Current government plans for digital health emphasise the importance of registering patients for services such as the NHS App.181 Much of the DF that we identified in our research also focused on registering patients for digital services, with less effort directed at ensuring that patients remain engaged with those services and continued to use them. But if the path from DF to the theorised benefits it brings requires more than this, for example, if patients need to be supported to build their digital confidence so that they can engage with multiple or changing digital services, then DF approaches will need to evolve to meet these needs if they are to be effective. In our discussion with national and regional stakeholders, the point was made that patients need to be encouraged to see digital channels as part of their relationship with their GPs, rather than as substitutes for that relationship.

Digital facilitation has a role in combatting potential inequalities in accessing health care in an NHS with enhanced digital opportunities and enhanced digital access to primary care services. Our survey of GP practices suggests that practices are aware of the need particularly to support groups such as older age groups, non-white ethnicities, those in lower socioeconomic groups, those in poorer health, and individuals in rural settings who may struggle to access digital services. However, our findings also suggest that DF efforts to support potentially vulnerable groups of patients fall short. Specifically, our surveys found that although older patients were frequently mentioned by GP practices as a target of DF efforts, in practice older patients were less likely to be aware of or use DF and were less likely to be told about digital services or helped to use them. Evidence from the literature review and focused ethnography suggests that assumptions and stereotypes about older patients being less digitally competent might make practice staff less likely to engage in facilitation efforts with them.

Implementation theory

Contextual factors affecting implementation

Our findings highlight the variation between general practices in terms of capacity for, and approaches to, DF. The populations served by general practice also differ. In our case studies we observed that a practice serving a young population of predominantly minority ethnic groups had very different requirements to a practice serving an older white British population where digital uptake was lower and health needs were different. To date, approaches to digitising general practice have been broadly uniform in their approach, without apparent tailoring for the individual needs and requirements of populations.

Moderating factors affecting implementation, barriers and facilitators

Our findings suggest, unsurprisingly, that constraints on the resources available to GP practices to engage in DF affect implementation. Most practices that we surveyed agreed that they did not have sufficient capacity to support patients in using online primary care services to the extent they might wish. The literature review highlights how automating some aspects of patient engagement through e-mail templates, protocols or scripts can help mitigate staff capacity pressures, but we saw only limited evidence of these types of tools being used in practice. Passive facilitation efforts such as posters and leaflets use less staff time than more active approaches, and there was some evidence from our survey that passive approaches are used more often by GP practices. In the ethnographic fieldwork, there were examples of posters on the wall in practices and information provided on practice websites, but these were not widely publicised or updated.

Findings from the ethnography suggest that DF benefits from having someone in the practice with specific responsibility for it. This person may be chosen because they are perceived to be the most knowledgeable digitally rather than for any formal reason. Approaches such as practice champions, where a dedicated staff member acts as a cheerleader and source of information for DF, can be challenging to put in place for capacity constrained practices.

The quality and usefulness of digital services also affect approaches to DF. Some digital platforms are more difficult to navigate or register with and may not be seen by patients or practice staff as bringing added value. Other platforms can be considered a good fit, being easy to use and bringing tangible benefits to patients and staff. Services that are a good fit require less DF, for example, online repeat prescription ordering, was found to be relatively well used across the survey and focused ethnographic case studies. Findings from the ethnography suggest that issues with more difficult-to-use services are seen by practice staff as being a problem with the digital service itself, and outside the ability or responsibility of staff members to resolve. Findings from the patient survey show that patients are less willing to engage with digital services for which they see little value. For DF to be successful, practices would therefore need to devote more resource to supporting and encouraging patients to use digital services that are a poorer fit.

This highlights some related barriers to DF. First, practice staff need to understand how to use the digital services themselves in order to provide support to patients to do so. In some practices visited for our ethnography, it seemed to just be assumed that younger members of staff would know how to navigate digital technologies, but this was not always true. Practices have a range of digital platforms, which sometimes change, in addition to the NHS App, which is outside of practices’ control. This diverse and changing mix of services can create barriers for staff in maintaining the requisite skills to provide facilitation. It can also present challenges for patients in understanding what services are available and for which support might be available.

Unclear lines of responsibility for DF also present a barrier to implementation, with practice staff, patients and other stakeholders (e.g. other NHS organisations, suppliers of digital technology) all assuming that at least a portion of the responsibility for facilitation lies with the others. This contributes to the reactive, single-issue style approach to DF we observed in our ethnographic research. When no one is responsible for supporting patients with broader issues of digital access such as digital confidence and literacy, then the limited resources available for support will be targeted at addressing immediate needs. Resources to develop thoughtful, potentially cross-practice or cross-platform, support for patients are unlikely to be invested in by practices or other stakeholders without clearer lines of responsibility.

Implementation could be supported by having a clear understanding of what DF the practice population needs, and how best to deliver it, and for this to be understood by all members of staff in the general practice. Conversely, a barrier to implementing DF is that (as evidenced by our literature review and focused ethnography) practice staff may make assumptions and stereotypes about older patients being less digitally competent and not worth engaging with in facilitation efforts.

Digital facilitation seemed to be functioning well where practices had allocated a specific role for DF and provided resource to support this. The literature review showed the importance of having buy-in from staff about the purpose of digital services and facilitation efforts, including ensuring that staff trust and understand the specific technologies being supported. Although the ethnography found that practices often lacked this clear rationale at a general level, conflating digital services with access to practice services, it found that when digital services were a good fit for practice and patient needs it was then easier to support patients to use those services. Clarifying the purpose and value of digital services is therefore a critical aspect of DF.

Evaluation framework

The aim of the study was to ‘identify, characterise and explore the potential benefits and challenges associated with different models of digital facilitation currently in use in general practice in England’ and to subsequently ‘design a framework for future evaluations of the effectiveness and cost effectiveness of such interventions’.

Our research has enabled us to understand the following:

• the concept of DF and how it is understood (or not) by all involved parties (patients, general practice staff, wider stakeholders)
• the types of DF currently in use
• to what extent DF is implemented and for what purposes
• the outcomes that are important to practices and their patients and what they expect DF to achieve, if anything
• the groups of patients who seek or have experienced DF
• the types of research methodology that are likely to be suitable to further investigate DF.

The level of understanding obtained via our research has allowed us to devise a framework to help to structure future research into DF. This is based on the synthesised findings of the study and theoretical framework findings.

The latest version of the Medical Research Council (MRC) framework for evaluation of complex interventions185 defines evaluation as ‘going beyond asking whether an intervention works’ and suggests a broader approach is taken that includes factors such as ‘theorising whether it works’ and ‘how it contributes to system change’. In line with these methodological developments, our framework takes a correspondingly broad approach to assessing effectiveness and cost effectiveness.

We begin by outlining how DF may be considered an intervention, who is responsible for DF, the key patient groups that should be considered in any future evaluation, the potential outcome measures for evaluation including those related to cost, and the possible different types of future evaluation including key areas of focus.

Digital facilitation as an intervention

Our findings have highlighted the amorphous nature of DF and indicate that DF is part of the complex processes involved in delivering access to care for patients in general practice. This has implications for the future evaluation of DF. DF is flexible and adaptable, functioning as a complex intervention. The end point, achieving access, is the important factor for all parties. Therefore, DF may be defined as the support required and provided to enable patients to achieve access digitally. DF is one or more of numerous possible, mutually non-exclusive interventions.

Where an intervention is used, consideration should be made for the understanding of the intervention by all involved parties with our evidence indicating that DF is not a concept that is easily grasped and may be better linked to how patients access and receive health services rather than being a standalone intervention. There should be clear rationale for introducing it.

Responsibility for digital facilitation

We observed that understanding about who delivers DF was key to whether it happened at all. Any future evaluation of effort at DF as an intervention should consider who is responsible for it. This would include:

• Who has overall responsibility for the success or failure of DF?
• What are the responsibilities of general practices and their staff with regard to ensuring safe and appropriate delivery of DF?
• Which staff groups are involved in delivering it? What does this involve and how does it fit with their current role?
• What are patient responsibilities when engaging with DF?
• If DF is delivered by an external party, how will they interface with the practice staff and patients and what responsibilities does this bring?
• How does DF fit into the wider health community beyond general practice?
• Will there be support from regional or national or other external organisations? This might include commercial suppliers of digital platforms, and charities to boost digital literacy in certain population groups.

Patient groups and potential for inequalities

A key element of this study has been to explore how different patient groups engage with and are affected by DF. This has had a particular focus on the potential for inequalities. Future evaluations should consider:

• where DF is most needed and most likely to be successful (there is little point in focusing DF in primary care on people without internet access, for example)
• differences in incidence of benefits and costs/savings to patients, for example, burden on patients and likelihood that it will lead to them being able to achieve access to health care in the way intended
• weight of importance for patients, which may be low relative to importance for practices and other interested parties.

Potential outcome measures for future evaluation

Potential outcome measures for use in future studies should aim to capture quantifiable impacts where possible, and also to explore qualitative aspects of DF. They should consider both intended (e.g. increase in levels of use of online services) and unintended consequences (e.g. staff finding themselves expected by some patients to act as a help desk for IT problems). Here we outline potential outcomes, but these are not exhaustive. The outcomes can be collected for subpopulations within the overall population so as to permit evaluation of inequalities, or their mitigation, between different groups within the population.

Service use and delivery-related outcomes

We have shown that access is an important outcome for practices and patients, and that DF plays a role in achieving it. The interface between access and digital is an important consideration when choosing outcome measures related to service use and delivery:

• awareness of digital services (where DF is positioned to change this)
• levels of registrations with digital services, levels of use
• change in use of online services at practice level and individual patient level. Specifically, levels of use after registration and attrition rate
• measures of access within the general practice: numbers of consultations, speed of access, rates of digital access versus other routes.

Patient and healthcare professional-related outcomes

Patient and healthcare professional outcomes are likely to be best assessed using both quantitative and qualitative approaches to allow triangulation of data. The exact nature of outcomes needed would depend on the composition of the intervention and extent of patient and healthcare professional engagement:

• Patient experiences with engaging in DF and with use of services facilitated. This could include existing patient-reported experience measures of satisfaction or specific intervention focused factors, for example, helpfulness of specific approach, reassurance gained by engaging with DF.
• Perceptions about whether DF has worked for them, for example, was it effective in helping them reach their desired end point?
• Patient confidence in using digital services.
• Practice staff reported experiences with facilitation, for example, ease of delivery, impact on workload. Views on patients’ use of services facilitated, for example, do staff think DF works?
• Perceptions about whether DF has worked for them, for example, does it provide a mechanism for staff to support patients in the way expected?

Cost outcomes

Any assessment of cost should take a societal perspective, moving beyond NHS and personal social services perspectives. Significant areas of costs and/or of likely cost savings that should be examined in future studies include:

• costs of DF to practices (training, staff time, materials and equipment)
• costs/savings to practices from increased use of online services by patients
• costs/savings to the rest of the NHS
• savings/costs to patients from using online services – for example, time savings from not attending a GP practice in person.

Different types of future evaluation including areas of focus

Given the difficulties in defining DF as an intervention and the many involved parties, measuring effectiveness would need to take a holistic approach. This fits with the latest MRC guidance on complex interventions which suggests that evaluation goes beyond whether an intervention works.

Our exploratory research has identified areas for focus that were previously unconsidered:

• the role of general practice administrative staff in supporting digital services and the appropriate ways to support this
• understanding how different approaches to DF work for population subgroups
• supporting sustained patient use of digital services, moving the focus beyond registration
• understanding the factors that need to be addressed to achieve a certain baseline of digital services use, and how service use can then be developed to support specific patient groups with particular needs
• understanding the extent of patient demand for digital services to access and use general practice and thus the level of DF needed to achieve this.

This is not an exhaustive list but highlights the complexity of this area and some appropriate areas of focus to ensure benefit from future research.

Deviations from the original proposal

There were no deviations in methods for WP4 to report from the original proposal.