Results

The eight case studies feature the scoping stages of evidence syntheses conducted by the HSDR Evidence Synthesis Centres between 2017 and 2020: three cases from the University of Exeter, three from the University of Sheffield and two from the University of York. Table 2 shows the considerable diversity of topics and types of synthesis method that these case studies covered. Please note that for ‘synthesis type’ we have used the terminology used in the source project and report, and that there is some variation and overlap in use of terms. In particular, evidence maps and mapping reviews are essentially the same in purpose and final product, and scoping reviews (as a defined product) should be distinguished from the scoping stage and processes that are the focus of all of the case studies. In addition, most of these named subtypes of evidence synthesis are also systematic reviews (in the sense that they had clearly defined questions and explicit methods for identifying, assessing and summarising included evidence sources). We have decided to retain the terms as used in each case study review, rather than to retrospectively impose a standardised typology. Further details of the specific synthesis methods used in each case study review, the rationale for their choice and any patient and public involvement (PPI) are in Appendix 1, Table 13.

The case studies that started with a clearly known policy customer/decision-maker and with a clearly stated review question or evidence gap/need are presented first, through to those that had neither a clearly stated policy customer nor a clear initial question. In this way, the order of presentation of the case studies should, in principle, move from those with clearer predefined initial scopes to those with more open-ended and uncertain scopes. The classification of case studies according to these two criteria is presented in Table 3.

Case study 1: rapid evidence synthesis of ‘Digital-First Primary Care’

This topic was given to the HSDR Evidence Synthesis Centre at York, having been identified as an urgent topic from an NHS England primary care workshop that focused on digital aspects of care. The York team had not been involved in the workshop and so sought further information from NHS England about evidence requirements and scope. Following initial clarification and discussion with representatives from NHS England in June 2018, the team undertook a rapid, responsive evidence synthesis between July and December 2018. Throughout the project, the York team maintained e-mail and telephone contact with a senior policy lead in the new business models team of NHS England’s Strategy and Innovation Directorate. This person co-ordinated the involvement of representatives from other teams (e.g. primary care).

The protocol was posted on the team’s webpage,17 as it was not eligible for registration in PROSPERO. The final report was published in full18 and a brief evidence summary produced.19

The original questions articulated at the workshop were as follows:

• What are the most effective automated systems management approaches that result in high levels of general practitioner (GP) engagement?
• How do you present data to ensure change in practice?
• What are the barriers to and motivators of using digital technology that drive cultural and behavioural change within primary care practitioners?

Following initial discussions with NHS England, which identified a broad and far-reaching list of themes and questions, an iterative production process was agreed to undertake the work in stages. We presented the findings after each stage to discuss progression onto the next stage. After the discussions that followed the second stage, the work was concluded and the report completed.

The rapid responsive evidence synthesis was undertaken to inform NHS England policy in ‘Digital-First Primary Care’. The principles and some aspects of systematic review methodology were applied to ensure transparency and reproducibility. The two stages of the synthesis consisted of (1) scoping the published review evidence and (2) addressing a refined set of questions produced by NHS England from the evidence retrieved during the scoping stage. Given that ‘a full systematic review was not possible, given the time and resources available’, the team ‘conducted a rapid synthesis of the most relevant evidence identified during the scoping exercise (stage 1) to establish if and to what extent these questions can be answered by the identified research’.18 Patient and public representatives were not directly involved in the development of the synthesis aims, methods or interpretation.

The following questions were addressed in the second stage:

• What are the benefits – to patients, GPs and the system – of digital modes and models of engagement between patients and primary care?
  • As GP workload and workforce is the main threat to primary care, how do we use these innovations to alleviate this, rather than only increasing patient convenience and improving their experience?
  • Which patients can benefit from digital (online) modes and models of engagement between patients and primary care?
  • What channels work best for different patient needs and conditions?
  • Are there differences in synchronous and asynchronous models?
• How do you integrate ‘digital first’ models of accessing primary care within wider existing face-to-face models?
• How do you contract such models and how do you deliver them (what geography size, population size)?

The final inclusion criteria were as follows:

• population – any primary care medical staff and patients (or their caregivers) of any age and/or other medical professionals
• interventions – as the known literature rarely conceptualised interventions as ‘digital primary care’, any form of non-face-to-face interaction, including e-mail, online/video, messaging and artificial intelligence-led systems or triage (or any of these alongside telephone consultation)
• outcomes – impact on care in terms of effectiveness and safety, patient access/convenience (including which patients are able to use digital consultations and what conditions are appropriate for non-face-to-face engagement), system-level efficiencies and related issues, such as workforce retention, training and satisfaction
• study design – systematic reviews, meta-analyses and other forms of evidence syntheses (any related primary studies encountered were included where relevant, although primary research evidence was not systematically searched).

Summary of key challenges/choices and scoping decisions

Challenge 1: clarifying the questions of interest and deciding the most appropriate method of synthesis

The brief topic outline we were given at the start of the project was broadened and expanded during the initial communications with NHS England. They were interested in patient-focused digital innovation in primary care and identified four broad themes and nearly 20 separate questions for which they wanted answers. The questions were wide-ranging, covering issues of contracting and implementation, as well as effectiveness, cost-effectiveness, benefits and risks. They reflected the interests and priorities of the various sections of NHS England involved in implementing ‘Digital-First Primary Care’.

It became clear in these discussions that it was not possible, at that stage, to identify a clear and focused research question (or questions) of high priority to NHS England that was amenable to evidence synthesis. Instead, we identified the need for a responsive and iterative approach to support the needs of the policy-makers.

We adopted a multistage approach to the work. We began by searching the research literature to scope available evidence syntheses. We extracted the key characteristics of all included documents and produced an interactive database of published and ongoing evidence that could be ordered or filtered according to these characteristics, incorporating links to the full-text versions where available. We produced an interim report that summarised the key evidence identified in this scoping exercise, along with an annotated bibliography and the interactive spreadsheet.

We presented the interim report to NHS England and asked them to decide whether it provided the information they needed or if a gap remained to be addressed by further rapid evidence synthesis. NHS England responded with seven questions that reflected specific ‘live’ policy areas in which they were most interested. Therefore, we undertook a second stage and conducted a rapid synthesis of the relevant evidence identified from the stage 1 scoping exercise with the seven research questions identified by NHS England, forming the basis of a thematic framework. Critical appraisal of included evidence was facilitated by relevant assessment tools and reporting standards used to inform judgements about the internal and external validity of included research results presented in the thematic synthesis.

We produced a report that combined both the initial scoping exercise and the rapid synthesis undertaken at stage 2. Again, we presented the findings through a summary report and teleconference to representatives from various NHS England teams (e.g. new business models, primary care, digital and workforce). Following this presentation, and the subsequent discussions, the topic was concluded.

Summary of response to challenge

We needed to adopt an iterative, responsive approach to this topic and revise our methodology accordingly. The initial scoping exercise was undertaken to provide a high-level overview of the available evidence, followed by a rapid evidence synthesis. A full systematic review was not possible, but aspects of systematic review research methodology (such as a priori inclusion criteria, critical appraisal of included evidence, and process measures to avoid bias and errors) were applied to introduce a level of transparency and reproducibility.

Challenge 2: identifying and incorporating recent and ongoing research

As a result of the initial scoping exercise and discussions with other researchers working in the field, we became aware of recent and ongoing projects, as well as two open NIHR calls for a proposal relevant to the topic. Although we searched for evidence syntheses, we incidentally identified some recent or ongoing primary research studies.

We were keen to ensure that we did not duplicate effort and also to alert NHS England to academic groups actively researching the topic. Therefore, we made contact with the researchers who were eager to engage, providing early sight of their draft reports in confidence to be included in our work. They were also keen to engage further with NHS England to inform ongoing policy work.

Summary of response to challenge

To avoid duplication of effort and to ensure that policy was informed by the most recent research, we highlighted the ongoing work alongside review evidence and facilitated contact between NHS England and the research authors.

Reflections and lessons

Multiple stakeholders from different areas within the same organisation identified a broad and far-reaching list of themes and questions that reflected the differing remits and priorities of the stakeholders. This required an iterative production process, undertaking the work in stages and presenting the findings at each stage to support their needs. We identified a tension between ongoing engagement with busy and changing stakeholders and fast-moving policy in an area of rapid and ongoing innovation. This required pragmatic adaptation of methods to meet the needs of stakeholders to balance methodological rigour with the usefulness of outputs, while maintaining transparency. In addition, in an area of rapid and ongoing innovation where peer-reviewed evidence may not be currently available, it was helpful to highlight ongoing work and academic colleagues working in the area for further direct dialogue with the stakeholder.

Disseminating the outputs from this multistage project was problematic. To respond to the stakeholder needs, we produced an interactive database of published and ongoing evidence (i.e. enabling sorting and filtering of the evidence base on key characteristics, accessing direct links to full publications and/or contact details of researchers). Reducing this to a final textual HSDR report stripped away this functionality and obscured much of the underlying work.

Case study 2: review of experiences of the ‘nearest relative’ provisions of the Mental Health Act 1983

Origin and context of the review topic

This evidence review topic was initially notified by the NIHR HSDR programme in direct response to an urgent request for help in gathering evidence for the independent review of the Mental Health Act 1983 (MHA),16 which was being conducted during 2018. In the UK, the MHA16 is the central piece of legislation that determines the circumstances and processes for when and how people experiencing mental distress can be compulsorily detained for assessment or treatment.

For the Exeter centre, it was the most rapid of the rapid systematic reviews that we had conducted during the contract period, as well as our first review project that was specifically about a piece of legislation (rather than a health-care intervention or model of health service delivery). We got our first indication of the review topic on 3 January 2018 and submitted the final report by the end of March 2018. The scoping process was, therefore, compressed and pragmatic, for example relying closely on the knowledge and stated needs of the main policy contact within the team conducting the independent review. Although we could identify a time point when the review questions and inclusion criteria were agreed, in other respects the scoping stage was less distinguishable from the main searches and review. In effect, it was a ‘live’ review protocol and the review was shaped and delivered through regular ongoing contact with the policy customer.

The scoping and systematic review was conducted by the HSDR evidence synthesis team at Exeter, and the full review protocol was published on the PROSPERO database (CRD42018088237).20

The review topic was initially stated as one of several very brief issues or aspects of the MHA16 sent to the team by NIHR. The subtopics that the policy leads initially prioritised as ‘the most pressing’ were:

• the rights of relatives
• consent and capacity
• the criminal justice system (tribunals and restricted patients)
• legal clarifications.

Our work was steered towards the first topic and, specifically, the ‘nearest relative’ provisions of the MHA.16 These are the legal requirements and associated practices that govern the involvement of the spouse or close biological relatives (i.e. the ‘nearest relative’) of a person who is being detained compulsorily for mental health reasons under the MHA16 (sometimes referred to as ‘being sectioned’). These persons are involved primarily as an advocate and support for the person in mental distress, especially in relation to decisions about care.

Although we had no specific review question, the remit of the independent review of the MHA16 included specific questions that it wanted to answer (Box 3) and these usefully shaped the possible directions of our rapid evidence review.

Overarching question: are the powers of the nearest relative within the MHA16 appropriate?

What is the role of the nearest relative under the MHA?16

What rights are available for nearest relatives and are these appropriate?

What are service users’ and carers’ experiences of nearest relative provision?

What is the impact on (1) service users’ access to family and carer support and (2) the experiences of families and carers involved in a person’s care?

How does this role impact information sharing and confidentiality?

Are there any possible alternative models (e.g. international best practice)?

Our contact person in the independent review team expressed the purpose of the needed review as the following main review question:

• Are the powers under the MHA16 relating to the appointment and involvement of the nearest relative appropriate (i.e. working)?

By the time we completed scoping the topic and finalised our review protocol, the planned systematic review aimed to address the following question:

• What are the experiences of services users, family members, carers and professionals of the use of the ‘nearest relative’ provisions in the compulsory detention and ongoing care of people under the MHA?

However, the remit and context of the independent review, together with guidance from the policy customer about the main perceived problems with the current legislation, revealed specific stages or aspects of these provisions that were of particular interest. These included the processes of identification of the nearest relative, displacement (i.e. replacement) of the assigned nearest relative, decisions about care, service users having access to support from those carers and loved ones who they wanted support from, and issues of patient confidentiality and information sharing. These stages/aspects were expressed in the PROSPERO protocol under the ‘outcomes’ of interest (see Table 8).20

We aimed to answer this question by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria. Table 4 summarises how the scoping process refined review questions and defined the inclusion criteria, compared with the initial, briefly stated review topic. The ultimate choice of a rapid form of qualitative evidence synthesis (i.e. framework synthesis) was mainly dictated by the review question’s focus on people’s experiences, combined with the very short time frame. Although there was some PPI in interpreting and writing up this rapid evidence synthesis, people or carers who had been directly affected by the nearest relative provisions were not involved in scoping the review or its questions.

Summary of key challenges/choices and scoping decisions

Key challenge 1: uncertainty about the type of evidence that would best answer the questions

The first question was whether or not any prior research had focused on patient and carer experiences of the nearest relative provisions within UK legislation. The questions of interest seemed so specific that we doubted if research would address them. Our review group’s contract and work remit extended to the potential for primary research or surveys, where such methods best answer policy-maker or NIHR HSDR questions. However, two early discoveries enabled us to focus on the evidence review component, rather than primary research. First, we identified a 2017 survey of people’s experiences of the MHA by the Mental Health Alliance and Rethink Mental Illness (London, UK).21 Second, we learned that the independent review team were, themselves, conducting a survey of carer experiences.

It became important to know early on whether the policy customer expected the review team to summarise research evidence or, given the distinct possibility of finding no or very little research, to extend coverage to other relevant non-research forms of evidence in the public domain (e.g. blogs, online discussion forums). Our first teleconference with the policy customer revealed a clear preference for research-based evidence, if available. This was to ‘reduce possible bias’ and achieve a breadth of evidence in the review process, given the other information sources and consultation on which the independent review processes would be drawing.

Therefore, the policy customer gave us a clear and early steer to search for and synthesise research evidence only. Nevertheless, at that time, we were not sure if there would be sufficient, or even any, good-quality UK-based research evidence on patient and carer experiences of the nearest relative provisions. Therefore, we also noted that, later, we might need to extend the scope to grey literature and less research-based sources of evidence, or include evidence from beyond the UK. This highlights pervasive uncertainty about the available evidence for answering alternative potential questions, and the motivation to make best use of a review team’s core skills and resources in responsive, commissioned policy-informing reviews.

Decision 1: to limit to research evidence only

The response of the team was to commit to reviewing evidence from research only. However, we also made some aspects of the review scope and protocol conditional (i.e. on the amount and quality of research evidence found). For such reviews, the scope and protocol are live and adjustable plans that are expected to change in response to the evidence that is found. They often lay out sequences of possible scopes, iteratively refined, realigned and renegotiated. Had the team found only one or two includable research studies, this decision may have been revisited. This kept options open within a very rapid review timescale.

Key challenge 2: deliverability within 8 weeks

This was the overarching challenge and a non-negotiable constraint, against which the other challenges and decisions were all weighed. The deliverability of any review would critically depend on the number, richness/quality and, therefore, amenability to formal synthesis of the studies found, and this would not be known until 4–6 weeks before the report submission deadline. Concerns about this were compounded by our awareness that synthesising qualitative research, usually being an interpretative process, takes time and requires a team approach. We had team discussions about the value of also searching for and including survey research (i.e. which may not be qualitative). However, having found enough qualitative research, we did not ultimately search for survey studies. (In any case, such a review would be unlikely to provide more recent and relevant evidence than a reanalysis of the 2017 Mental Health Alliance survey responses.)

Decision 2: manage expectations of the policy customer through regular contact and review of progress

We had teleconferences with the policy customer every 1 or 2 weeks, at a minimum, during the first half of the review. This was not so much a decision, but a direct response to our uncertainty (i.e. to keep open communications with the policy customer about whether or not we were finding evidence and what evidence we were finding). Had there been too much evidence to synthesise, there would be discussions during these teleconferences as to what evidence we should focus on. The short time frame of the review, undoubtedly, also played a part in us choosing not to look for evidence from beyond the UK or prior to 1998 (see Key challenge 3: how old and from which other jurisdictions would evidence be relevant to experiences of the UK Mental Health Act? and Decision 3: to include only UK evidence and evidence after 1998).

Key challenge 3: how old and from which other jurisdictions would evidence be relevant to experiences of the UK Mental Health Act?

As mentioned, this was our first review of an aspect of legislation. Although people in other countries might have experience of equivalent legislation and practices for including family members of close friends in decision-making for people experiencing severe mental distress, only people from the UK (i.e. the jurisdiction of application of the MHA16) would have directly relevant experience of the nearest relative provisions.

Decision 3: to include only UK evidence and evidence from after 1998

This was an early and relatively easy decision (i.e. to not include evidence from beyond the UK). The legislation and the nearest relative provisions within it were so specific to the jurisdiction of the UK that evidence from elsewhere would have very little applicability to the UK legislative and mental health-care context. It was also because the broad context of this review – and the scope of the interim report of the independent review of the MHA16 that it was to inform – was the creation of a fuller understanding of the current and past problems with legislation and its use. Had the review context required a stronger understanding of potential alternatives and legislative solutions, then perhaps an international and comparative review of equivalent provisions in mental health legislation in other countries may have been useful. Another factor that may have informed this decision was that we knew that some research evidence would originate from Scotland, where an alternative to the nearest relative provisions was already current practice. It was, therefore, likely that this evidence, and its comparison with studies in England and Wales, would be more likely to inform the independent review than studies from mainland Europe or North America.

The cut-off date of 1998 was partly to limit the size of the screening task (i.e. to address deliverability in 8 weeks) and partly because our stakeholders confirmed that the experience of the nearest relative provisions would have differed before and after the adoption of the Human Rights Act 199822 in the UK. Therefore, only evidence about experience of the mental health legislation after 1998 was judged as relevant to future possible legislation.

Key challenge 4: conducting rapid synthesis of qualitative research evidence

Given the extremely tight timelines, and the researcher-intensive nature of qualitative evidence synthesis, we were very nervous about the possibility of finding too many studies that met our inclusion criteria. This would have risked making the review undeliverable, rushed and of reduced quality, or having to retrospectively exclude studies.

Decision 4: using ‘framework synthesis’ and prioritising conceptually richer studies within the synthesis

Although a full thematic synthesis would have been the preferred approach, this would not have been possible within the limited time frame of this review. However, committing to conducting a rapid synthesis of qualitative research was less risky and more feasible because a streamlined, pragmatic approach to qualitative evidence synthesis was available, which could be applied to a modest number of studies. Fortunately, we found 35 papers from 20 studies; however, 22 papers provided only half a page of qualitative evidence relevant to the five study objectives.

This meant that we could, with some adaptations and innovations to the method, conduct a pragmatic, rapid, best-fit framework synthesis of the qualitative studies within the 6-week time frame. Within the method, a three-stage approach was used. First, relevant data were extracted according to the research objectives of our review from the 22 papers with half a page of relevant qualitative evidence. This process identified the six richest studies that contained the most data relevant to our research questions. Themes were selected from these studies to further refine the framework. In the third and final stage of the synthesis, thematic synthesis of the data enabled the corroboration and extension of the framework.23

Reflection and lessons

In this review, where review scoping and the conduct of the review overlapped almost completely, regular contact with the policy customer was an important way of staying responsive to the state of the evidence as it emerged. Fine-tuning of the review in response to the evidence needed to be in keeping with the overall protocol and fit with the available time and resources. These were decisions that could largely be manage from our side. However, the review also needed to be relevant to the policy customer’s needs, which demanded a close working relationship in such a rapidly evolving review.

The development of search strategies to scope the literature and identify pockets of evidence that were amenable to review underpinned many of the decisions that were made in the early stages. Scoping searches using Google Search and Google Scholar (Google Inc., Mountain View, CA, USA), and the initial bibliographic search development, played a key role in the decisions to exclude experiences of the MHA16 via criminal justice proceedings and to focus on research published after the 1998 Human Rights Act.22 Throughout this process, the integration of the information specialist in the review team was crucial. The pressing requirement to rapidly scope the available literature meant that our bibliographic database search strategy was developed and ready to run as soon as the final focus of the review was agreed. Supplementary searches were iterative and responsive to the emerging literature, within the prespecified boundaries set out in the protocol. In particular, this included iterative citation searching or ‘snowballing’ of key primary studies. Indeed, this approach to searching is a core part of qualitative searching, but the rapid nature of this review brought to the foreground the importance of this type of searching.

The searches also determined that there were no available frameworks or theories directly relevant to our research objectives that we could use to inform the initial framework for our framework synthesis. This informed our decision to structure our initial framework based on our initial research objectives, and then refine the framework using the themes from the primary studies included in our reviews, followed by thematic synthesis.

Case study 3: regulating and inspecting integrated health and social care in the UK

In 2018, the NIHR conducted a topic identification exercise within the broad area of professional regulation in UK health care. The exercise generated approximately 30 possible research topics, with some articulated as research questions and others as stated areas thought to be lacking in evidence. The NIHR assessed each topic and prioritised the following questions for referral to the York HSDR review team:

• What factors enable delivery of an effective system of regulation and inspection in an environment where services are increasingly being provided on a multiagency (including third sector) and local basis in or close to people’s own homes?
• How can we overcome the barriers to delivering effective joint regulation and inspection in a way that makes sense from the perspective of the individual accessing the care and services? To what extent is it possible to achieve this without the need for major legislative or structural change?

The research team conducted extensive stakeholder consultation to shape these questions into research questions, define terms and adapt the scope where necessary (see Summary of key challenges/choices and scoping decisions). There was some PPI in the review project, but at the report-writing and dissemination stages only. Ultimately, we planned to conduct a rapid scoping review to identify and classify published material that could potentially address four key questions:

1. What models of regulation and inspection of integrated care have been proposed (including approaches taken in other countries)?
2. What evidence is available on the effectiveness of such models?
3. What are the barriers to and enablers of effective regulation and inspection of integrated care?
4. Can barriers to effective regulation and inspection be overcome without legislative change?

The rationale for conducting a scoping review was based on these mainly descriptive, rather than evaluative, review questions (especially questions 1–3). The protocol was posted on the team’s webpage,24 as it was not eligible for registration in PROSPERO. The final report was published in full25 and a brief evidence summary produced.26

As can be seen by the above contrasting sets of questions, consulting with diverse stakeholders resulted in a broader scope, with a series of research questions focusing on the extent and nature of the evidence that were best suited to a scoping review. A map of the evidence was created, underpinned by the following inclusion criteria:

• Publication type – both empirical and non-empirical publications were eligible for inclusion. Empirical studies could be of a qualitative or quantitative design. Non-empirical publications could include discussion or theory papers, as well as other descriptive pieces, such as editorials. Letters or news articles were excluded, as were publications that reported the findings from inspections of care services.
• Setting – publications were primarily focused on the integration of health and social care provision, for example services delivered jointly by NHS providers and local authorities. However, publications could also focus on care provision that is delivered across other settings/sectors by different professional groups working together, for example across primary or secondary care. Care providers could be in the public, private or third sector, and services could be aimed at both adults and children.
• Focus – publications needed to have a primary focus on the regulation and/or inspection of integrated care. Reference to the governance of services more broadly was not sufficient for inclusion. Integration could be either horizontal or vertical in type and be at a macro-, meso- or micro-level.
• Outcomes – empirical studies could report on any outcome relevant to the regulation and/or inspection of integrated care. This could include issues related to implementation, such as views about barriers and enabling factors. Non-empirical publications could focus on any relevant issue, including proposed models of regulation or outcome frameworks.

Each included publication was coded on various key characteristics, including topic (i.e. regulation or inspection), country, population/setting and document type (e.g. empirical research, models or frameworks, or theoretical). This information was used to produce a high-level descriptive overview, which characterised the nature of current literature on the regulation and inspection of integrated health and social care in the UK, as well as identifying research gaps.

Summary of key challenges/choices and scoping decisions

Key challenge 1: reconciling multiple independent stakeholder consultations

The original questions prioritised by the NIHR originated from Health Inspectorate Wales (Merthyr Tydfil, Wales). A teleconference held between the York team and representatives of both Health Inspectorate Wales and Care Inspectorate Wales (Merthyr Tydfil, Wales), provided background and context to their proposed questions. However, before this teleconference could be arranged, the research team had already consulted other key stakeholders about the proposed questions.

Initial contact with representatives from the Professional Standards Authority (London, UK) and Care Quality Commission (London, UK) provided the team with an overview of health-care regulation in the UK and associated research. They expressed a willingness to assist with the proposed work and arranged for researchers to attend (1) the Professional Standards Authority’s Policy and Research Forum and (2) a meeting of the health and social care regulators. The former included representatives from the Professional Standards Authority and various regulatory organisations [e.g. the General Pharmaceutical Council (London, UK), the General Chiropractic Council (London, UK), the Health and Care Professions Council (London, UK), the General Optical Council (London, UK), the General Osteopathic Council (London, UK), the General Medical Council (London, UK), the General Dental Council (London, UK) and the Nursing and Midwifery Council (London, UK)]. The latter included senior managers from the Care Quality Commission, DHSC, the General Dental Council, the Health and Care Professions Council, the Local Government and Social Care Ombudsman (Coventry, UK), the Nursing and Midwifery Council, the General Pharmaceutical Council, the Parliamentary and Health Service Ombudsman (London, UK), the Professional Standards Authority, the Medicines and Healthcare products Regulatory Agency (London, UK), Social Work England (Sheffield, UK) and the General Medical Council. The research team also held a separate teleconference with a representative of the General Medical Council.

In each case, different stakeholders emphasised slightly different areas of interest, deviating to a greater or lesser extent from the original questions. The Health and Social Care Regulators suggested that interdisciplinary regulation of online primary care was the topic of interest. The diverse disciplines and regulatory organisations providing online care provoked questions about effective regulatory oversight and complaints related to care. The potential relevance and utility of the international evidence was particularly emphasised.

The General Medical Council agreed that, even given the differences in the regulatory architecture and frameworks across countries, there was scope to learn from other health and social care systems that face similar issues and potential risks. Of particular interest was multidisciplinary team working: understanding the barriers to, enablers of and issues around responsibility, given joint working and multidisciplinary collaboration.

The representatives of Healthcare Inspectorate Wales and Care Inspectorate Wales described recent policy initiatives to promote the integration of health and social care in Wales and the implications of such policies for regulators. They elaborated on their initial questions to pose a series of fundamental questions about the regulation and inspection of integrated health and social care provision, including what models exist, their effectiveness, the barriers to their implementation and non-legislative means of overcoming these barriers.

Decision 1: share all scoping conversations among the stakeholders and encourage ongoing engagement

It can be seen that – when given exactly the same information to scope a rapid evidence review – different stakeholders proposed differing but inter-related research questions. As these stakeholders are the likely audience for the review, we felt it crucial that they remain engaged with its development. Consequently, as part of the protocol development process, we shared our individual stakeholder discussions with the wider group. An introduction to the draft protocol outlined how these shared discussions informed the basis of a draft scope, which the stakeholders were then invited to further comment on and amend, as appropriate. Each of the stakeholder groups responded positively to this invitation.

Beginning with two detailed questions derived from a prioritisation exercise may have been a barrier to initial engagement from some stakeholders. Some stakeholders would defer to the questions as worded, focusing on interpreting the authors’ intentions rather than expressing their own perception of research priorities. The research team, therefore, encouraged stakeholders to use the initial questions as a starting point for the discussion about research priorities.

Key challenge 2: not all stakeholders are familiar with rapid evidence synthesis

It was clear that stakeholders’ experience and knowledge of rapid evidence synthesis varied widely, particularly in relation to formulating implementable research questions. We wanted to avoid the potential hazard of arriving at a scope that was simply amenable to evidence synthesis rather than reflecting a genuine stakeholder knowledge need.

Decision 2: empower stakeholders to take advantage of the available resource

The research team began each stakeholder group consultation with a presentation to ensure a common level of understanding before beginning the scoping work. This presentation covered:

• the nature of the work conducted by the HSDR Evidence Synthesis Centres
• the types of evidence synthesis and the related research processes
• the role of stakeholder involvement in evidence synthesis
• required stakeholder input for this specific project [e.g. helping develop the scope of the research question(s)].

The presentation aimed to create conditions in which stakeholders could lead the discussion, speak with confidence about their evidence needs and describe how rapid evidence synthesis might serve those needs. The research team could outline the types of research method that might be feasible or suitable for proposed research questions, but were careful not to influence topic-specific discussions. It became clear that stakeholders’ information needs were best served by quickly identifying and organising the apparently diffuse and disparate literature on regulation and inspection of integrated care. Consequently, a rapid scoping review was considered the most appropriate starting point.

Reflections and lessons

Successful scoping of rapid evidence synthesis questions requires the involvement of stakeholders who are knowledgeable, enthusiastic and engaged. However, it also requires researchers to developa framework in which stakeholders can lead conversations about the scope while remaining within the parameters of what can be achieved by a rapid evidence synthesis. This requires researchers to focus on listening and facilitation in the early stages of consultation, moving towards providing stronger guidance on possible methodologies once a level of consensus among stakeholders has been achieved. The time and effort required to develop these processes and practise these skills should not be underestimated.

Case study 4: social care access for BAME and LGBT+ populations – a rapid realist review

This evidence review topic was initially proposed by the Research Programmes Branch – Health and Care Section within the UK’s DHSC. The scoping and subsequent rapid realist synthesis was conducted by the HSDR Evidence Synthesis Centre at the School of Health and Related Research (ScHARR), University of Sheffield. The protocol was published on PROSPERO (CRD42019158250) and was also made publicly accessible on the NIHR’s published Sheffield Evidence Synthesis Centre web page (URL: https://njl-admin.nihr.ac.uk/document/download/2031791; accessed 21 June 2021). The review was conducted between November 2019 and the end of June 2020 and was published in 2021.27

The review focus was initially stated as the following topic:

• addressing diversity and inequalities in access to social care services.

Access to social care services had featured as a dominant theme within the James Lind Alliance’s Adult Social Work Top 10,28 reflecting a recent priority setting exercise. Rather than representing a specific priority from the James Lind Alliance exercise, this theme encapsulated several priorities. It was, therefore, felt by the NIHR and the DHSC that coverage of these priorities could be interpreted and understood by targeting the review at groups for whom access to social care could be particularly challenging. Initial discussion with the contacts at the DHSC focused on two particular groups of interest: ethnic minorities, and lesbian, gay, bisexual, transgender plus (LGBT+) people with social care needs. This process emphasises how a broad theme may be refined to particular populations, both as a specific target for policy initiatives and in the assumption that by addressing issues in access for specific groups, this can have wider implications for other population subgroups.

Rather untypically, the review customer (the DHSC) had already conducted a preliminary literature scoping around race disparities in the use of social care. The review team, therefore, faced two particular challenges: (1) how to add value to the initial literature review by complementary activity and (2) how to inform the social care access literature from the extensive work completed in health care. Both of these considerations shaped our subsequent approach to scoping the literature. In addition, the review scope and the perceived relevance of the review was discussed with members of the Sheffield Evidence Synthesis Centre’s PPI Group.

Although classic scoping of effectiveness review questions involves population (or patient type), intervention, comparator and outcomes, with the subsequent addition of study types (i.e. PICOS), once the quantity and quality of available literature has been established by scoping, we have identified the need for addition of a further ‘S’ [population (or patient type), intervention, comparator, outcome, study types, synthesis method (PICOSS)]. This additional ‘S’ relates to the type of ‘synthesis’ (i.e. a preliminary assessment of the literature allows us to identify what type of synthesis will be possible and useful to match the requirements of the topic to the needs of the commissioner). In this instance, we identified that realist methods would extend the evidence base beyond the descriptive literature review that the DHSC team had already explored to gain an understanding of how ‘access’ worked (or did not work) for different groups.

As part of the scoping, our jointly trained lead reviewer/information specialist identified a realist review on the topic of access in primary care.29 Our team was interested in the extent to which the conceptual models from that study might transfer to social care, particularly in relation to a trajectory or pathway of access.

Our initial scoping identified a significant number of data relating to the use of social care services by refugees and asylum seekers, particular by those with poor mental health. Checking with policy colleagues, via our direct point of contact, revealed that these groups would be included among the identified populations of interest. It further revealed a particular interest in the intersectionality between our two populations: ethnic minorities and LGBT+ people.

Following these explorations, discussions with the DHSC resolved the specifics of the review to the following research questions:

• What are the barriers to and facilitators of accessing social care for (1) ethnic minorities and (2) LGBT+ populations?
• Using ‘if–then–leading to’ statements (i.e. context–mechanism–outcome configurations), can we map access to social care against access pathways to health care (Ford et al.29) to provide additional explanations for what influences access to social care?

We aimed to obtain answers to these questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria, using a realist review approach. The rationale for conducting a realist review was to address the primary interest ‘for whom’ (or for which groups) access to social care works, as rapid realist methods have been specifically developed for work with policy-makers and realist methods also allow us to look at the contexts of access to social care and the role of mechanisms in determining the outcomes around access (see also table 13 in Booth et al.27). In addition, the DHSC acknowledged that the characteristics of their topic, which the team matched to the required review type (i.e. limited existing explanatory literature, mainly descriptive), were previously ‘reviewed’ in the UK literature and they were looking for added value from the evidence.

A key feature of realist approaches, however, involves prioritising studies or sources that can inform and develop candidate programme theories (i.e. provisional explanations of how interventions work). In this sense, even though the original research questions were suitably specified for developing the protocol, we continued to refine the focus of the scope as we started to interpret the evidence. An interim remote meeting was arranged between the ScHARR team and the DHSC to prioritise four or five programme theories to be addressed by the review. A notable feature of the review was the need to specify certain concepts early in the process. Table 5 summarises how the scoping process defined the inclusion criteria and shaped the scope.

Key challenges/choices and scoping decisions

Key challenge 1: the role of definitions in specifying relevant concepts

Dixon-Woods et al.34 make the distinction between an integrative synthesis ‘where the focus is on summarising data, and where the concepts (or variables) under which data are to be summarised are assumed to be largely secure and well specified’ and an interpretive synthesis with its concern with ‘the development of concepts, and with the development and specification of theories that integrate those concepts’. The reviews specified by the NIHR HSDR programme typically include at least one concept that has to be ‘discovered’ during the course of the review, whether that be unpacking the constituents of an intervention, the diversity of outcomes or, as in this case, in exploring if a health-oriented concept of access and candidacy could usefully be translated to social care. Elsewhere, Dixon-Woods et al.31 cite the distinction between the question as an ‘anchor’ [i.e. where four elements of population (or patient type), intervention, comparator and outcomes are prescribed and pinned down] and where the question is a ‘compass’ [i.e. where one element is pinned down (e.g. ‘adult social care’) and the implications of variation in the remaining concepts is explored during the course of the review].

These archetypal uses of review questions are, of course, end points of a continuum of approaches between having highly fixed inclusion criteria and questions and very fluid/flexible criteria and questions. Although the concepts of compass and anchor continue to offer useful heuristics at the question-generation stage, additional conceptual development could usefully explore their further utility and application, as they clearly also link to the core trade-off between the importance of the question(s) and the availability of relevant research to answer it (them).

In reality, when working on a review, we can recognise that all concepts are potentially either ‘compasses’ or ‘anchors’. Furthermore, definitions may be semisecure, such as the working definition provided in Thorlby et al.32 (see Table 5), or bound by statute, such as the Equality Act30 definitions associated with protected characteristics. Definitions may come from one of many different types of source (Table 6). Our lead reviewer for this project leads for diversity and inclusion at a faculty level and so was already familiar with the scope and variety of terms and concepts within this topic.

Key challenge 2: adding to the value of previous review work

Frequently, the review work that we undertake involves updating or complementing a previous review. In this case, the DHSC had already undertaken a rapid descriptive literature review. Part of our PICOSS challenge was to convey the added value of our preferred form of synthesis for this particular question. This can be conceived as a two-way dynamic flow between review commissioner and review team, in which the commissioner shares details of topic and research question and, in return, the review team seeks to communicate the synthesis methodology. Scoping, therefore, involved identifying existing evidence syntheses (such as Ford et al.29) that could communicate the potential of our planned review, act as templates for our own methods and outputs, and introduce the methods to be used.

Key to this process is the requirement to be familiar with diverse review methods so that matching between topic/question and methodology can be managed effectively. The emerging science of reviewing research (‘reviewology’) involves familiarisation with different review choices and engaging in an informed negotiation with the review commissioner. Evidence Synthesis Centres, therefore, need to include or have good access to experienced synthesis methodologists to perform this integral function.

Key challenge 3: scope and protocol as a work in progress

As mentioned above, the review team had a final opportunity to determine the scope of the rapid realist synthesis following completion of a process of sensitisation and rapid data extraction that preceded the interim remote meeting with the DHSC staff. The review team prepared and shared 13 candidate programme theories in a technical document, with accompanying illustrative extracts. At a subsequent meeting, the team at the DHSC indicated the relative priority of these programme theories and then confirmed these with policy colleagues. Engaging with such a vivid exemplification of the review scope in practice helped in finally determining the limits of the work. From January 2020 onwards, the review team worked within the scope of each of the five prioritised individual programme theories, selected by DHSC from the initial list, to organise analyse and interpret data within what are essentially ‘mini-reviews’ based on each programme theory. As befits an interpretative review, the parameters of the review product were determined relatively late and from within a wider sampling frame. Furthermore, the limits of the mini-reviews continued to emerge as decisions were made on the scope of each programme theory and their degree of complementarity, synergy or overlap.

Reflection and lessons

The scoping process involves extended negotiations, involving transactions and exchange of ideas, knowledge, references and topical and methodological artefacts between review commissioner and review team in moving to a shared understanding of the problem and its potential for resolution. The choice of review synthesis approach is not a decision to be made once the scope has been finalised. It represents an integral part of the scoping process (i.e. as an additional ‘S’ to the traditional PICOS framework).

More fundamentally, scoping also includes determining the extent to which concepts should be predefined or the extent to which they should be left to emerge from the review process or studies found. Concepts can themselves be viewed individually as ‘anchors’ or ‘compasses’ and definitions may be prespecified (i.e. temporarily fixed for the purposes of the review) or emerge from and give direction to an interpretive review method as part of the ‘findings’.

Case study 5: review of the effectiveness and implementation of strengths-based approaches to adult social work

This evidence review topic was initially proposed by the Chief Social Worker for Adults within the UK’s DHSC, in August 2019. The scoping and systematic review was conducted by the HSDR Evidence Synthesis team at Exeter and the full review protocol was published on the PROSPERO database (CRD42020166870).35 The review was conducted between November 2019 and the end of June 2020.

The review topic was initially stated as the following brief question:

• Different models of social work practice: what social work practice works best?

This directly translates into the more precise, typical language of systematic review questions as ‘What is the effectiveness of different models of social work practice?’

By the time we completed scoping the topic and finalised our review protocol, the planned systematic review aimed to address the following two questions:

1. What is the effectiveness of different strengths-based approaches used within adult social work?
2. What factors enable or inhibit the implementation of different strengths-based approaches in adult social work within the UK?

In other words, we had expanded the initial focus on effectiveness (i.e. ‘what works’) to include evidence on the implementation or embedding of strengths-based approaches into routine social work practice, and this case study mainly explores how that expanded focus arose. Ultimately, the review asked people with lived experience of using adult social care services to comment on research summaries and no service users were involved early enough in the process to shape the review questions and scope.

We aimed to obtain answers to these two questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria. Table 7 summarises how the scoping process refined the review questions and defined the inclusion criteria, compared with the initial, briefly stated review topic. Ultimately, as no effectiveness studies that met our inclusion criteria were found, it became a rapid synthesis of qualitative studies of implementation and (as with the review in case study 2) used framework synthesis because of its pragmatic aims and proven value for applied health care and informing policy.

Summary of key challenges/choices and scoping decisions

Key challenge 1: specifying the phenomenon of interest

Although the initially stated topic made no mention of specific models of social work practice, the earliest communications with the Chief Social Worker for Adults usefully and quickly clarified that one broad ‘model’ of social work practice was of most interest, that is strengths-based approaches. Such approaches had been enshrined in policy as part of the Care Act 2014,36 and were already strongly supported and promoted by the DHSC, the Social Care Institute for Excellence (SCIE) and NICE.

However, from expert advice and reading key reports, it quickly became clear that a strengths-based approach is not a single, well-defined ‘intervention’ or model of practice. Rather, it is a holistic way of working, with a multidimensional set of core ideas and principles about how social care can be effectively and positively provided in a way that respects people’s rights, desires, life and family circumstances, and draws on the capabilities and strengths of the individuals being supported, their families and the communities in which they live. At the same time, we had also been sent a table by the Chief Social Worker for Adults (i.e. the policy customer), which listed various subsidiary approaches, theories and models of practice, on which she had marked ‘those that more clearly fit under strengths-based approaches’.

Therefore, by 3 October 2019, when corresponding with NICE (about key challenge 2), we wrote:

We are aware from our previous scoping searches that there will be few, if any, comparative empirical evaluations of using strengths-based approaches vs. using traditional/usual approaches to social work practice. So we have provisionally, instead, decided to conduct a systematic review of some of the component therapeutic approaches or specific tools that are commonly seen as ways of delivering social work within a strengths-based approach.

In the likely absence of comparative, empirical evidence of the effectiveness of the overall strengths-based approach (based on our own scoping searches), the team had briefly considered conducting a theory-driven review (e.g. a realist review) to better specify the supposed underlying principles and mechanisms of strengths-based approaches (i.e. how the strengths-based approach improves social work outcomes). However, rather than using a review to better understand the effectiveness of strengths-based approaches from a theoretical perspective, we instead chose a more pragmatic approach to meeting the user’s stated evidence need.

Decision 1: focusing on subsidiary, named approaches aligned to strengths-based principles and practice

Mindful of the relatively short (5-month) timeline, we chose to conduct a systematic review of 17 named subsidiary approaches (i.e. models of social work practice and service change that are widely believed to be closely aligned with or to exemplify a strengths-based approach). As well as reflecting the reality of how strengths-based approaches were being fostered and adopted in different local authority adult social care teams, basing our review around such a list of named strengths-based approaches would enable targeted literature searches and give us flexibility at the evidence synthesis stage to not conduct syntheses where studies were either too few in number or too poor in quality. Our review protocol explicitly stated that the rapid review:

. . . will ultimately only include studies evaluating a selection of the approaches which are seen as a priority by the policy customer [DHSC], and [which] have an adequate number of studies for the synthesis to be more valuable.

The Chief Social Worker for Adults initially sent the team a table of named approaches and theories of social work (that had been produced by one local authority), and they had marked those that they saw as most aligned to a strengths-based approach. We supplemented this with an appraisal of named approaches within recent presentations/webinars, government reports and a roundtable meeting report about strengths-based approaches to social work. The final list of 17 strengths-based approaches that formed the basis of our searches and review were based on a table produced from these sources and, subsequently, corroborated by social work/social care expert stakeholders who had commented on the review protocol. Although acknowledging that the list was not exhaustive, the team believed it to capture the main subsidiary approaches that foster a strengths-based approach to practice or organisational change.

In summary, the nebulous definition of strengths-based approaches, being derived from various principles and an approach of how to work with vulnerable people rather than a single distinct model, offered two competing alternatives. We needed to focus on these underlying defining principles and, therefore, how strengths-based approaches improve outcomes (e.g. using a realist review), or focus on selected, named subsidiary approaches that are believed to foster or encapsulate a strengths-based approach, and effectively conduct a systematic review of each of these. For the reasons described above, we chose the latter approach.

Key challenge 2: rapidly becoming familiar with a new domain of policy and practice

The review team have primarily worked within health care and health policy research, and so the world of adult social care and social workers as a specific professional group was unfamiliar to all members of the review team at the beginning of this review. This meant that more time than usual had to be invested in talking to experts and reading key background reports to get to grips with the language, key concepts and basic context, such as what is adult social care? What is social work and social work practice? What are the types of adults and families in the UK who typically need support from social workers?

Decision 2: recruiting a wider group of stakeholders and involving experienced social workers in the review team

Rather than being an identifiable decision, this was a conscious ongoing effort to reach out to academics, practitioners and service leads in the social work field and adult social care services. It was boosted by two significant examples of collegial generosity and luck. First, a professor of social work who had co-authored one of the key recent government reports of strengths-based working was keen to help us in our work. The professor provided detailed commentary and e-mail feedback on emerging sections of the report and synthesis. Second, a health researcher from the Institute of Health Research (University of Exeter Medical School, Exeter, UK) who was a former social worker and had also been a leader and manager of adult social care services in two different local authorities assisted with the review.

Key challenge 3: assuring that we would not duplicate ongoing or recent evidence reviews by others

By following various leads in the literature and a tip-off from our policy customer about ‘a similar piece of work’, we became aware of (1) a recent policy document from NICE and the SCIE,37 which could have been based on a review of research evidence similar to our emerging topic, and (2) another university-based group conducting a review of the evidence relating to strengths-based approaches to social work. We, therefore, investigated both possibilities, through e-mails and telephone calls, to ensure that our planned review of the evidence relating to strengths-based approaches had not already been conducted.

In 2019, NICE and SCIE had jointly published Evidence for Strengths and Asset-Based Outcomes: A Quick Guide for Social Workers.37 Knowing that NICE evidence products are typically based on commissioned or internally produced systematic reviews, the review team contacted the Quick Guide authors at NICE to find out what evidence this guide was based on. Ultimately, e-mails and telephone conversations with two members of the social care and leadership team at NICE confirmed that no new or specific systematic reviews had been conducted to inform the Quick Guide. Rather, the evidence comprised a collation of evidence-based recommendations drawn from NICE, SCIE and other authoritative sources.

The second potential duplication of our planned review involved a small team based at the University of Kent (Kent, UK) that we had been told was ‘involved in developing a research proposal with others on strengths-based social work, through NIHR’. We spoke with both the principal investigator and the main researcher on this work to gain a clearer picture of their planned work. They confirmed that an ongoing project, conducted by the Adults Social Care Outcomes Unit (a DHSC policy research unit), involved exploring ‘strengths-based practice in social care’.

However, discussions with the lead researcher clarified that this was effectively a scoping review of relevant literature. We concluded that a broader focus and different methods (e.g. no specific study designs or inclusion criteria, etc.) meant that, notwithstanding shared inclusion of some studies, our systematic review would have a sufficiently different focus and methods (i.e. on effectiveness studies, and on specific, selected subapproaches within a strengths-based approach) to be both distinctive and valuable.

Key challenge 4: avoiding an empty review

A frequent criticism of systematic reviews is the apparently high prevalence of so-called ‘empty reviews’. Although empty reviews may give research commissioners impetus to fund more primary research to fill research gaps and give service commissioners greater freedom to encourage service innovations based on experiential knowledge and theory, they typically have less value in the immediate decision-making context of users of rapid reviews.

Decision 4: to extend the focus of the review to include the implementation of strengths-based approaches

The decision to add a second focus and review question about the implementation of strengths-based approaches was informed by three considerations. These were as follows:

1. Our scoping searches and advice from others with much greater experience in the field of social care research, indicated there would be no or, at best, very few comparative effectiveness studies of different strengths-based approaches. We judged that a review addressing the effectiveness question only would likely be an ‘empty review’ (i.e. no studies meeting our inclusion criteria).
2. Several of our expert stakeholders indicated that the research needs of adult social care teams were now less to do with whether or not strengths-based approaches are effective and more about how they can be effectively implemented by adult social care teams or individual social workers.
3. The view, shared by stakeholders and present in recent publications, that most social workers and other professionals working in social care ‘fundamentally supported a strengths-based approach within adult social work and social care, but often found it difficult to demonstrate, evidence and practice such an approach in practice’. Initial indications that local evaluations of models of strengths-based practice had included a focus on implementation (e.g. to identify enablers of or barriers to the adoption of the particular strengths-based approach).

Reflection and lessons

The fact that strengths-based approaches were already fully promoted by government and other agencies, that they are also widely assumed by social workers to be a positive and beneficial approach (reflecting the core goals and values of the profession) and the likelihood of there being no high-quality effectiveness studies of the overall approach led to us scoping a more useful review in two ways.

First, we added the second focus on the implementation of strengths-based approaches. This meant that if the effectiveness review was empty, we would have still provided a valuable evidence synthesis. Second, and relatedly, we avoided the challenge of defining what comprises or defines a strengths-based approach to social work practice by asking other stakeholders to identify named initiatives and subsidiary approaches that they deemed to be reflecting or fostering a strengths-based approach.

Scoping searches, therefore, played a critical role in defining what potentially synthesisable evidence existed, which fed back into the specification of productive review questions. Owing to the rapid timelines of this review, our main stakeholder at the beginning was the policy customer who was engaged with the process and setting the review goals. However, we were able to corroborate and expand our understanding of the evidence needs and which named initiatives were deemed to foster or exemplify strengths-based working by working with other social care and social work experts during the review, and by building on recent roundtable reports and webinars on strengths-based working. Without this broader engagement, we would have been less confident in the choice of the 17 strengths-based approaches that we ultimately included and less aware of the important differences between them (and, therefore, also less aware of limitations in the ultimate applicability of our findings). The lesson from this is that, even when you have a very engaged and knowledgeable policy customer, it is important to form a balanced view of the evidence needs using a broader range of experts and documentary sources.

Case study 6: review of hospital-led interventions to reduce the length of hospital stay for planned admissions of older people

Origin and context of the review topic

This case example was conducted by the Exeter HSDR Evidence Synthesis Centre. The review protocol was published on the PROSPERO database (CRD42017080637).38 The review was conducted between September 2017 and October 2018 and published in 2020 (with a pause of 3 months in early 2018 to address an urgent policy question) (see Case study 2: review of experiences of the ‘nearest relative’ provisions of the Mental Health Act 1983).39

The scope of the review was initially discussed at a teleconference between the review team and the scientific adviser at NIHR HSDR. The aims of the meeting were to obtain a preliminary understanding of (1) the background to the decision to prioritise the topic for review, (2) the main areas of uncertainty and (3) the purpose of the proposed review.

Two key aspects of the discussion informed our next steps:

1. Although the decision to focus on this area had partly been informed by key published references, the topic was considered a priority for the HSDR programme, with no other identifiable policy customer.
2. There was recognition that systems beyond hospitals often have an impact on timely discharge after unplanned care, but this should not necessarily be the case following planned admission. Potential topics of interest included –
   1. admission avoidance and non-emergency (hospital) care
   2. organisation of hospitals (i.e. hospital elements of control for planned care)
   3. population groups for which reducing length of stay may be particularly relevant (e.g. people receiving palliative care, people with dementia or delirium and people with learning disabilities).

Following a teleconference with the NIHR commissioners, a member of the review team ‘suggested that intervention or programme ideas could be looked at for [intended impacts on] length of stay – although narrow’ and it was agreed that this focus could extend to evidence on other outcomes of initiatives with this aim. This helped confirm the focus for our further exploration and scoping searches (i.e. a focus on interventions that explicitly aimed to reduce length of stay).

Given considerable uncertainty about the most useful review topic, our next steps were to familiarise ourselves with the topic and the body of evidence available. We (1) conducted broad scoping searches and (2) contacted local clinical experts with knowledge and expertise of caring for patients during planned hospital admission. The review team also met on three occasions with a group of four adults aged > 60 years, including one meeting on checking the review’s planned focus. Each individual had experience of being admitted to hospital overnight for a planned procedure.

On completion of scoping the topic and finalising the review protocol, the planned systematic review aimed to address the following two questions:

1. What is the effectiveness of hospital-led multicomponent interventions to reduce length of inpatient stay in hospitals for older adults following planned admission?
2. What is the cost-effectiveness of hospital-based multicomponent interventions to reduce length of inpatient stay in hospitals for older adults following planned admission?

We aimed to obtain answers to these two questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria that are typical of systematic reviews of effectiveness and cost-effectiveness, seeking quantitative outcome data from rigorous comparative evaluations [such as randomised controlled trials (RCTs)].9 Table 8 summarises how the scoping process refined the review questions and defined the inclusion criteria, compared with the initially stated review topic.

Summary of key challenges and decisions during scoping

Key challenge 1: absence of an explicit policy customer or question

With no explicit policy customer, it was not easy to identify a clear and specific clinical or policy need or evidence gap, within a given population. Our initial consultations with the HSDR programme provided insight into specific populations of interest; however, we did not know what evidence already existed for these populations or the scale and importance of the problems relating to their planned hospital admissions. Conversely, several relatively recent, high-quality systematic reviews indicated where there was an abundance of research and also which evidence had already been synthesised. We were also informed by our research commissioners about another review that was being conducted, which focused on ‘early discharge’-style interventions and mainly encompassed hospital-at-home and social care interventions after patients left hospital.

Decision 1: conduct broad and targeted literature searches to try and identify an unsynthesised body of relevant primary research

With a broad domain of health services only (planned hospital admissions) and a general perceived need for better evidence relating to it, we decided to conduct scoping searches to identify what questions, patient groups and hospital procedures had been the focus of primary research, and which evidence had already been effectively summarised and synthesised by others. This very quickly led to a second challenge.

Key challenge 2: lots of previous research and quite a few previous systematic reviews of it – but was there a gap?

Owing to the substantial cost of hospital admissions within all modern health systems, there has been a focus for decades on implementing and evaluating various types of service changes that aim to avoid hospital admissions in the first place (i.e. admissions avoidance) or shorten them by enabling people to be discharged earlier and supported at home or in other non-hospital settings. Increasingly, such intermediate care, hospital-at-home and early supported discharge schemes have also been seen as ways to further policy goals of person-centred care and, relatedly, providing care closer to or in people’s homes.

This has produced a wealth of research and also systematic reviews of that research. However, our research commissioners had felt that an evidence need remained in relation to this topic, but it was not clear if that need had already been met by primary research or syntheses. Early scoping activity centred on three population groups believed, by our commissioner, to exhibit greater uncertainty: (1) people with learning disabilities, (2) palliative care and (3) people with dementia or delirium.

Initial scoping searches of Ovid MEDLINE and Google Scholar identified several systematic reviews of the effectiveness of enhanced recovery after surgery and enhanced recovery programme interventions. However, we found no reviews that specifically focused on the three population groups of interest. We followed up this finding in two different ways. First, we searched for primary studies to ascertain whether or not a review of interventions focusing on these population groups would be feasible. Second, because we were unsure whether or not primary research would have been conducted specifically on these population groups, we considered the possibility of conducting a review of reviews, using subgroup analyses of the population groups of interest within systematic reviews of primary studies of general population groups. We also initiated discussions with external experts, colleagues and stakeholders about our proposed question.

Decision 2: focusing on hospital admissions of older people

These discussions guided us towards our final decision to focus on older people in our review, enabling us to consider evidence relating to at least two of the three population groups of interest: (1) people with dementia or delirium and (2) palliative care:

• A research associate at the Nuffield Trust (London, UK), and author of their related report,40 thought that achieving shorter hospital stays for older people was highly important because they are the most vulnerable to hospital-acquired harms.
• A professor of ageing and rehabilitation at the University of Exeter emphasised that older people would not necessarily receive an intervention unless it was deemed effective enough and, for this reason, enhanced recovery programmes could be an important option not being considered often enough for older people.
• A consultant geriatrician at Royal Devon and Exeter NHS Foundation Trust (Exeter, UK) suggested that we consider the need for evidence to inform ‘doing the right thing for the person’ in the treatment of older people with comorbidities, where interventions may not add much to quality of life. This was based on the assumption that enhanced recovery programmes are generally accepted as safe and effective, based on research evidence in younger and less complex patients.

Therefore, although the effectiveness of enhanced recovery programmes and similar initiatives to enable people to leave hospital earlier seemed to be accepted in general, there was shared uncertainty about whether or not such programmes would always be appropriate and effective in older people and, if not, for whom they would be effective. Returning to our scoping activity, we confirmed a gap in the synthesised research literature around organisational (i.e. hospital-based or hospital-led) interventions aiming to reduce the length of stay for older people following planned procedures.

Key challenge 3: defining and operationalising hospital-led ‘organisational interventions’ that aimed to reduce length of stay

Searching for evaluations of hospital-based interventions and hospital-led interventions to reduce length of stay revealed numerous studies that compared specific treatments or procedures. For example, many evaluations compared different drugs or protocols for anaesthesia, laparoscopic surgery with open surgery or different types of postoperative physiotherapy. Therefore, the team needed to distinguish ‘treatment interventions’ from broader ‘organisational interventions’ and changes in how whole teams work to support earlier discharge. In line with Medical Research Council definitions of complex interventions (as comprising multiple components41), checklists for describing components of enhanced recovery after surgery protocols42 and Cochrane Effective Practice and Organisation of Care group checklists,43 we decided to distinguish our interventions of interest as multicomponent interventions, as follows:

• Intervention has multiple components. As defined in the guidance for using the Intervention Complexity Assessment Tool for systematic reviews (iCAT_SR), commissioned by Cochrane, an intervention component is ‘a discrete, active element of an intervention that could be implemented independently of other elements’ (p. 6).43 Therefore, for inclusion, interventions should contain more than one component (e.g. education and exercise).

Therefore, organisational (multicomponent) hospital-based or hospital-led interventions in older people became the ultimate focus for our systematic review, and distinguished this review from those focusing explicitly on alternatives to hospital care or those focusing on single-component interventions (e.g. laparoscopic vs. open surgery or comparing different regimes for anaesthesia and analgesia for planned surgery). We also excluded strategies that were focused on discharge planning or pre-treatment assessment only (e.g. comprehensive geriatric assessment alone), as these do not affect other stages of the hospital stay.

In the absence of reliable intervention labels, the second challenging aspect of defining our included interventions was how to decide when an intervention aimed to reduce length of hospital stay (and, ideally, in a way that could be plausibly identified from a journal paper’s abstract). Our two main options were (1) to include only those papers that explicitly stated that its intention was to reduce length of stay (i.e. including papers in which length of hospital stay was the stated primary outcome), or (2) to also include those in which the aims implied that a reduction in length of stay was one of the aims [e.g. if the paper stated that it aimed to ‘improve’ or ‘enhance’ recovery (or equivalent language, e.g. ‘accelerate rehabilitation’)].

Reflection and lessons

The scoping of this systematic review exemplified many of our HSDR reviews in requiring iteration between the ‘technical’ (or data-driven, bottom-up) process of defining and refining searches to quantify research on different potential questions, and the ‘collective learning’ of a small group of reviewers working with clinical or research commissioner stakeholders to ‘home in’ on the most useful review questions (as a top-down process).

From both the searches and suggestions from the research commissioners, key published sources informed our understanding of the context of the review (e.g. the possible specification of organisational interventions), highlighted key pre-existing evidence syntheses on reducing patient length of stay and helped to develop our own search strategy. However, it was the combined views of several clinical and policy stakeholders that, on top of our learning from the searches, provided the critical decision and narrower focus (i.e. on older people). This decision allowed the scope to really take shape and helped to identify the main evidence gap to be addressed by our review. As one of our reviewers later reflected, such suggestions ‘stopped it becoming an academic exercise’. Our clinical stakeholders also aided our understanding of the stages of the patient journey through hospital and how these may differ according to age group and urgency of procedure. Again, this insight supported our definition of multicomponent, hospital-led interventions.

In conclusion, after reflection by the team, this scoping exercise was considered to have been ‘evidence’ rather than question driven. It started without a clearly defined question and the final scope and questions were based on scoping searches that showed a considerable body of unsynthesised research evidence related to stated evidence needs. In the absence of a clearly specified question at the beginning or a well-defined service delivery or policy area of uncertainty, it represented a general hunch about a lack of clear evidence relating to interventions that aimed to shorten the length of hospital stays for planned hospital admissions. This hunch was explored through considerable iteration between scoping searches and consulting clinical stakeholders. The ‘breakthrough’ decision, which provided an answerable review question that would likely yield useful evidence without too many included studies, was to focus on older people. All other decisions, including to focus on multicomponent and comprehensive models of care, such as enhanced recovery programmes, can be seen as following this main decision on the patient group. As a bonus, the focus on older people enabled the review to seek evidence in relation to people with dementia or delirium (another priority group mentioned by our commissioners).

Case study 7: implementation of interventions to reduce preventable hospital admissions for cardiovascular or respiratory conditions – an evidence map and realist synthesis

This evidence review topic was initially proposed by the HSDR programme, with no direct identifiable policy customer. Negotiations about the scope, therefore, took place with the HSDR technical adviser. Scoping and the subsequent evidence map and realist synthesis was conducted by the HSDR Evidence Synthesis team at ScHARR. The review was conducted in multiple stages between September 2017 and April 2019 and published in 2020.44

The review topic was initially stated in broad terms:

• interventions for preventing unnecessary hospital admissions.

Given that comparatively little primary research had focused on how to support the implementation of interventions to reduce preventable hospital admissions, an evidence map and realist synthesis was designed to address the following research question:

• With regard to the implementation of interventions to reduce preventable hospital admissions for cardiovascular and respiratory conditions, what works, for whom, how and in what circumstances?

The rationale for conducting a realist review was that the review’s commissioners (i.e. the NIHR HSDR programme) asked the team to consider the interventions as ‘proven interventions’ and, therefore, to provide greater understanding of how interventions that have been shown to reduce admissions for cardiovascular and respiratory conditions work in practice. The preceding mapping review was primarily to inform the sampling frame for the realist review.

Considerable effort and energy was expended at the beginning of the review to define the term ‘unnecessary’ (hospital admissions) from the originally stated review topic. Eventually, the team decided that conceptualising ‘unnecessary’ in terms of appropriateness was the realistic approach. The team and HSDR adviser agreed to focus on conditions characterised as ambulatory or primary care sensitive conditions (ACSCs). Ambulatory services are those services, like hospital outpatient services, where people attend and leave the appointments during the same day, and patients are, therefore, not classed as admitted patients and do not need an overnight stay. The logic is that where the severity or risk of acute episodes of a chronic condition is sensitive to the amount or quality of primary care or ambulatory care, then escalations and admissions to hospital can be prevented.

Table 9 summarises how the commissioned review sought to broaden and interpret the earlier review by Purdy et al.45

Summary of key challenges/choices and scoping decisions

Key challenge 1: targeting a representative part from a larger ‘whole’

An initial challenge was to select, from a long list of ACSCs on the DHSC website, a suitably productive focus to exemplify and, hopefully, to encapsulate issues for conditions generally. A frequent criticism is that realist syntheses fail to capitalise on the inherent analytical capacity of ‘what works for whom under what circumstances’ by failing to link qualitative insights to ‘what works’ (i.e. the effectiveness information). To avoid this pitfall, the team explicitly constructed a table (Table 10) that mapped the interim conclusions on effectiveness to different ACSCs. It immediately became apparent that cardiovascular or respiratory conditions had been sufficiently explored and adequately supported as effective to make these an appropriate focus for a realist inquiry. A telephone conversation with the technical adviser at the HSDR programme examined the case for each of the candidate conditions, in turn, to finally resolve the review scope. This approach to scoping, which resolves the coverage of a review by optimising both relevance and the likely explanatory power of the evidence, represents a risk-averse strategy when commissioning rapid reviews.

Realist synthesis typically requires prioritisation of programme theory to explore the phenomenon of interest in sufficient depth. The team had to ensure that, having identified review capacity to explore up to three chosen candidate programme theories, they optimised the analytical and explanatory power of their chosen foci. In consultation with the technical adviser from HSDR and the ScHARR Standing Advisory Group on Public Participation, the team selected foci to represent patient, carer and GP perspectives, respectively, within the candidate programme theory.

Key challenge 2: establishing comparability with the target population across diverse study populations

In a realist synthesis, evidence is excluded only if it does not relate to or inform the development of the programme theory. The practical focus of this review required that we exclude evidence with limited transferability to the NHS, such as avoidable admissions in low- and middle-income countries. In line with this decision, we prespecified five countries from which to derive direct evidence to the review, namely the UK, USA, Australia, Canada and New Zealand. These ‘big five’ were selected because of comparable health-care systems to the UK setting. The team continued to engage with the wider evidence base through systematic reviews, opinion pieces and direct reference to individual study reports, particularly where authors connected interventions to the UK context.

Key challenge 3: maximising the utility of review outputs

Much of the emphasis of published descriptions of scoping focuses on determining the conceptual boundaries, methodology options and logistic constraints of an individual review. In comparison, little attention has focused on the role of scoping in shaping the knowledge end product from the very beginning of the process. The scoping process offers an opportunity for commissioners and stakeholders, through their engagement, to exert an early influence on tailoring the final product to their requirements. So, for example, the focus on cardiovascular and respiratory conditions was not determined by evidence on effectiveness only, but also from the commissioner perspective, as articulated by the NIHR programme staff. For the review team, the resultant risk-averse strategy minimises the chances of producing a product that is neither useful nor appropriate. Scoping different options is one way of modelling different review configurations without embarking too far down a particular route.

Unlike other review processes that typically prioritise rigour and reproducibility (e.g. the internationally regarded NICE evidence synthesis products, with their explicit interest in effectiveness and cost-effectiveness), the emphasis of the HSDR programme has generally been to optimise a third ‘R’ of relevance (e.g. encompassing feasibility of implementation, acceptability and sustainability in different organisational contexts). This must continue to be underpinned by the preceding considerations of rigour/reproducibility. Scoping offers an early opportunity to sensitise the context within which the review is eventually situated. In this connection, subsequent attempts by the centres to further improve their working relationships with the patients, public and commissioners is particularly germane. Necessarily, this process of shaping review outputs does not simply involve determining the content, but also entails stimulating receptivity for the form of the final review. Review teams spend a significant amount of time, both in conversations and through the written protocol, cultivating the expectations of the commissioners with regard to less-standard review products, such as realist syntheses.

Reflection and lessons

In a research arena that is increasingly populated by systematic reviews and other types of evidence synthesis, this review and case study has demonstrated the increasingly frequent call to build on the foundations provided by a previous review. Previous reviews can be used in numerous ways (likened to the choice between bulldozing and reusing materials from an existing structure, building an extension or constructing a separate and adjacent annexe). In this particular case, the extension model was most evident. The review shaped the choice of a specific subset of studies that were extended and examined closely. Data on the viability of the chosen conditions, and the likelihood of their achieving an effect, drove the subsequent realist inquiry into how these interventions work. A long list of potential conditions to be examined was slimmed down rapidly into a short list of ambulatory sensitive conditions of potential interest. Conversely, this ‘extension’ approach required good familiarity with the original review, its implications and the definitions encapsulated within its methods.

The existence of a relevant systematic review also reduced some of the synthesis workload, allowing the team valuable time for exploring experimental initiatives reported in the grey literature or located in the ‘.nhs.uk’ internet domain. HSDR topics typically exhibit a shared concern with the established and evaluated, and the innovative and experimental. Although it is important to preserve the separate contributions of these two types of evidence, variously privileging rigour or relevance, the prior existence of a systematic review of effectiveness studies45 released search effort for exploration of ephemeral contributions. Potentially, NHS service initiatives extend the evaluation horizon by showcasing locally developed solutions with the, as yet unevaluated, potential to deliver health benefit. Faced with concentrated timescales, it is tempting to prioritise a formal scoping search of one or more bibliographic databases when the contribution of web sources may also be important.

Case study 8: access to primary and community health-care services for adults with intellectual disabilities – a mapping and targeted systematic review

This evidence review topic was initially proposed at the annual review meeting of the three evidence synthesis centres with members of the NIHR HSDR programme. The meeting forms the first stage of the process of scoping, where the teams identify prior experience and expertise in relation to a long list of topics generated by the HSDR priority generation process. At this time, the ScHARR review team identified a key researcher with expertise and networks in intellectual disabilities (IDs) to work with the review team to deliver against the identified topic.

The exact timing of this review was negotiated around the availability of the researcher and a joint reviewer/information scientist was scheduled to project manage the topic. As the review eventually took the form of an extensive mapping review, it was not formally included on PROSPERO. However, the protocol is published as a supplementary appendix to the final report. The review protocol was finalised in May 2018, following 2 months of scoping and iteration with the HSDR team, and the final report was delivered in December 2018 and published in 2020.46

The review topic was initially stated as follows:

• access and quality of health services for people with learning disability.

Within the context of the NIHR HSDR programme, this was a landmark topic, returning to the scene of a previous review by its predecessor, the Service Delivery and Organisation programme.47–50 The ‘footprint’ of this preceding review influenced several subsequent reviewing decisions, emphasising how each new synthesis fits within the evidence landscape. Indeed, even the title of the original review illustrates how both the topic and methodology had advanced. The title identified the output generically as a ‘literature review’, compared with our ‘mapping and targeted systematic review’, and the topic was described as ‘learning disabilities’ compared with the recent ‘IDs’.

By the time we undertook this review, we had consolidated lessons learned from the previous 3-year work programme.11 This endorsed a three-stage review process for most evidence synthesis topics, namely mapping, scoping and then systematic review.51 Although some researchers use mapping and scoping interchangeably, these processes are becoming distinct as their respective methods evolve. Mapping represents an attempt to obtain one’s bearings and take stock of the nature and quantity of research within a defined wider area of potential study, often with a view to identifying synthesis gaps where a subsequent potential review might be located. Scoping represents a process for identifying the limits or bounds to a specific planned review, whereby the implications in terms of both logistics and conceptual scope are explored, the limits and quantities of likely literature gauged and potential inclusion and exclusion criteria piloted. Therefore, scoping can be likened to drawing a line to describe the boundary of a review (i.e. deciding what is inside and what is outside its scope), whereas mapping is more about exploring and describing what studies are within particular bounds.

Although not every review requires a formal output at all three stages, these processes of mapping, scoping and reviewing feature in almost all of the three evidence synthesis teams’ projects. Therefore, we authored a protocol that involved two components: (1) a mapping review to offer breadth and then, following focused scoping, (2) a targeted systematic review to complement this with depth.

The mapping review aimed to address the following questions:

• What are the gaps in evidence about access to primary and community health care for people with ID?
• What are the barriers to accessing primary and community health-care services for people with ID and their carers?
• What actions, interventions or models of service provision improve access to health services for people with ID and their carers?

One feature of this scoping process was consulting people with IDs, family carers and formal paid carers so that the review could be informed by the views and experiences of stakeholders (for more details, see Appendix 1, Table 13). This consultation had multiple aims, namely to:

• illuminate the model of access to health care for people with IDs
• inform and refine our search strategies by identifying barriers to accessing health care and any solutions developed
• identify gaps in the literature.

We contacted the clinical director and senior commissioning manager of a Clinical Commissioning Group for services for people with IDs and asked them to identify relevant community groups for people with IDs and their carers. We sent information about the review to these groups and asked to visit them to discuss their experiences of accessing health care. Discussions were loosely guided by a topic guide, drawn from the previous review, covering how people identify a health need, what actions they take, the issues influencing their decision to take a particular course of action and the barriers to and facilitators of their access and use of the chosen service. The barriers and facilitators were used to identify relevant search terms and for future comparison with the barriers and facilitators identified in the qualitative literature.

Other scoping decisions proved important to the direction of the review. The original review distinguished between access and efficiency in focusing on ability to use a service (i.e. access), rather than whether or not the service was provided to a high standard (i.e. efficiency/effectiveness). In contrast, the update review sought studies reporting the effectiveness of any measures or interventions designed to improve access to the relevant services. Comparing the scopes of the original and more recent reviews illustrates how the HSDR programme, and its associated science, has developed and expanded how it conceptualises accessibility and related aspects of service delivery over the last decade or so.

The subsequent targeted systematic review focused on evidence from the UK, building on the findings of the mapping review. Key elements of the targeted systematic review were:

• development of the research questions based on the findings of the mapping review and information from PPI meetings
• a focused systematic database search following inspection of the mapping review findings;
• a full data extraction of relevant studies
• a quality assessment of included full peer-reviewed papers and no formal quality assessment of conference abstracts or grey literature.

Summary of key challenges/choices and scoping decisions

Key challenge 1: defining ‘relevance’

This review would include evidence from the UK only. This decision was taken to ensure relevance and to assist in managing the large volume of literature on this topic. A key scoping decision is which interpretation of relevance and context makes most sense in relating the study populations to the target population within each review. In some reviews, relevance is defined simplistically, as either geographical or chronological proximity. In other situations, relevance relates to similarities in how interventions work (i.e. conceptual homogeneity). For example, primary care settings for this review represent a ‘first point of contact’ for ID patients or their carers. In either case, the review team has to determine how to incorporate relevance into their review,7 either formatively, by methods used in the review, or summatively, in using one of many tools available for transferability or applicability.52,53

Key challenge 2: ‘mapping as scoping’

Broad topics are often characterised by uncertainties in relation to the quantity and quality of research. With limited time, the review team must decide how best to optimise the review resource. ‘Empty’ reviews are not helpful to decision-making, and yet overpopulated reviews are likely to either overrun timewise or prove superficial in their analysis and interpretation. A mapping review offers the possibility to offer breadth before the depth of a targeted systematic review. For this mapping review, initial descriptive analysis took place based on title and abstracts only. Subsequently, a subset (of approximately 20%) of the overall studies was sifted from the original retrieval results, supplemented by grey literature searches to form the data set for the targeted review. This approach keeps the scope as elastic as possible until informed decisions on coverage have been made. At this point, a formal protocol is finalised to provide the necessary scientific rigour. A further benefit of this staged approach is that it diversifies review outputs for different target audiences. Those requiring broad coverage can use the mapping review and those requiring a filtered approach that includes quality assessment are presented with a conventional systematic review.

Key challenge 3: building on previous systematic reviews

Although it may seem attractive to build on previous systematic reviews, the complexity of review decisions may offer minimal advantage from starting a review de novo. As illustrated by this review, not only had the preferred terminology changed, but also different fundamental assumptions about how to interpret ‘access’ shaped the subsequent review. Search terms need to be checked for new index terms and for the emergence of new terminology around the user community. Searches must be specified from the date of last search (not from publication date) together with overlap to avoid key items falling down a gap between reviews. Will the original data sources be replicated for fidelity to the original review? In the case of this particular review, the information specialist revisited the original list of sources to modify, and even improve, the original research. They also checked new index terms and terminology to ensure that the strategies continued to meet the requirements of the current review. This tension between comparability and contemporaneity needs to be managed according to the intended purpose of the review. Above all, the review team needs to decide whether to build on the foundation of the previous review, dismantle the previous review and reuse its original studies only alongside recent additions, or simply construct their own review, overlapping with the previous review.

Reflection and lessons

This review and case study has shown the value of recognising the distinct processes of mapping, scoping and systematic reviewing in managing the overall review, regardless of whether or not these lead to formal nominated review outputs. In general, unless the review customer has already conducted reliable searches and scoping of what evidence exists, some initial mapping and/or scoping needs to be conducted before any useful systematic review can be planned and conducted.

The case study also showed the potential contribution of different stakeholders in informing the scope and coverage of the review, including members of the public (in this case, adults with IDs). Stakeholder involvement included helping to identify gaps in the literature.

The case study also highlighted challenges that arise from changes in terminology and, importantly, adjustments in conceptualisation, as reflected in successive versions of related reviews. This may mean that a previous related systematic review cannot simply be updated, and the focus and intended value of the new review has to be carefully articulated. Even, as here, where the new systematic review is justified as updating and complementing previous reviews, the additional time and effort remains considerable compared with starting a new review from scratch.