Methods

There are seven stages to meta-ethnography (outlined in Figure 1). Stage 1 incorporates the rationale, aims and protocol development. Stage 2 involves deciding what is relevant to the synthesis through a systematic search, screening and appraisal. The analytical stages in meta-ethnography involve overlapping research activities: reading the studies (stage 3), determining how the studies are related (stage 4), translating the studies into each other (stage 5) and synthesising the translations (stage 6). The final stage (stage 7) involves the output and dissemination of findings.

FIGURE 1 The stages of meta-ethnography. (Figure not reproduced here; see the original article.)

Stage 1: getting started

This stage incorporates the background, rationale, aims and protocol development.

Stage 2: deciding what is relevant

In their original text on meta-ethnography, Noblit and Hare26 do not advocate an exhaustive literature search, and the number of studies included in meta-ethnographies ranges widely.22,24,27 The aim of meta-ethnography is not to perform statistical analyses but to draw on knowledge for conceptual development. Some argue that including too many studies makes the analysis ‘unwieldy’.27,29 However, we aimed to produce a conceptual analysis with a weight of evidence that has resonance with the health research community and thus undertook a systematic search of the published literature. Our previous meta-ethnography has demonstrated the value of undertaking a systematic search and including a larger number of studies into a QES.28

Searching and screening

Inclusions

We included studies that explored HCPs’ experiences of providing health care to adults with chronic non-malignant pain.

Exclusions

We excluded studies of acute pain, head pain, arthritis (including osteoarthritis and rheumatoid arthritis) or patient experience, and studies in which HCPs’ experiences could not be disentangled from the experiences of others (e.g. patients, carers or family members).

We searched five electronic bibliographic databases [MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO and Allied and Complementary Medicine Database (AMED)] using terms adapted from the InterTASC Information Specialists’ Sub-Group search filter resource.30–33 The InterTASC Information Specialists’ Sub-Group is a group of information professionals supporting research groups producing technology assessments for the National Institute for Health and Care Excellence (URL: www.york.ac.uk/inst/crd/intertasc/; accessed 24 July 2017).

We developed the search strategy with an information specialist and used a combination of specific subject heading terms and free-text terms (Table 1). We did not use the ‘clinical query limits’ option for qualitative research in our searches, as we had found that this can filter out relevant qualitative studies. To ensure value for money, we did not include citation checks, hand-searching, grey literature or Doctor of Philosophy (PhD) searches. Previous experience has shown us that these strategies do not necessarily add significant conceptual value to large meta-ethnographies.28 Two reviewers screened the titles, abstracts and full texts of potential studies for relevance.

Quality appraisal

Although there are many frameworks suggested for appraising the quality of qualitative research, there is no consensus on what makes a study ‘good’.27,34 However, a growing number of researchers are appraising studies for the purpose of QES.24

Although we did not intend to use rigid guidelines for determining inclusion, we used three methods of quality appraisal to frame our discussions:

1. The questions developed by the Critical Appraisal Skills Programme (CASP) for appraising qualitative research (Box 1).35 We assigned a numerical score to each question to indicate whether we felt that the CASP question (1) had not been addressed, (2) had been addressed partially or (3) had been extensively addressed, thus giving a possible score range of 10–30.8 We used the CASP in this way in a previous meta-ethnography of patients’ experience of living with chronic non-malignant pain, in which we found that satisfactory papers scored at least 19.8
2. A list of seven themes (Table 2) developed from a qualitative study embedded in a previous meta-ethnography funded by the Health Services and Delivery Research (HSDR) programme.36 Unlike CASP, these themes were developed specifically for meta-ethnography. For example, CASP does not focus specifically on conceptual clarity as a facet of quality, which is a distinguishing feature of meta-ethnography.
3. We categorised, as suggested by Dixon-Woods and colleagues,34 a ‘key paper’ that was ‘conceptually rich and could potentially make an important contribution to the synthesis’; a ‘satisfactory paper’; a paper that is irrelevant to the synthesis; and a paper that is methodologically fatally flawed.

1. Was there a clear statement of the aims of the research?
2. Is a qualitative methodology appropriate?
3. Was the research design appropriate to address the aims of the research?
4. Was the recruitment strategy appropriate to the aims of the research?
5. Were the data collected in a way that addressed the research issue?
6. Has the relationship between researcher and participants been adequately considered?
7. Have ethical issues been taken into consideration?
8. Was the data analysis sufficiently rigorous?
9. Is there a clear statement of findings?
10. How valuable is the research?

Two reviewers appraised each paper, and if they were unable to reach an agreement the study was sent to a third reviewer for the final decision.

GRADE-CERQual

We also utilised the Grading of Recommendations Assessment, Development and Evaluation Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual) framework37 (URL: www.cerqual.org/; accessed 24 July 2017), which aims to help reviewers to assess and describe how much confidence readers can place in review findings, in other words ‘the extent to which the review finding is a reasonable representation of the phenomenon of interest’.37 GRADE-CERQual proposes four distinct areas to rate for each review finding before assessing overall confidence. We aimed to explore how these domains were useful in assessing the confidence in a conceptual QES.

Methodological limitations

In GRADE-CERQual, methodological limitations are the ‘extent to which there are problems in the design or conduct of the primary studies that contributed evidence to a review’.37 Reviewers are required to provide an evaluation of the methodological quality of the studies supporting each of the review findings.

Relevance

In GRADE-CERQual, relevance is the ‘extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context specified in the review question’.37 Reviewers are required to provide an evaluation of the relevance of the studies supporting each of the review findings.

Adequacy of data

In GRADE-CERQual, adequacy of data is an ‘overall determination of the degree of richness and quantity of data supporting a review finding’.37 Rich data provide ‘sufficient detail to gain an understanding of the phenomenon we are studying’.37 Reviewers should describe the adequacy of data for each finding.

Coherence

In GRADE-CERQual, coherence considers whether or not the finding is well grounded in the primary studies. Reviewers should demonstrate that there has been no ‘attempt to create findings that appear more coherent through ignoring or minimising important disconfirming cases’,37 in other words demonstrate that they are not cherry-picking evidence to support an a priori concept. They should describe consistency and inconsistency along with possible explanations for any variations found across studies or cases.

Overall confidence

Finally, GRADE-CERQual reviewers give an overall rating of confidence for each individual review finding. The suggested ratings are high, moderate, low or very low. Lewin and colleagues37 indicate that ‘it may be difficult to achieve “high confidence” for review findings in many areas, as the underlying studies often reveal methodological limitations or there are concerns regarding the adequacy of the data’.

Stage 3: reading the studies

This stage of meta-ethnography involves thoroughly reading and re-reading the studies to identify and describe the ideas or concepts.26 The raw data of meta-ethnography are ideas or concepts in the primary studies. To allow us to refer to the original studies, we uploaded the study findings onto NVivo 11 software (QSR International, Warrington, UK) for qualitative analysis. NVivo is particularly useful for collaborative analysis as it allows the team to keep a record and compare the reviewers’ unique interpretations. NVivo 11 also allows the researchers to write memos and link them to data in order to keep track of developing analysis. We maintained a Microsoft Excel^® database (Microsoft Corporation, Redmond, WA, USA) of study demographics, appraisal and decisions on inclusion or exclusion. We read the studies in batches related to the topic/professional group and in the order in which they were retrieved, and did not use an index paper. In other meta-ethnographies, for example those of Campbell and colleagues,27 researchers have used an index paper as a way of ‘orienting the synthesis’.38 In these examples, concepts from an early or index paper are used for comparison with concepts from subsequent studies. However, we felt that there were methodological issues with using an index paper:

• How do we decide which paper to use as an index paper?
• How do we define a ‘classic’ paper with no consensus about what makes a study good?
• An index paper can have a dramatic effect on the resulting interpretation.
• Using an index paper can mean that we become constrained by a priori concepts. This is important because we will not necessarily find the conceptually rich papers first. The process of searching and analysing is iterative and analysis does not start when the full body of data is collected but continues alongside data collection.
• When synthesising a large number of studies, comparing concepts across these studies from an index paper can become unwieldy.

Stage 4: determining how studies are related to each other

Determining how studies are related to each other involves creating ‘a list of key metaphors, phrases, ideas and/or concepts’ (p. 28).26 The purpose of careful reading in meta-ethnography is to identify and describe the ‘metaphors’/ideas/concepts in each study and ‘translate’ or compare them to those found in other studies. This is fundamental to meta-ethnography because concepts are the raw data of the synthesis. However, at times it can be a challenge to decipher the concepts from primary studies.36 Two reviewers read each paper to identify and describe the concepts and compiled a list of concepts from the original papers. Our analysis was based on clearly articulated concepts from the originating papers. Schütz39 distinguishes between (1) first-order constructs (the participants’ ‘common sense’ interpretations in their own words) and (2) second-order constructs (the researchers’ interpretations of first-order constructs). In meta-ethnography, the ‘data’ are second-order constructs that are further abstracted to develop third-order constructs (reviewer’s interpretations of second-order constructs). However, the distinction between first- and second-order constructs is not clear-cut:40 primary authors choose narrative exemplars to illustrate a second-order construct. Meta-ethnography attempts to identify themes not from original narrative data but from the reported concepts. We did not recode narrative data as any attempt to recode is not embedded in the primary research process. Rather, we excluded data from analysis if both reviewers could not decipher a clear concept.

Owing to the scale of the study and the potential number of second-order constructs, our interpretation of each concept needed to combine clarity, precision and brevity. We therefore used a combination of the author’s description of the second-order construct (which briefly and clearly described the construct), and our interpretation of the construct (if the original was unclear or lengthy). In some cases, we found that there was a section of narrative exemplar provided by the original authors that was enough to give the essence of the concept. Our aim was to compile a list of concise interpretations of second-order constructs that were grounded in the primary studies.

Stage 5: translating studies into each other

The next stage in meta-ethnography involves exploring how the concepts are related to each other and, thus, sorting concepts into conceptual categories.26 All three reviewers organised, discussed and then reorganised the concepts into categories. ‘Translation’ is achieved through the constant comparative method41 through which the reviewers begin to see similarities and differences and organise concepts into further abstracted conceptual categories.

Stage 6: synthesising translations

The next stage of meta-ethnography is to synthesise, or make sense of, the conceptual categories by developing overarching themes and integrating these themes into a conceptual model. This is part of an ongoing process in which findings are further abstracted to form a conceptual framework. We planned to develop a line-of-argument synthesis, which involves ‘making a whole into something more than the parts alone imply’ (p. 28).26 In our experience of performing large meta-ethnographies, a line of argument can incorporate reciprocity and refutation into a useful conceptual model. This is achieved by constantly comparing concepts and developing ‘a grounded theory that puts the similarities and differences between studies into interpretive order’ (p. 64).26 We described and printed conceptual categories on postcards and sent these to our advisory group members to read and sort into thematic groups. Then, during the next advisory group workshop, members formed small groups to discuss and reorganise the postcards together. Each of the groups then chose a spokesperson to describe its themes. The complete advisory group then discussed and agreed on the final overarching themes. The aim of the workshop was to develop thematic groups that would underpin a conceptual model. Once the thematic groups had been finalised at the advisory group meeting, all three reviewers discussed and agreed on a conceptual model that they felt was greater than the sum of the individual themes.

Stage 7: expressing the synthesis

The final stage of meta-ethnography involves output and dissemination of findings. Our outputs included a short YouTube film (YouTube, LLC, San Bruno, CA, USA) (see Report Supplementary Material 1; URL: www.journalslibrary.nihr.ac.uk/programmes/hsdr/1419807/#/documentation). The script for the film was woven together from narrative exemplars from the primary studies to illustrate each overarching theme. The reviewers worked closely alongside a media company, RedBalloon (URL: www.redballoon.co.uk/; accessed 24 July 2017), which specialises in outputs from qualitative research. The team had worked successfully together to produce a film from a previous HSDR-funded meta-ethnography that explores patients’ experiences of living with chronic musculoskeletal pain (www.youtube.com/watch?v=FPpu7dXJFRI; accessed 24 July 2017).