About this chapter
Parts of this chapter have been reproduced or adapted with permission from Meiksin et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes additions and minor changes to the original text.
Parts of this chapter have also been reproduced or adapted with permission from Meiksin et al.57 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes additions and minor changes to the original text.
Parts of this chapter have also been reproduced or adapted with permission from Melendez-Torres et al.58 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes additions and minor changes to the original text.
Summary of key findings
Typology of intervention approaches
We developed a typology of interventions that categorised those included in the review by the structure of their content delivery, and identified two overarching types of interventions: time-limited or modular, which guided participants sequentially through intervention content from beginning to end, and open-ended, which were not designed as fixed and sequenced bodies of learning that all participants were intended to work through. Within these we identified five core categories. The three time-limited/modular categories comprised computer games and online modular interventions, both interactive and non-interactive. The two open-ended categories comprised interventions with content organised by participant needs, based on assessment, and those with generic content delivered to all participants. There was no clear pattern of particular intervention types addressing one or other of our outcomes, except that both open-ended interventions with content organised by assessment addressed sexual health and substance use and no open-ended interventions addressed mental health.
In our stakeholder consultations, all participants preferred interventions that provided some degree of personalisation and tailoring to the specific needs of individuals, ideally based on a risk assessment and providing content most relevant to their profile and/or needs. Stakeholders felt that this was critical in maintaining engagement with a diverse group of MSM who could benefit from these approaches. Interventions that were interactive and provided feedback mechanisms were favoured for increasing engagement and a sense of personal connection. Computer game approaches were seen by most stakeholders as patronising and unsuitable, except if these were specifically targeting young MSM or were exceptionally well designed. There was significant concern that these interventions would alienate much of their target population.
Theory of change synthesis
For this review, we synthesised the theories of change underpinning e-health interventions targeting sexual health outcomes, substance use and mental ill health among MSM. We developed a novel approach to doing so by using diagrams, rather than narrative themes, to summarise theories of change within and across interventions.
Although we have previously conducted reviews synthesising intervention theories of change using line-by-line coding of descriptive text,77,78 we found that this approach did not readily capture the often precisely described and complex inter-relationships between theoretical constructs presented in the body of literature for this review. Theories of change included in our past reviews, which addressed the integration of health and academic education78 and positive youth development interventions,77 were relatively simple and were either not significantly informed by existing scientific theories78 or informed by theories that are not typically portrayed visually.77 In contrast, theories of change underpinning the e-health interventions included in this review were more complex, more explicitly theorised and largely drew on existing scientific theories that are typically conceptualised in terms of diagrams indicating relationships between theoretical constructs, with constructs widely recognised and understood and therefore not always discussed at length. We found that, although thematic analysis is a good way of rendering explicit what is implicit, it is less appropriate when the literature itself is more explicit. We therefore developed a novel method of theory synthesis in which we created diagrammatic summative logic models of intervention theories of change. By inductively grouping these models according to their core constructs and using meta-ethnographic approaches, we identified three emergent theoretical approaches underpinning the included interventions and we created synthesised models of each approach. We have thus synthesised theories of change underlying interventions with similar approaches. These summarise and integrate existing theories of change rather than providing a novel overarching theory of change for such interventions.
Social cognitive theory and the IMB model featured most prominently in intervention theories of change, informing a small majority. Although reports cited a number of other existing scientific theories, each informed only a few interventions at most. In the ‘cognitive/skills’ synthesised theory of change, based on theory of change descriptions assessed as ranging from low to high quality, information and exercises were theorised to influence behavioural skills directly and via various cognitive factors relating to motivation/intention and self-efficacy/perceived control. All interventions represented in the ‘cognitive/skills’ grouping of theories of change targeted sexual health outcomes, either alone or in combination with substance use or both substance use and mental health. The synthesised theory of change also suggests that particular strategies can boost intervention impact, whereas other factors relating to the participant or (for sexual health interventions) to their type of partner (e.g. casual encounter, romantic interest or friend with benefits) moderate impact. In our consultations, this theory of change grouping resonated most strongly with stakeholders and was considered to be the best reflection of how face-to-face interventions with MSM are currently conceptualised in the UK.
Although represented in fewer interventions, two other distinct theory of change groupings emerged. The ‘self-monitoring’ grouping, based on theory of change descriptions assessed as being of low and high quality, focused more narrowly on the role of self-monitoring in triggering reflection, self-reward/critique and behavioural self-regulation. Interventions in the self-monitoring group of theories of change addressed all three domains of health outcomes for this review. Theory reports suggested that this synthesised diagram represents the basic components of more complex cognitive pathways119 and can sit alongside other distinct mechanisms of behaviour change in intervention theories of change.115 Underpinning interventions focused solely on mental health and based on theory of change descriptions assessed as being of medium quality, the ‘cognitive therapy’ grouping is rooted in cognitive therapy techniques, which can be augmented by mindfulness. In this approach, activities promoted awareness and recognition of a participant’s thoughts, feelings and situations and, via either challenging or accepting negative cognitions, aimed to reframe negative emotions to improve mental health. In our stakeholder consultations, the ‘cognitive therapy’ theory of change was felt to be useful in interventions specifically targeting mental health, but less useful for interventions addressing sexual health and substance use.
Intervention theories of change included in this review were informed by scientific theories that have been associated with greater impacts in reviews of e-health interventions specifically, including the transtheoretical model45,142 and the theory of planned behaviour,142 although these approaches underpinned a minority of interventions. Several also featured modelling and self-monitoring, and behaviour change techniques,143 which some evidence suggests might be effective in e-health interventions.142,144
As is common with e-health behavioural interventions,55,142 the theories of change underpinning interventions in this review tended to rely on individually focused psychological theories of behaviour or behaviour change. Although these are unlikely to address structural factors contributing to the syndemic among MSM such as marginalisation, homophobia and discrimination,22,145 the accessibility and anonymity of e-health interventions offer one approach to reducing barriers to service access stemming from stigma and discrimination146 and might form an important element of a broader mix of interventions addressing individual and structural factors.
Furthermore, we found that included interventions drawing on multiple theoretical approaches96,97,101,108,111,131 took into account sexual minority-related stressors. One accounted for such stressors as a theorised moderator of intervention impact,97 whereas others aimed to increase connectedness to the LGBT community111,131 and reduce internalised homophobia96,108,111,131 and associated shame101 to reduce sexual risk101,111,131 and improve mental health.96,108 However, theory reports included little discussion of the relationships between these constructs, limiting our ability to explicate their roles in the synthesised theories of change.
It was notable that some existing scientific theories that informed theories of change were theories of behaviour change (e.g. CBT theory) whereas others were theories of behaviour and its determinants (e.g. the health belief model). Authors generally did not report drawing on scientific theories oriented more explicitly towards selecting strategies for enacting behaviour change, such as the behaviour change wheel147 or the elaboration likelihood model.148 Intervention developers might usefully draw on such models.
Our theory synthesis aimed to develop overarching theories of change for different subtypes of e-health interventions to address sexual health outcomes, substance use and mental ill health among MSM, which we hope will help inform future intervention studies. Intervention developers might also draw on existing scientific theories that aim to integrate existing scientific theories of behaviour and behaviour change, such as plans–responses–impulses–motives–evaluation (PRIME) and capability, opportunity, motivation and behaviour models.147,149 Because we found that similar theories of change could underpin interventions addressing the different outcomes constituting the ‘syndemic’ affecting MSM, this provides some confidence that interventions can be theorised and developed that aim to address these interventions in combination.
Intervention, participant and contextual characteristics affecting intervention receipt
One-third of reports included in the overall review reported process evaluation data. All but one process evaluation took place in the USA. Most interventions targeted a single health domain of interest for this review (sexual health, substance use or mental health), with the majority focused on sexual health. However, two aimed to address aspects of all three.119,123 Some interventions employed personal tailoring, an approach that has been associated with effective e-health behaviour change interventions.45,144
Process evaluations rarely explored how intervention receipt varied between contexts. We found no eligible reports examining what factors affected intervention delivery as opposed to receipt. This seems to reflect the emerging state of process evaluations in e-health literature, with other reviews of e-health interventions reporting a similar pattern.150–153 There was some suggestion that slower internet speed could reduce acceptability of a multimedia intervention among rural MSM in the USA, who are less likely than non-rural residents to have high-speed internet at home.154 We did not find evidence that the factors that process evaluations identified as affecting acceptability varied by intervention type or by outcomes addressed.
In terms of intervention characteristics, as with e-health interventions among general populations,153 participants appreciated when interventions were easy to use and free of technical problems, while incompatibility with mobile platforms detracted from acceptability and could impede participation. In line with these findings, stakeholder consultations suggested that well-designed interventions with minimal technical problems were critical to the success of e-health interventions for MSM. There was a strong feeling among stakeholders that even minimal technical issues would have a profound impact on intervention engagement and effectiveness.
Privacy also emerged as an important aspect of acceptability in our review, suggesting that, with respect to e-health interventions addressing sexual health, detailed partner-level questions on sexual behaviour could feel intrusive and that features protecting app access and obscuring the manifest purpose of apps (for sensitive health domains) promote acceptability. The importance of privacy is also supported by existing evidence on behaviour change interventions for MSM155 and general populations.153
Participants liked content that was interactive and aesthetically pleasing, and they enjoyed the use of diverse media, such as animations, videos and graphics. However, among rural MSM, these media could also increase loading times for users with slower internet connectivity. Although modular approaches could support users to absorb programme content cumulatively, interventions that were too long detracted from acceptability and some users preferred that little or no time be required between sessions. The ideal number and duration of modules are likely to depend on a variety of participant, intervention and contextual factors.
Individual tailoring based on participant characteristics and risk profiles increased acceptability, highlighting this as a particularly promising approach and aligning with studies of e-health behavioural interventions for other populations.144,152,156 Participants valued interventions that presented scenarios and other content that reflected their experiences as MSM, an approach that stands in contrast to most existing e-health interventions targeting mental health and HIV prevention.56,157 When interventions targeted sexual minority groups more broadly, some participants suggested further tailoring based on the sexual and gender identities of its users. The language and tone of intervention content emerged as an important factor shaping acceptability for MSM, who appreciated the use of colloquial, direct, ‘up-beat’122 and sex-positive language. A candid tone and sex positivity were also highlighted in stakeholder consultations as important in engaging a diverse group of MSM, and matching intervention language to colloquial language used by MSM was felt to be essential. Stakeholders highlighted involvement of potential users in setting the tone and direction of these interventions as an important component of formative development.
Findings from our review also highlight the importance of paying careful attention to language and framing to ensure that these affirm sexual minority identities. That these concerns arose in interventions designed explicitly for sexual minority users, including one adapted for sexual minority young people using participatory approaches,107 suggests that this is an important area to explore during the pilot phase of intervention development.
As with studies of e-health interventions for general populations,150,153 perceived usefulness was key to acceptability. Participants liked gaining new knowledge and skills from e-health interventions and developing an awareness of the relationship between sexual behaviours and substance use.
Although reviews of e-health interventions for general populations report higher use and engagement among participants with higher levels of education,150,152,153 our findings suggest that in the context of a generally high use of electronic devices among MSM,47 the targeting of intervention content might be a more important determinant of the relationship between education level and receipt of e-health interventions than their electronic mode of delivery.130 Similarly, findings on the greater acceptability of the Keep it Up! intervention among black, Latino and other non-white users, compared with white users, suggest that e-health interventions can be developed to enhance inclusive acceptability among racially and ethnically diverse users.130 There was otherwise little evidence of engagement varying by sociodemographic factors, although findings on SES were mixed96,121 and Madkins et al.130 found that overall, and among white Keep it Up! users, those with lower levels of education reported higher intervention acceptability. Qualitative data suggest that e-health interventions can play a role in complementing external mental health support among MSM107,119 and that interventions targeting all adolescents might struggle to pitch content appropriately across this age range.107
Outcome data synthesis
Our systematic review of intervention effectiveness included 14 trials, of which 13 included active versus control comparisons. Trials included substance use, HIV, STIs and sexual risk behaviour outcomes, but not mental health outcomes. Substance use outcomes did not include alcohol use. Furthermore, all outcome estimates drew from ≤ 12 months of follow-up post intervention. A further two trials99,102 presented active versus active comparisons. Neither trial found a difference between tested interventions on sexual risk outcomes, and thus are not discussed further. The evaluation of the China–Gate HIV Prevention Program found a small but significant increase in benefits among participants with the least education, compared with those with university education or above;124 this is notable in the light of the process evaluation of Keep it Up!, which found that users with lower levels of education and white users reported higher intervention acceptability overall (but not non-white users).130 However, equity-relevant characteristics, for example moderation of intervention effectiveness by income, ethnicity and other social variables, were not meaningfully addressed by this body of evidence.
In active versus control comparisons, analysis for HIV infection drew on two studies, one with a short-term follow-up125 and one with a mid-term follow-up.113 Neither study suggested that an e-health intervention was effective at reducing infections, although short follow-up times and low event rates precluded meaningful comparison. The GRADE profile suggested that the certainty of these findings was low or very low. Analyses for STIs were similarly scant, drawing on two trials in the short term110,125 and one trial in the mid-term113 follow-up. Although a pooled analysis of short-term follow-ups suggested no impact of interventions on incident STIs with very low precision,110,125 the one trial informing the mid-term follow-up113 did suggest a meaningful and statistically significant reduction in incident STIs, with corresponding moderate certainty.
The largest analyses assessed sexual risk behaviour outcomes. Although the GRADE profile suggested that the certainty of conclusions was very low or low, primarily because of risk of bias in the included trials and possible publication bias, pooled estimates from mid-term follow-ups drawing on six trials suggested a small and statistically significant impact of e-health interventions in reducing sexual risk behaviour (d = –0.12). A pooled estimate from short-term follow-ups drawing on eight trials did not reach statistical significance, but suggested a trend towards reductions in sexual risk behaviour of similar magnitude (d = –0.14). These findings are in line with those from a 2019 review55 which concluded that e-health interventions targeting HIV/STI prevention among MSM could affect behaviours in the near term, but found few studies assessing change at 12 months, only one of which demonstrated a significant effect at that time point. We tested whether or not interactivity of interventions related to intervention effectiveness on sexual risk behaviours; however, a meta-regression did not suggest significant differences between interventions on the basis of this characteristic.
Findings for drug use drew on two studies with short-term follow-ups,115,125 which could not be meta-analysed because of extreme heterogeneity, and one study with mid-term follow-up.115 Together these studies did not present consistent evidence of effectiveness, with only one of these studies125 reporting evidence of a short-term impact from the Safe Behaviour and Screening open-ended intervention with general content on reduced recreational drug use. Furthermore, the GRADE profile for both analyses suggested that the certainty of the evidence was very low.
We found only two studies that examined the effects of e-health interventions on outcomes that spanned sexual health and drug use, with Reback et al.115 reporting no effects on sexual risk behaviour, but effects on one measure of drug use, and Chiou et al.125 reporting effects on measures of sexual risk behaviour and drug use, but not HIV infections or STIs. This lack of evidence for the effects of interventions that address more than one of the outcomes constituting the syndemic affecting MSM means that further trials will be required of interventions addressing such outcomes.
Economic evaluation synthesis
Our search identified only one economic evaluation eligible for inclusion in this review.132 This study suggested that the intervention may have been cost-effective in reducing episodes of CAI, but this finding was undermined by the lack of probabilistic sensitivity analyses examining the large degree of uncertainty around these results.
Deviations from the protocol
Appendix 4 summarises deviations from the protocol, most of which were minor, and the rationale for each. Deviations in the search included minor changes to the databases searched and the way in which search results were electronically managed. At the request of the funder, we conducted an updated search in April 2020, which was not included in the original protocol. Deviations in the theory synthesis included using a diagrammatic approach rather than line-by-line coding of descriptive text and, following on from this change, presenting an example of our approach to theory synthesis in the form of individual and overarching theory of change diagrams, rather than tables showing first-, second- and third-order constructs. Deviations in the outcome evaluation synthesis included (in addition to pooling outcomes by follow-up time in the meta-analysis, as initially planned) pooling outcomes across follow-up times, when appropriate, and narratively presenting findings by intervention type then follow-up time, rather than follow-up time then intervention type. In addition, stakeholder consultations were held as individual interviews rather than group discussions.
Limitations
Search and study selection
The original review searches involved multiple sources and methods, and aimed to maximise sensitivity. However, the updated searches were necessarily narrower because of the limits imposed by the COVID-19 pandemic. However, the sources that yielded all of the included study reports found as a result of the original electronic searches were included in the updated searches, so we think it unlikely that any studies were missed because of this reduced scope.
Typology of intervention approaches
We took an inductive approach to developing the typology of included interventions, which we grouped according to the structure of content delivery. Grouping based on other characteristics, such as delivery platform (e.g. SMS, mobile app), might be equally valid, and other reviewers might have developed a different typology of interventions. Categorising interventions based on behaviour change approaches such as the level of tailoring144 or interactivity or the use of modelling143 could also be useful approaches, but this was not possible because of inconsistent levels of detail available for different interventions.
Theory of change synthesis
Our synthesis was limited by the quality of the existing theory reports, which sometimes did not describe clear pathways from intervention activities to intended outcomes. In some cases, reviewers inferred relationships between theory of change constructs (which are made clear in our theory of change diagrams) and, in others, the relationships between specific activities, mediators and outcomes could not be determined. Although our approach to theory synthesis enabled us to systematically explore constructs and the relationships between them across intervention theories of change, synthesised diagrams do not capture aspects that theory reports suggest influenced theories of change when their role was not clear enough to be included in intervention-specific diagrams.
Our assessment of reported theories of change did not include assessment of the parsimony of theories of change, because we have found in past reviews that this is very difficult to consistently operationalise as a criterion of quality assessment; however, this is an important feature of theories of change. We also did not aim to systematically assess the evidence base for each of the scientific theories underpinning the intervention theories of change, because this was outside the scope of this review and would require assessing not only the evidence for the scientific theory, but also the evidence for the application of that theory to the outcomes targeted in the intervention theories of change it underpins.
Process evaluation synthesis
Our process evaluation synthesis was limited by the size and quality of eligible reports. Most were assessed as being of medium or high quality in terms of their reliability and usefulness. However, studies often lacked depth and breadth of analysis, and only around half were judged to privilege MSM’s perspectives.
Although the vast majority of interventions targeted MSM only and all were evaluated principally among MSM, three were assessed among samples that also included cisgender women.107,108,111,119 Author narratives and quantitative data did not always disaggregate MSM from other participants, raising the possibility that specific findings from these three studies might reflect data from other groups. The process evaluation of the smartphone self-monitoring intervention was the sole study contributing to findings on intervention benefits of self-monitoring and self-expression.119 Although the intervention targeted people of all genders and sexual identities living with HIV, > 80% of study participants identified as male and > 80% identified as gay or bisexual.119 In two studies, just under half of participants identified as female,107,108,111 but all themes informed by these studies also drew on other studies. The make-up of participants in these three studies is therefore unlikely to affect the validity of the themes to which they contributed. Studies of relevant interventions among broader sexual and gender minority populations might add further insight but could not be included as we could not be certain which findings reflected experiences of, or relevant to, MSM.
Outcome data synthesis
The meta-analysis drew on evidence of variable quality, with limited scope for meta-analyses. We were unable to account for heterogeneity between studies where this was present because of either scarcity of evidence or limitations of our evidence base. We were unable to undertake NMA, and most meta-analyses had too few studies to make meta-regression (e.g. comparing intervention type on outcomes) meaningful. In our analysis of sexual risk outcomes, which was the one model for which we were able to undertake meta-regression, we were unable to explain heterogeneity. Meta-regressions by outcome type to determine differential effectiveness on outcomes within sexual risk would have been uninterpretable because of the statistical methods used for meta-analysis, and because of multiple studies reporting outcomes across several domains. Probable publication and selective reporting biases across studies meant that several estimates of intervention effectiveness from included studies could not be included in our meta-analyses; in at least one case, outcomes stated in a trial protocol were not published in the main trial report. Finally, we were unable to locate evidence for some scoped outcomes.
Economic evaluation synthesis
The synthesis of economic evaluations was limited to one study132 eligible for inclusion in this review, which provided only limited information because of the uncertainty of its estimates.
Conclusion
Implications for research
Prior to this review, evidence suggested that e-health interventions are a feasible and acceptable approach for reaching MSM with targeted health interventions, particularly for men with lower-intensity needs or where access to face-to-face provision is limited.146 Previous evidence drawn from the general population or populations other than MSM suggests that e-health interventions might be effective in reducing sexual risk behaviour43,45 and substance use34 and addressing common causes of mental ill health,35–41 but effectiveness for MSM had not been adequately synthesised.
Our synthesis of theories of change identified three distinct theory of change pathways underpinning existing e-health interventions for MSM targeting sexual health, substance use and mental health outcomes, two of which underpin interventions targeting all three of these outcomes. Our review of theories of change suggests that interventions addressing these different outcomes may aim to exert impacts via common mechanisms of action, further adding to the potential for e-health interventions targeting the syndemic of sexual risk, substance use and poor mental health affecting some MSM. The synthesised theories of change are non-exclusive and may be combined to inform development of an e-health intervention holistically addressing this syndemic of multiple, often co-occurring, health issues among MSM. Although the ‘cognitive therapy’ synthesised theory of change is applicable only to interventions addressing mental health, the ‘cognitive/skills’ and ‘self-monitoring’ synthesised theories of change are relevant to all of the outcomes examined in this systematic review. Our synthesised theories of change could be augmented by use of the scientific theories on which they are based to inform a nuanced understanding of these theoretical underpinnings and how they can be most usefully applied in specific interventions. In the case of the ‘cognitive/skills’ synthesised theory of change, this would probably involve selecting a subset of mediators on which to focus, because a single intervention would not be expected to address the full range of constructs presented in this synthesised theory of change.
The findings suggest that the quality of existing theory reports is low to medium, with limited discussion of the inter-relationships between theoretical constructs and little attention to how mechanisms might vary by context. Improving the quality of theory reports would enable a better understanding of how interventions are intended to work and the evidence supporting this. It would also facilitate evaluations by identifying appropriate mediators and moderators of effects, the use of which could help outcome assessment to identify which components are triggering which mechanisms of change, and to what effect among which of its users.158 We suggest that intervention developers provide clear theories of change for their interventions, informed by existing scientific theories of behaviour and behaviour change relevant to the approach of the intervention and to the outcomes it seeks to address. Such theories of change can ensure that intervention activities align with their intended outcomes, and can ensure that evaluations are focused on the most appropriate measures of implementation, mediators and outcomes, and consider how mechanisms might vary by context and/or population.
The synthesis of process evaluations suggests that e-health interventions offer a feasible and acceptable approach to promoting the health of MSM, and allow MSM to access health promotion interventions privately, anonymously and at times that they find convenient. This synthesis identified several factors shaping the receipt of e-health interventions by MSM, which applied across interventions addressing substance use, mental ill health and sexual risk. These included ease of use, clear and comprehensive content, fun and enjoyable content, appropriate language and tone, interaction and personalisation, privacy and lack of intrusiveness, appropriate pacing and structure, content relevant to participants’ lives and relevance of intervention goals. Other factors should be considered carefully in designing interventions for MSM, including ensuring that language and tone are affirming of sexual minority identities and that content reflects the reality and experiences of MSM. The findings suggest that e-health interventions are acceptable for MSM across sociodemographic groups, although evidence in this area is limited and mixed. Different content for younger and older adolescents might be warranted. Variation in engagement and acceptability by participant characteristics, including ethnicity and level of education, should be explored in future research, and new interventions should be rigorously piloted to refine aspects affecting usability and acceptability.55,150
This review found that participants valued interventions that addressed the reality of their lives and the inter-relationships between the different domains of health, and the consultation with stakeholders found unanimous agreement among clinical and voluntary sector stakeholders that new interventions should holistically address the range of outcomes included in the review. Therefore, the findings suggest that e-health interventions simultaneously addressing sexual health, substance use and mental health might be particularly acceptable. The findings on the factors promoting acceptability of interventions can inform the development of future e-health interventions to address the syndemic of substance use, mental ill health and sexual risk among MSM and guide research questions for pilot and process evaluation studies. Those developing e-health interventions for MSM should use co-production and testing processes that ensure that the above factors are addressed and that interventions are acceptable. This approach is supported by findings from stakeholder consultations, which emphasised the importance of co-design for developing interventions that are relevant and accessible to MSM. Process evaluations should explore a broader range of individual, intervention and contextual factors that might affect implementation, and they should collect more in-depth, ideally qualitative, data privileging the perspectives of intended beneficiaries. Outcome evaluations of such e-health interventions should conduct linked process evaluations whenever possible, which would shed further light on factors affecting how they are delivered and received.158
We were interested in studies of interventions to address outcomes in the different domains of sexual health, substance use and mental health either in combination or separately. Studies addressing the domains in combination would indicate whether or not there is already good evidence for ‘holistic’ e-health interventions to address this syndemic of interclustered outcomes. Studies addressing them separately would provide some indication of the potential for developing and testing such a holistic intervention, particularly if no such holistic interventions have been evaluated to date. We found no outcome evaluations of holistic interventions addressing all of the outcome domains of interest. We found only two studies that examined the effects of e-health interventions on outcomes that spanned sexual health and drug use, with one115 reporting no effects of an e-health intervention on sexual risk behaviour, but an effect on one measure of drug use, and another125 reporting effects on measures of sexual risk behaviour and drug use, but not HIV infections or STIs. We found no evaluations of e-health interventions reporting effects for other combinations of outcome domains.
The quality and quantity of evidence supporting the effectiveness of e-health interventions for most of the outcomes that we set out to analyse was generally low or, in the case of alcohol or mental health outcomes, non-existent. The wide CIs surrounding many effect estimates suggest that many trials involved insufficiently large samples. Even when meta-analyses drew on multiple studies, issues with included trials precluded certainty in the evidence presented. Moreover, despite substantial heterogeneity in meta-analyses for sexual risk behaviour outcomes, we were unable to explain this heterogeneity. Although there was some evidence for intervention effects on sexual behaviours, there was inconsistent evidence regarding effectiveness preventing STIs and drug use, and no evidence for effectiveness preventing HIV infections.
Another key gap in this systematic review related to the inclusion of outcomes that accurately reflect current knowledge about minimising sexual risk. For example, a focus on condom use does not reflect that risk for HIV can be managed through effective biomedical means, such as adherence to HIV treatment for people living with HIV, or PrEP for those who are HIV negative. It is likely that interventions designed in the current context would more explicitly acknowledge biomedical approaches to managing risk.
One of the questions we set out to address in this systematic review was if the existing evidence suggests that our scoped outcomes could coherently, feasibly and effectively be addressed by a single health intervention addressing the syndemic of sexual risk, substance use and poor mental health affecting some MSM. It is clear, based on the meta-analyses presented, that the evidence does not, as yet, suggest that this is the case. This is largely because the majority of interventions were focused on individual, not syndemic, outcomes, as well as because of the patchy effects for outcomes that were assessed. Only two interventions115,125 assessed effects on substance use outcomes alongside sexual risk behaviour outcomes; for trials that otherwise reported outcomes over multiple categories, this was between sexual risk behaviour and either HIV or other STIs. No studies reported outcomes for depression or anxiety, despite poor mental health being a key syndemic condition, nor did any studies report outcomes relating to alcohol use, despite this being the most commonly used intoxicant in developed country settings.
Given the lack of evidence for e-health interventions to reduce risk across different outcomes, we cannot currently recommend scale-up of any e-health intervention aiming to address these outcomes synergistically. Given the lack of rigorous evidence for the effects of e-health interventions on certain outcomes (HIV and STIs, substance use and mental health in particular), a priority for future research is rigorously conducted studies of e-health interventions focused on these outcomes. Such studies may focus on a single domain of outcomes (e.g. mental health or alcohol use). However, given the syndemic of inter-related outcomes affecting some MSM, our findings that e-health interventions addressing diverse outcomes are often underpinned by similar theories of change and that several factors shape MSM’s receipt of such interventions applied across targeted health outcomes, our view is that it would be legitimate for future studies that focus primarily on one main domain of effects (e.g. alcohol use) to examine secondary outcomes in other domains (e.g. mental health) or to include multiple primary outcomes focused on these different domains. This could provide a more rapid means of identifying the potential for e-health interventions to address these syndemic multiple outcomes. Future trials of e-health interventions should also include several considerations. First, given the complete lack of evidence in this area, trials should consider how to address poor mental health among MSM, with a focus on how determinants of poor mental health among MSM relate to other outcomes considered here (sexual ill health and substance use) and to other antecedents (e.g. stigma). Second, trials should address a range of substance use behaviours that are syndemically linked to other relevant outcomes (mental well-being and sexual health), including alcohol use. Third, trials should involve interventions drawing on common and complementary theories of change to address multiple outcomes, and develop and test intervention content in collaboration with MSM. Fourth, trials should incorporate follow-up long enough, and sample sizes large enough, to detect a meaningful impact on HIV and other STIs, given the time needed to detect meaningful differences in HIV incidence. Fifth, trials should incorporate rigorous process and economic evaluations. Sixth, trials should ensure that interventions are not inequality-generating and that this is examined empirically by involving representative, diverse samples of MSM and examining how any intervention effects are moderated by factors such as gender identity, ethnicity, baseline health status and SES. This is important because interventions may unintentionally exacerbate inequalities between groups as a result of, for example, differential access to mental or sexual health services or substance use services. Seventh, to generate a joined-up, comprehensive e-health intervention that targets multiple outcomes, intervention evaluations should seek to generalise both mechanisms and components that are successfully used to achieve change in one outcome over multiple outcomes. Finally, e-health interventions are, of necessity, individualistic and inaccessible to men with limited internet access, and so should be complemented by other interventions, including community and structural interventions, addressing the broader upstream influences on the health of MSM.