Discussion

We developed a recruitment and retention strategy with young people and then tested this in a fRCT to see if it would be possible to conduct a future definitive trial of Wrapped (a digital intervention to increase condom use among users of web-based STI testing aged 16–24 years). This study demonstrates the feasibility of delivering the Wrapped intervention and of conducting the full trial. Furthermore, our nested qualitative study and implementation assessment showed that the intervention was acceptable and positively impacted condom-use beliefs and behaviours. Our independent DMEC and SSC recommended progression to full trial.

Recruitment and retention strategy

Strategies behind successful trial recruitment and retention are not well evidenced.71 Planning for recruitment and retention which involves consultation with key stakeholders maximises the potential for success.72 In preparation for a fRCT of Wrapped, we co-developed a comprehensive recruitment and retention strategy with a PPI group representing our target sample.18 Using evidence from a rapid review of strategies utilised in trials of digital interventions, and from focus groups with young people, a detailed plan was developed through iterative rounds of strategy refinement and voting by the PPI group. The resulting plan included six study adverts highlighting different value propositions; an agreed schedule for voucher payments which increased in value with study duration; agreement on the modes and frequency of messaging for invites and reminders; and selected strategies to maintain participant engagement over the 12-month duration of the study (e.g. study newsletters, personalised communications). Largely the strategy was delivered as planned, with a small number of in-study changes made predominantly in response to unexpected challenges. The success of the strategy was evaluated on completion of data collection with the PPI group. The ‘recommendations for future research’ outlined below reflect the consensus opinion of the PPI group and the research team.

Feasibility randomised controlled trial

In a two-armed parallel-group fRCT, in which our recruitment and retention strategy was applied, we assessed whether and how it would be possible to carry out a future definitive RCT of Wrapped.21 Over a 31-week period, 230 participants were recruited, representing 2% of the sampling pool, or 1.5% if the additional criterion of requiring self-report of baseline chlamydia test result was applied. This latter group, referred to as the ‘restricted sample’ is discussed hereon in, as it better reflects the true nature of the sample that would be recruited at full trial.

Recruitment

Recruitment duration was prolonged due to permanent and temporary losses of some local authority sites. One permanent loss was due to the contractor changing supplier at the end of their agreement term. Two temporary losses were due to service migration between different Preventx services in response to COVID-19. In any future RCT, it is imperative that partner areas have contracts with Preventx for at least the duration of the recruitment period and that agile arrangements are in place to allow recruitment to be transferred between Preventx services if required.

The overall proportion of the sampling pool recruited was lower than expected. While this does not threaten the feasibility of recruiting a sufficiently large sample at full trial given the size of the sampling pool available (see Chapter 3, Results, RO 6), if this can be increased then the rate of recruitment will also increase thus reducing trial length and costs. All adverts appeared to work equally well, and the change in voucher amount had no discernible effect on recruitment; changes to these variables at full trial are therefore unlikely to have any substantial impact. Visibility of the advert may however have been a critical factor. With support of a user experience (UX) expert, we have reviewed the recruitment user journey experienced by individuals invited to the study. This identified that anyone ordering a chlamydia self-sampling kit from Preventx via their mobile phone (applicable to 92% of Preventx service users; data provided by Preventx in personal communication) will not have seen the advert unless they actively scrolled down. Given that the advert was located on the ‘thank you’ page of Preventx, with messaging indicating the end of the ordering journey, it is likely that a substantial proportion of users closed their browser without scrolling down and seeing the advert. Furthermore, the advert could have been more visually distinct, as suggested by participants who took part in our qualitative study. Finally, as advised by the UX expert, the advert should aim to utilise the power of the Preventx brand, given that Preventx is trusted by our target population to perform their STI testing. The advert could include, for example, wording such as ‘Preventx are partnering with Wrapped to run a study to…’. This endorsement could also be carried through to the study project page to reassure participants that the study is genuine and credible. In any future trial, we will address these issues as a strategy to increase uptake.

There was evidence of some imbalances across demographic subgroups at baseline between the randomised groups. For the full sample, there appeared to be a lower proportion of bisexual participants in the intervention than the control group. Further, in comparison to the control group, there appeared to be lower proportion of 20-year-olds and a higher proportion of 21-year-olds in the intervention group. This is likely a product of our stratification strategy. To keep the total number of stratification categories to within reasonable limits, we did not stratify by all demographic characteristics and where stratification did occur, sub-categories of variables were collapsed. Age was one of the unstratified characteristics. Sexual identity was stratified but a binary categorisation of ‘heterosexual’ versus ‘other’ was used. These imbalances do not present concern in terms of a future full trial. This is because at full trial a stratification strategy would not be used as balance would be expected to occur naturally given the large number of participants required. The same imbalances across groups for bisexual participants and the above age groups carried through to the restricted sample, but also there appeared to be less females and more males in the intervention than the control group here. This difference across gender categorisations was introduced as a result of selecting the ‘restricted’ sub-sample of participants after randomisation. Again this does not present any concerns for a future full trial as we would not be selecting a sub-category of the sample for analysis; randomisation would occur once a participants’ STI status as baseline was known and all randomised participants would be included in the analysis. Chlamydia positivity at baseline in our sample (4.6%) was lower than the national figure for users of internet test settings (8.8%) at the time of data collection.29 This difference may in part have been as a result of the COVID-19 pandemic that coincided with data collection. The pandemic resulted in reductions in the frequency of sexual behaviour among young people, particularly for those not in a steady relationship.73,74 Given that this group are most likely to have new and casual sexual partners,75,76 both of which are key risk factors for STI transmission, it follows that instances of STI may have reduced during this time. It is possible therefore that the reduced levels of positivity evident in our study reflect this. Why there is a discrepancy with national level data on positivity is unclear. It may reflect local differences in behaviour or changes to service delivery in response to COVID-19 for areas participating in our study that are not evident in the national picture. A future trial conducted outside of this context may observe levels chlamydia positivity more typical of the national figure. Lower overall positivity in the sample may also reflect a tendency for participation among service users with lower sexual risk behaviour. This is potentially problematic for two reasons. Firstly, the sample size calculation for full trial is based on chlamydia positivity in the control group being 10% (see Report Supplementary Material 10) and lower than expected positivity may increase the sample size required to detect an intervention effect. Secondly, if participants have lower levels of chlamydia positivity (and sexual risk behaviour) than typical users of STI self-sampling services, this could result in biased study outcomes that threaten external validity. Other sexual health-related research is equivocal on whether recruited samples tend to exhibit higher or lower sexual risk behaviour.77–79 Less is known about the relationship between participation and STI status, but a recent longitudinal Dutch study reported that baseline STI positivity was a predictor of non-participation.77 While caution should be adopted given the small-scale nature of the present study and that chlamydia positivity at baseline was self-reported; it is prudent to consider how we could address disproportionate uptake of individuals by STI test outcome at full trial. Discussions with Preventx have indicated that they could identify service users with higher sexual risk behaviours prior to the point of study invite (service users are required to provide relevant information when ordering STI self-sampling kits) allowing us to customise the advert displayed to this subgroup. There is potential to work with a future PPI group (including those with a recent STI diagnosis) to develop and test alternative types of messaging that could be used at full trial to encourage their participation. Furthermore, we could introduce a stopping point within a full trial when half the sample has been recruited at which chlamydia positivity is assessed and a continuation decision made by the trial’s independent DMEC and SSC only if this is at the expected level.

Retention and data completeness

Follow-up for our primary outcome measure at main trial (i.e. chlamydia positivity, measured in the present study as the proportion who returned a valid chlamydia self-sample at M12) was 75.7%. It is worth noting that the response rate for the full sample, which includes those who did not self-report their chlamydia test result at baseline, was lower at 65.7%. As discussed, the restricted sample is most representative of the sample that would be used at full trial, but it is acknowledged that it is not the same. That is because at full trial, we would seek to obtain objective STI test result data from Preventx rather than relying on participant self-report. This increases uncertainty about whether recruitment estimates obtained in the present study can be confidently applied to decision-making at full trial. It is possible, for example, that the self-selecting nature of the restricted sample may have artificially inflated the M12 response rate, with the self-report of the baseline chlamydia test result (a requirement of membership of this subgroup) acting as a proxy for ongoing engagement. To test this hypothesis, post analysis, we sought objective baseline chlamydia test result data from Preventx for all participants in our sample and then examined the M12 response rate for all those with a valid result. Based on this, the response rate was higher at 78.8% (134 provided a valid self-sample at M12/170 with objective test data provided by Preventx at M0) indicating that we can be confident in obtaining a response rate of at least 75% at full trial. This is further supported by other studies which have achieved similar response rates, such as the recent sexual health trial by Free and colleagues which reported 74.8% return of STI self-sample at 12 months by 16- to 24-year-olds,80 and the recent longitudinal study by van Wees and colleagues where STI self-sample kit return at 6 months by 18- to 24-year-olds was 75.7%.77

Also measured in the present study was return of a valid chlamydia self-sample at M3, and survey completion and completeness at M0, M3, M6 and M12. Return of a valid sample at M3 was similar to that obtained to M12 at 75.1%, and survey response at all time points exceeded 90%. This level of survey response is equivalent to that of similar recent trials of digital sexual health interventions, for example 88% at M1 follow-up80 and 72% at M3 follow-up.81 Maintaining this level of completion across all surveys up to M12 follow-up is particularly encouraging given that trials of internet-delivered interventions can be particularly prone to attrition.82,83 Further, on items required to assess key primary and secondary outcomes at main trial there was over 95% completion indicating high levels of measure acceptability. These positive outcomes are likely the result of the investment in the recruitment and retention strategy and the repeated testing and refinement of the measure delivery and completion with our PPI group. This is further supported by the positive feedback received from participants in the nested qualitative study and by those who provided feedback on participant experience at the end of the M12 survey. These participants attributed their ongoing engagement to, for example, the ease of completing measures, personal touches such as hand-written notes included in self-sampling kits, and responsive communication from the research team.

Differential attrition by participant characteristics

Attrition by participant characteristics was examined to identify potential threats to internal and external validity. The demographic characteristics of participants who returned a valid chlamydia self-sample at M12 (primary outcome any future RCT) were compared to those of individuals in the sampling pool and at baseline to examine patterns of attrition and assess whether the sample available for analysis at full trial was likely to represent typical web-based STI-testing users. Participants at M12 were broadly equivalent to web-based STI testers in the sampling pool and to participants at baseline across all gender categorisations. Participants at M12 was also broadly equivalent to those at baseline across all sexual identity categorisations (note the same comparisons to the sampling pool could not be made as the required data were unavailable). There was however evidence of the sampling pool and the baseline sample being less well represented at M12 across other demographic characteristics. In terms of deprivation, there was some evidence that M12 participants were under-represented in deprivation quintile 3 and over-represented in deprivation quintile 5 (the least deprived) in comparison to the sampling pool. There was also some evidence of a continued slight under-representation of quintile 3 and slight over-representation of quintile 5 when comparing participants at M12 to those at baseline. These observed discrepancies partly reflect the under and over-recruitment of participants in quintile 3 and 5 respectively, but also a degree of attrition for quintile 3 and strong retention for quintile 5. Evidence of the over-recruitment of participants in the least deprived group, combined with high levels of retention, presents some concern for the future trial as this could threaten both internal and external validity. This picture warrants particular effort being made in any future trial to balance this out through improved recruitment and retention of participants across other deprivation quintiles. Encouragingly, representation of service users at both baseline and M12 (full and restricted samples) in deprivation quintile 1 (most deprived) was good. However, given that it is not unusual for trials to struggle to engage participants from the higher end of the deprivation spectrum,84 and that caution should be adopted in drawing too much inference from our findings given the small overall number of participants, a future trial strategy ought to be to focus most attention on the recruitment and retention of participants in quintiles 1 to 3. In a future trial, we would work with members of our PPI group to understand what incentivises young people from more deprived backgrounds to participate in research and what barriers exist to their ongoing participation. Any appropriate strategies recommended by the group will be implemented, and uptake and retention across deprivation groups carefully monitored throughout.

Participants at M12 appeared to represent all ethnic group identities at baseline thus there was no evidence of attrition by ethnicity. When comparing M12 participants to the sampling pool there was however evidence of a higher proportion of black participants and a lower proportion of white participants, reflecting differing levels of study uptake by these groups. While this pattern of recruitment is encouraging given that people of black ethnicity carry a disproportionately high burden of STI infection, and higher representation increases the potential for important subgroup analyses examining the equity of intervention effects, this does again threaten external validity. Furthermore, there was a tendency at M12 for lower representation of those aged 16–19 years, and higher representation of those aged 20–24 years, compared to the sampling pool. These differences by age group are the result of differential study uptake rather than dropout during the study itself as the M12 sample was representative of the baseline sample for this characteristic. While it is unlikely that measures would be taken at full trial to limit participation of participants according to their ethnic identity, efforts would likely be made to increase participation by those at the lower end of the age range. Lower participation by the under 20 seconds may be due to this age group being more likely to live at home and having concerns over parents/carers finding a STI self-sampling kit or intercepting notifications of results.85 This is partly supported by our examination of adverse events, whereby over half of cases of the reported event, ‘someone finding out I was testing for a STI when I didn’t want them to know’, were reported by participants under 20 years despite this age group representing only 13% of the total sample. The under-recruitment of younger participants may also have been because we were unsuccessful in developing adverts to attract them to the study. In a future full trial, we would work with younger members of our PPI group to review the study sign-up journey and identify changes that could help to increase uptake by this age group, for example developing bespoke adverts, and highlighting the discrete nature of STI kit delivery and result notification in the participant information. Regarding the latter strategy, it would however be important to strike a balance between providing reassurance while advising those who are particularly concerned about disclosure of STI testing not to participate in order to minimise adverse events.

As discussed above, at full trial we would carefully monitor the representativeness of the sample during recruitment taking corrective action if necessary. If we were concerned that the final sample did not reflect the sampling pool, then there would also be the further option of estimating a weighted intervention effect to match that of the sampling population. This strategy would be implemented if there was evidence that our primary outcome differed by demographic characteristics, suggesting that intervention effects were not equivalent across groups.

Dropout attrition

Dropout attrition, whereby the degree of attrition differs between the intervention and control group, is a threat to a trial’s internal validity. This can be a particular problem for trials of behavioural interventions where the blinding of participants to group allocation is more difficult to achieve.86 In the present study, retention (return of a valid chlamydia self-sample) was higher at M12 in the intervention group than the control group. This is at odds with evidence from meta-analyses which have examined attrition across trials of health behaviour change interventions, including for HIV-prevention, and have found slightly higher retention within control groups.87,88 This may be the product of participants experiencing the costs of participation in the intervention study (e.g. time and effort required to engage with intervention material) as greater than the benefits received, particularly if the targeted health risk behaviour is not perceived as problematic.88 In this sense our pattern of attrition reflects well on Wrapped, suggesting that at least within a research context, those in the intervention group did not perceive the costs of participation as greater than the benefits receiving Wrapped.

Despite our attempts to create an equivalent intervention experience for participants in the control group, the observed differential attrition may suggest that blinding was not entirely successful, and that participants’ perceptions of group allocation influenced their ongoing engagement. Why perception of being in the intervention group or control group might have led to higher and lower levels of retention respectively, is unknown. It may be that participants given access to the intervention materials (such as free condoms) felt more of an obligation to complete follow-up assessments than those in the control condition89 or that those in the control group felt disappointed, a sentiment which influenced ongoing engagement. Encouragingly, our examination of the demographic characteristics of participants at M12 showed that, with the exception of some observed differences that were a present at baseline, there were no systematic differences between the intervention and control groups indicating that attrition was not patterned. Any future trial should nonetheless introduce strategies aimed at increasing participant retention within the control group. This could include for example, ensuring that the research team effectively communicate to participants that all participation is of equal importance and value regardless of group assignment. Further at full trial, information that may be informative in terms of predicting missingness should be considered and measures taken to collect this so that multiple imputation can be performed as required.

Engagement with intervention materials

In this study, we successfully collected a breadth of data on participant engagement using Matomo analytics, information stored within the Wrapped website on product ordering and dispatch, and the fRCT surveys. Overall, Wrapped access levels were encouraging (68.7%) albeit with room for improvement. At a full trial, we would aim to increase this by sending access details through SMS in addition to e-mail, by sending more than one access reminder, and by potentially increasing the incentive amount up from £5. There was some evidence of higher access among younger participants (16–19 years) and those in the least deprived quintile (IMD 5), and lower access among those in deprivation quintile 2, relative to overall access levels. Given the small number of participants across these subgroups, these findings should be interpreted with caution. Nonetheless, it would be important to explore ahead of any future trial whether there may be any barriers to access among those in lower deprivation groups, for example, relating to perceptions of what a sexual health website may have to offer.

While information on access is informative from a trial perspective, this does not provide a valid assessment of use outside the confines of a RCT where there is no incentive for use or associated research measures to complete. To assess this, a future study should examine levels of Wrapped uptake and engagement, including by group, when it is made available outside of a research context. If implemented, this type of study could be run in parallel to a full trial in one discrete local authority area (not participating in the trial), with intervention uptake and use recorded unobtrusively using analytics data. To enable uptake and use to be compared across subgroups, analytics data could be linked to demographic and behavioural data shared by Preventx (subject to user consent and a data sharing agreement). These data are likely to be of importance to service commissioners should Wrapped be identified as effective and cost-effective and local implementation therefore being under consideration.

Users evaluated the website as useful and liked the design. Furthermore, we identified little evidence of friction points within the site, that is, aspects of the site that make it difficult or inconvenient for a user to complete an action such as registering, completing tailoring questions, or placing an order. This is backed up by data from the nested qualitative study (see Nested qualitative study). Higher levels of engagement were observed for product-related components than video-related components. High proportions of users ordered the products and reported using them as intended. The accounts of individuals participating in the nested qualitative study also attest to this. The large majority of those assigned to video components however did not access the relevant page, and those that did were unlikely to play the videos. Those that did play the videos were also unlikely to watch them in full. A user experience (UX) review of the video components is therefore necessary to explore and address any potential issues concerning usability and appeal.

Contamination

A large proportion of participants in the control group of the present study reported that they recognised the Wrapped website. Owing to technological measures put in place to prevent access to Wrapped by anyone other than participants assigned to the intervention group, and an analysis of analytics data which showed that only these individuals went on to gain access, we can be reasonably confident that no one in the control group gained direct access. An alternative possibility is that some participants in our study were known to each other, assigned to alternate conditions, and then showed each other the respective websites. While there was some limited evidence of participants being known to each other, there is no way of knowing whether content was shared. Either way, this would not account for the extent of contamination reported and instead the most likely explanation is that the question used to assess contamination was not reliable. Any future trial should only allow one participant per household to participate; checks performed at randomisation can be used to verify this. In addition, participants should be requested not to signpost others to the study and stress the importance of not sharing intervention materials with anyone else. Furthermore, the contamination question should be improved such that it can reliably assesses receipt of the active intervention.

Health economic assessment

The economic component of the study aimed to design processes, and capture data on the costs and outcomes associated with the intervention and control arms, to inform the methods for a future economic evaluation alongside a definitive trial. Overall, processes were successfully put in place to measure costs associated with delivering the intervention and the control (usual care). A range of outcome data were collected successfully, including for the EQ-5D-5L and SF-12. The cost data collected were comprehensive, and the private costs borne by the participants were well captured. The findings of this feasibility trial suggest that in a future definitive trial, it would be important to capture a broad range of costs, as participants reported that they had incurred a range of private costs which would need to be considered (e.g. condoms, self-sampling kits etc.). A difficulty encountered was capturing full data on NHS healthcare resource use (including information on contraceptive use and other products). We would recommend therefore that in any future trial, attention is paid to capturing further detail for both arms in relation to NHS resource use and private costs.

The relevant outcome data were collected successfully. This would permit different types of economic evaluation to be conducted in a future definitive trial. Firstly, a cost-effectiveness analysis could be conducted, using the primary clinical outcomes included in the trial, including on cost per STI averted. In addition, collection of EQ-5D-5L/SF-12 data would allow a cost–utility analysis to be undertaken as recommended by NICE.39,40 Finally, collection of data on more general well-being (via general well-being instruments such as ICECAP-A) would enable a broader analysis to be undertaken, which would focus on outcomes beyond health.

In a definitive trial the costs associated with attempting to improve condom use and sexual health would need to be offset against the longer-term savings associated with reductions in STIs. As is the case with many public health preventative programmes, longer term costs and outcomes would need to be considered.90 Hence, it is recommended that a decision-analytic model is used to extrapolate costs and benefits beyond the follow-up period. There is a range of evidence that could guide the development of the model, including the model used to inform NICE guidance on condom use and other more recent work in this area91,92 and adaptation of an existing model would need to be explored. The development of a model would require data on transmission rates and the longer-term sequalae associated with STIs.48 The aim of the decision-analytic model would be to provide information on the potential longer-term costs/savings and health benefits associated with the Wrapped intervention.

Nested qualitative study

Interviewees in the nested qualitative study were asked to reflect on the trial materials and procedures that they had experienced. Relevant feedback was also received within the M12 survey. Materials and procedures were viewed positively, with only a few minor amendments suggested. This experience is likely testament to the detailed work undertaken to develop the recruitment and retention strategy and the iterative testing of materials. In line with insights from our analytics data, feedback from interviewees also indicated few friction points within the Wrapped website. The condom sample pack was enjoyed and reportedly shifted views on the variety of condoms available. The ability to choose the colour of the box and the image displayed on the insert, while liked, was not considered necessary; this could be removed in any future delivery to reduce costs. Participants who had used the condom-ordering service also felt positively about this, enjoying the discrete nature of this service which for some removed the embarrassment of obtaining condoms in-person. The condom carrier however was valued less, in terms of its quality and potential usefulness. As a result, there is a need to reassess this product; given that some of the more positive feedback was from women who found the idea of the product empowering, it may be that different products/solutions need to be considered for different gender groups. Feedback on the videos fits with insights from the analytics data; there was a tendency for participants to lack awareness of their existence and to have preconceptions about their tone and content. As indicated above, a UX review of their position and framing within the site is required.

Implementation of Wrapped

Our discussions with local decision-makers indicated that there was receptivity in a range of different sexual health systems for the Wrapped intervention. In particular, there was recognition of the need and potential offered by digital health approaches, and it was felt that sexual health promotion and condom distribution were important areas where more digital provision was needed. Given the complexity of local commissioning arrangements and the changing nature of local landscapes, it was suggested that flexibility would be important in developing a strategy for the implementation of Wrapped. It thus seems that further exploration and analysis will be needed, taking into account evolving commissioning structures and partnerships to determine optimal routes for implementation, as part of a full RCT.

Strengths and limitations of the work

This study assessed the feasibility of conducting a future full trial of Wrapped. The methods that were tested addressed the limitations of previous trials of sexual health interventions.16,17 The generation and concealment of allocation sequences was successful; the allocation sequence was computer-generated within REDCap and researchers were not able to influence this process. Furthermore, all REDCap reports and datafiles identified allocation group by number only, thus group assignment was successfully concealed from the individuals performing data analysis. Finally, high follow-up was achieved over a good follow-up period (12 months) for a biological outcome measure (chlamydia positivity). Given this, it is likely that any future trial would have a low risk of bias and contribute robust evidence concerning the effect of Wrapped, and more broadly of behavioural interventions, on condom use and STI incidence.

The comprehensive work undertaken to develop a recruitment and retention strategy likely contributed to low attrition, high measure acceptability, and the positive feedback received from participants. Future studies may wish to draw on the strategies used to maximise retention. Uptake rate, while low, resulted in a baseline sample where participants appeared balanced across most demographic characteristics and largely representative of web-based STI testers. There was evidence of imbalance at month 12 between the intervention group and the control group, although encouragingly there was no evidence that this difference was patterned. Chlamydia positivity at baseline was lower than the national levels of chlamydia positivity for internet test settings. Strategies to address these imbalances in any future full trial have been discussed. Nonetheless, a stopping point should be implemented once half the sample has been recruited to assess levels of uptake and chlamydia positivity to determine ongoing trial feasibility.

The nested qualitative study was able to demonstrate the acceptability of trial procedures and measures and provided tentative evidence of impact on condom-use beliefs and behaviour. While we achieved our target of 30 interviews with trial participants, the sample lacked diversity constituting mostly of white female participants thus limiting the generalisability of findings. Obtaining a more diverse sample was challenging due to the dual importance of having a sample which also represented experiences of those in the intervention and control groups, and with different levels of intervention engagement. In a future trial, the increased number of participants available would widen these categories and facilitate achievement of a more representative qualitative sample. Interviews with the sexual partners of trial participants were limited in number and value. Twelve participants were willing to refer their partners for interview and while four viewed the participant information sheet, only three took part. Furthermore, their responses lacked depth and contributed little to our understanding of intervention impact. Accordingly, continuing this approach at full trial is not recommended.

Interviewees participating in the nested qualitative study also provided valuable feedback on the Wrapped intervention. This was combined with data from several other sources, including data analytics, ordering/dispatch logs, and the surveys, to provide detailed information on engagement. While this triangulation of data made for a robust approach, it also identified some evidence that Matomo did not pick up all cases of interaction with the Wrapped website. Following consultation with a Matomo expert, we believe that the most likely explanation for this inconsistency is that some of our participants had software in place on their device that blocked Matomo. This could be resolved in a future trial by changing the nature of the tracking code used.

Equality, diversity and inclusion

The burden of STIs is known to be unequal among young people, with those who identify as black, GBMSM, or who live in areas of higher deprivation, experiencing disproportionately high levels of diagnosis.1 Any trials testing interventions which aim to reduce STIs must therefore include representation of these groups within the sample and perform subgroup analyses to identify the impact on health inequalities. Given that the purpose of this study was to assess the feasibility of conducting a full trial, planned analyses included examination of whether the sample available for analysis represented the sampling pool across demographic characteristics, namely, age, gender, IMD quintile and ethnicity. As discussed above, the sample was identified as largely representative, with actions to address the exceptions proposed. Unfortunately, data on sexual identity were not made available by Preventx. This meant that we were unable to examine whether the study sample represented the sampling pool on this measure. We did however collect these data ourselves at baseline which were used to assess whether differential attrition was patterned by sexual identity along with the above participant characteristics. At full trial, it would be imperative to obtain data on sexual identity for the sampling pool to enable comparison with our sample to be made.

Our PPI group reviewed all participant materials thoroughly, assessing their accessibility and inclusiveness. Several iterations in line with this were made before sign-off by the group. It is however acknowledged that our PPI group, in addition to the wider research team, lacked ethnic diversity and therefore opportunities to improve the cultural competence of the materials and procedures may have been missed. Furthermore, the PPI group lacked representation of 16- to 19-year-olds, as did the sample of participants consulted in development of the recruitment and retention strategy. The latter was also predominantly female. While the research team would remain the same for any future full trial, a new PPI group would be formed such that it is more balanced and that all groups disproportionately affected by STIs have good levels of representation (see Public and patient involvement).

Public and patient involvement

PPI work was strongly embedded within this research. This included an assessment of all participant materials for the fRCT and nested qualitative study (participant information, consent statements, surveys, focus group and interview schedules), representation on the SSC, co-development of the recruitment and retention strategy, and development of recommendations for future research (see below). Their involvement undoubtedly improved the participant research experience and likely contributed to the observed level of retention and the positive feedback received from participants. As noted above (see Equality, diversity and inclusion) the PPI group, while representative of the target population (all were current or former users of Preventx), had some gaps in diversity. Although there was a good mix of males and females (four males, two females), no other gender groups were represented. All members identified as white ethnicity and were in the upper end of the age bracket (all 20–24 years). Information on sexual identity was not collected. If a full trial of Wrapped is undertaken, a new PPI group should be formed which represents underserved groups, particularly those who experience disproportionately high levels of STIs. Given that the fRCT under-recruited 16–19 years olds, having representation of this age group on the new PPI group would be important to enable focused work on how to maximise their uptake. It would also be important to include individuals with a recent STI diagnosis given the importance of maximising uptake and retention in this group.

Recommendations for future research

The findings of the fRCT and nested qualitative study were discussed with the PPI group at study end. Reflections were made on the success of the recruitment and retention strategy and changes that should be considered. Guidance was also received from the DMEC and SSC throughout. The following project-specific recommendations for full trial are the product of these discussions:

• Ensure partner sites have contracts with Preventx for at least the duration of the recruitment period.
• Work with Preventx to ensure that should an area migrate between their services, that there can be a seamless transfer such that recruitment is unaffected.
• Ensure that study adverts placed on the STI self-sampling website are visually distinct, utilise user trust in the Preventx brand, and are visible via a mobile phone interface without scrolling.
• Ensure that data on sexual identity of individuals in the sampling pool is made available by Preventx to allow comparisons to the sample to be made.
• Form a new PPI group which better represents the target population and includes good levels of representation by groups disproportionately affected by STIs.
• Work with the PPI group to develop and test alternative types of messaging that could be used to encourage participation of higher-risk service users.
• Review study process and procedures with the PPI group to identify changes and opportunities that could help to increase uptake by those under 20 years, and to increase the recruitment and retention of those from more deprived backgrounds.
• Explore and implement strategies aimed at increasing participant retention within the control group.
• To inform what input variables to include in multiple imputation at full trial, consider those which may predict missing data, or correlate with the value of the missing data, and ensure that relevant information is collected.
• Introduce a stopping point when half the sample has been recruited at which recruitment rate and chlamydia positivity of the baseline sample are assessed and a continuation decision made by the SSC/DMEC.
• If the final sample available for analysis does not match the sampling pool on one or more demographic characteristics found to interact with STI and/or behvioural outcomes, estimate a weighted intervention effect from the trial data to match that of the broader sampling population.
• Conduct a user experience (UX) review of the Wrapped video components to explore and address any potential issues concerning usability or appeal.
• Remove tailoring aspects of the condom sample pack (choice of box and insert) to reduce costs.
• Reassess the condom carrier with service users; consider whether different products/solutions are needed for different gender groups.
• Estimate the level of uptake and engagement with Wrapped, including by demographic group, should Wrapped be made available outside of a research context.
• Improve the reliability of the measure used to assess control group contamination.
• Exclude participation of individuals from the same household.
• Explore using a different form of Matomo tracking code that will not be blocked by device software.
• Capture detailed information via the surveys on NHS resource use and private costs.
• Use a decision-analytic model to extrapolate costs and benefits of Wrapped beyond the follow-up period.
• Ensure that individuals participating in any qualitative aspects of a future trial represent the target population.
• Conduct further stakeholder engagement (involving end users, sexual health commissioners, and decision makers working within commercial web-based STI-testing services) to assess the appropriateness and appeal of Wrapped thus maximising future opportunities for implementation.
• Continue to explore optimal routes for implementation, taking into account evolving commissioning structures and partnerships.

The following recommendations are made for others embarking on feasibility studies or full trials of behavioural interventions:

• As part of standard study risk assessments, carefully consider all assumptions regarding recruitment (e.g. continued ability of sites to recruit) and develop contingency plans that can be swiftly enacted if necessary.
• Work closely with PPI groups and UX professionals to review the entire participant journey from study advert, through engagement with intervention, and to completion of all study materials. Removing identified friction points could help to increase participant recruitment and retention.
• Develop a study recruitment and retention strategy with support from PPI members. As far as possible, base decisions on existing evidence of what works for the specific participant group and study context; review plans with representatives of the target population and adjust as necessary. If possible, test the strategies in a feasibility study prior to implementation in a full trial.
• Aim to develop a PPI group that reflects the participant sample as closely as possible. If there is evidence to suggest that any particular group(s) may be more challenging to recruit or retain, ensure that they are well represented by the PPI membership.
• During recruitment, work closely with the SSC and DMEC to review the number and representativeness of the participant sample and discuss/agree steps to address this where appropriate.
• Consider assessing the ‘real-world’ uptake of the intervention being tested by offering it outside of the study context. This information could be invaluable to future commissioners.
• Identify potential future commissioners and work with them to identify (and then collect) data that would be important to them in commissioning decisions, and also to identify/explore potential routes to market.

Conclusion

Using multiple methods and a multidisciplinary team, we have shown that the Wrapped intervention is acceptable and that a full definitive trial is feasible. We have recently been awarded funding from the NIHR to conduct that trial (NIHR award NIHR157903). The above research recommendations will be implemented to further enhance the robustness and value of this research.