Stage 1: intervention development, methods and findings

An adapted version of the following methods was published open access in Matthews et al.73 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

The HelpMeDoIt! study was undertaken in two stages: stage 1, an intervention development and formative evaluation phase; and stage 2, a feasibility RCT. This chapter describes stage 1, and is split into three key sections:

1. overarching approach to stage 1 intervention development
2. findings from stage 1
3. progression criteria from stage 1 to stage 2.

Multimethod approach to intervention development

The aim of stage 1 was to design the HelpMeDoIt! app and website with the help of users and to explore initial usability and acceptability. It was important that the development of the intervention was guided by the target audience, and was theory based built on the best available evidence for behaviour change. In addition to the app and website, the goal of stage 1 was to develop a comprehensive programme theory and logic model. This programme theory was tested and refined as part of both stage 1 and stage 2 of the study.

We took a novel approach to the design of the HelpMeDoIt! app and website by combining four approaches to intervention development. These were general intervention development methods in the form of the ‘6SQuID model’ (6 Steps in Quality Intervention Development);74 digital health-focused methods using the Person-Based Approach75 and the Behaviour Intervention Technology model;76 and identification of appropriate behaviour change theories and techniques using the behaviour change taxonomy16 and current theoretical evidence base.

A brief overview of each approach is provided below. The key components of each approach, and how they complement each other, are outlined in Appendix 1.

Approach 1: the 6 Steps in Quality Intervention Development (6SQuID) model

This model by Wight et al.74 was developed to address gaps in current guidance for the development of interventions. For example, the Medical Research Council (MRC) guidance on developing and evaluating complex interventions77 is primarily devoted to evaluation and does not provide sufficient detail on actual intervention development. The 6SQuID method involved six steps:

(1) defining and understanding the problem and its causes; (2) identifying which causal or contextual factors are modifiable: which have the greatest scope for change and who would benefit most; (3) deciding on the mechanisms of change; (4) clarifying how these will be delivered; (5) testing and adapting the intervention; and (6) collecting sufficient evidence of effectiveness to proceed to a rigorous evaluation.

We adhered to this six-step process, and used it as the overarching approach throughout our intervention development from initial idea to testing in the feasibility trial.

Approach 2: the Person-Based Approach

The Person-Based Approach is a framework for developing digital interventions that are based on a comprehensive understanding of the social, psychological and environmental context of the target group.75 This approach involves potential users in the development of the intervention and incorporates their perspectives. As well as focusing on the views of users about engagement, content and usability, crucially it also addresses the behaviour change techniques included in the intervention. We considered the Person-Based Approach complementary to the approaches we were already using to identify theory- and evidence-based approaches to changing weight-related behaviours. The Person-Based Approach provided a systematic approach to intervention development, relying on qualitative methods throughout the whole process to inform intervention design. This type of iterative, consultation approach, understanding user perspectives and incorporating key contextual influences, was key to developing an intervention that could be engaging and have a chance of being effective.

Approach 3: the Behaviour Intervention Technology model

Another digital health-related framework is the Behaviour Intervention Technology (BIT) model.76 The BIT model was developed to fill a gap in the literature on the design of behavioural intervention technologies, such as lifestyle interventions using web-based technology and mobile phones. The BIT model helps answer the questions why, what, how and when. For example, ‘why’ was reflective of the intervention aims; ‘what’ includes the BIT elements such as app notifications and report logs; ‘how’ includes behaviour change strategies, such as goal-setting, and technical characteristics, such as personalisation of the app; and ‘when’ refers to the navigation through the app/website as determined by either the software or the user. This model was particularly helpful in guiding the development of software characteristics.

Approach 4: utilising theory and behaviour change techniques

Between 36% and 89% of interventions that seek to change health behaviours are not clearly theory based.78 Interventions based on theories of behaviour change have been shown to be more effective, although reviews relating the impact of using theory to the success of interventions have shown mixed results, and theory is often poorly applied.79 The use of behaviour change techniques and how they relate to theoretical concepts is also often inadequately reported.80 We sought to address both of these issues by exploring multiple behaviour change theories and identifying the most appropriate candidate theories and associated behaviour change techniques that could be useful in the HelpMeDoIt! intervention. We used these theoretical underpinnings to develop a programme theory, identifying multiple causal mechanisms describing how the HelpMeDoIt! intervention could lead to positive outcomes for weight management in adults with obesity.

Figure 2 presents how these four approaches were combined. This involved the 6SQuID model being used as an overarching cradle-to-grave process, with the Person-Based Approach, the BIT model and behaviour theory approaches being incorporated to address more specific development issues. The other elements represented in Figure 2 will be described in detail throughout this chapter.

FIGURE 2 The combined framework for the development of the HelpMeDoIt! intervention. (Figure not reproduced here; see the original article.)

Overarching approach: 6SQuID

The remainder of this section is structured using the six steps of our overarching approach, the 6SQuID model. We describe under each step how and when we incorporated the other three approaches: the Person-Based Approach, the BIT model and behaviour change theories and techniques.

6SQuID step 1: ‘define and understand the problem and its cause’

The initial step of intervention development involved undertaking a literature review to identify up-to-date evidence on obesity. This involved gathering data on the prevalence, causes and associated risks. Obesity is a well-researched area, so there was extensive evidence providing insights into this public health problem. The causes of obesity are complex and multilayered, and the solutions are likely to involve intervening at multiple levels, including individual, interpersonal, organisational, community and policy. The HelpMeDoIt! intervention focused on the first two of these levels. An overview of our literature review findings was provided in Chapter 1. Further detail can be found in our study protocol (see Report Supplementary Material 1).

6SQuID step 2: ‘clarify which causal or contextual factors are malleable and have greatest scope for change’

As part of the above review, we explored causal and contextual factors that could be influenced as part of the intervention. The key influencing factors associated with weight loss are diet and physical activity. Harnessing positive social support was identified as an important factor, as were motivation and self-efficacy. These have already been discussed in Chapter 1.

6SQuID step 3: ‘identify how to bring about change: the change mechanism’

Identifying the potential mechanisms of change was a critical step in the development of the intervention. It was at this stage that we incorporated the first of our other intervention development approaches by identifying relevant behaviour change theories and techniques. We conducted a search to identify the intervention components most likely to contribute to successful weight loss, as well as components associated with successful and unsuccessful interventions. We also conducted a search for web-based and other technology-based interventions and those involving any kind of social support, whether from friends, family, colleagues or groups such as Weight Watchers. We also conducted an internet search for websites or apps aimed at changing health behaviours around diet and physical activity. We found many sites that utilised monetary incentives or prizes, competition with others, behavioural goals, and social support elements such as chat forums. However, we did not identify any that included family members and friends to encourage and promote weight-related behaviour change in the way we envisaged. Our review of the literature and 6SQuID steps 1 and 2 provided us with the rationale to develop an intervention for adults with obesity, using technology in the form of an app/website, to facilitate social support for weight loss. The resulting intervention is, thus, a combination of (1) individual behaviour change approaches, (2) social support and (3) the use of technology.

Behaviour change theories and techniques

The literature review identified the most appropriate candidate theories and associated behaviour change techniques that could inform the early-stage development of the ‘version 1.0’ programme theory and logic model (Figure 3). This initial logic model broadly focused on goal-setting, self-monitoring and social support,as these techniques have been shown to be effective for weight loss.11,81 It also included a number of other evidence-based techniques, such as boosting motivation and increasing autonomy. We mapped these behaviour change techniques to behaviour change theories relevant to our intervention.82,83 These were Social Cognitive Theory,18 Self-Determination Theory,84 Control Theory19 and Social Support Theories.27,85 Elements of the version 1.0 logic model that related to each of these theories are highlighted in Table 1.

FIGURE 3 Version 1.0 of the HelpMeDoIt! programme theory and logic model. (Figure not reproduced here; see the original article.)

Table 1 presents the mapping of individual behaviour change techniques used in the intervention to these four behaviour change theories.

6SQuID step 4: ‘identify how to deliver the change mechanism’

Version 1.0 of the logic model was based on the research evidence, using 6SQuID steps 1–3. The logic model was used as a starting point for then engaging with stakeholders, with a view to further refining the mechanisms of change and intervention content. At this point we began developing the intervention alongside potential users and technical experts from a software company. This was a critical stage in our intervention development and involved the use of the two other approaches: (1) the Person-Based Approach75 and (2) the BIT model.76 Each of these explored how to effectively deliver the mechanisms of change and specific content of the interventions.

The Person-Based Approach

We adopted the Person-Based Approach as an appropriate method of involving key stakeholders in the development of the intervention, allowing us to iteratively explore usability, engagement and content, including relevant behaviour change techniques. The first step of the Person-Based Approach was to develop guiding principles on which to base the intervention. Our HelpMeDoIt! guiding principles are outlined in Appendix 2. The second step was to gather insights from key stakeholders, namely potential users of the app/website and members of the software company (described below). The Person-Based Approach helped identify which key components should be included, taking into account context and implementation, in a way that was acceptable and convincing to the target group.

The Behaviour Intervention Technology model

The purpose of using the BIT model was to guide the practical build of the intervention in relation to its conceptual aim of weight loss using social support. The BIT model was crucial in providing us with a digital health-focused framework for ensuring that the intervention’s technical aspects aligned with its objectives of goal-setting, self-monitoring and social support via helper interaction. The model is based on five elements: (1) why the software is being developed, for example to promote weight loss; (2) how the software is conceptually considered to achieve the overall goal of weight loss, for example via goal-setting, self-monitoring and social support; (3) what elements the software requires to ensure that these aspects of the intervention are achieved, for example use of notifications and reminders to use the app; (4) how these features will technically be delivered by the software to meet the needs of the participant, for example specific requirements, such as choosing days of the week, for the goal-setting feature; and (5) when the various elements of the intervention are delivered; for example, the timing of some features will be led by participants, compared with other features that will be time-based and set by the software, such as push notifications. Table 2 provides examples of how the BIT model helped develop the technical build of the app and website. For this phase, we worked alongside a software company, ensuring that we captured their expertise in and insights into developing the intervention into an app and website.

Methods for involving users in the development phase

We recruited a development panel of 10 users. An overview of the methods used with the development panel is provided in Table 3. The development panel was instrumental in helping us develop features of the app and website that were evidence based but user led. It allowed the study team to gather critical insights into the psychosocial context as well as the perspectives of potential users. It enabled us to explore ideas for engagement, and helped refine the key elements and delivery mechanism of the app and website. Findings are presented in Results of stage 1 intervention development.

6SQuID step 5: ‘test and refine on a small scale’

In addition to our development panel, we used a separate testing group to gather feedback at various points throughout the development process.

Methods for involving users in the testing phase

An overview of the methods used with our testing group is provided in Table 4. In addition to gathering feedback on the content, look and navigation of the app and website, this testing stage helped identify technical bugs, software issues such as navigation errors, and areas of the app/website that could be strengthened further. Findings are presented in Results of stage 1 intervention development.

Heuristic evaluation

Further testing was performed via a heuristic evaluation. This standardised process involved the app and website being assessed by two independent technology experts.88 Each expert applied and assessed set heuristic criteria to both the app and the website (see Appendix 3). The aim of the heuristic evaluation was to identify strengths and limitations of the software and highlight areas for further refinement.

Logic model and programme theory development

The valuable insights gathered by 6SQuID steps 1–5, incorporating the Person-Based Approach, BIT model and behaviour theory/technique exploration, helped refine and strengthen the HelpMeDoIt! programme theory and logic model. We developed and continually updated the programme theory used to design our intervention and plan the evaluation, and this is represented in the logic model. The original version 1.0 of the programme theory logic model (see Figure 3) identified general links between the software elements of the intervention and the proposed outcomes. The ongoing user involvement and feedback throughout stage 1 allowed us to consider additional contextual factors and further refine ‘version 2.0’ of the programme theory and logic model. The revised logic model now included (1) further mechanisms of action (e.g. increased motivation and increased autonomy); (2) a reduced number of intermediate outcomes (e.g. less sedentary time and increased physical activity and healthy eating were condensed into healthy habit formation); and (3) multiple contextual factors (e.g. availability of people in participants’ social network to act as helpers) (Figure 4).

FIGURE 4 Version 2.0 of the HelpMeDoIt! programme theory and logic model. (Figure not reproduced here; see the original article.)

6SQuID step 6: ‘collect sufficient evidence of effectiveness to justify rigorous evaluation/implementation’

In step 6 of the 6SQuID process, the resulting version 2.0 logic model and intervention were ready for implementation and testing in a feasibility RCT with accompanying process and health economic evaluation (stage 2). The methods and findings for this stage are presented in Chapters 3–6.

Results of stage 1 intervention development

As described above, the intervention was developed in collaboration with a development panel of users, a testing group of users and a software company and used evaluation feedback from software experts. Details of how this input helped inform the development of the HelpMeDoIt! intervention are presented below in the following order:

1. findings from the development panel
2. findings from the testing group
3. usage statistics for the app and website
4. findings from the heuristic evaluation
5. findings from the USE questionnaire.

Findings from the development panel

We recruited 10 individuals with a range of characteristics to our development panel. We had a good gender balance (female, n = 6; male, n = 4) and spread of ages (18–70 years). All participants were interested in losing weight. We also attempted to include individuals with characteristics associated with a higher likelihood of obesity, and thus included participants of non-white ethnicity (n = 2) and a greater number of individuals from areas of lower socioeconomic status (n = 7). We included two individuals with limited experience of using smartphone apps (the majority of people who responded to the study advert were experienced in this). When participants were unable to attend one of the focus groups, their feedback was collected via a one-to-one interview (n = 1) or by e-mail (n = 2).

Insights and feedback from the development panel were useful and informative at all data collection time points. Participants were engaged, motivated and creative in their approach to the software development and many of their ideas and suggestions were incorporated into the prototype of the app and website. Some suggestions were not included as they were beyond the scope of the app, such as taking photographs of a meal and the app providing an accurate nutritional analysis.

Thematic analyses of the focus groups identified three main themes and related subthemes. These are detailed in Table 5 along with examples of how the different themes were addressed within the app and website.

Theme 1: software design

This theme included discussion around how the design of the app and website could influence whether or not it was adopted and used successfully.

Subtheme 1.1: avoiding non-adherence

This included discussion on barriers to and facilitators of using the app and website, and how to facilitate ongoing engagement. Our review of the literature had highlighted that ongoing engagement with apps was a significant challenge for software designers. Therefore, the development panel were asked for their insights on what features might help maintain their engagement with the app. Various methods were discussed, such as wording that boosted intrinsic motivation, as well as gamification, daily messages/tips from the app, and a method of receiving instant feedback from their helper(s). These ideas were explored further in the three focus groups. The agreed features were (1) gamification, where participants and helpers could earn points (which would convert to medals and trophies); (2) regular tips via the app or e-mail, which included tips for weight loss, physical activity, well-being, SMART (specific, measurable, attainable, relevant and time-bound) goal-setting, self-monitoring and social support; (3) daily messages via the app or e-mail, which provided motivation, inspiration and encouragement; and (4) instant feedback between participants and helpers via fun animations, for example an animated ‘high five’ from helpers to participants when the latter achieved a goal.

The development panel also discussed likely barriers to and facilitators of using the app. For example, when they were asked to identify reasons they might not use the app, their responses included having trouble with the login process; not having in-app rewards; if the app shows adverts or asks for money; a lack of engagement from the nominated helper; difficulties with using the interface; or if they found another app that performed better.

Subtheme 1.2: adaptability of the app

Here, issues about flexibility and the individual’s ability to tailor certain aspects of the app to suit their routine were discussed. Participants provided useful insights into contextual factors that might encourage use, for example being able to personalise the time and frequency of notifications to match their work schedule:

You could also change it, like if you do get a reminder, you can set snooze for 1 hour, 2 hours. I don’t know if that’s possible. I don’t know, but it’s an option. Because you may think, ‘I am at work just now, but maybe when I get in I want to be reminded’ and you can upload that information.

Not all members of the development panel viewed the use of short message service (SMS) text messaging positively. Although some participants did not mind receiving text messages, others felt that this was ‘intrusive’:

I don’t like that [receiving messages via text]. The same as you get text messages from all these companies, stuff like that . . . I feel like junk spam kinda text messages.

It was agreed that the main method of communication from HelpMeDoIt! would be set as a preference by the participant (i.e. they would have the choice of receiving push notifications via the app or by e-mail). Several participants indicated they would choose to receive notifications via the app, with some stating that they enjoyed logging into an app each day and seeing a different message. This provided variety and helped with ongoing engagement. However, SMS text messaging was not considered intrusive in the context of helper interaction. Participants liked the idea of receiving supportive text messages from their helper(s). It was agreed that a ‘quick access’ icon would be included in the app to allow participants and helpers to send text messages to each other.

Subtheme 1.3: usability of the app

This related to technical issues such as the app interface, colour and font, and the software itself. The development panel provided feedback on the initial designs and then finally on the prototype of the app and website. Overall, their feedback was positive. The majority of participants liked the layout, colour scheme, design and key features. The development panel also shared critical feedback of the app and website that identified areas for further consideration and refinement. Criticisms of the intervention included difficulties with the initial login via the third-party test app; numerous software bugs affecting their ability to test run the app (e.g. absence of screen animations, buttons obscured by text); not having enough guidance on how to initially use the app or earn rewards; and not having the ability to earn points, and, thus, rewards, during the early stages of testing.

Theme 2: intervention content and features

This theme included the actual intervention elements, what they might look like within the app and website, and how they could help or hinder the process of adopting a healthy lifestyle change.

Subtheme 2.1: key features of the app

This consisted of insights related to goal-setting, self-monitoring and helper interaction.

Goal-setting

Participants agreed that having template goals developed by the research team would be useful to help them understand what an appropriate and manageable goal was. They felt that it was important that the template goal could be edited so that they could tailor it to their own circumstances. They also suggested that focusing on a small number of goals would be beneficial, and that goals should focus on sustainable lifestyle change rather than simply weight loss:

It was good that it [the app] gave you the option of already made goals, if that makes sense. Something you struggle to think of. Having one there and being able to work on it gives you more ideas. It’s a good idea.

Self-monitoring

Graphs were unanimously participants’ preferred method of viewing and monitoring progress. Participants shared positive experiences of using graphs on other apps and requested that the HelpMeDoIt! app use a simple yet informative style similar to that of apps they had previously used. They understood that monitoring their progress could help them ‘learn to succeed’:

. . . you are proving to the app like I can do this. Not saying something like ‘I am going to change the world’ but at least like ‘to walk more’, ‘take one bus stop less’. Yes, more like lifestyle changes.

Helper interaction

Early discussion highlighted that participants were engaged with the concept of helpers, and in particular with helpers interacting via an app. They felt that the intervention could be helpful and effective but they spoke about other lifestyle apps that they used (e.g. MyFitnessPal) and emphasised that HelpMeDoIt! needed to identify its uniqueness in relation to the helper interaction. They shared numerous ideas in relation to interaction with their helper(s). This included insight into the motivation of helpers, who potentially would be more willing to log on to the app and support their friend initially for the fun and novelty. Some shared thoughts on how they imagined the helper would use the app for instant feedback (e.g. a husband or wife messaging their partner from across the room). Participants agreed that having a ‘pat on the back’ from their helper would be encouraging and motivating, and they thought they would enjoy receiving motivating messages from their helper(s). To maximise the support that helpers provide, it was recommended that as much information as possible was shared with them, with helpers basically seeing the same version of the app as the participant. This would allow helpers to see the achievement of small goals, as well as larger weight loss goals. Helpers should therefore receive notifications of new goals, goal progress and goal lapses; however, participants recognised that the frequency of notifications should be limited to avoid their helper(s) becoming annoyed and disengaged with the app:

[I like] having other people helping you. No one pats you on your back unless you go around [saying] ‘I reached my target weight’. All right there are no like celebrations, no cake, no anything like that.

Subtheme 2.2: feedback from the app

This related to app engagement around progress graphs, reminders, prompts and rewards. Participants were aware that receiving too many push notifications and e-mails could be a potential barrier to engagement. The majority agreed that either a weekly or a fortnightly summary e-mail of their progress would be useful and non-intrusive. They also agreed that push notifications were a good method of receiving feedback, as these reminded them to use the apps on their phone:

I think in Fitbit [Fitbit, Inc., San Francisco, CA, USA] every week you get an e-mail to see how many steps you’ve done and your overall progress or whatever, which is I suppose it saves you being bombarded.

Subtheme 2.3: key features of the website

This addressed content that participants considered useful and informative to have on the accompanying website. The majority considered that they would use the website as a source of information, but would use the app as the main source of the intervention. They did not want all of the information on the app, a suggestion that was also supported by the software company, which also suggested trying to keep the app simple and develop any extra detail for inclusion on the website. Ongoing discussion with the development panel identified that the website should provide guidance (1) for participants on how to safely and effectively lose weight and get the most out of their social support; and (2) for helpers on how to be a good helper and how to support their friend when they experience setbacks, etc.:

. . . maybe for some web page maybe some particular advice, I don’t know . . . What to eat? Or . . . I don’t know. Because that would . . . for me reinforcement is you would go to a web page for reinforcement. For me it’s some sort of psychological reinforcement.

Theme 3: people and context

This theme covered social support and contextual factors in the participant’s life that could affect the likelihood that they would be able to make lifestyle changes. This theme included how people in the participants’ social network could help them achieve their goals and what that their social network looked like in terms of supportive and less supportive individuals. It also touched on factors related to the motivation needed to effect change. Finally, this theme included discussion of contextual factors in terms of work obligations, family responsibilities and friends, and the participants’ own histories of health and attempts to lose weight.

Subtheme 3.1: social support for lifestyle change

This related to peer modelling, characteristics of helpers, types of social support offered, and barriers to and facilitators of being a helper. Participants discussed the different people they had in their social network and how some of them might be good helpers because they would be good at motivating them, while others might be supportive because they too were embarking on lifestyle change. Some were reluctant to choose helpers who had an active healthy lifestyle, feeling that these helpers might lack empathy and understanding of a weight loss journey. Most participants liked the idea of nominating helpers from their social network rather than strangers:

I think having someone who can push me would be very good because I lack motivation on my own. But I think it is better to have one single person than different persons.

Subtheme 3.2: motivation

This encompassed participants’ motivations for using the app, lifestyle change, sustaining lifestyle changes, and whether motivation was intrinsic or extrinsic. Some participants were motivated by recently diagnosed health problems, for which their lifestyle change was a form of disease management. Others described how they were motivated by the change in lifestyle rather than setting goals, and therefore sustained lifestyle change was the ultimate goal. They also shared important insight into their motivation for continuing to use the app, which typically involved the app being simple, engaging and fun to use:

. . . from my perspective, I am not looking to achieve my target weight. I am looking to generally improve my lifestyle [. . .] drinking water rather than Irn Bru [AG Barr, Cumbernauld, UK]. Rather than actual 18, 15, 10 stone. The rest of it is no target for me [. . .] ideally I would like it to be sustainable for the rest of my life.

Subtheme 3.3: context

This included insights into how participants might fit the intervention into their everyday life, including working patterns, childcare, problems with their short-term memory, comparison with other apps and previous experiences of app use. Participants described how they used apps on their phone daily, and typically checked their phone when they woke up in the morning. Others noted how their busy work day made it difficult to manage apps during the day; however, they used their evenings to ‘catch up’ on notifications. Some participants described the challenges of sustaining lifestyle change around shift work, and how weekly rather than daily goals were likely to have greater chance of success:

I don’t have some apps on my phone, because I don’t have time during the day. I only do it on my iPad [Apple Inc., Cupertino, CA, USA] at night-time so I am not harassed by notifications. But I like to be harassed at night. That’s when I fill everything in.

First thing I do is check my phone when I get up.

Table 6 presents a summary of the key features agreed by the development panel for inclusion in the app and/or website.

Findings from the testing group

Feasibility of the app and website was also evaluated by the 28 participants recruited to the testing group. We purposely aimed to recruit participants with a range of characteristics: 19 were women and 9 were men; age range was 18–64 years; nine participants were from the top two quintiles of socioeconomic deprivation; two participants were from minority ethnic backgrounds; and two individuals had limited experience of using apps (the majority of people who responded to the study advert were experienced in the use of smartphone apps). Participants were recruited via word of mouth (n = 15) (e.g. by members of the original developmental panel to people in their social network), study adverts on Gumtree (www.gumtree.com) or Facebook (Facebook, Inc., Menlo Park, CA, USA) (n = 8), and a university-wide staff e-mail announcement (n = 4). Although feedback was collected on an ongoing basis between August and December 2015, participants provided feedback on only one occasion as a ‘user’ (n = 11), a ‘helper’ (n = 11) or both (n = 6). Detailed feedback from the testing group is presented in Appendix 4. A brief overview is provided below. Overall, feedback from the testing group was constructively critical.

Early feedback at the first time point (August and September 2015) focused on gathering feedback on the initial ideas and designs for the app and website. Feedback was positive, with participants in agreement about the proposed key features of goal-setting, self-monitoring and helper interaction, as well as how these would be delivered. They provided feedback on the initial design, look and feel of the app, and the majority of responses were positive comments on the simple design and colour scheme. Helpful suggestions related to grammar were received to help make the app more engaging. They also provided insights to help the team refine the initial design, for example guidance to choose from more than one goal category.

Feedback collected at time point 2 (October and November 2015) focused on participant insights on our refined app and website ideas. Feedback highlighted that some guidance was required to help participants choose and set goals using the app. This phase of testing also identified useful content to support helpers in their supportive role. Ideas participants provided for the helper section of the website included (1) example conversations between the helper and their friend; (2) guidance on what a helper could do if their friend was struggling to meet their goals; (3) things not to say to their friend; and (4) an online quiz to engage and motivate helpers:

When mentoring at my work it’s about trying to be encouraging so maybe some examples of encouraging dialogue.

At the final time point (December 2015), participants were presented with the prototype version of the app. This was accessed using a third-party testing platform. This enabled participants to access and use the app in its beta version and sync to the latest version as features were updated and errors were rectified.

Feedback was gathered via the ‘think-aloud’ approach.87 This approach facilitated identification of software bugs and errors, which the software company worked on rectifying immediately (e.g. icons obscured by text, inconsistent display of progress graphs). Many of the software bugs were caused by the interaction of our software with a third-party testing platform. Some of these were addressed when the third-party testing platform released an update near the end of our testing phase. In addition to software issues, the feedback helped highlight navigation issues, things that were not intuitive in the design and areas of the app that could be strengthened further. One key suggestion was the need for tutorial guidance to help first-time users understand the process of choosing template versus custom goals:

I’m not sure what I’m meant to do here [on goal-setting screen]. Do I press this button? How do I go back and see the list of goals again?

When time allowed, some participants in the testing group also provided feedback on the website (n = 8). Feedback for the website was very positive, with the majority of participants commenting on the simple layout, easy navigation, clear display of information and fun animations. Several suggestions were made to strengthen the website further, for example embedding hyperlinks within the text and simplifying some of the grammar.

Usage statistics for the app and website

There were 498 individual logins recorded for the app, showing that the development panel accessed the software regularly during the testing period. All key features of the app were used. The website was accessed by the development panel on 70 occasions, with 687 web page views in total. The average login for each member of the development panel was four sessions of around 10 minutes’ duration. During this initial testing phase, a number of software issues and bugs were identified and several participants had difficulty logging in. This is normal and expected when beta-testing software.

Findings from the heuristic evaluation

A heuristic evaluation was undertaken by two independent technical experts. The aim of the evaluation was to identify strengths and limitations of the software and to highlight areas for further refinement. This involved applying and assessing set heuristic criteria to both the app and the website. The heuristic criteria were scored on a scale of 1–5 (1, very poor; 5, excellent).

Overall, the findings of the heuristic evaluation were positive. The full heuristic report is presented in Appendix 3. The majority of criteria for the website (71%; n = 35 of 49 criteria) and the app (71%; n = 24 of 34 criteria) were assessed as either ‘good’ or ‘excellent’. Eleven issues were identified as needing improvement. Most of these were minor, for example slowing down the images on the website homepage, and clarifying the goal template headings in the app. Two issues with the app were rated as ‘poor’ and related to a lack of a clear method of returning to (1) the main navigation menu and (2) the main dashboard. All issues highlighted by the heuristic evaluation were addressed by the study team and the software company.

Findings from the Usefulness, Satisfaction, and Ease of Use questionnaire

During the final focus group, participants were asked to complete the Usefulness, Satisfaction, and Ease of Use (USE) questionnaire.86 This explored the acceptability of the software and also allowed the study team to assess the feasibility of using this questionnaire as a process measure in the trial. Overall, the questionnaire was quick and simple to complete and it was identified as useful for stage 2. Participants responded to 30 individual statements on a scale of 1 (strongly disagree) to 7 (strongly agree). The results are presented in Appendix 5. Despite the app being a prototype, all of the mean responses (with the exception of question 29) had a score of ≥ 4 [i.e. all mean participant responses were greater than or equal to a score of 4 (neutral)]. However, it should be noted that in this initial test run some of the USE questions were difficult to answer as participants did not have access to the software over a longer period of time to be able to assess items such as ‘the software helps me be more effective’. In addition, some participants had access to the software for less than 1 week (owing to difficulties with the login process caused by third-party software), which affected their ability to answer all of the USE questions, as some were not applicable because they referred to longer-term use. The USE questionnaire would therefore benefit from a ‘not applicable’ response option.

The HelpMeDoIt! intervention content

The findings from stage 1 informed the resulting intervention. HelpMeDoIt! was delivered via smartphone app and website. The list of app and website features, and how they map to individual components of the logic model, is presented in Appendix 6. Social support was the unique aspect of this intervention. The core element involved participants nominating one or more ‘helpers’ from their social network (friends, family or colleagues). Participants could nominate one or more people to be an official helper. There was no limit to the number of the helpers they could nominate; however, the website suggested one to five helpers as a manageable number. They were not restricted to choosing helpers from the UK.

The app and website had different functions that complemented each other; therefore, participants and their nominated helpers were encouraged to make use of both the app and the website. This combined approach was agreed with user representatives during our initial intervention development phase. Substantial amounts of text or content on apps were identified as a barrier to engagement. Users from our development phase highlighted the preference for a separate website that contained more detailed information. The website was designed to be accessible and viewable from both desktop computers and smartphone devices. Example screenshots of our app and website content are available online.73

The HelpMeDoIt! website

The website presented evidence-based information for participants on healthy eating, physical activity and weight loss, and guidance on how to select and/or be a good helper.

Participant information included:

• guidance on how to use the app to set SMART goals and monitor progress (tracking goals achieved and recording weight) and how to identify appropriate helpers using helpful hints to evaluate the support around them
• evidence-based information on healthy eating (e.g. daily portions of fruit and vegetables, reading food labels), physical activity (current UK guidelines) and behavioural strategies to support weight loss (e.g. regular sleep pattern)
• ‘top tips’ for weight loss (based on the key points of the evidence-based information, e.g. ‘add volume to your meal with liquid or fibre)
• ‘helpful links’ to other relevant pages [e.g. NHS Eat Well (www.nhs.uk/live-well/eat-well/; formerly NHS Food Mixer) for healthy recipe ideas].

Helper information aimed to provide helpers with the guidance needed to be an effective helper for the participant who was trying to lose weight. This information included:

• tips on how to be a good helper (e.g. ‘be a cheerleader not a coach’)
• methods of positive feedback and encouragement to the participant (e.g. the option to send animated smiles or send a well-done text message when their friend completed a goal)
• examples of non-food rewards to help motivate the participant [e.g. new music for their friend’s iPod (Apple Inc., Cupertino, CA, USA)]
• examples of dialogue and motivational language to support the participant (e.g. helpful phrases for helpers to use when their friend had experienced a difficult week)
• Helpers could interact with participants over the telephone, in person, by sending a text message or by sending ‘smiles’ via the app.

The HelpMeDoIt! app

The app focused on the three key elements of the intervention: social support, goal-setting and self-monitoring.

The participant version of the app included the following features.

• Goal-setting: participants were encouraged to set goals for healthy eating, physical activity and other lifestyle factors, such as sleep. There was no maximum goal limit; however, participants were advised that five or more goals could be difficult to manage. They could either choose predefined template goals or write their own custom goals. Custom goal-setting was a two-step process that encouraged participants to action plan and problem-solve as they set their goals. This involved setting a short title for each custom goal, followed by adding SMART goal detail; for example, ‘stand up more’ – watching evening television (TV) on Monday and Wednesday I will stand up each time there is an advert break’.
• Monitor progress: participants could update the app to either log their most recent weight or report when a goal had been achieved. Personal progress was presented over several graphs, including one that showed their change in weight over the past month and also their change in weight since they started using the app; a figure showing their most recent weight; and a list of all set goals, each with a ‘% completed’ bar.
• Nominate helpers: participants could enter the contact details of one or more individuals from their social network that they wanted to invite to be a helper. Nominated helpers received an invitation by e-mail. Participants could also use this feature of the app to remove a helper, for example if the helper was not supportive.
• Helper interaction: the app provided several ways to interact with helpers. These included sending animated smiles (e.g. ‘Great help this week’), sending text messages and making telephone calls.

The helper version of the app included the following features:

• display of the participant’s goals, where they were able to view new, completed and missed goals
• display of the participant’s progress for weight and goals completed
• methods of giving positive feedback and encouragement to the participant, for example sending animated smiles (e.g. ‘Keep up the good work’), sending text messages and making telephone calls.

An element of ‘gamification’ was used within the app to encourage frequent use and to support the ongoing engagement of both participants and helpers. This involved both participants and helpers receiving points for (1) regularly inputting progress data, (2) interacting with each other and (3) achieving goals. Once participants and/or helpers had accumulated a certain number of points, they were awarded virtual medals (i.e. bronze, silver or gold) or a trophy. Ongoing engagement was also supported through the use of (1) push and e-mail notifications (e.g. informative messages, progress summaries and notifications of new badges); (2) daily motivating messages (e.g. ‘Doing well? Think about how you can progress one of your goals this week’); and (3) weekly reminders for uncompleted goals. Helpers were sent (1) daily messages (e.g. ‘A kind word can do wonders for motivation’) and (2) regular prompts via push and e-mail notifications to remind them to provide encouragement, celebration or further support (e.g. ‘Your friend hasn’t set any goals recently. Get in touch to find out how you can help them’).

Overall, the guidance given to participants and their helpers regarding how to establish social support for weight loss was not prescriptive or structured. Participants and their helpers were encouraged to use strategies that worked best for them, and as part of this feasibility study we were interested to find out what the range of approaches were. Participants and their helpers were encouraged to interact with each other either via or outside the app and examples of how they could do this were given in both the app and the website.

Progression criteria from intervention development (stage 1) to feasibility trial (stage 2)

The HelpMeDoIt! study was overseen by an independent Trial Steering Committee. To progress to the stage 2 feasibility RCT, the intervention needed to demonstrate successful achievement of the stage 1 progression criteria, which the Trial Steering Committee had previously approved. The progression criteria are listed in Table 7, along with key associated findings. The results of stage 1 were discussed with the Trial Steering Committee, which agreed that the progression criteria had been met and approved progression to the stage 2 feasibility study.

Stage 1 conclusions

This rigorous multimethod approach facilitated the development of an intervention that was developed with input from the target audience. A comprehensive programme theory was developed, incorporating behaviour change techniques and relevant theory, and addressing multiple contextual factors. Stage 1 progressed well and resulted in an app and a website that were both feasible and acceptable to participants and their helpers. All stage 1 progression criteria were achieved and no significant threats or weaknesses were identified.