Some sections of this chapter have been reproduced from the ORBIT trial protocol, which has been published1 under the CC-BY-4.0 licence.
Trial design
ORBIT was a parallel-group, single-blind, non-commercial, randomised controlled superiority trial with an internal pilot for CYP with tics. Participants were randomised in a 1 : 1 ratio to receive 10 weeks of treatment of either online, remotely delivered, therapist-supported BT for tics or online, remotely delivered, therapist-supported psychoeducation on tics. Participants were followed-up at mid-treatment and at 3, 6, 12 and 18 months post-randomisation. Months 3 and 6 were per-protocol follow-ups in which participants were encouraged not to change medication or start alternative therapies for tics. Months 12 and 18 were naturalistic follow-ups where participants might be using alternative treatments in accordance with standard practice recommended by their usual treating clinician. A sub-sample of participants and parents were purposively selected to participate in process evaluation interviews after the 3-month follow-up time point. A flow chart of the study design is shown in Figure 1.
FIGURE 1 Trial flow. Note: Figure published in Hall et al.1 (Figure not reproduced here; see the original article.)
Internal pilot
The objective of the internal pilot was to determine whether recruitment, engagement with the intervention and retention to the trial were sufficient to allow the trial to progress and provide a definitive answer on the effectiveness of the intervention. The internal pilot ran for the first 9 months of recruitment. Allowing for a staggered start to recruitment, the stop–go rules for the internal pilot were as follows:
- The study needed to have recruited 66 patients by the end of the ninth month of recruitment.
- At least 60% of participants needed to have completed the intervention (with completion defined as completing at least the first four child chapters).
- 80% of participants who had reached the relevant time window needed to have completed the primary outcome measure (YGTSS) at the primary end point (3 months) within the specified time frame for measure completion.
The success of the internal pilot was judged by the independent Trial Steering Committee (TSC) and Data Monitoring Committee (DMC).
Ethical approval and research governance
The trial was conducted in compliance with the principles of the Declaration of Helsinki (1996), the principles of good clinical practice (GCP) and in accordance with all applicable regulatory requirements including but not limited to the Research Governance Framework and the Mental Capacity Act 2005.
Ethical and Health Research Authority (HRA) approval was received from Northwest Greater Manchester Research Ethics Committee on 23 March 2018 (protocol v2.0; ref.: 18/NW/0079). The published trial protocol1 was approved by an independent TSC and DMC. Two substantial amendments were made and approved; these are described below. The trial was prospectively registered with the ISRCTN (ISRCTN70758207) and ClinicalTrials.gov (NCT03483493).
Substantial amendments
One subsequent amendment was made to the protocol (v.3.0, 16 April 2018) after initial Research Ethics Committee (REC)/HRA approval. This amendment was to allow the 10-week intervention to be delivered over a 12-week period to account for periods of therapist absence during bank holidays or unforeseen circumstances. The amendment was approved by the committee on 15 May 2018. There was one further subsequent substantial amendment; however, this was not to trial protocol. An amendment was submitted on 28 June 2018 to give REC/HRA approval for the interview schedules as part of the process evaluation; this was approved on 2 July 2018 for interview schedules. The limited number of amendments from the original protocol is testament to the fidelity to the original proposal and indicates that the study procedures and interventions could be successfully delivered within the ORBIT trial.
Study oversight
The study was overseen by three groups.
Trial Management Group
The full Trial Management Group (TMG) consisted of all the co-investigators listed on the protocol and a representative from the patient and public involvement (PPI) group when possible. The TMG met at least every 6 months to discuss study progress and overall conduct of the trial.
Trial Steering Committee
The role of the TSC was to provide overall supervision of the trial on behalf of the Trial Sponsor and Trial Funder and to ensure that the trial was conducted to the rigorous standards set out in the Medical Research Council’s (MRC) Guidelines for GCP. The TSC met every year, with an additional meeting to review the internal pilot.
Data Monitoring Committee
The DMC assessed whether there were any ethical or safety reasons why the trial should not continue. The DMC met annually and reviewed the internal pilot.
Participants
The study sought to recruit CYP with tic disorders.
Inclusion criteria
- Aged 9–17 years: patient confirmed through screening.
- Suspected or confirmed TS/chronic tic disorder:
- - Including moderate/severe tics: score > 15 on the YGTSS-TTSS; TTSS score > 10 if motor or vocal tics only: researcher confirms at screening appointment.
- Competent to provide written informed consent (parental consent for child aged < 16 years): researcher confirms at screening appointment.
- Broadband internet access and regular PC/laptop/Mac user, with mobile phone SMS: patient confirmed through screening.
Exclusion criteria
- Previous structured behavioural intervention for tics (e.g. HRT/CBIT or ERP) within last 12 months: patient confirmed through screening.
- Change to medication for tics (start or stop of tic medication) within the previous 2 months: patient confirmed through screening, and subsequent medication/interventions commenced throughout the trial recorded at each time point for analysis.
- Diagnoses of alcohol/substance dependence, psychosis, suicidality or anorexia nervosa: confirmed through parent development and well-being assessment (DAWBA).
- Moderate/severe intellectual disability: confirmed through qualitative judgement of the assessor at the telephone screen [and confirmed at baseline through child and adolescent intellectual disability screening questionnaire (CAIDS-Q)] through questions relating to type of school the child attends and previous diagnoses.
- Immediate risk to self or others: confirmed through screening questions and DAWBA.
- Parent or child not able to speak or read/write English: patient confirmed through screening by the assessor.
Recruitment procedures
Participant identification
There were three streams to participant identification. Patients could be identified via previous or current referrals at (1) one of the two study sites or (2) the patient identification centres (PICs). For both instances, a member of the usual care team identified potential participants from the patient records or current referrals held at the two study sites. Patients were provided with a brief information sheet and ‘consent to contact’ (C2C) form, which was passed to the research team once completed.
The third stream was through public recruitment campaigns. Specifically, the study advertised for participant recruitment via the Tourettes Action website (a national charity for people with tics) and a study website. A brief information sheet and a C2C form were hosted online.
Screening and baseline appointment
Once C2C had been established, patients were contacted by a member of the research team to go through the telephone screening questionnaire. The screening questionnaire was developed by the research team to understand the patient’s eligibility for the trial and took approximately 20–30 minutes to complete.
Patients who met the eligibility requirements were invited to attend a screening/baseline appointment. The appointment was held at one of the two study sites. All patients were reimbursed for their travel costs to attend this appointment. Before participants attended the screening/baseline appointment parents were asked to complete an online DAWBA.62 Further details on the DAWBA are described under the Measures section.
At the screening/baseline appointment the researcher took informed consent and went through the eligibility criteria. As part of the eligibility check, the YGTSS assessment and the CAIDS-Q63 were conducted. Further details on measures are described under the Measures section.
After completion of baseline measures, patients were randomised into the study by the researcher via the web-based system hosted on a secure server by Sealed Envelope. At this appointment participants were introduced to the ORBIT therapy platform and provided with login details. Participants set their own start date for therapy but were encouraged to make it with 24–48 hours of their baseline appointment.
Consent
All participants provided written informed consent before completion of screening measures. For young people under 16 years old legal consent was sought from parents/carers and verbal or written assent from the young person. At 16 years of age or over written consent was sought from both the young person and their parent/carer. The original signed and dated consent forms were held securely as part of the trial site file, with copies sent to the participant, their general practitioner (GP) and the referring site (if applicable) for their records.
Randomisation, concealment and blinding
Randomisation was conducted using the Sealed Envelope online randomisation system and managed by Priment Clinical Trials Unit. Participants were randomised online by an outcome assessor. Randomisations were on ratio 1 : 1 and stratified by study site using block randomisation with varying block sizes.
The outcome assessor, statisticians, health economists, trial manager and chief investigator were blind to the treatment allocation, and the therapist was notified by the randomisation system of the treatment allocation. Although participants were not directly informed of their treatment allocation, it was likely that they would have been able to guess their allocation once the intervention had started.
Interventions
The two interventions have been described in detail in Chapter 2. The experimental arm consisted of 10 weeks of online, therapist-supported ERP therapy and the control arm consisted of 10 weeks of online, therapist-supported psychoeducation for tics.
Follow-up
Follow-up measures were completed at the mid-treatment point (5 weeks) and at 3 and 6 months (this formed phase 1, per-protocol design). For phase 2 (a naturalist design), follow-up measures were obtained at 12 and 18 months. Follow-up measures were completed remotely (via videoconferencing or telephone) and via an online database.
Measures
Primary outcome measure
The primary outcome was the severity of tics as measured by the TTSS (0–50) on the YGTSS.4 The primary end point was 3 months post-randomisation. The primary outcome (YGTSS-TTSS) was measured at baseline (pre-intervention; face-to-face), at 3 months (primary end point) and at 6, 12 and 18 months post-randomisation (online via videoconferencing or telephone where this was not possible).
The YGTSS was administered by a blinded assessor as an investigator-based semistructured interview focusing on motor and vocal tic frequency, severity and tic-related impairment over the previous week.
In this study, four index YGTSS scores were obtained: Total Motor Tic Score, Total Phonic Tic Score, TTSS (primary outcome) and Overall Impairment Rating (secondary outcome). The Total Motor Tic Score is derived by adding the five items pertaining to motor tics (range 0–25); the Total Phonic Tic Score is derived by adding the five items pertaining to phonic tics (range 0–25); the TTSS (range 0–50) is derived by adding the Total Motor Tic Score and the Total Phonic Tic Score.
The YGTSS takes between 15 and 35 minutes to administer.
All outcome assessors underwent training alongside 6-month rater-agreement checks in the YGTSS (see Appendix 1, Table 31).
Secondary outcome measures
Yale Global Tic Severity Scale: impairment scale4
The impairment scale forms one of the four index YGTSS scores described above. The impairment rating is on a 50-point scale ranging from 0 (no impairment) to 50 (severe impairment). The rating focuses on distress and impairment experienced in interpersonal, academic and occupational realms.
Parent tic questionnaire64
The parent tic questionnaire (PTQ) assesses the number, frequency and intensity of motor and vocal tics. Frequency ratings are made on a 1–4 scale (constantly, hourly, daily and weekly) and intensity ratings are made on a 1–4 scale. A separate score for each tic is calculated by adding the frequency and intensity ratings, giving a score ranging from 0 to 8. Motor and vocal tic severity scores are computed by summing the scores for all motor and vocal tics respectively and a severity score computed by summing the two sub-scores. The PTQ was completed by parents/carers at all measurement time points (baseline, mid-treatment, 3-, 6-, 12- and 18-month follow-up) via the online web-based data platform.
Clinical Global Impressions-Improvement Scale65
The Clinical Global Impressions-Improvement Scale (CGI-I) provides an overall clinician-determined summary measure that takes into account all available information to determine improvement since initiation of the intervention. The CGI-I consists of one item scored on a seven-point scale from 1 (very much improved) to 7 (very much worse). The measure was completed online via the online data platform at each follow-up time point after treatment (3, 6, 12, 18 months) by the outcome assessor who completed the YGTSS.
Children’s Global Assessment Scale66
The Children’s Global Assessment Scale (CGAS) is a 0–100 scale that integrates psychological, social and academic functioning in children as a measure of psychiatric disturbance. Scores above 70 indicate functioning in a normal range. The CGAS was completed by the researcher who completes the YGTSS via the online data platform at baseline and at 3, 6, 12 and 18 months of follow-up.
Strengths and difficulties questionnaire (parent completed)67
The strengths and difficulties questionnaire (SDQ) is a brief measure of behavioural and emotional difficulties. The SDQ consists of 25 items that are rated on a three-point Likert scale (not true, somewhat true and certainly true). The items are designed to be divided between five sub-scales, each consisting of five items, which can be used to create scores for emotional symptoms, conduct problems, hyperactivity–inattention, peer problems and pro-social behaviour. The items for all but pro-social behaviours can be summed to generate a ‘total difficulties score’.68 The standard SDQ can be supplemented with a brief impact supplement that assesses the impact of the child’s difficulties in terms of distress, social impairment, burden and chronicity.67 The SDQ was completed by parents/carers as part of the DAWBA at baseline and via the online data platform at 3, 6, 12 and 18 months of follow-up.
The mood and feelings questionnaire (child-completed version)69
The moods and feelings questionnaire (MFQ) is a 33-item questionnaire designed to report depressive symptoms. The items are rated on a three-point scale (0 = not true, 1 = sometimes, 2 = true). The MFQ is scored by summing together the values for each item. The MFQ was completed by the child/young person at each time point (baseline, mid-treatment, 3, 6, 12 and 18 months of follow-up) via the online data platform. The measure was used to check for side effects as well as outcomes in this trial.
Spence Child Anxiety Scale (self-report)70
The Spence Child Anxiety Scale (SCAS) is a child self-report measure designed to evaluate symptoms relating to anxiety. The SCAS consists of 44 items. Children are asked to rate on a four-point scale (0 = never, 1 = sometimes, 2 = often, 3 = always) the frequency with which they experience each symptom. The ratings are summed from the 38 anxiety items to provide a total score (maximum = 114), with high scores reflecting greater anxiety. The SCAS was completed by the child/young person at baseline and at 3, 6, 12 and 18 months of follow-up via the online data platform.
Child health utility 9D (parent- and child-completed versions)71
The CHU9D is a paediatric QoL measure for use in health-care resource-allocation decision-making. The questionnaire consists of nine items, each with a five-level response category. There are two versions of the questionnaire: a self-report measure and a proxy measure, which can be completed by the parent/carer of the child. The CHU9D was completed at baseline and at 3, 6, 12 and 18 months of follow-up via the Swedish BASS data platform (online) by the parent/carer and the child/young person.
The Child and Adolescent Gilles de la Tourette Syndrome Quality of Life Scale72
The Child and Adolescent Gilles de la Tourette Syndrome Quality of Life Scale (CandA-GTS-QoL) is a disease-specific measure of health-related QoL (HRQoL) designed for children and adolescents with TS. There are two versions of the measure: one for children aged 6–12 years and one for young people aged 13–18 years. The questionnaire consists of 27 items, each with a five-level response category. The CandA-GTS-QoL was completed at baseline and at 3, 6, 12 and 18 months of follow-up via the online data platform by the child/young person.
Client service receipt inventory73
The client service receipt inventory (CSRI) is a flexible research instrument developed to collect information on service receipt, service-related issues and income. The questions of the CSRI are largely structured in a multiple-choice format, but, to contend with the complexity of community care arrangements, a few open-ended questions are also asked. The measure also asks about school attendance since the last measure completion time point (3/6 months). A modified version of the CSRI was to be completed at baseline and at 3, 6, 12 and 18 months of follow-up via the BASS platform (online). This modified version also combines elements of the child and adolescent service use schedule (CA-SUS).74 The questionnaire takes < 15 minutes to complete.
Adverse events/side effects
Adverse events/side effects were recorded on a modified version of the side effects scale developed by Hill and Taylor.75 The scale consists of 17 short items relating to common side effects (such as headaches, anxiety, sleep and low mood). The participant is asked to respond on a five-point scale ranging from ‘not at all’ to ‘all the time’ to describe the presence of each item. The scale was completed at baseline (to ascertain the presence of these symptoms prior to treatment), mid-treatment and at 3 and 6 months of follow-up via the BASS platform (online) by the parent/carer, with input from the child/young person.
Treatment credibility
To assess treatment credibility, we administered a short questionnaire created by the research team. The questionnaire consisted of two items that are scored on a five-point scale. The questionnaire asks how well the internet treatment suits children for managing tics and how much improvement they expect from the treatment. The questionnaire was completed 3 weeks into the treatment by parents/carers and CYP via the online data platform.
Treatment satisfaction and need for further treatment
To assess treatment satisfaction, the research team created a brief questionnaire consisting of seven items. Six of the seven items are scored on a five-point scale and ask how helpful the treatment was and whether the participant would recommend it to others. The seventh item is a three-choice option asking whether the families would prefer face-to-face treatment, had no preference or would prefer internet treatment. The need for further treatment questionnaire consists of one item that asks the CYP and parents/carers and to rate whether they feel they/their child needs more treatment for their tics. This is rated on a five-point scale ranging from ‘I/my child doesn’t need any more treatment’ to ‘I/my child needs a lot more treatment’. Both the satisfaction and need for further treatment questionnaires were completed online at the 3-month follow-up point by parents/carers and CYP.
Concomitant interventions
Parents completed a short questionnaire that asked about other treatments/interventions/medications in progress. This was completed at baseline and then again at 3, 6, 12 and 18 months of follow-up with the researcher via online videoconferencing/telephone.
Screening measures
Two measures were used as screening measures.
Development and well-being assessment
The DAWBA is a package of interviews and questionnaires completed by parents and teachers and designed to generate International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) and Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV)/Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) psychiatric diagnoses for CYP. The DAWBA computer algorithm estimates the probability of having a psychiatric disorder in bands of < 0.1%, 0.5%, 3%, 15%, 50% and > 70% based on large community-based population studies.62 DAWBA was used to exclude people who are rated as being likely (50–70%) to experience self-harm, psychosis and anorexia nervosa or suicidality. If the DAWBA indicated a high likelihood of suicidality, the participant’s GP or usual treating clinician would be informed.
Child and adolescent intellectual disability screening questionnaire76
The CAIDS-Q determines the presence of intellectual disability and was used at baseline only. The questionnaire contains seven items answered in a yes/no format by someone who knows the person well. A total score is calculated and a cut-off (by age group) indicates whether the child is likely to have an intellectual disability or not. The score can also be used as a proxy for IQ in situations where only an approximate indication of intellectual ability is needed, and it was used to exclude participants who were likely to have an intellectual disability.
Sample size
To detect a clinically important average difference of 0.5 of a standard deviation (SD) between intervention and comparator with 90% power at p < 0.05 (two-sided), after making an allowance of 20% for dropout, requires a total sample size of 220 participants. Our systematic review77 found the average estimate for the SD of the YGTSS-TTSS from 19 trials of behavioural intervention for tics was 6.6. Thus, the trial was powered to detect an average change of 3.3 on the YGTSS, which was sufficient to ensure the risk of missing a clinically significant effect in the trial was low.
Analysis
Statistical analyses were conducted using Stata® (version 16; StataCorp LP, College Station, TX, USA) in line with a predefined statistical analysis plan (SAP) approved by the TSC. The SAP stated that confidence intervals (CIs) rather than p-values were to be reported. Analysis was performed on a modified intention-to-treat basis, in which participants were analysed according to their allocated group for cases where data were available. Baseline demographic characteristics of participants, as well as their clinical and mental health outcomes at baseline and at 3 and 6 months of follow-up, were summarised by randomised group using mean (SD) or count (percentage), respectively, for continuous and categorical data.
Phase 1 analysis
This analysis has been described and published; sections have been reproduced from the Hollis et al.78 publication under the CC-BY 4.0 Licence.
The primary outcome was estimated using a linear regression model with YGTSS-TTSS at 3 months as the outcome and study group as the main explanatory variable, adjusting for YGTSS-TTSS at baseline and site (Nottingham/London).
Linear regression models were also fitted to estimate the effect of the intervention on secondary outcomes at mid-treatment and at 3 and 6 months of follow-up (post-randomisation). The statistical model for the CGI-I did not adjust for baseline as this is a measure of change. Using CGI-I to indicate response to treatment, the scale was dichotomised to define response as ‘improved’ or ‘much improved’ versus non-response as ‘minimally improved’, ‘stayed the same’, ‘worse’ or ‘very much worse’. Two unplanned subgroup analyses explored whether the effect of the intervention on the primary outcome was modified by either anxiety diagnosis or ADHD diagnosis. The statistical models were the same as for the main analysis of the primary outcome, with the addition of a fixed effect of the comorbidity (anxiety or ADHD) and an interaction between the comorbidity and the study arm. All statistical analyses were conducted on complete cases. This analysis has been described in Hollis et al.78
Phase 2 analysis
For the phase 2 follow-up, a sample power calculation was not specified. Outcomes at 12 and 18 months were summarised by randomised group, for continuous data using mean (SD) or for categorical data using count (percentage). A single linear mixed model was fitted for each outcome, with measures from all available time points (at mid-treatment and at 3, 6, 12 and 18 months of follow-up) as the repeated-measures outcome and a random effect of participant to account for correlations between the repeated measures on each individual at different time points. The main explanatory variables were treatment, time and the treatment by time interaction, adjusting for site and the baseline measure of the outcome. Since correlations are commonly smaller over longer time periods, we adjusted for baseline through an interaction with time, which allowed correlations with baseline to differ between follow-up times. The effect of the intervention at 12 and 18 months was estimated from this model.
As with the phase 1 analysis, the statistical model for CGI-I did not adjust for baseline as this is a measure of change. Response to treatment was compared between study arms using separate logistic regression models at 12 and 18 months, adjusting for site. Estimated effects are reported with 95% CIs.
We summarised changes in other (non-trial) tic treatments (medication or therapy) between 6 and 12 months and between 6 and 18 months by study arm and for the sample overall using counts (N) and percentages (%). The phase 2 analysis is currently under peer review with the Journal of Child Psychology and Psychiatry.79