Economic evaluation

Introduction

This chapter presents an exploratory economic assessment of the cluster-randomised Stand Out in Class pilot study, addressing objective 12. The health economic evaluation of the Stand Out in Class study represents a particularly valuable contribution to sedentary behaviour research. During a scoping review, we found few intervention studies that explicitly considered the magnitude of the potential health effect of observed changes in sedentary behaviour. Fewer still examined any potential resulting change in cost of health-care resource use.

Given interest in this area as a potential public health intervention, and tightly constrained budgets across the health and education sectors, it is important to maintain an explicit consideration of cost-effectiveness in future work of this nature. Health economic models in this area are underdeveloped and would benefit from refinement to closer reflect the current evidence base of health effects of physical (in)activity. This would enable better representation of the potential health opportunity cost of investments to reduce sedentary behaviour rather than other possible public health investments.120

Introduction to methods

The economic analysis is split into two main sections:

1. A within-trial analysis of health and education resource use, and summary measures of health and education effects, collected at baseline and at the end of the trial follow-up period. This is presented as a cost–consequence analysis.
2. An extrapolated trial analysis modelling the cost and quality-of-life impact over the long term. This is presented as an indicative cost–utility analysis. An expected value of perfect information (EVPI) analysis is also presented.

Value of information analysis presents an upper estimate of the monetary value of further research into resolving all decision uncertainty of the cost-effectiveness of the current ‘Stand Out in Class’ intervention at varying thresholds, given the changes in physical activity and sedentary behaviour observed in the pilot study. EVPI estimates capture the value of both further research into the effectiveness of this intervention at reducing sedentary time, and the associated health effect of any change in physical activity and/or reduction in sedentary time.

All analyses are presented from a health system and Personal Social Services perspective. A limited range of education outcomes are, however, included in the within-trial analysis. Within-trial analysis presents a cost–consequence analysis of reported health and education outcomes over the trial time horizon (4 months). It was not possible to conduct a cost–utility analysis for within-trial health outcomes. Although the EQ-5D-Y106 and PEDS-QL105 were both collected at baseline and at follow-up, to date these outcome measures have no valuation tariff for valuing surveyed health states into quality-adjusted life-years (QALYs). All self-report measures used to inform these analyses are described in detail in Chapter 2. The education outcomes collected are also limited, as ‘education resource use’ and ‘strengths and difficulties’ questionnaires lack external validity in predicting relevant educational attainment outcomes, such as the outcome of national curriculum assessments.

Extrapolated trial analysis aims to identify the magnitude of potential causal links between changes in sedentary time and/or physical activity observed, and long-term health outcomes. Public health interest in this trial area is driven by the expectation that reducing the level of sedentary behaviour in a cohort of children is important for ongoing health outcomes, including consideration of a residual health effect beyond the scope of the intervention itself.

General methods

Data sources

Impact of intervention

Within-trial analysis utilised three key data sources: (1) health-care resource use was measured by parent-reported survey, which detailed use of a range of common health services over the previous 3 months; (2) health-related quality of life was measured by the EQ-5D-Y (including visual analogue score) and PEDS-QL; and (3) an educational resource use and educational outcome questionnaire relating to pupils’ behaviour (Strengths and Difficulties Questionnaire102) was completed for each pupil by teacher-reported survey. Baseline measurements were taken in January/February 2017 and post-intervention assessments were completed in July 2017.

Parent reports of health-care contacts were combined with closest matching unit costs. Reference costs for non-hospital care were obtained from the latest Personal Social Services Research Unit (PSSRU) report121 in which resource costs for identical or most similar items were available. Hospital-related incidents were costed using the 2016 NHS schedule of reference costs.122 Data sources for each health-care resource use item are presented online in Report Supplementary Material 2. When relevant, costs were inflated to 2016 pounds using the health and social care factor (Hospital and Community Health Service) reported in PSSRU 2016.

Extrapolated trial analysis utilised the results of surveyed levels of physical activity in the intervention and control groups, measured by the ActiGraph accelerometer. Although the Stand Out in Class study primarily targeted classroom sitting time, there is currently very limited evidence available for the effect of classroom sitting time and other sedentary behaviours on health costs and health-related quality of life, both of which are necessary for health economic modelling (see Scoping review). Physical activity levels (referred to as ‘activity’ throughout this chapter) were therefore measured in METs,123 when health economic evidence was available. The difference in measured activity between the intervention and control groups was calculated and used in long-term disease incidence modelling. Changes in disease incidence resulted in changes in projected health-related costs and quality of life, which informed long-term cost-effectiveness estimates.

Education outcomes are also presented in the economic within-trial analysis. There was insufficient trial reporting on relevant standardised education outcomes to inform detailed evaluation. Although 9- to 10-year-olds in England follow a standardised curriculum, progress tests are not uniform between schools, or even necessarily valid for time point comparisons within schools. The main education outcome considered was therefore the change in reported behavioural difficulties on a teacher-reported Strengths and Difficulties Questionnaire. A cognitive function test was performed; however, individual results were not recorded in the case report form.

Intervention costs

The trial protocol allowed for six desks in each intervention school, allocated between a total class size of 30 children. In each intervention school, access to standing desks was granted to the full class, whether children were enrolled in the trial or not. As class sizes were at full capacity of 30 pupils in all but one school (n = 27), exposure to the intervention was approximately equal if the trial protocol was followed in each school.

In the intervention arm, standing desks were assumed to be an upfront cost split between the current full cohort of 30 pupils and nine future full cohorts of 30 pupils. These future cohorts represent future beneficiaries of the capital assets over an expected 10-year lifespan. Manufacturer correspondence (Carrie Schmitz, Ergotron Inc., 2018, personal communication) advised the best available price for the desks provided in-kind to the trial (Ergotron LearnFit) was £379 per unit, with no discount available on the trial cohort of 24 desks.

The intervention cost, per school, of six standing desks used in the trial was therefore costed as £2274. No financing arrangements were assumed, nor any setup, future service or maintenance costs to the intervention schools, with the manufacturer-supplied 10-year warranty assumed to be comprehensive and costless at point of claim. These assumptions were deemed appropriate for the relatively small number of desks provided to each school; however, any financing cost and opportunity cost of time and labour, for example, to set up standing desks on a greater scale, such as across a full school, would need to be accounted for in a larger intervention.

The intervention cost per pupil over 10 years for a full class size of 30 pupils was calculated to be £7.58. This assumes no opportunity cost of upfront capital investment (e.g. lost bank interest on cash over the 9 additional cohort years). The assumption of no opportunity cost of capital over 10 years would require re-examination for a larger-scale intervention. In the control arm, no environmental changes were made and, thus, no desk-related costs were assumed to be accrued over the course of the trial.

Missing data

Missing data are a concern in cost-effectiveness analysis because costs or health outcomes in individuals with missing data may be systematically different from those with fully observed information. Inappropriate missing data strategies have the potential to bias resource allocation decisions.124 We followed a recent presentation framework in setting out our missing data strategy below (see Missing data strategy).125

Description of missing data

Patterns of missingness in data sources used in economic within-trial analysis are detailed online in Report Supplementary Material 2. Participant responses were regarded as ‘complete’ if all items were completed unambiguously. ‘Incomplete’ responses included both partially complete and fully incomplete responses. When responses indicated a range of values, the mean was used. Remaining ambiguous entries were removed and treated as null entries.

The missing data mechanism was primarily participant non-response, meaning that all resource items were incomplete (e.g. due to failure to return a completed survey). Only six respondents were completely lost to follow-up, with no response to any outcome measure (four in the control arm and two in the intervention arm). There was also a small amount of item non-response for individual questions within otherwise complete outcome measures, particularly for the parent-reported health-care resource use. Missing data are non-monotone, with only 33 of 176 (18.8%) respondents having complete data in outcomes of interest in all six survey categories (four outcome measurements, two resource use measurements) at baseline and at follow-up. Thirty-three per cent of respondents had missing data in only one outcome at one of the two collection time points.

Missing data patterns indicated some variation in missingness by trial group, with 23.3% of respondents having no missing data for all main outcome variables in the intervention arm, compared with 14.4% of respondents in the control arm. There was also some indication of variation in missingness patterns by ethnicity, with 54.46% of non-white respondents having missing data in main outcomes for up to only one time point, compared with 48% of white respondents. Missing data were most frequent in the parent-reported health-care resource use survey (44.3%, aggregated across time points) and least frequent in education resource use (0%) and EQ-5D-Y profiles (5.4%, aggregated time points).

For health-care resource use, missingness at baseline appeared a strong predictor of missingness at follow-up (p = 0.01); however, there was strong clustering of missingness at the school level. Accounting for clustering reduced the significance of the null hypothesis of no association between time points (p = 0.11). There was also quite strong evidence for association of missingness between time points in the activity data (p = 0.04) (measured using the ActiGraph), although there was also clustering of completeness at the school level. Accounting for clustering, there was some evidence for rejecting the null hypothesis of no association in missingness in activity levels between time points (p = 0.12). In other outcome categories, there was no evidence of association in missingness between time points.

There was no evidence of association of missingness between outcome types. In the outcome categories with the most missing data, health-care resource use and accelerometer-measured activity, there was only very weak evidence to reject the null hypothesis of no association (p = 0.81 and p = 0.79 adjusted for clustering).

Missing data strategy

Our missing data strategy consisted of two analyses:

1. a multiply imputed analysis, using multiply imputed data sets generated by predictive mean matching from nearest neighbours
2. a complete-case analysis, including only participants with complete responses in each survey type at baseline and follow-up.

Missing data were multiply imputed assuming missingness at random in the base case, with results compared with the complete-case analysis (reported online in Report Supplementary Material 2). Sensitivity to the assumption of missingness at random was not otherwise investigated and testing robustness of conclusions to plausible departures from the ‘missingness at random’ assumption should form a part of future work. Conditional on the observed data, the probability of cost-effectiveness data being missing may still depend on the underlying unobserved values (i.e. data may be ‘missing not at random’).126 The missing data strategy makes no distinction between within-trial and extrapolated trial outcomes. All relevant outcomes were combined into one data set, allowing utilisation of complete outcomes to inform other, incomplete outcomes.

Multiple imputation methods

Missing data were multiply imputed by chained equations using the mi impute command in Stata^® 15 (StataCorp LP, College Station, TX, USA). Nearest neighbour matching with replacement using the 10 nearest neighbours was used to help protect against misspecification of regression equations.127 Fifty multiply imputed data sets were generated to minimise Monte Carlo error, approximately matching the percentage of missing data in the outcome with the most missing data (health-care resource use).128 Each imputation had a burn-in of 50 estimates to help the imputation model achieve stable estimates.

In estimating health-care resource use, it was not possible to multiply impute health-care items individually, due to non-convergence. For this reason, resource use was categorised into four broad types: (1) primary care, (2) hospital, (3) mental health and (4) community care. Categorisation of individual resource use items into broad categories is presented in Table 17. Resource use was imputed on a cost scale, after prior calculation of costs from individual items, as there was large variation between costs of individual types of resource use within item categories. Imputing directly on the cost of resource use was therefore more likely to reflect uncertainty in resource cost in each category, rather than trying to later recreate cost outcomes from the predicted number of appointments in each category.

Multiple imputation estimates were conducted simultaneously by chained equations and included intervention group, school number, as well as participants’ age, sex and ethnicity (16 possible categories) as covariates. A random seed was specified to aid replicability of analysis, with the imputation trace saved to perform post-imputation diagnostics to establish convergence performance. Anthropometric measurements regarded as potential predictors of outcome variables were included in chained equations. These included BMI, fat mass and average blood pressure measurements at baseline and at follow-up.

Multiple imputation used a single-level linear model with no interactions, accounting for clustering at the school level only, using an indicator variable, rather than using multilevel modelling as advised best practice.129 Single-level (fixed-effect) modelling accounts for some between-cluster variation, explained by cluster-level predictors; however, it is unlikely to explain all between-cluster heterogeneity. Multilevel (random-effects) imputation was not conducted owing to perceived difficulty in implementation, given the small numbers of individuals in each cluster, particularly within the time constraints of the research. Small numbers within each cluster also prevented model fitting of a single-level sensitivity analysis – in imputing data separately for each cluster, which would allow the distributions of imputed values to differ among clusters. Comparable single-level imputation models would not converge using the standard maximum likelihood approach.130 Future work that investigates the sensitivity of our conclusions to a multilevel imputation strategy would be a valuable contribution to understanding the intervention effect studied in this trial.

The total cost of health-care resource use across all categories and activity values aggregated to a simulated year (accounting for school holidays) were registered as passive outcomes and reconstructed post imputation from imputation estimates of non-aggregated values. EuroQol-5 Dimensions (EQ-5D) responses were imputed, as other variables using predictive mean matching, at the index level. It was not possible to conduct imputation on individual dimensions using ordered logit or probit regression, nor EQ-5D tariff-valued scores, as per best practice.131 As a result, individual dimensions were reconstructed from the imputed index for each imputation.

Post-imputation estimation

Post-imputation estimation was performed using the Stata command mi estimate to implement Rubin’s rules, adjusting for clustering at the school level.132 This method was used for all outcome variables, apart from the EQ-5D. The clustering method ‘vce (cluster)’ assumes that observations are independent across groups (in this case, at the school level), but not necessarily within groups. Observations were not clustered by observations across time points within individuals. The mi estimate command also accounts for between-imputation uncertainty in estimation of standard errors and, thus, presents wider Cls than standard analysis, which assumes that the value for each observation is certain.

However, reconstructed EQ-5D dimension scores across 50 imputations were combined using their mode value for each individual. This approach therefore does not fully represent the degree of uncertainty introduced by variation between imputations. However, this method was judged the best approach to ensure eradication of nonsense non-integer values introduced by post-imputation estimation commands and also judged most appropriate for the recommended method of presentation of non-tariff-valued EQ-5D profiles.133

Within-trial analysis

Methods

Cost–consequence analysis

Costs and consequences across health and education sectors are presented in the intervention and control groups at follow-up. It was not possible to conduct a full cost–utility analysis on data collected in health, as youth tariffs to convert health profiles to QALY weightings are not currently available.133 Best practice indicates not to use adult weightings to value children’s health state measures.134 In education, there is currently no known available cost-effectiveness framework outside cost–consequence analysis.

Base-case analysis

All base-case analysis was performed on multiply imputed data sets. A complete-case analysis is presented in scenario analysis, reported online in Report Supplementary Material 2. We compiled simple unadjusted aggregate outcomes for health-related costs and consequences in health and education. We utilised an intervention cost, as set out in Data sources, Intervention costs, of £7.58 per child. A comparison of outcomes at baseline and follow-up is presented. As previously mentioned, no valuation tariffs were available for calculation of QALYs in the trial groups and, for this reason, no incremental cost-effectiveness ratios (ICERs) could be calculated. We did not conduct any regression analysis of within-trial outcomes, nor attempt to stratify results by predefined subgroups, such as ethnicity.

Health-care resource use

The number of health-care contacts by type are presented from the results of complete-case analysis. This is inconsistent with presentation of other within-trial results. Imputation methods used did not directly estimate the number of health-care contacts in patients with missing data. The number of patients with complete data in all categories are presented in each group. Each inpatient day was classified as a separate health-care event. Estimated health-care costs during the survey period were calculated for all patients from reported usage. Mean patient costs in four health categories are presented at baseline and at follow-up for the intervention and control groups.

Education resource use

Education resource use by teacher-reported survey is reported at each time point. Education outcomes are not monetised.

Health outcomes

The EQ-5D-Y and PEDS-QL health profiles are presented at baseline and at follow-up. The percentage of pupils in each EQ-5D health state is reported. PEDS-QL mean scores in each dimension are reported. Presentation of PEDS-QL scoring follows the survey provider’s guidance.135 PEDS-QL scores are scaled 0–100, with 0 being worst and 100 being best (therefore higher scores represent better health-related quality of life). Valuation of health outcomes into QALYs from EQ-5D-Y profiles was not possible as there were no valid available preference-based tariffs.134 Use of a mapping algorithm for PEDS-QL was considered; however, mapping methodology relied on stable population preference-based valuation of EQ-5D-Y tariff.136

Education outcomes

Strengths and Difficulties Questionnaire scores were categorised into four bands, as recommended by the survey developer.137 The percentage of pupils in each category is reported at baseline and at follow-up, with ‘normal’ representing the best categorisation and ‘very high’ representing children with the most difficulties.

Results

Table 17 summarises resource use categorisation and corresponding reference costs, per resource use, collected from PSSRU121 and NHS Reference Costs 2015–16.122 For survey items for which a recent reference cost had not been calculated, a close comparator was used for costing. This was applicable for only one resource item, where the survey item ‘other counsellor/therapist’ was assumed to be comparable to the cost of a GP counsellor. Estimating the cost of some other resource items required further assumptions, such as resource items requiring home visits, which involved assumption of the amount and cost of travel time. Assumptions, when made, were kept consistent between closely comparable resource items. Full details of the assumptions for each item are presented in Report Supplementary Material 2 alongside their source.

Recent reference costs were readily available for survey items with the highest reported use, such as GP surgery visits. In less frequently utilised services, costs were less readily available in the literature. For this reason, given relatively low reported use, the impact of assumptions to approximate health-care resource costs in categories where these costs were not directly available is expected to be minimal. Inpatient stays were costed as an average of hospital costs per day, rather than by individual Healthcare Resource Group coding for any procedure named in complete cases. This was expected to limit unexplainable variation in imputed hospital costs due to different Healthcare Resource Group costs of conditions treated in each arm by inpatient stay recorded in complete cases.

Costs are presented in 2016 Great British pounds. When cost estimates were gathered from historical sources, they were inflated to 2016 prices using the Hospital and Community Health Service Index reported by the PSSRU.121 Total resource costs and outcomes in health and education are presented in Table 18.

The number of health-care contacts in complete cases in each trial arm appears to vary. This is largely driven by a small number of high-use patients. In the control arm at follow-up, 11 of 15 instances of ‘psychological care’ are for one pupil who reported weekly counselling appointments. Similarly, in the intervention arm at follow-up, 15 of 17 ‘hospital care’ contacts were for a pupil who received surgery for a burst appendix and therefore required an extended inpatient stay.

In the full base-case cohort, mean costs of hospital care in the intervention arm appear large; however, this was, in part, driven by the previously described surgery for one pupil, costed at £5836.50. Overall, there appears to be little interpretable difference in estimated health-care costs in the prior 3 months between trial arms. The mean cost of the intervention per pupil was comparatively small, and if costed over the same 3-month time horizon, rather than a full year, the mean cost of the intervention during the trial period reduces from £7.58 to £1.90 per pupil.

Education resource use across all categories appears slightly higher in the intervention group than in the control group at follow-up. However, when baseline patterns of resource use are considered, the level of education resource use in the intervention arm appears stable between the two time points, whereas resource use increases in the control arm between summer and winter. Education outcomes collected are, however, difficult to interpret. For example, interpretation of an increase in pupils receiving additional tutoring in the control arm between time points is highly context dependent. This could be due to a change in teaching style within the school, or by revision of expectations by parents. Again, there appears to be little interpretable evidence of divergence in resource use between the intervention and control arms during the trial period that could be attributed to the intervention.

Comparing EQ-5D-Y profiles at follow-up suggests that health outcomes also diverge little between groups. The largest difference was a 6% difference in report of pupils experiencing some pain and discomfort, with a higher number reporting ‘no problems’ in the intervention arm. This corresponds roughly to a difference of five pupils. There were, however, 4% more pupils reporting some anxiety in the intervention arm than in the control arm and a similar 4% increase in reported problems in self-care in the same group.

When assessing EQ-5D-Y profiles between the two time points, a more significant trend emerges of changes in the characteristics of reported health profiles between baseline and follow-up. In both trial arms, the percentage of pupils reporting ‘no difficulties’ in mobility, self-care and pain/discomfort decreases between baseline and follow-up, meaning a higher number of pupils report at least some difficulties in these dimensions. Both trial arms report an increase in mobility problems between baseline and follow-up, a change of 6% in the control arm and 8% in the intervention arm.

The EQ-5D-Y profiles reveal some further divergence between trial arms. Although in the control arm there is an 11% increase in the number of pupils reporting at least some pain, in the intervention arm there is an increase in reported pain of only 1% between time points. In the ‘usual activities’ dimension, there is a 4% increase in reported problems in the intervention group between time points, whereas there is a simultaneous 4% decline in reported problems in the control arm. Reported anxiety simultaneously increases by 3% in the intervention arm, but declines by 1% in the control arm.

The PEDS-QL dimensions follow a similar pattern to EQ-5D-Y outcomes. Health profiles are comparable at follow-up; however, observed differences between groups in health profiles at baseline mean that pupils in the two trial groups have experienced some difference in trends between time points. Both groups report a decline in ‘school function’ between time points, with a 6.98-point decline in the control arm and a 2.08-point decline in the intervention arm. ‘Emotional function’ decreases by 2.82 points in the control arm, but improves by 2.4 points in the intervention arm. There is greater similarity in trends in other dimensions, with both trial arms reporting a decline in physical function between time points (slightly larger in the control group), and small changes in social function showing a slight decrease in the control arm compared with a slight improvement in the intervention arm.

Comparing education outcomes at follow-up, teacher-reported Strengths and Difficulties Questionnaires reveal a greater number of more severe reported difficulties in the intervention arm than in the control arm. There is a 5% difference in reported difficulties, which are distributed among the two highest categorisations of pupil difficulties. When accounting for baseline values, however, an improvement in strengths and difficulties profiles is apparent in both arms, with a much stronger difference observed in the intervention arm. In the intervention group, there is a 7% improvement in the number of pupils categorised within normal ranges by teachers. There is an apparent improvement in reducing reports of both higher- and lower-severity difficulties. In the control arm, there is a 3% improvement in aggregated difficulties profiles. This is due to fewer pupils being classified as ‘high’ difficulty; however, there is a corresponding increase in the number of pupils with difficulties profiles still classified outside a ‘normal’ range. The results of complete-case analysis are presented in scenario analysis (see Report Supplementary Material 2).

Discussion

Main findings

Within-trial analysis reveals no substantial difference in outcomes between trial arms. Suggestive patterns in some outcomes do not appear to sum to a coherent picture across all outcomes describing any clear distinction between trial groups. Overinterpretation of small numbers is a significant risk, with this study underpowered to detect differences in outcomes with a high level of sampling variability.

There is some evidence of differences between the trial arms in health and education outcomes, but less evidence of differences in estimated resource use. Difference in trend was most apparent in PEDS-QL scores and in ‘strengths and difficulties’ profiles, in which there was some evidence to suggest a slight improvement in measured outcomes between time points in the intervention arm compared with the control arm. However, little is known about the sensitivity or external validity of these measures to influence a judgement of the relative importance of sample-measured improvement on these outcomes.

There was no clear difference in EQ-5D-Y profiles between groups. There were changes in reported anxiety and levels of pain and discomfort between time points, which could be interpreted to favour the trend in the intervention arm; however, the balance of other outcomes appears to either favour the control arm or show no difference in the aggregated health profile. It is especially difficult to interpret the relevance of any change in the population-aggregated EQ-5D-Y profile without a valuation tariff to provide generalisation to a measure of average health-related quality of life in each group by an aggregate quality-of-life measure, such as the QALY. Without valuation, it is not easy to establish the relative weight to attribute to any improvement in one domain compared with relative declines in another area. Changes in an individual’s measured health-related quality of life depend on reported values for that individual across all dimensions, rather than one dimension alone; thus, it is particularly difficult to interpret any obvious trend in individual health outcomes from a population-aggregated EQ-5D-Y health profile.

In health-care resource use, although survey-reported estimates suggest a higher cost of total resource use in the intervention arm than in the control arm, this is largely attributable to a single high-cost participant for an apparently unrelated inpatient stay. All cost categories show a high degree of variation, with a potential problem in attribution to the intervention. Any change in health-care resource use resulting from the Stand Out in Class intervention is likely to follow a meandering causal pathway, probably over a long period, and therefore will require a much larger sample size to measure and attribute any difference between trial arms at an appropriate degree of accuracy.

No obvious pattern emerges from the within-trial pilot survey outcomes. Sampling variability was heightened by differential patterns of outcome completeness in health-care resource use between clustered sites. In education resource use, it is similarly difficult to interpret changes in resource use not only due to previously mentioned difficulty in interpretation of the outcomes collected, but also due to little understanding of the sensitivity, stability across time points and external validity of outcomes collected. A larger trial powered to detect significant difference may resolve parameter uncertainty; however, thought should also be given to ensure that outcomes collected in any future trial have a ready, intuitive interpretation.

Impact of scenario analysis

Complete-case analysis reveals a similar picture to the base case, with the largest divergence from the multiply imputed base case in outcomes with highest levels of incompleteness. This meant that although health and education outcomes and education resource use were broadly similar, there was some observed difference in the estimated mean cost of health-care resource use in the complete-case analysis. Complete-case analysis suggested a higher mean cost of health-care resource use in both arms, suggesting some difference in patterns of missing resource information accounted for in regression adjustment on observable characteristics in multiple imputation.

The difference in mean health-care resource use between scenarios was greatest at follow-up. Mean resource use was 9% higher, on average, in complete-case analysis than the multiply imputed base case. The difference was particularly large for the control arm at follow-up (complete-case health-care resource use mean costs were 14% higher) and for the intervention arm at baseline (complete-case health-care resource use mean costs were 11% higher than base-case estimates). For this reason, based on multiply imputed estimates, it appears that high-cost patient subgroups are over-represented in complete-case survey responses of health-care resource use, which is probably only partially corrected by imputation on observed characteristics in base-case analysis. Patterns of resource use in each arm hold roughly consistent proportions between scenarios, but with similarly little distinctive pattern in resource use in complete-case analysis across health cost categories.

Comparison with other studies

Limited randomised research has examined the impact of sedentary behaviour interventions aiming to create an active classroom in primary schools.56,69 This study adds to existing literature by examining resource use outcomes, as well as a limited range of health and education outcomes. In particular, information regarding health-care resource use during observation periods has not been found in previous known studies of active learning interventions. The study shows that it is feasible to collect outcomes of interest for an economic evaluation. However, evidence gathered remains small scale and over a short time period. Our study does not demonstrate significant new information than previous studies, which have already demonstrated the feasibility of incorporating sit–stand desks into the primary school environment in the short term.29,58,59

Our study conclusion of no measured difference in education resource use in the trial arm concurs with previous study results of no disruption caused by a sit–stand desk intervention to teaching, learning or behaviour.29 Education outcomes collected could not, however, be used to corroborate previously gathered evidence of improved levels of academic engagement or achievement.64,65 Economic analysis of surveyed health outcomes did show a slight improvement in trend in reported pain on EQ-5D-Y profiles, which could be interpreted to corroborate with previous study evidence demonstrating that sit–stand desks lead to improvements in posture and musculoskeletal comfort in children63,64 and adults.138 However, the origins of reported changes in pain profiles were not analysed in economic analysis. Observed changes in aggregate health profiles were difficult to interpret, with little convincing evidence of differences between arms on EQ-5D-Y profiles; however, some limited evidence of improvement on PEDS-QL-measured quality-of-life scores were seen. Poor balance on baseline characteristics between arms is likely to have impaired our ability to estimate the treatment effect accurately for all outcomes collected in this pilot study.

Strengths and limitations

Health-care resource use surveys were incomplete for a significant proportion of trial participants. Efforts should be made in any future trial analysis to make parental surveys easier to complete, and incentives to encourage responses should be considered. In any future trial, reporting of health-care resource use could be streamlined to minimise reporting and recall bias (e.g. allowing parents to record health-care incidents in real time using a trial-specific application or website). This would also allow closer analysis of the effect of clustering of health-care resource use by school, recording the exact timing of resource use.

It was not possible to calculate QALYs based on the quality-of-life surveys used. Although it is perhaps unlikely that quality-of-life difference would be observed during the short period of within-trial analysis, it is hoped that future work may be able to utilise forthcoming valuations of EQ-5D-Y measures to collate individual health profiles. Difference in average health-related quality of life may be observed over a longer time period, if possible, to calculate individual-level standardised quality-of-life values.

Educational outcomes utilisable as intermediate outcomes for extrapolation to final end points, such as national examination attainment, were absent from trial data collection. Education resource use and Strengths and Difficulties Questionnaires lack significant external validity for measures of interest to policy-makers in education. In consultation with head teachers and teachers/assessment co-ordinators via PPI, further consideration should be given to measuring academic progress to identify appropriate national curriculum-related or standard assessments used by schools, which will enable direct comparison between participating schools in a future trial. In the absence of consensus over such measures, proxies of academic progress, such as attendance and/or in-class behaviour data, which can be measured consistently across schools, should be identified in collaboration with head teachers and teachers. Future analysis should also consider how best to cost limited education resource use items collected.

Some residual imbalance between intervention and control groups, despite randomisation, may have biased the trial results presented, as there is some evidence of differences in key observable characteristics between trial arms at baseline. Outside imbalances mentioned directly in economic analysis, we draw attention to differences between trial arms described in previous chapters. There is an observed difference in school-level averages of English as first language (44.85% intervention arm, 52.9% control arm), reflecting differential ethnic composition, and also a difference in number of pupils in receipt of free school meals (27.1% intervention arm, 35.1% control arm). Both differences are likely to reflect an underlying difference between arms in socioethnic strata and reflect differential barriers to children’s participation in society. Perhaps more crucially, there were also variations in intervention implementation as described in Chapter 5, with only one of four schools implementing the Stand Out in Class intervention as per trial protocol.

The effect of these differences was not investigated formally, nor accounted for in the economic analysis presented. Results presented should therefore be viewed in the context of heterogeneity in baseline characteristics, and type and magnitude of intervention exposure.

Scoping review

Aims and objectives

A scoping review was carried out to identify candidate economic models or comprehensive meta-analyses evaluating the effect of childhood sedentary behaviour on a range of long-term health outcomes. Sedentary behaviour can be defined as time spent sitting, lying or reclining accompanied by low levels of energy expenditure (1.0–1.5 METs).1 Sedentary behaviours exist across various daily domains: in work, leisure and commuting.33,139,140 Some distinctions between types of sedentary behaviour go further, categorising sedentary behaviours as discretionary (such as watching television or reading) and non-discretionary (such as sitting at school or during a commute).141

Our primary research question was:

Are there existing economic or epidemiologic models that could be adapted for economic evaluation, which link childhood sedentary behaviour to expected short- and long-term health outcomes?

A secondary aim of our scoping review was to give an oversight of existing research into the link between sedentary behaviour and health outcomes to guide our methods of evaluation and highlight key issues in the field.

Our secondary research question was therefore:

What is known from the existing literature about the causal association between childhood sedentary behaviour and health outcomes?

In consultation with an information strategist, preliminary MEDLINE searches were carried out (see Search strategy). Only English-language publications were searched.

Search strategy

A comparison of initial search strategies is presented online in Report Supplementary Material 2. Search strategy 1 required article titles to contain ‘sedentary’ in the main title, whereas search strategy 2 allowed ‘sedentary lifestyle’ or variants to be contained in the main text of articles. Search strategy 1 identified 78 records, whereas strategy 2 identified 1540 records.

Abstracts from strategy 1 and a sample of 20 abstracts from strategy 2 were examined for relevance. Given the greater sample size of papers judged potentially relevant, search strategy 2 was selected, with the phrase ‘[review OR economic evaluation]’ added to the search terms to refine the search to more closely identify potential modelling studies. This search identified 208 papers, 182 of which were not duplicates.

All 182 abstracts were examined for relevance to the research question. Papers were examined in more detail if they were determined from the abstract to:

• examine associations with a quantified health effect (e.g. BMI, cognitive development, depression)
• consider a relevant population (i.e. aged < 16 years)
• use an objective measure of physical activity and/or sedentary behaviour
• include a study design that could indicate evidence of a causal association
• be a meta-analysis or systematic review, containing evidence regarding more than one health effect, or outcomes relevant for an economic evaluation (e.g. quality of life).

Articles were ranked according to these criteria to prioritise articles for review. There were 12 articles fulfilling four criteria and seven articles fulfilling all five criteria (one was at protocol stage). Results of the scoping review are shown in Report Supplementary Material 2.

Our scoping search revealed studies of varying quality, mostly in high-income countries; however, it also found some evidence gathered in a variety of low- and middle-income settings. The 33 papers examined highlighted some key themes within the field and identified further relevant references. It was apparent that many studies have been undertaken to ascertain population trends and general correlates of sedentary behaviour, as well as testing the effectiveness of interventions aimed to reduce sedentary behaviour. In this field, there was a good range of evidence type, including studies with defined control groups, some with randomised components. Health effects of sedentary interventions were, however, less often studied.

When examining sedentary time, proxies were often used, such as reported television viewing time, rather than objective measurement. The lack of objective measurement and uniform scale, and method of assessment, of sedentary behaviour made comparisons between studies difficult due to levels of heterogeneity in exposure, often hindering the usefulness of reviews or meta-analyses. The type of sedentary behaviour studied is important, with the possibility of health effects from television viewing, for example, to be mediated by unhealthy dietary patterns less common in people who spend more time in other sedentary behaviours (such as reading or studying).17,142 Comparatively few studies aimed to specifically track sedentary behaviours to short- or long-term health effects in children. In gathering evidence for short-term health effects, comparatively few analyses convincingly account for residual confounding of physical activity levels. Our search highlighted a wider range of evidence for the health effects of sedentary behaviours in adults; however, our assessment of this evidence was limited by our choice of search terms, which explicitly attempted to focus on a paediatric and adolescent population.

Studies that examined health effects of sedentary behaviours in children most often focused on a limited range of outcomes, most commonly around cardiometabolic or adiposity outcomes. Most frequently, short-term outcomes were observed, such as high-density lipoprotein cholesterol or BMI. The six reviews ranked most relevant (i.e. those scoring 5/5) focus primarily on these outcomes; however, they also report other major outcomes.12,14,142–145 The reviews present a mixed evidence base, with the most relevant reviews suggesting a negative impact of higher sedentary behaviour on paediatric health outcomes, particularly from observational studies not controlling for physical activity levels. The evidence base is, however, inconclusive or shows no health effect of sedentary behaviour in a paediatric population for prospective study designs or when physical activity levels or diet are controlled for.12,144,145 A full summary of evidence presented in the reviews considered is presented in Report Supplementary Material 2.

Narrative summary of studies considered

Carson et al.,14 de Rezende et al.142 and Tremblay et al.13 focus on non-prospective sources, as well as intervention studies. Perhaps due to the greater number of non-prospective sources considered, the direction of evidence presented suggests an association between lower sedentary time, lower BMI and improved psychosocial outcomes. Tremblay et al.13 provide a meta-analysis for adiposity outcomes; however, all above papers use a narrative review as their main vehicle of evidence synthesis. No review attempts to control for levels of physical activity, neither do they readily differentiate the conclusions of evidence gathered from objective measures of sedentary behaviour from other sources. Authors cite heterogeneity in exposure and methods of outcome measurement in underlying studies as a limitation to further quantitative synthesis. Nevertheless, Tremblay et al.13 conclude that the limited RCT evidence examined, ‘coupled with the qualitative syntheses of data from the other study designs, provides consistent evidence of the inverse relationship between sedentary behaviour and health outcomes’.

Biddle et al.,144 Cliff et al.12 and van Ekris et al.145 take a more sceptical view of the evidence available. These reviews focus primarily on prospective studies or those using objective measurement of sedentary behaviours. Cliff et al.12 provide an estimate of the health impact of sedentary behaviours controlling for moderate to vigorous physical activity levels. Although Carson et al.14 argue against this approach, due to potential for collinearity in physical activity levels and sedentary behaviour patterns in a regression framework, Cliff et al.12 nevertheless utilise this approach. As Cliff et al.’s12 focus was on evidence gathered from experimental sources (often based on multicomponent physical activity interventions), Carson et al.’s14 criticism is less applicable. Carson et al.’s14 criticism is most pertinent for evidence gathered from observational sources, where causality is more difficult to attribute; thus, adjusted regression approaches may eliminate the intervention effect.

Cliff et al.12 conclude that ‘there is limited available evidence that volume or patterns of sedentary behaviour are associated with health and development in children and young people, particularly from studies with low [risk of bias] that adjust for MVPA’.12 van Ekris et al.145 give an evidence synthesis stratified by sedentary behaviour type to help account for heterogeneity in exposure type. This analysis presents a small, non-significant increase in adiposity (BMI) for each additional reported hour of television viewing or computer use. There was no evidence of effect on adiposity in studies using objective measurement of sedentary behaviour. Biddle et al.144 also conclude that studies using objective accelerometer measurement yield null association between sedentary behaviour (measured by screen time) and adiposity, despite a small dose–response relationship observed from cross-sectional evidence. Biddle et al.144 conclude that there is no evidence of a causal association between sedentary behaviour and adiposity in youth using Bradford Hill criteria.

Results

The summary of the evidence base of the health effects of sedentary behaviour in children was surprising to us. The evidence base for adverse effects of childhood sedentary behaviour on common health outcomes was expected to be stronger. All authors acknowledge limitations in the evidence base, citing the need for more long-term studies in this area, particularly with robust study designs. Nevertheless, the observed divergence from evidence gathered in adults, for whom there is currently stronger indication of adverse health effects of sedentary behaviours, may have plausible explanation. Some authors suggest that children accumulate their sedentary behaviour in a different way to adults.146 Altenburg et al.146 found the children studied to spend ‘most of their sedentary time in bouts of ≥ 5 minutes’ and ‘bouts of ≥ 20 minutes were rare’. Meanwhile, adult studies indicate that prolonged uninterrupted sitting is especially detrimental for cardiometabolic health.147–149 van Ekris et al.145 therefore hypothesise ‘that frequent interruptions in children’s sedentary time may prevent them from negative health consequences’. The underlying biology may be the same, but the behaviour was sufficiently different to prevent this being observed.

An alternative explanation may be borne out in the inadequacy of measurement methods to detect sedentary behaviours accurately. Much research uses non-objective measures, such as self- or proxy-reported questionnaires, with serious risk of bias. However, Carson et al.14 equally note that ‘accelerometers may not be sensitive enough to capture patterns in sedentary time’, and propose future research using inclinometers, ‘which can more accurately capture postures . . . to better understand the health impacts of patterns of sedentary behaviour’.14,150 The authors therefore believe that the use of accelerometers as the primary instrument of measuring sedentary behaviour objectively limits the potential to find evidence of a health effect. Nevertheless, evidence of the paediatric health effect of sedentary behaviours from inclinometer measures is currently unavailable and therefore cannot be considered in economic model building. Perhaps more notably, evidence of the health effects of sedentary behaviours gathered using any means from screen time studies are likely to be largely incomparable with the in-school evidence generated in this study, if the possible mediating effects of increased snacking while sedentary during screen time is considered a plausible confounding effect. It is not expected that this mediating pathway would be observed to a similar degree within the school environment.

Candidate models

The scoping search found no candidate economic models to assess the health impact of sedentary behaviour in children. This is not surprising, given the limited literature providing robust evidence of an association between sedentary behaviour and adverse health outcomes in children, independent of physical activity levels. Meta-analysis data gathered from the literature was too diverse to attempt to incorporate into a new economic model, given budget and time constraints imposed by the project. Outcomes, exposure types and dosing reported by meta-analysis were considered difficult to translate to a modelling approach that examined health outcomes suitable for economic evaluation. Meta-analyses that did report relevant information on exposure type and dosing generally reported intermediate biological end points as main outcomes, which also lacks sufficient linking evidence for a paediatric population to long-term health outcomes that could be used to understand potential health-related costs and quality of life. A project with greater research resources could undoubtedly make better use of existing evidence in attempting to create a simple economic model synthesising all existing evidence and aiding decision-makers’ understanding of key remaining research uncertainties.

The research team therefore decided to focus on evaluations conducted in the wider sport science literature. Communication with other health economists working in sport science evaluation revealed three candidate economic models, focusing on physical activity rather than sedentary behaviours.151–154 This approach would provide indicative evidence of the potential health effects and economic value of the trial intervention in a related area in which more robust evidence was available and was, therefore, considered the best use of the quantitative evidence collected. This approach would also aid comparability with existing public health interventions valued with these tools, with the relative comparability of modelling assumptions. Similarly, this would best aid decision-makers by producing generalised outcomes measured in health-related costs and QALYs, which can be used for comparison between different programmes.155 Nevertheless, patterns of sedentary behaviour are hypothesised to have a health effect independent from overall levels of physical activity, rather than being defined simply by the absence of physical activity.9,156,157 These papers raise evidence in health outcomes of sedentary behaviours in adults (except Treuth et al.156), from cardiometabolic disease and all-cause mortality, to a variety of physiological and psychological problems. The limitations of our approach in focusing solely on health effects arising from ‘physical activity’ are therefore recognised and could bias the results against the cost-effectiveness of the intervention, if the true health effects arising from sedentary behaviour are clinically significant.

All three models were Markov-type structured, with each model’s key features summarised in Report Supplementary Material 2. MOVES v.2 model was chosen primarily for its ability to be adapted to model individual-level health effects from person-specific activity values collected during the trial at baseline and at follow-up. Assumptions were more transparent and adaptable than other models considered, and therefore despite evidence being tailored to an adult population (aged > 16 years), this model was deemed most appropriate to make best use of the granularity of continuous individual-level activity data collected during the trial, without aggregation to a binary activity variable. Further detail of the mechanism of the MOVES model is provided in Introduction to MOVES model and also in the updated MOVES v.2 user guide.151

Extrapolated trial analysis

Introduction

This section presents long-term modelling estimating changes in health-related quality of life and health costs arising from changes in physical activity levels observed in the Stand Out in Class trial over a 30-year time horizon.

Methods

Introduction to MOVES model

The MOVES model is designed to estimate the health-related quality-of-life and cost impact of a change in physical activity induced by a specific intervention. The tool uses epidemiological evidence to estimate the health impact of a change in physical activity on projected incidence of eight common health conditions over a chosen time horizon up to 30 years. The model focuses on the studied health effects of physical activity in adults aged ≥ 16 years, for whom the evidence base of a health effect is greatest.

The tool estimates the incidence of eight health conditions with and without a physical activity intervention to estimate the incremental cost and health-related quality-of-life effect of the intervention over a chosen time horizon. Increased physical activity is modelled to reduce the projected incidence of eight common health conditions for which a dose–response relationship has been rigorously studied. The projected number of incident and prevalent cases over the chosen time horizon are combined with disease-specific cost and quality-of-life sources gathered from the literature. This information is used to estimate the impact of projected changes in disease incidence due to physical activity on health service costs associated with treating and managing the considered health conditions.

Quality-of-life impact is similarly modelled, comparing quality-of-life outcomes in the two projections of disease prevalence with and without the physical activity intervention with age-adjusted general population values. Quality-of-life changes in the model arise directly from changes in the projected prevalence of health conditions and each condition’s associated severity.158 Quality-of-life changes from incident conditions are modelled as absolute values rather than decrements to age-adjusted values. This means that it is possible that an incident health condition may increase health-related quality of life in the model. It was not possible to rectify this modelling issue in the scope of this work; however, this model deficit is most important for elderly populations who have a lower health-related quality of life, on average, than the young population considered in our analysis.

The magnitude of change in physical activity is modelled using METs, a physiological measure expressing the energy cost of the physical activity undertaken. This is a standard measure of physical activity used regularly in the sport science literature.159,160 A reference value of 1 MET, for example, is considered the resting metabolic rate.20 The measure has ratio properties, meaning that an activity of 2 METs can be considered double the energy expenditure than that of an average person at rest.

The MOVES tool models a diminishing health benefit of additional exercise for already active individuals. Similarly, the health decrement from decreasing activity levels from a high baseline activity will result in a smaller change in modelled health than for a less active individual. Thus, the model is less sensitive to changes in activity for highly active individuals than for relatively inactive individuals. This decrement was modelled with an adult population in mind; any change in this inflection for a child population was not considered in the scope of this research.

The model is parameterised using a value of ‘METs per week’, with and without intervention. In the base case, the baseline value was assumed to be the average activity level observed in the control arm at follow-up. The intervention effect was assumed to be the difference between the average activity level observed in the intervention arm and control arm at follow-up. This is reported as the ‘unadjusted’ analysis. We investigate the sensitivity of conclusions to an intervention effect controlling for difference in the activity levels observed in intervention and control arms at baseline. This is reported as the ‘adjusted’ analysis.

Activity levels were measured by the ActiGraph accelerometer in both trial arms (see Chapter 2). MET values were obtained directly from internal ActiGraph processing software (ActiLife) and averaged over a standardised waking wear time period (06.00 to 23.00, as determined by diary entries). Non-wear time was detected during the processing of the ActiGraph data and automatically removed.

Although METs are a standard tool in physical activity research, there is some limitation to their applicability in analysing interventions targeted at reducing sedentary behaviours.154,161 A growing evidence base suggests an independent association between sedentary behaviours, such as prolonged sitting, and health outcomes, regardless of the levels of physical activity undertaken. Some evidence is suggestive of this health effect in conditions such as obesity, cardiovascular/metabolic markers and some cancers,162,163 leading to characterisations such as the ‘physically active couch potato’ or the ‘active non-couch potato’.13,164

This highlights a weakness in our modelling approach. By focusing only on the average levels of activity at the two time periods, we do not analyse how that level of activity was achieved, the intensity of bursts of vigorous activity and the length of unbroken sedentary periods. For example, subject A and subject B may achieve the same average level of activity, with one engaging in short periods of high-intensity activity and another achieving a longer dose of lower-intensity activity. Evidence suggests that the distribution of activity may be important, with a hypothesised change in health risk if activity is regularly interspersed throughout the day, rather than achieving the same dose of physical activity in a single burst.165 However, as above, our scoping review failed to discover any existing models suitable or adaptable for health economic evaluation that assessed the independent health effects of sedentary behaviour from physical activity (see Scoping review).

Model mechanism

MOVES models the projected change in disease incidence resulting from a change in physical activity for type 2 diabetes mellitus, breast cancer, colorectal cancer, coronary heart disease, cerebrovascular disease, dementia, depression and hip fracture. Modelled conditions were based on the UK Chief Medical Officers’ 2011 Department of Health and Social Care report Start Active, Stay Active, summarising the health conditions judged to have strong evidence of an association between physical activity and health, which could be modelled with a dose–response relationship.23 Evidence has been presented of the impact of physical activity on up to 20 long-term conditions; however, evidence of a health impact on other conditions was not judged sufficiently robust by the Chief Medical Officers’ 2011 report.23,166

Annual population incidence and treatment effect of physical activity were estimated by the model authors for the eight health conditions. Physical activity interventions are not assumed in the model to affect the number of already prevalent cases, only the incidence of future cases. Population incidence of disease rates are calculated for the general population (which therefore includes active and non-active participants), with incidence rates divided into five age categories, with categories split into 15-year periods until 60 years of age. The modelling approach assumes average population incidence of each disease condition to occur at population average rates of physical activity (adjusted for age group). The model gathers evidence for age-specific incidence from a global source, so performs a direct standardisation based on UK 2011 census data of UK population size in each age category for each sex.167,168

Evidence for the treatment effect attributed by the model is based on two meta-analyses.169,170 The source for each health condition is reported in the model user manual, alongside the model authors’ quality assessment of underlying studies.151 Costs and quality-of-life effects from hip fracture were not considered relevant outcomes for our trial population, with no estimated change in outcomes. Dose–response relationships were not reported stratified by age group in either meta-analysis, so the treatment effect of exercise is assumed to be uniform across age groups. This is likely to bias the direction of modelled association if the true effect of physical activity on health is greater in a population subgroup, such as the very young or very old.

Model authors have added smoothing to treatment effects reported in the meta-analyses to extrapolate over a wider range of potential activity values (0–70 METs/week). Although meta-analyses considered the effect of a fixed dose of physical activity on health, the model aims to estimate the health response of physical activity across a range of values. A comparison of raw and smoothed curves is presented in Report Supplementary Material 2. Smoothing appears generally reasonable for study outcomes derived from the Kyu et al.169 meta-analysis, with some divergence in modelled patterns of type 2 diabetes mellitus from meta-analysis evidence, with meta-analysis suggesting a stronger health effect of activity than that modelled. It was not possible to compare raw and modelled outcomes for dementia or depression outcomes gathered from the Woodcock et al.170 study. Insufficient detail was provided in the model handbook to understand the model authors’ choice of smoothing effect for all conditions and this prevented further investigation. Default model assumptions of the shape of the dose–response relationship were therefore not altered in our study.

Further detail of model mechanism is available in the model manual.151 A full list of adjustments to the standard model specification are presented in Report Supplementary Material 2. Parameters used in the model were not updated to aid comparability and replicability with the standard model.

Model parameterisation

Our analysis uses each participant’s daily average of total measured activity to estimate the average level of wake-time activity at baseline and follow-up in each trial arm. There was no minimum valid observation period of accelerometer wear, as data sparsity meant that the trial protocol of a minimum of 3 days of accelerometer wear time was dropped. This was judged the most pragmatic approach with data available. Activity levels during school hours and out of school time were estimated separately from trial data to understand the independent effect of the intervention on activity during and after school hours. This separation was important owing to potential for a compensation effect in total measured activity levels (i.e. a compensating reduction in magnitude/intensity of out-of-school activity due to any increase in activity during the school day).171

Activity values measured in METs per week within and outside school during the trial period were therefore scaled up to create a simulated year. This simulated year attempts to describe any variation in activity between the academic year and school holidays. Children were assumed to be in school for 190 days per year (government-mandated minimum).172 Data collected during weekends were assumed to be representative of activity during holiday periods and therefore multiplied by 175.25 days. This created an estimate of activity over a full year, which was used to simulate the ‘average’ MET per week in each trial arm at baseline and at follow-up.

Metabolic equivalents per week were estimated in the control and the intervention arm by linear regression, adjusting standard errors in mean values for clustering at the school level. In the base case, estimated activity measured in METs per week over the simulated year were compared at follow-up (unadjusted analysis). In a corresponding scenario, the observed difference between groups at follow-up was adjusted for baseline activity values (adjusted analysis). Regression modelling did not investigate the effect of any clustering of activity within individuals between time points.

Estimated values were fed into the model, with probabilistic modelling choosing a random value around the mean value from Student’s t distribution around the modelled standard error, with degrees of freedom specified by regression output. In multiply imputed data sets, the average degrees of freedom from regression estimates were used (4.97–5.46 degrees of freedom). In complete-case analysis, 7 degrees of freedom were used, with t distribution sample size adjusted to match the appropriate complete-case sample. Monte Carlo simulation modelling was used to generate 10,000 simulations for each scenario to represent the effect of parameter uncertainty on cost-effectiveness results.

The MOVES model allows specification of a parameter determining how long the health effect from any change in an individual’s activity is expected to be maintained. Previously reported modelling studies for interventions aiming to reduce sedentary behaviour assume that intervention effects are fully sustained for the first year and then decay at a rate of 50–55% per annum thereafter.173–176 A 50% decay rate per annum in the health effect of the intervention was therefore assumed in the base case, and sensitivity analysis investigated the effect on cost-effectiveness conclusions when decay rates of 0% and 100% were assumed. These scenarios represent, respectively, a full ongoing health effect across 30 years or an exponential decay to the health effect after the year of the trial intervention.

Estimated costs and health-related quality of life over time horizons considered were discounted at 3.5% per annum, with intervention costs assumed as a one-off event at the beginning of the modelled time horizon. As in within-trial analysis, desks used in the intervention and control arm were modelled to have an expected lifespan of 10 years, meaning that the intervention cost was assumed to be £7.58 per child. Scenario analysis investigated the impact of using a public health discount rate of 1.5% per annum. Probabilistic modelling is presented for all scenarios.

Expected value of perfect information was computed for the base-case model and relevant scenarios. This is computed as the difference between the (1) average of maximum net monetary benefit of either Stand Out in Class or a ‘do nothing’ strategy and (2) average net monetary benefit from the Stand Out in Class intervention.177 This is calculated over 10,000 scenarios and calculated at maximum and minimum stated willingness-to-pay thresholds set by the National Institute for Health and Care Excellence’s (NICE’s) technology assessments.178 A 10-year discounted population of all children in the UK aged 9.5 years was chosen to represent the target population of the intervention over its expected time horizon, representing the expected size of the Year 5 school cohort in the UK. Values for the UK population size by age were taken from mid-2017 population estimates.179

Results

Physical activity

Activity values at baseline and follow-up in each arm are presented in Table 19, with multiply imputed and complete-case values presented alongside for comparison. Multiply imputed values were used for base-case analysis. ‘Simulated average’ values represent the mean METs per day expected over the simulated year described above. These values were used to create the expected METs over the course of the simulated week used as a model parameter.

As is apparent from the complete-case analysis, presented in Table 19 for comparison, there was a higher level of completeness in accelerometer measurement at baseline than at follow-up, which is reflected in the increased standard error around the sample means at follow-up. The increased standard error in the intervention arm at follow-up may also highlight heterogeneity between subgroups within the intervention group, some of whom may have responded particularly strongly to the standing desk intervention in their activity patterns. This possibility was not investigated further in this analysis owing to potential for overinterpretation of small numbers.

Activity levels are higher, on average, across all measures at follow-up than at baseline. As highlighted previously, follow-up data collection was undertaken during the summer months, with longer daytime hours than for the baseline data collection in winter. For this reason, average activity values were collected over a longer period, with an applied wake-time filter of 06.00 to 23.00 at both time periods (based on sleep and wake-time diary entries), thus potentially capturing an increase in daily activity during the summer period.

Comparing trial arms in the multiply imputed data set at follow-up, there is a slightly higher average daily activity level measured in the intervention arm than in the control arm during weekday and weekend monitoring. The difference between outcomes is highest at weekend measurement, in which average daily activity is 1.54% higher in the intervention arm than in the control arm. The difference between arms narrows to 1.12% in weekday activity at follow-up. This translates into an activity difference of 1.34% between arms at follow-up in the simulated year.

In complete-case analysis, the difference between arms in activity levels at follow-up suggests a slightly different pattern. The difference in activity levels between arms during weekday measurement is only 0.22%, with a higher pattern of activity in the control arm. The difference in activity levels widens during weekend observation to 2.95%, translating to a difference in activity over the simulated year to 1.53%. Missing data in complete-case analysis therefore appear to follow a pattern captured by baseline characteristics included in regression adjustment in multiple imputation equations, with imputation estimates rebalancing average activity values by differential adjustment in some of the imputation subgroups considered. The sensitivity of conclusions to alternative specification of multiple imputation equations was not investigated in this analysis; however, this could plausibly form a part of future work.

When also considering the pattern of baseline values, evidence of a difference between groups is starker. If considering the trend between baseline and follow-up time periods, the intervention arm shows a 0.60% lower increase in measured activity between summer and winter time points than in the control arm. The difference between arms is strongest during weekend measurement, for which the difference in trend between time points is 26.83%, with the intervention arm demonstrating considerably less increase in activity from winter to summer. This corresponds to an overall difference in trend between trial arms over the simulated year of 18.73%, showing a larger increase in activity in the control arm than the intervention arm during the trial period.

To describe the potential importance of imbalance in baseline values to study conclusions, mean differences between the intervention and control group at baseline are presented. In both the multiply imputed base case and complete-case analysis, there is some evidence of difference between the levels of activity observed in the intervention and control groups at baseline. In the multiply imputed base case, the difference in activity levels over a simulated average week shows already higher activity in the intervention arm than in the control arm (0.3669 METs/week, 95% CI –0.1537 to 0.8874 METs/week). At follow-up, there is less mean difference between treatment and control groups, albeit with wider CIs, particularly in the complete-case analysis. The reader’s judgement regarding the relevance of slight evidence of true difference between the control and intervention arms’ baseline activity in this small sample should be used to inform the reader’s view of the appropriateness of adjusting estimates of the difference between intervention and control arms at follow-up for any existing difference in baseline activity levels in long-term cost-effectiveness modelling.

Long-term cost-effectiveness

Table 20 shows the main results of long-term probabilistic cost-effectiveness modelling over a 30-year time horizon. In the base case, in which there was no adjustment for baseline activity levels, ‘unadjusted’ modelling of the incremental cost-effectiveness of the Stand Out in Class intervention produced an ICER of £78,986 per QALY. For unadjusted activity levels, probabilistic modelling estimated a 26.67% probability of the Stand Out in Class intervention being cost-effective at the current maximum national willingness to pay for a unit of health (£30,000/QALY gained).

When differences between groups in baseline activity levels were accounted for, the Stand Out in Class intervention was dominated by a strategy of ‘do nothing’. In this case, the intervention was, on average, more costly and less effective in improving population health than making no changes to the classroom environment. This is due to a smaller observed change in physical activity levels between time points in the intervention arm than the control arm. At a maximum value of the current NICE threshold, adjusting for differences in baseline levels of physical activity, the estimated probability of the Stand Out in Class intervention being cost-effective was 10.56%. This means that, accounting for uncertainty in the change in physical activity observed, there is only a small chance of the intervention being cost-effective under these assumptions.

Our long-term cost-effectiveness conclusions were moderately sensitive to choice of health services or public health discount rate, with the probabilistic ICER reducing in the unadjusted case by £18,556 when using the public health discount rate of 1.5%. However, it remained unlikely that the intervention was cost-effective, with an estimated 34.33% chance that the intervention was cost-effective at the maximum NICE threshold at the new unadjusted ICER of £60,430 per QALY.

Our conclusions were, however, highly sensitive to assumptions surrounding the ongoing health effect of physical activity beyond the intervention itself. When the health effect of past exercise was assumed to be maintained for the whole modelled time horizon of 30 years, the intervention was highly cost-effective in the unadjusted case at £10,546 per QALY, with a 60.32% chance of being cost-effective at the maximum threshold value. Conversely, in the adjusted case, due to the observed reduction in physical activity being maintained over the whole trial period, there was a corresponding increase in modelled costs and decrease in modelled QALYs. Owing to the increased importance of uncertainty around the health effect when modelled over a longer period, in the adjusted case there was still a moderate chance of the intervention being cost-effective (34.84% at a threshold of £30,000/QALY).

As expected, when the health effect was assumed to decay exponentially after the period of the intervention, the modelled probabilistic ICER increased to £116,278 per QALY in the unadjusted case (14.22% chance cost-effective at £30,000/QALY). The adjusted case remained dominant, with a 5.13% chance of being cost-effective at £30,000 per QALY. Cost-effectiveness results for complete-case analysis are presented in Report Supplementary Material 2. Cost-effectiveness acceptability curves for base-case scenarios are also available in Report Supplementary Material 2.

Expected value of perfect information

Table 20 presents the EVPI to resolve all parameter uncertainty considered in the model. EVPI is calculated for each scenario at the lower and upper NICE cost-effectiveness thresholds, per person and at the population level.

The EVPI results are more sensitive to the scenario modelled than whether or not post-intervention measures of activity levels adjust for baseline activity. At the upper bound of the NICE threshold (i.e. a higher willingness to pay per population QALY generated), there is increased value placed on resolving decision uncertainty regarding the proposed intervention’s cost-effectiveness. For this reason, in the base case, EVPI in the baseline-adjusted analysis was modelled at £0.43–0.90 per person at national thresholds of between £20,000 and £30,000 per QALY. In unadjusted analysis, EVPI per person is modelled at £0.66–1.69 (threshold £20,000–30,000/QALY). At the higher threshold, the EVPI is 11.9% of the modelled cost of the Stand Out in Class intervention per person in the adjusted case and 22.3% of the desk cost in the unadjusted case.

There was only one scenario in which EVPI to resolve all modelled parameter uncertainty was at or higher than the value of the cost of the intervention per person. When the health effect of the standing desk intervention was assumed to be maintained for the full time horizon of 30 years, the EVPI per person in the baseline-adjusted analysis was £7.57–11.97 (threshold £20,000–30,000/QALY). In the unadjusted analysis, the EVPI per person was £11.82–15.61 (threshold £20,000–30,000/QALY). At the upper threshold value, this puts the EVPI in this scenario at 157.9% of the cost of the Stand Out in Class intervention in the adjusted case and 205.9% of the intervention cost in the unadjusted analysis.

The EVPI was calculated at a population level, considering the expected discounted UK population of Year 5 pupils over the intervention lifetime of 10 years. In the base case, the population value of resolving all modelled parameter uncertainty in the health effect of the Stand Out in Class intervention is between £3.0M and £6.2M when differences in baseline activity levels are considered, or £4.6M and £11.6M when not adjusting for difference in baseline activity (threshold £20,000–30,000/QALY). Again, the assumed ‘ongoing health effect’ parameter, determining the proportion of ongoing health effect beyond the intervention period, produced considerable variation between scenarios in population EVPI for the discounted population. Population EVPI values for the discounted population of 10 ‘Year 5’ cohorts across all scenarios considered are presented in Table 20. Across all scenarios considered, the maximum expected value of research to resolve all decision uncertainty was £107M at a threshold of £30,000 per QALY. A comparison plot of the EVPI at different willingness-to-pay thresholds for unadjusted base-case scenarios is presented in Report Supplementary Material 2.

Discussion: long-term cost-effectiveness modelling

Key findings

Based on long-term cost-effectiveness modelling over a 30-year time horizon, the Stand Out in Class intervention is not cost-effective at a threshold of £30,000 per QALY, with a low probability of the intervention being cost-effective based on modelled uncertainty in existing evidence. ICERs show large variation between scenarios; however, in all but one of eight modelled scenarios, this variation does not result in a predicted mean ICER below the upper bound of the UK cost-effectiveness threshold for new health technologies. In base-case analysis, we estimated an ICER of £78,986 per QALY in ‘unadjusted’ analysis, with the treatment being dominated by a strategy of ‘do nothing’ in ‘adjusted’ analysis, which controlled for differences between groups in levels of baseline activity.

The Stand Out in Class intervention is only cost-effective when the ‘unadjusted’ health effect of a standing desk is assumed to be maintained for the whole 30-year period. At face value, this appears to be an unreasonable assumption. However, this is equivalent to assuming that the intervention would produce a long-term behaviour change in the treated population, which meant that their activity levels were consistently above the untreated (at the current magnitude) population over the 30-year period. When phrased as such, this assumption may appear plausible, given recent dialogue of the potential for early intervention to deliver long-term maintenance of intervention-induced behaviour change.180 However, when incorporating this assumption into a model that adjusts the treatment effect in activity levels for observed differences in activity between trial groups at baseline, the Stand Out in Class intervention is still shown to be more costly and less effective than a ‘do nothing’ strategy, as in all other scenarios that control for differences between groups in baseline activity levels.

Estimates of EVPI present the maximum possible value of future research to resolve all remaining decision uncertainty regarding the cost-effectiveness of the Stand Out in Class intervention, based on the modelling approach used. ‘Perfect information’ can eliminate the possibility of making the wrong decision. For this reason, to maximise population health gain, a health system should be willing to pay for additional evidence to inform a decision, up to a maximum budget constraint, after which the cost of research will exceed the population health benefit from further research.181 In this case, a proportion of this expected research value is in resolving parameter uncertainty in the health effect of physical activity. A further proportion of this research value is in determining the true value of expected health costs and quality of life for a population with a known disease burden. The remaining value of research uncertainty is in parameter uncertainty regarding the activity inputs (i.e. knowing the true levels of activity in each intervention group at baseline and follow-up).

The expected population value of perfect information for all parameters in base-case analysis is £6.2M in the ‘adjusted’ case and £11.6M in the ‘unadjusted’ case. This assumes the higher value of the national threshold value to produce one ‘unit’ of health (£30,000/QALY). In this case, the value of research that resolved all parameter uncertainty involved in this funding decision would be worth between £6.2M and £11.6M to a national decision-maker.

In the scenario in which we assume an ongoing health effect throughout the intervention period, the EVPI increases to £107.4M in the ‘unadjusted’ case and £82.4M in the ‘adjusted’ case. When the health effect of the intervention is maintained for this longer period, knowing the true change in activity induced by the intervention matters proportionately more.

Neither the expected value of perfect parameter information nor the expected value of sample information were calculated as part of this work. For this reason, we do not have a clear indication of the upper bound of the expected value of further research into activity levels alone, as collected in this trial. We also do not have a clear indication of the expected value of an additional trial, which without incurring unfeasibly large costs to obtain an unlimited sample size, would resolve only a proportion of the total research uncertainty.182 For this reason, we can conclude only that the further value of research into determining the true health effect of the Stand Out in Class trial is bounded by EVPI estimates, with the value of resolving uncertainty in true levels of activity pre and post intervention being only a proportion of the total EVPI over all parameters presented above.

Comparability with other studies

To the best of our knowledge, this chapter represents the first examination of the long-term cost-effectiveness of active classroom interventions, such as sit–stand desks. Despite public health interest in targeting childhood sedentary behaviours, with the intention of changing these behaviours over the longer term, the cost-effectiveness of such interventions has been less frequently considered.183 Such interventions are an obvious candidate for public health cost-effectiveness modelling, given that trial outcomes collected are typically short term and expected payoffs occur over a much longer time horizon, with potential overspill between sectors, such as improved education outcomes leading to better health outcomes in later life.184

Although one systematic review185 found school-based physical activity interventions to be among the most cost-effective in increasing energy expenditure, a separate review186 linking activity gains to changes in health using a generalised cost–utility framework measured in QALYs, revealed all school-based interventions considered to be cost-ineffective at a threshold of US$50,000 per QALY.185,186 Current work, including this report chapter, has, however, not yet explicitly integrated consideration of potential intersectoral impacts of active classroom interventions into cost-effectiveness analysis.187 Explicit consideration of intersectoral effects could provide a new perspective on the estimated long-term cost-effectiveness of interventions, such as the Stand Out in Class intervention.

In contextualising the changes in activity levels measured in the trial, in a recent systematic review57 of eight studies of classroom standing desks and sedentary behaviour (three RCTs), all studies reported an increase in standing time and a decrease in sitting time. This review57 did not report activity outcomes in terms of change in METs per hour. Using an average of point estimates of energy expenditure in sitting-type activities, we could expect an approximate activity increase of 0.193 METs per hour for each additional hour spent standing rather than sitting.188 For this reason, the additional metabolic expenditure of an estimated 40-minute increase in standing time in children, reported by the single study59 reporting outcomes in minutes over waking hours, is likely to have a limited impact on daily average activity. This translates to an estimated increase of 0.0107 METs over a 12-hour waking day. The small impact of the Stand Out in Class intervention on metabolic expenditure levels therefore does not appear inconsistent with wider literature.

Strengths and limitations

In this trial, intervention cost-effectiveness was undermined by study results that demonstrated only small changes in activity, measured by METs. METs are sometimes sidelined in newer research studies of sedentary behaviour. Newer studies often measure the impact of sedentary behaviour interventions on specific related outcomes, such as changes in sitting time, rather than providing a generalised measure. This is due to the perceived insensitivity of METs to detect changes in patterns of sedentary behaviour and the hypothesis of a health effect of sedentary behaviour independent of physical activity.157,189 However, although there is some evidence of sedentary behaviour patterns influencing a limited range of intermediate health outcomes, such as short-term measures of blood pressure, cholesterol or blood lipids, these outcomes are not currently utilisable in health economic evaluations as their impact on meaningful generalisable measures of population health (such as the QALY) is highly variable.190 For example, the relevance of changes in blood composition will depend on the cardiac risk profile of the population considered and how long any change in blood composition is maintained. There is currently insufficient evidence not only in the effect of measured sedentary behaviour on surrogate health outcomes, but also of the link between the measured intermediate health outcomes to robust measures of final end points, such as health-related quality of life. This absence is particularly apparent in children.

Our study conclusions utilise a health mechanism mediated by physical activity levels, in which causal links are more readily established.23 Future work could investigate the plausibility of updating our modelling strategy with current best evidence on the independent health effect of sedentary behaviour; however, such work will require a significant increase in research resources. Given current limitations in available evidence, such research is also unlikely to resolve a meaningful proportion of decision uncertainty in relation to the cost of any such research undertaking.

Even in using METs, significant uncertainty remains in estimating the health impact of changes to activity levels. Questions of the nature of dose–response between activity and health outcomes remain unresolved.191 Estimation of the dose–response relationships of hypothesised health effects, given different levels of baseline activity, require very large sample sizes.192 For example, the current model assumption of an attenuating dose–response relationship at higher levels of physical activity is likely to require further examination. There is also particular scepticism regarding current modelled health costs and outcomes in depression, with the evidence base currently heavily relying on use of inconsistent self-report measures of physical activity, with varying use of subjective and direct measures of defining depression criteria.193,194 This presents a barrier to understanding the nature of a dose–response relationship with an adequate degree of precision.

In this study, we have assumed that health effects of changes in activity in children are equivalent to the modelled risk in young adults (aged ≥ 16 years). This is despite difference in national population guidelines of the recommended level of physical activity per week in the two populations, with recommended levels of activity in children being significantly higher.23 Any difference in risk profile from additional activity in children compared with an adult population was not addressed as part of this research and may be an important consideration in future analysis. There was also no account in modelling for changes in the trajectory of physical activity with ageing of the modelled cohort (i.e. the population becoming more sedentary with age); additional economic modelling could be used to address this deficiency.139,195