Last year, we at DECIPHer published findings from a randomised controlled trial of the National Exercise Referral Scheme in Wales. These indicated that patients who had been referred to the scheme with coronary heart disease risk factors (such as diabetes, or being overweight) were significantly more physically active where referred to the intervention that where they were not. Patients who had been referred for mental health reasons did not experience improvements in physical activity, but did demonstrate reductions in anxiety and depression.
NERS represents an example of a rigorously evaluated scheme which conferred significant benefits for many patients. But, while evaluating outcomes in this way is necessary for knowing whether or not an intervention works, by itself it doesn’t allow us to understand why it did (or didn’t) work. For this, we need process evaluation. Our paper, published in Health Education, explores what was actually delivered, and how NERS worked.
Information on how the scheme was implemented was gathered using a combination of:
- Observations of tape recorded consultations between participants and exercise professionals;
- Interviews with exercise professionals and coordinators of the scheme;
- Routinely collected programme monitoring data;
- Qualitative interviews with 32 patients (in six case study centres) and 38 exercise professionals.
Process evaluation allows us to look beyond simply whether an intervention ‘works’ or not, and understand the processes by which outcomes are (or aren’t) achieved.
What was the intended intervention?
According to intervention protocols (a manual which sets out how the scheme is intended to take place), NERS comprised the following core components:
- Advice from a health professional on how to increase activity, and referral to a local authority leisure centre;
- A first appointment with a level three qualified exercise professional, including a health check, motivational interviewing and agreement of goals;
- A 16 week programme of discounted, supervised, mostly group-based exercise opportunities;
- Follow-up contact from exercise professional at four weeks, to review goals;
- After 16 weeks, ‘signposting’ to exercise opportunities outside the scheme;
- Follow-up contact from exercise professional at eight and 12 months to prevent relapse.
In practice, NERS provided referral to an exercise facility, where patients attended a first appointment with an exercise professional, before entering a programme of discounted group exercise opportunities, supervised by the level three qualified instructor. However, some components of NERS were not fully delivered.
Motivational interviewing was not delivered during the trial period. The practice of goal setting was often not consistent with goal setting theory, often involving somewhat ambiguous goals (e.g. ‘get fitter’, or ‘lose weight’, rather than clear goals). Patients who stopped attending were also not routinely followed up at four weeks, and four- and 12-month follow up was inconsistent.
As previously mentioned, NERS was effective for many. However, the process evaluation found that these outcomes were not produced through all of the mechanisms we had anticipated.
How did it work, and for whom?
Interviews with patients and professionals emphasised the role of social processes in encouraging participants to stay with the programme, and in increasing their physical activity. Patients highlighted the value of the exercise professionals’ supervision in safely introducing them to unfamiliar exercise machines, and support their motivation. They also valued the provision of the guidance needed to minimise the risk of making their injury or illness worse through overexertion. Indeed, many described having previously attempted to become more active independently, but having worsened their conditions.
The patient-only classes within NERS provided patients with realistic role models; other patients who had been in the scheme for a few weeks and begun to see changes could provide encouragement and support to new entrants. The patient-only classes also provided a supportive context, in which struggling to overcome illness while being in the exercise environment was normalised. Understanding the importance of the social role of the scheme, many of the exercise professionals involved developed strategies to foster the development of social networks, in which people participating in NERS could support each other’s activity when the scheme ended.
Participants in the NERS found that supervised, patient-only classes provided a supportive context.
We also found that engagement with the intervention was impacted by a range of factors. For example, car owners were more likely to attend the scheme, with classes described as more difficult to access in more rural areas. Older patients were more likely to complete the scheme, with qualitative data indicating that this was in part because many centres ran classes at times during which working age patients could not attend. Additionally the fact that the majority of referrals were older patients may have meant that these patients benefitted most from the social context of the scheme.
While NERS was not delivered entirely as planned during the trial period, data on outcomes indicated substantial benefits for many patients. The process evaluation, which focused on implementation and causal processes, offered important insights into how the scheme was carried out in practice, and how different aspects of it worked.
NERS continues to be offered to patients throughout Wales. Findings from the process evaluation are feeding into discussions regarding the future form of the programme as it evolves to meet changing demands, while attempting to retain the activities which appeared to have positive outcomes.
Moore G, Raisanen L, Moore L, Din N, Murphy S. ‘Mixed-method process evaluation of the Welsh National Exercise Referral Scheme’ Health Education 2013; 113(6).
Dr. Graham Moore is a research fellow at DECIPHer, based at Cardiff University. Graham is currently leading the development of MRC guidance for process evaluations of complex public health interventions.
For further discussion of the need for process evaluation in complex public health interventions, and the need for guidance, see the following piece:
Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. ‘Process evaluation in complex public health intervention studies: the need for guidance’. Journal of Epidemiology and Community Health 2013. Published online first: 10 September 2013. doi:10.1136/jech-2013-202869