Having recovered from the realisation that last week’s policy trials event was the 19th in the Health Challenge Wales ‘Evidence for policy’ seminar series (it seemed like only yesterday that we were planning the first session!), I thought I’d reflect briefly on what was a hugely enjoyable and stimulating day.
The idea to submit a proposal for the seminar came about at a Behavioural Insights Team launch event earlier in the year for what Ben Goldacre has affectionately referred to as the “Ladybird Book of Randomised Policy Trials”. At this event, Dr. Simon Murphy presented a paper on our experiences in Wales of running policy trials, with particular reference to the National Exercise Referral Scheme. While encouraged by the government’s increased recognition of the importance of robust policy trials in the development of public policy, we were left with a sense of frustration that our work in Wales has not received the recognition that it deserves.
Fast forward six months and this event, hosted by DECIPHer as part of the ESRC’s 2012 Festival of Social Science, provided an opportunity to recognise the track record of research in Wales, as well as an examination of how a range of policy trials have been developed and implemented. So, what themes emerged from the day? To borrow from Archie Cochrane, here are a few “random reflections” on policy trials.
First, I can’t overstate the importance of early and effective collaboration – for example, between academic researchers, their counterparts within government, policy makers and practitioners. From experience, an effective policy trial depends on early engagement to inform how a policy is developed and – crucially – rolled out.
Second, a supportive research infrastructure is a key facilitator, ensuring the availability of suitably skilled people. The Public Health Improvement Research Network, one of the Registered Research Groups established by NISCHR with backing from health improvement policy colleagues within Welsh Government, has facilitated the development of a number of policy trials.
Third, it is naïve to believe that policy trials will be unproblematic. They are likely to be contested, and present many challenges, from the potential methodological compromises required when working in a political environment to issues of cost or ethics. Despite this, they are part of the evidence toolkit that should inform the policy process, and we now have much experience that we can draw on in Wales.
Finally, a nuanced view of policy trials is needed. We must recognise that policy trials come in many shapes and sizes, and should not underestimate the importance of embedded process evaluations. The studies presented during the event were all pragmatic, with a focus on effectiveness and implementation – far removed from the traditional view of a clinical trial that many still have.
So, while policy trials won’t always be possible, they can be done with a degree of foresight and should incorporate early collaboration between relevant partners. Perhaps the challenge now is to see if what can be learnt from Wales’s impressive track record in the area of health improvement policy can be applied to other aspects of public policy.
Chris Roberts (@OrientChris) is Research Lead for the Health, Social Services and Children Analytical Team in the Welsh Government. For more information about the Health Challenge Wales Evidence for Policy Seminar Series and to download slides from past seminars, please see the HCW Seminar Series page.