Cristina Rosemberg is a partner at Technopolis Ltd, a specialist consultancy working across science, technology and innovation.
A challenge-driven approach to funding R&D
Policy makers in the UK and abroad are increasingly embracing the idea of using industrial and innovation policy to tackle ‘grand challenges’ and to orient resources and efforts towards areas of strategic importance.
This has been central in the thinking behind the Industrial Strategy Challenge Fund, set up by UKRI in the past three years, which funds a suite of challenge-driven programmes to support research and innovation in key areas identified in BEIS’ Industrial Strategy. The programmes bring together a package of measures to provide different types of government-backed interventions across the R&D cycle to support the development of nascent sectors. A challenge-driven approach (‘making the UK the market leader in Immersive Content’ or ‘increasing the uptake of AI solutions in the professional service sector’) calls for a mix of interventions.
The Audience of the Future programme (AotF), for instance, funds collaborative R&D projects (where two or three organisations have come together to develop ideas at an early stage of development); demonstrators (led by larger consortia of eight to ten organisations that come together to test new business models and gather market intelligence); and a National Centre for Immersive Storytelling (to upskill the UK creative workforce in the use of immersive technologies). The programme also has support from the Digital Catapult to disseminate the knowledge emerging from the demonstrators, and engage with a wider immersive content community.
Understanding if those initiatives work, and how, is key to informing future public investment decisions and the design of future programmes. Our evaluation of the AotF (and other ISCF programmes*), although still at an interim stage, has begun to shed some light on its effects, and how this sits within a wider national and international context.
Why do we evaluate?
Evaluations offer an opportunity not only to assess the scale of the success (or lack thereof) of interventions, and to understand what works best, but also to collect intelligence to understand the national and international context in which the researchers and innovators, and their solutions and ideas operate.
Evaluating a mix of interventions requires balancing the need to assess the individual interventions, while at the same time understanding if the whole has been more than the sum of its parts, exploring the interconnections that are at play.
Evaluating a relatively new approach, such as the ISCF, also calls for a need to combine summative elements (‘what has been achieved?’) with formative ones (‘why is this the case, and what can we learn from this process?’). Understanding the design and implementation becomes as important as assessing the outcomes. This has required combining methods that explain the ‘what’ but also the ‘how’ and ‘why’, with a toolbox of quantitative and qualitative measures.
Predicting the future
Investments such as those delivered via the ISCF programmes are expected to generate results in the long term, but the need for evidence to inform the next policy cycle is more immediate. This requires setting up evaluations that are able to demonstrate the direction of travel and intermediate results. In a marked departure from other programmes, the ISCF programme evaluations have been set up with a medium term perspective, covering two or three phases. This involves collecting evidence at the outset of the programmes, at a mid-point and after one or two years of completion of most projects, producing insights as they emerge, gathering some intelligence that could help to inform a change in course, and concluding with a better view of what the programmes have achieved.
Evaluators cannot predict the future, but they can identify the signals of positive change and ‘precursor’ indicators that can shed some light on the direction of travel.
Further effects are expected to materialise beyond the life of the evaluation, with additional investments and efforts being made by the organisations involved, and tracking some of the initiatives funded by the programme over time could help to further showcase how (and if) the programme has acted as a catalyst for change in the medium and long term. A recent report from the National Audit Office, while being positive of the efforts made by ISCF so far, also pointed to the need to capture these long term effects.
Programming for change
By nature, research and innovation projects require a degree of exploration and the ability to change course. If we knew all the answers already, we wouldn’t be doing research. Publicly funded programmes then find themselves having to walk the line between monitoring the appropriate use of public resources, and letting the projects follow their natural, non-linear, progression. The COVID pandemic has tested the implementation of research and innovation projects in many ways, not least in terms of the ability of the existing UKRI process to adjust to change, and the ability of the projects to adapt to new circumstances. This is the case for the four AotF demonstrators set up to test an immersive content offer, with audiences at the centre of it, within new partnerships that brought together large players with new tech companies, academics, and start-ups. All four demonstrators needed to adapt to a new (current) reality of no live audiences and reassess their focus and business model. The ‘process’ element of the AotF evaluation shows that UKRI has been able to cope effectively with the changing circumstances, although it is fair to say that the unprecedented circumstances have tested the limits of the organisation’s ability to change gear in an agile way.
Connecting the dots
The evaluation of AotF and other ISCF programmes reveals that intervening at different stages can be quite effective, but that more can be done at the design stage to optimise the level of cross-fertilization across different strands.
This may be achieved by identifying interesting ideas funded at an early experimental stage (through, for instance, collaborative R&D funds), and progressing them to other types of support included within the programme (such as demonstrators, or investor-matching activities).
The ability of the ISCFs to do this was limited the first time around, with the majority of the funding being disbursed in the first two years.
However, these are valuable lessons that could help to inform future generations of the programmes, which could opt for advancing some of the achievements in the first stage (helping them to progress for instance from initial R&D projects to demonstrators) while at the same time being open to new ideas and projects as they emerge.
More challenge driven, agile interventions
More activity is expected in the future around challenge-driven, agile interventions. Just some weeks ago, government announced the launch of a new agency dedicated to the funding of high risk, high reward research. The Advanced Research & Invention Agency (ARIA) will have an initial funding envelope of £800 million and be ‘broadly modelled’ on the US Defence Advanced Research Projects Agency (DARPA), although the details are still unclear.
ISCF was, to some extent, also modelled with DARPA in mind, with Challenge Directors appointed to provide strategic drive to the investments. In submission to the Science and Technology Committee, Sir Mark Walport (UKRI’s former CEO), however, noted that Challenge Directors have not had sufficient freedom to run their programmes, and there are doubts on whether UKRI, in its current configuration and degree of Ministerial oversight, could cope with operating in a more agile, risk taking mode. In contrast, the new agency is expected to have more freedom for identifying and funding opportunities – echoing Sir Mark’s aspiration.
Whatever the future of the new agency and ISCF, the appetite for strategic, goal-oriented and high-risk investment looks likely to drive future R&D investment decisions in the UK, which is a positive development.
*The evaluation is being conducted by Technopolis & BOP Consulting. Technopolis is also involved in the evaluation of five other ISCF programmes.
Cristina Rosemberg presents some of the learning from the Evaluation of the UKRI Audience of the Future programme over on the CRAIC Videos page
March 2021