Pathways to Impact: From Research into Policy, Practice and Interventions, Nov.30th 2018
On Friday, Nov. 30th, over 50 BSG members and non-members, speakers from academia, lay and non-governmental organization (NGO) stakeholders, met to discuss how ageing research achieves impact. The event was extremely well received by delegates as a great opportunity to engage in conversations about impact from the perspectives of academics and practioners as well as a great opportunity to network. Thank you to all our speakers and delegates who contributed to the event.
For many of the academics their interest lay with the forthcoming Research Excellence Framework (REF) assessment in 2021 where the broad definition of impact follows that of its origins in 2014:
The sub-panels will assess the ‘reach and significance’ of impacts on the economy, society, culture, public policy or services, health, the environment or quality of life that were underpinned by excellent research conducted in the submitted unit. This element will carry a weighting of 25 per cent. (REF 2021, Draft Guidance on submissions, 2018, p12)
In 2015, the BSG had also been open to learning about how research on ageing had been included in REF 2014 and commissioned Bangar, Hargreaves and Mountain (2015) to review the publically available data of impact case studies from all HEIs. Out of 6,679 published impact case studies, 311 involved ageing research (4.7%) and related to 33 out of 36 Units of Assessment (UoAs) indicating the interdisciplinary nature of gerontological research.
Across all 311 case studies ‘health’ was the most commonly occurring category of impact.
The morning started with a consideration of two 4* Impact Case Studies from REF2014. Assessment focused on ‘originality, significance and rigour’ and 4* quality needed to achieve ‘outstanding impacts in terms of reach and significance’.
The impact case studies currently being developed for REF 2021 are using templates similar to those of 2014 and include a profile of: underpinning research, references to the research, details of impact, and sources that can corroborate this impact (see REF 2021, Draft Guidance onsubmissions, 2018, Annex G p113-16).
The first impact case study discussed was led by Professor Anthea Tinker from Kings College London (KCL) entitled ‘Maximising independent living for the UK’s rapidly ageing population’ and the second by Professor Julienne Meyer from University of London (City) entitled ’Research that has impact on the quality of life in care homes for older people’ (links to both case studies are given below plus powerpoint presentation and the programme from the event). Anthea and Julienne are senior members of academic staff with a long history of teaching and research in social gerontology and nursing. They have both collaborated outside and inside academia enabling them to develop strong networks. Their comments were followed by the views of stakeholders – Jeremy Porteus, Managing Director of the Housing Learning and Innovation Network (HousingLIN) and Des Kelly, former Executive Director of the National Care Forum and currently Chair of the Centre for Policy on Ageing. They had used, communicated and supported the findings in relation to policy development and practice and reported on their role.
The following points were identified:
• Decisions over which UoA research is submitted is often directed by the HEI. (Anthea’s work was submitted under UoA Sociology and Julienne’s under UoA Allied Health Professions, Dentistry, Nursing and Pharmacy);
• References to research on the impact case studies usually indicate prominent research publications as outputs. As the KCL’s case focused on the work of one academic and her colleagues, titles of research proposals and funding awarded that led to impact were listed instead of publications (see case study; REF 2021 Draft Guidance on submissions, 2018 p114-5);
• Case studies may demonstrate the way research strengths are developed over time. The KCL work considered 12 projects related to Anthea and colleagues, whereas the City work included 6 projects.
• Current draft guidance talks of continuous case studies where research themes may develop over time with new impact being made in more recent years;
• Research in gerontology is often undertaken through multi-disciplinary academic and professional teams, which can be used to develop different routes to impact. Anthea talked about links with architects, Occupational Therapists, economists and engineers; whereas Julienne highlighted more the links with policy and practice.
• Impact case studies go through many drafts and how evidence for impact is derived and presented is important. It can take time to find local, national and international evidence. The importance of record-keeping for on-going evidence of change linked to research was discussed.
• Both Jeremy and Des talked about their links with central and local government, policy and practice. Jeremy spoke of Anthea’s influential work in relation to the Royal Commission on Long Term Care, and its subsequent impact on policy initiatives such as the Long Term Care Revolution, Lifetime Homes, Lifetime Neighbourhoods and, more recently, Housing our Ageing population: panel for Innovation. Des was the original Founder of My Home Life (MHL) wanting to change negative public and professional perceptions of this sector. He was actively supporting the training mission of My Home Life. Here the commitment of the stakeholders to developing change was clear, which researchers need to recognise.
• Knowledge Exchange, in preference to knowledge translation, was seen as crucial, Julienne talked about how the impact of her work was delivered through an interplay of research, enterprise and social action. There was seen to be a need to open up dialogue between researchers, policy makers and practitiotners across traditional borders using more creative and targeted methods of communication. MHL had a clear vision: being evidence-informed, relationship-centred, appreciative, and focused on action. The team tried to ‘walk the talk’, when promoting and co-creating how to act on this vision through its research, leadership support and influencing work.
• Anthea also reported that each of the projects she discussed produced a short ‘Findings’ type document that was integral to their outputs. Researchers need to cost such dissemination within their proposals – it should not be an afterthought.
• Anthea mentioned a valuable report by Sasse and Haddon (2018) concerning working with government.
The programme then turned from specific impact case studies to considering how a programme of research can support researchers in their ability to develop their skills and make an impact. Introduced by Anthea Tinker (Deputy Chair of the NDA) consideration was given to the New Dynamics of Ageing (NDA) Programme of research between 2005/6-15 funded by five research councils. Two evaluations took place: one relating to the impact on policy, practice, business and other research users (National Development Team for Inclusion (NDTI, 2014), and the second in relation to Academic quality and impact (Holland, Gwyther, Griffiths, Nabney and Tritter, 2017). The ESRC have only put the Executive Summary of the latter online and the full NDA review can be obtained through Holly Gwyther -email@example.com . It will also appear on ResearchGate.
Professor Carol Holland (University of Lancaster) commented on both studies and made these observations regarding impact development from the NDTI review which related closely to discussion in the first session:
• Stakeholder engagement needs to be strategic and embedded from the outset: building support and ownership, ensuring outputs and messages are relevant
• Academic-practitioner partnerships are most effective where they’ve developed over time with increasing mutual understanding, rather than just set up for that project
• Outputs (publications or events) can make an important contribution to impact but as a starting point, not an end in themselves, generating interest among stakeholders and opportunities for further impact work.
• Older people played a key role in maximising impact: challenging also assumptions, raising the profile of older consumers, acting as powerful advocates for change
• Synchronicity and happenstance. Not just luck; rapid, assertive action is required to exploit opportunities
In continuing to regard academic quality and impact she was able to show how quality ratings of 3* and above were seen for project outcomes and that there was little difference in citations across the five research council focused projects. Multidisciplinary collaboration was important and some projects managed to combine non-academic impact and quality academic outputs. The NDA programme was seen to build capacity and to have enduring influence which saw the inclusion of older adults in research on ageing that was just developing at this time, as well as the development of multidisciplinary and interdisciplinary working with influence on the development of European research strategies. While ‘issues of timing, funding duration (or availability of follow on funding) and researcher continuity had a bearing on the variable number of academic outputs available’, the importance of multidisciplinary programmes of ageing research was seen as a crucial development.
Tessa Harding, a member of the NDA Advisory Group was able to verify some of Carol’s comments and spoke of the important development of ‘The NDA Handbook’ (Harding, 2015) which gave a summary of the key findings from the programme that had been packaged for a wide audience,especially older people themselves. She had worked with the Older People’s Reference Group to distil the essence of each piece of research. The Handbook has the potential to be a key impact tool. Older people need to be involved from the earliest stages in any research and should have a role in defining what needs to be researched. They had felt at various points of the programme that people were speaking a different language to themselves but they had been able to come face to face with the researchers on the projects. They were keen to see practical outcomes.and value for money. She reminded the audience that older people are often skilled communicators and campaigners with the advantage of having first hand experience, which makes a strong contribution.
In the short discussion that followed we saw the value of the over-arching supportive leadership of Professor Alan Walker and his team at Sheffield University. This had enabled researchers across the UK to not only meet together but also to find ways of accessing other networks. This was true for researchers at all stages of their career.
In the afternoon, the meeting turned to the views of those who are committed to making change happen and whom academic researchers often approach as co-applicants or supporters of their work. What is their view when this happens? How are partnerships made between different types of stakeholder and academic researchers?
First, members of three NGOs – Catherine Foot, Director of Evidence, Centre for Ageing Better, Phil Ambler, Director of Evidence and Policy, Thomas Pocklington Trust and Sally West, Policy Manager, Age UK – gave their views on how they operate.
Catherine said that they are a ‘what works’ centre and do things in partnership with many different ways of collaborating with researchers, practitioners and policymakers. Ageing Better tries to be crystal clear as to what they are trying to do, spending time looking at barriers to change, and developing different theories of change. Research projects are therefore developed with a clear intent behind them for how they will support the organisation to achieve change. Evidence may be gathered through pre-existing and new reviews of the literature, secondary data analysis, qualitative research and consulting front line practice, using a wide range of methodologies appropriate to the research questions. They test the ‘so what?’ message recognising that recommendations from research need to be made deliberately and collaboratively. To date the research projects they have supported have been relatively small scale. They are looking at commissioning larger and longer pieces of research in responsive mode or work with a consortia of research organisations alongside the one-off small group projects.
Sally commented on behalf of Age UK with contribution from Libby Archer (Senior Research Manager). Researchers turn to them because they work with and for older people as a large national Charity with contacts with politicians, policy developers and the media. Academics are keen to use Age UK as an organisation that disseminates but also helps with recruitment. They do not offer direct research funding but for those who wish to liaise with them they have a process outlined on their web-site that will guide you through gaining access. Sally recommended that researchers wishing to partner with Age UK, should actively involve them in the early stages of the research development rather than approach them with a ‘next week’ deadline. She represents central Age UK and reminded people also to think locally as local Age UK Groups may be able to be involved in project development and support. Charities don’t always have the capacity to work with researchers but are keen to do so where they can, and where projects link to their priorities.
Phil set out his case as Director of Evidence and Policy for an organisation involved with people with vision impairment of all ages. He commented on how they seek to enable people and influence others who need this knowledge. He asked researchers to consider ‘what is the change that you are trying to make through this research?’ Talk to people who have vision impairment don’t assume you know their experience. Language and accessibility are important. Phil mentioned a particular project where research concerning care homes and issues of sight impairment had been turned into a funded toolkit which, rather like the work of My Home Life, was linked to training sessions.
In the discussion that followed, the idea of training Early Career Researchers (ECR) through apprenticeships with NGOs was considered. Internships and swapping jobs was also seen as having potential. One of the speakers suggested that it is always useful to get academics to try to discuss their research in lay terms in 3 minutes. It really helps researchers clarify their aims and objectives. They returned to the ‘so what?’ discussion and questioned ‘so what and for whom?’ – and ‘making a chance for whom? You have to understand the context of all – those developing services may have a different perspective to end users.
Caroline Kenny from the Parliamentary Office of Science and Technology, was unable to join the event due to Parliamentary work that had to take precedence. Fortunately Professor Norah Keating, University of Swansea, Wales; University of Alberta, Canada; and North-West University, South Africa agreed to add global perspective on impact. She stressed the importance of understanding how challenges of population ageing are understood across diverse contexts. Building international credibility and potential for impact requires involvement with key international stakeholders in ageing including researchers, government and private sector policy makers and international NGOs such as the World Health Organization (WHO). She suggested 3 pathways to global impact: present academic work at international conferences where research links can be fostered; seek interaction with key NGOs who are addressing issues to which you can contribute; get involved in international professional organisations. She recommended the IAGG–EU meetings in Gothenburg Sweden in May 2019 as one starting point for making some of those connections.
Finally, we turned to the REF process itself and to three members of the BSG who have, and will be, intimately involved with REF 2021. Interdisciplinary research has been picked up as important in the next REF and each sub-panel UoA will have an interdisciplinary advisor. For the following subpanels they are:
• UoA Social Work and Social Policy, Professor Roger Burrows, Newcastle University;
• UoA Sociology, Professor Jane Elliott, University of Exeter and
• UoA Allied Health Professions, Dentistry, Nursing and Pharmacy, Professor Yvonne Barnett, Nottingham Trent University.
There is a central Interdisciplinary Research Advisory Panel (IRAP) and past President of the BSG, Professor Judith Phillips from University of Stirling is a member. As she was not able to join us in person, Judith sent a short video indicating that ways of working for this group are still being developed as is liaison between UoAs and IRAP. An article in The Times Higher (Willis, Oct.12th, 2017) suggested that disciplinary diversity, methodological clarity and adventurousness might be topics to be discussed.
Brief comments were made by Professor Alison Bowes, a member of the sub panel for UoA Social Work and Social Policy in REF 2014, also from University of Stirling and Professor Alisoun Milne, University of Kent, of UoA Social Work and Social Policy 2021. Points raised included:
• Sub panels are meant to be supportive of those being assessed;
• Impact represents a greater proportion of the overall weighting in 2021 and evidence in support of impact needs to be clear and robust;
• Continuing case studies will have to show how impact has continued to be developed;
• Teaching can be considered in terms of impact but it needs to fulfil the same criteria as any other sort of impact (see REF 2021 Draft Guidance on Submissions, 2018, p19) and be supported by evidence;
• The REF process is a coherently and tightly managed process;
• Double-weighting is encouraged – where an output e.g. a book might count as two rather than one output. It is up to the HEI to identify outputs as double weighted and can offer a ‘reserve’ output if the panel consider that the item does not qualify as double weighted. The sub panel cannot identify an output as double weighted if the HEI does not do so.
The final discussion clarified that guidance for REF 2021 is being finalised; it will be pubished early in 2019. HEIs will be holding mock REF submissions to enable their own sub groups to assess their contributions and talk with staff about their own positions. Impact case studies will be written and re-written. This event indicated that the BSG as the central Learned Society concerned with social gerontology has a role supporting its members during this assessment period. Also, post REF 2021 consideration should be given once again to analysing the outcome for ageing research indicating the strengths and weaknesses revealed through this process. Alisoun, Judith and Sheila have suggested capturing gerontologcal research ‘presence’ including Impact Case Studies post REF 2021.
Note: Several powerpoint presentations given at the event are available online via https://www.britishgerontology.org/events-and-courses/bsg-events/bsg-age-uk-pathways-to-impact. Access is also given to the two 4* case studies discussed in the first session.
Bangar,S., Hargreaves,S & Mountain,G (2015) The Impact of Ageing Research within the Research Excellence Framework 2014: an evaluation. University of Sheffield, British Society of Gerontology
Harding,T (2015) The NDA Handbook, A Summary of the key findings from the New Dynamics of Ageing Research Programme, The University of Sheffield. ISBN 978-0-9932547-0-3
Holland,C Gwyther,H Griffiths,H Nabney,I and Tritter,J, (2017). https://esrc.ukri.org/files/research/research-and-impact-evaluation/evaluation-of-the-new-dynamics-of-ageing-programme-executive-summary/
National Development Team for Inclusion (NDTI) (2015) Evaluating the Impact of the New Dynamics of Ageing Research Programme, ESRC. https://esrc.ukri.org/files/research/research-and-impact-evaluation/evaluating-the-impact-of-the-new-dynamics-of-ageing-research-programme-final-report/
REF 2021(REF2018/01) Draft Guidance on Submissions, July 2018
Sasse,T & Haddon,C (2018) How Government Can Work with Academia, London: Institute for Government, June 2018-12-10
Willis,M (2017) How will interdisciplinarity be assessed in the REF?, Times Higher Education, Oct 12th