EVALUATION & IMPACT

LIANZA is working this year with a wonderful cohort from across the library and information sector, who are keen to  gain confidence and capability in evaluation research to tell ‘the story’ of New Zealand libraries services and programmes.

This second LIANZA Evaluation and Impact initiative will run from March to October 2022. It will be led by Marge Ainsley, a cultural consultant in the UK with specialist expertise in research and evaluation.

In 2020, LIANZA provided evaluation and impact professional development through four workshops and a nine-month community of practice with Marge. Participants shared their work and experiences at six online hui and at the LIANZA conference in 2021. You can read more about their work and experiences in the LIANZA Evaluation and Impact 2021 Case Studies here

This initiative is supported by the NZ Libraries Partnership Programme.

TIMELINE

Four workshops will be held at fortnightly intervals from March to May. Participants will then engage in the monthly community of practice from May to September, sharing their learning in October at online hui whakamahi.

FUNDING

With the support of the NZ Libraries Partnership Programme, LIANZA is funding all participants to attend the four workshops and engage in the community of practice.

PROGRAMME

Workshop One Welcome to the programme! First steps in evaluation: getting on the same page and what to expect from our time together Tuesday, March 15
9.00am - 11.00am
Workshop Two How to develop an evaluation framework logic model Thursday, March 31
9.00am - 11.30am
Workshop Three An introduction to quantitative and qualitative research instruments Tuesday, April 26
9.00am - 11.30am
Workshop Four Analysing and interpreting your data: what is it telling you and how can you share your story? Tuesday, May 17
9.00am - 11.30am
Community of Practice Hui Tahi Welcome to the CoP! Meet the class of 2021 and get support on completing your evaluation framework or research plan Thursday May 26
9.00am - 10.15am
Community of Practice Hui Rua Reporting back on prototyping 1 and planning for prototyping 2 Thursday June 30
9.00am - 10.15am
Community of Practice Hui Toru Reporting back on prototyping 2 and thinking about analysis tools/what the data is telling you Thursday July 21
9.00am - 10.15am
Community of Practice Hui Wha Reporting on analysis tools and insight + sharing your story Tuesday August 30
9.00am - 10.15am
Community of Practice Hui Rima Celebration: what you’ve learned and how you’ll use the data going forward lightning talks Thursday September 22
9.00am - 10.15am
Hui Whakamahi - sharing learning Sharing learning at LIANZA online hui whakamahi October - TBC

WORKSHOPS

All workshops will be delivered using Zoom for 2.0 – 2.5 hours including a break and plenty of time for Q&A. The workshops will be recorded, with permission and shared with participants afterwards, along with copies of slides or handouts. The sessions will be as interactive as possible, including the use of break out rooms for group tasks where appropriate. Workshop content is tailored to meet the needs of participants, using information supplied by successful applicants in advance of the programme. Therefore, the  schedule below is a guide only.

WORKSHOP ONE: Welcome to the programme! First steps in evaluation: getting on the same page and what to expect from our time together

Getting to know your peers and an introduction to the programme. This 2.0-hour introductory hui and Q&A is designed to show that evaluation and monitoring isn’t boring!  Through drawing on her own experience working across the cultural sector, Marge will set the tone for the rest of the programme, introducing the basics of monitoring and evaluation so that participants confidently start the programme!

In this session we will:

  • get to know everyone in the cohort, where they’re from and their interests
  • introduce Slack, our platform for sharing our resources and learning with each other
  • demystify the jargon that often surrounds this topic
  • reflect on the opportunities and challenges of evaluation in the sector now
  • introduce evaluation principles and the difference between monitoring, evaluation, and research
  • give a flavour of what’s to come, including getting started with evaluation frameworks, qualitative and quantitative research tools, ethical considerations and more.
WORKSHOP TWO: How to develop an evaluation framework logic model

In this workshop we’ll focus on how to produce an evaluation roadmap for a project. Using a logic model format, the session will give participants a clear, straightforward process to follow along with desktop tools and templates to use after the session. Reporting frameworks such as the New Zealand Local Government Community Well-being and the UN 2030 Sustainable Development Goals may be drawn on to shape evaluations plans developed using the logic model.

By the end of the session, you will:

  • learn what goes in an evaluation framework and why it’s important to have one
  • understand the difference between an evaluation framework and an exploratory research plan
  • be clear on the definitions of aims, activities, outputs, outcomes, indicators, and data collection tools
  • get to grips with using a logic model template that you can implement in future projects and within the community of practice
  • think about the realities of embedding evaluation within an organisation’s existing workload to ensure that your plan doesn’t sit on a shelf
  • gain tips on non-user consultation and primary exploratory research
  • create a research plan
  • gain tips for getting buy-in around evaluation, monitoring and research
WORKSHOP THREE: An introduction to quantitative and qualitative research instruments in the industry

It’s not all about surveys and focus groups. In this workshop Marge will go into more detail about data collection tools, inspiring participants to think differently about how they can measure what matters. We’ll assess the robustness or otherwise of available quantitative and qualitative tools; and discuss the ethical considerations when using them. Marge has a particular interest in inclusive tools and online qualitative methods so there will be a focus on these.

By the end of the session, you will:

  • think about how an evaluation framework or research plan informs data collection tools
  • be inspired by various practical examples in the field of quantitative and qualitative methods
  • understand what to consider when selecting data collection tools and pitfalls to avoid
  • consider ethical considerations and safety in the time of COVID-19
  • think about the best ways to devise research questions and understand the role of the researcher in data collection
WORKSHOP FOUR: Analysing and interpreting your data: what is it telling you and how can you share your story?

In this session Marge will share how to use the data you’ve collected. She will provide tips on how to analyse quantitative and qualitative data, with a specific focus on how to interpret qualitative data. Using a live case study, Marge will share how data can be strategically used for audience development and will inspire you to consider how you can share your story beyond a written report.

By the end of the session you will:

  • understand where to start when analysing your quantitative and qualitative data
  • explore the different qualitative data analysis (QDA) methods available to you
  • understand the challenges that interpreting data can bring, such as unconscious bias
  • be inspired to use data collected for strategic audience development planning
  • gain new ideas for presenting your data and powerfully sharing your story
COMMUNITY OF PRACTICE HUI

A community of practice will support participants to put their learning into action.

The purpose of the community of practice is to test and trial evaluation ideas and activity in a safe space, and to foster shared learning and networking opportunities through regular reflection workshops. Marge will facilitate five community of practice hui which will each have a defined thematic area with ‘homework’ to keep the impetus and motivation levels high and to add accountability for learning. The sessions will directly link to the four training workshops and each participant will choose a project and follow an evaluation process to put learning into practice. Hui will be held monthly to allow adequate time for piloting and ‘doing’, with check-ins and dialogue with Marge and other participants in-between via the online support channel.

The Community of Practice is participatory, and peer led. Participants will share their learning and support others in a welcoming and supportive environment. It will be facilitated by Marge and the LIANZA team.

Marge Bradshaw - Photo by Chris Payne

ABOUT THE FACILITATOR - MARGE AINSLEY

Marge is a well-respected cultural consultant with specialist expertise in research and evaluation. A Member of the Market Research Society, she was previously voted one of the 50 best freelancers in the UK by marketing magazine, The Drum.

Since launching her consultancy in 2008, she’s worked with a range of libraries, museums, and archives across the UK and internationally on a range of projects which seek to evidence insight and impact. These have ranged from evaluation of major grants programmes through to delivering training and facilitation in monitoring, evaluation and research for organisations with limited in-house expertise.

Over the last fourteen years as a consultant she’s worked with numerous arts organisations across the UK and internationally including LIANZA, Manchester International Festival, National Museums Liverpool, Carnegie UK Trust, Cumbria Archives, Cheshire Archives, Signal Film & Media, Manchester Museum, Manchester Art Gallery, The Whitworth, Dales Countryside Museum, Macclesfield Museums Trust, West Cheshire Museums, Scottish Book Trust, London Transport Museum, Arts Council England and Fountains Abbey (National Trust, Ripon).

Marge is an active member of the museums, libraries, and archives community, whether that’s sharing skills and fostering innovative ideas through publications, or by disseminating evaluation findings at sector conferences.

Further details regarding Marge's approach, testimonials and a full client-base can be found on her website: http://margeainsley.co.uk/

Photo credit: Chris Payne