EVALUATION & IMPACT

LIANZA is delighted to offer the library sector an opportunity to gain confidence and capability in evaluation research and telling the story of New Zealand libraries.
This initiative is supported by the NZ Libraries Partnership Programme

Libraries collect membership, issues and other data, but may be less confident in collecting evidence and using it to tell the story of their impact. This evaluation and impact initiative aims to support participants to work out what exactly needs to be monitored, how to measure what matters and ways to communicate effectively about the impact your library makes in your community. Rather than taking a ‘build it and they will come’ approach, evaluation and research will enable libraries and GLAM institutions to understand the wants, needs and motivations of your users and potential users, and to work out where’s best spend your time and money to grow and support your ‘audience’.

LIANZA has offered an evaluation and impact professional development of four workshops and a nine-month community of practice, led by Marge Ainsley, a cultural consultant in the UK with specialist expertise in research and evaluation.

Applications were received by September 14 and 69 participants in three cohorts were included in the initiative. The first cohort began on September 24.

PARTICIPANTS

COHORT ONE COHORT TWO COHORT THREE
Name Library Name Library Name Library
Annalese Weber Hamilton City Libraries Kareen Carter Wellington Medical & Health Sciences Library Bernie Hawke Dunedin Public Libraries
Bonnie Sarkar Christchurch City Libraries Paula Broekhuizen Hauora Tairawhiti Clinical Library Deborah Gordon Kapiti Coast District Libraries
Emma Sherie Buller District Libraries Rebecca Dames Waikato DHB Medical Library Nicola Smith Far North District Libraries
Emma Waywyn Napier Library Dyane Hosler Puke Ariki Kay Huia Hamilton City Council
Geoffrey Lush Parliamentary Library Nicole Noldus Hutt City Libraries Hamish Lindop Auckland Libraries
Heather Furniss Matamata-Piako District Libraries Andrew Henry Auckland Libraries Lalita Sudhakar South Waikato District Libraries - Tokoroa Public Library
Jane Thomsen Hewitson Library Presbyterian Research Centre Vicki Woodrow Clutha District Libraries Laura Caygill Waimakariri Libraries
Jennifer Hutchinson Thames Coromandel District Council Senga White Invercargill City Libraries Jo Cocker Auckland Libraries
Jill Watson Ashburton Public Library Alice Cruickshank Christchurch City Libraries Ebony Curtin South Waikato District Libraries
Joanna Boyd Far North District Libraries Lisa Kemp Auckland Libraries Saskia Hill Cashmere High School Library
Katie Brazil Christchurch City Libraries Rob Baigent Hamilton City Libraries Kylie Smith Rosehill College Library
Krystel Woodcock Westland District Library Lillian Pak Hutt City Libraries Miriam Tuohy National Library
Laura Findlay Nelson Public Libraries Jennifer Kirkwood Waimakariri Libraries Jacinda Boivin University of Otago
Nicola Stedman NZ Law Society Library Ngaio Hughey-Cockerell Waimakariri Libraries Susan Eady Unitec Library
Paul Tudor Tonkin & Taylor LTD Library Fiona Kerr Waitaki Libraries Paula Eskett Waimakariri Libraries
Philippa Robinson Auckland City Art Gallery Victoria Passau Auckland War Memorial Museum Kirstin Kane Auckland Libraries
Rachel Watson Dargaville Public Library Helena Westwick Mental Health Foundation of NZ Jeanne Clifton Auckland Libraries
Rebecca Keenan Dunedin Public Libraries Lily Phillips Parliamentary Library Zayna Hussein Auckland Libraries
Reesha Ranchod Hutt City Libraries Simon Wetherill Walsh Memorial Library MOTAT Debbie Williams Central Hawkes Bay
Steven Harley Nelson Public Libraries Jennifer Klarwill Department of Corrections Esther Casey Sylvia Park School
Suani Nasoordeen Auckland Libraries Iris Zhu Auckland University of Technology Judith Porter Environmental Protection Authority
Vicki Darling Central Otago District Libraries Rhea Fohs Upper Hutt Libraries Pamela Dais Tararua District Library
Kate Reynolds Wintec Library Tom Avery Open Polytechnic
Chelsea Renshaw Walsh Memorial Library MOTAT Angela Jowitt Puke Ariki Libraries
Anna Hodson Tauranga Library

FUNDING & COMMITMENT

Funding
With the support of the NZ Libraries Partnership Programme, LIANZA has funded all participants to attend the four workshops and engage in the nine-month community of practice.

Commitment
Library managers were required to indicate their commitment to supporting their participant with release time to engage in the workshops and community of practice, undertake an evaluation project and to share their learning with the library sector.

All applicants wererequired to send in an application detailing why they wish to attend and to indicate their commitment to engaging in the initiative, including sharing their learning at a LIANZA regional hui whakamahi in September, 2021 and at the LIANZA Conference in November, 2021.

Marge Bradshaw - Photo by Chris Payne

FACILITATOR

Marge Ainsley
Marge is a well-respected cultural consultant in the UK with specialist expertise in research and evaluation. She has 19 years’ experience working within the museums, libraries and archives sector.

A member of the Market Research Society, she was previously voted one of the 50 best freelancers in the UK by marketing magazine, The Drum. Marge launched her consultancy in 2008 and over the last twelve years as a consultant she’s worked with numerous arts organisations across the UK including Manchester International Festival, National Museums Liverpool, Cumbria Archives, Cheshire Archives, Signal Film & Media, Manchester Museum, Manchester Art Gallery, The Whitworth, Dales Countryside Museum, Macclesfield Museums Trust, West Cheshire Museums, Silverstone Heritage Experience (Silverstone Circuits Ltd), Scottish Book Trust, St Helens Libraries,  London Transport Museum, Arts Council England and Fountains Abbey (National Trust, Ripon).

Marge is an active member of the museums, libraries and archives community, whether that’s sharing skills, and fostering innovative ideas through publications such as Public Library News or by disseminating evaluation findings at sector conferences e.g. Research Libraries UK / The National Archives DCDC, which she most recently presented at in November 2019. She is also a volunteer ‘critical friend’ of the programming committee at The Portico Library in Manchester.

http://margeainsley.co.uk/
Photo credit: Chris Payne

WORKSHOP OUTLINES

All workshops are being delivered using Zoom for 1 – 2.5 hours including a break and plenty of time for Q&A. The workshops will be recorded (with permission) and shared with participants afterwards, along with copies of slides or handouts. The sessions will be as interactive as possible, including the use of break out rooms for group tasks where appropriate.

Workshop One: First steps in evaluation: a whistle-stop tour of evaluation
This introductory 60-minute presentation and Q&A is designed to show that evaluation and monitoring isn’t boring!

Through drawing on her own experience working across the cultural sector, Marge will bring inspiring examples and case studies to introduce the basics of monitoring and evaluation so that participants leave feeling invigorated about, and confident in getting started with this area of work.

By the end of the session, you will:

  • Understand the difference between monitoring, evaluation and research (demystifying the jargon that often surrounds this topic)
  • Understand why monitoring and evaluation is important and how to keep it realistic based on your time and capacity
  • Understand the importance of an evaluation framework and the various options available to produce one
  • Be inspired by creative approaches to quantitative and qualitative methodologies
  • Be reminded about the ethical considerations when undertaking evaluation work
  • Reflect on the opportunities and challenges of evaluation in the sector at the moment
  • Leave with links to further resources, tools and tips for further study in this area.

Workshop Two: How to develop an evaluation framework logic model
In this two-hour workshop we’ll focus on how to produce an evaluation roadmap for a project. Using a logic model format, the session will give participants a clear, straightforward process to follow along with desktop tools and templates to use after the session. Reporting frameworks such as the New Zealand Local Government Community Well-being and the UN 2030 Sustainable Development Goals may be drawn on to shape evaluations plans developed using the logic model.

By the end of the session, you will:

  • Learn how to create a robust monitoring and evaluation framework for projects
  • Find out how to embed ‘agile’ evaluation within an organisation or project team’s workload – to ensure your plan doesn’t sit on a shelf and responds to changing circumstances or needs of the audience/organisation
  • Understand the definition of aims, activities, outputs, outcomes, indicators and data collection tools and why it’s important to be clear on each term
  • Find out more about how to develop outcomes using frameworks such as Inspiring Learning For All’s Generic Learning Outcomes
  • Think more broadly about the range of quantitative and qualitative methodologies available, focusing on measuring what matters
  • Leave with a range of tools, templates and tips to enable you to produce your logic model.

Workshop Three: An introduction to quantitative and qualitative research instruments in the industry
It’s not all about surveys and focus groups. In this two-hour workshop Marge will go into more detail about data collection tools, inspiring participants to think differently about how they can measure what matters. We’ll assess the robustness or otherwise of available quantitative and qualitative tools; and discuss the ethical considerations when using them. Marge has a particular interest in inclusive tools and online qualitative methods so there will be a focus on these.

By the end of the session, you will:

  • Understand how to choose the best tools based on the needs of your evaluation framework
  • Understand the pitfalls and mistakes to avoid when designing surveys
  • Be inspired by creative participatory methods being used in the sector and beyond – and be confident in assessing their robustness or lack of robustness
  • Find out about the latest online qualitative research approaches which are used in the wider market research industry, including asynchronous and synchronous designed methods
  • Understand options available for analysis, especially qualitative data
  • Leave with resources and further reading, including for health and wellbeing evaluation tools.

Workshop Four: Using data strategically within audience development
Data is gold dust for audience development planning and marketing. Indeed, understanding your existing and potential audiences and key stakeholders is essential for communicating the 'story' of your library. In this two-hour workshop you’ll learn how organisations can use quantitative and qualitative data strategically for audience development planning. Using a real-world case study, Marge will walk through the process used and will give practical tips to increase confidence and ability in using data.

By the end of the session, you will:

  • Understand the types of data organisations can collect to support their audience development planning process
  • Learn how segmentation works and how marketers use tools like Audience Spectrum and Culture Segments for audience development planning.
  • Gain confidence in understanding how to work out who your existing and potential audiences are
  • Understand the requirements of an audience development plan and what we mean by ‘audience development’.

Community of Practice Hui
A community of practice will support participants from both cohorts to put their learning into action.

The purpose of the community of practice is to test and trial evaluation ideas and activity in a safe space, and to foster shared learning and networking opportunities through regular reflection workshops.  Monthly hui will be an opportunity for the participants to reflect and discuss issues, share ideas collectively and share learning and will be supported by Ana Pickering, LIANZA Executive Director.

Marge will facilitate five bi-monthly hui which will have a defined thematic area with ‘homework’ to keep the impetus and motivation levels high and to add accountability for learning. The sessions will directly link to the four workshops and each participant will choose an evaluation project for their library and follow an evaluation process to put learning into practice. Hui will be held monthly to allow adequate time for piloting and ‘doing’, with check-ins and dialogue with Marge and other participants in-between via an online support channel.

EVALUATION & IMPACT DELIVERY


  Cohort One Cohort Two Cohort Three
Workshop One First steps in evaluation Thursday September 24, 9-10am Friday October 16, 9-10am Thursday November  12, 9-10am 

 

Workshop Two How to develop an evaluation framework logic model Wednesday October 21, 9-11am Friday November 27, 9-11am Thursday December 3, 9-11am

 

Workshop Three An introduction to quantitative and qualitative research instruments Thursday November 19, 9-11am Wednesday December 9, 9-11am Thursday January 14,  9-11am

 

Workshop Four Using data strategically within audience development Wednesday December 16, 9-11am Friday January 22, 9-11am Thursday February 4, 9-11am

 

Community of practice   Starts early February Starts mid February Starts late February
Community of Practice Hui Tahi Choosing a project focus for evaluation + reminder of how to create a framework February, 2021 February, 2021 February, 2021
Community of Practice Meetup   March, 2021 March, 2021 March, 2021
Community of Practice Hui Rua Reporting back on framework + prototype methods/tools April, 2021 April, 2021 April, 2021
Community of Practice Meetup   May, 2021 May, 2021 May, 2021
Community of Practice Hui Toru Reporting on piloting/prototyping of tools + analysis tools June,  2021 June,  2021 June,  2021
Community of Practice Meetup   July, 2021 July, 2021 July, 2021
Community of Practice Hui Wha Reporting back on data and insights  + turning analysis into something meaningful & creative ways to share your story August, 2021 August, 2021 August, 2021
Hui Whakamahi - sharing learning Sharing learning at LIANZA regional hui whakamahi September, 2021 September, 2021 September, 2021
   

Reporting back on findings/shared story + thinking about how to use the data strategically.

Celebration: what you’ve learned and how you’ll use the data going forward

 

October, 2021 October, 2021 October, 2021
LIANZA Conference 2021 - sharing learning Sharing learning at LIANZA 2021 Conference, Wellington November, 2021 November, 2021 November, 2021