EVALUATION & IMPACT

LIANZA is delighted to offer the library sector an opportunity to gain confidence and capability in evaluation research and telling the story of New Zealand libraries.
This initiative is supported by the NZ Libraries Partnership Programme

Libraries collect membership, issues and other data, but may be less confident in collecting evidence and using it to tell the story of their impact. This evaluation and impact initiative aims to support participants to work out what exactly needs to be monitored, how to measure what matters and ways to communicate effectively about the impact your library makes in your community. Rather than taking a ‘build it and they will come’ approach, evaluation and research will enable libraries and GLAM institutions to understand the wants, needs and motivations of your users and potential users, and to work out where’s best spend your time and money to grow and support your ‘audience’.

LIANZA has offered an evaluation and impact professional development of four workshops and a nine-month community of practice, led by Marge Ainsley, a cultural consultant in the UK with specialist expertise in research and evaluation.

Applications were received by September 14 and 69 participants in three cohorts were included in the initiative. The first cohort began on September 24.

EVALUATION & IMPACT DELIVERY

  Cohort One Cohort Two Cohort Three
Workshop One First steps in evaluation Thursday September 24, 9-10am Friday October 16, 9-10am Thursday November  12, 9-10am 
Workshop Two How to develop an evaluation framework logic model Wednesday October 21, 9-11am Friday November 27, 9-11am Thursday December 3, 9-11am
Workshop Three An introduction to quantitative and qualitative research instruments Thursday November 19, 9-11am Wednesday December 9, 9-11am Thursday January 14,  9-11am
Workshop Four Using data strategically within audience development Wednesday December 16, 9-11am Friday January 22, 9-11am Thursday February 4, 9-11am
    Cohort One Cohort Two Cohort Three  
Community of Practice Hui Tahi Introduction and ground rules for the CoP.

Inspire & reflect

Sharing mini-evaluation or research plans (pre-task) in small groups

Ideate

Data collection tools prototyping in small groups

Weds 10th Feb 9am – 10.30am Weds 17th Feb 9am – 10.30am Weds 24th Feb 9am – 10.30am  
Community of Practice Hui Rua Inspire & reflect

Sharing prototyping experiences in small groups

Ideate

Second prototyping sprint + data analysis method

Tues 9th Mar 9am – 10am Tues 16th Mar 9am – 10am Tues 23rd Mar 9am – 10am  
Community of Practice Hui Toru Inspire & reflect

Sharing second prototyping in groups + how are you analysing the data and creating insight (what is the data you’re collecting telling you?)

Ideate

How will you tell your story and share your data insight

Weds 14th April 9am – 10am Weds 21st April 9am – 10am Weds 28th April 9am – 10am  
Community of Practice Hui Wha Inspire & reflect

Sharing data insight and storytelling progress in small groups

Full group reflection on learning so far.

Weds 9th June 9am – 10am Weds 16th June 9am – 10am Weds 23rd June 9am – 10am How have you turned your insight into something useful?

What tools are you trying out for this?

What support do you need from others in the group?

Celebration event Celebration and reflection event Weds 7th July 2021 9am – 10am

 

Weds 14th July 9am – 10am Weds 21st July 9am – 10am Sharing of stories with each other (lightning talks).  To what extent  have you learned something new through this programme?

What’s next?

Did anything disappoint you  with the programme or  was missing?

What would you  have liked more  of?

What have  been the blockers to success during  the CoP?

How?

Hui Whakamahi - sharing learning at LIANZA regional hui

 

All cohorts present an aspect of learning at a LIANZA regional hui. Details to follow. Sept 2021 (date tbc)      
LIANZA Conference 2021 - sharing learning

 

Sharing learning in-person or virtually at LIANZA 2021 Conference, Wellington, details to follow. Nov 9-11, 2021 (date tbc)      

FUNDING & COMMITMENT

Funding
With the support of the NZ Libraries Partnership Programme, LIANZA has funded all participants to attend the four workshops and engage in the nine-month community of practice.

Commitment
Library managers were required to indicate their commitment to supporting their participant with release time to engage in the workshops and community of practice, undertake an evaluation project and to share their learning with the library sector.

All applicants wererequired to send in an application detailing why they wish to attend and to indicate their commitment to engaging in the initiative, including sharing their learning at a LIANZA regional hui whakamahi in September, 2021 and at the LIANZA Conference in November, 2021.

Marge Bradshaw - Photo by Chris Payne

FACILITATOR

Marge Ainsley
Marge is a well-respected cultural consultant in the UK with specialist expertise in research and evaluation. She has 19 years’ experience working within the museums, libraries and archives sector.

A member of the Market Research Society, she was previously voted one of the 50 best freelancers in the UK by marketing magazine, The Drum. Marge launched her consultancy in 2008 and over the last twelve years as a consultant she’s worked with numerous arts organisations across the UK including Manchester International Festival, National Museums Liverpool, Cumbria Archives, Cheshire Archives, Signal Film & Media, Manchester Museum, Manchester Art Gallery, The Whitworth, Dales Countryside Museum, Macclesfield Museums Trust, West Cheshire Museums, Silverstone Heritage Experience (Silverstone Circuits Ltd), Scottish Book Trust, St Helens Libraries,  London Transport Museum, Arts Council England and Fountains Abbey (National Trust, Ripon).

Marge is an active member of the museums, libraries and archives community, whether that’s sharing skills, and fostering innovative ideas through publications such as Public Library News or by disseminating evaluation findings at sector conferences e.g. Research Libraries UK / The National Archives DCDC, which she most recently presented at in November 2019. She is also a volunteer ‘critical friend’ of the programming committee at The Portico Library in Manchester.

http://margeainsley.co.uk/
Photo credit: Chris Payne

WORKSHOP OUTLINES

All workshops are being delivered using Zoom for 1 – 2.5 hours including a break and plenty of time for Q&A. The workshops will be recorded (with permission) and shared with participants afterwards, along with copies of slides or handouts. The sessions will be as interactive as possible, including the use of break out rooms for group tasks where appropriate.

Workshop One: First steps in evaluation: a whistle-stop tour of evaluation
This introductory 60-minute presentation and Q&A is designed to show that evaluation and monitoring isn’t boring!

Through drawing on her own experience working across the cultural sector, Marge will bring inspiring examples and case studies to introduce the basics of monitoring and evaluation so that participants leave feeling invigorated about, and confident in getting started with this area of work.

By the end of the session, you will:

  • Understand the difference between monitoring, evaluation and research (demystifying the jargon that often surrounds this topic)
  • Understand why monitoring and evaluation is important and how to keep it realistic based on your time and capacity
  • Understand the importance of an evaluation framework and the various options available to produce one
  • Be inspired by creative approaches to quantitative and qualitative methodologies
  • Be reminded about the ethical considerations when undertaking evaluation work
  • Reflect on the opportunities and challenges of evaluation in the sector at the moment
  • Leave with links to further resources, tools and tips for further study in this area.

Workshop Two: How to develop an evaluation framework logic model
In this two-hour workshop we’ll focus on how to produce an evaluation roadmap for a project. Using a logic model format, the session will give participants a clear, straightforward process to follow along with desktop tools and templates to use after the session. Reporting frameworks such as the New Zealand Local Government Community Well-being and the UN 2030 Sustainable Development Goals may be drawn on to shape evaluations plans developed using the logic model.

By the end of the session, you will:

  • Learn how to create a robust monitoring and evaluation framework for projects
  • Find out how to embed ‘agile’ evaluation within an organisation or project team’s workload – to ensure your plan doesn’t sit on a shelf and responds to changing circumstances or needs of the audience/organisation
  • Understand the definition of aims, activities, outputs, outcomes, indicators and data collection tools and why it’s important to be clear on each term
  • Find out more about how to develop outcomes using frameworks such as Inspiring Learning For All’s Generic Learning Outcomes
  • Think more broadly about the range of quantitative and qualitative methodologies available, focusing on measuring what matters
  • Leave with a range of tools, templates and tips to enable you to produce your logic model.

Workshop Three: An introduction to quantitative and qualitative research instruments in the industry
It’s not all about surveys and focus groups. In this two-hour workshop Marge will go into more detail about data collection tools, inspiring participants to think differently about how they can measure what matters. We’ll assess the robustness or otherwise of available quantitative and qualitative tools; and discuss the ethical considerations when using them. Marge has a particular interest in inclusive tools and online qualitative methods so there will be a focus on these.

By the end of the session, you will:

  • Understand how to choose the best tools based on the needs of your evaluation framework
  • Understand the pitfalls and mistakes to avoid when designing surveys
  • Be inspired by creative participatory methods being used in the sector and beyond – and be confident in assessing their robustness or lack of robustness
  • Find out about the latest online qualitative research approaches which are used in the wider market research industry, including asynchronous and synchronous designed methods
  • Understand options available for analysis, especially qualitative data
  • Leave with resources and further reading, including for health and wellbeing evaluation tools.

Workshop Four: Using data strategically within audience development
Data is gold dust for audience development planning and marketing. Indeed, understanding your existing and potential audiences and key stakeholders is essential for communicating the 'story' of your library. In this two-hour workshop you’ll learn how organisations can use quantitative and qualitative data strategically for audience development planning. Using a real-world case study, Marge will walk through the process used and will give practical tips to increase confidence and ability in using data.

By the end of the session, you will:

  • Understand the types of data organisations can collect to support their audience development planning process
  • Learn how segmentation works and how marketers use tools like Audience Spectrum and Culture Segments for audience development planning.
  • Gain confidence in understanding how to work out who your existing and potential audiences are
  • Understand the requirements of an audience development plan and what we mean by ‘audience development’.

Community of Practice Hui
A community of practice will support participants from both cohorts to put their learning into action.

The purpose of the community of practice is to test and trial evaluation ideas and activity in a safe space, and to foster shared learning and networking opportunities through regular reflection workshops.  Monthly hui will be an opportunity for the participants to reflect and discuss issues, share ideas collectively and share learning and will be supported by Ana Pickering, LIANZA Executive Director.

Marge will facilitate five bi-monthly hui which will have a defined thematic area with ‘homework’ to keep the impetus and motivation levels high and to add accountability for learning. The sessions will directly link to the four workshops and each participant will choose an evaluation project for their library and follow an evaluation process to put learning into practice. Hui will be held monthly to allow adequate time for piloting and ‘doing’, with check-ins and dialogue with Marge and other participants in-between via an online support channel.

PARTICIPANTS

COHORT ONE COHORT TWO COHORT THREE
Name Library Name Library Name Library
Annalese Weber Hamilton City Libraries Kareen Carter Wellington Medical & Health Sciences Library Bernie Hawke Dunedin Public Libraries
Bonnie Sarkar Christchurch City Libraries Paula Broekhuizen Hauora Tairawhiti Clinical Library Pamela Dais Tararua District Library
Emma Sherie Buller District Libraries Helena Westwick Mental Health Foundation of NZ Nicola Smith Far North District Libraries
Emma Waywyn Napier Library Dyane Hosler Puke Ariki Kay Huia Hamilton City Council
Geoffrey Lush Parliamentary Library Nicole Noldus Hutt City Libraries Hamish Lindop Auckland Libraries
Heather Furniss Matamata-Piako District Libraries Simon Wetherill Walsh Memorial Library MOTAT Lalita Sudhakar South Waikato District Libraries - Tokoroa Public Library
Jane Thomsen Hewitson Library Presbyterian Research Centre Vicki Woodrow Clutha District Libraries Laura Caygill Waimakariri Libraries
Steven Harley Nelson Public Libraries Senga White Invercargill City Libraries Jo Cocker Auckland Libraries
Jill Watson Ashburton Public Library Alice Cruickshank Christchurch City Libraries Ebony Curtin South Waikato District Libraries
Joanna Boyd Far North District Libraries Iris Zhu Auckland University of Technology Saskia Hill Cashmere High School Library
Katie Brazil Christchurch City Libraries Rob Baigent Hamilton City Libraries Kylie Smith Rosehill College Library
Krystel Woodcock Westland District Library Lily Phillips Parliamentary Library Miriam Tuohy National Library
Laura Findlay Nelson Public Libraries Jennifer Kirkwood Waimakariri Libraries Jacinda Boivin University of Otago
Nicola Stedman NZ Law Society Library Ngaio Hughey-Cockerell Waimakariri Libraries Tom Avery Open Polytechnic
Paul Tudor Tonkin & Taylor LTD Library Chelsea Renshaw Walsh Memorial Library MOTAT Paula Eskett Waimakariri Libraries
Philippa Robinson Auckland City Art Gallery Jennifer Klarwill Department of Corrections Kirstin Kane Auckland Libraries
Rachel Watson Dargaville Public Library Jeanne Clifton Auckland Libraries
Rebecca Keenan Dunedin Public Libraries Zayna Hussein Auckland Libraries
Reesha Ranchod Hutt City Libraries Debbie Williams Central Hawkes Bay
Esther Casey Sylvia Park School
Lillian Pak Hutt City Libraries