Measuring your impact effectively part 2

Do you have the right measurements in place to track against your goals? Learn how to measure, quantify and communicate your impact to prove your charity’s worth.

Organisations are no longer measured purely on their outputs but also on how they achieve social and environmental goals. We know that, for many of you, measuring and reporting on your impact can be quite daunting and complex.

So, throughout 2020 we’re pleased to offer a free, two part webinar on ‘Measuring the Difference Your Work Makes’ delivered by our expert partners at the Foundation for Social Improvement.

This two-part webinar series is open to all causes.

This is a recording of the second webinar held in September 2020. It provides an overview of evaluation approaches and will equip you with the skills to effectively demonstrate the impact of your organisation.

Please allow 90 minutes to watch the recording in full. If you’d like to view the first webinar you can find this here.

If you’d like to join an upcoming live presentation of this training with the opportunity to ask your own questions, look out for invitations via our Aviva Community Fund Facebook Group.

Measuring impact series: Considerations to be aware of when conducting your own research

Ethical considerations

All research should consider the wellbeing of the participants completing the research and the researcher / interviewer throughout the research process. This not only ensures the safety of all, but that participants take part giving their consent, understand what the research is about and are not mislead in any way. Some key principles that need top be abided by are as follows:

  • All participation should be voluntary
  • Informant consent should be obtained in all instances
  • No harm should come to participants or researcher
  • Ensure anonymity and confidentiality throughout
  • Transparency and not deceiving participants

The MRS (Market Research Society) Code of Conduct

The Market Research Society (MRS) is a professional body for the market research industry. They have devised a code of practice which all MRS members are obliged to follow and is also an excellent guide for those not working in the market research sector. For more information and for the full, official code of conduct, see

  1. GDPR (General Data Protection Regulation) in research

Data protection is extremely important in research, not only to abide by the law, but to protect sensitive and personal information from participants. Ultimately, to ensure participants feel comfortable completing surveys and uphold research credentials in the industry.

There are many considerations that need to be taken on board and implemented in research in order to be compliant with GDPR guidelines and regulations. The most important ones to note are:

  • Data collection
  • All participants need to give informed consent and made aware upfront if they will be asked sensitive topics or questions. If you want to invite your customers to take part in a survey, you can only reach out to them if they have given explicit permission to be contacted for these purposes. In addition to this, you will need to gain consent in the first wave of research to recontact participants again if you have any farther questions to ask them.
  • Ensure all data from participants, regardless of whether this is quantitative, qualitative, face-to-face or online, is anonymised and remains confidential unless specific consent is gained at the beginning of the research.
  • Data storage & transfer
  • Make sure all data, even when it is anonymised, is stored securely. If any personal information is obtained, it needs to be securely destroyed within sensible retention period.
  • If the participant contacts you and asks for their data to be deleted, you will need to quickly respond to them and delete their data from all sources as requested.

All research agencies should be abiding by these steps and have a variety of measures in place to ensure no data breaches happen.

Adapting research in accordance with target audience & other considerations

Research among children

If the research involves interviewing a child under 16, parental (or locus parentis) consent is required. For further information on points to consider see – point 16-22

Research among vulnerable people

The research design and process will need to be reviewed if research is carried out among vulnerable people. Those running the research will need to make sure that participants are able to give informed consent and careful consideration will need to think about their wellbeing during the research which may mean introducing specific safeguarding measures.   

For more information visit: – point 23-24

Research on sensitive topics

Before any research is carried out, consideration should be taken if the survey or interview covers any questions or areas that may be sensitive to others or cause embarrassment. Such topic areas could be, sex, health, religion, ethnicity etc.

To minimise this:

  • Make sure topic area is outlined at the beginning of the research and informed consent is gained before research commences
  • Potentially provide details of a confidential helpline telephone number and email address which participants can contact if they need support or information.
  • Reassure participants about the anonymity of their data
  • Make sure participants are not forced to answer any sensitive questions or provide demographic information if they feel uncomfortable doing so.

For more information visit:

Conducting pilot research and/or participatory research methods help to minimise some of the risks associated with these types of research. It is always good practice to reach out to market research specialists if you have any worries or concerns.

This article was kindly contributed by Hannah Kilshaw, Research Director at ICM Unlimited.

Measuring impact series: Different ways to carry out qualitative research

Ultimately, qualitative research is used to explore and understand a research problem (rather than quantitatively measure it). It focuses on attitudes, feelings, language and expression.

  1. Qualitative Research Approaches

Broadly speaking, there are three types of qualitative research….

A. Groups e.g. focus groups, co-creation workshops, online bulletin boards
B. Depth interviews e.g. one-to-one interviewing, in pairs or threes
C. Observation e.g. passive observation and ethnography

Groups and depth interviews can be conducted both online and face-to-face. As there are several techniques, here’s a brief overview of the most well-known and frequently used ones.

Focus Groups

A standard face to face focus group includes 6-8 participants per group. The attendees are normally recruited on certain criteria e.g. they all live alone for example. Focus groups are usually 90-120 minutes long. An online focus group will hold fewer participants but can still last an hour and a half.


• Range of views in a short space of time; highlights similarities and differences quickly
• More creativity and debate – participants spark off one another
• Can be observed by others from your organisation
• Can be relatively quick turnaround, and cost efficient


• Strong personalities in the group can dominate, affecting other’s willingness to talk or open up and be honest
• Group affect – can lose individual differences, in preference for group consensus
• Hot-housing – focusing too narrowly on an issue, losing perspective

Depth interviews

Depth interviews are often used when independent recollection is needed, for decision making processes, e.g. how they accessed your services, or for intimate and personal subjects. You can conduct 1-2-1 depth interviews or have paired / triad interviews. This will depend on the research aims, nature of the topic, locations, budget etc.


• More depth of information; more probing
• Less sense of embarrassment / more sensitivity
• Understand the individual story without fear of what a wider group think
• Useful for independent, individual experiences and where peer discussion isn’t needed


• Time-consuming – travel, conducting the interviews, analysis; therefore more costly per participant
• Less opportunity for creativity and interaction
• Sometimes one-to-ones are mistakenly viewed in a quantitative way


A typical qualitative observation is when data collection is usually a passive process where the researcher does not interact with the observed person. The researcher needs to remain objective when interpreting ambiguous behaviour. Qualitative observation can also involve other aspects of customer behaviour e.g. accompanying a participant whilst they use an aspect of your services, observing their behaviour, audio / video-recording as well as using a guide for discussion.


• Can reveal discrepancies between what people say and what they actually do
• Can be conducted as an observation therefore removing interviewer effect


• Subject to observer’s interpretation
• If observation only with no other interaction, reasons for behaviour can be missed

Creating a discussion guide

What is a discussion guide? Why is it useful, and typical length

A discussion guide is used to provide structure to the interview / group. Qualitative research should still be unambiguous, clear and concise, unbiased, and rigorous and reliable. This is so the research can be replicated by other interviewers or facilitators if need be.

The guide is used by the person leading the group to help direct the flow of questions (with prompts and probes as required) and allows the researcher to follow a consistent path across groups / depths.

Discussion guides can be creative, and a wide array of stimulus can be used, e.g.
• Existing materials, communications (websites, newsletters, email) or promotional materials (adverts, brochures). These are easiest to use.
• New ideas (concepts, pack designs, storyboards), which can be valuable to test before launch.

Moreover, projective techniques (replacing direct questions with inventive / creative ones) can be used in qualitative research and gives participants the opportunity to talk more openly about the subject. These techniques are particularly helpful when participants feelings are subconscious, irrational, too personal or embarrassing. An example of a projective technique is ‘personification’ – If our charity was a celebrity, who would they be?

Further reading & information
Additional information and guidance on qualitative research approaches can be found here:

  1. MRS events and conferences
  2. Yvonne McGivern. 4th Edition (2013). The Practice of Market Research: An Introduction
  3. Get in touch with a market research specialist

This article was kindly contributed by Hannah Kilshaw, Research Director at ICM Unlimited.

Measuring impact series: Designing your questionnaire

Questionnaire structure

The structure of a questionnaire is important to ensure it is clear and easy to complete, and that participants are guided through the questions in a chronological order.

Also consider the order of your questions as you want to minimise priming or framing topics to reduce bias if they are asked about later in the questionnaire.

A typical questionnaire will be structured as follows:

  • Screening section at the beginning of the survey will ensure you are talking to the right people i.e. your target sample
  • Main body of the questionnaire will cover the key questions you want to find out, starting broadly, then becoming more focused as the questionnaire progresses
  • Lastly, any final demographic information not essential in the screening section is gathered for profiling and analysis

Types of questions

The questionnaire should include different types of questions to achieve robust data and maintain participant engagement throughout the survey. Different question types include:

  • Structured questions (closed and open-ended)
  • Question styles e.g. yes/no, choose from list, agree/disagree, drag ‘n’ drop etc.
  • Open-ended questions – participants can freely type or say what they want to answer the question

Also think about question wording: questions should be clear, specific, and easy for every participant to answer.

Questionnaire length

Overall, you need to be mindful of the length of the questionnaire. An optimal online questionnaire will last no more than 10-15 minutes in length. Any longer than this tends to lead to respondent fatigue and potentially a higher level of drop out.

If a face-to-face method is used, the questionnaire length may need to be very short if intercepting people in the street. Conversely, a longer questionnaire may be acceptable and appropriate if you are completing an in-home face-to-face survey.

  1. Further reading & information

Additional information and guidance on quantitative research approaches can be found here:

  1. MRS events and conferences
  2. Yvonne McGivern. 4th Edition (2013). The Practice of Market Research: An Introduction
  3. Get in touch with a market research specialist

This article was kindly contributed by Hannah Kilshaw, Research Director at ICM Unlimited.

Measuring impact series: Different ways to carry out quantitative research

Quantitative research methods are typically used to measure how many people feel, think or act – essentially, they quantify how many people say ‘X’, do ‘Y’, think ‘Z’. Standard quantitative methods are polls or surveys. Here we look at some of the different ways to carry out quantitative research.

  1. Quantitative Research Approaches

There are two main ways quantitative research can be completed by the target sample, either by ‘self-completion’ or ‘interviewer administered’. The former means that the participant (a person who has agreed to take part in the research) completes the research themselves, whilst the latter means that an interviewer asks the questions and records the participants’ answers for them.

There is no ‘right’ or ‘wrong’ approach to quantitative data collection, and self-completion or interviewer administered techniques are chosen for several different reasons. Factors such as budget, timings, subject matter of the research and the target sample need to be considered.

Self-completion methods


An online survey is one that participants complete themselves on a device (e.g. laptop, tablet or mobile phone).


  • Is typically cheaper & faster than other methods
  • Large sample sizes can be achieved nationally and internationally
  • No interviewer bias
  • Stimuli such as videos, can be included in the survey


  • Potential sampling issues can occur –hard-to-reach participants can be difficult to identify. Consider whether your online survey is accessible by all recipients.
  • Technical issues – some online surveys may not open correctly on certain browsers or devices – carry out a quick test before sending if possible.

Using an interviewer:                                                                                                                 


A face-to-face survey can be carried out almost anywhere; in-home, at an event or by asking people on the street.


  • Rapport – theinterviewer can build rapport, which is important for sensitive research subjects
  • Targeted – you can very selective about geographical areas and/or demographics
  • High quality – has a reputation for delivering representative samples and high quality data.


  • Cost and time – additional costs, such as interviewers time, need to be accounted for and time completing the interviews can mean the project can be time-consuming.
  • Reach – can be difficult, or at least very costly, if you need to reach all regions/areas of the population.
  • Bias – the rapport and influence of the interviewer can generate bias



  • Sampling – localised geographical targeting can be achieved more easily. Reaching customers via telephone may be required if you’re lacking email addresses of those you want to survey.
  • Access – overcome geographical and technical barriers, no travel required.
  • High quality – has a reputation for delivering representative samples and high quality data.


  • Cost – additional costs, such as interviewer’s time and cost of the calls, need to be accounted for so tend to be more expensive than other self-completion methods.
  • Visual aid(s) not possible – participants will not be able to see stimuli such as videos

Measuring impact series: Conducting your own research – top tips

What to consider when carrying out some research

What is market research and why do it?

Simply, market research is the collection and analysis of data to uncover what people think (their attitudes or opinions) and what they do (their behaviour). It is used to produce quality information for planning and decision-making, using robust/credible evidence to ensure:

  • Better products and services
  • Better relationships with customers / beneficiaries / other stakeholders
  • Greater longevity for the organisation
  • Better quality decision-making and use of resources
  • More effective Government policies

How to decide on what research method is best?

Stage 1 – What do you want from the research?

  1. Define the research problem

‘The problem’ is a formal way of identifying the ‘core need’ for the research to be undertaken.

Before starting any research, or considering reaching out to any external research agencies to undertake the work for you, it is a good idea to be clear on the background (what triggered the need), the objective (how will the results impact your organisation) and what the research objective(s) are (what is the research actually trying to achieve).

For example, is it to help develop a service you provide? Is it to understand what your beneficiaries think of your organisation? Would you like to know more about the characteristics of a group of people?

  • What data / insight do you need?

Also ask yourselves, what do you need the research for? What type of data or insight do you need at the end of the research?

For example, are you looking for data from a small number of people that is qualitative in nature and may go into more detail about the ‘why’ or are you looking for data from a large number of people in which you can quantify their answers?

Stage 2 – Creating the research design

  • Do you need to conduct primary research?

Once you know the type of data you are looking for and what you want it to tell you, it is a

good idea to consider what research already exists. For example, is primary research (research you are conducting yourself and is not publicly available elsewhere) actually needed? Is there already sufficient data available internally in your organisation or publicly available? Is it feasible to undertake the primary research?

Once questions like these have been considered, you can go on to identify what research method is most appropriate and confirm what the research requirements are.

  • Who do you want to talk to?

Who in particular would you like to talk to – whether that be members of the general public, your beneficiaries, your employees and volunteers, other stakeholders etc.?

When this has been ascertained, you will need to think more about the sampling strategy i.e. who the target population is, how you can access this group of people (the sampling approach) and the sample size (the number of people you want to research).

  • Choosing the data collection method

Once you have thought about what it is you want to research, what you require from the data, who you want to talk to and how you would get access to this group of people, then you should have a clear idea of whether you want to conduct qualitative research or quantitative research.

Next steps are to think about whether you can conduct the research in-house or need support from an external research agency.

For more information on quantitative and qualitative research methods see our other articles.

Thanks to Hannah Kilshaw, Research Director at ICM Unlimited for contributing to this series of articles.

Using Design Thinking to come up with innovative new ideas

Design thinking as a specific process became prominent in the mid-20th century as designers of many different things began using it to think about the end user’s experience.

By the 1950s and 1960s, people who were designing homes, technology and consumer goods were all doing so by thinking about the people who would eventually use those things. This is the origins of modern design thinking.

Design Thinking is an iterative process in which we seek to understand the user, challenge assumptions, and redefine problems to identify alternative strategies and solutions that might not be instantly apparent with our initial level of understanding. It can be applied to anything – services, products, teams, organisations, events – and even just everyday life. And that’s why it’s great for charitable organisations because they often don’t focus on just one product or one service but a whole host of support mechanisms and relationships.

Adopting design thinking is a great way for charities to be innovative. Design thinking is human centred, as it focusses on the user, requiring empathy to really understand what is happening for them – and we know empathy is something you’ve got a lot of!

What are the Steps to Problem Solving this Way?

The key steps in design thinking are highlighted in the diagram below. Work from left to right starting with ‘Empathise’ through to Test’.

1. Empathise

It is imperative to really understand the underlying challenge. At this stage observing, engaging and empathising with people to understand their experiences and motivations, as well as immersing yourself in the physical environment so you can gain a deeper personal understanding of the issues involved. Understanding of the users, their needs, and any problems that may arise will help you with the development of any solutions to meet these needs.

2. Define (the Problem)

”If I were given one hour to save the planet, I would spend 59 minutes defining the problem and one minute resolving it.” Albert Einstein.

Using the information gathered in the ‘Empathise’ stage, define the core problems that you and your team have identified up to this point. You should seek to define the problem as a problem statement in a human-centred manner. The way to do this is as follows:

  • Who is your user? (Note as many specific details as possible.)
  • What is their deep, unmet need?
  • Why is this insightful? (List the insights you gleaned from your empathetic need-finding process.)

3. Ideate

Now’s the time to start identifying new solutions to the problem statement you’ve created, and start to look for alternative ways of viewing the problem. Typically techniques such as brainstorm, brainwrite, worst possible idea are used to stimulate free thinking. Try and generate as many ideas or problem solutions as possible at the beginning of the Ideation phase.

4. Prototype

Produce a number of inexpensive, scaled down versions of the solution, so you can investigate the possible solutions to the problem generated in the previous stage. This is an experimental phase, and the aim is to identify the best possible solution for each of the problems identified during the first three stages. The solutions are investigated and either accepted, improved and re-examined, or rejected on the basis of the users’ experiences.

5. Test

Now is the time to vigorously test the complete solution using the best approaches identified during the prototyping phase. As design thinking is an iterative process, the results generated during the testing phase may be used to reconsider some of the problem statements and how people think, behave, and feel.

6. Implement

Now you’ve identified through testing solutions what ideas have the best chance of succeeding, you decide which ones you want to expose to a wider audience. It’s over to you!

Final thoughts…

Has this article convinced you that the technique of design thinking can be applied in charitable organisations? If you’re interested in how others have used it successfully, why not have a look at IDEO – one of the leading organisations in design thinking. They’ve shared a number of case studies that highlight how they’ve used this technique to develop brilliant and innovative solutions for causes they’ve identified.

For further information, tools and resources visit here

Seven myths about innovation

It’s a reality that many charitable organisations operate in a competitive environment.

Pressure exists to expand into new audiences and deepen relationships with beneficiaries, as well as keeping up with the times and exploring new opportunities. There’s a lot of talk about the need for charities to innovate. Indeed, our own Community Fund asks for causes to submit a forward-thinking idea because we believe taking a risk and testing something new, can be a fantastic way to open up new opportunities and lead to a more confident future.

But we understand that innovation can at first feel risky, time consuming or out of reach. It’s often perceived as something only big charities have the freedom, funds and right people to do. In this article, one of Aviva’s Senior Innovation Managers, Richard, aims to highlight some of the common myths that exist about the term innovation to help you think about the word differently.

Myth 1: Innovation is identifying something brand new

Not necessarily. Innovation is about finding a better solution (than exists today), that adds value by meeting a new or unrealised need. It can occur anywhere in the charity e.g. engagement, creative, research, strategy process etc.

Myth 2: Innovation is having a lightbulb moment

Innovation is not just a fresh idea. Ideas, whilst part of the innovation mix, don’t necessarily meet a need or add value alone. Innovation is about borrowing and developing ideas to suit the situation and crucially meet a need and adds value. Innovation involves: iterating, testing, stakeholder buy-in and refinement to allow you to move from concept to delivery.

Myth 3: Innovation is all about using technology

It’s true that technology often enables or accelerates innovation e.g. by allowing easy access to new audiences, scaling processes or removing the need for large numbers of employees. It doesn’t, however, need to be a prerequisite for innovation success. Innovation is a lot wider – it can mean cultural shifts or new ways of working or new beneficiaries. It can be done by one person or a whole organisation.

Myth 4: Innovation is popular.

Humans generally dislike change. Self-preservation makes us conformists and it takes a lot of effort to accept and support something new. Most organisations have an innate ‘immune system’ that kills things that go outside the perceived norm. Recognising this and appreciating when it is happening can help you move forward.

Myth 5: We haven’t got the right minds or people in our organisation to innovate.

You don’t need someone technically minded or a digital guru – what you need are lots of good ideas and you most likely already have the basis of these already available in your organisation. Your employees and volunteers, even your beneficiaries, will have excellent insights into what needs to be done, what could change, what could be tested. The key is prioritising and investing in the most suitable innovations for your organisation and at the right time.

Myth 6: Innovation is done elsewhere.

Whatever your role and whatever your team, you can benefit from at least looking at how you can be more innovative. As a concept, a principle and an approach, it’s there for everyone.

Myth 7: Innovation is expensive and exclusive

Innovation doesn’t need to cost anything. Why not try adopting at least a few techniques, principles or processes to start you off and see what new ideas and benefits it might lead to!

If you’ve found this article interesting, why not also read:

How relevant is innovation for charities?

Innovation doesn’t need to mean expensive

Four Ways to Future Proof your Charity in the Digital World

There are many benefits for charities to improve their digital skills today.

Embracing digital today can be the key to your charity’s success in the future. In a digital-first world, it’s important to understand how emerging trends and channels can enhance your charity’s activities over the next years.

A strategic approach towards digital can help you:

  • Increase impact
  • Improve service delivery
  • Explore an improved used of data
  • Increase income from new channels

Looking ahead, here are four ways to future-proof your charity in the digital world.

According to the Charity Digital Skills report, only 35% of charities are up to speed with how digital trends are affecting their charity’s work. Do you want to find out more? please see charity digital tool kit. 

1. Set up a digital strategy – Many charities are using digital channels to reach their supporters but not all of them have a set digital strategy. There’s no need to be a digital expert to go from a tactical to a more strategic digital use. You can start by mapping out your current channels and how you will make the most of them.

2. Build your digital skills – One of the biggest challenges for charities is the lack of funding to build up the team’s digital skills. You don’t necessarily need a big budget to access free digital courses and webinars that can help you improve your digital skills. What you need is the buy-in from your stakeholders to embrace digital transformation from your IT infrastructure to the processes that can future-proof your charity.

3. Embrace a digital culture – You can’t plan your charity’s digital future without the internal buy-in and support from your board. Many charities feel that they lack the internal digital culture to support a broader digital transformation. That’s why it’s important to get your leaders and trustees to understand the power of digital and how it can benefit your organisation. A more agile and collaborative environment can be the key to cultivating a digital mindset that favours your future digital plans.

4. Think of the users’ needs first – The best way to future-proof your charity is to focus on your supporters’ needs and the tactics to address them. Digital channels should help you learn more about the changing user needs and behaviours. You should be able to explore how social media channels, email communications or even your website can bring you closer to your supporters. It can even be a great opportunity to increase your impact and improve your service delivery by reaching your supporters on their preferred method of communication

When it comes to future-proofing your charity, aim to be digitally ambitious. Start small and explore how you your digital ambitions can help you deliver the highest value for your users even with limited resources and time.

Please see extra links which may help you and

Measuring your impact effectively – part one

Do you have the right measurements in place to track against your goals? Learn how to measure, quantify and communicate your impact to prove your charity’s worth.

Organisations are no longer measured purely on their outputs but also on how they achieve social and environmental goals. We know that, for many of you, measuring and reporting on your impact can be quite daunting and complex.

So, throughout 2020 we’re pleased to offer a free, two part webinar on ‘Measuring the Difference Your Work Makes’ delivered by our expert partners at the Foundation for Social Improvement.

This two-part webinar series is open to all causes.

This is a recording of the first webinar held on Wednesday 29th January. It provides an overview of evaluation approaches and will equip you with the skills to effectively demonstrate the impact of your organisation.

Please allow 90 minutes to watch the recording in full. A printable version of the slides used can be found here.

If you’d like to join an upcoming live presentation of this training with the opportunity to ask your own questions, here are the dates. Look out for invitations to join via our Aviva Community Facebook Group.

Dates for future impact measurement webinars: