A case study from the Royal Institution

Digital is undoubtedly the way to go when it comes to reaching out to and engaging your audience. For charities and non-profits, this often means parting with sizeable funds to create engaging online content such as films or podcasts to draw in supporters, inspire donors and inform beneficiaries in a more creative way. Yet, in a less than certain funding environment, money spent on digital resources that don’t deliver could put off funders and detract from the good work of a charity.

So, how can we be sure that the content produced is reaching the right people in the right way? How do you know that people find your online content engaging? The Royal Institution (Ri) approached Pro Bono Economics (PBE) as they wanted to know more about viewer engagement with their online science videos. Here are some of the main findings from the report [full report here].

1) Who is your audience/who do you want to reach?

The first thing the Ri needed to consider was the audience they wanted to reach and the best way of doing this. PBE economists helped the Ri understand which of their videos were most popular (measured by views and minutes watched), how engaging their videos were according to likes, comments and shares and gave them an overview of the demographics of their audience as well as the devices they use. If you’re interested in knowing who your audience is, you might choose to analyse demographics, or, if you’re more interested in knowing which content is the most popular, looking at consumption and engagement metrics might be the best use of your time.

"At the Ri, one of our aims is to encourage people to ‘think more deeply about the wonders and applications of science’. Whilst we already knew which of our videos were most viewed, and hence most popular, PBE volunteer economists helped us to better understand how our audience were engaging with them. Through working with FTI, we now have metrics which we use to measure whether we are achieving our objective of encouraging our audience to ‘think more deeply’.”

Cassie Williams, Digital Manager at the Royal Institution

2) What does “success” look like?

Now that you’ve decided on the audience you want to reach, what denotes success? Those familiar with analytics are probably aware that a video “view” is not always suggestive that the viewer has truly engaged with its content. For the Ri, engagement with the sciences is integral to its founding mission. PBE economists advised the Ri that the level of viewer engagement may be better measured using metrics that relate to whether videos: (1) inspire conversation (for example, with comments and debate below videos); (2) result in applause (for example, by collecting “likes” or being added to favourites playlists); or (3) get shared.

Such metrics can be expressed in absolute terms (for example, the total number of comments received by a video) or as ratios (for example, the proportion of views that results in a comment). These metrics are better measures of how engaging videos are than the number of views alone.

PBE economists found that longer films in the Ri’s collection were on average more popular and more likely to be commented on and shared. They also found that longer films and talks were responsible for recruiting over a third of subscribers to their YouTube channel. This has helped the Ri to consider which content to focus on in the future.

“It is encouraging to find that certain types of content have proved to be particularly popular and engaging in the past. We would advise charities and non-profits to interpret these statistics carefully before deciding what type of content to produce in the future. There are three important issues here. First, there is a distinction between correlation and causation: is the content wholly responsible for its own success, or have other factors – such as a special marketing effort or fortuitous timing – contributed too? Second, there may be diminishing returns to producing certain types of content: is the fifth video on the origin of the universe (for example) likely to be as popular as the first? Third, are certain types of content still worth producing, despite their being less popular than others?”

Ravi Kanabar, Director at FTI Consulting.

3) Viewer comments as feedback

Getting people talking about your online content is a great way to deepen interest in your cause. Monitoring the comments section under videos can help your organisation evaluate their impact. Having a designated moderator within your communications team can help to steer debate and answer any questions that may arise. If you don’t have time for this, think about provoking questions to share on social media every time you post content to get conversation flowing. The Ri often posts content on twitter that requires engagement such as quizzes and questions to stimulate discussion.

4) Don’t misinterpret your findings

Getting positive results can be exciting, but it’s always important to remember that there are deeper questions that need to be asked. For the Ri, video consumption and viewer engagement are suggestive of learning, but how can they know that this is actually the case? By measuring pick-up, the Ri can now make a strong case to funders that their digital content is engaging and widely shared, however science learning is perhaps harder to measure. The Ri might consider adding a short quiz to the end of their videos to enable them to gauge the level of learning.

“Viewer engagement metrics are arguably a better guide to genuine learning than simple counts of the number of times a video has been viewed. Short quizzes could provide further information on the extent to which the viewer has retained and understood the knowledge a video was intended to convey.”

Tim Battrick, Senior Director at FTI Consulting.

5) What next?

Now that you have a clearer idea of your audience and/or the content they most prefer, what next? Knowing the impact of your digital content is one thing, but acting on that insight is another. Be prepared to make changes and explore new approaches. For example, the Ri have now made a point to focus on engaging more women and girls in their revised digital content strategy.

Whether you want to hone in on your existing audience by giving them more of what they want or extend to harder to reach audiences, measuring the impact of your digital content is a fantastic way of doing this. Ensuring you have the very best content and the most informed plans for dissemination will help you spread your organisational message as far and wide as possible, whilst making a strong case to funders.

30th March 2016