Tim Harford at the Royal Institution
"Good Causes and Bad Statistics"
Pro Bono Economics hosted its flagship lecture on Wednesday 10th February 2016 with guest speaker Tim Harford at the Royal Institution. The evening saw almost three hundred attendees from across the charity sector and economics profession arrive at the famous Faraday Theatre to hear Harford speak on the theme “Good Causes and Bad Statistics.” Lord Gus O'Donnell, who had been announced as the new PBE Board Chair the previous week, provided an introduction.
Harford began his speech by talking about “Blue Monday,” a day in the middle of January reported to be the most depressing day of the year and used by two mental health charities to highlight the rise in mental illnesses across the UK. The formula behind “Blue Monday” has been disparaged by scientists as a fictitious construct designed by travel companies to sell package holidays. The use of claims for effect by charities, says Harford, serves to stimulate interest for particular causes with little regard for whether these claims are true.
A second and humorous example explored the pitfalls of sampling bias when utilising statistics. The charity Guide Dogs UK fell foul of this when it asked guide dog users on twitter to fill in a survey about the risk to the blind from cyclists. From this small sample, Guide Dogs UK put out a statement that 25% of guide dog owners in London have had dangerous encounters with cyclists, which they later withdrew.
Harford’s third example demonstrated that it’s possible to misinterpret findings, with the British Stroke Association stating that strokes in the middle-aged population have increased by more than 25% since last year based on the number of 40-54 year old men admitted to hospital with stroke symptoms. After the statistic was called into question by a leading expert on strokes, further analysis showed that the increased number of people visiting hospitals was actually due to the circulation of better information on the first signs of strokes and people taking action on this. The number of strokes within the age group has, in fact, marginally decreased.
The famous Oxfam campaign, claiming that the 85 richest people are as wealthy as the poorest half, is a claim that Harford considers to be true, however misleading. Every year Oxfam extrapolates data from the Credit Suisse Global Wealth Data Book to calculate global inequality. The way that wealth is calculated means that anyone in negative equity, which will include many in the UK with sizeable student loan repayments and mortgages, may well be part of the bottom billion. An African farmer, who will never have the opportunity to accrue this kind of debt, will be in positive equity and might therefore be part of the middle five billion.
"The number is true, but it’s also BS because it doesn’t help us understand the big issue – nor was it designed to do that. The purpose was to drive people to [Oxfam’s] website and harvest their emails."
Harford suggested that the best way to navigate oneself through “bad statistics” is to ask yourself whether numbers provoke thought or just feelings.
“You must ask yourself, I wonder how they measured that.”
Harford proposes that with the truth, i.e. accurately calculated statistics, charities will be able to ask themselves more serious questions about what must be done to tackle the issues their beneficiaries face. A way that charities can grab headlines is to spend more time figuring out what works and less time using bad statistics that do little more than harm their reputation.
"Bad facts breed cynicism from the general public and dilute the discourse."
Harford’s address was followed by responses from a panel of charity sector professionals: Dawn Austwick (Big Lottery Fund, CEO), Mike Ellicock (National Numeracy, CEO), Hugh Stickland (Citizens Advice, Chief Economist) with trustee and co-founder of PBE Andy Haldane as chair.
Dawn Austwick suggested that charities often use “hysterical data,” as “humble data” doesn’t get the kind of media attention necessary to boost support for the cause.
Mike Ellicock said that charity professionals are often put on the spot about their knowledge of their cause, which may result in the over amplification of particular facts and statistics.
Hugh Stickland talked about the internal capacity of Citizens Advice to collect data, analyse statistics and measure impact lead by himself as Chief Economist. He suggested that PBE can help charities to do this when they do not have the capacity themselves.
The panel was followed by a Q&A session with the audience. One attendee, in response to Tim’s speech, suggested that charities often publish bad statistics in a hurry due to pressures from the media. The Director of Giving Evidence and author of “It ain’t what you give, it’s the way that you give it: making charitable donations that get results” Caroline Fiennes wanted to know how the third sector can fix incentives for charities, as charities often feel the need to use bad statistics in order to compete for funding. Dawn Austwick responded by saying that this was something funding organisations were aware of, but that competition between organisations can also be a good thing to improve charity effectiveness.
Follow the twitter hashtag #GoodCausesBadStats.