My love for graphs has a most unfortunate side-effect. I end up shouting and swearing at most news reports with line graphs, bar graphs or pie charts. Regularly. In fact, most days.
What I see are spurious graphs, and they make me angry.
What I see are spurious graphs, and they make me angry.
Spurious means:
♦️ of illegitimate birth; bastard
♦️ outwardly similar or corresponding to something without having its genuine qualities; false♦️ of falsified or erroneously attributed origin; forged; of a deceitful nature or quality.
Bastard, false, deceitful! Yep, they are some of the words I shout at the TV.
Graphs provide authority for the claims made in research and news reports. They are also easy to view and interpret quickly, so they fit into our fast-paced news cycle.
Spurious graphs are a boon for marketing, PR and politics. They turn up in advertising for dubious health products, plausible but disingenuous political claims, and shoddy science interpretation. (Think The Coalition's 2019 misleading claim that Australia's carbon emissions were actually going down.)
Here are some disturbing examples that might make you laugh at how bad they are, until you remember these are real but spurious graphs produced to manipulate you and me. I explain the misleading nature, and also suggest things for you to look out for.
Example 1: The first graph starts at 10 instead of zero on the Y axis, and runs from 10 to 15 only - so this exaggerates the difference between the length of each bar. Someone has manipulating the graph to show a bigger change than there actually is. This would matter if they were claiming their funding program has made a big difference. In the right-hand graph where this is corrected to run from zero to 15, it looks very different.
So, first thing for any graph, check the scale on the Y axis: does it start at zero? What does it go up to? (A scale up to 10 or 100 can make a very different impression of results, for example.) Is the scale a reasonable sort of way to measure this type of content?
The figures are from real data, right? The plots and lines are mathematics, so how can you argue with them? They have a seductive power; their apparent objectivity gives us a sense of trust.
Reserve that trust: graphs are all too often a tool of propaganda.
Spurious graphs are aimed at deliberately fooling you.
A spurious graph is manipulated to make illegitimate claims with false authority and objectivity.
Graphs provide authority for the claims made in research and news reports. They are also easy to view and interpret quickly, so they fit into our fast-paced news cycle.
 |
| Fig 1: Important Graph |
As an example, you can see just by glancing to the right that there has been a massive increase, 180%, in the number of news articles featuring a graph over the last 20 years. The upward trend is clear and the clean graph layout suggests the information is objective and credible.
The problem is that I made up that claim, and I made up the graph too. If you look more carefully, it is essentially meaningless. It has no labels, no scale, a pointless title. It's a spurious graph that actually says nothing. Except if you just glance at it, the lines infer 'Up, ever upward.' If such a graph briefly features in a political announcement about the economy, then you can be easily reassured that things are going well. Well, you shouldn't.
The problem is that I made up that claim, and I made up the graph too. If you look more carefully, it is essentially meaningless. It has no labels, no scale, a pointless title. It's a spurious graph that actually says nothing. Except if you just glance at it, the lines infer 'Up, ever upward.' If such a graph briefly features in a political announcement about the economy, then you can be easily reassured that things are going well. Well, you shouldn't.
If you pay attention, you will see spurious graphs everywhere.
 |
| Example 1: from Flowing Data |
So, first thing for any graph, check the scale on the Y axis: does it start at zero? What does it go up to? (A scale up to 10 or 100 can make a very different impression of results, for example.) Is the scale a reasonable sort of way to measure this type of content?
Example 2: The top graph shows a very short time period on the X axis. In this short 'cherry picked' time period, the financial returns on this particular stock appear to have done well. The graph below it shows a longer, more representative time period for the trading pattern, which shows the stock is on a downward trend.
Second key thing to check (mainly for line graphs) then, is the time period on the X axis; does it include enough information for the claim they are making, or has someone done some cherry picking to make things look better or worse than they are?
Second key thing to check (mainly for line graphs) then, is the time period on the X axis; does it include enough information for the claim they are making, or has someone done some cherry picking to make things look better or worse than they are?
Example 3: In this graph, the uneven spaces in the points of time suggest something has been omitted. The graph implies there has been a sudden increase in the number of episodes. But there is too little space on the right-hand side if you realise the final data point is 9 years later than the previous data point? This should be spread out over more space on the X axis (just like the earlier points on the graph), and then I would imagine the trend line would be fairly similar.
So, third check: Do both axes have consistent spacing and sensible labels for the type of content? And has the spacing been manipulated to imply something?
So, third check: Do both axes have consistent spacing and sensible labels for the type of content? And has the spacing been manipulated to imply something?
Example 4: The graph compares households with any person on welfare on the left with every individual with a full-time job on the right, not sensible to compare, but works rather well to look outrageous. To be legitimate, they should have compared households for both welfare and jobs OR individuals for both welfare and jobs. It doesn't start at zero either, so it implies a shockingly greater number of people on welfare than those working, when it's quite disproportionate. This is clearly manipulated information to mislead in order to stoke the outrage machine.
So final check: if the findings look shocking or outrages, check the type of data used to make the graph. Is it sensible to compare these two things or has someone compared apples with oranges to try to make you angry at fruit?
In summary, these graphs are spurious because they do not represent the data faithfully or accurately. A valid graph should:
Accuracy is essential. But there is a bit more to avoiding being misled by spurious graphs.
So final check: if the findings look shocking or outrages, check the type of data used to make the graph. Is it sensible to compare these two things or has someone compared apples with oranges to try to make you angry at fruit?
- have an X and Y scale suitable for the type of content, staring at zero
- include all the data and enough data over time
- have clear labels on the axes
- compare things of the same nature.
Accuracy is essential. But there is a bit more to avoiding being misled by spurious graphs.
Graphs are tools of communication; they tell us things about the world.
When we consider what a graph is telling us, we also need to think about how that information aligns with our existing knowledge of the world. Graphs communicate how things relate to each other. It's up to us to then think about why this might be, and why it might be relevant and what that knowledge allows us to do.
Exploring how things relate to each other in our complex world involves more than numbers and objective methods of plotting on a grid. It involves us thinking carefully about the world and the sorts of relationships between things that could possibly exist.
Exploring how things relate to each other in our complex world involves more than numbers and objective methods of plotting on a grid. It involves us thinking carefully about the world and the sorts of relationships between things that could possibly exist.
Interpreting graphs requires us to think about how things relate in the real world.
Tyler Vigen has created a wonderful website Spurious Correlations to demonstrate just how easily graphs and data can be manipulated to make it appear that two things relate to each other when they don't, to show lines that trend in the same pattern and then appear to show a correlation (cor meaning 'together' + relation).
He has technically accurate graphs that show completely unrelated things as somehow, mysteriously, but so very definitely related to each other, if you just go by the graph - the evidence is there in those lines! So, the per capita consumption of chicken is related to US crude oil imports? That makes no sense. Not if you think about it for three seconds.But see how easy it is to appear to show that two things relate, or imply that one thing causes another? You can make your own spurious graph at his site on this page. Do try it; it's fun and not just for graph nerds. (Well, I wouldn't exactly know.)
Graphs depend on the creator's decisions and selection of data to use.
While graphs are appealing and powerful because they seem objective, they depend on a whole pile of decisions that are subjective.
Those making a graph should start from a genuine interest in finding out about the world using all the available data, and not some limited figures or time periods to suggest some relationship you want to imply. Graphs are also constructed on assumptions about how things in the world can relate to each other, and what factors to track together to investigate relationship and correlation. To say anything meaningful, they must refer to a nomological network which precisely defines each factor on the graph. Graphs emerge from our underlying understandings of how things relate to each other in our world.
These subjective decisions are value-laden, and they are open to misunderstanding, bias and, as we have seen, deliberate manipulation.
Vigen's website aims to reminder us that interpreting graphs takes some care as well as asking the appropriate questions, firstly about graphs, but also about the world and the sorts of relationships between things that could possibly exist.
Vigen's website aims to reminder us that interpreting graphs takes some care as well as asking the appropriate questions, firstly about graphs, but also about the world and the sorts of relationships between things that could possibly exist.
Spurious graphs are everywhere so being alert to them can be a demanding task. It's easier said than done.
Because we trust a trending line on a graph to mean something objective, we maybe let our guard down when we really shouldn't.
Because we trust a trending line on a graph to mean something objective, we maybe let our guard down when we really shouldn't.
Do you trust a graph that suggests there a relationship between a person's political leanings and their tendency to violence? Vigen's site suggests that anyone could make a graph that appears to say this. If you believe your political opponents are not very nice people, that graph would be satisfying.
But it might just be spurious.
The use of spurious graphs in the media is part of keeping up sales.
This is not new, as social psychologist Milton Rokeach wrote in 1968:
| ♦️ “The kinds of data … disseminated in the mass media seem designed more to entertain than to inform. … The quality of the information conveyed seems not much different from that conveyed in the sports pages or, better yet, the daily racing form." |
And as for people who deliberately manipulate graphs as propaganda*, as in some of the examples above, all I can say is protect yourself with a lot of scepticism and a little knowledge.
Spurious graphs are used and reused to entertain, to generate clicks and revenue, but also to confuse and to manipulate. All with the cloak of authority and objectivity.
The misuse of my much-loved graphs gets right up my nose. Strangely, shouting at the TV does not seem to improve things.
The least we can all do is think twice about what an upward trending line really says.
The least we can all do is think twice about what an upward trending line really says.
*RE 'fake news', I prefer the term propaganda which says more about the purpose than the specific item of 'news'. For those interested, this article explores this issue.
Images of other people's fabulous graphs
- https://flowingdata.com/2017/02/09/how-to-spot-visualization-lies/
- https://venngage.com/blog/misleading-graphs/
- https://noijam.com/2017/07/12/how-to-spot-a-misleading-graph/
- https://www.statisticshowto.datasciencecentral.com/misleading-graphs/
- http://tylervigen.com/spurious-correlations
- https://blogs.lse.ac.uk/usappblog/2015/05/27/why-do-we-pay-more-attention-to-negative-news-than-to-positive-news/
Cari8 October 2019 at 00:31
ReplyDeleteThanks for the article. Personally I find shouting at the tv and the radio very soothing. Cari
Gina Shivvin 9 October 2019 at 15:09
Glad I'm not alone, Cari, hahaha!
Comments copied from the original post at AdjAngst https://adjangst.blogspot.com/2019/06/spurious.html
ReplyDelete