Statistics are usually included in media to support facts, reinforce arguments or provide additional information to the viewer. However, we must not forget that they are a powerful tool of persuasion, and must be interpreted with caution, as statistics can be deliberately manipulated or skewed by the author to shape the opinions of viewers. Data may be manipulated in the following common ways:
- Cherry-picking data - discarding results that are unfavourable.
- Wrong measure used - we have seen that different the measures of centre and spread may give different impressions of the data if outliers are present.
- Biased surveys - the phrasing of survey questions and the interviewer can often influence the responses given.
- Sampling technique - a sample should be representative of the population of interest. A poor sampling technique can introduce bias to the results. Practices likely to produce bias include very small samples or samples not selected randomly, such as from volunteer phone polls.
- Over-generalisation - when asserting a conclusion from a particular group to a wider population when the initial group is not representative of the larger population.
- Overstating significance of findings - sometimes it is not clearly stated how large the sample was or how large the margin of error is. If the error margin is large then then result may be significantly off the stated figure.
- Correlation and causation - if a relationship is observed between two variables A and B, it is tempting to jump to the conclusion that A causes B. However, it may be that B causes A, the two may in part cause each other, there may be a third factor that causes both or the relationship may be purely by chance.
- Comparing apples and oranges - when comparing two data sets that cannot be meaningfully compared against each other. (Example)
- Misinterpreting statistics or probability - probabilities, in particular, are commonly misunderstood and misrepresented in media court cases. (see prosecutor's fallacy)
Misrepresentation of statistics often happens unintentionally by a source who is reporting on a subject for which they are not an expert or are not familiar with the statistics quoted. However, misrepresentation may also be intentional to lead viewers to a particular conclusion. Critical evaluation of figures given in articles is very important in our information rich community. This video has some tips for looking deeper than the attention grabbing headlines.
Another way to commonly misrepresent data is using an inappropriate visual representation. Graphs are an important tool to convey statistics, however, a poorly constructed graph may lead a reader to an incorrect conclusion. Review the common ways misleading graphs can be constructed.
Practice questions
Question 1
Lachlan asks $120$120 Year 12 students at his school how much time they spend on homework per night. $78$78 Year 12 students say they do more than $3$3 hours. At a meeting of the student council Lachlan reports "$65%$65% of students at this school do too much homework.
Which one of the following explains why this is misleading?
The survey does not represent the population of the school.
AThe question should have been multiple choice.
BThe question was biased.
CThe sample size was too small.
D
Question 2
Question 3
Refer to the graph to answer the following questions.
A line graph illustrating fourth-grade mathematics scores on the Main NAEP (National Assessment of Educational Progress) from $1990$1990 to $2000$2000. The horizontal axis represents years and is labeled with four points in chronological order: $1990$1990, $1992$1992, $1996$1996, and $2000$2000. These points are spaced at irregular intervals, with gaps of $2$2, $4$4, and $4$4 years respectively. The vertical axis represents the scale score. It ranges from $210$210 to $240$240, increasing in increments of $2$2. In $1990$1990, the score is marked at $213$213. In $1992$1992, the score rises to $220$220. In $1996$1996, the score continues upward to $224$224. In $2000$2000, the score reaches$228$228. A silhouette icon of three students appears above the $2000$2000 data point at $228$228, drawing attention to this value. To the left of the graph, accompanying text highlights "The Main NAEP shows dramatic gains in maths. From $1990$1990 to $2000$2000, fourth grade math scores increased fifteen points – equivalent to about $1.2$1.2 years of learning".
Source: Brookings report on American education
What is a fault with this graph?
By cropping the bottom section of the graph the author has made the decrease in math scores appear larger than it really is
ABy cropping the bottom section of the graph the author has made the increase in math scores appear larger than it really is
BBy cropping the bottom section of the graph the author has made the decrease in math scores appear smaller than it really is
CBy cropping the bottom section of the graph the author has made the increase in math scores appear smaller than it really is
DWhat is another fault with this graph?
The labels on the vertical axis are not evenly spaced
AThe labels on the horizontal axis are not evenly spaced
BThe graph does not have a scale break
CWhy is this a problem?
It has made the increases in the 4-year intervals 1992-1996 and 1996-2000 appear faster than they really are (relative to the rate in the 2-year interval 1990-1992)
AIt has made the increases in the 4-year intervals 1992-1996 and 1996-2000 appear slower than they really are (relative to the rate in the 2-year interval 1990-1992)
BIt has made line segment in 1990-1992 interval appears more steep than it should be
CIt is not a problem
D
Question 4
The Australian Labour Party released this graph after Tony Abbot was elected as Prime Minister.
A vertical bar graph titled "Women in Cabinet around the World". The vertical axis consists of numbers from $0$0 to $12$12, marked and labeled uniformly in increments of $2$2. There are $15$15 countries listed on the horizontal axis. Each country is represented by a blue vertical bar, labeled by a number just above the horizontal axis. The countries are arranged from the one with the highest bar on the left to the lowest bar on the right. Canada is on the leftmost with $12$12. Rwanda has $11$11. Cuba and Uganda each have $8$8. Indonesia has $7$7. New Zealand has $6$6. Liberia and Zimbabwe each have $5$5. Afghanistan, Egypt, India, and the United States each have $3$3. China and Malaysia each have $2$2. Australia is last, with $1$1. The bar for Australia is marked with a downward arrow labeled “Tony Abbott’s new cabinet.”
Which of the following comments apply:
This graph is misleading because the scale on the vertical axis is not uniform.
AThis graph is misleading because it claims that there are always an equal number of male and female cabinet members to choose from
BThis graph is misleading because it claims that all cabinet sizes are the same.
CThis graph is misleading because there are no European countries included.
D