7 Summarising quantitative data

Connection to the curriculum: This is the seventh in a series of statistical articles for mathematics teachers. In this article, we discuss the notion of central or ‘typical value’ in a sample of data (mean, median) and the amount of variation (range, standard deviation, interquartile range). These concepts are referred to in Essential Mathematics Unit 2, Topic 1, Representing and comparing data; General Mathematics Unit 2 Topic 1 Univariate data analysis and the statistical investigation process.

The mean monthly streamflow at the Mary River in February is 1 123.2 ML. The median monthly streamflow for the same data is 146.1 ML, about ten times smaller. Why are these two measures of average so different? Which, if either, is better, and why?

7.1 Introduction

The scientific process answers research questions based on evidence, which we call data. Once the data are obtained, the data needs to be described (i.e., identifying the data type; Dunn and Marshman. 2020b) and summarised. Then, statistics extracts answers to the research question from the data. In this chapter, we describe creating numerical summaries for quantitative data, a precursor to analysis (the third step in the statistical investigation process) (Dunn and Marshman, 2020a).

A crucial distinction in this article is that a statistic is a number that summarises a sample, and a parameter is a number that summarises a population. A sample is used to study a population. Statistics are computed from samples, to estimate the unknown values of the parameters of a population. In practice, we almost never have a population. The value of statistic varies from one sample to another, depending on which sample is selected from the population. In contrast, the values of the parameters do not change from sample to sample.

7.2 Measuring the central value of sample data

The two most common ways of finding the average of a sample of quantitative data are the sample mean (or sample arithmetic mean) and the sample median. These can be used to estimate the population mean or population median respectively (ACMEM050; ACMGM030).

The word ‘average’ is very broad, and is used to refer to the mean, median or any other measure of central tendency. The specific word ‘mean’ or ‘median’ should be used to avoid ambiguity when appropriate. Even the word ‘mean’ itself has multiple meanings. The usual definition, and the one in the curriculum, is the arithmetic mean; however, geometric mean, harmonic mean, weighted mean, truncated mean, and Winsorized mean are also commonly-used in various applications.

The arithmetic mean is computed as EQUATION where is the sample size and corresponds to the observations. (The large capital sigma represents adding.) This is how the mean is computed; but what does it measure?

The sample mean is the ‘balance point’ of the observations; that is, the sum of the positive and negative distances from the mean is zero. We can model this by balancing small counters on a ruler and balancing this on a fulcrum. The place of balance of the ruler on the fulcrum is the mean (Hardiman, Well and Pollatsek, 1984). The mean is also the value such that the sum of the positive and negative distance from the mean sum to zero.

Consider an example. To answer the research question ‘What is the average percentage of butterfat from mature jersey cows?’, Sokal and Rohlf (1981) (as cited in Hand et al., 1996) considered milk from a small sample of mature Jersey cows and collected the percentage of butterfat (Table 1). The sample can be used to estimate the population mean. The sample arithmetic mean, as the balance point, is shown in Figure 1(left panel) as 5.250%. This is also demonstrated in Figure 1 (right panel), where the sum of the positive and negative distances from the mean are zero. (Animated diagram of this idea are available in Dunn (2024)).

Table 1: Percentage butterfat from 10 Jersey cows, shown in increasing order.

Percentage butterfat from jersey cows

4.5 4.8 4.8 5.2 5.2 5.2 5.2 5.4 5.7 6.5

1.                      b.

   Figure 1. The meaning of the mean. Left: the mean as a balance point. Right: the mean is where the sum of differences is zero.

The arithmetic mean is the most frequently used measure of central location, and is computed by most frequently-used calculators. The arithmetic mean is most suitable for approximately symmetric data. If the data contain outliers (observations quite different from the other observations), or the data are highly skewed, then the arithmetic mean will not be a good representation of the central value for the sample. In general, if the data contain large outliers the arithmetic mean will be too large, and if the data contain very small outliers the arithmetic mean will be too small.

Sometimes other means (besides the arithmetic mean) are more appropriate to use. The geometric mean uses multiplication, rather than addition, of the observations: , where the symbol means to multiply (akin to . The geometric mean is usually defined only when all observations are positive. The geometric mean is useful for averaging values that are multiplied. For instance, finding the average annual return on a financial investment over time whose value compounds.

The harmonic mean is useful for averaging rates, and is defined as , and again is usually only defined when observations are all positive. Consider a car travelling from A to B at (say) \(60\) km/h, then returning the same distance from B to A at \(100\) km/h; the ‘average’ speed is the harmonic mean of \(60\) and \(100\) (i.e., \(75\) km/h), not the arithmetic mean (i.e., \(80\) km/h).

Another measure of ‘average’ is the median, which is not affected by outliers or skewness. The median can be defined as a value of that minimises . The median is the value which has half the values in the sample larger than it and half the values less than it. For a dataset with values, the median is the th value in the ordered sequence of observations. For a sample where the number of observations is odd, the median is the middle observation when the observations are ordered. For a sample where the number of observations is even, the median is any number between the two middle values; commonly, the median is often quoted as half-way between the two middle numbers (i.e., the arithmetic mean of the two middle numbers). However, other definitions for the median are also possible when n is even. For instance, consider the (obviously contrived) data set:

1 2 4 8 16 32 64 128

For this dataset, , so the median is halfway between 8 and 16: the median would be 12. However, the data are not-equally-spaced, so taking the half-way point between two observations seems unintuitive. Taking logarithms to base 2 yields the integers from 0 to 7, with a median of 3.5; thus, the median of the original data set may be quoted as 23.5 = 11.3 approximately.

For the Jersey cow data (Table 1), with observations, the sample median is between the observations 5.2 (ordered observation 5) and 5.2 (ordered observation 6); that is, the median is 5.20%.

7.3 Measuring the variation of sample data

While the mean or the median describes the sample data using a single value, the variation also needs to be described. Consider a class with an average assignment mark of 60%; students may be interested in the amount of variation too. If there is no variation, then everyone received 60% (and some students will be very disappointed, and some very pleased!). However, the reactions are different if the class is also told that the marks range from 10% to 100%.

The simplest measure of the variation is the range: the maximum value minus the minimum value. The range is often too simplistic to be useful. (Sometimes both the maximum and minimum values are given as the range.) For the Jersey cow data, the largest value is 6.5%, and the smallest value is 4.5%. Therefore, .

The most common method of describing the variation is the standard deviation. The standard deviation ‘indicates the degree to which the individual data values are spread around their mean’ (QCAA, 2019, p.71); roughly (but not exactly), the standard deviation is like the average distance that observations are from the mean. The standard deviation of a sample is defined as

where is the sample mean. For calculation, the equivalent formula

is used, which is easier for computation. A table (Table 2) is useful for computing the standard deviation if a calculator or software is not used.

Table 2. Computing various measures of variation for the butterfat data in Table 1.

Data
Distance from mean
Distance squared
Data squared
Absolute distance from mean
Absolute distances from median


















4.5
-0.75
0.5625
20.25
0.75
0.7



4.8
-0.45
0.2025
23.04
0.45
0.4



4.8
-0.45
0.2025
23.04
0.45
0.4



5.2
-0.05
0.0025
27.04
0.05
0.0



5.2
-0.05
0.0025
27.04
0.05
0.0



5.2
-0.05
0.0025
27.04
0.05
0.0



5.2
-0.05
0.0025
27.04
0.05
0.0



5.4
0.15
0.0225
29.16
0.15
0.2



5.7
0.45
0.2025
32.49
0.45
0.5



6.5
1.25
1.5625
42.25
1.25
1.3

Sum: 52.5 0.00 2.7650 278.39 3.70 3.5

Using the information in Table 2, both formulas give the same value, but the second requires less calculation. Using the definition, %. Using the computational formula, %.

A common question asked by students is why we divide by , rather than , in the calculation (which would be more in line with the “arithmetic mean” distance of observations from the mean). Dividing by \(n - 1\), rather than \(n\), obviously increases the value slightly, though the differences are negligible for larger data sets. One answer is that the distances are distances from the sample mean, and the sample mean itself is an estimate whose value varies from sample to sample, so the value is inflated slightly to account for this additional source of variation. Another answer is that the estimate is purposely chosen to minimise the distances from the observations to the mean, so will tend to underestimate the variation as a result. Mathematically, the reason is that dividing by rather than means that (the sample variance) is an unbiased estimate of the population variance (i.e., the expected value of the sample estimate is equal to the population value).

Another common question asked by students is why we sum the square of the distances of the observations from the mean, then take the square root, rather than just use the absolute value of the distances from the mean. Using absolute values is perfectly acceptable, and doing so produces mean absolute deviation: the mean absolute deviation for the butterfat data is . (Mathematically, no bias correction is needed, hence the division by .)

The value of the standard deviation is larger than the value of the mean absolute deviance, as the squaring in the standard deviation formula has the effect of placing greater emphasis on the larger values when compared to the mean absolute deviation. The median can be used rather than the mean also: the median absolute deviation is .

The mean and median absolute deviations are perfectly acceptable as ideas, but are not often used because the absolute values cause mathematical difficulties (e.g., taking derivatives of absolute values). Furthermore, many theoretical results in statistics rely on using the standard deviation because of these nicer mathematical properties.

Another measure of variation is the interquartile range (or IQR). IQR is not affected by outliers in the data. Quartiles divide the data into four parts with approximately equal numbers of observations. We say “approximately equal”, since dividing the data into four parts with the same number of observations in each is clearly only possible when the sample size is a multiple of four; hence, quartiles are often approximations. The first quartile (or ) is effectively the median of the smaller half of the data (halfway between the minimum value and the median). The smallest 25% of observations are less than or equal to .

The second quartile is also the median: about 50% of observations are less than .

The third quartile is effectively the median of the larger half of the data (halfway between the median and the maximum value). The largest 25% of the data are greater than or equal to .

The middle 50% of the data lie within the range of the IQR . Since the IQR effectively ignores the smallest 25% and largest 25% of the data, outliers have no impact on the value of the IQR.

As with the median, these rules are convenient for computing quartiles by hand, but other definitions are possible. In fact, Hyndman and Fan (1996) discuss nine different methods used by software for computing quantiles. In other words, quartiles computed manually using the convenient rules above may be slightly different than those computed by software. A boxplot is the graph of the minimum value, the quartiles and the maximum value (called the 5-number summary). For the Jersey cow data, the 5-number summary is 4.5 (the minimum), 4.8 (), 5.20 (median), 5.4 (), 6.5 (maximum); the boxplot is shown in Figure 2.

Figure 7.1: Boxplot of the Jersey cow data.

#> Warning in par(old.par): graphical parameter "cin" cannot
#> be set
#> Warning in par(old.par): graphical parameter "cra" cannot
#> be set
#> Warning in par(old.par): graphical parameter "csi" cannot
#> be set
#> Warning in par(old.par): graphical parameter "cxy" cannot
#> be set
#> Warning in par(old.par): graphical parameter "din" cannot
#> be set
#> Warning in par(old.par): graphical parameter "page" cannot
#> be set

By convention, outliers (ACMGM031) can be defined in terms of how far observations are from or . Extreme outliers can be defined as observations more than smaller than , or more than greater than . In addition, mild outliers can be defined values more than smaller than , or greater than (that are not already defined as extreme outliers). The numbers \(1.5\) and \(3\) used here are arbitrary, but in common use. Any observations declared as a mild or extreme outlier are often plotted explicitly in boxplots to highlight them *(see Figure 4, for example). Any declaration of an outlier is arbitrary. Importantly, outliers are not necessarily mistakes, and should not be removed from the data without further consideration of the source of the outlier (see Dunn and Smyth, 2018; p. 138). Boxplots summarise an entire data set with just five values, so detail is lost. For this reason, histograms (Dunn and Marshman, 2021) are usually recommended for showing the distribution of a single quantitative variable. Boxplots are excellent for comparing the distribution of two or more sets of quantitative data. EXAMPLE USING DIFFERENT DATA? Side-by-side (parallel) boxplot

7.4 Teaching using the Mary River streamflow data

This activity was designed to help students understand the differences between the mean and the median, and to make decisions about when the sample mean or the sample median gives the best estimate of the ‘average’ for the population. Teachers, preservice teachers and students often have misunderstandings of these two concepts (Nguyen et al., 2023), and noted that there was a preference for using the algorithm to calculate mean rather than for understanding its meaning. For many data sets, the mean and median produce similar values, so students may not think it matters which is used.

Consider the data of the daily river flow volume (called ‘streamflow’) of the Mary River from 1 October, 1959 to 17 January, 2019, summarised by month in Table 1 (extracted from Queensland DNRM data). The February ‘average’ daily streamflow () could be quoted using either the mean or the median. Before showing your students the data, you could ask: Would the mean or median be the better way to measure the average daily streamflow? Does it even matter (since both measure the ‘average’ of the same data)? Explain your answer.

After the discussion, the mean and median can be shown to the students (Table 1), who will see that the two statistics give very different values for the ‘average’; the mean daily flow in February is 1 123.2ML whilst the median daily flow is 146.1ML (almost ten times smaller). Students can be asked to discuss:

Why do these two statistics (both measuring the ‘average’ of the same data) give such different values? Which average is the ‘best’? How do you decide what is meant by ‘best’?

Table 3. Daily streamflow summaries at Mary River (Bellbird Creek), in ML, from 1 October 1959 to 17 January 2019 summarised by month; the mean and median for the daily streamflow in each month is shown.

Month Mean streamflow (ML) Median streamflow (ML) Month

January 849.3 71.3 1783

February 1123.2 146.1 1650

March 793.9 194.9 1829

April 622.5 141.7 1770

May 348.4 118.4 1829

June 376.7 83.6 1741

July 259.3 68.8 1810

August 108.6 55.5 1829

September 100.9 48.0 1758

October 151.2 37.6 1835

November 186.6 45.3 1775

December 330.8 64.1 1834

Obtained from data extracted from the Queensland Department of Natural Resources and Mines (DNRM; http://watermonitoring.dnrm.gov.au)

Teachers should anticipate how their students may justify their decision to use the mean or median (Lampert, 2001; Stein et al., 2008). As students are working on their responses, teachers should monitor the students’ progress and look at the actual statistics the students are discussing, assess the validity of their ideas, and make sense of their statistical thinking even when not quite correct (Stein et al., 2008). While teachers are doing this, they need to select which student responses will be shared (Lampert, 2001; Stein et al., 2001). The appropriate selection of students ensures that common misconceptions are aired publicly, understood and corrected, and that the key learning is brought to all students’ attention. Students’ responses can be sequenced in many ways. Teachers may begin with the response that most of the class have used, or begin with a strategy based on a misconception, clarify and move on, or they could use related or contrasting strategies one after the other to make it easier to compare (Schoenfeld, 1998; Stein et al. 2008).

Finally, teachers need to help students to connect the statistical ideas that have been discussed (Boaler & Humphries, 2005; Stein, 2008), and have a correct understanding of why the median is the most appropriate measure of average. Plots of the daily streamflow in February at the Mary River from 01 October 1959 to 17 January 2019 with a dot plot (Figure 3, left panel) and a histogram (Figure 3, right panel) show highly right-skewed data with very large outliers. In fact, no plot shows the data well, unless logarithms of the data are used (after adding one to all values, since some daily streamflows are zero).

Locating the mean (1 123ML) and the median (146ML) on the graphs is pointless as they are both so small compared to the largest value (near 150 000ML); the maximum value is more than one hundred times larger than the mean and one thousand times larger than the median. Furthermore, about 86% of the observations are less than the mean. In contrast, about 50% of the values are less than the median (by definition). For these data, the mean is hardly a central value; medians are better measures of the ‘average’ for highly skewed data, or data with large outliers. These points need to be drawn out in the discussion.

Figure 7.2: The daily streamflow at the Mary River in February. Left panel: a histogram of the data (left panel). Right panel: a jittered dot plot, where each dot represents the daily streamflow for one day in February (the vertical dimension has no meaning (hence is unlabelled) but is necessary since otherwise most points are indistinguishable). The vertical grey lines correspond to the mean or median (on the scale used on the horizontal axis, they are indistinguishable).

The boxplot of the Mary River data shows the data is very highly right-skewed (Figure 4), and so the median should be used in preference to the mean. From a practical point-of-view, however, even more useful is to understand that using means or median both seem insufficient, and a measure of the variation is needed. Using the quartiles: , and , while the maximum is 156 000 ML. This shows that the largest 25% of the data span an enormous range.

Percentiles can be used to understand the data also (Dunn, 2024). Percentiles are like quartiles, but refer to breaking the data into 100 approximately-equal numbers of observations. This means, for example that the first quartile is the same as the 25th percentile, and the median (and ) is the same as the 50th percentile. For the streamflow data, estimating the 99th percentile may be more useful for constructing infrastructure (corresponding to a 1-in-100 year event). The 99th percentile is 19 043 ML. Constructing infrastructure using a mean (1 123.2 ML) or median (146ML) streamflow event seems rather inadequate.

Figure 7.3: Boxplot of the February streamflow at the Mary River in February.

#> Warning in par(old.par): graphical parameter "cin" cannot
#> be set
#> Warning in par(old.par): graphical parameter "cra" cannot
#> be set
#> Warning in par(old.par): graphical parameter "csi" cannot
#> be set
#> Warning in par(old.par): graphical parameter "cxy" cannot
#> be set
#> Warning in par(old.par): graphical parameter "din" cannot
#> be set
#> Warning in par(old.par): graphical parameter "page" cannot
#> be set

In summary, the mean and standard deviation are best used when the data are approximately symmetric data, because the mean is influenced by outliers and skewness. In other situations, the medians and IQR are best used as they are not influenced by outliers and skewness.

The idea of the distribution is explored further in Data Files 8.