Calculate and let calculate – How simulations enrich teaching

Hypothesis testing is a fundamental tool in statistics, but complex mathematical formulae cause many students to fail. Yet the process could be much simpler and even deliver more accurate results. By computer, that is.

Hypothesis tests may be less interesting for Faustian characters who want to know “what holds the world together at its core”. But anyone who is interested in finding out where in the world pure chance is at work and where it is not can no more avoid the subject than students of statistics. After all, hypothesis tests can be used to test assertions (including the “perceived truths” so popular today) on the basis of random samples. Of course, one hundred per cent certainty could only be achieved by examining the entire population, but that would be quite time-consuming. It would also mean that statisticians would no longer be needed and I would be out of a job. It is therefore simpler (and more profitable for me) to analyse only random samples and describe the remaining uncertainty as a probability calculation. To do this, we start with an initial hypothesis or null hypothesis (H₀), set an alternative hypothesis or unity hypothesis (H_a) against it so that it is not so lonely, and off we go.

Investigation using the slightly alcoholised traditional example

To explain hypothesis tests, the famous but somewhat dusty example of Lady Ottiline is often used, who claims that she can tell whether tea or milk was poured into her cup first. As I prefer to drink coffee myself (without milk), I’d like to suggest a little update with Scottish single malt whisky: So Lady Ottiline claims she can taste whether the whisky or the water was in the glass first. Admittedly, this is an example that needs explaining: Probably not everyone knows that you usually add a few drops of water to a single malt, but why not impart a bit of culture en passant?

So we give the lady eight glasses of whisky with three drops of water in each and let her taste it. Our H₀  is: she can’t decide which ingredient was in the glass first. Then her hit rate would be 50 per cent at best (p <= 0.5). The opposite H_a  is: She can decide. Then her hit rate would have to be greater than 50 per cent (p > 0.5). To make a long story short: Our Lady Ottiline may be a little tipsy at some point, but she’s right on all 8 glasses. If we were just guessing, her hit rate would have been p ≈ 0.004 at best (readers check for themselves). So we discard our H₀ and accept the H_a. So far, so simple. The students are happy. They have understood that.

Diced probabilities

Unfortunately, that’s not enough, because the nasty word random fluctuations makes things a bit more complicated: let’s assume we want to go to the casino to get rich against all odds at the dice table. There’s a shady guy lurking around outside the entrance who wants to sell us a special throwing technique that is supposed to increase the probability of rolling a 6. Naturally, we demand a sample from him and have armed ourselves with hypotheses. H₀ : The guy wants to cheat us, so the probability of a 6 is at best 1/6, p <= 1/6. H_a: He can do what he claims, then the probability should be higher than 1/6, p >1/6. So we let the guy roll the dice 56 times and count the sixes. He manages 13, which corresponds to a relative frequency of 0.232 and is therefore actually higher than 1/6=0.1667. Wow. But could this result just be a random fluctuation of the null hypothesis? This can be calculated, but unfortunately it is quite complicated. So let’s take a sip of Lady Otterline’s whisky and take a deep breath before we utter the evil word binomial distribution (Fig. 1) – knowing full well that we will lose half of the students when we explain this formula, regardless of whether they have secretly sipped Lady Otterline’s whisky or not.

Fig.1: Binomial distribution

How to make students happy

If you like looking into empty eyes under smoking heads, you can follow up this calculation with one on the normal distribution (Fig. 2), where more than just two values can be assumed. For real maths nerds, this may be a nice exercise, but for the others (and apart from minor random fluctuations, that’s usually everyone), it will take all the fun out of statistics. And rightly so. Because the following applies to both binomial and normal distributions: the probabilities you are looking for cannot be calculated manually. You need at least a good pocket calculator or, if possible, a computer. And because these things happen to be on every desk anyway, there is nothing to stop the effect of chance being simulated automatically. Modern software packages make this easy to implement and the experiment can be repeated tens of thousands of times in a matter of seconds – without having to resort to binomial coefficients and other nasty formulae. Happy students!

Fig. 2: The normal distribution

Long live progress!

Thanks to modern statistical software with didactically prepared packages, such simulations can be easily realised with intuitive keywords. All you have to do is learn a few code snippets (to put it in new German) and – with good instructions – you can also use GenAI to create the command lines. My experience with this approach in the degree programmes in the Department of Economics has been extremely positive. If problems arise at all when using the computer, they are less to do with understanding the simulation. Students are more likely to fail to open an Excel file or to find it again after saving it. But it really can’t hurt to learn this. Just as little as a few splashes of water in a good single malt. But please: always pour the whisky first and feel your way around drop by drop!