Statistical Decisions

When statistical errors have real consequences

What is at stake?

Statistical decisions are often presented as technical decisions. But the consequences of being wrong are not necessarily symmetrical.

For each situation, construct the complete decision matrix. Decide which outcome is a Type I error, which is a Type II error, and which outcomes contain no statistical error.

You must also decide what actually happens in each of these four situations. Only after the complete matrix is correct can you move on to the more difficult question: which mistake is worse?

Remember: a Type I error occurs when H₀ is true, but you reject it. Its probability is α.

A Type II error occurs when H₁ is true, but you fail to reject H₀. Its probability is β. Statistical power is 1 − β.

Choose a situation:

H₀

H₁

Construct the decision matrix

For every box, make two independent decisions. First decide whether the statistical outcome is a Type I error, Type II error, or no error. Then select the real-world consequence that belongs in that box.

True situation
H₀ is true	Statistical classification Consequence	Statistical classification Consequence
H₁ is true	Statistical classification Consequence	Statistical classification Consequence

Now make the difficult decision

Complete the decision matrix correctly before making this decision.

The statistical classification has a correct answer. The moral comparison does not necessarily have one.

Compare the consequences of the two errors. Which mistake do you consider worse? Then independently decide which statistical error rate should receive more attention.

Which error is worse?

Which error rate should be minimized more strongly?