Hypothesis Testing - How to make a decision when you do not know

Types of Hypothesis

Suppose you are asked to determine whether a coin is is fair or biased in favour of heads. The statement that the coin is fair and it shows same probability each time tossed, is null hypothesis while the statement that the coin is biased in favour of heads is alternative hypothesis.

• H0: The coin is fair (Probability of getting a head= 50%)
• Ha: The coin is biased in favour of heads (Probability > 50%)

Once null and alternative hypotheses are defined, we should specify how much error we will accept when concluding if the null hypothesis (H0) is True or False.

Significance Level

When we make conclusion for an experiment, we can make correct decisions when the null hypothesis is True or False. However, in some case we could potentially make decision that the null is False in fact when it is True. This is called Type 1 Error (ie. False Positive).

This error can occur since we do not test with a population but with a sample. The random sampling potentially causes Type 1 Error. Say we give Drug A to two groups (1 and 2) and they will mostly show similar recovery rate since it is the same drug. However, due to random sampling (eg. The Group 2…