Thu 07 December 2017 | tags: statistics, hypothesis testing, -- (permalink)

So I recently had a first stage phone interview, that didn’t go as well as I wanted. The biggest thing was that a statistics question threw me off. The frustrating part was that I’m pretty sure that I knew the answer. I was just nervous and thrown off my game.

The interviewer person threw me a question about calculating significance levels. It was a broad question that I probably had encountered somewhere before.

I wanted to document my thoughts and essentially make sure I am able to better deal with hypothesis testing type questions better in the future. Hence the peapod post about hypothesis testing and significance!

Hypothesis Testing

To understand the deluge of data that is available today, statistics is more important than ever before. One of the biggest implementations is in understanding the results of an experiment that you may have conducted. Statistics allows you to test a hypothesis that you may have formed in response to a problem you may have.

A hypothesis test essentially allows you to analyze two contradictory statements about the results of an experiment and determine which statement is best supported by the observed experimental data.

I think to best understand this concept, its good to go back to basics and understand what a hypothesis and a null hypothesis is.

What is a hypothesis?

A hypothesis is essentially a statement that explains a problem. It links an independent variable to a dependent one. From my experience, the best clearly worded hypotheses are ones that utilize an if-then statement.

Example hypothesis statements are:

1. If the price of the cars are lowered, then more customers will come to the car dealership.
2. If school started later in the day, then students would have better grades.

These hypotheses are proposed solutions to a problem.

• If the price of the cars are lowered, then more customers will come to the car dealership.

Problem: How can the number of customers coming to the dealership be increased?

Dependent Variable: Number of customers coming to the dealership

Independent Variable: Lowering the price of the cars

• If school started later in the day, then students would have better grades.

Problem: How can student grades be improved?

Dependent Variable: Student academic performance

Independent Variable: Changing the start of school to later in the day

What is a Null Hypothesis?

Null hypotheses are an important concept in statistics. It is used to essentially check the validity of your hypothesis based on the experiment results. Null hypotheses assume that any kind of observed difference you see in your dependent variable after conducting an experiment is essentially due to chance.

For our examples, the null hypotheses would be:

1. If the price of the cars are lowered, then there would be no change in the number of customers coming to the dealership.
2. If school started later in the day, then students would perform the same as they did with an earlier start time.

Okay, now that we have those terms settled, now what?

Lets think of this in terms of a real imaginary experiment. Say that we are testing a new process that improves a chemical process for the production of product A. The improvement was tested on 100 samples and produced units that had a mean length 3.2 inches with a standard deviation of 0.5. The old process produced product A that had lengths of 2.9.

Did the new process really improve the lengths of product A?

This is essentially a question of Statistical Significance.

Statistical Significance.

So in this example experiment, the hypothesis was that If this new process is used, then the length of Product A will increase. The results of the experiment produced an increase of lengths to 3.06 inches with a standard deviation of 0.5.

To understand the significance of this result, we have to first assume that the null hypothesis is true. The null hypothesis in this scenario is If the new process is used, it will have no affect on the lengths of Product A. This essentially means that the lengths of Product A would not change regardless of the new process.

Assuming that the lengths are distributed normally, if the null hypothesis is true, the grades would be normally distributed about a length of 2.9. To figure out if the results are statistically significant, a calculation on the probability of getting a average length of 3.06 with a standard deviation of 0.5 must be done.

To perform this calculation, the standard deviation of the sampling distribution must be found. This is done by the following formula:

The standard error isn’t known so we can approximate standard deviation of the sampling distribution using the sample standard deviation of 0.5. This leads to a value of 0.05.

We are essentially asking how many standard deviations away is 3.06 from 2.9, this involves finding the Z-score. The formula for Z-score can be found here.

The calculated Z-score for this analysis is 3.2. This means that the length of 3.06 inches is 3.2 standard deviations away from the mean of 2.9 inches. Looking up the associated p-value on a standard normal probabilities table. I also used this handy dandy tool.

If we assume that the new process has no affect, then the probability of getting a length of 3.06 is 0.00687%. Meaning that this is a strong indicator that the null hypothesis is wrong and the new process is significant in improving the length of Product A.

The threshold p-value is typically 0.05, anything smaller than this value indicates strong evidence against a null hypothesis.

Another way to express this is to say that the probability of obtaining a Type 1 error is very low. A Type 1 error is the mistake of rejecting the null hypothesis despite it being true. A type 1 error is essentially a False Positive.

The opposite of this would be keeping saying a null hypothesis is correct when it really isn’t. This type of mistake is known as a Type II error or a False Negative.

One-tailed or Two-tailed Hypothesis Test

To quickly touch on this concept, the scenario painted above is a one-tailed hypothesis test. In these types of tests, one is usually trying to see if an experiment can improve or make worse a known observation. These tests are only testing one or the other.

Two-tailed hypothesis tests are just trying to see if any change can occur when an experiment is done.

For our hypothetical scenario, if we wanted to reword our hypothesis and make it into a two-tailed test. It would be as follows: If this new process is used, there will be a change in the length of the Product.

This link gives a pretty good overview.

In the next couple posts I’ll quickly go over a couple of types of hypothesis tests that are relevant in businesses today.

Thanks for reading! See you in the next pod.