大数定律

The law of large numbers can be summed up with a quick example!

Let's say you have a single die that's numbered $1$ to $6.$ Somebody asks you what the expected value of the dice is. Now you might not know what the law of large numbers is, but you probably know how to calculate the expected value of a dice! It's just:

$\frac{1}{6}(1+2+3+4+5+6) = \frac{21}{6} = 3.5$

Let's think about why that is. Well, the probability of rolling each of the numbers is technically equal. You might roll a dice and get $6$ $1$s, or you might roll it again and get $2$ $2$s and $4$ $5$s. You're not always going to get each number from $1-6$ in $6$ rolls! That's just what the expected value predicts.

Basically, what the law of large numbers says is that the average of the results of the experiment performed a very large number of times should be close to the expected value. To translate this over to the dice problem: the 'experiment' is the rolling of the dice $6$ times and finding an average value for the $6$ rolls. If you do this experiment a million, a billion, or infinity times, the average of all of these average values should equal the expected value, $3.5!$

The law of large numbers

The law of large numbers is a probability principle where given enough trials or instances, an event with the same likelihood of occurrence evens out.

In other words, as the number of experiments increases, the probability of an event occurring will converge on the theoretical, or expected probability.

For instance, if we keep flipping a coin for all of eternity we will see that seeing a heads or a tails will happen with an equal 50/50 probability. However if we only flip it a couple of times, we may see all heads or all tails or some other non-equal amount of heads and tails.

Law of Large Numbers

For experiments with a high number of trials, we have a theory called the Law of Large Numbers.

The Law of Large Numbers says that: The average of the results of the experiment performed many times should be close to the expected value.

It might be easier to think of an example! Think of a coin toss, a class example of a random experiment where the outcome (heads or tails) is determined by chance. This is also an example of an IID (Independent and Identically Distributed trial). The Law of Large Numbers tells us that as the number of trials increases (the sample size increases), you are more and more likely to get exactly half heads and half tails.

(Video) Law of Large Numbers - Explained and Visualized

by TED-Ed

Watch this video to learn more about the law of large numbers!

As this video explains, the law of large numbers states that the average results of an experiment will get closer to the expected value as the number of experiments increase.

Follow along with the example about flipping a coin. The first few times you flip a coin, you may get a lot of heads in a row, which will skew your proportion of heads to tails. However, as you continue flipping coins, you will start to get more tails. Eventually, your proportion will even out to a 0.5 split between heads and tails. However, I encourage you to look at the graph in this video. You'll see that the probability between heads and tails goes up and down, eventually settling at the expected value of 0.5.