Probability: Linearity of Expectation

Linearity of Expectation

Suppose we have random variables $X$ and $Y$.

Linearity of expectation says that $\mathbb{E}[X + Y] = \mathbb{E}[X] + \mathbb{E}[Y]$.

We can prove this using definitions. $$\begin{align*} \mathbb{E}[X + Y] &= \sum_{x \in \Omega} \sum_{y \in \Omega} (x + y)Pr(X = x, Y = y)\\ &= \sum_{x \in \Omega} \sum_{y \in \Omega} x Pr(X = x, Y = y) + y Pr(X = x, Y = y)\\ &= \sum_{x \in \Omega} x \sum_{y \in \Omega} Pr(X = x, Y = y) + \sum_{y \in \Omega} y \sum_{x \in \Omega} Pr(X = x, Y = y)\\ &= \sum_{x \in \Omega} x Pr(X = x) + \sum_{y \in \Omega} y Pr(Y = y)\\ &= \mathbb{E}[X] + \mathbb{E}[Y] \end{align*}$$

This proof can be generalized to any finite number of random variables, as well as constant multipliers on the random variables.

The Linearity of Expectation

In probability, we can apply the Linearity of Expectation.

The Linearity of Expectation is a fundamental property in probability theory that tells us that the expected value of the sum of random variables is equal to the sum of their individual expected values.

We can apply this concept very generally, so it's applicable across probability and computer science.

When we are dealing with random variables, such as the number of heads obtained when flipping a coin multiple times, we often need to figure out the expected value of their sum. The linearity of expectation tells us that we can do this by breaking down the calculation into simpler components. For example, we can find the expected value of heads on a single flip, which will yield us a value of $0.5$.

If we were to make this situation more complex, such as flipping a coin twice, we can apply the linearity of expectation by summing the expected values of each individual random variable. So, we could say that the expected number of heads in two flips is simply the sum of the expected number of heads in each flip. The expected value would be $0.5 + 0.5 = 1$.

This may seem familiar to the concept of weighted average, where each random variable's contribution is weighted by its probability. By calculating the expected value of a sum using the linearity of expectation, we streamline the process, which will be very helpful in more complex math.

(Video) Linearity of Expectations

by Eat That Logic

In this video by Eat That Logic you will learn about Linearity of Expectation with a walkthrough example.

Summary

You will review how to calculate the expected value of a random variable using the possible values and their probabilities. The video then introduces properties that can make it easier to find the expected value of a new random variable. You will see how constants and expressions involving a random variable can be used to calculate an expected value.

The video also uses examples to show how these properties work when solving problems. By the end, you will understand how the properties of expected value can help you solve problems without calculating every value from scratch.

Hint: Expected value is linear, which means you can use rules for sums, differences, and constant multiples to simplify calculations.