Taylor Series of f(x) at x=a

Where does the Taylor Series come from?

Suppose that $f$ has a power series, so

$$f(x)=a_0+a_1x+a_2x^2+a_3x^3+\dots$$

To isolate one coefficient, we can make all of the variables on the right cancel by taking $f(0)$. To make the variables on the left cancel, we take the derivative a sufficient amount of times until they cancel. For instance, say we want to find $a_3$ in this series. To cancel $a_0$, $a_1$, and $a_2$, take the third derivative:

$$f'(x)=a_1+2a_2x+3a_3x^2+4a_4x^3+\dots$$ $$f''(x)=2a_2+6a_3x+12a_4x^2+20a_5x^3+\dots$$ $$f'''(x)=6a_3+24a_4x+60a_5x^2+120a_6x^3+\dots$$

Then plug in $x=0$ to cancel the right terms.

$$f'''(0)=6a_3$$

Therefore,

$$a_3=\frac16f'''(0)$$

To generalize for the $a_n$ term, we focus just on the $a_nx^n$ term because the other terms will cancel:

$$a_nx^n\to na_nx^{n-1}\to n(n-1)a_nx^{n-2}\to n(n-1)(n-2)\dots(n-k)a_nx^{n-k}$$

Take $k=n-1$ to get $n(n-1)(n-2)\dots(2)(1)a_nx^1$ or $n!a_nx$. Take the derivative one more time to get $n!a_n$. Since the derivative was taken $n$ times to cancel the other terms, we get that

$$n!a_n=f^{(n)}(0)\implies a_n=\frac{f^{(n)}(0)}{n!}$$

Plugging this back in for each coefficient, we get

$$f(x)=f(0)+f'(0)x+\frac{f''(0)}2x^2+\frac{f'''(0)}6x^3+\dots+\frac{f^{(k)}(0)}{k!}x^k+\dots$$

This is the Taylor series formula!

Now, we come to the fascinating subject of Taylor Series. Before, we delve into this topic, let us do a quick check.

To get started, let us consider the function $y = \frac {1}{1-x}$. If you have practiced a lot with geometric series, you might recall and/or notice that this is the sum of a geometric series with first term $1$ and common ratio $x$; namely $$1 + x + x^2 + \cdots$$Believe it or not, you just witnessed a Taylor Series!

So, first off, what's our motivation for write a nice-looking function such as $y = \frac {1}{1-x}$ as a series that looks much longer (in fact, of infinite length)? Well, it's not immediately with a function such as this one since the series and function are so simple. Let's take a more interesting, namely $y = e^x$. Now, let us say we wanted to find $e^{0.1}$.

Usually, we like to know things. Most of the time, we like to be exact as we can, especially in math. There's one thing we have seen earlier in the Expii calculus map. Do you remember what it was?

Indeed, we can use the linear approximation. Then, we get that, if we set $f(x) = e^x$, we get $f'(x) = e^x$, so $$\begin {align*} f(0.1) &\approx f(0) + 0.1 \cdot f'(0) \\&= e^0 + 0.1 \cdot e^0 \\&= 1 + 0.1 = 1.1. \end {align*}$$ It turns out, unsurprisingly, that this is much closer the exact value of $e^{0.1}$.

We can go one step further and use a quadratic approximation. We know that $f''(x) = e^x$ and we get $$\begin {align*} f(0.1) &\approx f(0) + f'(0) \cdot 0.1 + \frac {f''(0)}{2} \cdot 0.1^2 \\&= 1 + 0.1 + \frac {0.1^2}{2} \\&= 1.105. \end {align*}$$Lo and behold, this is even closer. A natural question ask is $-$ if we get closer at each step, can we ever hit the exact value.

It turns out we can continue to use higher-order approximations: the cubic, the quartic, etc. However, we never get the exact value until we use an infinite number of terms. However, we can get remarkably close.

Now, let's look back at what we just did. We wrote a two-term sum for the linear approximation, a three-term sum for the quadratic approximation, and in general, we can write an $n$-term sum for the $(n+1)^{\text{th}}$-order approximation. And, of course, we can write an infinite series that includes all the terms possible, so that, if we want the $k$th order approximation, we just take the first $k+1$ terms and, boom, we have it! We call this infinite series a Taylor series.

Fun fact. A Taylor series centered at $x=0$ is called a Maclaurin series.

By now, you should have been able to guess the general form of the series for $f(x) = e^x$, since we conveniently have that $f^{(n)}(e^x) = e^x$, for all $x$. (Recall that $f^{(n)}(x)$ is the $n$th derivative of the function $f(x)$.) Centered around $x=0$, $$e^x = \displaystyle\sum_{n=0}^{\infty} \frac {1}{n!} x^n = 1 + x + \frac {1}{2} x^2 + \frac {1}{6} x^3 + \frac {1}{24} x^4 + \cdots$$ Exercise for the reader. Try to figure out why this is the case.

A natural question we might and should ask, at this point, is, can we write a Maclaurin series for any function $f(x)$. The answer is yes, and we leave it as an exercise for the reader to try to see why this is the correct formula: $$f(x) = \displaystyle\sum_{n=0}^{\infty} \frac {f^{(n)}(0)}{n!} \cdot x^n = f(0) + \frac {f'(0)}{1!} \cdot x + \frac {f''(0)}{2!} \cdot x^2 + \cdots$$ Now, let us say we do not want to center it at $x = 0$. Instead, we want to compute something like $f(1.1)$ and $f(1)$ is very trivial. Then, we might want to center it at $x_0 = 1$ for example. So, for a general "root" $x_0$, we can say the Taylor series defining the function is $$f(x) = \displaystyle\sum_{n=0}^{\infty} \frac {f^{(n)}(x_0)}{n!} \cdot \left(x-x_0\right)^n \\= f(x_0) + \frac {f'(0)}{1!} \cdot \left( x - x_0 \right) + \frac {f''(0)}{2!} \cdot \left( x - x_0 \right)^2 + \cdots$$ You can practice creating Taylor series in the "Practice" part of this section and you can learn more about manipulating Taylor series in the sections after this.

A Taylor Series, in simplistic terms, is a Taylor polynomial of infinite degree. More formally, if $P(x)$ is the Taylor series for a function $f(x)$, infinitely differentiable at a point $a$, about that point, and $f_n(x)$ is the $n$th degree Taylor polynomial approximation about the same point, then

$P(x) =$ $\lim_{n\to\infty} f_n(x)$.

This limit function will take the form

$P(x) = f(a) + f'(a)(x-a) + \frac{1}{2} f''(a)(x-a)^2 + \frac{1}{6} f'''(a)(x-a)^3 +$ $...$

or

$P(x) =$ $\sum_{n=0}^{\infty} \frac{f^{(n)} (a)(x-a)^n}{n!}$.

0 is often the best choice for the point $a$ to make the Taylor series easy to evaluate, but not always. For example, the function $ln(x)$ is not defined at 0, and neither are any of its derivatives. Think about what point would make a better choice for the center of the Taylor series expansion.

The best choice would be $x=1$. About this point the series becomes

$1 + (x-1) - \frac{1}{2} (x-1)^2 + \frac{1}{3} (x-1)^3 - ...$

However, of note is the fact that is we try to evaluate this function at a point $x$ such that $|x-1| \geq 1$, the resulting series does not converge! This problem will be investigated further in the next section.

By know, we're familiar with the Taylor polynomials for a function $f$ about a point $a$ - and we know that they can actually be very good approximations of $f$!

Ultimately, however, the Taylor polynomials $g_n$ are just approximations of $f$. And if $f$ is not a polynomial, $g_n$ will never be equal to $f$ for any $n$. However, it turns out that for many functions, a truly remarkable thing happens. Consider the infinite series: $$T_a(x) = \sum_{n=0}^{\infty} \frac{f^{(n)}(a)}{n!} (x - a)^n$$ Clearly, we have $$\lim_{n \rightarrow \infty} g_n(x) = T_a(x)$$

In a sense, $T_a$ - the Taylor series of $f$ at $a$ - is the best "approximation" we can make. It takes the limit of all of our approximations, each of which get closer to $f$, in the hope that we actually reach $f$. The astounding thing is that for many functions, we do! In particular, this is true for $\sin, \cos, e^x$, and many others.

(A small sidenote: when $a = 0$, the Taylor series of $f$ at $a$ is sometimes called the MacLaurin series of $f$).

Let's try to find some!

Correct!

Great!

Yes! This fact also gives us a new way to calculate the value of $e$ - simply evaluate the series at $x = 1$. This gives us the remarkable equation: $$e = \sum_{n=0}^{\infty} \frac{1}{n!}$$

Taylor polynomials and Taylor series are fantastic mathematical tools, and one of the most beautiful parts of calculus. The idea that many of the more interesting functions we've described so far can be written down as power series is stunning.

Unfortunately, life is not always as good as it is with $\sin, \cos$ and $e^x$. The next section will talk about how we can know how good our approximations are: when is it true that $T_0(x) = f(x)$, and when are they not the same?

Overview of Taylor series

The Taylor series of a function $f(x)$ at a point $x=a$ is a power series centered at $x=a$ modeled off of Taylor polynomial approximations. It has a radius of convergence like any other power series. In ideal situations, the Taylor series will not only converge, but converge to the original function on an open interval containing $a$. In this case, the Taylor series is sometimes called the (power or Taylor) series expansion centered at $x=a$. (Furthermore, the Taylor series is actually the unique power series converging to $f(x)$⁠—which makes Taylor series all the more natural!)