What Is Quadratic (2nd Order) Taylor Approximation?

Now, after figuring out the zeroth order approximation, we will try to find out the formula for a quadratic approximation, or an approximation of degree 2 at x = a.

First, the formula for a quadratic approximation is for a function at x = 0 is: $$f(x) \approx f(0) + f'(0)x + \frac{f''(0)}{2}x^2$$

The quadratic approximation is also called the second degree Taylor polynomial. The rationale behind the formula will be explained below. First, let's practice, and see how this can be useful:

Let's now see why our formula is a good one. Consider a function $f(x)$. Its zeroth (green) and first order (red) approximations are graphed below.

Now, in non-rigorous terms, if we want our approximation to be more accurate, we have to "bend" it in an attempt to make the two curves coincide. For example, consider our original function and the two approximations. Each degree reduces the error of the approximating function significantly, as each degree makes the approximation "bend" toward the curve. With high degrees, the difference becomes virtually nothing over certain intervals. For illustration, consider the 4th order approximation of $f(x) = e^x$. (The nth order explanation is explained fully in a later topic.)

Notice that the green and blue curves are nearly coincident from x = 0 to x = 10.

Let's see how close our quadratic approximation formula is.

The graph of $1+x+\frac{1}{2}x^2$ looks like this, and is a fairly good approximation of $f(x)=e^x$ near $x=0$:

Anyway, our goal with this is to find a quadratic approximation for a function $f(x)$ at $x=a$. To do this, we can use a little bit of intuition. As stated before, we desire to "bend" our approximations closer and closer to our original function. To do this, we can set the first and second derivatives of our approximation equal to the original function. This will allow for both curves to have the same derivative and it will allow for both functions to be either concave-up or concave-down. This second part ensures that the functions are at least bending in the same direction.

Therefore, our approximation of $f(x)$ at $x=a$, now referred to as $A(x)$, must be of the form $A(x) = b_1 + b_2(x-a) + b_3(x-a)^2$.

This function's zeroth and first derivatives are: $$b_1 + b_2(x-a) + b_3(x-a)^2$$ $$b_2 + 2b_3(x-a)$$ respectively.

At $x = a$, these derivatives become $$b_1$$ $$b_2$$ $$2b_3$$ Finally, remember that for reasons explained above, we wish to set the first and second derivatives of A(x) and f(x) equal. Therefore, $$\leftrightarrow A(a) = f(a) , A'(a) = f'(a), A''(a) = f''(a)$$ $$\leftrightarrow b_1 = f(a), b_2 = f'(a), 2b_3 = f''(a)$$

This means that our final quadratic approximation is $A(x) = f(a) + f'(a)(x-a) + \frac{f'(a)}{2}(x-a)^2$

In the next article, the error of the second-degree approximation will be discussed.

We'd like to have a better approximation than the constant one, of course. The function $h(x) = 0$ is not a very good approximation of $\sin$, after all, even near $0$. So let's try with a linear function. The core idea that Taylor had is to make the first few derivatives (in fact, the first $n$ derivatives) of $g_n$, the $n$-th order approximation to $f$, match up with those of $f$ at $a$. So, to match, we want to find a linear function $g_1$ such that $g_1(a) = f(a)$ and $g_1'(a) = f'(a)$.

Correct! In fact, ultimately our $g_n$s will be polynomials in $(x - a)$. Now, let's try to define $g_2$. For $g_2$, we want the following to hold:

Correct! If we try this for $\frac{e^x}{2}$ around $0$, we note that our linear and quadratic approximations are much better than the constant one, as you can see below:

In the next couple of sections, we will do some more analysis of these quadratic approximations.

Problem to Illustrate accuracy Quadratic Approximation!

Here is a problem for you to think about!

Given the function $f(x) = 7(1-e^{-0.7x})^2$ (inspired by Morse potential energy) we find the quadratic approximation, $Q(x)$, of the same at its minima. Let us call an approximation valid if the value of $f(x)$ differs from $Q(x)$ by $25\%$. Where does the quadratic approximation break down for $x > 0$ and $x < 0$?

Here is the plot of the Morse potential function (blue) imposed with the quadratic approximation (green) and the linear approximation (red).

We're going to work on a pretty hard problem in this exercise. Don't be scared though!

You're given an infinite increasing sequence of positive integers $\{a_i\}$ such that $$\sum_{i = 1}^\infty \frac{1}{a_i} = \infty.$$

This is the same as saying that $\sum_{i = 1}^\infty \frac{1}{a_i}$ diverges.

Show that $$\prod_{i = 1}^\infty \left(1-\frac{1}{a_i} \right) = 0$$

Let's build some intuition with some example sequences, starting with the simple sequence $a_i = i+1$. Then $\sum_{i = 1} ^ \infty \frac{1}{a_i} = \sum_{i = 1} ^ \infty \frac{1}{i+1}$. This is called the Harmonic Series, which diverges.

Let's find $\prod_{i = 1}^\infty \left(1-\frac{1}{a_i} \right) = \prod_{i = 1}^\infty \left(1-\frac{1}{i+1} \right).$

Good job! I'm sure you see a pattern. Let's work on proving our guess. When we simplify $1-\frac{1}{i+1}$, we get $\frac{i}{i+1}$.

I'll write it out for $n = 5$. We're multiplying $$\frac{1}{2} \cdot \frac{2}{3} \cdot \frac{3}{4} \cdot \frac{4}{5} \cdot \frac{5}{6} = \frac{1}{\not{2}} \cdot \frac{\not{2}}{\not{3}} \cdot \frac{\not{3}}{\not{4}} \cdot \frac{\not{4}}{\not{5}} \cdot \frac{\not{5}}{6} = \frac{1}{6}.$$ Consecutive numerators and denominators cancel.

Therefore, $\prod_{i = 1}^n \left(1-\frac{1}{i+1} \right) = \frac{1}{n+1}.$ Surely, as $n$ tends to infinity, the product goes to zero. So my claim is true in this special case. Now let's tackle our original problem, a more general case.

To be formal, let $P = \prod_{i = 1}^\infty \left(1-\frac{1}{a_i} \right).$ Taking the natural logarithm of both sides gives us $$\ln P = \ln \left(\prod_{i = 1}^\infty \left(1-\frac{1}{a_i} \right) \right) = \sum_{i = 1}^\infty \ln \left(1-\frac{1}{a_i} \right),$$

by our logarithm rules. Seeing many terms of the form $\ln \left(1-\frac{1}{a_i} \right)$, let's look at the Taylor expansion of $\ln(1-x)$.

If we expand out $\ln(1-x)$ even further, we can see that $\ln(1-x) = -x-\frac{x^2}{2}-\frac{x^3}{3}- \dots - \frac{x^n}{n} - \dots .$ If you don't believe me, check Wolfram|Alpha:

http://www.wolframalpha.com/widgets/view.jsp?id=f9476968629e1163bd4a3ba839d60925

In particular, $\ln(1-x) \le -x$, because all terms after that in the Taylor expansion are negative.

Let's continue. Using our inequality above, $$\ln P = \sum_{i = 1}^\infty \ln \left(1-\frac{1}{a_i} \right) \le \sum_{i = 1}^n -\frac{1}{a_i} = -\infty$$ by our initial condition.

Let's take $e$ to the power of both sides now of the equation $\ln P = -\infty.$ We get: $e^{\ln P} = e^{-\infty} \implies P = e^{-\infty}.$

$e^{-\infty} = (e^{\infty})^{-1} = \frac{1}{e^\infty} = \frac{1}{\infty} = 0.$ Remembering what $P$ is, we are done. :)

You can also formalize my above work, since using infinity as a number the way I did is not completely rigorous.

Quadratic approximation of quadratics

What is the quadratic approximation of the function $f(x)=x^2+x+1$ at $x=1$?

We get $f(x)\approx (x-1)^2+3(x-1)+3$. Now, expand this answer as a polynomial in $x$. What do you get?

That's right! We get $x^2+x+1$ back again. The quadratic approximation of a quadratic at $x=1$ is just a different way of writing the same polynomial.

Try the same thing at $x=2$. Do you still get the same polynomial?