Lagrange Error Bound

Suppose that we have a function and its $n$th order Taylor approximation. We wish to know how well the Taylor approximation approximates the value of the function. The Lagrange Error Bound provides a simple and powerful bound on the error of the Taylor approximation.

The Lagrange Error Bound is as follows:

Let $f$ be a function that is continuous and has all of its derivatives also continuous. Let $P_n(x)$ be the $n$th order Taylor approximation of $f(x)$ centered at $a$, and let the error function be $E_n(x)=f(x)-P_n(x)$. Then: $$|E_n(x)|\le \frac{M}{(n+1)!}|x-a|^{n+1}$$ where $M$ is a value such that $|f^{(n+1)}(x)|\le M$ on the interval between $a$ and $x$.

Note that the absolute value signs are necessary to force everything to be positive, since the signs generally do not behave as nicely as we would wish them to. However, since we usually just care about how close our estimate is to the real value of the function and not whether our estimate is too high or too low, this theorem is very useful.

It turns out that the proof is actually quite simple! All you need to do is take the inequality, true on the interval between $a$ and $x$: $$|E_n^{(n+1)}(x)|\le M$$ Then, you just integrate it $(n+1)$ times, using the fact that $E^{(k)}_n(a)=0$ for all $0\le k\le n$: $$\int_{a}^{x}|E_n^{(n+1)}(x)|dx\le\int_{a}^{x}Mdx$$ $$|E_n^{(n)}(x)|\le M|x-a|$$ $$\int_{a}^{x}|E_n^{(n)}(x)|dx\le\int_{a}^{x}M|x-a|dx$$ $$|E_n^{(n-1)}(x)|\le \frac{M}{2}|x-a|^2$$ $$...$$ $$\int_{a}^{x}|E'_n(x)|dx\le\int_{a}^{x}\frac{M}{n!}|x-a|^{n}dx$$ $$|E_n(x)|\le \frac{M}{(n+1)!}|x-a|^{n+1}$$ This is exactly what we wanted, so we are done.

That formula may be a little bit tough to digest, but it is actually quite intuitive. Let us reason intuitively as to why the above error bound holds.

Note that the Taylor approximation has been engineered so that: $$P_n(a)=f(a)$$ $$P'_n(a)=f'(a)$$ $$P''_n(a)=f''(a)$$ $$...$$ $$P^{(n)}_n(a)=f^{(n)}(a)$$ Therefore, $E^{(k)}_n(a)=0$ for all $0\le k\le n$. This means that the first derivative that could be nonzero is the $(n+1)$th order derivative. However, there is actually a very simple function that also has this property!

Note that the polynomial function $g(x)=K(x-a)^{n+1}$ satisfies the relation $g^{(k)}(a)=0$ for all $0\le k\le n$. Thus, we expect that the error function $E_n(x)$ behaves similarly to the polynomial $g(x)=K(x-a)^{n+1}$. But what should be the value of $K$?

If we differentiate $g(x)$ $(n+1)$ times, we get: $$g(x)=K(x-a)^{n+1}$$ $$g'(x)=K(n+1)(x-a)^{n}$$ $$g''(x)=K(n+1)(n)(x-a)^{n-1}$$ $$...$$ $$g^{(n+1)}(x)=K(n+1)!$$

Thus, if we want to approximate $E_n(x)$ by a polynomial $g(x)=K(x-a)^{n+1}$, we should set: $$K=\frac{E_n^{(n+1)}(x)}{(n+1)!}$$ But the value of $E_n^{(n+1)}(x)$ need not be constant; we want a constant be the coefficient of our approximating polynomial. Well, since we want an upper bound on the error, it would make sense to use the maximum magnitude of $E_n^{(n+1)}(x)$ on the relevant interval, which is the interval between $a$ and $x$.

Now, if we let $M=\max(|E_n^{(n+1)}(x)|)$, then we could approximate the function $E_n(x)$ by the polynomial $g(x)=\frac{M}{(n+1)!}(x-a)^{n+1}$. Then, it would seem to make sense that: $$|E_n(x)|\le \frac{M}{(n+1)!}|x-a|^{n+1}$$ This is exactly what we were searching for! Thus, if you have trouble remembering the Lagrange Error Bound, just try to estimate the error function using a simple power function.

Overview of Lagrange error bound

If $T_n(x)$ is the degree $n$ Taylor approximation of $f(x)$ at $x=a$, then the Lagrange error bound provides an upper bound for the error $R_n(x) = f(x) - T_n(x)$ for $x$ close to $a$. Namely, if $|f^{(n+1)}(x)| \le M$ on an interval $I = (a-R,a+R)$ centered at $a$, for some radius $R>0$, then $$|R_n(x)| \le M\frac{|x-a|^{n+1}}{(n+1)!} \le M\frac{R^{n+1}}{(n+1)!}$$ for all $x$ in the interval. This will be useful soon for determining where a function equals its Taylor series. (If you're curious about the proof of the Lagrange error bound, there are basically two common ways to prove it: the Lagrange mean-value form of the remainder, or the integral form of the remainder.)