Differential (dy) Notation and Intuition

Introduction

Now that you've learned what a derivative is, you've seen things like this: $$\frac{dy}{dx} = 4x + 2$$ The $dy$ and $dx$ are called differentials, but you're probably wondering: what does $dy$ mean? What does $dx$ mean? Unfortunately, these are some of the most confusing questions in calculus. The purpose of this post is to explain why these are difficult questions and to explain the possible answers.

You might be saying, "the possible answers?!? I thought this was math!" Well, sorry. If you thought math was free of subjectivity, you were mostly right, but ultimately wrong. There are choices that have to be made: What symbols should we use? How should we define certain concepts? What are our axioms? What counts as a proof? Once these choices are made, the rest is as objective as can be; however, the right choices aren't always obvious.

The Differential as an Infinitesimal Quantity

Let's take a trip back in time. The first historical traces of calculus are generally attributed to the ancient Greeks. Democritus considered dividing an object into infinitely many cross-sections, and adding the volumes of the cross-sections together to get the total volume. He ultimately concluded the method was invalid and never actually used it. Archimedes used Eudoxus's method of exhaustion to calculate areas of circles and even areas bounded by parabolas, dividing areas into small rectangles and adding them in a way strikingly reminiscent of modern calculus.

The idea of an infinitesimal figured heavily into ancient Greek mathematics. An infinitesimal was thought of as an infinitely small number, too small to describe, and so small that infinitely many can be added to get a finite number. Sounds strange, right? It should make you wonder, is there really any such thing? Can we actually use them in math?

In the 17th century, these questions still hovering, Leibniz and Newton brought calculus into its own. They offered definitions for the derivative and integral, and they proved the fundamental connection between the two. Each developed his own notation, but Leibniz's stuck; we owe to Leibniz the integral sign, $\int$, and our present topic, the differentials $dy$ and $dx$.

For Leibniz, the differentials $dy$ and $dx$ represented infinitesimal differences in $y$-values and $x$-values, respectively. Recasting Pierre de Fermat's earlier idea of "adequality" into new terminology, he imagined starting from a point $(x, y)$ on a function's graph, taking another point infinitely close to that point, calling the differences in $y$-values and $x$-values $dy$ and $dx$, and calculating slope as $dy/dx$. Since the second point is infinitely close to the first, he reasoned, the slope $dy/dx$ is the slope of the tangent line:

You can use differentials to calculate derivatives. Let's say we have $y = x^2$. Then, if we're thinking of differentials the way Liebniz did, we have: $$y + dy = (x + dx)^2$$ $$y + dy = x^2 + 2x\cdot dx + (dx)^2$$ $$dy = 2x\cdot dx + (dx)^2$$ $$\frac{dy}{dx} = 2x + dx$$ Then since $dx$ is infinitesimally small, we can ignore it, so that $dy/dx = 2x$.

Critiquing the Concept of Infinitesimals

Not all were convinced that these methods were valid. For example, we ignored the $dx$ in the last equation above, but we did not ignore the $dx$ in the first equation! On what basis can we ignore some $dx$'s but not others? Or, we might ask, what are these $dx$'s and $dy$'s anyway? Are they real numbers (as in, numbers in $\mathbb{R}$)? Or something else? As mathematicians became more and more concerned with putting all of mathematics on solid axiomatic footing, these questions took on grave significance.

The answer came in the form of limits. Rather than thinking of the derivative as a quotient of infinitesimals $dy$ and $dx$, we began to think of the derivative as a limit of quotients $\Delta y/\Delta x$, as $\Delta x$ becomes infinitesimally small--or, as we now say, as $\Delta x$ approaches 0.

Mathematicians succeeded in putting this conception of calculus on solid axiomatic footing, capped by Bolzano's introduction of a fully rigorous $\epsilon$-$\delta$ definition of a limit in the early 1800's. The concept of an infinitesimal seemed destined to fade into history and out of mathematical practice. In 1919, Bertrand Russell wrote:

"The conception of a 'limit' is one of which the importance in mathematics has been found continually greater than had been thought. The whole of the differential and integral calculus, indeed practically everything in higher mathematics, depends upon limits. Formerly, it was supposed that infinitesimals were involved in the foundations of these subjects, but Weierstrass showed that this is an error: wherever infinitesimals were thought to occur, what really occurs is a set of finite quantities having zero for their lower limit." (Introduction to Mathematical Philosophy, p. 88)

The Differential as a Difference Along a Tangent Line

Despite the thought that infinitesimals were an "error," Leibniz's $dy/dx$ notation persevered, even in a calculus based on limits, not differentials. Some might say it persevered merely as a historical relic; others might point to its ability to evoke the concept of slope. Whatever the reason, the notation remained, though it took on a different meaning: $dx$ became a small difference in $x$, and $dy$ became the corresponding difference in $y$ along a tangent line.

This leads to nice calculations of tangent-line approximations. Let's say we want to estimate $\sqrt{1.01}$. We take $y = \sqrt{x}$, and consider a tangent line at $x = 1$. By evaluating the derivative at $x = 1$, we get: $$\frac{dy}{dx} = \frac{1}{2}$$ $$dy = \frac{1}{2}\cdot dx$$ Then our estimate is $$y + dy = 1 + \frac{1}{2}\cdot 0.01 = 1.005$$

Nice, right? But make sure you notice, these differentials $dy$ and $dx$ don't quite mean the same thing that Leibniz meant when he first penned them. They are merely small real numbers, not infinitesimals. And they are not differences along a function; they are differences along a tangent line.

The Differential as an Infinitesimal, Revisited

Still, a few rogue mathematicians persisted in trying to save the concept of the differential as an infinitesimal, trying to put Leibniz's approach to calculus on firm mathematical ground. And actually, they succeeded.

In the early 1960's, Abraham Robinson originated the field of non-standard analysis, a rigorous treatment of calculus using infinitesimals rather than limits as its fundamental concept. He defined a set of numbers called the hyperreal numbers, which essentially enlarged the set of real numbers to contain both infinitely small and infinitely large numbers.

These infinitely small numbers are numbers greater than zero, but smaller than every real number. They are not real numbers, but can be compared to real numbers, added to real numbers, and so on. Sound sketchy? You may think so, but Robinson managed to prove that the hyperreals can be constructed from the set of real numbers $\mathbb{R}$. That means that if the real numbers are logically consistent (which we generally assume is true), Robinson's hyperreals are too.

Another mathematician, Edward Nelson, developed an alternative way of doing non-standard analysis, called Internal Set Theory, which doesn't involve introducing any new numbers. Instead, he introduces three axioms to the usual ZFC set axioms, along with a new predicate, "standard." Some real numbers are standard, and some are not, and you can't exactly say which are which, but intuitively, "standard" numbers are those real numbers that you could describe in a reasonable amount of time. As unusual as this may seem, Nelson proved that any proof in this system carries over to our usual system of mathematics. This results in some beautifully strange proofs of the basic results of calculus.

The point here is that the concept of differentials as infinitesimal quantities is not an incoherent notion; it is merely a notion that belongs to a different formal approach to calculus than the one most of us are used to. In fact, introductory undergraduate calculus courses have been taught using this approach; see here. Russell turned out to be wrong. Infinitesimals were not an error; they were merely a house that had not yet found its foundations.

Conclusion

In summary, there are two different concepts of the differentials $dx, dy$. The difference is not due to ignorance, but to two different formal approaches to calculus, both of which are valid, and both of which include all of the same results, but different ways of reaching them.

Under the more common approach, the approach based on limits, $dx$ and $dy$ refer to small differences along a tangent line. As we saw, this leads to nice tangent-line approximations and can be considered a convenient re-appropriation of Leibniz's notation.

Under the more ancient but now less common approach, the approach based on infinitesimals, $dx$ and $dy$ refer to infinitesimal differences along a curve. This helps reinforce our intuition for what a derivative is, and allows us to manipulate $dx$'s and $dy$'s algebraically like we feel we should be able to.

Now that you're aware of this distinction, you can react however you'd like. You may choose to reject one approach in favor of the other, which will help you solidify your knowledge of one particular formal system. Or, you may choose to draw from the fruit of each approach, thinking of derivatives as limits at some times and as ratios of infinitesimals at others, whichever suits the moment.

Either way, allow yourself to be amazed by the simple elegance of each approach, and by this complexity that lies even at the very foundations of the subject of calculus.

A differential $dx$ is not a real number or variable! It is just a common component of notation in calculus.

However, it can intuitively be thought of as "a very small change in $x$", and it makes lots of the notation in calculus seem more sensible.

The derivative $\frac{dy}{dx}$, for example, intuitively means "if we add a very small change to $x$, what is the very small change that $y$ experiences?"

Since both $dy$ and $dx$ are so small, we can't actually evaluate either one as a real number, but we can compare "how small" the changes are by taking their ratio. If $x$ increases by $dx$, and this causes $y$ to increase by approximately $7 \cdot dx$, then the derivative $\frac{dy}{dx} = 7$.

Let's play with the simple $y = x^2$ near $x = 1$ to understand this better.

We might write that $\Delta y = 0.21$ in this case. Note that $\Delta x$ and $\Delta y$ are real variables. They have an actual size, and we can compute $$\frac{\Delta y}{\Delta x} = 2.1$$ as a normal division of reals.

Unfortunately, they're not small enough.

Again, we can define two normal variables as a normal division, we compute $$\frac{\Delta y}{\Delta x} = 2.0001.$$

You can probably imagine doing this for smaller and smaller $\Delta x$ and computing $\Delta y / \Delta x$. Unfortunately, we'll never quite reach the derivative we want, which is $2$; there will always be a little change left over.

But we also can't just set $\Delta x$ to zero; there has to be some change in $x$ to produce a change in $y$, or we'd be dividing by zero.

So, instead of writing a real variable $\Delta x$, we write the differential $dx$. This is a pretend-variable that conceals a limit --- it means "a very small change" and tells the reader to imagine that its limit is being taken to zero. The downside of this is that it's impossible to specify exactly how much that change is, so it isn't a real variable and we can't manipulate it like we can manipulate real values unless we're extremely careful.