In this chapter, we study the rate of change of functions \(f: I \to \mathbb{R}\), where \(I \subset \mathbb{R}\) denotes an interval.
For a linear function \(L: \mathbb{R} \to \mathbb{R}\), \(L(x) = ax+b\), where \(a, b \in \mathbb{R}\) are fixed, the rate of change from the point \(\xi \in \mathbb{R}\) to the point \(x \in \mathbb{R}\) is given by
\(\frac{L(x) - L(\xi)}{x - \xi} = \frac{ax+b - a\xi-b}{x - \xi} = a.\)
Letting \(x \to \xi\), we obtain the rate of change of L at the point \(\xi\):
\(\lim_{x \to \xi} \frac{L(x) - L(\xi)}{x - \xi} = a.\)
For any function \(f: I \to \mathbb{R}\), we can try to compute the rate of change at a point \(\xi \in I\) by the same approach, i.e., we can try to find the limit
\(\lim_{x \to \xi} \frac{f(x) - f(\xi)}{x - \xi}.\)
Definition.
i) A function \(f: I \to \mathbb{R}\) is differentiable at \(\xi \in I\) if the limit
\(f'(\xi) := \lim_{x \to \xi} \frac{f(x) - f(\xi)}{x - \xi}\)
exists. In this case, we call \(f'(\xi)\) the derivative of f at the point \(\xi\). If f is differentiable at every point of I, we say that f is differentiable (on I), and the function \(f': I \to \mathbb{R}\) is the derivative of f.
ii) A function \(f: I \to \mathbb{R}\) is \(\{\text{right/left}\}\) differentiable at \(\xi \in I\) if the limit
\(f'_{\pm}(\xi) := \lim_{x \to \xi\pm} \frac{f(x) - f(\xi)}{x - \xi}\)
is well-defined and exists. In this case, we call \(f'_{\pm}(\xi)\) the \(\{\text{right/left}\}\) derivative of f at \(\xi\). If f is \(\{\text{right/left}\}\) differentiable at every point of I, then we say that f is \(\{\text{right/left}\}\) differentiable (on I), and the function \(f'_{\pm}: I \to \mathbb{R}\) is the \(\{\text{right/left}\}\) derivative of f.
Remark. i) There are different ways to denote derivatives, e.g., we also write
\(f'(x) = \frac{df}{dx}(x) = \frac{d}{dx} f(x).\)
ii) A function \(f: (a, b) \to \mathbb{R}\) is differentiable at \(\xi \in (a, b)\) if and only if f is right and left differentiable at \(\xi\) and \(f'_+(\xi) = f'_-(\xi)\). In this case, we have \(f'(\xi) = f'_+(\xi) = f'_-(\xi)\).
iii) If \(f: I \to \mathbb{R}\) is differentiable at \(\xi \in I\), then f is continuous at \(\xi\):
\(\lim_{x \to \xi} (f(x) - f(\xi)) = \lim_{x \to \xi} \left\{ \frac{f(x) - f(\xi)}{x - \xi} (x - \xi) \right\}\)
\(= \lim_{x \to \xi} \frac{f(x) - f(\xi)}{x - \xi} \cdot \lim_{x \to \xi} (x - \xi) = 0.\)
Examples. i) Let \(k \in \mathbb{N}\) be a fixed integer and \(f: \mathbb{R} \to \mathbb{R}, f(x) = x^k\). Then we have for \(x \neq \xi\)
\(\frac{f(x) - f(\xi)}{x - \xi} = \frac{x^k - \xi^k}{x - \xi} = \sum_{\nu=0}^{k-1} x^\nu \xi^{k-1-\nu}\)
\(\to \sum_{\nu=0}^{k-1} \xi^\nu \xi^{k-1-\nu} = k \xi^{k-1}\), as \(x \to \xi\).
\(\implies\) f is differentiable on \(\mathbb{R}\) with \(f'(x) = k x^{k-1}\).
ii) The exponential function \(\exp: \mathbb{R} \to (0, \infty)\) is differentiable on \(\mathbb{R}\) with \(\frac{d}{dx} \exp(x) = \exp(x)\). To see this, let \(\xi \in \mathbb{R}\) be a point in \(\mathbb{R}\), then we have
\(\lim_{x \to \xi} \frac{\exp(x) - \exp(\xi)}{x - \xi} = \lim_{x \to \xi} \frac{e^x - e^\xi}{x - \xi}\)
\(= \lim_{x \to \xi} e^\xi \frac{e^{x-\xi} - 1}{x - \xi} = e^\xi \lim_{x \to \xi} \frac{e^{x-\xi} - 1}{x - \xi}\)
\(= e^\xi \lim_{t \to 0} \frac{e^t - 1}{t} = e^\xi = \exp(\xi).\)
iii) The function \(f: \mathbb{R} \to \mathbb{R}, f(x) = |x|\), is not differentiable at \(\xi = 0\) as
\(f'_- (0) = \lim_{x \to 0-} \frac{|x|}{x} = -1 \quad\) but
\(f'_+ (0) = \lim_{x \to 0+} \frac{|x|}{x} = 1.\)
Theorem 35.
i) Let \(f, g: I \to \mathbb{R}\) be differentiable at \(\xi \in I\). Then, for any \(\alpha, \beta \in \mathbb{R}\), the function \(\alpha f + \beta g\) is differentiable at \(\xi\) with
\((\alpha f + \beta g)'(\xi) = \alpha f'(\xi) + \beta g'(\xi) \quad \text{"sum rule"}\),
the function \(f \cdot g\) is differentiable at \(\xi\) with
\((f \cdot g)'(\xi) = f'(\xi) g(\xi) + f(\xi) g'(\xi) \quad \text{"product rule"}\),
and, if \(g(\xi) \neq 0\), the function \(\frac{f}{g}\) is differentiable at \(\xi\) with
\((\frac{f}{g})'(\xi) = \frac{f'(\xi) g(\xi) - f(\xi) g'(\xi)}{g(\xi)^2} \quad \text{"quotient rule"}.\)
ii) Let \(f: I \to J\) and \(g: J \to \mathbb{R}\) two functions such that f is differentiable at \(\xi \in I\) and g is differentiable at \(\eta = f(\xi)\). Then \(g \circ f: I \to \mathbb{R}\) is differentiable at \(\xi\) with
\((g \circ f)'(\xi) = g'(f(\xi)) f'(\xi) \quad \text{"chain rule"}.\)
Proof. i) For any \(x \in I \setminus \{\xi\}\) we have
\(\frac{\alpha f(x) + \beta g(x) - \alpha f(\xi) - \beta g(\xi)}{x - \xi}\)
\(= \alpha \frac{f(x) - f(\xi)}{x - \xi} + \beta \frac{g(x) - g(\xi)}{x - \xi} \xrightarrow{x \to \xi} \alpha f'(\xi) + \beta g'(\xi),\)
which proves the sum rule. Moreover, for \(x \in I \setminus \{\xi\}\), we have
\(\frac{f(x) g(x) - f(\xi) g(\xi)}{x - \xi} = \frac{f(x) - f(\xi)}{x - \xi} g(x) + f(\xi) \frac{g(x) - g(\xi)}{x - \xi}\)
\(\xrightarrow{x \to \xi} f'(\xi) g(\xi) + f(\xi) g'(\xi),\)
proving the product rule. Furthermore, for \(x \in U_\delta(\xi) \setminus \{\xi\}\), where \(\delta > 0\) is chosen small enough to ensure that \(g(x) \neq 0\) on \(U_\delta(\xi)\), we have
\(\frac{\frac{f(x)}{g(x)} - \frac{f(\xi)}{g(\xi)}}{x - \xi} = \frac{1}{g(x) g(\xi)} \frac{f(x) g(\xi) - f(\xi) g(x)}{x - \xi}\)
\(= \frac{1}{g(x) g(\xi)} \left\{ \frac{f(x) - f(\xi)}{x - \xi} g(\xi) - \frac{g(x) - g(\xi)}{x - \xi} f(\xi) \right\}\)
\(\xrightarrow{x \to \xi} \frac{1}{g(\xi)^2} \{ f'(\xi) g(\xi) - f(\xi) g'(\xi) \},\)
which proves the quotient rule.
ii) We define \(r: J \to \mathbb{R}\) by
\(r(y) := \begin{cases} \frac{g(y) - g(\eta)}{y - \eta} - g'(\eta), & y \in J \setminus \{\eta\} \\ 0, & y = \eta. \end{cases}\)
\(\implies\) r is continuous at \(y = \eta\) with \(r(\eta) = 0\). Using this function r, we can write for \(y \in J\)
\(g(y) - g(\eta) = g'(\eta) (y - \eta) + r(y) (y - \eta)\)
\(\implies (y = f(x))\) For \(x \in I\) we have
\(g(f(x)) - g(f(\xi)) = (g'(f(\xi)) + r(f(x))) (f(x) - f(\xi))\)
\(\implies\) For \(x \in I \setminus \{\xi\}\) we have
\(\frac{g(f(x)) - g(f(\xi))}{x - \xi} = (g'(f(\xi)) + r(f(x))) \frac{f(x) - f(\xi)}{x - \xi}\)
\(\xrightarrow{x \to \xi} g'(f(\xi)) f'(\xi). \quad \square\)
Examples. i) Let \(p: \mathbb{R} \to \mathbb{R}, p(x) = \sum_{k=0}^n a_k x^k\), be a polynomial, then, by the sum rule, p is differentiable on \(\mathbb{R}\) with derivative
\(p'(x) = \frac{d}{dx} \sum_{k=0}^n a_k x^k = \sum_{k=0}^n a_k \frac{d}{dx} x^k = \sum_{k=1}^n a_k k x^{k-1}.\)
ii) Let \(r: I \to \mathbb{R}, r(x) = \frac{p(x)}{q(x)}\), be a rational function, where \(q(x) \neq 0\) on I. Then, by the quotient rule, r is differentiable on I with
\(r'(x) = \frac{p'(x) q(x) - p(x) q'(x)}{q(x)^2}.\)
In particular, for \(k \in \mathbb{N}\) and \(x \neq 0\), we have
\(\frac{d}{dx} \frac{1}{x^k} = -k x^{-k-1}.\)
iii) Let \(a > 0\) and \(f: \mathbb{R} \to (0, \infty), f(x) = a^x = e^{x \log a}\).
Then, by the chain rule, f is differentiable on \(\mathbb{R}\) with
\(f'(x) = e^{x \log a} \log a = a^x \log a.\)
iv) Using the series representation and the Cauchy product, we can show that the following addition formulas hold for sine and cosine:
\(\sin(x+y) = \sin x \cos y + \sin y \cos x\)
\(\cos(x+y) = \cos x \cos y - \sin x \sin y, \quad x, y \in \mathbb{R}.\)
We can use these formulas to differentiate sine and cosine, e.g., we have for any \(\xi \in \mathbb{R}\)
\(\lim_{x \to \xi} \frac{\sin x - \sin \xi}{x - \xi} = \lim_{x \to 0} \frac{\sin(\xi + x) - \sin \xi}{x}\)
\(= \lim_{x \to 0} \frac{\sin \xi \cos x + \sin x \cos \xi - \sin \xi}{x}\)
\(= \lim_{x \to 0} \cos \xi \frac{\sin x}{x} + \lim_{x \to 0} \sin \xi \frac{\cos x - 1}{x} = \cos \xi.\)
\(\implies\) The sine is differentiable on \(\mathbb{R}\) with \(\frac{d}{dx} \sin x = \cos x\). Similarly, we can show that cosine is differentiable on \(\mathbb{R}\) with \(\frac{d}{dx} \cos x = -\sin x\).
Moreover, using the quotient rule we see that
\(\tan x := \frac{\sin x}{\cos x}\), for \(x \neq \frac{\pi}{2} + k\pi, k \in \mathbb{Z}\),
is differentiable with
\(\frac{d}{dx} \tan x = \frac{\cos^2 x + \sin^2 x}{\cos^2 x} = \frac{1}{\cos^2 x}.\)
Next, we deal with the differentiation of inverse functions.
Theorem 36. Let \(f: I \to \mathbb{R}\) be strictly monotonic and continuous on I, and suppose that f is differentiable at \(\xi \in I\) with \(f'(\xi) \neq 0\). Then the inverse \(f^{-1}: f(I) \to I\) is differentiable at \(\eta = f(\xi)\) with
\((f^{-1})'(\eta) = \frac{1}{f'(f^{-1}(\eta))} = \frac{1}{f'(\xi)}.\)
Proof. The existence of \(f^{-1}\) and its continuity is clear from Thm 33. For any \(y \in f(I) \setminus \{\eta\}\) we define \(x := f^{-1}(y) \neq \xi\) (f is strictly monotonic).
\(\implies \frac{f^{-1}(y) - f^{-1}(\eta)}{y - \eta} = \frac{x - \xi}{f(x) - f(\xi)} = \left[ \frac{f(x) - f(\xi)}{x - \xi} \right]^{-1} \xrightarrow{x \to \xi} \frac{1}{f'(\xi)},\)
as \(y \to \eta\), where we use that \(x = f^{-1}(y) \to f^{-1}(\eta) = \xi\), as \(y \to \eta\). \(\square\)
Examples. i) We know that \(f = \exp\) satisfies the assumptions of Thm 36, hence \(\log: (0, \infty) \to \mathbb{R}\) is differentiable on \((0, \infty)\) with
\(\frac{d}{dx} \log x = \frac{1}{\exp(\log x)} = \frac{1}{x}, \quad x > 0.\)
ii) Let \(\alpha \in \mathbb{R}\) and \(f: (0, \infty) \to \mathbb{R}, f(x) = x^\alpha = e^{\alpha \log x}\). Then, by the chain rule, f is differentiable on \((0, \infty)\) with
\(\frac{d}{dx} x^\alpha = \frac{d}{dx} e^{\alpha \log x} = e^{\alpha \log x} \frac{\alpha}{x} = \alpha x^{\alpha-1}.\)
Next, we deal with local extrema and mean value theorems.
Definition. The function \(f: D \to \mathbb{R} (D \subset \mathbb{R})\) has a local maximum at \(\xi \in D\) if there exists some \(\epsilon > 0\) such that \(f(x) \le f(\xi)\) for all \(x \in U_\epsilon(\xi) \cap D\). The function f has a local minimum at \(\xi \in D\) if there exists some \(\epsilon > 0\) such that \(f(x) \ge f(\xi)\) for all \(x \in U_\epsilon(\xi) \cap D\). Local minima and local maxima are also called local extrema.
Theorem 37. Let \(f: (a, b) \to \mathbb{R}\) have a local extremum at \(\xi \in (a, b)\) and suppose that f is differentiable at \(\xi\). Then \(f'(\xi) = 0\).
Proof. As f is differentiable at \(\xi\), we know \(f'(\xi) = f'_+(\xi) = f'_-(\xi)\).