W.l.o.g. let us assume that f has a local maximum at \(\xi\). Then we have
\(f'_+(\xi) = \lim_{x \to \xi+} \frac{f(x) - f(\xi)}{x - \xi} \le 0\), \(f'_-(\xi) = \lim_{x \to \xi-} \frac{f(x) - f(\xi)}{x - \xi} \ge 0\)
\(\implies f'(\xi) = 0\).
Remark. The condition \(f'(\xi) = 0\) is only necessary but not sufficient for a local extremum. For example, consider \(f: \mathbb{R} \to \mathbb{R}, f(x) = x^3\), then we have \(f'(0) = 0\) but f does not have a local extremum at 0.
Theorem 38. (Rolle's Theorem)
Let \(f: [a, b] \to \mathbb{R}\) be continuous on \([a, b]\) and differentiable on \((a, b)\). Further, suppose that \(f(a) = f(b)\), then there exists some \(\xi \in (a, b)\) such that \(f'(\xi) = 0\).
Proof. By Thm 34 we can find \(\xi_1, \xi_2 \in [a, b]\) such that \(f(\xi_1) = \min_{x \in [a, b]} f(x), f(\xi_2) = \max_{x \in [a, b]} f(x)\).
If \(\xi_1 \in (a, b)\) or \(\xi_2 \in (a, b)\), then the statement follows from Thm 37. Let us assume now that \(\xi_1, \xi_2 \in \{a, b\}\).
\(\implies f(\xi_1) = f(\xi_2) = f(a) = f(b)\)
\(\implies f\) is constant on \([a, b]\)
\(\implies f'(\xi) = 0\) for every \(\xi \in (a, b)\).
Theorem 39. (Generalised Mean Value Theorem)
Let \(f, g: [a, b] \to \mathbb{R}\) be continuous on \([a, b]\) and differentiable on \((a, b)\). Then there exists some \(\xi \in (a, b)\) such that
\((f(b) - f(a)) g'(\xi) = (g(b) - g(a)) f'(\xi)\).
Proof. The proof follows immediately from the application of Thm 38 to the function
\(h: [a, b] \to \mathbb{R}, h(x) = (f(b) - f(a)) g(x) - (g(b) - g(a)) f(x)\).
Remarks. i) If \(g'(x) \neq 0\) on \((a, b)\), then \(g(b) \neq g(a)\) (by Thm 38), so we can write the statement of Thm 39 as
\(\frac{f(b) - f(a)}{g(b) - g(a)} = \frac{f'(\xi)}{g'(\xi)}\).
ii) In the special case \(g(x) = x\), we obtain
\(f(b) - f(a) = f'(\xi) (b-a)\) for some \(\xi \in (a, b)\).
This statement often is called Mean Value Theorem.
iii) As a consequence, we can observe for any function \(f: [a, b] \to \mathbb{R}\), which is continuous on \([a, b]\) and differentiable on \((a, b)\), that
\(f\) is constant on \([a, b] \iff f'(x) = 0\) on \((a, b)\).
Theorem 40. Let \(f: (a, b) \to \mathbb{R}\) be differentiable. Then we have
i) \(f'(x) > 0\) on \((a, b) \implies f\) is strictly increasing on \((a, b)\)
ii) \(f'(x) < 0\) on \((a, b) \implies f\) is strictly decreasing on \((a, b)\)
iii) \(f'(x) \ge 0\) on \((a, b) \iff f\) is increasing on \((a, b)\)
iv) \(f'(x) \le 0\) on \((a, b) \iff f\) is decreasing on \((a, b)\).
Proof. "\(\implies\)" in all four parts: For \(x_1, x_2 \in (a, b)\) with \(x_1 < x_2\), by Thm 39, we find a \(\xi \in (x_1, x_2)\) such that \(f(x_2) - f(x_1)=f'(\xi)(x_2 - x_1)\).
"\(\impliedby\)" in parts iii, iv: W.l.o.g. we assume that \(f\) is increasing. For any \(\xi \in (a, b)\) we have
\(f'(\xi) = \lim_{x \to \xi} \underbrace{\frac{f(x) - f(\xi)}{x - \xi}}_{\ge 0} \ge 0\).
Example. The tangent \(\tan: (-\frac{\pi}{2}, \frac{\pi}{2}) \to \mathbb{R}\), \(\tan x = \frac{\sin x}{\cos x}\), is strictly increasing on \((-\frac{\pi}{2}, \frac{\pi}{2})\):
\(\frac{d}{dx} \tan x = \frac{1}{\cos^2 x} > 0\).
Theorem 41. (L'Hôpital's Rule)
Let \(f, g: (a, b) \to \mathbb{R}\) be differentiable functions with \(g'(x) \neq 0\) on \((a, b)\), where \(-\infty \le a < b \le \infty\), and suppose that
i) \(\lim_{x \to a+} f(x) = \lim_{x \to a+} g(x) = 0 \quad\) or
ii) \(\lim_{x \to a+} f(x) = \lim_{x \to a+} g(x) = \infty\).
If \(\lim_{x \to a+} \frac{f'(x)}{g'(x)}\) exists, then \(\lim_{x \to a+} \frac{f(x)}{g(x)}\) exists and
\(\lim_{x \to a+} \frac{f(x)}{g(x)} = \lim_{x \to a+} \frac{f'(x)}{g'(x)}\).
Proof. We only focus on the case i) and \(a > -\infty\) (the other cases are similar). We define \(f(a) := g(a) := 0\) and choose some \(b' \in (a, b)\).
\(\implies f, g\) are continuous on \([a, b']\) and differentiable on \((a, b')\). By Thm 39, for any \(x \in (a, b')\), we find some \(\xi \in (a, x)\) such that \(\frac{f(x)}{g(x)} = \frac{f(x) - f(a)}{g(x) - g(a)} = \frac{f'(\xi)}{g'(\xi)}\).
\(\implies\) As \(x \to a+\), we obtain
\(\lim_{x \to a+} \frac{f(x)}{g(x)} = \lim_{\xi \to a+} \frac{f'(\xi)}{g'(\xi)}\).
Examples. i) We have for \(a, b > 0\)
\(\lim_{x \to 0+} \frac{a^x - b^x}{x} = \lim_{x \to 0+} \frac{a^x \log a - b^x \log b}{1}\)
\(= \log a \lim_{x \to 0+} e^{(\log a)x} - \log b \lim_{x \to 0+} e^{(\log b)x}\)
\(= \log a - \log b = \log \frac{a}{b}\).
ii) \(\lim_{x \to 0+} x \log x = -\lim_{x \to 0+} \frac{-\log x}{1/x} = -\lim_{x \to 0+} \frac{-1/x}{-1/x^2}\)
\(= -\lim_{x \to 0+} x = 0\).
Next, we deal with higher-order derivatives.
Definition. Suppose \(f: I \to \mathbb{R}\) is differentiable on I. If \(f': I \to \mathbb{R}\) is differentiable at \(\xi \in I\), we say that f is twice differentiable at \(\xi\) and call
\(f''(\xi) := (f')'(\xi)\) the second derivative of f at \(\xi\). If \(f''(\xi)\) exists at every \(\xi \in I\), we say that f is twice differentiable on I.
More generally, for some \(m \in \mathbb{N}\), suppose that
f has a derivative of order \(m-1\) on I, i.e., \(f^{(m-1)}: I \to \mathbb{R}\) exists (we define \(f^{(0)} := f\)).
If \(f^{(m-1)}\) is differentiable at \(\xi \in I\), we say that f is m-times differentiable at \(\xi\) and call
\(f^{(m)}(\xi) := (f^{(m-1)})'(\xi)\) the derivative of order m of f at \(\xi\). If \(f^{(m)}(\xi)\) exists at every \(\xi \in I\), we say that f is m-times differentiable on I and call \(f^{(m)}: I \to \mathbb{R}\) its derivative of order m on I.
Moreover, we define
\(C_n(I) := \{f: I \to \mathbb{R} : f \text{ is n-times differentiable on I and } f^{(n)} \text{ is continuous on I} \}\),
"set of n-times continuously differentiable functions on I",
\(C_\infty(I) := \bigcap_{m=0}^\infty C_m(I)\) "set of smooth functions on I".
Remarks. i) We have \(f^{(0)} = f, f^{(1)} = f', f^{(2)} = f''\).
ii) \(C_0(I)\) is the set of continuous functions on I.
Definition. Let \(f: I \to \mathbb{R}\) m-times differentiable on I, for some \(m \in \mathbb{N}_0\), and let \(x_0 \in I\). Then
\(T_m(x) := \sum_{k=0}^m \frac{1}{k!} f^{(k)}(x_0) (x-x_0)^k\)
is the m-th-order Taylor polynomial of f centered at \(x_0\).
Theorem 42. (Taylor's Theorem).
Let \(f: I \to \mathbb{R}\) be \((m+1)\)-times differentiable on I for some \(m \in \mathbb{N}_0\), and let \(x_0 \in I\) and \(p \in \mathbb{N}\). Then for \(x \in I\) we have \(f(x) = T_m(x) + R_m(x)\), where the remainder is given by
\(R_m(x) = \frac{1}{m! p} f^{(m+1)}(x_0 + \vartheta(x-x_0)) (1-\vartheta)^{m+1-p} (x-x_0)^{m+1}\)
for some \(\vartheta = \vartheta(x, x_0, m, p) \in (0, 1)\).
Proof. Let \(x, x_0 \in I\) with \(x \neq x_0\), \(m \in \mathbb{N}_0\) and \(p \in \mathbb{N}\). We define for \(t \in I\)
\(G(t) := f(x) - \sum_{k=0}^m \frac{1}{k!} f^{(k)}(t) (x-t)^k\), \(g(t) := (x-t)^p\).
\(\implies G(x) = g(x) = 0, G(x_0) = R_m(x)\),
\(G'(t) = -\frac{1}{m!} f^{(m+1)}(t) (x-t)^m\), \(g'(t) = -p (x-t)^{p-1}\).
\(\implies \frac{R_m(x)}{(x-x_0)^p} = \frac{R_m(x)}{g(x_0)} = \frac{-G(x_0)}{-g(x_0)} = \frac{G(x) - G(x_0)}{g(x) - g(x_0)}\)
\(\stackrel{\text{Thm 39}}{=} \frac{G'(\xi)}{g'(\xi)}\) for some \(\xi\) between \(x_0\) and \(x\),
which we write as \(\xi = x_0 + \vartheta(x-x_0)\) for some \(\vartheta \in (0, 1)\).
\(\implies \frac{R_m(x)}{(x-x_0)^p} = \frac{-f^{(m+1)}(x_0 + \vartheta(x-x_0)) (x - x_0 - \vartheta(x-x_0))^m}{-m! p (x - x_0 - \vartheta(x-x_0))^{p-1}}\)
\(= \frac{1}{m! p} f^{(m+1)}(x_0 + \vartheta(x-x_0)) (1-\vartheta)^{m+1-p} (x-x_0)^{m+1-p}\).
As an application of Taylor's Theorem we obtain the following sufficient condition for local extrema.
Theorem 43. Let \(f \in C_2((a, b))\) and suppose that \(\xi \in (a, b)\) with \(f'(\xi) = 0\) and \(f''(\xi) \neq 0\). Then \(f\) has a local maximum at \(\xi\) if \(f''(\xi) < 0\), and \(f\) has a local minimum at \(\xi\) if \(f''(\xi)> 0\).
Proof. Applying Thm 42 with \(m=1, p=2, x_0 = \xi\) to f, gives us
\(f(x) = f(\xi) + f'(\xi)(x-\xi) + \frac{1}{2} f''(\xi + \vartheta(x-\xi))(x-\xi)^2\)
\(= f(\xi) + \frac{1}{2} f''(\xi + \vartheta(x-\xi))(x-\xi)^2\) for some \(\vartheta \in (0, 1)\).
As \(f''\) is continuous and \(\neq 0\) at \(\xi\), we find
a \(\delta > 0\) such that \(f''(\xi + \vartheta(x-\xi)) < 0\), \(x \in U_\delta(\xi)\), if \(f''(\xi) < 0\), and \(f''(\xi + \vartheta(x-\xi))> 0\), \(x \in U_\delta(\xi)\), if \(f''(\xi) > 0\).
Another application of Taylor's Theorem leads to power series representations of \(C_\infty\)-functions.
Definition. Let \(f \in C_\infty((a, b))\), where \(-\infty \le a < b \le \infty\), and let \(x_0 \in (a, b)\). The power series
\(\sum_{k=0}^\infty \frac{1}{k!} f^{(k)}(x_0) (x-x_0)^k\)
is called the Taylor series of f centered at \(x_0\). Moreover, the function f is real-analytic at \(x_0\) if there exists a \(\delta > 0\) such that
\(f(x) = \sum_{k=0}^\infty \frac{1}{k!} f^{(k)}(x_0) (x-x_0)^k\), \(x \in U_\delta(x_0)\).
Remark. A function \(f \in C_\infty((a, b))\) is real-analytic at \(x_0 \in (a, b) \iff f(x) = \lim_{m \to \infty} T_m(x)\) on \(U_\delta(x_0)\), where \(T_m(x)\) is the Taylor polynomial of m-th order of f centered at \(x_0\).
Examples. i) Not every smooth function is real-analytic: Consider \(f: \mathbb{R} \to \mathbb{R}\)
\(f(x) = \begin{cases} e^{-1/x^2}, & x \neq 0 \\ 0, & x = 0, \end{cases}\) and \(x_0 = 0\).
It is not difficult to show that \(f \in C_\infty(\mathbb{R})\) with \(f^{(k)}(0) = 0\) for all \(k \in \mathbb{N}_0\), so that
\(\sum_{k=0}^\infty \frac{1}{k!} f^{(k)}(0) x^k = 0\) for all \(x \in \mathbb{R}\), while
\(f(x) \neq 0\) for \(x \neq 0\).
ii) Let \(f: (-1, \infty) \to \mathbb{R}, f(x) = \log(1+x)\), and \(x_0 = 0\).
Then \(f'(x) = \frac{1}{1+x}, f''(x) = \frac{-1}{(1+x)^2}, f^{(3)}(x) = \frac{(-1)^2 \cdot 2}{(1+x)^3}\)
\(\implies f^{(k)}(x) = \frac{(-1)^{k-1}(k-1)!}{(1+x)^k}, k > 0, \implies f \in C_\infty((-1, \infty))\)
\(\implies f^{(k)}(0) = (-1)^{k-1}(k-1)!, k > 0\).
We show that f is real-analytic. To this end, we write \(f(x) = T_m(x) + R_m(x)\) with
\(T_m(x) = \sum_{k=0}^m \frac{1}{k!} f^{(k)}(0) x^k = \sum_{k=1}^m \frac{(-1)^{k-1}}{k} x^k\).
\(0 \le x \le 1\): From Thm 42 with \(p=m+1\) we obtain
\(|R_m(x)| = | \frac{1}{(m+1)!} f^{(m+1)}(\vartheta x) x^{m+1} | = \frac{1}{m+1} \left( \frac{x}{1+\vartheta x} \right)^{m+1} \to 0, m \to \infty\).
\(-1 < x < 0\): From Thm 42 with \(p=1\) we obtain
\(|R_m(x)| = | \frac{1}{m!} f^{(m+1)}(\vartheta x) (1-\vartheta)^m x^{m+1} |\)
\(= \left( \frac{1-\vartheta}{1+\vartheta x} \right)^m \frac{|x|^{m+1}}{1+\vartheta x} \to 0, m \to \infty\).
\(\implies\) For \(x \in (-1, 1]\) we have
\(\log(1+x) = \sum_{k=1}^\infty \frac{(-1)^{k-1}}{k} x^k\) "Mercator series"
In particular: \(\log 2 = \sum_{k=1}^\infty \frac{(-1)^{k-1}}{k}\).
iii) Let \(f: (-1, \infty) \to \mathbb{R}, f(x) = (1+x)^\alpha\), \(\alpha \in \mathbb{R}\) fixed.
Then, by a similar reasoning as in ii), we can show that for \(x \in (-1, 1)\)
\((1+x)^\alpha = \sum_{k=0}^\infty \binom{\alpha}{k} x^k\), "Binomial series"
where \(\binom{\alpha}{k} := \frac{1}{k!} f^{(k)}(0) = \frac{1}{k!} \alpha(\alpha-1) \dots (\alpha-k+1)\).