Week 2: Preparation

Week 2: Preparation#

Reading Material#

We recommend that you read the textbook. Watching YouTube videos on the week’s topics can be useful, but it should not replace proper preparation for the week’s program and is not recommended as a standalone approach.

Read and study the following:

Key Concepts#

After reading, you should be able to explain the following key concepts:

  • Vector functions of multiple variables

  • Directional derivatives

  • Differentiability

  • The Jacobian matrix (or just the Jacobian)

  • The gradient vector

  • The chain rule

  • The Hessian matrix (or just the Hessian)

This week, we will explore these key concepts in great detail. We expect you to have familiarized yourself with these topics before lectures.

Preparatory Exercises#

I: A Composite Function#

Let \(g : \mathbb{R}^2 \to \mathbb{R}\) be given by

\[\begin{equation*} g(x,y) = e^{2x+y}, \end{equation*}\]

and let \(\pmb{f} : \mathbb{R} \to \mathbb{R}^2\) be given by

\[\begin{equation*} \pmb{f}(t) = \bigl(t^2, \sin(t)\bigr). \end{equation*}\]

Question a#

Find the composite function \(g \circ \pmb{f}\).

Hint

First, find the expression for the composite function \((g \circ \pmb{f})(t)\)

Hint

Let \((x,y) := \pmb{f}(t) = (t^2, \sin(t))\) and substitute the expression for \(x\) and \(y\) into \(g(x,y)\).

Hint

Then, determine the domain and codomain of the composite function.

Answer

\(g \circ \pmb{f}: \mathbb{R} \to \mathbb{R}\) is given by \((g \circ \pmb{f})(t) = g(t^2, \sin(t)) = e^{2t^2 + \sin(t)}\) for \(t \in \mathbb{R}\).

Question b#

Calculate the derivative

\[\begin{equation*} \frac{d}{dt}(g \circ \pmb{f})(t). \end{equation*}\]

Hint

The easiest approach is to differentiate the expression \(e^{2t^2 + \sin(t)}\) with respect to \(t\), but you can also use the chain rule.

Answer

\[\begin{equation*} \frac{d}{dt}(g \circ \pmb{f})(t) = e^{2t^2 + \sin(t)} \; \bigl(4t + \cos(t)\bigr). \end{equation*}\]

Question c#

Is the composite function \(\pmb{f} \circ g\) well-defined?

Answer

Yes, in this case, it is. It is a function of the form \(\pmb{f} \circ g: \mathbb{R}^2 \to \mathbb{R}^2\). Try to find the functional expression yourself.

II: Partial Derivatives and Directional Derivatives#

Let \(f : \mathbb{R}^2 \to \mathbb{R}\) be given by

\[\begin{equation*} f(x,y) = x^2 y + y^3. \end{equation*}\]

Question a#

Calculate the partial derivatives \(\frac{\partial f}{\partial x}\) and \(\frac{\partial f}{\partial y}\) at the point \((1,2)\).

Hint

Differentiate \(f\) with respect to \(x\) and \(y\) separately, where you treat \(y\) and \(x\) as a constant, respectively.

Answer

\[\begin{equation*} \frac{\partial f}{\partial x}(x,y) = 2xy, \quad \frac{\partial f}{\partial y}(x,y) = x^2 + 3y^2. \end{equation*}\]

At the point \((1,2)\):

\[\begin{equation*} \frac{\partial f}{\partial x}(1,2) = 2 \cdot 1 \cdot 2 = 4, \quad \frac{\partial f}{\partial y}(1,2) = 1^2 + 3 \cdot 4 = 1 + 12 = 13. \end{equation*}\]

Question b#

Calculate the gradient vector \(\nabla f(x,y)\), and state \(\nabla f(1,2)\).

Hint

The gradient of a function \(f\) is a vector consisting of its partial derivatives.

Hint

For a function of two variables the gradient is \(\nabla f(x,y) = \left( \frac{\partial f}{\partial x}(x,y), \frac{\partial f}{\partial y}(x,y) \right)\).

Answer

We have from the previous question that \(\nabla f(x,y) = (2xy, x^2 + 3y^2)\) and \(\nabla f(1,2) = (4, 13)\).

Question c#

Calculate the directional derivative in the direction given by \(\pmb{e}_1=[1,0]^T\) at the point \((1,2)\). Also, calculate the directional derivative in the direction given by \(\pmb{e}_2=[0,1]^T\) at the point \((1,2)\).

Hint

The directional derivative in a “standard direction” corresponds to the partial derivatives.

Answer

At the point \((1,2)\), the directional derivative in the direction given by \(\pmb{e}_1\) is \(4\), and in the direction given by \(\pmb{e}_2\) it is \(13\).

Question d#

What do the answers in questions b and c have to do with each other?

Answer

The partial derivatives are the directional derivatives in the “standard directions” \(\pmb{e}_1\) and \(\pmb{e}_2\).

III: A Vector Function in Three Variables#

Let \(\pmb{f} : \mathbb{R}^3 \to \mathbb{R}^2\) be given by

\[\begin{equation*} \pmb{f}(x,y,z) = \bigl(xy + z, x - yz\bigr). \end{equation*}\]

Question a#

Calculate \(\pmb{f}(1,2,3)\).

Hint

Substitute \((x,y,z) = (1,2,3)\) into the expression for \(\pmb{f}\).

Answer

\[\begin{equation*} \pmb{f}(1,2,3) = \bigl(1 \cdot 2 + 3, 1 - 2 \cdot 3\bigr) = (5, -5). \end{equation*}\]

Question b#

Find the Jacobian matrix \(\pmb{J}_{\pmb{f}}(x,y,z)\). Calculate \(\pmb{J}_{\pmb{f}}(1,2,3)\).

Hint

The Jacobian matrix is a matrix that contains all the partial derivatives of all the coordinate functions. The vector function \(\pmb{f}=(f_1,f_2)\) has two coordinate functions, so what is the size of the Jacobian matrix?

Hint

The Jacobian matrix has the size: \(\pmb{J}_{\pmb{f}}(x,y,z) \in \mathbb{R}^{2 \times 3}\); remember that \(n=3\) is the number of input variables, and \(k=2\) is the number of “output” coordinate functions.

Answer

The Jacobian matrix \(\pmb{J}_{\pmb{f}}(x,y,z)\) is:

\[\begin{equation*} \pmb{J}_{\pmb{f}}(x,y,z) = \begin{bmatrix} \frac{\partial (xy + z)}{\partial x} & \frac{\partial (xy + z)}{\partial y} & \frac{\partial (xy + z)}{\partial z} \\ \frac{\partial (x - yz)}{\partial x} & \frac{\partial (x - yz)}{\partial y} & \frac{\partial (x - yz)}{\partial z} \end{bmatrix} = \begin{bmatrix} y & x & 1 \\ 1 & -z & -y \end{bmatrix}. \end{equation*}\]

At the point \((x,y,z)=(1,2,3)\):

\[\begin{equation*} \pmb{J}_{\pmb{f}}(1,2,3) = \begin{bmatrix} 2 & 1 & 1 \\ 1 & -3 & -2 \end{bmatrix}. \end{equation*}\]