{
"cells": [
{
"cell_type": "markdown",
"id": "d7f3a4ab",
"metadata": {},
"source": [
"# Week 4: The Spectral Theorem"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81d5cbf7",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"from sympy import*\n",
"from dtumathtools import*\n",
"init_printing()"
]
},
{
"cell_type": "markdown",
"id": "ac4fa0f9",
"metadata": {},
"source": [
"## Exercises -- Long Day"
]
},
{
"cell_type": "markdown",
"id": "42dab11a",
"metadata": {},
"source": [
"### 1: Types of Matrices\n",
"\n",
"Consider the matrices:\n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} 1 & 0 & 0 \\\\ 0 & 2 & 0 \\\\ 0 & 0 & 3 \\end{bmatrix}, \\quad\n",
" B=\\begin{bmatrix} 1 & 2 & 3 \\\\ 3 & 1 & 2 \\\\ 2 & 3 & 1 \\end{bmatrix}, \\quad\n",
" C=\\begin{bmatrix} 1 & 2+i & 3i \\\\ 2-i & 1 & 2 \\\\ -3i & 2 & 1 \\end{bmatrix}, \\quad\n",
" D=\\begin{bmatrix} i & 2 & 3 \\\\ 2 & i & 2 \\\\ 3 & 2 & i \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"Determine for *each* matrix whether it is symmetric, Hermitian, and/or normal. You may use SymPy to check if the matrices are normal. For convenience, the matrices are provided here:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6dcfa272",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A = Matrix.diag(1, 2, 3)\n",
"B = Matrix([[1, 2, 3], [3, 1, 2], [2, 3, 1]])\n",
"C = Matrix([[1, 2 + I, 3*I], [2 - I, 1, 2], [-3*I, 2, 1]]) \n",
"D = Matrix([[I, 2, 3], [2, I, 2], [3, 2, I]])"
]
},
{
"cell_type": "markdown",
"id": "39d85591",
"metadata": {},
"source": [
"### 2: Hermitian 2-by-2 Matrix. By Hand"
]
},
{
"cell_type": "markdown",
"id": "d3a1c57f",
"metadata": {},
"source": [
"We consider the Hermitian matrix $A$ given by: \n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} 0 & i \\\\ -i & 0 \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"This exercise involves computing a **spectral decomposition** of $A$, which we know exists due the Spectral Theorem (the complex case). We find this decomposition of $A$ in three steps:"
]
},
{
"cell_type": "markdown",
"id": "8e5e2270",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "d829778e",
"metadata": {},
"source": [
"Find all eigenvalues and associated eigenvectors of $A$. Check your anwer using SymPy's `A.eigenvects()`."
]
},
{
"cell_type": "markdown",
"id": "2f04a7d6",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "84d72409",
"metadata": {},
"source": [
"Provide an orthonormal basis consisting of eigenvectors of $A$."
]
},
{
"cell_type": "markdown",
"id": "73b49d6c",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "1f10e40c",
"metadata": {},
"source": [
"This result holds for general $n \\times n$ matrices. Show that $A = U \\Lambda U^*$ if and only if $\\Lambda = U^* A U$, when $U$ is unitary."
]
},
{
"cell_type": "markdown",
"id": "b30da04a",
"metadata": {},
"source": [
"#### Question d"
]
},
{
"cell_type": "markdown",
"id": "d25e1407",
"metadata": {},
"source": [
"Write down a unitary matrix $U$ and a diagonal matrix $\\Lambda$ such that $A = U \\Lambda U^*$. This formula is called a spectral decomposition of $A$. Check your result using the SymPy command:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58274c50",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A = Matrix([[0, I], [-I, 0]])\n",
"A.diagonalize(normalize = True)"
]
},
{
"cell_type": "markdown",
"id": "75257da1",
"metadata": {},
"source": [
"### 3: Orthogonality of Eigenvectors of Symmetric Matrices\n",
"\n",
"Let $C$ be a $2 \\times 2$ real, symmetric matrix with two different eigenvalues. Show that the eigenvectors $\\pmb{v}_1$ and $\\pmb{v}_2$ corresponding to the two different eigenvalues are orthogonal, i.e., that\n",
"\n",
"\\begin{equation*}\n",
"\\langle \\pmb{v}_1, \\pmb{v}_2 \\rangle = 0.\n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"id": "5f0c2d65",
"metadata": {},
"source": [
"### 4: Symmetric 3-by-3 matrix"
]
},
{
"cell_type": "markdown",
"id": "b82f72dd",
"metadata": {},
"source": [
"We are given the *real and symmetric* matrix\n",
"\n",
"\\begin{equation*}\n",
" A=\\begin{bmatrix} -2 & 1 & 1 \\\\ 1 & -2 & -1 \\\\ 1 & -1 & -2 \\end{bmatrix}.\n",
"\\end{equation*}\n",
"\n",
"Find a **spectral decomposition** of $A = Q \\Lambda Q^T$. In other words, provide a *real orthogonal* matrix $Q$ and a *diagonal matrix* $\\Lambda$ such that \n",
"\n",
"\\begin{equation*}\n",
" A = Q \\Lambda Q^T\n",
"\\end{equation*} \n",
"\n",
"or, equivalently, \n",
"\n",
"\\begin{equation*}\n",
" Q^T \\, A\\, Q=\\Lambda\n",
"\\end{equation*} \n",
"\n",
"is fulfilled. As in the previous exercise we know from the Spectral Theorem (the real case) that these matrices exist."
]
},
{
"cell_type": "markdown",
"id": "db7054f5",
"metadata": {},
"source": [
"### 5: Spectral decomposition with SymPy"
]
},
{
"cell_type": "markdown",
"id": "a83151b3",
"metadata": {},
"source": [
"We consider the following matrices given in SymPy:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "707f9b02",
"metadata": {},
"outputs": [],
"source": [
"A = Matrix([[1, -1, 0, 0], [0, 1, -1, 0], [0, 0, 1, -1], [-1, 0, 0, 1]])\n",
"B = Matrix([[1, 2, 3, 4], [4, 1, 2, 3], [3, 4, 1, 2], [2, 3, 4, 1]])\n",
"A, B"
]
},
{
"cell_type": "markdown",
"id": "b43ca7d2",
"metadata": {},
"source": [
"We are informed that both matrices are real, *normal* matrices. This can be checked by:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d6b620f0",
"metadata": {},
"outputs": [],
"source": [
"A.conjugate() == A, B.conjugate() == B, A*A.T == A.T*A, B*B.T == B.T*B"
]
},
{
"cell_type": "markdown",
"id": "46b9aa34",
"metadata": {},
"source": [
"We are furthermore informed that the eigenvalues are, respectively:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "730ae136",
"metadata": {},
"outputs": [],
"source": [
"A.eigenvals(multiple=True), B.eigenvals(multiple=True)"
]
},
{
"cell_type": "markdown",
"id": "9353e2a4",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "589781da",
"metadata": {},
"source": [
"Will the following SymPy commands give us the matrices involved in the spectral decompositions of $A$ and $B$? The call `A.diagonalize(normalize=True)` returns $(V, \\Lambda)$, where $A = V \\Lambda V^{-1}$, with *normalized* eigenvectors in $V$ and the eigenvalues of $A$ being the diagonal elements in the diagonal matrix $\\Lambda$ (according to the eigenvalue problem studied in Mathematics 1a)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7c4a8d4f",
"metadata": {
"tags": [
"remove-output"
]
},
"outputs": [],
"source": [
"A.diagonalize(normalize = True), B.diagonalize(normalize = True)"
]
},
{
"cell_type": "markdown",
"id": "7f415aa0",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "84e8a239",
"metadata": {},
"source": [
"Does a unitary matrix that diagonalizes *both* $A$ *and* $B$ exist? Meaning, does **one** unitary matrix exist such that $A = U \\Lambda_1 U^*$ and $B = U \\Lambda_2 U^*$, where $\\Lambda_1$ is a diagonal matrix consisting of the eigenvalues of $A$ and where $\\Lambda_2$ is a diagonal matrix consisting of the eigenvalues of $B$?"
]
},
{
"cell_type": "markdown",
"id": "d415fb3a",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "9afa6f61",
"metadata": {},
"source": [
"You have seen the matrix $U$, or maybe $U^*$, before (possibly with its columns in a different order). What kind of matrix is this?"
]
},
{
"cell_type": "markdown",
"id": "47e4b4b9",
"metadata": {},
"source": [
"### 6: Diagonalization and Reduction of Quadratic Forms"
]
},
{
"cell_type": "markdown",
"id": "1319e64d",
"metadata": {},
"source": [
"We consider the function $q : \\mathbb{R}^3 \\to \\mathbb{R}$ given by\n",
"\\begin{equation*}\n",
" q(x,y,z)=-2x^2-2y^2-2z^2+2xy+2xz-2yz+2x+y+z+5.\n",
"\\end{equation*}\n",
"\n",
"Note that $q$ can be split into two parts: a part containing purely the quadratic terms: $k(x,y,z)=-2x^2-2y^2-2z^2+2xy+2xz-2yz$, and a part with the remaining terms, which is a linear polynomial: $2x+y+z+5$."
]
},
{
"cell_type": "markdown",
"id": "16b5f98c",
"metadata": {},
"source": [
"We are given the symmetric matrix\n",
"\n",
"\\begin{equation*}\n",
"A=\\begin{bmatrix} -2 & 1 & 1 \\\\ 1 & -2 & -1 \\\\ 1 & -1 & -2 \\end{bmatrix}.\n",
"\\end{equation*}"
]
},
{
"cell_type": "markdown",
"id": "fa3ad9a3",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "4ea03ccd",
"metadata": {},
"source": [
"Provide a real, orthogonal matrix $Q$ and a diagonal matrix $\\Lambda$, such that\n",
"\n",
"\\begin{equation*}\n",
" Q^T \\, A\\, Q=\\Lambda.\n",
"\\end{equation*}\n",
"\n",
"You must choose $Q$ such that it has $\\operatorname{det}\\,Q=1$. You may use SymPy for this exercise.\n",
"\n",
"**Note:**\n",
"Real, orthogonal matrices always have $\\operatorname{det} Q = \\pm 1$ (why do you think that is?). If your chosen $Q$ has $\\operatorname{det} Q = -1$, you can simply change the sign of any column or row. Real orthogonal matrices with $\\operatorname{det} Q = 1$ are sometimes called *properly oriented*. In $\\mathbb{R}^3$, this simply means that the orthonormal basis which constitutes the columns in $Q$ forms a *right-handed* coordinate system. This does not play a significant role for us in this problem.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "89f35007",
"metadata": {},
"source": [
"State the functional expression of $k(x,y,z),$ convert it to matrix form, and *reduce* it."
]
},
{
"cell_type": "markdown",
"id": "4df175d4",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "53accdf9",
"metadata": {},
"source": [
"Find a properly oriented orthonormal basis for $\\mathbb{R}^3$ in which the formula for $q$ has no mixed terms. Determine the functional expression."
]
},
{
"cell_type": "markdown",
"id": "75f6c795",
"metadata": {},
"source": [
"### 7: The Partial Derivative Increases the most in the Gradient Direction"
]
},
{
"cell_type": "markdown",
"id": "c17e8d54",
"metadata": {},
"source": [
"This exercise is taken from the textbook, and the goal is to argue why in the gradient method, one moves in the direction of the gradient vector.\n",
"\n",
"Let $f: \\mathbb{R}^{n} \\to \\mathbb{R}$ be a function for which all directional derivatives exist at $\\pmb{x} \\in \\mathbb{R}^{n}$. Assume that $\\nabla f(\\pmb{x})$ is not the zero vector."
]
},
{
"cell_type": "markdown",
"id": "eb822966",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "7a01bcc3",
"metadata": {},
"source": [
"Show that $\\pmb{u} := \\nabla f(\\pmb{x}) / \\Vert \\nabla f(\\pmb{x}) \\Vert$ is a unit vector."
]
},
{
"cell_type": "markdown",
"id": "8eb05748",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "4562f022",
"metadata": {},
"source": [
"Show that the scalar $|\\nabla_{\\pmb{v}}f(\\pmb{x})|$ becomes largest possible when $\\pmb{v} = \\pm \\pmb{u}$."
]
},
{
"cell_type": "markdown",
"id": "72e95125",
"metadata": {},
"source": [
"### 8: Standard Equations for the Three Typical Conic Sections"
]
},
{
"cell_type": "markdown",
"id": "b95006f7",
"metadata": {},
"source": [
"In the following examples, we consider quadratic forms without *mixed terms* (since we can eliminate these through diagonalization, as in the previous exercise). Here, it is possible to take it a step further and remove the first-degree terms as well. This technique is called [completing the square](https://en.wikipedia.org/wiki/Completing_the_square#Basicexample). In the following, we will use this technique in order to identify so-called conic sections."
]
},
{
"cell_type": "markdown",
"id": "7ccdd6d1",
"metadata": {},
"source": [
"#### Question a"
]
},
{
"cell_type": "markdown",
"id": "a1468620",
"metadata": {},
"source": [
"An ellipse in the $(x, y)$ plane with center $(c_1, c_2)$, semi-axes $a$ and $b$, and symmetry axes $x = c_1$ and $y = c_2$ has the standard equation \n",
"\n",
"\\begin{equation*}\n",
" \\frac{(x-c_1)^2}{a^2}+\\frac{(y-c_2)^2}{b^2}=1.\n",
"\\end{equation*}\n",
" \n",
"An ellipse is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" 4x^2+y^2+8x-6y+9=0.\n",
"\\end{equation*}\n",
"\n",
"Complete the square, put the equation in standard form, and specify the ellipse's center, semi-axes and symmetry axes."
]
},
{
"cell_type": "markdown",
"id": "439f597a",
"metadata": {},
"source": [
"#### Question b"
]
},
{
"cell_type": "markdown",
"id": "5815a48f",
"metadata": {},
"source": [
"A hyperbola in the $(x,y)$ plane with center $(c_1,c_2),$ semi-axes $a$ and $b$, and symmetry axes $x=c_1$ and $y=c_2$ has the standard equation\n",
"\n",
"\\begin{equation*}\n",
" \\frac{(x-c_1)^2}{a^2}-\\frac{(y-c_2)^2}{b^2}=1.\n",
"\\end{equation*}\n",
"\n",
"Alternatively (if it isn't horizontally but vertically oriented):\n",
"\n",
"\\begin{equation*}\n",
" \\frac{(y-c_2)^2}{a^2}-\\frac{(x-c_1)^2}{b^2}=1.\n",
"\\end{equation*}\n",
"\n",
"A hyperbola is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" x^2-y^2-4x-4y = 4.\n",
"\\end{equation*}\n",
"\n",
"Complete the square, put the equation in standard form, and specify the hyperbola's center, semi-axes and symmetry axes."
]
},
{
"cell_type": "markdown",
"id": "776cf534",
"metadata": {},
"source": [
"#### Question c"
]
},
{
"cell_type": "markdown",
"id": "d876b620",
"metadata": {},
"source": [
"A parabola in the $(x, y)$ plane with its vertex (stationary point) at $(c_1, c_2)$ and symmetry axis $x = c_1$ has the standard equation\n",
"\n",
"\\begin{equation*}\n",
" y-c_2=a(x-c1)^2.\n",
"\\end{equation*}\n",
"\n",
"Alternatively, if the parabola is not vertically but horizontally oriented, in which case the symmetry axis becomes $y=c_2$:\n",
"\n",
"\\begin{equation*}\n",
" x-c_1=a(y-c2)^2.\n",
"\\end{equation*}\n",
"\n",
"A parabola is given by the equation\n",
"\n",
"\\begin{equation*}\n",
" 2x^2+12x-y+17=0.\n",
"\\end{equation*}\n",
"\n",
"Complete the square, put the equation in standard form, and specify the parabola's vertex and symmetry axis."
]
},
{
"cell_type": "markdown",
"id": "182d6785",
"metadata": {},
"source": [
"___\n",
"\n",
"## Theme Exercise -- Short Day"
]
},
{
"cell_type": "markdown",
"id": "46bb04d9",
"metadata": {},
"source": [
"This week the Short Day will be dedicated to [](tema:2)."
]
}
],
"metadata": {
"jupytext": {
"formats": "ipynb,md:myst"
},
"kernelspec": {
"display_name": "python3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}