3.1 Introduction to Derivatives of a function with multiple variables

Partial Derivatives

In the lecture we have already seen the partial derivative of a function with multiple variables. We can calculate the partial derivative of a function \(f(x,y)\) with respect to \(x\) by treating \(y\) as a constant:

\[ \frac{\partial f}{\partial x} \]

We can interpret this as how much \(f(x,y)\) changes when \(x\) changes. We can do this for all variables of \(f\). Hence, we can also calculate the partial derivative of the function \(f(x,y)\) with respect to \(y\) by treating \(x\) as a constant.

For our example from above \(f(x,y) = 0.5*sin(x) + cos(y)\) we would get:

\[ \frac{\partial f(x,y)}{\partial x} = 0.5*cos(x) \]

for the partial derivative with respect to \(x\) and

\[ \frac{\partial f(x,y)}{\partial y} = -sin(y) \]

for the partial derivative with respect to \(y\).

As an example, let’s take a look again at the plot of the lecture were we plotted the partial derivative of \(f(x,y)\) with respect to \(x\) and \(y\) seperately:

import Pkg
Pkg.instantiate()

# Add new Packages
# import Pkg
# Pkg.add("Revise")
# Pkg.add("GLMakie")
# Pkg.add("ForwardDiff")
# Pkg.add("FiniteDifferences")
# Pkg.add("ForwardDiff")
# Pkg.add("ReverseDiff")

using Revise
include("./PartialDerivativesPlot.jl")
using .PartialDerivativesPlot
using GLMakie

function parabola(x, y)
    return @. x^2 + y^2
end

angle = LinRange(0, 2*pi, 64)
radius = LinRange(0, 1, 16)

x = radius .* cos.(angle')
y = radius .* sin.(angle')
z = parabola(x, y)

tangentPoint=[0.0, 0.5]

with_theme(theme_dark()) do
    Makie.inline!(true)
    fig = Figure(resolution = (800, 650), fontsize = 22)
    ax = Axis3(fig[1, 1], azimuth =0.3*pi, elevation = 0.05 * pi)

    sm = surface!(ax, x, y, z; colormap = :viridis, colorrange = (minimum(z), maximum(z)),
        transparency = false)

    # partial derivative w.r.t. x
    PartialDerivativesPlot.directionalbox!(ax, parabola, tangentPoint, :x, tangentColor=GLMakie.Colors.HSL(0, 1, 0.4))

    # partial derivative w.r.t. y
    PartialDerivativesPlot.directionalbox!(ax, parabola, tangentPoint, :y, tangentColor=GLMakie.Colors.HSL(125, 1, 0.4))

    display(fig)
end

GLMakie.Screen(...)

Total Derivatives

The total derivative of a function \(f(x,y)\) with respect to \(x\) and \(y\) is defined as:

\[ d f = \frac{\partial f}{\partial x} {d x} + \frac{\partial f}{\partial y} {d y} = f_x {d x} + f_y {d y} \]

Remember, the total derivative gives us the relationship of changes in the function \(f(x,y)\) when \(x\) and \(y\) change. Hence \(d x\) and \(d y\) are not variables we can calculate with: \(d x \neq \Delta x\). We can make an approximation:

\[ \Delta x \approx \frac{\partial f}{\partial x} {\Delta x} + \frac{\partial f}{\partial y} {\Delta y} \]

Chain rule

If we want to calculate the derivative of a function \(f(x(t),y(t))\) with respect another variable \(t\), where \(x(t)\) and \(y(t)\) depend on \(t\) we can use the chain rule:

\[ \frac{d f}{d t} = \frac{\partial f}{\partial x} \frac{d x}{d t} + \frac{\partial f}{\partial y} \frac{d y}{d t} \]

This can also be written in a more general way:

\[ \frac{\partial f}{\partial t} = \frac{\partial f}{\partial x} \frac{\partial x}{\partial t} + \frac{\partial f}{\partial y} \frac{\partial y}{\partial t} \]

This form is also valid for functions with more indepedant variables.

Gradient

The gradient of a function \(f(x,y)\) is defined as:

\[ \nabla f = \begin{bmatrix} \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \end{bmatrix} \]

and we can write the total derivative as:

\[\begin{split} d f = \nabla f \cdot \begin{bmatrix} d x \\ d y \end{bmatrix} \end{split}\]

Jacobian Matrix

If we want to calculate the total derivative of a system of equations, we can define a function as \(\mathbf{f}= \begin{bmatrix} f_1 \\ f_2 \end{bmatrix}\) and the total derivative of this function is defined as:

\[\begin{split} d \mathbf{f} = \begin{bmatrix} \frac{\partial f_1}{\partial x} & \frac{\partial f_1}{\partial y} \\ \frac{\partial f_2}{\partial x} & \frac{\partial f_2}{\partial y} \end{bmatrix} \cdot \begin{bmatrix} d x \\ d y \end{bmatrix} = \mathbf{J} \cdot \begin{bmatrix} d x \\ d y \end{bmatrix} \end{split}\]

The matrix \(\mathbf{J}\) is called the Jacobian matrix of the function \(\mathbf{f}\). The Jacobian matrix is a matrix of all partial derivatives of the function \(\mathbf{f}\) with respect to all variables. The columns of the Jacobian matrix are the gradients of the function \(\mathbf{f}\) with respect to the variables:

\[\begin{split} \mathbf{J} = \begin{bmatrix} \nabla f_1 \\ \nabla f_2 \end{bmatrix} = \begin{bmatrix} \frac{\partial f_1}{\partial x} & \frac{\partial f_1}{\partial y} \\ \frac{\partial f_2}{\partial x} & \frac{\partial f_2}{\partial y} \end{bmatrix} \end{split}\]