A derivative takes a function $f$ and a position $x_0$ as an argument, and returns a number representing the slope of that function at that point.

• ­$(f,x_0)\to f^{\prime }(x_0)$

$f^{\prime }(x_0)\equiv \underset{ϵ\to 0}{lim}\frac{f(x_0+ϵ)-f(x_0)}{ϵ}$

Numerical derivative

$f^{\prime }(x_0)\equiv \underset{ϵ\to 0}{lim}\frac{f(x_0+ϵ)-f(x_0)}{ϵ}$

$f^{\prime }(x_0)\approx \frac{f(x_0+ϵ)-f(x_0)}{ϵ}$

Fun fact

$f^{\prime }(x_0)\equiv \underset{ϵ\to 0}{lim}\frac{1}{ϵ}\mathrm{\Im }f(x_0+iϵ)$

Fun fact

$f^{\prime }(x_0)\approx \frac{1}{ϵ}\mathrm{\Im }f(x_0+iϵ)+O({ϵ}^2)$

$f^{\prime }(x_0)\equiv \underset{ϵ\to 0}{lim}\frac{f(x_0+ϵ)-f(x_0)}{ϵ}$

• ­$\frac{d}{dx}{x}^3=3x^2$
• ­$\frac{d}{dx}\sin x=\cos x$
• ­$\frac{d}{dx}(x^3+\sin x)=\frac{d}{dx}{x}^3+\frac{d}{dx}\sin x$

• Linearity
  • ­$(f(x)+g(x){)}^{\prime }=f^{\prime }(x)+g^{\prime }(x)$
  • ­$(f(x)-g(x){)}^{\prime }=f^{\prime }(x)-g^{\prime }(x)$
  • ­$(c\cdot f(x){)}^{\prime }=c\cdot f^{\prime }(x)$

• Product rule
  • ­($f(x)\cdot g(x){)}^{\prime }=f^{\prime }g+g^{\prime }f$
• ­Quotient rule
  • ­($\frac{f(x)}{g(x)}{)}^{\prime }=\frac{f^{\prime }g-g^{\prime }f}{g^2}$

• Chain rule
  • ­$(f(g(x)){)}^{\prime }=f^{\prime }(g)\cdot g^{\prime }(x)$
  • ­$\frac{df}{dx}=\frac{df}{dg}\frac{dg}{dx}$
  • ­$\frac{df}{dx}=\frac{df}{da}\frac{da}{db}\frac{db}{dc}\dots \frac{dy}{dz}\frac{dz}{dx}$

Dual numbers

$z=(f,f^{\prime })$

Where:

• ­$f$ represents a function
• ­$f^{\prime }$ represents its derivative
• ­Both of them evaluated at a given point

Addition

$(f,f^{\prime })+(g,g^{\prime })=(f+g,f^{\prime }+g^{\prime })$

Subtraction

$(f,f^{\prime })-(g,g^{\prime })=(f-g,f^{\prime }-g^{\prime })$

Multiplication

$(f,f^{\prime })\cdot (g,g^{\prime })=(fg,f^{\prime }g+f{g}^{\prime })$

Division

$\frac{(f,f^{\prime })}{(g,g^{\prime })}=(\frac{f}{g},\frac{f^{\prime }g-f{g}^{\prime }}{g^2})$

Powers

$(f,f^{\prime }{)}^n=(f^n,n{f}^{n-1}\cdot f^{\prime })$

Nice trick

But how is this useful?

Consider this

$f((x,1))=(f(x),f^{\prime }(x))$

Is true for any algebraic function!

What about non-algebraic?

$\sin ((u,u^{\prime }))=(\sin u,\cos u\cdot u^{\prime })$

Addition

$(f,f^{\prime })+(g,g^{\prime })=(f+g,f^{\prime }+g^{\prime })$

function +(self::Dual, other::Dual)::Dual
    return Dual(self.x + other.x, self.dx + other.dx)
end

Multiplication

$(f,f^{\prime })\cdot (g,g^{\prime })=(fg,f^{\prime }g+f{g}^{\prime })$

function *(self::Dual, other::Dual)::Dual
    y = self.x * other.x
    dy = self.dx * other.x + self.x * other.dx
    return Dual(y, dy)
end

Division

$\frac{(f,f^{\prime })}{(g,g^{\prime })}=(\frac{f}{g},\frac{f^{\prime }g-f{g}^{\prime }}{g^2})$

function /(self::Dual, other::Dual)::Dual
    y = self.x / other.x
    dy = (self.dx * other.x - self.x * other.dx) / (other.x)^(2)
    return Dual(y, dy)
end

Teach also the basics

$+(f,f^{\prime })=(f,f^{\prime })$

+(z::Dual) = z

$-(f,f^{\prime })=(-f,-f^{\prime })$

-(z::Dual) = Dual(-z.x, -z.dx)

Teach also the basics

$(f,f^{\prime })>(g,g^{\prime })⟺f>g$

>(self::Dual, other::Dual) = self.x > other.x

Teach also the basics

$(f,f^{\prime })\equiv (g,g^{\prime })⟺f\equiv g,f^{\prime }\equiv g^{\prime }$

==(self::Dual, other::Dual) = (self.x == other.x) && (self.dx == other.dx)

Example

poly = x -> x^(3) + x^(2) + x  
z = Dual(3, 1)  
poly(z)  
  
> Dual(39, 34)

Example

""" Equivalent to previous example. Just uglier """  
function poly(x)  
    aux = 0 # Initialize auxiliary variable  
    for n in 1:3  
        aux = aux + x 
    end  
end  
  
poly(z)  
  
> Dual(39, 34)

Table of derivatives

'''Enables one-liners like:

operator(z::Dual) = _factory(<fun>, <derivative>)(z)

'''
function _factory(f::Function, df::Function)::Function
    return z -> Dual(f(z.x), df(z.x) * z.dx)
end

Table of derivatives

$\frac{d}{dx}\sin x=\cos x$

sin(x::Dual) = _factory(sin, cos)(x)

Table of derivatives

$\frac{d}{dx}\cos x=-\sin x$

sin(x::Dual) = _factory(sin, cos)(x)
cos(x::Dual) = _factory(cos, x -> -sin(x))(x)

Table of derivatives

$\frac{d}{dx}\tan x=?$

sin(x::Dual) = _factory(sin, cos)(x)
cos(x::Dual) = _factory(cos, x -> -sin(x))(x)
tan(x::Dual) = sin(x) / cos(x)

Table of derivatives

csc(x::Dual) = 1 / sin(x)
sec(x::Dual) = 1 / cos(x)
cot(x::Dual) = 1 / tan(x)

Found typo: the code above requires coercing Real into Dual at quotient definition. See code here. I hid this in the slides for the sake of simplicity.

Kudos for Suvayu Ali for noticing.

Example

fun = x -> x + tan(cos(x)^(2) + sin(x)^(2))  
  
z = Dual(0, 1)  
fun(z)  
  
> Dual(1.557407724654902, 1.0)

Materials available at

pabrod.github.io

• ­Including:
  • ­Link to these slides
  • ­Code examples in Python and Julia
  • ­Links to packages that do this for you
  • ­Links to better materials than mine

Thanks for your attention