Derivative Rules

Power, product, quotient, and chain rules with common derivatives.

Basic rules

Here $c$ is a constant and $f, g$ are functions of $x$.

Function	Derivative
Constant $f(x) = c$	$f'(x) = 0$
Power $x^n$	$n \, x^{n-1}$
Scalar multiple $c \, f(x)$	$c \, f'(x)$
Sum $f(x) + g(x)$	$f'(x) + g'(x)$
Product $f g$	$f' g + f g'$
Quotient $f/g$	$\dfrac{f' g - f g'}{g^2}$
Chain $f(g(x))$	$f'(g(x)) \cdot g'(x)$

Common functions

Function	Derivative
$e^x$	$e^x$
$a^x$	$a^x \ln a$
$\ln x$	$1/x$
$\sin x$	$\cos x$
$\cos x$	$-\sin x$
$\sqrt{x}$	$\dfrac{1}{2\sqrt{x}}$

Activation-function derivatives

Written in terms of the output where that is cheaper to compute (as autograd does in practice).

Function	Derivative
Sigmoid $\sigma(x) = \dfrac{1}{1+e^{-x}}$	$\sigma(x)\,(1 - \sigma(x))$
Tanh $\tanh(x)$	$1 - \tanh^2(x)$
ReLU $\max(0, x)$	$1$ if $x > 0$ , else $0$ (undefined at $0$ ; take $0$ )
Leaky ReLU (slope $a$ )	$1$ if $x > 0$ , else $a$
Softplus $\ln(1 + e^x)$	$\sigma(x) = \dfrac{1}{1+e^{-x}}$