Notation

Functions

Symbol Meaning
\Sigma Sum
\Pi Product
\sigma Sigmoid function
\mathbb{E} Expectation
\log Natural logarithm
|x| Absolute value of x

Sets

Symbol Meaning
\mathbb{R} The set of real numbers
\mathbb{R}^{n} The set of vectors of real numbers of length n
\mathbb{R}^{m \times n} The set of matrices of real numbers of size m \times n

Calculus

Symbol Meaning
f' Derivative of f, shorthand for \frac{df(x)}{dx}
f'' Second derivative of f
\frac{dy}{dx} Derivative of y with respect to x
\frac{d^2y}{dx^2} Second derivative of y with respect to x
\frac{\partial y}{\partial x} Partial derivative of y with respect to x
\frac{\partial^2 y}{\partial x^2} Second partial derivative of y with respect to x

Information theory

Symbol Meaning
D_{KL}(P||Q) KL-divergence between two distributions, P and Q

Linear algebra

Symbol Meaning
x A vector
X A matrix
X^T Transpose of X
X^* Conjugate transpose of X
X^{-1} Inverse of X
||x||_2 Euclidean norm of x
I Identity matrix
X*Y Element-wise product of X and Y
x \cdot y Dot product of x and y
tr(X) Trace of X
det(X) Determinant of X

Probability

Symbol Meaning
X A random variable
P(X) Probability of a particular value of X. Shorthand for P(X=x_i)
U(a,b) Uniform distribution
N(\mu,\sigma^2) Normal distribution

Statistics

Symbol Meaning
\mu Mean
\sigma Standard deviation
V(X) Variance of X
Cov(X,Y) Covariance of X and Y

Machine learning

Symbol Meaning
\theta Parameters of the model
O Observations or data
x Feature vector
L(...) Loss function
y Label
\hat{y} Prediction
g_t Gradient at time t
u_t Parameter update at time t
\eta Learning rate

Reinforcement learning

Symbol Meaning
\pi(s_t) Policy
a_t Action at time t
s_t State at time t
r_t Reward at time t
V(s_t) Value function
A Action set
\gamma Discount factor