# Density estimation¶

The problem of estimating the probability density function for a given set of observations.

## Empirical distribution function¶

Compute the empirical CDF and numerically differentiate it.

## Histogram¶

Take the range of the sample and split it up into n bins, where n is a hyperparameter. Then assign a probability to each bin according to the proportion of the sample that fell within its bounds.

## Kernel Density Estimation¶

The predicted density function given an a sample is:

Where is the kernel and is a smoothing parameter.

A variety of kernels can be used. A common one is the Gaussian, defined as:

### Disadvantages¶

The complexity at inference time is linear in the size of the sample.

## Mixture Model¶

Estimates the density as a weighted sum of parametric distributions. The predicted density function for a sample is:

Where is the number of distributions and each distribution, , is parameterised by . It is also weighted by a single scalar where .

The Gaussian is a common choice for the distribution. In this case the estimator is known as a **Gaussian Mixture Model**.

All of the parameters can be learnt using Expectation-Maximization, except for which is a hyperparameter.