# Graphical models¶

## Bayesian network¶

A directed acyclic graph where the nodes represent random variables.

Not to be confused with Bayesian neural networks.

### The chain rule for Bayesian networks¶

The joint distribution for all the variables in a network is equal to the product of the distributions for all the individual variables, conditional on their parents.

where denotes the parents of the node in the graph.

## Boltzmann Machines¶

### Restricted Boltzmann Machines (RBMs)¶

Trained with contrastive divergence.

### Deep Belief Networks (DBNs)¶

### Deep Belief Machines (DBMs)¶

## Clique¶

A subset of a graph where the nodes are fully-connected, ie each node has an edge with every other node in the set.

## Conditional Random Field (CRF)¶

Discriminative model that can be seen as a generalization of logistic regression.

Common applications of CRFs include image segmentation and named entity recognition.

### Linear Chain CRFs¶

A simple sequential CRF.

## Markov chain¶

A simple state transition model where the next state depends only on the current state. At any given time, if the current state is node i, there is a probability of transitioning to node j, where is the transition matrix.

## Markov property¶

A process is said to have the Markov property if the next state depends only on the current state, not any of the previous ones.

## Markov Random Field (MRF)¶

A type of undirected graphical model which defines the joint probability distribution over a set of variables. Each variable is represented by one node in the graph.

One use for an MRF could be to model the distribution over the pixel values for a set of images. In order to keep the model tractable edges are only drawn between neighbouring pixels.

## Naive Bayes Model¶

A simple classifier that models all of the features as independent, given the label.