Markov model

A Markov model. The numbers are the probabilities that it moves from one "state" to another. States are letters in this picture.

A Markov model is when things are predicted from other things that came immediately before. For example, if it usually snows after it rains, you might guess that it will snow tomorrow if it is raining today. A Markov model is part of probability theory. Mostly, it tries to predict "states" from previous states. Some Markov models include the Markov chain and the Markov decision process.

Markov chain

See the main article: Markov chain

The Markov chain is a simple Markov model. It has "the Markov property," which means that future states of the variable are because of the previous states of the variable.

Hidden Markov model

An example of a hidden Markov model (sometimes called HMM). The states are at the top. The probability of a transfer from a state to a state and also between states and observations are shown as the numbers.

A hidden Markov model is a type of Markov chain. This is very similar to a basic Markov model, but when the state is only partially observable or noisily observable. In other words, past states are related to the state of the system, but they don't tell us enough to perfectly know the state we are trying to predict. This is used commonly. For example, it helps computers automatically turn speech audio into words.

Markov decision process

See the main article: Markov decision process

A Markov decision process is a type of Markov chain. A Markov decision process goes from one state to another. It uses a vector to do this.

Uses

Markov chains are used to predict the future. They can predict prices and wind power.