Generative model

A generative model is a model for randomly generating observed data, typically given some hidden parameters. Generative models are used in machine learning for either modeling data directly (i.e., modeling observed draws from a probability density function), or as an intermediate step to forming a conditional probability density function. A conditional distribution can be formed from a generative model through the use of Bayes' rule.

Examples of generative models include:

• Gaussian distribution
• Gaussian mixture model
• Multinomial distribution
• Hidden Markov model
• Generative grammar

If the observed data are truly generated by the generative model, then fitting the parameters of the generative model to maximize the data likelihood is optimal. However, data rarely truly arises from the generative models used. Therefore, it is often more accurate to model the conditional density functions directly: i.e., performing classification or regression analysis.