Latent Dirichlet Allocation Algorithm

Latent Dirichlet allocation, known as LDA, is a generative statistical algorithm used in natural language processing for topic modeling, describing a collection of text documents in terms of a set of underlying topics that are never directly observed but must be inferred from the words the documents actually contain. It treats each document as a random mixture of these latent topics and each topic as a probability distribution over words, so that a topic about politics might assign high probability to words such as president, government and election while a topic about sports assigns high probability to words such as team, game and score; given only the observed documents, the algorithm works backward through this generative process, typically using Bayesian inference techniques such as Gibbs sampling or variational inference, to estimate both the topic mixture of every document and the word distribution of every topic. Jonathan Pritchard, Matthew Stephens and Peter Donnelly first developed the underlying model in 2000 for population genetics, and David Blei, Andrew Ng and Michael Jordan adapted it for machine learning and text analysis in 2003, after which it became one of the most widely used topic modeling techniques.