Bayes' Rule (Bayes' Theorem)
Context
“Bayesian inference is important because it provides a normative and general-purpose procedure for reasoning under uncertainty.”
Inductive Reasoning: Experimental, Developmental, and Computational Approaches, edited by Aidan Feeney and Evan Heit
Bayesian inference refers to an approach first proposed by Rev. Thomas Bayes (opens in a new tab) (1702-1761), whose rule allows calculating the probability of an event upon observing an event .
Bayes' Rule
Bayes' rule or Bayes’ theorem relates the conditional and marginal probabilities of events and (provided that the probability of is not equal to zero). More specifically, Bayes' rule allows calculating the conditional probability of event given event with the inverse conditional probability of event given event A.
Posterior
is the conditional probability of event given event . It is also called the "posterior" probability because it depends on knowledge of event . This is the probability of interest.
Note that referring to "posterior" should not be interpreted in a temporal sense, i.e., it does not imply a temporal order between events and .
Prior
is the prior probability (or “unconditional” or “marginal” probability) of event A. The unconditional probability P(A) was first called “a priori” by Sir Ronald A. Fisher (opens in a new tab). It is a “prior” probability because it does not consider any information about event B.
is the prior or marginal probability of event .
Note that "prior," just like "posterior," does not imply a temporal order.
Likelihood Ratio
is the Bayes factor or likelihood ratio.