Log-Loss

Definition

The Log-Loss $LL(E)$ reflects the number of bits required to encode an n-dimensional piece of evidence (or observation) $E$ given the current Bayesian network $B$. As a shorthand for "the number of bits required to encode," we use the term "cost" in the sense that "more bits required" means computationally "more expensive."

where ${{P_B}(E)}$ is the joint probability of the evidence $E$ computed by the network $B$:

In other words, the lower the probability of $E$ given the network $B$, the higher the Log-Loss $LL(E)$.