Log-Loss

Definition
The Log-Loss LL(E)LL(E)LL(E)
 reflects the number of bits required to encode an n-dimensional piece of evidence (or observation) EEE
 given the current Bayesian network BBB
. As a shorthand for "the number of bits required to encode," we use the term "cost" in the sense that "more bits required" means computationally "more expensive."
​LLB(E)=−log⁡2(PB(E)),L{L_B}(E) = - {\log _2}\left( {{P_B}(E)} \right),LLB​(E)=−log2​(PB​(E)),
​
where PB(E){{P_B}(E)}PB​(E)
 is the joint probability of the evidence EEE
 computed by the network BBB
:
​PB(E)=PB(e1,...,en)P_B(E) = P_B({e_1},...,{e_n})PB​(E)=PB​(e1​,...,en​)
​
In other words, the lower the probability of EEE
 given the network BBB
, the higher the Log-Loss LL(E)LL(E)LL(E)
. 
Note that EEE
 refers to a single piece of n-dimensional evidence, not an entire dataset.
Normalized Entropy
Expected Log-Loss
Last modified 1mo ago