Kullback-Leibler Divergence (Arc Force)

Definition 
In BayesiaLab, the Kullback-Leibler Divergence (or KL Divergence) is used to measure the strength of the relationship between two nodes that are directly connected by an arc. 
We commonly refer to the KL Divergence also as Arc Force.
Formally, ​the Kullback-Leibler Divergence DKL measures the difference between two distributions PPP
 and QQQ
.
​DKL(P(X)∥Q(X))=∑XP(X)log2P(X)Q(X)D_{KL}(P({\cal X})\|Q({\cal X}))=\sum_{\cal X}P({\cal X})log_2\frac{P({\cal X})}{Q({\cal X})}DKL​(P(X)∥Q(X))=∑X​P(X)log2​Q(X)P(X)​
​
For our purposes, we consider PPP
 the Bayesian network that does include the arc for which we wish to compute the Arc Force, and QQQ
 the Bayesian network that does not contain that arc but is otherwise identical.
We interpret this difference DKL as the "force of the arc" or Arc Force.
Note that Filtered Values are taken into account for computing the Arc Force.
Throughout this website, we use Kullback-Leibler Divergence, KL Divergence, and Arc Force interchangeably.
Expected Log-Loss
Information Gain
Last modified 1mo ago