Mutual Information
Mutual Information measures how much observing reduces uncertainty about , on average.
The Venn Diagram below illustrates this concept:
Loading SVG...
Conditional Entropy is the uncertainty about after observing , averaged over the possible observations of . It is the Expected Log-Loss in bits when predicting from .
Mutual Information can also be computed from the joint probabilities or from the conditional probabilities.
Usage
For a given network, BayesiaLab can report the Mutual Information in several contexts:
- Menus > Analysis > Report > Target > Relationship with Target Node.
- Note that this table shows the Mutual Information of nodes such as and only with respect to the target node, .


- The Mutual Information can also be shown by selecting Menus > Analysis > Visual > Overall > Arc > Mutual Information and then clicking the Show Arc Comments icon or selecting Menus > View > Show Arc Comments.
Loading SVG...
- Note that the corresponding options under Menus > Window > Preferences > Analysis > Visual Analysis > Arc's Mutual Information Analysis have to be selected first:
Loading SVG...
- In Preferences, Child refers to the Relative Mutual Information from the Parent onto the Child node, i.e., in the direction of the arc.
- Conversely, Parent refers to the Relative Mutual Information from the Child onto the Parent node, i.e., in the opposite direction of the arc.