Symmetric Normalized Mutual Information

Symmetric Normalized Mutual Information


The following Venn Diagram illustrates that the Mutual Information is symmetrical for the two variables XX and YY, i.e., I(X,Y)=I(Y,X)I(X,Y)=I(Y,X).

However, the variables XX and YYcan each have a different number of states. Therefore, their respective entropies can be very different.

This means that the absolute value of Mutual Information cannot be interpreted without context. In the Venn Diagram, for instance, I(XY)I(X|Y) reduces H(Y)H(Y) by a bigger percentage than does H(X)H(X). As such, I(X,Y)I(X,Y) would be more "important" with regard to YY than it would be with regard to XX.


The Symmetric Normalized Mutual Information measure takes the difference of the respective entropies of X and Y into account:

ISN(X,Y)=2×I(X,Y)log2(SX)+log2(SY){I_{SN}}(X,Y) = 2 \times \frac{{I(X,Y)}}{{{{\log }_2}({S_X}) + {{\log }_2}({S_Y})}}

As a result, we have an easy-to-interpret measure that relates to both XX and YY together.


For a given network, BayesiaLab can report the Symmetric Normalized Mutual Information in several contexts:

  • Menu > Analysis > Report > Relationship Analysis:
  • The Symmetric Normalized Mutual Information can also be shown by selecting Menu > Analysis > Visual > Overall > Arc > Mutual Information and then clicking the Show Arc Comments icon or selecting Menu > View > Show Arc Comments.
  • Note that the corresponding options under Menu > Preferences > Analysis > Visual Analysis > Arc's Mutual Information Analysis have to be selected first:
  • In Preferences, Child refers to the Relative Mutual Information from the Parent onto the Child node, i.e., in the direction of the arc.
  • Conversely, Parent refers to the Relative Mutual Information from the Child onto the Parent node, i.e., in the opposite direction of the arc.

For North America

Bayesia USA

4235 Hillsboro Pike
Suite 300-688
Nashville, TN 37215, USA

+1 888-386-8383

Head Office

Bayesia S.A.S.

Parc Ceres, Batiment N 21
rue Ferdinand Buisson
53810 Change, France

For Asia/Pacific

Bayesia Singapore

1 Fusionopolis Place
#03-20 Galaxis
Singapore 138522

Copyright © 2024 Bayesia S.A.S., Bayesia USA, LLC, and Bayesia Singapore Pte. Ltd. All Rights Reserved.