BayesiaLab
Minimum Description Length Score

Minimum Description Length Score

Definition

The Minimum Description Length Score (MDL Score) is derived from Information Theory and has been used extensively in the Artificial Intelligence community.

It consists of the sum of two components that estimate:

  • the minimum number of bits required to represent a model, and
  • the minimum number of bits required to represent the data given the model.

However, in the specific context of Bayesian networks, we need to explain the exact meaning and the notation of these two components:

  • Calculating Complexity: DL(B)
  • Calculating Fit: DL(D|B) "the minimum number of bits required to represent a model" is denoted DL(B)DL\left( {B} \right) (="Description Length of the Bayesian network BB") and refers to the structural complexity of the Bayesian network model BB, which includes the network graph and all probability tables.
    • For brevity, we often use the shorthand "complexity" or "structure" to refer to DL(B)DL\left( {B} \right).
    • Small values of DL(B)DL\left( {B} \right) suggest a simple model structure, and large values a complex model.
    • The goal of this structural part is to apply Occam's Razor, or the law of parsimony, i.e., to choose the simplest hypothesis, all other things being equal.
  • "the minimum number of bits required to represent the data given the model" is denoted DL(DB)DL\left( {D|B} \right) (="Description Length of the data DD given the Bayesian network BB") and refers to the likelihood of the data DD with respect to the Bayesia network model BB.
    • The data likelihood is inversely proportional to the probability of the observed dataset, as inferred by the Bayesian network model.
    • Put simply,DL(DB)DL\left( {D|B} \right) refers to the "fit" of the model to the data.
    • Small values of DL(DB)DL\left( {D|B} \right) suggest a well-fitting model; large values, conversely, imply a poor fit.

BayesiaLab attempts to minimize the MDL Score by evaluating candidate networks during structural learning.

Learn More About Calculating Complexity & Fit


For North America

Bayesia USA

4235 Hillsboro Pike
Suite 300-688
Nashville, TN 37215, USA

+1 888-386-8383
info@bayesia.us

Head Office

Bayesia S.A.S.

Parc Ceres, Batiment N 21
rue Ferdinand Buisson
53810 Change, France

For Asia/Pacific

Bayesia Singapore

1 Fusionopolis Place
#03-20 Galaxis
Singapore 138522


Copyright © 2024 Bayesia S.A.S., Bayesia USA, LLC, and Bayesia Singapore Pte. Ltd. All Rights Reserved.