Skip to Content

Step 2: Variable Clustering

BayesiaLab’s Variable Clustering is a hierarchical agglomerative clustering algorithm that uses Arc Force (i.e., the Kullback-Leibler Divergence) for computing the distance between nodes.

At the start of Variable Clustering, each manifest variable is treated as a distinct cluster. The clustering algorithm proceeds iteratively by merging the “closest” clusters into a new cluster. Maximum Cluster Size limits the number of variables in each group.

In the current Variable Clustering launch dialog, enable Maximum Cluster Size and set it to 5. Leave Post-Processing of Leaf Nodes off and disable Generate Cluster Names with Hellixia to retain the Factor names used throughout this example. Earlier versions exposed a Stop Threshold in Preferences; that control is absent from the current Variable Clustering preferences.

Variable Clustering with maximum cluster size 5, leaf post-processing off and Hellixia naming off

Maximum Cluster Size is a modeling choice. For building PSEMs, we recommend a value between 5 and 7, for reasons that will become clear when we show how latent variables are generated. If, however, the goal of Variable Clustering is dimensionality reduction, we suggest increasing the Maximum Cluster Size to a much higher value, thus effectively eliminating it as a constraint.

The Variable Clustering algorithm can be started via Main Menus > Learning > Clustering > Variable Clustering or by using the shortcut S.

Learning menu with Clustering and Variable Clustering selected

With these settings, the current version initially proposes 14 clusters. Set the cluster-count selector to 15 to retain the 15-factor example used throughout this chapter. Each node is now color-coded according to its cluster membership. The following image shows the standalone graph, outside the BayesiaLab window, for better legibility.

Loading SVG...
Click to Zoom

BayesiaLab offers several tools for examining and editing the proposed cluster structure. They are accessible from an extended menu bar (highlighted in the screenshot below).

BayesiaLab with the clustering toolbar and cluster-count selector set to 15
Click to Zoom

Dendrogram

The Dendrogram allows us to review the linkage of nodes within variable clusters. It can be activated via the corresponding icon in the extended menu. The lengths of the branches in the Dendrogram are proportional to the Arc Force between clusters.

The Dendrogram window beside the complete color-coded network
Click to Zoom

Also, the Dendrogram can be copied directly as a vector or bitmap graphic by right-clicking on it. Alternatively, it can be exported in various formats via the Save As… button. As such, it can be imported into documents and presentations. This ability to copy and paste graphics applies to most graphs, plots, and charts in BayesiaLab.

Loading SVG...

Cluster Mapping

Mapping offers an intuitive approach to examining the just-discovered cluster structure as an alternative to Dendrogram. It can be activated via the Mapping button in the menu bar.

Loading SVG...
Click to Zoom

By hovering over any cluster “bubbles” with the cursor, BayesiaLab displays a list of all manifest nodes connected to that particular cluster. Each list of manifest variables is sorted according to the intra-cluster Node Force. This also explains the names displayed on the clusters. By default, each cluster takes on the name of the strongest manifest variable.

Trust cluster tooltip listing Trust, Fulfilled, Bold, Active and Character

Number of Clusters

The cluster-count selector lets us revise the proposed grouping. Here, it is manually set to 15 for continuity with the rest of the example.

Cluster-count selector manually set to 15

The Dendrogram and the Mapping view respond dynamically to any changes to the number of clusters.

Cluster Validation

The result of the Variable Clustering algorithm is purely descriptive. Once the number of clusters is settled, click Validate Clustering in the toolbar and answer Yes when asked whether to keep the current clustering. BayesiaLab creates one Class per Cluster, named Factori\mathit{Factor}_i, with ii representing the factor identifier, and opens the Variable Clustering Report. The cluster colors are retained; the current version does not display a separate Replace Colors prompt.

Current confirmation asking whether to keep the selected clustering

The Clusters are now saved, and the color-coding is formally associated with the nodes. A Clustering Report provides a formal summary of the new Factors and their associated Manifest variables.

Variable Clustering Report showing the first Factor classes and their manifest variables
Click to Zoom

Note that we use the following terms interchangeably: “derived concept,” “unobserved latent variable,” “hidden cause,” and “extracted factor.”

The Classes icon in the lower right-hand corner of the window confirms that Classes have been created corresponding to the Factors. Validation automatically ends the clustering session and closes its extended toolbar.

Editing Factors

Beyond our choice with regard to the number of Clusters, we also have the option of using our domain knowledge to modify which Manifest Nodes belong to specific Factors. This can be done by right-clicking on the Graph Panel and selecting Edit Classes, which opens the Class Editor directly. Select a class and click Edit to inspect or change it. Alternatively, we can click the Classes icon . In our example, however, we show the Class Editor just for reference, as we keep all the original variable assignments in place.

Graph Panel contextual menu with Edit Classes highlighted
Class Editor showing Factor_0 through Factor_14 with sizes totaling 47 variables
Click to Zoom

Cluster Cross-Validation

We now examine the robustness of the identified factors, i.e., how these factors respond to changes in sampling. This is particularly important for studies that are regularly repeated with new data, e.g., annual customer satisfaction surveys. Inevitably, survey samples are going to be different between the years. As a result, machine learning will probably discover somewhat different structures each time and, therefore, identify different clusters of nodes. Therefore, it is important to distinguish between a sampling artifact and a substantive change in the joint probability distribution. In the context of our example, the latter would reveal a structural change in consumer behavior.

We start the validation process via Menus > Tools > Resampling > Variable Clustering > Data Perturbation.

Tools menu with Resampling, Variable Clustering and Data Perturbation selected

This brings up the dialogue box shown below.

Variable Clustering Cross-Validation by Data Perturbation window with EQ, 100 tests, and a maximum cluster size of 5

These settings specify that BayesiaLab will learn 100 networks with EQ and perform Variable Clustering on each, with a maximum of 5 nodes per cluster. For this run, set both Minimum Structural Coefficient and Maximum Structural Coefficient to 1, leaving Structural Coefficient Standard Deviation at 0.1, and enable Fixed Seed with value 31. Leave Taboo and Post-Processing of Leaf Nodes off. Upon completion, we obtain a report panel, from which we initially select Variable Clustering Report.

Arc Confidence Analysis Report with an arc, inverted arc, edge, and total frequency for each arc
Click to Zoom

The Variable Clustering Report consists primarily of two large tables. The first table in the report shows the cluster membership of each node in each network (only the first 12 columns are shown). Here, thanks to the colors, we can easily detect whether nodes remain clustered together between iterations or whether they “break up.”

Variable Clustering Report table giving each node's Factor membership in each of the learned networks
Click to Zoom

The second table shows how frequently individual nodes are clustered together.

Node Association Frequencies table showing how often each pair of nodes lands in the same cluster
Click to Zoom

Clicking the Clustering Frequency Graph provides yet another visualization of the clustering patterns. The thickness of the lines is proportional to the frequency of nodes being in the same Cluster. Equally important for interpretation is the absence of lines between nodes. For instance, the absence of a line between Flowery\mathit{Flowery} and Modern says that they have never been clustered together in any of the 100 samples. If they were to cluster together in a future iteration with new survey data, it would probably reflect a structural change in the market rather than a data sampling artifact.

Loading SVG...
Click to Zoom