Step 2: Variable Clustering
BayesiaLab’s Variable Clustering is a hierarchical agglomerative clustering algorithm that uses Arc Force (i.e., the Kullback-Leibler Divergence) for computing the distance between nodes.
At the start of Variable Clustering, each manifest variable is treated as a distinct cluster. The clustering algorithm proceeds iteratively by merging the “closest” clusters into a new cluster. Maximum Cluster Size limits the number of variables in each group.
In the current Variable Clustering launch dialog, enable Maximum Cluster Size and set it to 5. Leave Post-Processing of Leaf Nodes off and disable Generate Cluster Names with Hellixia to retain the Factor names used throughout this example. Earlier versions exposed a Stop Threshold in Preferences; that control is absent from the current Variable Clustering preferences.
Maximum Cluster Size is a modeling choice. For building PSEMs, we recommend a value between 5 and 7, for reasons that will become clear when we show how latent variables are generated. If, however, the goal of Variable Clustering is dimensionality reduction, we suggest increasing the Maximum Cluster Size to a much higher value, thus effectively eliminating it as a constraint.
The Variable Clustering algorithm can be started via Main Menus > Learning > Clustering > Variable Clustering or by using the shortcut S.
With these settings, the current version initially proposes 14 clusters. Set the cluster-count selector to 15 to retain the 15-factor example used throughout this chapter. Each node is now color-coded according to its cluster membership. The following image shows the standalone graph, outside the BayesiaLab window, for better legibility.
BayesiaLab offers several tools for examining and editing the proposed cluster structure. They are accessible from an extended menu bar (highlighted in the screenshot below).
Dendrogram
The Dendrogram allows us to review the linkage of nodes within variable clusters. It can be activated via the corresponding icon in the extended menu. The lengths of the branches in the Dendrogram are proportional to the Arc Force between clusters.
Also, the Dendrogram can be copied directly as a vector or bitmap graphic by right-clicking on it. Alternatively, it can be exported in various formats via the Save As… button. As such, it can be imported into documents and presentations. This ability to copy and paste graphics applies to most graphs, plots, and charts in BayesiaLab.
Cluster Mapping
Mapping offers an intuitive approach to examining the just-discovered cluster structure as an alternative to Dendrogram. It can be activated via the Mapping button in the menu bar.
By hovering over any cluster “bubbles” with the cursor, BayesiaLab displays a list of all manifest nodes connected to that particular cluster. Each list of manifest variables is sorted according to the intra-cluster Node Force. This also explains the names displayed on the clusters. By default, each cluster takes on the name of the strongest manifest variable.
Number of Clusters
The cluster-count selector lets us revise the proposed grouping. Here, it is manually set to 15 for continuity with the rest of the example.
The Dendrogram and the Mapping view respond dynamically to any changes to the number of clusters.
Cluster Validation
The result of the Variable Clustering algorithm is purely descriptive. Once the number of clusters is settled, click Validate Clustering in the toolbar and answer Yes when asked whether to keep the current clustering. BayesiaLab creates one Class per Cluster, named , with representing the factor identifier, and opens the Variable Clustering Report. The cluster colors are retained; the current version does not display a separate Replace Colors prompt.
The Clusters are now saved, and the color-coding is formally associated with the nodes. A Clustering Report provides a formal summary of the new Factors and their associated Manifest variables.
Note that we use the following terms interchangeably: “derived concept,” “unobserved latent variable,” “hidden cause,” and “extracted factor.”
The Classes icon in the lower right-hand corner of the window confirms that Classes have been created corresponding to the Factors. Validation automatically ends the clustering session and closes its extended toolbar.
Editing Factors
Beyond our choice with regard to the number of Clusters, we also have the option of using our domain knowledge to modify which Manifest Nodes belong to specific Factors. This can be done by right-clicking on the Graph Panel and selecting Edit Classes, which opens the Class Editor directly. Select a class and click Edit to inspect or change it. Alternatively, we can click the Classes icon . In our example, however, we show the Class Editor just for reference, as we keep all the original variable assignments in place.
Cluster Cross-Validation
We now examine the robustness of the identified factors, i.e., how these factors respond to changes in sampling. This is particularly important for studies that are regularly repeated with new data, e.g., annual customer satisfaction surveys. Inevitably, survey samples are going to be different between the years. As a result, machine learning will probably discover somewhat different structures each time and, therefore, identify different clusters of nodes. Therefore, it is important to distinguish between a sampling artifact and a substantive change in the joint probability distribution. In the context of our example, the latter would reveal a structural change in consumer behavior.
We start the validation process via Menus > Tools > Resampling > Variable Clustering > Data Perturbation.
This brings up the dialogue box shown below.
These settings specify that BayesiaLab will learn 100 networks with EQ and perform Variable Clustering on each, with a maximum of 5 nodes per cluster. For this run, set both Minimum Structural Coefficient and Maximum Structural Coefficient to 1, leaving Structural Coefficient Standard Deviation at 0.1, and enable Fixed Seed with value 31. Leave Taboo and Post-Processing of Leaf Nodes off. Upon completion, we obtain a report panel, from which we initially select Variable Clustering Report.
The Variable Clustering Report consists primarily of two large tables. The first table in the report shows the cluster membership of each node in each network (only the first 12 columns are shown). Here, thanks to the colors, we can easily detect whether nodes remain clustered together between iterations or whether they “break up.”
The second table shows how frequently individual nodes are clustered together.
Clicking the Clustering Frequency Graph provides yet another visualization of the clustering patterns. The thickness of the lines is proportional to the frequency of nodes being in the same Cluster. Equally important for interpretation is the absence of lines between nodes. For instance, the absence of a line between and Modern says that they have never been clustered together in any of the 100 samples. If they were to cluster together in a future iteration with new survey data, it would probably reflect a structural change in the market rather than a data sampling artifact.