Node Editor — Assessments

The mechanism of knowledge elicitation in BayesiaLab allows a set of experts to provide assessments on the Conditional Probability Tables of nodes.

A facilitator can mediate and ask relevant questions to obtain the experts’ assessments.

The experts must be defined in the network using the Expert Editor. Two ways of capturing the assessments are available:

• Each assessment is input directly into BayesiaLab.
• Each assessment is input via the online assessment tool.

When a node has assessments, the following icon is displayed at the bottom left of the node. The more significant the disagreement between the assessments for this node, the darker the icon’s background.

Editing Assessments

Assessments can be entered from the Probability Distribution tab in the Node Editor. Experts must already be created.

• A button Assessment becomes active when a cell is selected. After pressing the button, assessment editing is done for the selected line.

When an assessment is made for a line in the conditional probability table:

• The corresponding cells have a green border.
• An icon indicates the level of disagreement between the experts for this cell. The more visible the icon, the greater the disagreement.

Hovering over a cell displays a tooltip with:

• The minimum probability provided by an expert for this cell.
• The maximum probability provided by an expert for this cell.
• The number of assessments for this cell.

Editor

The Assessment Editor displays the list of assessments for the selected line in the conditional probability table.

Each line in this table defines an assessment consisting of:

• Probabilities for each state of the node, summing up to 100. These values are editable by double-clicking or typing directly.
• The name of the expert who made the assessment, chosen from a drop-down menu.
• The confidence the expert assigns to the assessment (editable).
• A comment by the expert about the assessment (displayed as a tooltip when hovering over the cell).
• The time taken to complete the assessment, measured in seconds (editable).

The Add button allows adding a default assessment to the end of the list, which can be edited manually. It is possible to have multiple assessments from the same expert, but this is not recommended for clarity. The Delete button removes selected assessments.

When an assessment is selected, the expert’s image is displayed on the right side:

Assessment Validation

When the Accept button is pressed in the dialog box:

• Assessments are automatically normalized if necessary (the sum of probabilities equals 100).
• A consensus is computed by averaging the assessments for each state, weighted by the confidence levels of the experts. Assessments with zero confidence are not included.

Online Editing

In an online knowledge elicitation session, the Post Assessment button is displayed. Pressing this button posts the current question online, allowing experts to answer it via a dedicated web interface. A window displays in real time the experts who answered the question:

Web Interface

Once the online session is opened, an expert can connect using a secure URL, entering their name and the session name. After logging in, a waiting interface is displayed until a question is posted. The interface shows:

• The current states of each parent node.
• A slider and input field for editing probabilities.
• A field for entering comments and confidence levels.

The Validation button submits the expert’s response to the server.

Tools

The context menu of the expert indicator allows generating an assessment report. Tools for exporting assessments are available in Main Menu > Tools > Assessment, and assessment sensitivity analysis can be accessed via Main Menu > Analysis > Graphic.