Modeling and simulation of dynamic systems

The maintenance policy optimization of a complex industrial process requires a dynamic model allowing the simulation of its behavior. Dynamic bayesian networks represent the ideal tool for representing compactly such systems. They allow computing the probability distributions of the system states with respect to the time and the maintenance actions. Dynamic bayesian networks can then be used for the estimation of the system reliability (cf [1]), or for the evaluation and the automatic learning of maintenance policies.

In order to illustrate the methodology that can be used with BayesiaLab, we use the simple dynamical system presented in [1] and described in the figure below. It is made of 3 valves that control the distribution of a fluid. Each valve as two failure modes: Remains Opened (RO) and Remains Closed (RC).

The dynamic bayesian network below represents this system. Valve1, Valve2 and Valve3 represent the state of the valves at time t. Valve1 t+1, Valve2 t+1 and Valve3 t+1 represent the state of valves at time t+1. The conditional probability tables associated allow specifying the failure rates. Remains Opened and Remains Closed can be used to classify the system's failure (for a system that distributes gas, these two types of failure certainly don't have the same consequences). In short, Available determines whether or not the system can be controlled.  

A temporal simulation over 1 000 time steps allows computing the evolution of the system availability. The screen shot below describes that evolution.

The Decision nodes of BayesiaLab can be added to the network (blue square nodes) for the action modeling. The dynamic Bayesian network below describes for example our simple valve process where the maintenance system allows repairing only one valve at a time, The Utility nodes of BayesiaLab (purple diamond nodes) can also be added to the network for the evaluation of the states qualities. The network below associates a fixed cost to the process, repairing costs for each valve, and income and raw material costs depending on the system availability.

It is then possible to exploit the reinforcement learning algorithm of BayesiaLab to learn a maintenance policy optimizing the discounted sum of the expected utilities. The screen shot below describes the evolution of the probability of the system availability with the automatically learned policy. The quality table associated to the Decision node allows describing the maintenance policy: the blue background cells correspond to the best action to apply in the corresponding state.