Robust fault estimation and fault-tolerant control for nonlinear Markov jump systems with time-delays

The problem of the robust fault estimation and robust fault-tolerant control, for a class of nonlinear time-delayed Markov jump systems (MJSs) with both actuator and sensor faults, was studied. Firstly, by extending the system state, the actuator fault state vector, the sensor fault state vector and the original system state vector were extended to auxiliary state variables, and the original system was extended to a generalized system. Then, for the singular system, a generalized observer was designed to achieve the simultaneous estimation of its actuator faults, sensor faults and original system state. In addition, based on the observer estimation, a state feedback fault-tolerant controller was designed to make the closed-loop system stable and meet certain performance indicators. By solving linear matrix inequality, sufficient conditions for the existence of generalized observer and fault-tolerant controller were given. Finally, a numerical example and a practical example were given to demonstrate the effectiveness of the proposed approach.


Introduction
Markov jump systems (MJS) are systems consisting of several subsystems or modes that can be abstracted as time evolution and event-driven mechanisms that randomly switch from one mode to another at different times. During the operation, it will be affected by the external disturbance, human intervention and sudden changes in the external environment or damage to the internal components of the system. Because of this characteristic, MJS have been extensively studied by scholars in the field of control in the past few decades, and some achievements have been made.
In the actual control system, the faults of components, such as actuator and sensor faults inside the system, sudden changes in the external environment, the presence of external disturbances and network transmission delays, often lead to sudden changes in the structure and parameters of the system, reduce the stability performance of the system, and even lead to catastrophic accidents. In order to ensure that the system can still work normally in the presence of faults and external interference, the fault tolerance and antiinterference ability of the system are very important. Therefore, the stability analysis [1][2][3][4][5], fault detection and filtering [6][7][8][9], fault estimation and fault-tolerant control [10][11][12][13][14] of the system have attracted extensive attention of scholars, and have become a research hotspot in the field of control.
In [1], stochastic MJS with mixed time-varying delays and unknown partial transfer rate are discussed, and their exponential stability in the mean square is analysed. To obtain the less conservative stabilization condition, an appropriate weighting method is proposed in [2]. In [6], the H∞ filtering problem is studied for MJS with unknown transition probability. In [7], the H∞ filtering problem of discrete state MJS with time-varying delay is studied. A sliding surface is then constructed and a sliding mode controller is synthesized to ensure that the associated Markovian jump systems satisfy the reaching condition. Moreover, an observer-based sliding mode control problem is investigated in [9]. Although some attempts on fault estimation and fault-tolerant control for MJS have already been reported [14][15][16][17][18], the achieved results are only focused either on actuator faults [14] or on sensor faults [16]. Specially, the [15][16][17][18] have studied the problem of fault estimation and fault-tolerant control for a class of MJS with both actuator faults and sensor faults, but the upper bounds of faults and their derivatives need to be learned in advance. According to the above introduction, it is of great significance and practicality to carry out fault estimation and fault-tolerant control for MJS with actuator faults, sensor faults and external interference. In [19], the paper focuses on the fault-tolerant control (FTC) approach using fault estimation (FE) and fault compensation within the control system in which the design is achieved by integrating together the FE and FTC controller modules. In [20], the paper is concerned with the sliding mode control of uncertain nonlinear systems against actuator faults and external disturbances based on delta operator approach. At present, there are few studies on this issue at home and abroad, so it is worthwhile to discuss these issues in depth [21,22].
In this paper, a generalized observer design method is proposed for a class of nonlinear Markov jump systems with time-delays to estimate the original state of the system, actuator faults and sensor faults simultaneously. Based on the observer estimation, a design method of state feedback fault-tolerant controller is proposed to make the system stable and meet certain performance indexes in the case of simultaneous actuator and sensor fault and external disturbance, and the effectiveness of the proposed method is verified by numerical simulation.
The contributions of this paper are concluded as follows: (1) In the case of uncertain state transition probability, the simultaneous estimation of actuator and sensor is given for a class of MJS with delay link and parameter uncertainty. (2) This paper assumes that the state transition probability matrix has uncertainty in its estimated value, which is more practical than [15,16]. (3) In the design process of this paper, the information about actuator or sensor faults is not required to be determined in advance, thus leading to great advantages over the results in [22][23][24].

Problem statements
Consider a class of nonlinear time-delayed Markov jump systems with the following form: (1) where x(t) ∈ R n is the state vector, u(t) ∈ R m is the control input vector, y(t) ∈ R q is the control output vector, w(t) ∈ R p is the external disturbances input which belongs to L 2 [0, ∞), L 2 [0, ∞) represents the square integrable vector function space, f a (t) ∈ R s and f s (t) ∈ R w are unknown actuator faults and sensor faults, respectively. ϕ(t) is the continuous initial state function on [−τ , 0], τ is the lag time constant of the system. x 0 and r 0 represent the initial state and the initial mode of the system, respectively. g(x(t), x(t − τ )) represent the non-linear term of the system. The mode jumping process {r t , t ≥ 0} is a Markov process with continuous time discrete state in the finite space = {1, 2, · · · , N}, and its state transition probability is as follows: where h > 0, lim h→0 = o(h)/h = 0, π ij represents the state transition probability from state i at time t to state j at time t + h. If j = i, then π ij > 0, else π ii = − N j=1,j =i π ij .A(r t ), A d (r t ), B(r t ), D(r t ), E(r t ), C(r t ) and F(r t ) are constant matrices with appropriate dimensions.
Before system analysis, the following assumptions are made: Assumption 1: We assume that the actuator faults f a (t), sensor faults f s (t) and the external disturbance ω(t) are divisible and satisfy the following equations

Assumption 2:
The matrix N i , L i , T i and Q i with appropriate dimensions satisfy the following equation where E{·} represents the mathematical expectation, and || · || represents the European norm of the vector.

Definition 2 ([22]):
Given the scalar γ > 0, the Markov jump system (1) is said to be stochastically stable and meet γ disturbance attenuation, if there exists a constant M(x 0 , r 0 ) and M(0, r 0 ) = 0, such that the following inequality holds:

Lemma 2 ([26]):
Given the matrix X and Y with appropriate dimension, if there exist a scalar ε > 0 and vector x ∈ R n , y ∈ R n such that where The system (8) is a generalized system with state variables including the original system state, actuator faults and sensor faults. If an effective observer is designed for the system (8), then the original system state, actuator faults and sensor faults can be estimated simultaneously.

Robust observer design
For the system (8), we propose a generalized observer that can simultaneously estimate the original system state, actuator and sensor faults. The observer design is as follows: where z(t) ∈ R n+q+w is the observer intermediate vari- , then from the formula (7), formula (9) and assumption 2, we can obtain the following formula: then we can obtain the following error dynamic system: According to assumption (2), Then the error dynamic system (11) can be simplified to the following formula: Let A = 1, then A set of special solutions to equation (14) are as follows: The following theorem is one of the main results of this paper, which not only gives sufficient conditions for the existence of the generalized observer, but also guarantees the robust stability of the error dynamic system (15).
In this paper, * stands for symmetric transposition.
π ij P j + I, 12 = P i T iĀdi , 13 = P i T iDi , then there is an observer (9), which can make the error dynamic system (15) asymptotically stable and can meet H ∞ performance γ .
Proof: Defined the following Lyapunov-Krasovskii function Then along with the error dynamic system (15), the weak infinitesimal operator of V(e(t), i) is as follows: From lemma 1 and lemma 2, we can obtain that Substituting the formula (16) and (20) into the formula (19), we can obtain: If assuming that ω(t) = 0, then we can obtain that It can be seen from the formula (17) that 1 < 0, that is V(e(t), i) ≤ 0. As can be seen from definition 1, the error dynamic system (15) is asymptotically stable. It is discussed below that under the zero initial condition, the error dynamic system (15) meets H ∞ performance γ .
Under the zero initial conditions, for any nonzero external disturbance ω(t) ∈ L 2 [0, ∞], the weak infinitesimal operator of V(e(t), x(t), i) along the error dynamic system (15) is as follows: Introduce the following performance function: It can be seen from the formula (17), 2 < 0, that is, According to Dynkin's formula, where e 0 , r 0 are the initial values of corresponding variables, respectively. It can be obtained from the formula (26) that It can be seen from definition 2 that the error dynamic system (15) is asymptotically stable and meets H ∞ performance γ . This completes the proof. According to the conclusion of theorem 1, the algorithm for solving the control law is given as follows, and the specific design mechanism in Figure 1.
Step 1: From assumption 2, we can obtain the matrices T i and Q i .
Step 2: Apply the LMI toolbox to solve the inequality (17). If there is a solution, we can obtain the matrix K i through K i = P −1 i Y i . Step 3: Substituting K i into the formula (16), we can obtain matrices N i and L i .
Step 4: The coefficient matrices of the observer have been solved so that the error of the observer converges to zero. In addition, the estimation of the state, actuator faults and sensor fault can be obtained fromx(t) = I n 0 0 x (t),f a (t) = 0 I q 0 x (t) and f s (t) = 0 0 I w x (t), respectively.

Fault-tolerant controller design
Based on the observer design, we propose a method of state feedback fault-tolerant control design, which makes the system relatively stable.
The design of state feedback fault-tolerant controller is as follows: wherex(t) andf a (t) are the estimated values obtained by the observer formula (9). According to the assumption 3, substituting formula (28) into formula (1), the closed-loop system is as follows: then there is a robust fault-tolerant controller (28), which can make the closed-loop system (29) asymptotically stable and can meet H ∞ performance δ = μ 2 + βγ 2 , β is a positive scalar given in the proof below.
Proof: Defined the following Lyapunov-Krasovskii function: Then along the closed-loop system (29), the weak infinitesimal operator of V(x(t), i) is as follows: The analysis process of theorem 2 is similar to that of theorem 1, then As can be seen from the formula (30), where Defined a new Lyapunov-Krasovskii function as follows: Given a positive scalar β > (σ 2 1 /σ 0 ), as can be seen from the formula (34), According to Dynkin's formula, where e 0 , x 0 and r 0 are the initial values of the corresponding variables. According to the formula (39), we can obtain that It can be seen from definition 2 that the closed-loop system (29) is asymptotically stable and meets H ∞ performance δ. This completes the proof.

Proof: Let
Multiplying diag{ W i I I} on the left and right of the formula (30), according to Schur complement theorem, formula (30) is equivalent to formula (41). This completes the proof.
The fault-tolerant controller design procedure is listed following the Step 4 in the previous section.
Step5: Solving the formula (41), we can obtain matrixK i , then according to the estimated statex(t) and actuator faultsf a (t), the fault-tolerant controller (28) is constructed.

Numerical examples
The parameters of the nonlinear Markov jump timedelayed system are selected as follows [22,27]: Mode 1 Mode 2 In mode 1 and mode 2, we assume that ε 1 = ε 2 = 1, the delay time is 3s, and the transition probability Finally, according to step 5, we can obtain δ = 2.4062, and the state feedback gain matrix is as follows:    For the convenience of simulation, we set the initial state as x 0 ( 0.1 −0.01 0.1 ), the actuator faults and the sensor faults as f a (t) = sin(t) and f s (t) = cos(t), respectively. The system state variable estimation is shown in Figures 2-4.
It can be seen from Figures 2-4 that the observer designed in this paper has a good estimation effect  on the state variables. The fault-tolerant controller is designed to make sure the system under the condition of actuator and sensor faults has a certain antiinterference ability and fault tolerance, the simulation results prove the effectiveness of the method.
In addition, it is compared with the method in [26]. But in this paper, the sensor failure and actuator failure are considered, which is more consistent with the actual control system. Considering that the states of the jump system cannot be directly obtained, in [26], under the action of the observer and the controller, the closedloop state curves x 1 , x 2 and x 3 are shown in Figures  5-7, respectively. After about 2s, the system reached stability. If the method in this paper is adopted, the state curvesx 1 , x 2 and x 3 are shown in Figures 2-4, respectively. After about 1.5s, the system is stable, and the oscillation of the state trajectory curves is smaller. It can be seen that the controller designed in this paper is slightly better than [26] in terms of dynamic response speed. At the same time, in this paper, the sensor failure and actuator failure are also considered, more in line with the actual control system.

Practical examples
In order to further verify the method designed in this paper, a simulation is carried out for a practical example to verify the effectiveness of the designed method. A linearized quadcopter model is selected [22,27], the parameter matrix is as follows: wherein β(t) is the uncertain model parameter, which is subjected to a Markov process r(t) with N = 2: The selection of other parameters is the same as that of the numerical example in Section 4.1.
For the convenience of simulation, we set the initial state as x 0 ( −0.1 −0.1 −0.1 ), the actuator faults and the sensor faults as f a (t) = sin(t) and   f s (t) = cos(t), respectively, and the delay time as 3s. The system state variable estimation is shown in Figures 8-10.
It can be clearly seen from Figures 8-10 that the system has certain fluctuations in the initial operating state. After a short adjustment process, the system reaches a stable state. The simulation results verify the effectiveness of the proposed method.

Conclusion
In this paper, robust fault estimation and faulttolerant control for a class of nonlinear Markov jump time-delayed systems with both actuator and sensor faults were studied. Firstly, a generalized system is constructed by expanding the system state. Then a generalized observer design method is proposed for the system, and the simultaneous estimation of the original state, actuator fault and sensor fault of the nonlinear Markov jump time-delayed system is realized. In addition, based on the estimation of observer, a design method of state feedback fault-tolerant controller is proposed to make the system stable and meet certain performance indexes under the condition of simultaneous actuator and sensor failure and external interference. The sufficient conditions for the existence of generalized observers and fault-tolerant controllers are given by linear matrix inequalities. At last, a numerical example is given to demonstrate the effectiveness of the proposed method.

Disclosure statement
No potential conflict of interest was reported by the author(s).