H2−H∞ control of discrete-time nonlinear systems using the state-dependent Riccati equation approach

ABSTRACT A novel H2−H∞ State-dependent Riccati equation control approach is presented for providing a generalized control framework to discrete-time nonlinear system. By solving a generalized Riccati equation at each time step, the nonlinear state feedback control solution is found to satisfy mixed performance criteria guaranteeing quadratic optimality with inherent stability property in combination with H∞ type of disturbance attenuation. Two numerical techniques to compute the solution of the resulting Riccati equation are presented: The first one is based on finding the steady-state solution of the difference equation at every step and the second one is based on finding the minimum solution of a linear matrix inequality. The effectiveness of the proposed techniques is demonstrated by simulations involving the control of an inverted pendulum on a cart, a benchmark mechanical system.


Introduction
The Hamilton-Jacobi equation (HJE) is a traditional approach to characterize the optimal control of nonlinear systems. The solution of the HJEs provides the necessary and sufficient optimal control conditions for system modelled by nonlinear dynamics. When the controlled system is linear time-invariant and the performance index is linear quadratic regulator (LQR), the HJEs can be reduced to algebraic Riccati equations (AREs). As for H ∞ nonlinear control problem, the optimal control solution is equivalent to solving the corresponding Hamilton-Jacobi inequalities (HJIs). However, HJEs and HJIs, which are first-order partial differential equations and inequalities, cannot be solved for more than a few state variables.
Motivated by the success of linear system optimal control methods, there has been a great deal of research involves in approximating the solutions of HJEs and HJIs over the last decade. As powerful alternatives to HJE/HJI techniques: the state-dependent linear matrix inequality (SDLMI) and the state-dependent Riccati equation (SDRE) techniques have provided us very effective algorithms for synthesizing the nonlinear feedback controls. Both SDLMI and SDRE utilize state-dependent linear representations, some of the earliest work can be found in Cloutier (1997), Cloutier, D'Souza, and Mracek (1996); Huang and Lu (1996) and Mohseni, Yaz, and Olejniczak (1998). The purpose behind SDLMI is to convert a nonlinear system control design into a convex optimization problem involving state-dependent linear matrix inequality solutions. The recent development in numerical algorithms for solving convex optimization provides very efficient means for solving LMI (Boyd, Ghaoui, Feron, & Balakrishnan, 1994). If a solution can be expressed in LMI form, then there exist efficient algorithms providing globally optimal numerical solutions. Therefore, if the LMIs are feasible, then SDLMI control technique provides optimal solutions at each step for a given state for nonlinear system control problems. As pointed out in Jeong, Feng, Yaz, and Yaz (2010), Yaz (2009), Wang, Yaz, and and , SDLMI provides us an effective method to synthesize nonlinear feedback control in achieving nonlinear quadratic regulator (NLQR), H ∞ and positive realness performance criteria.
The SDRE control has emerged as general design method since the mid-1990s, which provides a systematic and effective design framework for nonlinear systems. Motivated by linear quadratic regulator control by algebraic Riccati equation (ARE), Cloutier et al. extended the result to nonlinear quadratic regulator problem by using state-dependent coefficient matrices as pointed out in Cloutier (1997) and Cloutier et al. (1996). A discrete SDRE method is developed in Dutka, Ordys, and Grimble (2005). Due to the computational advantage and guaranteed local stability, the SDRE method is of practical importance and has a wide range of applications, including robotics, missiles, aircraft, satellite/spacecraft, unmanned aerial vehicles (UAVs), ship systems, autonomous underwater vehicles, automotives, process control, chaotic systems, biomedical systems, guidance and navigation, etc. A recent survey of the development of SDRE method can be found in Cimen (2008Cimen ( , 2010. Traditionally, the SDRE method approaches address the nonlinear quadratic regulator problem. The contribution of this manuscript is to propose a novel H 2 −H ∞ SDRE control approach with the purpose of providing a generalized control framework to discrete-time nonlinear systems. By solving the generalized SDRE at each time step, the optimal control solution is found to satisfy mixed performance criteria guaranteeing quadratic optimality with inherent stability property in combination with H ∞ type of disturbance reduction (Basar & Bernhard, 1995;Van der Shaft, 1993). Two numerical solution procedures: one involving the steady-state solution of a generalized Riccati difference equation and the other involving a state-dependent LMI are also given. The effectiveness of the proposed technique is demonstrated by simulations involving the control of a benchmark mechanical system. The paper is organized as follows: In the second section, the system model and the performance criteria are introduced. In the third section, the derivation of the H 2 −H ∞ SDRE controller is provided. Optimal control solution can be obtained by solving the generalized SDRE. To solve the generalized SDRE, a difference SDRE and an SDLMI solution are also presented to provide computational alternatives. The fourth section contains an illustrative example involving the control of the inverted pendulum on a cart. Finally, the conclusions are summarized in the fifth section. The following notation is used in this work: x ∈ n denotes n-dimensional real vector with norm x = (x T x) 1/2 where (·) T indicates transpose. A ≥ 0 for a symmetric matrix denotes a positive semi-definite matrix. l 2 is the space of infinite sequences of finitedimensional vectors with finite energy: ∞ k=0 x k 2 < ∞.

System model and performance index
Consider the input affine discrete-time nonlinear system given by the following difference equation: where x k ∈ n is the state vector, u k ∈ m the applied input, w k ∈ q the l 2 type of disturbance and A k , B k , F k the state-dependent matrices of known structure. Note that the simplified notation for time-varying matrices A k , B k , etc. is used to denote the state-dependent matrices. The performance output function z k ∈ p is generalized as follows: where C k , D k , G k are state-dependent coefficient matrices of known structure. It is assumed that the state feedback is available. Otherwise, estimated state variable can be obtained from a nonlinear state estimator. The nonlinear state feedback control input is given by Consider the quadratic energy function for the following difference inequality: Note that upon summation over k, Equation (5) yields Notice that Q k and R k are state-dependent counter parts of the weighting matrices in the traditional linear quadratic (H 2 ) control approach and γ 2 is the H ∞ bound. By properly specifying the value of the weighing matrices Q k , R k , C k , D k , mixed performance criteria can be used in nonlinear control design, which yields a mixed NLQR in combination with H ∞ performance index.

Main results
The following theorem summarizes the main results of the paper: Theorem 1: Given the system (1), performance output (2), and control input (3), the mixed performance index (6) can be achieved by using the control feedback where P k is obtained from the generalized SDRE: Proof: By applying system (1), performance output (2), control input (3), performance index (5) can be written as Equivalently, we have Therefore, we have where By applying the Schur complement (Boyd et al., 1994), we obtain which yields The minimum value of P k is achieved when the inequality above is satisfied as an equality. Since the iterative solution starts at P ∞ and runs backward in time and for P k+1 = P k convergence occurs, the difference equation becomes an algebraic equation (Dutka et al., 2005) as follows: By collecting terms, we have Equivalently, the equation can be simply written as where By completing the square in the controller gain K k , we have For Equation (18) to be equal to Equation (20), we must have Therefore, the optimal feedback gain When K k = K o k , the minimum P k is defined by the positive-definite solution of the following generalized SDRE: Equation (23) is the generalized discrete SDRE equation. By solving P k from Equation (23), the H 2 −H ∞ SDRE control can be achieved by Equation (22).

Remark 1:
As a special case, if there is no H ∞ component in the performance index, i.e. the problem is of nonlinear quadratic regulator control, then the following controller can be derived as a special case of the above results: By neglecting the noise term, the system equation becomes The optimal feedback control gain as where P k is defined by the positive-definite solution of the following generalized SDRE: (26) Therefore, the conventional discrete SDRE solution (Dutka et al., 2005) is derived as a special case of our results.

Remark 2:
The generalized SDRE (23) can be numerically difficult to solve. To facilitate the computation process, the following two results provide two alternative numerical solutions to the generalized SDRE in Theorem 1. Method 1 provides us the solution by solving the difference SDRE (28) until the steady state is reached, instead of (23). Method 2 provides us a state-dependent linear matrix inequality approach.

Numerical method 1 (H 2 −H ∞ difference SDRE control)
Given the system (1), performance output (2), control input (3) and performance index (6), optimality can be achieved by using the control feedback where P k is obtained as the steady solution to the following difference SDRE equation: At time step k, the difference equation (28) is iterated starting with an arbitrary initial condition P k,0 > 0 until P k,i converges to P k,i+1 , for i = 1, 2, 3, . . .. Hence, the solution to the generalized SDRE equation (23) can be found using this method. In practical applications, we can choose as the starting value for iterations to calculate P k .

Numerical method 2 (state-dependent LMI control)
Given the system equation (1), performance output (2), control input (3) and performance index (6), if there exist matrices M k = P −1 k > 0 and Y k for all k ≥ 0, such that the following state-dependent LMI holds (Wang, Yaz, & Long, 2014a, 2014b: where 12 = −αM k C T k D k + 0.5 · βM k C T k , and M k+1 ≥ M k , where max π k s.t. M k ≥ π k I, (32) then inequality (5) is satisfied. The nonlinear feedback gain of the controller is given by Proof: Inequality (10) is equivalent to the ≤ 0 following inequality: By adding and subtracting the same term in Equation (34), the following inequality results: Therefore, subject to P k+1 ≤ P k , Equation (35) can be rewritten as By pre-multiplying and post-multiplying the matrix with block diagonal matrix diag{M k , I, I}, where M k = P −1 k , the following inequality as follows: Hence, if the LMI (41) holds, inequality (5) is satisfied. The following initial conditions are assumed: x 1 = 1, x 2 = 0, x 3 = π/4 and x 4 = 0.
Simulation results for different design parameter values are compared in Figures 1-5 for performance: the  classical SDRE or NLQR result (Dutka et al., 2005), the new H 2 −H ∞ controller for a set of design parameter values computed by using the difference equation technique, new controller for two different sets of parameter   values computed by the SDLMI technique and the traditional LQR control based on linearization. From these results, one can choose the controller that suits the designer's expectation best. Note that Figures 1, 3 and 4 show that the traditional LQR technique loses control of the state variables. Figure 5 shows that the lowest control magnitude is needed by the linearization-based LQR technique at the expense of losing control of the state trajectory.

Conclusions
A novel H 2 −H ∞ control of discrete-time nonlinear systems with SDRE approach is presented in this paper. The optimal control solution can be obtained by solving generalized state-dependent Riccati equations or statedependent LMIs. The inverted pendulum on a cart is used as an illustrative example. For future work, the mixed H 2 −H ∞ SDRE control approach will be extended to nonlinear systems with nonaffine structure.

Disclosure statement
No potential conflict of interest was reported by the authors.