H2−H∞ control of continuous-time nonlinear systems using the state-dependent Riccati equation approach

ABSTRACT This paper presents a novel state-dependent Riccati equation (SDRE) control approach with the purpose of providing a more effective control design framework for continuous-time nonlinear systems to achieve a mixed nonlinear quadratic regulator and H∞ control performance criteria. By solving the generalized SDRE, the optimal control solution is found to achieve mixed performance objectives guaranteeing nonlinear quadratic optimality with inherent stability property in combination with H∞ type of disturbance reduction. An efficient computational algorithm is given to find the solution to the SDRE. The efficacy of the proposed technique is used to design the control system for inverted pendulum, an under-actuated nonlinear mechanical system.


Introduction
The Hamilton Jacobi equation (HJE) is a traditional approach to characterize the optimal control of nonlinear systems. The solution of the HJE provides the necessary and sufficient optimal control condition for system modelled by nonlinear dynamics. When the controlled system is linear time-invariant and the performance index is linear quadratic regulator (LQR), the HJE reduces to an algebraic Riccati equation (ARE).
The H ∞ approach has shown its effectiveness in design controller for modern control systems in minimizing the effect of external disturbances (Doyle, Glover, Khargonekar, & Francis, 1989). A parameterization of H ∞ dynamic compensators for the state feedback and full information cases was given in Zhou (1992) in terms of the Hamiltonian matrix and the Youla-Kucera parameterization. As for H ∞ nonlinear control problem, the suboptimal control solution is equivalent to solving the corresponding Hamilton Jacobi Inequalities (HJIs). However, HJEs and HJIs, which are first order partial differential equations and inequalities, cannot be solved in closed form for more than a few state variables, and can impose numerical problems (Anderson & Moore, 1990;Basar & Bernhard, 1995;Huang & Lu, 1996;Khalil, 2002;Van der Shaft, 1993).
Motivated by the success of linear system optimal control methods, there has been an in-depth research in approximating the solutions of HJEs and HJIs over the CONTACT Xin Wang xwang@siue.edu last decade. As powerful alternatives to HJE/HJI techniques: state-dependent linear matrix inequality (SDLMI) and state-dependent Riccati equation (SDRE) techniques have provided us very effective algorithms for synthesizing the nonlinear feedback controls. Both SDLMI and SDRE utilize state-dependent state space representations and some of the earliest work can be found in Cloutier (1997), Cloutier, D'Souza, and Mracek (1996), Huang and Lu (1996), and Mohseni, Yaz, and Olejniczak (1998). The purpose behind SDLMI is to convert a nonlinear system control design into a convex optimization problem involving SDLMI solutions. The recent development in numerical algorithms for solving convex optimization provides very efficient means for solving LMI. If a solution can be expressed in LMI form, then there exist efficient algorithms providing global numerical solutions (Boyd, Ghaoui, Feron, & Balakrishnan, 1994). The work by Scherer (1995) was one of the earliest that discuss the H ∞ problem with mixed H 2 − H ∞ objectives. As pointed out in Yaz (2009), Wang, Yaz, andJeong (2010), , , SDLMI provides us an effective method to synthesize nonlinear feedback control in achieving nonlinear quadratic regulator (NLQR), H ∞ and positive realness performance criteria. However, SDLMI approach may not provide a reliable control solution if the linear matrix inequality is found to be infeasible to solve.
As a powerful alternative to SDLMI method, the SDRE control has emerged as a general design method since the mid-1990s, which provides a systematic and effective design framework for nonlinear systems. Motivated by LQR control by ARE, Cloutier (1997) and Cloutier et al. (1996) extended the result to NLQR problem by using state-dependent coefficient matrices. A discrete SDRE method is developed in Dutka, Ordys, and Grimble (2000). The NLQR and H ∞ control of discrete time nonlinear systems using SDRE approach can be found in Wang, Yaz, Schneider, and Yaz (2011). Due to the computational advantage, stability and effectiveness in control, the SDRE method is of meaningful and practical importance and has a wide range of applications, including robotics, missiles, aircraft, satellite/spacecraft, unmanned aerial vehicles, ship systems, autonomous underwater vehicles, automotives, process control, chaotic systems, biomedical systems, guidance and navigation, etc. A survey of the recent development of SDRE method has been summarized by Cimen and can be found in Cimen (2008).
Traditionally, the SDRE method has been used for NLQR, as appeared in most publications (Cloutier, 1997;Cloutier et al., 1996). The contribution of this manuscript is to propose a novel mixed H 2 − H ∞ SDRE control approach with the purpose of providing a more general control design framework to continuous-time nonlinear system. The form of the result is simple to implement and can be used to select the parameter matrices to satisfy the general design criteria. By solving the generalized SDRE, the optimal and robust control solution is found to satisfy mixed performance criteria guaranteeing nonlinear quadratic optimality with inherent stability property in combination with H ∞ type of disturbance reduction. The effectiveness of the proposed technique is demonstrated by simulations involving the control of an under-actuated mechanical system. The purpose of this work is to provide a more effective control framework and a more flexible design method for nonlinear systems SDRE control to achieve a mixed NLQR and H ∞ performance.
The paper is organized as follows: In the second section, the system model and the performance criteria are introduced. In the third section, the derivation of the NLQR -H ∞ SDRE controller is provided. Optimal control solution can be obtained by solving this generalized SDRE. The fourth section contains an illustrative example, involving the control of the inverted pendulum system. Finally, the conclusions are summarized in the fifth section.
The following notation is used in this work: x ∈ n denotes n-dimensional real vector with norm x ∈ (x T x) 1/2 , where (·) T indicates transpose. A ≥ 0 for a symmetric matrix denotes a positive semi-definite matrix. L 2 is the space of infinite sequences of finite dimensional vectors with finite energy: The derivative of x with respect to time:ẋ = dx/dt will be used in the development.

System model and performance index
Consider the input affine continuous-time nonlinear system given by the following differential equation: where x ∈ n is the state of the dynamical system, u ∈ m the applied input, w ∈ q the L 2 type disturbance, and A, B, F the known coefficient matrices of appropriate dimensions, which can be the functions of x.
The performance output z ∈ p is generalized as follows: where the coefficient matrices can also be functions of x. It is assumed that the state feedback is available. Otherwise, estimated state variable can be obtained from nonlinear state estimator. The nonlinear state feedback control input is given by ( 3 ) It is desired to find the state variable feedback gain K such that the closed loop system Consider the quadratic energy function with P(x) ≥ 0, a function of x, for the following differential inequality, which serves as the performance criteria of controller design: where Q(x) > 0, R(x) > 0 are weighing matrices and can be functions of x. Note that upon integration over time from 0 to T, Equation (5) yields dt representing the NLQR (H 2 control) performance objective is always positive semi-definite. By assuming the initial condition that Since Equivalently, the H ∞ disturbance rejection can be realized by minimizing γ , subject to the condition of Call γ * the minimum gain for which this occurs. Doyle et al. and Scherer have shown procedure for calculate the minimum gain using quadratic convergent algorithm in Scherer (1995) and Zhou (1992) for linear system control problems. Therefore, by properly specifying the value of the weighing matrices Q, R, C, D, G, this performance objective can be used in nonlinear control design, which yields mixed H 2 and H ∞ performance criteria.

Main results
In this section, we shall drop the argument x to simplify the notation. The following theorem summarizes the main results of the paper: Theorem: Given the system equation (1), performance output (2) and control input (3). If the following conditions are satisfied: Assume that (A, C) is detectable, (A, B) is stabilizable.
To satisfy Lasalle's stability condition, the quadratic energy function V = x T Px must be strictly decreasing. Therefore, we haveV < 0, which implieṡ In order to apply Schur complement, we assume Denote the matrix and −1 matrix always exists.
Then the performance criteria (6) can be achieved by using the optimal state feedback control gain: where P is solved from the generalized SDRE: Proof: By applying system Equation (1), performance output (2) and control input (3), performance criteria (5) becomes Equivalently, we have By applying the Schur complement result, we have the following inequality: Equivalently, we have In order to guarantee the controller stability, the quadratic energy function V = x T Px must be decreasing, that is, −Ṗ > 0. Since we are trying to minimize weighing matrix P, the minimum value is achieved when the inequality (19) is satisfied as the following equality: By collecting terms with different powers of K, we have Equivalently, Equation (21) can be simplified as where By completing the square in controller gain K, we have For Equation (24) to be equal to Equation (22), we must have Therefore, the optimal feedback gain is found to be When K = K o , the minimum P is defined by the positive definite solution of the following Generalized SDRE: which concludes the proof.

Corollary 1: NLQR SDRE Control If we do not have H ∞ component in the performance index, that is, only NLQR control exists, then the following controller can be derived as a special case:
If we neglect the noise term, then system equation becomesẋ For F and G identically equal to zero, the optimal feedback control gain can be derived as P is defined by the positive definite solution of the following SDRE: Therefore, the conventional SDRE solution proposed by Cloutier (1997) and Cloutier et al. (1996) is derived as a special case of our results.  and (A, B) is stabilizable. Then K is a state feedback that stabilizes system (1) and guarantees L 2 gain bounded by γ > γ * , and can be written as where P = P T ≥ 0 is the solution to: A special case of Corollary 2 for linear systems control application with the condition G(x) = 0 matches the results from Theorem 1 in Gadewadikar, Lewis, Xie, Kucera, and Abu-Khalaf (2007).

Solution algorithm
Although the Generalized SDRE (14) seems quite complicated, we have found a method which is fairly easy to implement, and standard SDRE solvers can be used to the solution of the generalized SDRE to find the optimal control gain. To facilitate the computation process, the following notation is defined: Then, the equivalent form of generalized SDRE (14) can be rewritten as follows, using the notation above.
The equivalent form of Equation (13) is given as follows: Therefore, a standard SDRE solver can be used to find the solution for the Generalize H 2 − H ∞ SDRE control without increasing any computational complexity.

H 2 − H ∞ SDRE control of inverted pendulum on a cart
The inverted pendulum problem is a classical control problem and is used widely as a benchmark for testing control algorithms. The pendulum mass is above the pivot point which is mounted on a horizontally moving cart. This real example is used in this work to demostrate the effectiveness and robustness of the generalized SDRE control approach. Figure 1 shows the physical representation of the inverted pendulum system. A beam attached to the cart can rotate freely in the vertical twodimensional plane. The angle of the beam with respect to the vertical direction is denoted at angle θ . The cart moves in the one-dimensional track, with position x. The external force F, the control input acting on the cart, is desired to stabilize this highly nonlinear system while satisfying general performance objectives. The control objective is to find the SDRE control to set cart position x, velocity of the cartẋ, angle of the beam θ and angular velocityθ all to zero.
Traditional nonlinear control techniques assume that θ is a very small angle, cos(θ ) ∼ = 1, sin(θ ) ∼ = 0, and linearize the system equation around its equlibrium point afterwards. Other nonlinear control methods might be applicable also. However, it can be shown that the control is not guaranteed to be optimal or stable. In this paper, we will not resort to the usual linearization approach. That is why a detailed account of the system modelling is provided. A model of the inverted pendulum problem can be derived using the Euler-Lagrange equation: where M is the mass of the cart, m the mass of the pendulum, b the friction coefficient between cart and ground, L the length to the pendulum centre of mass. The total length of the pendulum equals 2L, I = 1 3 m(2L) 2 the inertia of the pendulum, F the external force, input ot the system. Denote x 1 = x, x 2 =ẋ, x 3 = θ , x 4 =θ and Considering the effect of external disturbance and noise, the state space model for the system can be written as where It should be noted that the state space formulation in the state-dependent format is not a process of linearization, but a process of state-dependent parameterization. To avoid the division by zero, the sin(x 3 ) x 3 term is substituted for x 3 = 0 by the limit      Assume the initial condition x 1 = 1, x 2 = 0, x 3 = π/4, x 4 = 0.
All of the above mixed criteria control performance results are shown in Figures 2-6. Simulation results show that the classical SDRE control has the fastest response time. Predominant H 2 control shows very similar response with the classical SDRE control, since both of them predominant satisfy the same performance objective. Predominant H ∞ shows better capability of disturbance rejection, but slower response rate.

Conclusions
A novel generalized SDRE control of continuous-time nonlinear systems based on a mixed H 2 − H ∞ performance criteria is presented in this paper. The optimal control solution can be obtained by solving the generalized SDRE. It is shown that the conventional SDRE for H 2 − H ∞ criteria are special cases of the Generalized SDRE approach when the performance index is restricted. An efficient computational algorithm is provided to find the solution of control feedback gain. And the inverted pendulum is used as an illustrative example to demonstrate the efficacy of the proposed method. For future work, the mixed H 2 − H ∞ SDRE control approach will be extended to nonlinear systems with nonaffine structure.

Disclosure statement
No potential conflict of interest was reported by the authors.