Synthesis analysis for data driven model predictive control*

This paper shows our new contributions on data driven model predictive control, such as persistent excitation, optimal state feedback controller, output predictor and stability. After reviewing the definition of persistent excitation and its important property, the idea of data driven is introduced in model predictive control to construct our considered data driven model predictive control, whose state information and output variable are generated by measured data online. Variation tool is applied to obtain the optimal controller or predictive controller through our own derivation. Furthermore, for the cost function in data driven model predictive control, its preliminary stability is analysed by using the linear matrix inequality and one single optimal state feedback controller is given. To bridge the gap between our derived results and other control strategies, output predictor is constructed from the point of data driven idea, i.e. using some collected input–output data from one experiment to establish the output predictor at any later time instant. Finally, one simulation example is given to prove the efficiency of our derived results.


Introduction
Model predictive control (MPC) is emerged as one successful feedback strategy in many industry fields, including process industries in particular. Model predictive control is based on the conventional optimal control that is obtained by minimization or mini-maximization of some performance criterion for a fixed finite horizon or for an infinite horizon. The basic concept of model predictive control is formulated as follows. At the current time, optimal controls for open loop or closed loop are obtained on a fixed finite horizon from the current time. Among the optimal controls on the fixed finite horizon, only the first one is implemented as the current control law. The procedure is then repeated as the next time with a fixed finite horizon from the next time. Owing to this unique characteristic, there are several advantages for wide acceptance of model predictive control, such as closed loop structure, guaranteed stability, good tracking performance, input-output constraint handling, simple computation. Generally, model predictive control is to construct one constraint optimization problem with input-output constraints to get the optimal control, solving it with some numerical optimization algorithms, such as Newton algorithm, split algorithm, convex optimization algorithms and so on.
CONTACT Hong Jianwang wangjianhong210016@163.com * This paper was not presented at any IFAC meeting.
The main mission of advanced control theory is to design a detailed controller in open loop or closed loop structure, so that this designed controller can drive the output of a plant to track an expected set point or to satisfy a given target. Two categories exist for controller design, i.e. model based approach and data driven approach. Consider the first model based approach, a mathematical model of the considered plant is required for the next controller design. It tells that no mathematical model means no controller. Constructing the corresponding mathematical model for the unknown plant is very necessary for this first type of model based approach, and it is also the most difficult step, as it needs some knowledge of other subjects, such as probability theory, linear and nonlinear system theory, etc. This modelling process corresponds to model identification or system identification, which is adopted to obtain the mathematical model exploiting measured data from experiment on the considered open loop or closed loop system. The whole steps for system identification include four main steps, i.e. model structure selection, optimal input design, parameter estimation and model validation. These above four steps are implemented iteratively until to getting one satisfying model, so system identification is the first step or premise for the next controller design, i.e. the idea of identification for control. Usually, trying to apply different system identification strategies into produce a mathematical model, this mathematical model maybe high order and high property of nonlinearity, then it leads to controllers with high order and high nonlinearity. Thus due to the controllers with high nonlinearity, one extra controller reduction procedure is added in the practical application, because the complex controllers are difficult or costly to design and implement. Generally, the obtained controller, designed by model based approach, depends on the identified model for the unknown plant. It means the above four identification processes are repeated again and again, while guaranteeing that the identified model can be used to replace the original plant perfectly.
To alleviate the dependence on the identified model for the controller, then notion of data driven approach is widely studied in recent years. The attracting property of data driven approach is that the controller is designed directly based on measured data. As data driven approach is still in its infancy, different names are called in the references to describe it, such as data driven, data based, model free.... To the best of our knowledge, the principles between data driven approach and system identification are similar to each other, as the measured data are applied to get the mathematical model for the unknown plant in the framework of system identification, but get the approximated controller for the case of data driven approach. The idea of direct data driven control was first proposed in machine learning, then it attracted many researchers in the advanced control field recently. Now this data driven theory is widely applied in control field, for example, direct data driven control, data driven estimation, data driven detection, data driven optimization, etc. The common property among them is that the measured data are used to achieve our main goals, then it means some useful information are extracted from these measured data. On the other hand, data driven approach needs lots of measured data, i.e. the number of measured data is sufficiently large. This requirement is feasible in our information period, and data driven approach was born to overcome the limitation of model based approach, so data driven approach is studied very popularly from theory and practice application.
Due to the application of data driven approach widely in control field, and the similar point between data driven approach and system identification, we call their combination as identification for control, i.e. system identification for direct data driven control. More specifically, we describe a concise introduction or contribution on system identification for direct data driven control, which belongs to data driven approach. In case of the unknown but bounded noise, one bounded error identification is proposed to identify the unknown systems with time varying parameters. Then one feasible parameter set is constructed to include the unknown parameter with a given probability level. In Alamo et al. (2009), the feasible parameter set is replaced by one confidence interval, as this confidence interval can accurately describe the actual probability that the future predictor will fall into the constructed confidence interval. The problem about how to construct this confidence interval is solved by a linear approximation/programming approach, which can identify the unknown parameter only for linear regression model. According to the obtained feasible parameter set or confidence interval, the midpoint or centre can be deemed as the final parameter estimation, further a unified framework for solving the centre of the confidence interval is modified to satisfy the robustness. This robustness corresponds to other external noises, such as outlier, unmeasured disturbance (Bertsekas & Goyal, 2012). The above-mentioned identification strategy, used to construct one set or interval for unknown parameter, is called as set membership identification, dealing with the unknown but bounded noise. There are two kinds of descriptions on external noise, one is probabilistic description, the other is deterministic description, corresponding to the unknown but bounded noise here (Blackmore et al., 2011). For the probabilistic description on external noise, the noise is always assumed to be one white noise, and its probabilistic density function (PDF) is known in advance. On the contrary for deterministic description on external noise, the only information about noise is bound, so this deterministic description can relax the strict assumption on probabilistic description. In reality or practice, bounded noise is more common than white noise. Within the deterministic description on external noise, set membership identification is adjusted to design controllers with two degrees of freedom, it corresponds to direct data driven control or set membership control. Set membership control is applied to design feedback control in a closed loop system with nonlinear system in Campi and Garatti (2016), where the considered system is identified by set membership identification, and the obtained system parameter will be benefit for the prediction output. After substituting the obtained system parameter into the prediction output to construct one cost function, Campi et al. (2009) take the derivative of the above cost function with respect to control input to achieve one optimal input. Set membership identification can be not only applied in MC but also in stochastic adaptive control (Callawy & Hiskens, 2011), where a learning theory kernel is introduced to achieve the approximation for nonlinear function or system. Based on the bounded noise, many parameters are also included in known intervals in prior, then robust optimal control with adjustable uncertainty sets are studied in Zhang et al. (2017), where robust optimization is introduced to consider uncertain noise and uncertain parameter simultaneously. To solve the expectation operation with dependence on the uncertainty, sample size of random convex programs is considered to replace the expectation by finite sum (Zhang et al., 2015). Generally, many practical problems in systems and control, such as controller synthesis and state estimation, are often formulated as optimization problems (Garatti & Campi, 2013). In many cases, the cost function incorporates variables that are used to model uncertainty, in addition to optimization variables, and Farina and Giulioni (2016) employ uncertainty described as probabilistic variables. Abdelrahim et al. (2017) study data driven output feedback controllers for nonlinear system and apply event triggered mode to analyse the robust stability. Data driven estimation is used to achieve hybrid system identification (Pillonetto, 2016), whose nonlinearity is described by one kernel function. During these recent years, the first author studies this direct data driven control too, for example, the closed relation between system identification and direct data driven control (Jianhong & Ramirez-Mendoza, 2020), and data driven model predictive control (Jianhong et al., 2021). Based on above descriptions on direct data driven control and our existed research about system identification, model predictive control, direct data driven control and convex optimization theory , etc., our mission in this paper is to combine our previous results and apply them in practical engineering. During these 2 years, a new interesting subject about persistently of excitation is studied again in data driven control and model predictive control. Willem's fundamental lemma from Willems et al. (2005) gives a data-based parametrization of trajectories for one linear time invariant system. Based on this Willem's fundamental lemma, one parametrization of linear closed loop system is derived to pave a way to study important controller design problems (De Peris & De Tesi, 2020). (Van Warrde & De Peris, 2020) assert that all trajectories of a linear time invariant system is obtained from a single given one on the condition that a persistently of excitation. One necessary and sufficient condition on the informativity of data is derived for some data driven control and analysis problems (Van Waarde et al., 2020).
Based on above detailed descriptions and our previous contributions on data driven control and model predictive control, this paper combines the data driven and model predictive to form our considered data driven model predictive control, i.e. the idea of data driven is introducing into model predictive control. More specifically, when the condition of persistent exciting holds, then state information or output predictor, existing in one performance index or const function for model predictive control, are replaced by a single persistently exciting trajectory. Consider the concept of persistent exciting, it guarantees the observed data includes all the implicit information about the considered system, being from the system identification. A general principle of system identification is that the identified model can only capture behaviour that is exhibited by the system and embodied in the measured data. For example, the long-period dynamics cannot be modelled based on data from a short-duration manoeuvre. Similarly, if the effect of a particular control surface is to be modelled, that control surface must be moved during the testing, and the movement must be sufficiently different from simultaneous variations in other explanatory variables. The same principle applies to all explanatory variables. The fact that the explanatory variables cannot be moved independently for a flying airplane is part of what makes effective experiment design for aircraft system identification a challenging task. If it is known that some surfaces will always be moved in a specified way (such as inboard flaps that always move symmetrically with the same amplitude), then these surfaces can be treated as a single control surface for modelling purposes. This simplifies the analysis, since only one combined effectiveness parameter will be estimated, and reduces flight test and instrumentation requirements, since there is only one effective surface being moved. In general, control surfaces should be tested in the same way as they will be used. For example, if surfaces will always be deflected together symmetrically or asymmetrically, test them that way, rather than moving each control surface individually. This ensures that any interaction effects are properly characterized. The negative aspect of this approach is that if the scheduled movement of the surfaces is changed, more testing will be required, where the surfaces will have to be moved according to the new schedule or individually, to characterize the control effectiveness properly. System identification can be viewed as processing information from measured data to produce model parameter estimates and associated error bounds. For a fixed amount of information from a given manoeuvre or set of manoeuvres, as more model parameter estimates are requested, the parameter estimates get less accurate. In a sense, the fixed amount of information is spread more thinly over the estimation of more model parameters. Because of this, more complicated model structures with more model parameters require more information to get accurate model parameter estimates. This additional information comes from more or better manoeuvres with information content applicable to the model being identified.
Using the property of the persistent exciting, it means an input-state trajectory of the considered system is generated from a finite number of measured ones.
Consider this performance index for model predictive control, whose state information and output predictor are generated online by finite data, the detailed processes are given to derive the optimal controller, i.e. predictive controller. Within the framework of persistent exciting condition, one data-based representation is proposed to give the output predictor only through some basic matrix operations, as this obtained output predictor can be applied not only in this data driven model predictive control but also in classical subspace predictive control. Furthermore, the detailed derivation about that predictive control is from our own work, and it is similar to the variation analysis. It is well known that stability is an important property for all nature systems. If one system is unstable, then it is not useless, so here stability analysis is studied for our considered data driven model predictive control by using the tool of linear matrix inequality.
The main contributions of this paper are formulated as follows: (1) Apply persistent excitation into data driven model predictive control to generate the state information and output predictor by measured data.
(2) Variation tool is used to get the optimal controller, and output predictor is derived by data.
(3) Stability analysis of data driven model predictive control is considered with linear matrix inequality.
This paper is organized as follows. In Section 2, preliminaries about one linear time invariant system and persistent excitation are introduced, and some existed results about the input-output response are reviewed. Then Section 3 combines persistent excitation into data driven model predictive control, whose state information and output predictor are generated by data online. The optimal controller, i.e. predictive control is derived from the point of variation analysis. In Section 4, one data-based representation is given to describe the output predictor, applying in other control strategy. Stability analysis is studied in Section 5 for data driven model predictive control, and our used tool is dependent of linear matrix inequality. In Section 6, one simulation example illustrates the effectiveness of the considered data driven model predictive control. Section 7 ends the paper with a final conclusion and mentions the next topic.

Preliminaries
Consider the following linear time invariant system: where in Equation (1), x(k) ∈ R n , u(k) ∈ R m , y(k) ∈ R p are the state, input and output of the system at the discrete time instant k respectively. Matrices {A, B, C, D} are four matrices with approximated dimensions.
Although linear system does not exist in our normal life, but it is the nonlinear system. After linearizing the nonlinear system at some equilibrium points, then one approximated linear system can be used to replace the original nonlinear form. Furthermore, our proposed theories from this paper can be also applied to the nonlinear system directly, and it is our ongoing work. Due to four matrices {A, B, C, D} exist in above linear time invariant system (1), the problem of identifying these four matrices can be dealt with the subspace identification strategy, i.e. using the input-output data sequence {u(k), y(k)} to yield the four matrices {A, B, C, D} directly. From the linear system theory, the input-output response or trajectory of that system (1) in one time interval [0, t − 1] is formulated as follows: are controllability matrix and observability matrix with order t. Vectors u [0,t−1] and y [0,t−1] are in vectorized forms of the time interval [0, t − 1], i.e.
and I t is one identity matrix, 0 t×n is the zero matrix with approximated dimension. Set (u d,[0, , y d,[0,T−1] ) be the input-output data of the considered system (1), sampled from an experiment, its corresponding Hankel matrix be that.
Similarly to Equation (2), it holds that x d (i) are named as state samples, collected from one experiment. During this paper, the condition of persistent excitation is important for latter section, so first the definition of persistent excitation is reviewed.

Definition 2.1 (Willems et al., 2005): The signal z d,[0,T−1] is persistent excitation of order L if and only if the following matrix
has full rank mL.
Signal with persistent excitation of order L guarantees the considered system is excited completely, then all of its intrinsic modes or properties are shown to us. The important role of persistent excitation means the trajectory space is spanned by time shifts of the measured trajectory, which is a fact as the following Lemma 2.1.

Lemma 2.1 (Willems et al., 2005): If system (1) is controllable, then the following conclusions hold.
(1) If u d, [T−1] is persistent excitation with order n + t, then any t long input-output trajectory of system is expressed as where g ∈ R T−t+1 .
(2) Given a T long input-output trajectory of system, any linear combination of the columns of the matrix (5) is a t long input-output trajectory.
From this above Lemma 2.1, we see after some inputoutput data (U 0,t,T−t+1 , Y 0,t,T−t+1 ) are collected or measured within only one experiment, then other input0output data at later time instants have one simple linear form on the basis of the formal input-output data (U 0,t,T−t+1 , Y 0,t,T−t+1 ), so these input-output data (U 0,t,T−t+1 , Y 0,t,T−t+1 ) are similar to the basic data, as every input-output trajectory of system (1) a linear combination of this basic data, i .e. a finite number of measured data.

Combining persistent excitation into data driven model predictive control
From Section 3, we start to introduce our new synthesis analysis for data driven model predictive control, such as persistent excitation, output predictor and stability analysis.

Data driven model predictive control
Model predictive control is an efficient strategy for practical systems with some physical and operational constraints. Specifically, consider that system (1) again, at each sampling time instant k, model predictive control solves the following numerical optimization problem to obtain the optimal control input u(t).
where x(k) ∈ R n is the current state information at time instant k, Q and R are two semi-definite weighting matrices with their suited order. r(k) is a sequence of reference state strategy. This optimization problem (10) defines one quadratic tracking problem, due to its quadratic cost function.
Based on the definition of persistent excitation in Definition 2.1, and that important property in Lemma 2.1, the above quadratic tracking problem can be changed to the following data driven model predictive control, which replaces (10) with min g,u [1,N] ,x [1,N] J (x, u where (U 0,t,T−t+1 , X 0,t,T−t+1 ) is an input-state trajectory of system (1) and collected from one priori experiment. The reason about why these two optimization problems are equivalent depends on Lemma 2.1. Observing the optimization problem (11) again, the idea of data driven tells us that after an input-output trajectory of system (1) is collected, then other input-state trajectory is a linear combination of this basis trajectory.
The obvious merit of model predictive is to consider the input constraint or output constraint, then the problem of model predictive control is turned to be one numerical optimization problem, so lots of existed optimization algorithms can be used, such as Newton algorithm, gradient algorithm, etc. In this paper, my task is on applying persistent excitation and data driven idea into model prediction control, i.e. use the past measured data to express the future predictive output. Moreover, constraints are considered in those two numerical optimization problems, corresponding to our data driven model prediction control.

Optimal controller
Due to the equivalence between two optimization problems (10) and (11), we only give the detailed derivation for the optimization (10) to obtain the optimal controller u * (k). We first set a Hamiltonian function as.
where k ∈ [1, N], and N is the time horizon, then we have the necessary condition for u * (k) to be an optimal controller is A necessary condition for u(k) to minimize H(k) is ∂H(k) ∂u(k) = 0, thus Since the matrix ∂ 2 H(k) ∂u 2 (k) = 2R is positive definite, and H(k) is a quadratic form in u(k), so the optimal solution u * (k) is.
Assume that After substituting, we get.
Solving for p(k + 1) to yield substituting p(k + 1) into Equation (5), it holds that combining Equation (13) and (17), we have Then we choose K(k + 1) and g(k) as follows: for a zero reference signal, g(k) becomes to be zero, then it is simplified to where the optimal controller is one state feedback controller.

Extended form
Observing Equations (10) and (11),it is the state information x(k) that exists in cost function.But in more general or extended form, output variable y(k) is to replace state information x(k), i.e.
substituting that output equation The cost function (23) is formulated as Similarly Hamiltonian function is set as It holds that A necessary condition for u(k) to minimize H(k) is ∂H(k) ∂u(k) = 0, thus we have ∂H(k) ∂u(k) = 2(Cx(k) − r(k))QD i.e. the optimal controller is that As state information x(k) exists in the optimal controllers, we need to analyse it. Due to the state equation in system (1), we have This recursive form about state information x(k) can be applied in Equation (29) to compute the optimal controller. Comment: Observing these above two optimal controllers (22) and (29), these two optimal controllers are all state feedback controllers. Optimal controller (22) corresponds to the cost function with state information, but optimal controller (29) is for one cost function, being related with the output predictor. And the output predictor is constructed by past measured data. In Section 5, stability analysis of the first model prediction control with the state information is exemplified.

Data-based representation
As output variable y(k) always exists in cost function for some control strategies, for example, linear quadratic control, subspace predictive control , etc., so it is necessary to analyse the output variable for our considered data driven model predictive control, i.e. generating the output predictor to replace the original output variable online. For convenience, let s and f be the number of past and future data, N is the number of data, similarly to the cost function (10). After iterating recursively Equation (1) to yield the following relation: Similarly we have Then it holds that . . .
The following input-output relations holds: where Y k = y(k) y(k + 1) · · · y(k + N − 1) In Equation (34), the estimation corresponding to E 0 is given asÊ where Z * [k−s,k) is the pseudo inverse matrix of data matrix Z [k−s,k) Set the prediction horizon as f, then the past inputoutput data matrix is expressed as follows in our control problem.
Then the future f step output predictor is obtained aŝ where and U [k,k+f −1) is the future control input, i.e.
Based on the estimationÊ 0 , we substitute and formulate above terms to yield.
From Equation (40), output predictorŷ(k) is independent of control input, and this output predictor is generated from data online, so it is benefit in the cost function for our considered data driven model predictive control, i.e. the output variable in cost function (23) can be replaced by data.

Stability analysis
Stability is one important issue for all system, due to unstability means useless for system. Here we only give a preliminary contribution on data driven model predictive control, and the deep detailed result about stability will be our future work. Consider that cost function again Our aim is to design one state feedback controller u(k) = Hx(k), while minimizing the above cost function, for example, in Equation (22).
Assume that V(x) has the form and satisfies the following inequality: As Q is one semi-definite matrix, so quadratic term satisfies that x T (k)Qx(k) ≥ 0. After considering this to obtain that with u(k) = Hx(k), the inequality (43) is changed as The left side of inequality (44) is expanded to be that Changing above inequality (44) into one linear matrix inequality, then Equation (44) is satisfied if there exists H and K such that Applying Schur complement on Equation (45) to get Then Equation (46) is simplified as Continuing to rewrite above Equation (47) as follows: Generally, stability about that data driven model predictive control is formulated as the following Lemma 2. Furthermore, that optimal state feedback controller H can be solved easily. As optimal controller H must satisfy Then that optimal state feedback controller H * is yielded as follows: Observing Equation (22) and (53), these two kinds of optimal state feedback controllers are similar to each other.

Simulation example
We consider one linear time invariant system, described as follows: x(k + 1) = Ax(k) + Bu(k) y(k) = Cx(k) + Du(k) Assume that the system order n = 7 is known, from an open loop experiment, an input-output trajectory {u(k), y(k)} N−1 k=0 of data length N = 1000 is measured. All system matrices are given as Based on this priori known input-output trajectory {u(k), y(k)} N−1 k=0 of data length N = 1000, Equation (40) is proposed to yield the output predictor during the latter time interval. Figure 1 shows the resulting output predictors as well as the true output for comparison. From Figure 1, we see the output predictors are nice, and the errors can be neglected.

Conclusion
In this paper, synthesis analysis on data driven model predictive control is considered from the point of persistent excitation, optimal state feedback controller, output predictor and stability. Although many subjects are mentioned, but they are expanded around the idea of data driven model predictive control, whose cost function included the generated state information and output predictor from data online. As stability analysis here corresponds to our preliminary work, its deep results about stability for data driven model predictive control are our ongoing work.