The practical analysis for closed-loop system identification

Abstract As the existed theories for closed-loop system identification are very mature, and in this short note, one common closed-loopclosed-loop system is considered, then the classical prediction error identification is reviewed for closed-loop system identification. Based on above-existed results on closed-loop system identification, here we continue to do some research on closed-loop system identification from the practical perspective. It means all mathematical derivations and results, proposed here are more general than those existed results from references, and all results are more suited for the practical application. More specifically, the cost function, used to minimize with respect to the unknown parameter, is derived again to its more simplified form. The identification for the unknown plant is obtained again from the point of its usual spectral analysis estimation. After choosing one suited coefficient matrix into the explicit expression about the cost function, one optimal feedback controller or optimal control input is derived to be global minimum. Our derived spectral analysis estimation, simplified cost function, and optimal feedback controller are beneficial for embodying the relations with all input– output signal and other variables, such as the true plant, parameter estimator, spectral analysis estimation etc. Finally, one simulation example has been performed to demonstrate the effectiveness of the theories proposed in this paper.


PUBLIC INTEREST STATEMENT
The cost function, used to minimize with respect to the unknown parameter, is derived again to its more simplified form. The identification for the unknown plant is obtained again from the point of its usual spectral analysis estimation. After choosing one suited coefficient matrix into the explicit expression about the cost function, one optimal feedback controller or optimal input is derived to be global minimum. Generally, the above-simplified cost function, optimal feedback controller and their explicit expressions are useful for practical application, so they are the main reasons of studying closed-loop system identification from the point of practical application.

Introduction
Now many practical systems operate under feedback control situation, due to required safety of operation or to unstable behavior of the plant, as occurs in many industrial processes, such as paper production, glass production, separation process like crystallization, etc. As there does not exist any feedback in open-loop structure, so the output corresponding to the plant affects the input less. In closed-loop system structure, a feedback controller is added to return the collected output back to the excited input. Then, one error signal between the excited input and feedback output can be imposed on the plant to generate one correction action, which makes the output converge to its given value. The essences of closed-loop system are to decrease the error by using the negative feedback controller, and to correct the deviation from the given value automatically. As the closed-loop system can suppress the errors coming from the internal or external disturbances and achieve the specified control goal, so the closed-loop system is most needed in all engineering fields.
There are many subjects on research for the closed-loop system, for example, closed-loop system identification, closed-loop controller design, closed-loop performance monitoring and diagnosis etc. When to consider closed-loop system identification, generally three identification methods exist for closed-loop system, i.e. direct approach, indirect approach, and joint inputoutput approach, where the feedback is neglected in direct approach and then plant model is identified directly by using only input-output data. For the indirect approach, the feedback effect is considered and the input-output data from the whole closed-loopclosed-loop condition are used to identify the plant model. The joint input-output approach is very similar to indirect approach. It means that the joint input-output approach requires two separated steps, (1) Identification of the closed-loop system, and (2) Recalculation of the open-loop model. For the problem of how to design feedback controller in closed-loop system, generally two strategies are used to design the feedback controller in closed-loop structure, i.e. model-based design and direct data-driven design. The primary step of model-based design is to construct the plant model in closed-loop system using system identification theory and apply this mathematical model in designing feedback controller. Conversely for the direct data-driven design, the modeling process is not needed and the controller is directly designed by using only the observed input-output data under closed-loop closed-loop condition. Through comparing these two strategies, the direct data-driven design method is worth studying deeply in future research. But now as the first model-based design strategy is more applied widely, and then we need to do much research on closed-loop system identification. closed-loop performance monitoring and diagnosis comprise a crucial step in maintenance of model-based control system. In the event of performance degradation, diagnostic tools allow us to verify if the unsatisfactory closed-loopclosed-loop operation results from the idea of plant-model mismatch. As above all subjects need the mathematical model corresponding to the closed-loop system, it is necessary to propose new identification strategy for closed-loop system under different situations.
Some references on closed-loop system identification are given as follows. In (Ljung, 1999), the above three methods are presented to identify closed-loop system. In (Boyd & Vandenberghe, 2004), research on the detailed system identification theory are introduced in time domain. Similarly, the frequency domain system identification is given in (Pintelon & Schoukens, 2001), where lots of other methods are proposed to solve the closed-loop system identification, such as maximal likelihood estimation, prediction error method, bias correction method and subspace method etc. A new virtual closed-loopclosed-loop method for closed-loop identification is proposed in (Forssel and Ljung, 1999). In (Augero, 2011), one projection algorithm is proposed on the basis of the recursive prediction error method. In (Forssell & Ljung, 2000), when many inputs exist in closed-loop system, whether can closed-loop system be identified with parts of the inputs controlled? The closed relationship between closed-loop system identification and closed-loop control is obtained in (Leskers, 2007). In (Hjalmarssion, 2005), the linear matrix inequality is used to describe the problem of optimal input design in closed-loop system. Further, the least cost identification experiment problem is analyzed in (Hjalmarssion, 2008). The power spectral of the input signal is considered to be an objective function and the accuracy of the parameter estimations is the constraints (Van Mulders, 2013). In (Georgiou & Lindquist, 2017), H infinity norm from robust control is introduced to be the objective function in the optimal input design problem. Based on the H infinity norm, the uncertainty between the identified model and nominal model is measured and the optimal input is chosen by minimizing this uncertainty (Hostettler & Brirk, 2016). The selection of the optimal input can also be determined from the point of asymptotic behavior corresponding to the parameter estimation (Care et al., 2018). The persistent excitation input in closed-loop system is analyzed and we obtain some conditions about how to achieve persistent excitation (Weyer & Campi, 2017). In (Bravo et al., 2017), the problem of how to apply closed-loop system identification into adaptive control is solved, so that bias and covariance terms are isolated separately. All above results hold when the number of the observed signals will converge to infinity (Tanaskovic et al., 2017). In our own paper (Jian-hong & Yan-xiang, 2017a), we consider the problem of model structure validation for closed-loop system identification, and two probabilistic model uncertainties are derived from some statistical properties of the parameter estimation. The probabilistic bounds and optimum input filter are based on an asymptotic normal distribution of the parameter estimator and its covariance matrix, which was estimated from infinite sampled data. Using some derived results from our own paper (Jian-hong & Yan-xiang, 2017b), a new technique for estimating bias and variance corresponding to the model error is suggested, then one bound described as an inequality corresponds to a condition on the model error. The abovementioned references, used to identify the closed-loop system, often assume that an open-loop system is closed by a feedback mechanism, containing a known, linear time invariant controller. As a result, the prediction error and the inverse covariance matrix about the parameter estimation are the functions of the true parameter, but this true parameter is always unknown. This difficulty can be resolved by replacing the true parameter with an initial estimate, that is obtained from a previous experiment. Further the inverse covariance matrix is a function of the sensitivity function, relating with the closed-loop system. Thus the sensitivity function must also be known, it means that when the system is in closed-loop with a known linear time invariant controller, the sensitivity function is trivially calculated.
From our above descriptions on closed-loop system identification, generally after modeling the considered plant by using system identification theory, then the process of system identification is finished and the process of control design is started. In this short note, one common closed-loop closed-loop system is considered, and the classical prediction error identification is reviewed for closed-loop system identification. Based on above existed results on closed-loop system identification, here we continue to do some research on closed-loop system identification from the practical perspective. It means all derivations and results, proposed here are more general than those existed results from references, and all results are more suited for the practical application. More specifically, the cost function, used to minimize with respect to the unknown parameter, is derived again to its more simplified form. The identification for the unknown plant is obtained again from the point of its usual spectral analysis estimation. After choosing one suited coefficient matrix into the explicit expression about the cost function, one optimal feedback controller or optimal input is derived to be global minimum. Generally, the above-simplified cost function, optimal feedback controller and their explicit expressions are useful for practical application, so they are the main reasons of studying closed-loop system identification from the point of practical application, i.e. the practical analysis for closed-loop system identification.
This short note is organized as follows. In section 2, some preliminaries of closed-loop system are described, providing a model formulation for further study. In section 3, the classical prediction error identification is reviewed to identify the unknown parameter, coming from the closed-loop system. In section 4, the cost function is rewritten to its reduced form, and the unknown plant is identified again by its spectral analysis estimation. In section 5, one optimal feedback controller is derived through choosing one suited coefficient matrix in cost function. In section 6, a simulation example illustrates the effectiveness of our derived results. Section 7 ends the paper with final conclusion and points out the next subject of ongoing research. Here in this short note, all the mathematical derivations are obtained by our own contributions.

closed-loop system description
Consider the following linear time invariant controller CðzÞ, regulating a single input single output system, consisting of output yðtÞ and input uðtÞ (see Figure 1).
In Figure 1, G 0 ðzÞ is a true plant, H 0 ðzÞ is a noise filter, they are all stable, discrete time transfer functions, and H 0 ðzÞ is a monic and minimum phase. CðzÞ is one stable controller. The excitation signal rðtÞ and external disturbance or noise eðtÞ are uncorrelated, eðtÞ is a white noise with zero mean value and variance λ 0 . vðtÞ is a colored noise, obtained by passing white noise eðtÞ through the noise filter H 0 ðzÞ. z is the delay operator, it means that zuðtÞ ¼ uðt þ 1Þ. Then, it holds that In above closed-loopclosed-loop system structure, after some direct computations, we derive one transfer function form.
Continuing to do some computations and we get.
To simply the latter analysis process, define the sensitivity function as.
Then the input and output relations corresponding to the closed-loop system may be rewritten as.
As here our goal is to identify this closed-loop system without any priori knowledge about the controller CðzÞ, so in next section the identification problem is formulated and the classical prediction error identification is reviewed to solve our identification process, as the classical prediction error identification is similar to the direct approach.

Classical prediction error identification
Introduce one unknown parameter vector θ in above closed-loopclosed-loop system, its parameterized forms are given as.
where θ denotes the unknown parameter vector, it exists in the parameterized plant Gðz; θÞ and noise filter Hðz; θÞ,respectively. The goal of closed-loop system identification is to estimate the unknown parameter vector θ N from one collected input-output data set Z N ¼ fyðtÞ; uðtÞg N t¼1 , where N is the number of total observed data.
According to Equation (6), the prediction for output yðt; θÞ can be calculated as the following one step ahead prediction.
Computing the one step ahead prediction error or residual εðt; θÞ, it becomes.
In the standard prediction error identification, when using the input-output data set Z N ¼ fyðtÞ; uðtÞg N t¼1 with the number N of data, the unknown parameter vector θ is identified by.
As one step ahead prediction error or residual εðt; θÞ is a smooth function of parameter vector θ for every time instant t, the parameter estimator is equivalently given as a root of the following function.
Then the unknown parameter vector θ is identified by where notation @ θ means partial derivative with respect to θ, and @ θŷ ðt; θÞ is expanded as that.
A point estimator may then be derived by equating DðθÞ to zero, and solving for θ. It means that the parameter estimator is given as a root of the function DðθÞ. When to solve the root of the function DðθÞ, our proposed Robbins-Monro algorithm can be applied to achieved goal (Jian-hong, 2011). Section 2 gives a coincide description on closed-loop system, and from our own understanding on closed-loop system identification, section 3 reviews the classical prediction error identification.

Practical analysis for cost function
From this section 4, we start to give our own derivations on closed-loop system identification, and all mathematical derivations are obtained from the point of practical application, i.e. practical analysis. In Figure 1, assume frðtÞg is a quasi stationary reference signal with spectrum ϕ r ðwÞ, and due to the while noise with zero mean value and variance λ 0 , so the white noise eðtÞ has spectrum λ 0 , i.e. colored noise vðtÞ has spectrum H 0 ðe jw ÞH � 0 ðe jw Þ ¼ jH 0 ðe jw Þj 2 λ 0 , where � means complex conjugate.
Observing Equation (5) and using the condition about that frðtÞg and feðtÞg are uncorrelated, then some spectrums are easily obtained. For example, the input spectrum is.
where ϕ r u ðwÞ and ϕ e u ðwÞ are two components of the input spectrum, originating from the reference signal and the noise, respectively.
The above equity holds on condition that N tends to infinity, i.e.
Consider that cost function (9), there always exists one true parameter vector θ 0 such that.
This assumption shows that the identified model is contained in the considered model set. To show what make us to propose stealth identification strategy for closed-loop system structure, we concentrate on that prediction error εðt; θÞ in Equation (8). Due to Equation (8) is implicit and it can be expanded easily, we use its another expression, which is similar to direct approach.
Based on above-derived prediction error expression, we get another cost function for Equation (9), i.e.
Applying Equation (13) into above Equation (21) to get that Equation (22) means when N tends to infinity, the minimum value for cost function V N ðθÞ satisfies

Practical analysis for optimal feedback controller
During above mathematical derivation on Equation (17) and (22), corresponding to the spectral analysis estimate Ĝ ðe jw Þ and cost function V N ðθÞ, feedback controller CðzÞ exists in all above explicit expressions. But in practice, this feedback controller CðzÞ may be unknown, the goal of this section is to design this feedback controller CðzÞ.
Here we use the asymptotic variance expression to design optimal feedback controller CðzÞ. A general design criterion based on the asymptotic variance expression is derived as.
It is obvious that the criterion (28) is minimized with the condition that CðzÞH 0 ðzÞS 0 ðzÞ ¼ À MðzÞ It holds that Then the optimal feedback controller CðzÞ is chosen as Using the optimal feedback controller CðzÞ into Equation (2), then the closed-loop design for any extra input rðtÞ. That is our derived optimal feedback controller CðzÞ gives the global minimum of the criterion (28) or (23).

Simulation example
In this section we apply our derived results on a single input and single output system controlled by a model predictive controller. A true data generating system considered here is given as.
The data generating system is operated in closed-loop system with a model predictive control based on our commissioning model ðG init ; H init Þ, where θ init is chosen as that.
θ init ¼ À 1:7 0:7 À 0:4 0:8 0:15 0:1 0:4 0:1 ½ � T All above parameters are chosen in such way that the obtained closed-loop system must satisfy the bounded input and bounded output stability, which is proposed in adaptive control. The model predictive control is tuned so that we get sufficient performance for the commissioning model ðG init ; H init Þ.The model predictive control is set to have a prediction horizon of 100 and a control horizon of 100. The output variable yðtÞ has a weight of 10, and the input variable uðtÞ has a weight of 1. An excitation signal rðtÞ has a bound, i.e. À 1 � rðtÞ � 1.
The applied input signal is given in Figure 2, and we measure the output signal yðtÞ by some measuring devices, where observed output signal is plotted in Figure 3.
When using prediction error identification method to identify unknown parameters, if estimation error δ ¼kθðtÞ À θ k = k θ k is less than one very small value 0.005, then terminate the recursive methods. To verify the efficiency of the identified model Gðθ N Þ by our derived results and make sure that this identified model can be used to replace the true model, we compare the Bode responses among true plant G 0 ðqÞ, our identified model Gðθ N Þ and classical plant, respectively in Figure 4. From Figure 4, we see that these three Bode response curves coincide with each other, and our identified model Gðθ N Þ can converge to its true plant quickly than classical identified plant. This means that our model error G ðqÞ will converge to zero with time increases very quickly.
More specifically, three curves exist in Figure 4, where the first red curve is Bode response for the true plant model, the second black curve corresponds to Bode response from our identified plant model and the third blue curve is for classical plant model. The classical plant model can be chosen through classical least squares method. After observing Figure 4 again, two curves deviate the first red curve during the initial time, but the deviation will be increased with time increase. When comparing the second black curve and third blue curve, the deviation magnitude from the second black curve is less than the third blue curve.

Conclusion
In this short note, one common closed-loopclosed-loop system is considered, and the classical prediction error identification is reviewed for closed-loop system identification. Based on the existed theories on closed-loop system identification, we give one spectral analysis estimation for that unknown plant and reduce one cost function to its more simplified form. After choosing one suited coefficient matrix into the explicit expression about the cost function, one optimal feedback controller or optimal control input is derived to be global minimum. But the identifiability analysis is one more important problem in closed-loop system identification, so we will study this problem in our next subject.