Safe-update of bi-layered controller and its application to power systems

In real-world social infrastructures such as power systems, proven controllers are already implemented and operated stably. To further improve the control performance reliably, bi-layered control with inheriting the existing controller is proposed: the existing controller generates the baseline control signal in the upper layer, while multiple sub-controllers individually coordinate the signal to actuate the infrastructure system in the lower layer. The sub-controllers are characterized by few parameters that represent the degree of coordination. Then, the update strategy of the parameter with guaranteeing the safety of the overall system is proposed. The effectiveness of the bi-layered control is shown via a numerical experiment with a power system model.


Introduction
The scale of social infrastructures such as power systems and traffic systems has been getting larger in recent years. Social infrastructures must work reliably, while they have to overcome some problems caused by their scale increase. One of the problems can be seen in the operation and control of the power system. In the worldwide growing awareness of environmental preservation, a large number of renewable energy sources (RESs) are integrated into the existing power system. The high penetration of RES may increase the uncertainty of power supply and lead to deteriorating frequency stability [1]. To prevent deterioration, the controller needs to be updated depending on the RES increase. In other words, the main requirement for next-generation power systems is the ability of adaptation in the controller.
Most of the existing controllers designed for social infrastructures, e.g. load-frequency control (LFC) for the power system, are implemented in a centralized fashion [2]: the controller aggregates the measured information from connected sub-plants and actuates them all at once. This controller is referred to as a global controller. Since practical social infrastructures are spatially distributed, the measurement and actuation for the global controller need to be of low-rank [3]. For example, LFC measures the average frequency from generators and actuates them by broadcasting requests. Due to the severe limitations in the measurement and actuation, the achievable performance by existing global controllers is not satisfactory. This motivates us to relax the limitation and to readdress the controller structure for social infrastructures.
Some works address decentralized structure in the controller and/or apply data-based controller design. Although the works [4,5] show the effectiveness of decentralized control and data-based design, it is not always realistic to completely replace the existing reliable controller with a new one. As is pointed out in [6,7], the safety of the control systems may be impaired during the update of the controller. In other words, the state may largely fluctuate beyond the allowable operation range, which causes hardware failures or puts users at risk. Also, the complete renewal of the controller is not acceptable for users and operators due to their psychological resistance. This article addresses a novel control framework, where the existing controller is not replaced, but inherited to the updated control system.
We consider the bi-layered structure in the controller as shown in Figure 1. In the bi-layered control, multiple local sub-controllers {K i } i∈{1,...,n} are dispersively installed below the global controller K 0 . A similar controller structure can be seen in the concept of glocal control [8], where independently defined global/local objectives are achieved simultaneously. On the other hand, the bi-layered control in this article pursues a common objective by the cooperative design of K 0 and {K i }. It is assumed that the upperlayered controller plays the role of the baseline controller, i.e. K 0 is preliminarily implemented and works well. Then, the design problem of lower-layered controllers is addressed: {K i } are designed to assist the operation of the upper-layered controller by using their local measurement of sub-plants {P i } i∈{1,...,n} . Since the bi-layered controller utilizes the local measurements in addition to the existing global one, it has the potential to further improve the overall control performance.
A variety of controller design methods can be compatible with the presented general framework of bilayered control. A promising one is the application of data-based update, which brings the ability of adaptation in the control system. This article also addresses the update of lower-layered controllers with the safety guarantee, which is the main concern of data-based control [6].
The rest of this article is organized as follows: in Section 2, the design problem of the bi-layered controller is formulated. Each lower-layered controller is characterized by designable parameters, and its design problem is formulated. Section 3 addresses the design method of the parameters based on operating plant data. In addition, some propositions are given, where the safety of the overall system is guaranteed during the parameter update. In Section 4, the demonstration of the bi-layered control using a power system model is shown. Section 5 concludes the article and shows future works.

Notation:
The symbol e i is a unit vector whose ith element is 1, and the symbol 1 denotes the all-ones column vector, i.e. 1 := [1 · · · 1] T . For a vector v, the symbol (v) denotes the diagonal matrix composed of the elements of v. Given matrices and Y, the linear fractional transformations (LFTs) are defined by which are called the lower LFT and upper LFT, respectively [9].

System description
We consider a feedback control system illustrated in Figure 2. The system is composed of the following three parts: one is the plant system defined by the interconnection of the sub-plants {P i } via the connection matrix L, another is the baseline controller K 0 , and the other is the set of additional sub-controllers {K i } to be designed. Each sub-plant P i is described by the following discrete-time state space model: where x i ∈ R m i , u i ∈ R, w i ∈ R, y i ∈ R, and z i ∈ R denote the state, control input, disturbance input, measured output, and control output, respectively, and ξ i ∈ R and η i ∈ R represent the interaction signals between the sub-plants {P i }. In addition, the symbols A pi , B ui , B wi , B ξ i , C yi , C zi , and C ηi denote constant matrices with appropriate dimension. The set of sub-plants {P i } is described by where x ∈ R m 1 +···+m n , u ∈ R n , w ∈ R n , y ∈ R n , z ∈ R n , ξ ∈ R n , and η ∈ R n are the stacked vectors of In addition, the coefficients in (5) denoted by A p , B u , B w , B ξ , C y , C z , and C η are the diagonal matrices composed of their corresponding coefficients of (4). For example, A p := diag(A p1 , . . . , A pn ). We let P(s) ∈ C 3n×3n represent the transfer function matrix of {P i }. As sketched in Figure  2, {P i } are connected via the connection matrix L ∈ R n . This is modelled by using the following equation: Here, we let z 0 be the aggregation of z, i.e. z 0 = 1 T z holds. Then, it follows from (5) and (6) that the overall plant system P all is described by The controllers K 0 and {K i } constitute a bi-layered structure: the baseline controller K 0 is located on the upper layer, while the additional sub-controllers {K i } are on the lower one. The baseline controller K 0 , namely, upper-layered controller, is an SISO static system described by where k 0 is a constant, and r 0 ∈ R and y 0 ∈ R denote the baseline control signal and the aggregation of the measurements {y i }, i.e. y 0 = 1 T y, respectively. The baseline signal r 0 is broadcast to the sub-controllers {K i }. Then, lettingr = 1r 0 , the input-output response from the measured output y to thisr is described bȳ We see from (9) that the upper-layered controller K 0 plays the role of "global" control: the controller is driven by the averaged behaviour of the sub-plants {P i } and broadcasts a baseline control signal to the subcontrollers {K i } [8]. Each sub-controller K i , namely, lower-layered controller, is described in the parametrized form: where α i is the parameter of the function K i . A simplistic class of K i is the linear controller of the form which is characterized by two design parameters α ri and α yi . The lower-layered controller K i works as the "local" coordinator, i.e. it assists the upper-layered controller K 0 by individually coordinating the baseline signal r 0 based on each local measurement y i . It follows from (9) and (11) that the overall bi-layered controller is described by where α r := [α r1 · · · α rn ] T and α y := [α y1 · · · α yn ] T , respectively.
Although the details are omitted to simplify the statements in this article, the following discussion is valid with slight modifications even for the case of the dynamic controller where k 0 , α r , and α y in (12) are replaced by the transfer functions k 0 (s), α y (s), and α y (s), respectively.

Problem setting
We consider that K 0 is already implemented in the control system and it stably regulates the plant system P all without {K i }. In other words, no coordination ofr is performed by {K i } and (α r , α y ) = (1, 0) holds in (12). Then, letting G 0 denote the existing baseline control system, we have the expression An assumption is imposed on G 0 .
This article aims at designing {K i } to further improve the control performance in the sense of the H 2 norm. In the design, it is assumed that K 0 is given and that Assumption 2.1 holds. For simplicity of discussion, we let α = [α 1 · · · α n ] T and holds. Letting G α denote the overall bi-layered control system, we have the expression (15) Then, the following design problem is addressed.
where G α H 2 represents the H 2 norm of α, i.e. letting z 0 be the unit impulse response of G α , it holds that

Method of parameter design
It is known that structured controller design such as Problem 2.1 is intractable even in numerical computation. In particular, as summarized in e.g. [10,11], the design problem of output feedback controllers with static gain and/or decentralized structure cannot be convex and cannot be solved efficiently. This article addresses the solution algorithm of finding a local optimum. To this end, the gradient of the cost function in (16) is studied as follows.
As a preliminary, suppose that the models of {P i } and K 0 are available for the design of parameters α. Then, an analytical expression of the gradient of J(α) can be derived and is given in the following proposition: where W and Y are the solutions of Lyapunov equations respectively, and A cl (α) is given by The proof of the proposition is omitted in this article. The proposition is shown in the same way as its more generalized result given in Theorem 3 of [12]. Based on Proposition 3.1, we can update the parameters by the following algorithm.

Algorithm 1
Step 0 Let c and λ be positive constants, and p max denote the maximum iteration limit. In addition, let p = 0 and determine the initial guess α (0) = α 0 for some constant vector α 0 .
Step 1 Based on the current guess α (p) , solve the Lyapunov equations (19) and (20) to obtain W = W (p) and Y = Y (p) .
As implied in Proposition 3.1 and Algorithm, the models of {P i } and K 0 are required for the parameter design. However, their accurate models are generally not available for controller-designers, in particular, for large-scale plant systems such as power systems. Another way to the parameter design is to combine the algorithm above with data-driven model reconstruction. In the following discussion, the method of model reconstruction based on the operating plant data is stated.
To this end, let the control input generated by (12) be modified as where u ID := [u ID1 , . . . , u IDn ] T ∈ R n denotes the measurable noise or the identification input to be injected by the controller-designer. Recall the baseline control system G 0 , where (α r , α y ) = (1, 0) and {K i } do not work. Then, we see that the state x of G 0 follows the state-space equation of the form: where A cl (·) is given by (21). It is assumed that all of {x(k), u ID (k)} are measurable. In other words, the input-state data of all of the sub-plants {P i } is available for data-driven model reconstruction. Then, the model reconstruction is reduced to estimating the coefficient matrices A cl (0) and B u in (24) based on the input-state data. LetÂ cl andB u denote the estimates of A cl (0) and B u , respectively. By gathering (24) from k = 0 to k = N−1, the following equation holds: where (26) Assuming that F(N) is of full rank, the least squares solution is given by

= X(N)F(N) T F(N)F(N)
With this , the gradient (18) is approximately calculated and is utilized for the controller update at Steps 1 and 2 of Algorithm. Finally, it should be noted that the dimension of the state space of the reconstructed model (24) is generally different from that of the actual plant. In many practical situations, some of the sub-plants are not accessible, and only a part of {x i (k), u IDi (k)}, instead of {x(k), u ID (k)}, are available for data-driven model reconstruction. Then, the state behaviour x(k) cannot be perfectly reconstructed from the data. Then, the resulting model (24) with estimatedÂ cl andB u is lower-dimensional than the plant system (5), and the modelling error inevitably exists. Algorithm combined with the low-dimensional model is verified in a demonstration given in Section 4.

Stability and safety analysis
Updating lower-layered controllers {K i } can ultimately improve the overall control performance, evaluated by the H 2 norm. However, the safety of the system is not guaranteed in the process of the update: the state may largely fluctuate and be beyond an allowable range. This section addresses additional structure imposed on the lower-layered controllers {K i } such that any safety requirement is satisfied. The discussion in this section holds even for general K i stated in (10). To this end, the saturation function is additionally introduced to each K i , described by (10), such that every u i does not largely vary from r 0 : where The behaviour of the saturation function is illustrated in Figure 3. The centre of the possible output is r 0 , which is the baseline control signal generated by K 0 , while the range is γ i , which is to be designed. One can interpret the modified control input, generated by K si , as with a bounded pseudo-disturbance d i . Due to the saturation function, it holds with this d i that This interpretation of (28) by (29) with the bounded disturbance is illustrated in Figure 4. The boundedness plays a central role in guaranteeing the stability and safety of the overall control system. This is seen as follows.
Due to the linearity assumption on {P i } and K 0 , the influences are independent each other. Let G safe denote the overall control system defined by the feedback connection of F l (P(s), L) and {K si }, and let z 0 be the output difference between G safe and the baseline system G 0 , described by (13). Then, letting d := [d 1 · · · d n ] T , by a straightforward calculation, we have the following inequality: where A cl (·) is given by (21). We now have the following two propositions: one is on the stability and the other is on the safety.

Proposition 3.2: Suppose that Assumption 2.1 holds.
Then, the overall control system G safe is BIBO stable.

Proposition 3.3: Suppose that Assumption 2.1 holds.
Then, the performance deterioration in G safe by any design of {K i (α i , r 0 , y i )} is bounded, in other words, (31) holds. Proposition 3.3 states the worst-case fluctuation in z 0 caused by the undesirable coordination of the lowerlayered controllers {K i }. The fluctuation caused during the update is "designable" by the choice of γ i . Small γ i suppresses the worst-case fluctuation and satisfies severe safety requirement, while equivalently deteriorating the achievable performance by the controller update. Conversely, large γ i sacrifices the safety, while improving the achievable performance. There is an inevitable trade-off between the safety guarantee and the performance limit.

Numerical experiment
In this section, we demonstrate the bi-layered control via a numerical experiment using a power system model. The model is originally developed by the Institute of Electrical Engineers of Japan Power and Energy [13] and consists of 107 buses, 191 branches, and 30 generators of eastern Japan. It is assumed here that renewable energy (RE) farms are connected to the generator 1 and 6, denoted by P 1 and P 6 , respectively. Then, we suppose that the active power output of the RE farm is suddenly changed and behaves as a disturbance to the power system causing its frequency deviation. We aim at suppressing the overall frequency deviation by the design of sub-controllers K 1 and K 6 utilizing the operation data on P 1 and P 6 .  To simplify the discussion, we let (α r , α y ) = (1, 0). Then, the design of K 1 and K 6 is reduced to that of α 1 and α 6 . In the experiment, the operation data is collected until the first 5 s to reconstruct the twodimensional model (24). Note here that since the IEEJ simulator model, from which the operating data is generated, is of 90-dimensional, the reconstructed model (24) is relatively low-dimensional and cannot perfectly express the simulator dynamics. Based on the reconstructed model, the parameter is updated gradually at every 0.2 s until 8.0 s.
The parameter update during the experiment is shown in Figure 5. In the figure, both α 1 and α 6 converge on the optimum solution. Figure 6 shows the result of the disturbance response. In Figure 6, the red line represents the overall frequency deviation controlled by only the existing controller K 0 , while the blue line represents the deviation controlled by the proposed bi-layered controller. Until 5 s, both lines are overlapping because the lower-layered controllers are not operating. After the lower-layered controller starts operating, the fluctuation of the blue line is suppressed compared with the red one. This means that the disturbance suppression is achieved by using the bi-layered controller.

Conclusion
This article addressed the novel control framework for large-scale systems. The bi-layered control system was proposed, where the existing reliable controller is inherited and only the lower-layered controllers are updated to improve the overall H 2 performance. The algorithm for the controller update was also presented and the class of the lower-layered controllers was extended such that any safety requirement is satisfied during the update. Finally, a numerical demonstration of the proposed control system was performed in the control problem of the power system.
There are many future works on this bi-layered control. The class of the upper-and lower-layered controllers will be extended to nonlinear and time-varying ones. It is also important to apply the bi-layered control system to other social infrastructures such as traffic systems.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Toshiki Homma received the B.E. deg rees in Applied Physics and Physicoinformatics from Keio University in 2020. His research interests include control theory for large-scale systems and applications to power systems control. Jun-ichi Imura received the M.E. degree in Applied Systems Science and the Ph.D. degree in Mechanical Engineering from Kyoto University, in 1990 and 1995, respectively. He served as a research associate in Kyoto University, from 1992 to 1996, and as an associate professor in Hiroshima University, from 1996 to 2001. Since 2001, he has been with Tokyo Institute of Technology, where he is currently a professor. His research interests include modelling, analysis, and synthesis of nonlinear systems, hybrid systems, and network systems. Dr. Imura is a member of IEEE, SICE, ISCIE, and The Robotics Society of Japan.

Masaki
Kengo Urata received the B.E. and M.E. degrees in Applied Physics and Physicoinformatics from Keio University in 2015 and 2017, respectively, and he received the Ph.D. degree in Systems and Control Engineering from Tokyo Institute of Technology. He is currently working at NTT network technology labo-ratories. His research interests include control theory for dynamical systems and applications to power system control and communication network control.