Distributed adaptive control for optimal tracking of uncertain interconnected dynamical systems

This paper presents a distributed model reference adaptive control scheme for optimal tracking of an interconnected dynamical system in the presence of system/interconnection uncertainties. A reference model selection which achieves an optimal tracking for the nominal system is introduced by using linear quadratic regulator theory. Then an adaptive control law is developed for the uncertain interconnected dynamical system, where it employs the specified reference model. It is shown that the control law achieves the desired behaviour such that the output of the system asymptotically tracks the output of the reference model in the presence of the uncertainties. An explicit error bound regarding optimal tracking is also established.


Introduction
Adaptive control is a methodology which can deal with uncertain systems because it can tolerate large parametric uncertainties to ensure the desired tracking performance. Thus it has been used in many applications [1]. The distinctive feature of adaptive control is that the knowledge of plant parameters is not required and can still achieve the boundedness of signals, system stability, and/or asymptotic output/state tracking. In particular, model reference adaptive control such that the closed loop system is matched with a reference model has been extensively studied and its theory and design techniques have been developed for decades [2].
Since several uncertainties should be treated in large-scale systems composed of several subsystems and interconnections, decentralized adaptive control was proposed in [3,4], where bounded disturbances and nonlinear uncertainties are considered. After that, such a decentralized adaptive scheme was further investigated in [5,6], where it is proved that the output of the system can asymptotically track the output of the reference model if all decentralized controllers share their prior information. On the other hand, the paper [7] presented a distributed control, where each controller which applies to each subsystem uses the information about her neighbours.
Performance guarantee is also an issue even in decentralized adaptive control. In fact, the paper [7] provided an error bound of tracking based on a Lyapunov solution of a given reference model. This technique has further been investigated in the context of several types of uncertain systems in [8][9][10]. In this regard, when we introduce a performance index and consider an optimal tracking, it is natural to explore a performance guarantee with respect to the index. However, an available result does not exist. For example, the paper [11] considered an adaptive optimal control for large-scale systems, where an asymptotic optimality was investigated, while any error bound of tracking is not provided. It is important to evaluate the performance degradation caused by adaptation in terms of the performance index of the reference model which achieves optimal tracking. Then, an appropriate selection of control input which incorporates the evaluation of performance degradation is needed so that the tracking error is minimized for the nominal system and an explicit error bound of the optimal tracking is obtained.
In this paper, we consider a class of large-scale dynamical systems with uncertain interconnection between the subsystems. We investigate a distributed control, where each controller which applies to each subsystem uses the information about her neighbours. We first introduce a performance index and consider an optimal tracking problem for each nominal subsystem.
We construct a reference model as optimal tracking for the nominal subsystem, where a Riccati solution of an LQ regulator determines the model. Then we develop a distributed adaptive tracking control law for the interconnected uncertain dynamical system, where the Riccati solution for optimal tracking is used in the update rule of the adaptive gain. We show that the proposed adaptive control law achieves the desirable optimal tracking asymptotically as well as the boundedness of all signals. We also establish an explicit error bound with respect to the nominal optimal tracking, where a role of the learning rate of the update rule is clarified. Numerical examples illustrate that the theoretical results developed in this paper are useful.
A preliminary version of this paper was presented at a conference [12], where an adaptive tracking of a single plant is considered for a step-type reference signal. On the other hand, the present paper deals with a distributed adaptive tracking of an interconnected system for a general reference signal, which clarifies a possible performance guarantee for a type of adaptive control law.
This paper is organized as follows. We state notations and definition in Section 2. We describe the statement of problem in Section 3 and select the desired reference model as optimal tracking in Section 4. We discuss the distributed adaptive control scheme for uncertain systems with interconnections in Section 5 and evaluate the performance degradation to the original optimal tracking in Section 6, Lastly, in Section 7, the effectiveness of the proposed scheme is demonstrated to show results of the theoretical findings and the concluding remarks are discussed in Section 8.

Notations and definitions
In this paper, R denotes the set of real numbers, R n denotes the set of n-dimensional real column vectors, and R n×m denotes the set of n × m real matrices. In addition, we write Q T for the transpose of a real matrix Q, R −1 for the inverse of a matrix R, rankS for the rank of a matrix S, and trU for the trace of a square matrix U.

Problem formulation
Let us consider an interconnected system consisting of N uncertain dynamical subsystems with uncertain interconnection. The topology of the interconnection is expressed by a graph G = (V, E), where V = {1, 2, . . . , N} is the set of the nodes each of which corresponds to a subsystem, and E ⊆ V × V is the set of the edges which represents the interaction among the subsystems. The set of neighbourhood of the ith subsystem is denoted by N i = {j ∈ V|(i, j) ∈ E}, where the ith subsystem is affected by the jth subsystem through uncertain interconnection if j ∈ N i . Here we assume that G is known and time invariant.
With the graph G, we describe the dynamics of the ith subsystem aṡ where x i (t) ∈ R n i is the state, u i (t) ∈ R m i is the control input restricted to the class of admissible controls consisting of measurable functions, y i (t) ∈ R m i is the controlled output. Throughout the paper, the subscripts i and j correspond to the ith and jth subsystems, respectively, i.e. i ∈ V and j ∈ N i ⊂ V. The matrices A i ∈ R n i ×n i , B i ∈ R n i ×m i , and C i ∈ R m i ×n i represent the nominal part of the subsystem, where the pair (A i , B i ) is controllable and the pair (C i , A i ) is observable. On the other hand, the matrix i ∈ R m i ×m i and the vector-valued function i,j : R n j → R m i represent the uncertain part of the subsystem as well as the uncertain interaction among the subsystems. That is, i,i (x i (t)) expresses the uncertain part of the ith subsystem itself, while i,j (x j (t)) (j = i) expresses the uncertain influence from the jth subsystem to the ith subsystem.
In this regard, we introduce the following assumption for i and i,j (x j (t)).

Assumption 3.1:
The control effectiveness i of the ith subsystem is an unknown symmetric and positive definite matrix. The state-dependent uncertainty j is an unknown weight matrix and α i,j : R n j → R s i,j is a given basis function.
For each subsystem (1), we define the corresponding nominal system aṡ That is, when i = I and i,j (x j (t)) ≡ 0 for all j ∈ N i ∪ {i}, the ith subsystem (1) takes its nominal behaviour.
In this paper, we consider a reference signal r i (t) ∈ R m i generated bẏ where x ri (t) ∈ R r i , the eigenvalues of A ri are on the imaginary axis and all distinct from one another, and the pair (C ri , A ri ) is observable. That is, x ri (t) and thus r i (t) of (3) are bounded signals which are represented as a linear combination of a constant signal and sinusoidal signals having several frequencies and phases. The boundedness of x ri (t) will be used for establishing the zero steady-state tracking error. We define the initial time t = 0 at the time when the reference signal is applied. The initial state x ri0 of (3) is arbitrary. It is known that the controlled output y i (t) of the nominal subsystem (2) can follow any reference signal for all eigenvalues λ ri of A ri . We assume this condition for all subsystems (1). The objective of this paper is to construct a distributed adaptive control law for u i (t) of the form such that the output y i (t) of the given system (1) asymptotically tracks the reference signal r i (t) of (3) in the presence of the system uncertainty described by i and (x i,j ) satisfying Assumption 3.1.
Here, x N i (t) denotes the set of x j (t) with j ∈ N i , and thus the control law (5) utilizes the knowledge of the ith subsystem itself and its surrounding neighbours only. In this sense, we call the form (5) a distributed adaptive control law. Then, we employ an optimal tracking control law for each nominal subsystem (2) and derive a distributed adaptive tracking control law for the uncertain interconnected system (1) with a performance guarantee related to the nominal optimality.

Reference model selection
In this section, we select suitable reference models for our distributed adaptive control. To this end, we consider the nominal system (2) and revisit a standard optimal tracking for the reference signal (3) [13].
Under the assumption (4), there exist a unique state x si (t) and a unique control input u si (t) described by for which the controlled output y i (t) is identical to the reference signal r i (t) [14], where L xi and L ui are defined by We denote the variations of x i (t) and u i (t) from x si (t) and u si (t) bỹ and the tracking error of the controlled output y i (t) by Using these notations with (7), we have the variation systemẋ To obtain a good transient behaviour of tracking to the reference signal r i (t), we apply linear quadratic regulator theory to the variation system (9) with the performance index where Q i ∈ R m i ×m i and R i ∈ R m i ×m i are symmetric and positive definite matrices. Then the optimal control law which minimizes J i with respect to (9) is given byũ where and P i ∈ R n i ×n i is a symmetric and positive definite solution of the Riccati equation When the control law (11) is applied to the variation system (9), the resultant closed loop system is stable, Using (6) and (8), we rewrite the control law (11) as for the nominal system (2), where That is, the optimal tracking control law for the nominal system (2) is composed of a feedback from the state x i (t) and a feedforward from the state x ri (t). The resultant control system with (2) and (15) is described bẏ When there is no uncertainty in the system (1), the optimal tracking (17) represents the best achievable behaviour of each subsystem. We therefore consider a distributed adaptive control framework for (1) which asymptotically realizes the optimal tracking (17). To this end, we emploẏ as the reference model for each subsystem, where x mi (t) ∈ R n i is the state, K i is given by (12) based on the performance index J i of (10), and H i is given by (16) with this K i .

Distributed adaptive control scheme
Let us go back to the uncertain interconnected system (1). According to the selected reference model (18), we rewrite each subsystem (1) aṡ where we define (5). In fact, from Assumption 3.1, the signal δ i (t) must be linearly parameterized by using an unknown weight W i ∈ R m i ×q i and the corresponding basis function σ i : Then we introduce a distributed adaptive control law where we define the update rule of the adaptive gain The signal x mi (t) of (21) is given by (18). The learning rate η i is a positive real number and P i is the symmetric and positive definite solution of the Riccati equation (13). We see that the control law (20) with (21) is distributed indeed. In fact, the basis is distributed. Thus, if the interconnection of the overall system (1) is sparse, we enjoy a sparse structure of the control, where the control law (20) with (21) utilizes the knowledge of the ith subsystem itself and its surrounding neighbours only. See also the numerical example in Section 7 for further details. Now, let us define the errors from the ideal case as where x mi (t) and y mi (t) are defined in (18). With (18)-(21), we havė which describes the error dynamics from the reference model (18). The next theorem presents the result of this section.

Proof: Consider a candidate of Lyapunov function
where η i and P i are taken from (21) and i of (1) satisfies Assumption 3.1, which means that P i = P T i > 0 and i = T i > 0. Thus the function V i (x ei , W ei ) is a continuously differentiable function such that V i (0, 0) = 0 and V i (x ei , W ei ) > 0 for all (x ei , W ei ) = (0, 0), which implies that V is also positive definite.
Differentiating each component V i (x ei , W ei ) of this candidate V along the trajectories of (22) and (23), we havė where we use the fact that Riccati equation (13) can be rewritten as with K i of (12). Since Q i = Q T i > 0 and R i = R T i > 0, we see thatV holds true for all t ≥ 0. Hence all of the solutions (x ei (t), W ei (t)) (i = 1, 2, . . . , N) given by (22) and (23) are bounded. Now, let us recall a standard fact which holds for all t ≥ 0. With (24), we have for all t ≥ 0. We therefore see that and x ri (t), we can see that (27) In fact, x ei (t) and W ei (t) are bounded as we have proved above, while x ri (t) of (3) is bounded as we have assumed. The boundedness of x i (t) and x N i (t) follows the boundedness of x ei (t) (i = 1, 2, . . . , N) and x mi (t) (i = 1, 2, . . . , N) of (18). Using (26), (27), and [15] with Q i = Q T i > 0, we conclude that all of the tracking errors y ei (t) (i = 1, 2, . . . , N) satisfy lim t→∞ y ei (t) = 0 for any (x ei (0), W ei (0)) (i = 1, 2, . . . , N).
Theorem 5.1 establishes the boundedness of the error signals generated by the proposed distributed adaptive control in the presence of the system/inter connection uncertainties. The theorem also shows that zero steady state tracking error is achieved by this control.

Performance evaluation
Since the distributed adaptive control given by (20) and (21) employs the optimal tracking system (18) as the reference model, one of our interests should be to evaluate the performance degradation from the optimal response.
To this end, let us rewrite the minimum value (14) of the performance index (10) for the nominal system as where K i is the optimal gain (12). For the reference model (18), this means that Referring the above, we define a performance index for the adaptive control as This index (29) is reasonable for evaluating the degradation caused by adaptation since its weight (C T i Q i C i + K T i R i K i ) coincides with that of (28). In fact, when we evaluate the tracking error from the preferable state x si (t) which achieves y i (t) = r i (t) rather than the tracking error from the reference model state x mi (t), if we introduce the performance index we immediately obtain its evaluation as by employing the triangle inequality. Notice also that the value J ei becomes 0 (i.e. the value J esi becomes J mi ) if the perfect tracking x i (t) = x mi (t) is achieved. For this index (29), we have the following result.     Proof: In the proof of Theorem 5.1, we have established (24) and (25), which implies that holds true for all t ≥ 0. Thus it turns out that which establishes the bound (30).
The upper bound given by this theorem guarantees the transient performance of the proposed adaptive control. It shows that the performance of the distributed adaptive control applied to the uncertain interconnected dynamical system captured by x ei (t) cannot  be more than the right hand side of (30) at each subsystem. In addition, if we make the learning rate η i large, the transient performance will be better, which will be confirmed in the numerical example in the following section.

Numerical example
Let us consider a mass-spring-damper system having N masses in one line shown in Figure 1. Each mass m i (1 < i < N) is possibly connected with its neighbours m i−1 and m i+1 by springs k i−1,i , k i,i+1 and dampers where q i (t) ∈ R is the position of the mass m i to be controlled, andq i (t) andq i (t) are its velocity and acceleration, respectively. The force u i (t) ∈ R is applied to the mass m i as the control input, while we assume that all physical parameters m i > 0, k i−1,i ≥ 0, k i,i+1 ≥ 0, c i−1,i ≥ 0, and c i,i+1 ≥ 0 are unknown. The cases i = 1 and i = N having one neighbour are not explicitly stated in this section, though it is clear that they can be described in a similar way.
The mass-spring-damper system (31) stated above is consistent with the system description (1). In fact, we can rewrite (31) aṡ where we define That is, all unknown parameters are included in i and i,j (x j (t)). Apparently, is observable, and the uncertainties i and i,j (x j (t)) satisfy Assumption 3.1. That is, the massspring-damper system (31) can be represented as (1), where we define N i = {i − 1, i + 1}.
For this system, we consider a sinusoidal reference signal such as sin ω i t, i.e. we define the coefficient matrices of (3) as where ω i > 0. We see that (C ri , A ri ) is observable. Also, the rank condition (4) is satisfied for the eigenvalues of A ri . Actually, we have the solutions of (7) as Regarding the performance index J i of (10) with for Q i = 1 and R i = 1, we obtain the positive definite solution of the Riccati equation (13) as Then the optimal tracking gains (12) and (16) are In this way, we can construct the reference model (18) which achieves the optimal tracking for the sinusoidal reference signal.
According to this reference model, we can rewrite the system (32) as (19), i.e.
It should be noted that its uncertainty is described as That is, due to the sparse structure of the system (31) of Figure 1, the basis x ri (t)) contains only the states of its neighbours, i.e. x i−1 (t) and x i+1 (t). Since the adaptive control law (20) and the update rule of the adaptive gain (21) have the form , these become in fact a distributed control law thanks to the sparse structure of the basis σ i (x i (t), x N i (t), x ri (t)). Now, let us consider a numerical simulation. We investigate the case N = 3, where we set the unknown and uncertain parameters as m 1 = m 2 = m 3 = 3, k 1,2 = k 2,3 = 2, and c 1,2 = c 2,3 = 1. We chose all initial states are zero except for the reference signal generators, where we used x r1 = x r2 = x r3 = 0 1 T . We set Figures 2-7 show the tracking responses and the gain behaviours of the proposed distributed adaptive control for subsystems 1, 2 and 3, respectively, where we set the learning rates as η 1 = η 2 = η 3 = 50. On the other hand, Figures 8-13 show the tracking responses and the gain behaviours of the proposed adaptive control, where we use η 1 = η 2 = η 3 = 15. Note that y 1 (t), y 2 (t), y 3 (t) are indicated as solid lines and y m1 (t), y m2 (t), y m3 (t) are indicated as dashed lines in Figures 2, 4, 6, 8, 10, and 12. The elements of the adaptive gainsŴ 1 (t),Ŵ 2 (t) andŴ 3 (t) for subsystems 1, 2 and 3 are indicated as solid lines in Figures 3,5,7,9,11,and 13. In Figures 2-7, all signals are bounded and y i (t) tends to y mi (t) (i = 1, 2, 3) as t tends to infinity, which       Figures 8, 10, and 12, we see that a larger learning rate η i gives a better performance, which is consistent with Theorem 6.1.

Conclusion
In this paper, we have investigated a distributed adaptive control such that the output of an uncertain interconnected dynamical system asymptotically tracks the output of a reference model in the presence of system/interconnection uncertainties. We have constructed the reference model as optimal tracking for the nominal system, where a Riccati solution of an LQ regulator determines the model. We have employed this Riccati solution in the update rule of the adaptive gain and have shown that the proposed adaptive control law actually achieves the desirable tracking as well as the boundedness of all signals. We have also established an explicit error bound regarding optimal tracking. The numerical examples have shown that the theoretical results developed in this paper are useful. Although the state feedback case under no disturbance has been investigated in this paper, the extension to the output feedback case in the presence of disturbance is an important future work. In this regard, a preliminary result [16] has been obtained, where L 2 disturbance is considered for a single plant in an H ∞ tracking setting.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This work was supported by JSPS KAKENHI Grant Number JP20K04547.

Notes on contributors
Atsede G. Gebremedhin is a doctoral student at the Department of Information and Physical Sciences of Osaka University in Japan. She received her B.S. degree in Electrical and Computer Engineering from Jimma University, Ethiopia, and her M.S. degree in Control and Instrumentation Engineering from Mekelle University, Ethiopia. Before studying at Osaka University, she was a lecturer at Mekelle University, Ethiopia.