Adaptive output feedback control with cerebellar model articulation controller-based adaptive PFC and feedforward input

This paper deals with a design scheme of adaptive output feedback control system with an adaptive parallel feedforward compensator (PFC) and an adaptive feedforward input based on the cerebellar model articulation controller (CMAC). It is shown that, by applying the CMAC strategy to PFC design, we can obtain a PFC, which makes the resulting augmented system almost strictly positive real approximately, for a nonlinear system. Moreover, an adaptive feedforward input to attain output tracking for the practical output is also provided via CMAC. The stability of the obtained control system will be analysed and the convergence of the mean value of tracking error to a small range will be shown. In addition, the effectiveness of the proposed method will be validated through numerical simulations.


Introduction
Adaptive output feedback controls for linear systems based on the almost strictly positive real (or ASPR) property of the controlled system have been well recognized as a simple and robust control strategy with respect to disturbances and uncertainties of the systems [1,2]. The method can guarantee the stability of the adaptive control system via ASPR-ness of the considered system. A linear system is said to be ASPR if there exists a static output feedback such that the resulting closed-loop system is strictly positive real (SPR). This property is called output feedback exponentially passive (or OFEP) for nonlinear systems. The conditions for the system to be ASPR (or OFEP) have been clarified such as (1) the system has a relative degree of 1, (2) the zero dynamics of the system is stable, i.e. the system is globally minimum-phase, and (3) high-frequency gain of the system is positive [1][2][3][4]. Unfortunately, however, since most of the practical systems do not satisfy all of ASPR (or OFEP) conditions, the conditions impose a severe restriction on the practical application of the ASPR (or OFEP)-based adaptive controls. To overcome this problem, the introduction of a parallel feedforward compensator (PFC) has been proposed [5]. The introduction of a PFC in parallel with the non-ASPR controlled system makes it possible to redesign the relative degree and zeros of the resulting augmented system. Therefore, the conditions for designing the adaptive output feedback control system can be fulfilled by introducing the PFC accordingly so as to render the resulting augmented system ASPR. Thus, the ASPR-based adaptive control strategy can be applied to the ASPR augmented system. As for the design strategy of the PFC which renders the resulting augmented system ASPR, several methods have been also proposed [6][7][8]. However, most of these methods are kinds of model-based design scheme and a static PFC is obtained based on a priori information of the given system model, and these are just for linear systems. In the case where the system is changing slowly by secular change and/or the case where the system has nonlinearities, it might be difficult to design the PFC that maintains the ASPR-ness of the augmented system. With this in mind, adaptive types of PFC design strategies have been also investigated [9,10]. However, most of these methods were just for linear systems too. Moreover, designing the control system for the augmented system with the PFC, one cannot attain the control objective exactly because of the affect from the PFC output. This is an additional issue to practical application of the ASPR based control strategy, and several control system design strategies including augmented model strategy [2,6], two-degree-of-freedom design strategy with adaptive feedforward input [4,11,12] have been proposed to solve the problem.
With these issues in mind, an adaptive PFC design strategy via the cerebellar model articulation controller (CMAC) has been proposed [13] in order to make it possible to maintain the ASPR-ness (or OFEP property) of the augmented system for nonlinear systems in a considered operating range. Moreover, a feedforward input to attain output tracking for the practical output has been also designed adaptively via CMAC in [13]. The CMAC is a kind of multilayer neural network, and it has low calculation cost and robustness for nonlinearities [14,15]. This method has been also applied to adaptive type controller design for nonlinear systems. In [16], an intelligent PID controller is proposed based on CMAC strategy, and CMAC based adaptive PFC design scheme has also been proposed in [17]. However, no stability analysis for the obtained control system had been provided in these methods. In [18], a feedforward controller is designed via CMAC based on the feedback error learning (FEL) strategy. In this method, although a PFC is introduced in order to make a stable control system by the ASPR-ness of the system, the PFC that maintains the ASPR-ness of the augmented system should be known. The method proposed in [13] is a new adaptive design method of PFC and feedforward input for output regulation based on the CMAC strategy, and stable control system design method has been provided. However, the results in [13] only showed the boundedness of all the signals in the resulting control system.
In this paper, we expand the analysis of the resulting control system and provide the detailed stability analysis of the control system. It is shown and clarified that the mean value of the output tracking error converges to a small range by setting appropriate design parameters in the adaptive algorithms according to the convergence of the augmented output tracking error. By applying CMAC to the design of PFC, one can obtain a PFC, which makes the ASPR augmented system approximately, for nonlinear systems. Furthermore, a CMAC-based feedforward input can also attain the output tracking of nonlinear system easily. Finally, the effectiveness of the proposed method will be validated through numerical simulations.

Problem statement
Consider the following nth order nonlinear stable controlled system: where x(t) ∈ R n is a state vector, u(t) ∈ R is a control input and y(t) ∈ R is an output of the system. f (x), g(x): R n → R n , h(x): R n → R are smooth nonlinear functions with f (0) = 0, h(0) = 0. For this controlled system (1), let us consider an ideal ASPR linear model G * a (s) with the state space model oḟ with the appropriate system parameters of A * a ∈ R n a ×n a , b * a , c * a ∈ R n a . Here we aim to realize a PFC with which the resulting augmented output y au (t) corresponds to the output y * au (t) of the ideal augmented system G * a (s). Suppose that the desired PFC, which renders the ideal augmented system's output: is given by the form oḟ We try to realize this PFC adaptively via the CMAC technique later, and for the resulting ASPR (or OFEP) system, the objective in this paper is to design an adaptive output tracking control system in which the output y(t) of the system (1) is required to track the reference signal r(t) generated by the following exosystem: A feedforward control input (FF control input), which attains desired output tracking, is also realized adaptively using the CMAC technique later under the following assumptions on the reference signal: Assumption 2.1: The exosystem (5) is neutral stable.

Assumption 2.2:
The reference signal r(t) and the derivative of the reference signalṙ(t) are bounded.

Ideal PFC for a linear system
Firstly, we show the parametric representation of the ideal PFC for a linear system [10]. Express the ideal ASPR augmented output of G * a (s) given in (2) as where the notation y(t) = W(s)[u(t)] denotes the output y(t) of a system W(s) with an input u(t). Now, the ideal PFC output y * fu (t) can be obtained by utilizing the output y(t) of the system as follows: Moreover, in the case where the considered system is a linear system, the ideal PFC can be given by an n f th and thus the ideal PFC output is expressed by (see Figure 1) Considering an n f th order stable filter: the ideal PFC output can be represented from (8), (9) by and , This is a parametric representation of the ideal PFC for a linear system [10].

Linear approximation of ideal PFC for a nonlinear system
Let's consider an ideal PFC for a nonlinear system. Suppose that the ideal PFC output y * * fu (t) exists for a state on which z * u (t) ∈ z ⊂ R 2n f . Moreover, suppose that there exist subsets z,j z (j z = 1, . . . , N z ) such as z,1 ∪ z,2 ∪ · · · ∪ z,N z ⊂ z (14) satisfying the following assumption (See Figure 2): Assumption 3.1: For z * u (t) ∈ z,j z , there exists an optimal parameter vector ρ * j z : such that the ideal PFC output can be obtained by That is, there exist compact subsets z,j z in which the ideal PFC output y * * fu (t) can be approximated adequately by a linear approximation.
In practical cases, since optimal parameter vectors ρ * j z are unknown, we consider adaptively adjusting these ρ * j z via CMAC later.

Ideal feedforward input
In order to achieve output tracking, we consider constructing two-degree-of-freedom control system for the ASPR augmented system. The existence of the ideal feedforward control input v * (t) to achieve perfect output tracing y(t) ≡ r(t) is guaranteed under Assumption 2.1. It is well recognized that, under Assumption 2.1 for the exosystem generating the reference signal, the ideal state and ideal input, which achieve perfect output tracking, exist and can be represented by as functions of ω with mild conditions on the controlled system [19]. Suppose that for the considered controlled system, there exists an ideal feedforward (FF) control input v satisfying the following assumption: with a bounded error That is, there exist compact subsets v,j v in which the ideal FF control input v * (t) can be approximated adequately by a constant value.
We also consider adaptively adjusting this FF control input v * j v via CMAC later.

Ideal control system
Consider an ideal PFC output y * * fu (t) with an input u(t). From Assumption 3.1, it can be expressed by using ideal parameter ρ * j z as wherez * The ideal PFC output y * * fv (t) with an input v(t) can be also expressed by Moreover, the ideal PFC output y * * fu e (t) in Figure 3 can be approximated by using the ideal linear approximated PFC output y * fu e ,j z (t) in z,j z with ideal parameter ρ * j z defined in Assumption 3.1 as follows: Therefore the ideal augmented system's output y * where y v * (t) is the virtual output of the system with the and then ε ini (t) is an error caused by the initial value.
Therefore, we have the following ideal error system: where (27) can be represented as whereε Consider the control input by In (26), and thus we have there exists an ideal feedback gain k * such that the resulting closed-loop system is stable with u e (t) = −k * e * a (t). Unfortunately, however, v * (t) and k * are unknown in practice, in the following section, we consider adjusting these parameters adaptively.

Parameter adjustment by cerebellar model articulation controller (CMAC)
CMAC is well known as a type of neural network based on a mathematical model of the mammalian cerebellum. In the CMAC, inputs to the input space are transformed into the label set, and it outputs the average value of the weights by referring the weights in the activated cells with a distributed shared memory structure based on the input label. Since the CMAC is trained based on variables in the specified area, it is simple and might achieve quick learning and adjusting the weights in the activated cells through the error observed at the output. We utilize this CMAC strategy for adjusting parameters for the linearly approximated system in a specific domain. Thus, for a system satisfying Assumptions 3.1 and 3.2, in each subsets z,j z and v,j v , we adjust parameters ρ j z of PFC and v j v of FF control input via CMAC strategy.

Input space and weight tables
For simplicity, consider one input case. In this case, the input space of the CMAC is labelled as {L 1 , L 2 , . . . , L P }, and then prepare C weight tables W l , l = 1, 2, . . . , C. Each weight table W l is divided into N blocks D l,j , j = 1, 2, . . . , N and the blocks D l,j are coupled by labels in the input space (see Figure 4). Concerning the set of the block D l,j , we impose the following assumptions: Assumption 4.1: The domain of sum of sets D j of jth blocks D l,j on each weight table W l are included in the set of z,j z (or v,j v ) (see Figure 4). That is, 1, 2, . . . , N v ).

Parameter adjusting -Output of CMAC-
The adjusting parameters ρ(t) of ρ * for PFC and v(t) for FF control input are given as follows as the output of the CMAC: where ρ l z ,j hz (t) and v l v ,j hv (t) are adjusted parameters in l v , respectively, corresponding to the label L h in the input space, and ρ h z (t) and v h v (t) are the output of CMAC corresponding to the label L h in the input space (See Figure 4).

Update of the weight table of CMAC
The parameters ρ l z ,j hz (t) and v l v ,j hv (t) in the weight table W l corresponding to the label L h in the input space are updated adaptively by the following adjusting laws: is defined later as in (42) and (43). The remaining parameters in the weight table, which do not correspond to the label L h in the input space, do not update. That is, for j = j ḣ

Adaptive PFC
Using the adjusted parameter ρ(t) = ρ h z (t) and the adaptive FF control input v(t) = v h v (t) given in (33), (34) through CMAC, define and define the following signal: where The PFC output y fu e (t) is given by

Adaptive controller design
The adaptive control system is designed as follows (See Figure 5): where k(t) is an adaptive feedback gain adjusted bẏ

Error system representation
Applying the control input designed in (45), the error system is expressed from (26) as Since it follows from (22), (44) that with ρ h z (t) = ρ h z (t) − ρ * j z , the error system can be represented by where

Boundedness of the control system
Since G * a (s) is ASPR, the obtained error system can be represented by (53) where A η is a stable matrix, and thus there exist symmetric positive definite matrices P η and Q η such that Now, consider the following positive definite function: In the case of |e a (t)| > ε e a , we design feedback input from (46) as follows: And then, taking the time derivative of V 1 (t) for the case in z * u (t) ∈ z,j z and v * (t) ∈ v,j v for any j z and j v , we havė Taking the time derivative of V 2 (t) to V 5 (t) and consid-eringV 1 (t) given in (57), we havė On the other hand, we design feedback input from (46) in the case of |e a (t)| ≤ ε e a as follows: Taking the time derivative of V 1 (t), Similarly, given that the time derivative of V 2 (t) to V 5 (t), we haveV(t) aṡ Consequently, considering a sufficiently large k * and constants δ 1 , δ 2 , δ 3 , δ 4 such as where Thus, we can conclude that all the signals in the obtained control system are bounded.

Convergence of the actual error e(t)
We investigate the convergence range of the mean value of the tracking error |e(t)|. Now, setting δ 3 and δ 4 as follows: then we obtain We have from (65) thaṫ Integrating (69) from 0 to T, we have and then we obtain Finally, from the fact that V(t) is bounded, it follows that as T → ∞. Thus, the mean square value of |e a (t)| can be evaluated as Similarly, we can evaluate the mean square of k(t) as follow: Thus, we have from Schwarz's inequality that and Moreover, we have from (75) and (76) that Finally, considering the case where the design parame- we can obtain that Thus, the mean value of |e a (t)| and |k(t)e a (t)| converge to a small range by considering the sufficiently large k * and setting the design parameters to satisfy (78). Consequently, the mean value of the input u e (t) for the PFC designed in (46) also converges to a small range and it leads that the mean value of the PFC's output |y fu e (t)| also converges to a small range. Thus, we can conclude that the mean value of the tracking error |e(t)| also converges to a small range by setting appropriate design parameters.
Theorem 6.2: By setting the design parameters in controller (33), (34) and (45) to (49) adequately, that is, set the parameters so as to satisfy for sufficiently large ideal feedback gain k * which satisfies Theorem 6.1, the mean value of the error |e a (t)| and the mean value of |k(t)e a (t)| converge to a small range such as provided that ρ ε is set so as to satisfy ε * < ρ ε , where Thus, the mean value of the actual tracking error |e(t)| of the controlled system converges to a small range too.

Validation through numerical simulations
The effectiveness of the proposed method is confirmed through numerical simulations for a Hammerstein system. The considered Hammerstein system in this simulation is given by The static characteristic of the system is shown as in Figure 6.
In order to design an adaptive control system for this system, we set the following ideal ASPR model and stable filter: For designing CMAC, we first determined the number of partition of the input space as P z = P v = 28, that is, Figure 6. Static characteristic of the Hammerstein system. the input space was divided into 28 and was labelled by {L 1 , L 2 , . . . , L 28 }. We also determine the number of blocks in the weight table so as to satisfy Assumption 4.1. For the considered system with a static characteristic shown in Figure 6, we considered dividing the weight table into around 10 blocks such as N z = N v = 10. The number of weight table C was determined according to the following relation provided in [18]: where x := min{n ∈ Z | x ≤ n} denotes ceiling function. For P z = P v = 28 and N z = N v = 10, C z and C v could be determined as , the label L h of input space was distributed as follows: • For the first block with j = 1, the label satisfying 1 ≤ L h ≤ C − l + 1 is distributed in the block D l,1 . • For the block with j ≥ 2, the label satisfying

Remark 7.2:
In the simulation, we set design parameters γ k , σ k , ρ z , ρ ε , ε e a , H , σ H , γ v , and σ v so as to satisfy the given conditions in Theorem 6.2 with k * = 10. k * = 10 was supposed to be a value to satisfy the condition in (64). This value can be roughly determined from the ideal ASPR augmented system G * a (s). Moreover, it is valuable to note that setting a large γ k and small σ k leads to a small range of convergence of output error due to resulting high gain feedback control system. It may also be noted that, in a practical situation with some output noise, one needs to design σ k lager, and of course, we will have bigger output error depending on the magnitude of noise in this case.
The simulation results are shown in Figures 7-11. Figure 7 shows a result by the proposed method via online adaptation and learning with initial conditions of ρ l z ,j z (0) = 0 and v l v ,j v (0) = 0 in CMAC weight tables. By setting appropriate design parameters, a good control performance and accurate output tracking were obtained even when the reference signal was widely moving.   Figure 9 shows the result with fixed controller parameters in CMAC after pre-learning the parameters in CMAC through a random step-type reference signal as shown in Figure 8. An acceptable control result is obtained; however, the tracking accuracy degrades at the set points near r(t) = 10 since the reference signals for learning did not include the signal around r(t) = 10.
The result shown in Figure 10 is the one by online adaptation and learning with the initial parameters, which were pre-learned through the step-type reference signal given in Figure 8. The control results are improved without oscillating around set points of r(t) = 10 compared with the results shown in Figure 9. Having the learned parameters as the initial values and implement the online adaptation and learning, a better result is expected.
Next, we try to get parameters in CMAC by learning more for 1800 s with an exhaustive step-type reference signal ranging from r(t) = 0 to r(t) = 10. The control result is shown in Figure 11. It is clear that the result is improved compared with Figure 9. Thus, if the training range covers the range of operation, we can obtain adequate CMAC parameters by appropriate training even if the learning was done through some specified signals as a step-type reference signal.
Finally, from these obtained simulation results, we can confirm that the online adaptation via CMAC allows us to apply the ASPR based adaptive control for linear system to nonlinear systems with the appropriate division of operation range. Moreover, off-line use of CMAC after adequate training is also useful, but in the case where the operating range will be changed, the online learning can improve the control performance effectively.

Remark 7.3:
In the simulations, the pulsive signal appears in the input signal. This phenomenon is occurred due to the discontinuous derivative of the reference signal r. For the ASPR-based adaptive output feedback control, this phenomenon occurs frequently when the reference signal is suddenly changed. Fortunately, however, it has been confirmed through many   practical applications that this kind pulsive signal may cause no trouble to practical applications.

Conclusions
In this paper, an adaptive PFC design strategy via the cerebellar model articulation controller (CMAC) was proposed in order to make it possible to maintain the ASPR-ness (or OFEP property) of the augmented system in a considered operating range for a nonlinear system. A PFC for a nonlinear system, which makes the ASPR augmented system approximately, was obtained adaptively by applying the CMAC. Moreover, an adaptive feedforward input to attain output tracking for the practical output was also proposed via CMAC strategy. The stability of the obtained control system was analysed and the effectiveness of the proposed method was validated through numerical simulations for a Hammerstein system.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This work was supported by JSPS [grant number JP 21K04126].