Direct data-driven control approach of reference shaping for two degree of freedom control

In this study, we propose a novel data-driven approach for control of two degrees of freedom systems. The proposed method uses one-shot experimental data to derive the reference governor for improving control performance. The reference governor consists of feedback-feedforward controller, desired transfer function, and virtual feedforward controller. Virtual feedforward controller can be obtained using data-driven control approach such as ERIT, FRIT, and VRFT. Since a model is not required in our method, the cost related to modelling can be reduced. Numerical simulations were used to validate the proposed method. Verification results demonstrate that the desired control response was achieved by reference shaping alone, with the feedback and feedforward controllers not requiring any updates. The proposed method can realize the desired control performance in cases where the plant is non-minimum-phase system.


Introduction
Feedback control is widely used in the control systems industry due to the ease with which a desired response can be achieved by adequately tuning linear feedback gains. Various auto-tuning methods, such as the Ziegler-Nichols method, the Chien-Hrones-Reswick method, and optimization based on a cost function [1], have been proposed. The gain may be automatically derived by applying conventional auto-tuning methods to control system designs. However, appropriate controller parameter-tuning methods need to be determined by experimental trial and error. Therefore, simple auto-tuning methods are desirable in linear feedback controllers. In data-driven control approaches, iterative feedback tuning [2], virtual reference feedback tuning (VRFT) [3], and fictitious reference iterative tuning (FRIT) [4] have been proposed to solve this problem. Linear feedback gains can be tuned using experimental data. The FRIT and VRFT methods can tune the feedback controller using the initial closedloop/open-loop experiments to realize the desired control response. These methods are simple processes because the control system can be easily designed without the need for modelling.
The data-driven control approach involves, as the name suggests, a data-driven attitude to reference shaping, explicitly based on the abovementioned studies [5][6][7]. Takahashi and Kaneko proposed a data-driven approach to reference shaping for a control system with a single degree of freedom [5]. This method uses the data-driven prediction results of the closed-loop system to design the feedforward controller for reference shaping. The desired closed-loop response can be realized by the feedforward controller, even if the tuning results of the feedback controller are not inadequate. However, the conventional method can only be applied when the closed-loop system itself can be updated. There are several cases where it is impossible to reprogram the implemented controllers -for instance, in cases where the controller has already been embedded in the equipment, it cannot be reconfigured even if a more desirable controller is later designed. As another example, if the process plant is already up-and-running, it is not preferable to stop its operation to reprogram the implemented controller with a more desirable controller; in such cases, the preference is to achieve the desired performance without halting the ongoing operations. Kuwabara et al. proposed a new data-driven approach that does not require tuning the feedback controller of closed-loop systems [6,7] -the reference governor, in such cases, can realize the desired control responses without tuning the feedback controller of the closed-loop system. Therefore, data-driven reference shaping can realize the desired control response without terminating the operation of control systems. This approach is inexpensive for controller design. However, the conventional method cannot be applied to two-degree-of-freedom (2DOF) control. From a practical perspective, it is immensely relevant to provide data-driven reference shaping for 2DOF controls since they are widely used in the control industry. In the SICE Annual Conference 2021 [8], we proposed a new data-driven reference shaping method for a 2DOF control system that was explicitly based on the abovementioned studies. The method proposed in that study can achieve the desired control responses without tuning the feedback and feedforward controllers. However, it cannot be used when the plant is a non-minimumphase system.
This study is an improvement on the work mentioned above [8]; here, we have expanded the proposed method to include non-minimum-phase systems. The primary objective of this study was to design a reference governor for 2DOF control that does not require updating closed-loop systems. Our method was validated using numerical simulations. The remainder of this paper is organized as follows. The problem formulation is presented in Section 2, with the fundamentals of reference shaping described in Section 3. The proposed method for the configuration of non-minimum-phase systems is presented in Section 4. Numerical simulations of minimum and non-minimum phase systems are presented in Sections 6 and 7, and the conclusions of this study are presented in Section 8.
Herein, we have omitted the variable s from the rational functions and time-series -e.g. we use G instead of G(s) and y instead of y(s) when it is clear that these rational functions and time series are Laplace transforms.

Problem formulation
In this study, we investigated a control system wherein the feedback controller, feedforward controller, and control object are linear time-invariant systems. A block diagram of the control system is shown in Figure 1. In this system, P, C FB , and C FF0 are assumed to satisfy the following assumptions: Assumption 1. The control object P is an unknown, single-input-single-output (SISO), linear time-invariant system. Assumption 2. The control system is stabilized by the feedback controller C FB .
Assumption 3. The orders of the numerator and denominator of the control object P is known.
Assumption 4. The initial-closed loop experiment data had been measured.
In Figure 1, r, u, and y are the reference, control, and output signals, respectively. We denote the desired transfer function, T d , from r to y. Here, T d is given as a minimum phase system. In this problem, the feedback and feedforward controllers C FB and C FF0 cannot, by themselves, realize y = T d r. Therefore, they must be tuned such that the closed-loop systems from r to y are similar to the desired transfer function T d . However, they cannot be redesigned because the control system is in operation. Hence, we aim to achieve the desired control performance by shaping the reference signal in such a way that y is tracked to T d r as much as possible. However, y cannot be tracked to T d r if P is a non-minimum phase system. Therefore, the new desired transfer function for non-minimum-phase systems should be estimated.
This problem is defined as follows.
Problem Statement: Consider the closed-loop system shown in Figure 1, which satisfies assumptions 1-4. For the desired transfer function T d and reference signal r, we identify the shaped reference signal r * such that y * , defined as depicted in Equation (1), is similar to T d r. If model P is known, then r * can be calculated as However, the reference governors of Equation (2) cannot be derived because P is unknown. Furthermore, the new desired transfer function for non-minimum-phase systems should be estimated from initial closed-loop data if P is a non-minimum phase system.

Fundamentals of data-driven reference shaping for 2DOF control
In this section, we explain the concepts underlying the data-driven approach to reference shaping [8], which forms the basis for the work carried out in this study.

Design reference governor
A block diagram of the reference governor is shown in Figure 2. The initial output signal y ini was measured in the initial closed-loop experiments and C FF was incorporated as a virtual feedforward controller using data-driven tuning. For example, a virtual feedforward controller can be designed using estimated response iterative tuning (ERIT) [9]. ERIT updates the feedforward controller such that the closed-loop transfer function is similar to T d r. Of course, we can utilize VRFT [10] or FRIT [11] for 2DOF control. The virtual feedforward controller C FF is not implemented in closed-loop systems because this tuning is hypothetical. The reference governor can be designed such that: Therefore, the shaped reference signals, r * , are calculated as By inputting r * from Equation (4) into the closed-loop system, the desired control performance can be realized without updating the feedback and feedforward controllers. The procedure for the proposed method is as follows: (1) The initial dataset of y ini was obtained from the closed-loop experiments in the initial control systems, as shown in Figure 1.

Analysis of the reference governor
In this subsection, we analyze the reference governor.
If the cost function of data-driven tunings is extremely low, the desired transfer function can be expressed as follows: Substituting Equations (4) and (5) in Equation (1), the closed-loop response is derived as follows: The reference governor can cancel the numerator exactly in the third line of Equation (6) since C FB , T d , and C FF0 in the closed-loop system are known.
From this result, the desired control performance can be realized by inputting r * to the closed-loop systems. Next, we analyze the stability of the reference governor for the case in which Equation (6) is achieved. From Equations (5) and (6), the reference governor can be expressed as: Since the reference governor includes an inverse model of P, it has an unstable pole when P has an unstable zero point. Therefore, the proposed method cannot realize the desired control responses when P is a non-minimum phase system.

Remarks
When the control signal is saturated, the proposed method cannot realize the desired control responses because Equation (1) is not satisfied. If Equation (1) is not satisfied, the output is not tracked to T dr since the numerator P(C FB T d + C FF0 ) and denominator C FB T d + C FF0 are not cancelled in the third line of Equation (6). Therefore, data-driven reference shaping cannot be performed while considering the input saturation of closed-loop systems. This is a common problem in conventional methods and in exploratory methods proposed in other works. Furthermore, the presence of significant levels of noise is another scenario in which the proposed method cannot realize the desired control responses, because Equation (1) would not be satisfied then. This can be overcome by adopting a controller design strategy, such as the use of a low-pass filter, for noise elimination. A more detailed exploration of noise effects should be made in future studies. The advantage of the proposed method compared to the simple solution is explained as follows. When feedback and feedforward controller were embedded, a simple solution would be to construct an outer-loop controller such as cascade control system. However, the outer-loop controller of cascade system must be designed such that the cascade control system is stabilized. It is difficult to stabilize the control system if P is an unknown system. The proposed method can stabilize the control systems if the reference governor F become stable. Therefore, we can easily stabilize the control system by using the proposed method if the initial closed-loop system is stable. Another simple solution is to identify the closed-loop system and use its inverse as F. Compared to the reference governor based on the inverse-closed loop model, the advantage of data-driven approach is in the processing cost [6]. Tunable parameter number of the proposed reference governor F is extremely low because the orders of the denominator/numerator of the inverse closedloop model are higher than those of virtual controllers. Therefore, the processing cost for designing the reference governor is shorter than that of the conventional method.

Extension to non-minimum-phase systems
In this section, we expand the data-driven approach of 2DOF control to non-minimum-phase systems.

Design reference governor for non-minimum-phase systems
The initial output signal, y ini , was measured in the initial closed-loop experiment. The data-driven tuning of non-minimum-phase systems was used to derive C FF as a virtual feedforward controller. In a non-minimumphase system, the nominal model P m can be expressed as: where G n and G m are transfer functions of the nonminimum-phase and minimum-phase systems and ρ is a tunable parameter. The feedforward controller has an unstable pole when it is set as C FF (ρ) = T d P m −1 . Therefore, the desired transfer function should include the non-minimum-phase properties of the plant [12,13]. The new desired transfer function for non-minimum-phase systems, T n , is replaced as follows: To prevent system instability, the virtual feedforward controller for non-minimum-phase systems is represented as follows: In this study, the virtual feedforward controller C FF was not implemented in closed-loop systems because this tuning is hypothetical. For example, the new desired transfer function and virtual feedforward controller can be designed using ERIT of the non-minimum-phase system [12], where the cost function of ERIT determines the relative error between P and P m . Therefore, C FF and T n can be updated by minimizing the cost function of the ERIT. We can also achieve this by applying FRIT to the non-minimum phase [13]. The reference governor can be designed such that By inputting r * based on Equation (12) into the closedloop system, the desired control performance can be realized without updating the feedback and feedforward controllers. The procedure for the proposed method is as follows: (1) An initial dataset of y ini was obtained from the initial closed-loop experiments in the initial control systems, as shown in Figure 1.
(2) C FF and T n are derived from the data-driven tunings of the non-minimum-phase systems. (3) The reference governor is designed using Equation (12).

Analysis of the reference governor
In this subsection, we analyze the reference governor.
If the cost function of data-driven tunings is extremely low, the virtual feedforward controller can be expressed as follows: Substituting Equation (4), (12) and (13) into Equation (1), the closed-loop response is derived as follows: From this result, the desired control performance can be realized by inputting r * to the closed-loop systems. Next, we analyze the stability of the reference governor for the case in which Equation (14) is achieved. From Equation (14), the reference governor can be expressed as: If the cost function of data-driven tunings is extremely low, G n would be equal to the non-minimum-phase transfer function of P. Therefore, the reference governor would be stable because the unstable zero point of P would stand cancelled. The proposed method can realize the desired control responses when P is a nonminimum-phase system.

Numerical simulation of minimum-phase systems
The method proposed in this study was validated through numerical simulations of the minimum-phase systems, implemented on MATLAB software. The sample time was set as 0.01 s.

Simulation conditions
In the simulation, plant P was set as a second-order system and its transfer function was expressed as: Furthermore, the desired transfer function is expressed as: The initial nominal model, P m0 , is expressed as: Based on the nominal model and desired transfer function, the initial feedforward controller is derived as follows: The feedback controller is expressed as: This ensures the stability of the closed-loop system.

Simulation results
Results of the initial simulation are presented in Figure 3, in which the solid and dash-dotted lines represent the simulation data and reference signals, respectively. The output and control signals were observed to vibrate. The output signal could not be tracked to the desired control response because the nominal model was not equivalent to the control object. Since the feedback and feedforward controllers could not be updated, the desired control performance must be realized by shaping the reference signal. The reference governor was derived using the initial closed-loop response data. Here, C FF is expressed as: where ρ = [ρ 1 , ρ 2 , ρ 3 ] are tunable parameters and the cost function was minimized using a particle-swarm optimization (PSO) algorithm [14]. The parameters of the virtual feedforward controller were tuned to ρ = [1.501, 0.2, 0.3501]. The control responses and r * are shown in Figure 4 and (c), respectively. The shaped reference signal was observed to oscillate. Although the feedback and feedforward controllers were not updated, the output signal could be tracked to the desired control response. This result indicates a similarity between the closed-loop response and the desired closed-loop transfer function.

Numerical simulation of non-minimum-phase system
The method proposed in this study was validated through numerical simulations of the non-minimumphase systems, implemented on MATLAB software. The sample time was set as 0.01 s. Here, the method was applied to the benchmark systems of a four-wheeled vehicle [15].

Simulation conditions
In the simulation, the transfer function of the control object P was set as follows: Furthermore, the desired transfer function, T d , was expressed as: The initial nominal model, P m0 , was expressed as: Based on the nominal model and desired transfer function, the initial feedforward controller was derived as follows: = − (s 3 + 50s 2 + 425s + 1900) (s 4 + 85s 3 + 2760s 2 + 40, 400s + 224, 000) The feedback controller is expressed as: ensuring the stability of the closed-loop system.

Simulation results
Results of the initial simulation are presented in Figure 5, in which the solid and dash-dotted lines represent the simulation data and reference signals, respectively. The output signal could not be tracked to the desired control response because the nominal model was not equivalent to the control object. Since the feedback and feedforward controllers could not be updated, the desired control performance must be realized by shaping the reference signal. The reference governor was derived using the initial closed-loop response data. Here, G m and G n are expressed as: where ρ = [ρ m1 , ρ m2 , ρ m3 , ρ m4 , ρ m5 , ρ m6 , ρ n1 ] are tunable parameters, and the ERIT cost function was minimized using the PSO algorithm. The parameters of the virtual feedforward controller were tuned to ρ = [−394, −1.79 × 10 4 , −2.24 × 10 5 , 39.0, 369, 1850, 23.3]. The control responses and r * are shown in Figure 6 and (c), respectively, and the desired response is updated for non-minimum phase systems. Although the feedback and feedforward controllers were not updated, the output signal could be tracked to the desired control response. This result indicates a similarity between the closed-loop response and the desired closed-loop transfer function.

Simulation example 2: unstable zero is 0.1
In the previous simulation, the unstable zero of the control object P was 23.4. In general, the control system design is difficult if unstable zero is closer to an imaginary axis. Therefore, we should verify the validity of the proposed method for such a case as well. Therefore, the transfer function of the control object P was changed as   The unstable zero of P is 0.1. The desired transfer function and the feedback-feedforward controller are set as Equations (24)-(27). Results of the initial simulation are presented in Figure 7, in which the solid and dash-dotted lines represent the simulation data and reference signals, respectively. The output signal could not be tracked to the desired control response because the nominal model was not equivalent to the control object.
Since the feedback and feedforward controllers could not be updated, the desired control performance must be realized by shaping the reference signal.
Here, G m and G n are expressed as Equations (28)   systems. Although the feedback and feedforward controllers were not updated, the output signal could be tracked to the desired control response. This result indicates a similarity between the closed-loop response and the desired closed-loop transfer function.

Conclusion
In this study, a data-driven approach to 2DOF control was proposed. This approach was particularly expanded to non-minimum-phase systems. The reference governor consists of feedback-feedforward controller, desired transfer function, and virtual feedforward controller. The proposed method can realize desired closed-loop response of the non-minimum phase system since virtual feedforward controller can be obtained using data-driven control approach such as ERIT for non-minimum phase systems. The method was validated using numerical simulations and results indicated that the proposed method could achieve good control performance without resorting to updating the feedback and feedforward controllers. In future works, we will further verify the validity of this method through experimental verification of non-minimumphase systems.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Motoya Suzuki He received his BE, ME, and PhD from Tokyo Denki University, Japan, in 2016, 2018, and 2019, respectively. In 2018 and 2019, he joined the Mitsubishi Electric Company and Isuzu Advanced Center, Ltd. In 2020, he joined the University of Electrocommunication. His work primarily involved the development of auto-driving for commercial vehicles. His research interests include control systems and their applications.

Osamu Kaneko
He received his BE and ME from the Nagaoka University of Technology, Japan, and PhD from Osaka University, Japan, in 1992, 1994, and 2005, respectively. In 1999 and 2000, he joined Osaka University and Kanazawa University as an associate professor. In 2015, he joined the University of Electro-Communication as a professor. He received the Best Paper Award from the SICE in 2012. His research interests include control system design and system theory.

ORCID
Motoya Suzuki http://orcid.org/0000-0003-2864-8020 The cost function evaluates a relative error between the transfer function T d and the closed-loop transfer function. Therefore, the feedforward controller can be updated by minimizing Equation (E.1).
Next, ERIT for non-minimum phase systems is introduced [12]. The initial output signal y ini is measured via initial closed-loop experiments. The feedforward controller has an unstable pole when it is set as C FF (ρ) = T d P m −1 . Therefore, the desired transfer function should include the nonminimum-phase properties of the plant. The new desired transfer function for non-minimum-phase systems, T n , is set as Equation (10). The feedforward controller is set as Equation (11). Here, we denote the cost function as: 2) The cost function evaluates a relative error between the transfer function T n and the closed-loop transfer function. Therefore, the feedforward controller and the nonminimum-phase properties can be updated by minimizing Equation (E2).