Design of cross-coupled CMAC for contour-following – a reinforcement-based ILC approach

ABSTRACT One of the most popular applications of a bi-axial motion stage is precision motion control. The reduction of tracking error and contour error is one of the most coveted goals in precision motion control systems. The accuracy of a motion control system is often affected by external disturbances. In addition, system non-linearity such as friction also represents a major hurdle to motion precision. In order to deal with the aforementioned problem, this paper proposes a fuzzy logic-based Reinforcement Iterative Learning Control (RILC) and a Cross-Coupled Cerebellar Model Articulation Controller (CCCMAC). In particular, the proposed fuzzy logic-based RILC and a LuGre friction model-based compensation approach are exploited to improve motion accuracy. The fuzzy logic-based RILC aims at reducing tracking error and compensating for external disturbance, while the LuGre friction model is responsible for friction compensation. In addition, the CCCMAC consisting of a cerebellar model articulation controller and a cross-coupled controller aims at reducing contour error and dealing with the problem of dynamics mismatch between different axes. Performance comparisons between the proposed fuzzy logic-based Reinforcement Iterative Learning Cross-Coupled Cerebellar Model Articulation Controller (RIL–CCCMAC) and several existing control schemes are conducted on a bi-axial motion stage. Experimental results verify the effectiveness of the proposed RIL–CCCMAC.


Introduction
Recently, due to the thriving development of the Information Technology (IT) industry, the Computer Numerical Control (CNC) machine tools used in manufacturing IT products have been in high demand. Most CNC machine tools are used in precision machining applications, where high precision multiaxis contour-following is essential. Consequently, the development of a sophisticated motion control scheme that is able to achieve high multi-axis contour-following accuracy is crucial [1][2][3][4]. In general, tracking error and contour error are two of the popular performance indices used to assess the contour-following accuracy of a CNC machine tool. Smaller tracking error and contour error indicate better contour-following accuracy and product quality.
In practice, due to different causes such as non-linearity, external disturbance and dynamics incompatibility between different axes, the motion accuracy of a multi-axis motion platform may deteriorate [5][6][7]. To deal with the problem of dynamics incompatibility between different axes, in addition to the well-known feedforward controller, the Cross-Coupled Controller (CCC) proposed by Koren is likely the most popular [8]. The idea of the CCC is to estimate contour error based on the tracking error of each axis. The estimated contour error is used as a basis to calculate suitable compensation torque/force to each axis, yet the original version of the CCC is only suitable for the case of linear trajectory. In order to enhance the applicability of the CCC, many researchers have developed different kinds of contour error estimation algorithms that are suitable for non-linear trajectories or even free-form parametric curves [9,10]. Recently, the parameterbased contour error estimation algorithm was reported to exhibit good contour error estimation performance [11]. Therefore, this paper will incorporate the parameter-based contour error estimation algorithm into the CCC for application to different types of trajectories, as well as improve the effectiveness of the CCC. In addition to the contour error estimation algorithm, one of the possible solutions to improve the effectiveness of the CCC is the design of cross-coupled gain. In the original CCC structure, the cross-coupled gain is fixed for the case of linear trajectory. However, it is not reasonable if the cross-coupled gain is fixed for other types of trajectories.
On the other hand, due to the rapid development of artificial neural networks and other learning approaches for the past several decades, more and more learning control algorithms are exploited to improve contour-following accuracy of multi-axis motion stages [12][13][14][15][16][17][18]. For instance, Wen and Cheng proposed a recurrent fuzzy Cerebellar Model Articulation Controller (CMAC) with a self-tuning learning rate for the control of a bi-axial piezoelectric-actuated micro-motion stage [18]. Shi and Shen proposed an adaptive sliding mode fuzzy CMAC structure for the control of a high-order pneumatic artificial muscles system [19]. Inspired by these previous studies, in order to cope with the problem of dynamics incompatibility between different axes, this paper incorporates the CMAC into a CCC structure (Cross-Coupled Cerebellar Model Articulation Controller, CCCMAC) to tune the cross-coupled gain based on the contour error information provided by a real-time contour error estimation algorithm.
Although the proposed CCCMAC can deal with the dynamics incompatibility problem, it may not be as effective when coping with periodic external disturbance and non-linearities such as friction resulting from repetitive contour-following motions. In order to overcome this difficulty, the fuzzy logic-based Reinforcement Iterative Learning Control (RILC) scheme proposed in this paper and the LuGre friction model [20][21][22][23] are used to suppress external disturbance and compensate for friction, respectively. Several contourfollowing tasks are conducted to verify the effectiveness of the proposed approach.
The rest of the paper is organized as follows. Section 2 introduces the CCCMAC with parameter-based contour error estimation algorithm. The proposed fuzzy logic-based RILC is elaborated in Section 3. Experimental set-up and contour-following results are given in Section 4. Conclusions are addressed in Section 5.
2. Cross-coupled CMAC with parameter-based contour error estimation 2.1. Real-time calculation of contour error for free-form parametric curves As mentioned previously, contour error is an important performance indicator in contour-following applications. The calculation of contour error for special types of trajectories such as straight lines and circles is routine. However, it is not the case for free-form parametric curves. In general, real-time calculation of exact contour error is very difficult if not impossible. As a result, many real-time contour error estimation algorithms have been proposed. For instance, based on the idea of straight line approximation, Cheng and Lee proposed a real-time algorithm that is able to estimate contour error for free-form parametric curves [24]. The drawback of this method is that the reference point used to estimate the contour error is not on the parametric curve. In order to circumvent this issue, Chen et al. [11] proposed a parameter-based contour error estimation algorithm, in which the reference point used to estimate contour error is on the parametric curve. Compared with several existing contour error estimation algorithms, the approach proposed by Chen et al. [11] was reported to exhibit better estimation accuracy. In addition, cross-coupled control-based approaches require knowledge of contour error. Better contour error estimation results will help to boost the performance of the cross-coupled control scheme so as to cope with adverse effects due to dynamics mismatch among different axes. In this paper, the parameter-based contour error estimation algorithm proposed in [11] is employed to provide accurate contour error estimation to the cross-coupled control scheme.

Cross-coupled CMAC controller
The original CCC structure proposed by Koren [8] was designed for straight line-following tasks. Based on contour error information, the CCC aims at reducing contour error caused by dynamics incompatibility among different axes. In order to extend the use of CCC to other types of contour-following tasks, amendments to real-time contour error estimation algorithms are essential. Figure 1 illustrates the block diagram of a bi-axial motion control scheme that contains a CCC teamed up with the parameter-based contour error estimation algorithm. In Figure 1, R x and R y are the position commands for x-axis and y-axis, respectively; P x and P y are the real current positions for x-axis and y-axis, respectively; K px and K py are the position loop gains for x-axis and y-axis, respectively; e x and e y are the tracking errors of x-axis and y-axis, respectively; E 0 c is the estimated contour error, K c is the gain of the CCC, while C x and C y are the cross-coupled gains for x-axis and y-axis, respectively. In the original CCC developed for straight line-following tasks, the crosscoupled gains are defined as follows: where u is the angle between the straight line contour and x-axis. Figure 1. Block diagram of a bi-axial motion control scheme that contains a CCC in conjunction with the parameter-based contour error estimation algorithm proposed in [11].
However, in many contour-following applications, the contour to be followed is not a straight line. That is, the angle u required in the calculation of crosscoupled gains described by (1) and (2) is no longer available since the contour to be followed is not a straight line. One way to cope with this problem is to redefine the angle u. In this paper, the tangent vector at the current position command is computed. u is defined to be the angle between the tangent vector and x-axis.
The CMAC proposed by Albus [25,26] is an artificial neural network belonging to the category of associative memory. It uses local basis functions and has attractive characteristics such as fast learning and good approximation abilities. It has a simple structure and is suitable for online control. In general, the input space S of a CMAC is quantized into many hypercubes and each hypercube is associated with a memory cell. In addition, the content of a specific memory is called its weight. When used in a control system, Equation (3) is employed to compute the control output of a CMAC, while Equations (4) and (5) are used to update the weights of memory cells.
where a Sn is the associated memory index vector of state s N , w represents the memory weight vector for stored data and Nm is the size of the total physical memory. Figure 2 illustrates the block diagram of the CCCMAC proposed in this paper, in which the CCC K c in the original CCC structure is replaced by a CMAC. The idea of the proposed CCCMAC is to adjust the amount of compensation to each axis using a CMAC so as to reduce contour-following errors caused by incompatibility among different axes. Note that the weights of the proposed CCCMAC are trained based on the estimated contour error obtained from the parameter-based contour error estimation algorithm.

Fuzzy logic-based reinforcement ILC with friction compensation
This section will introduce the proposed fuzzy logicbased Reinforcement Iterative Learning Cross-Coupled Cerebellar Model Articulation Controller (RIL-CCCMAC) with LuGre model-based friction compensation.

LuGre model-based friction compensation
Friction is one of the most crucial non-linear phenomena that may lead to deterioration in motion accuracy for a servomechanism. Unlike the classical friction model, the LuGre friction model proposed by Wit et al. [20] is a dynamic model. According to Wit et al. [20], the LuGre friction model is described by where z is the average deflection of the bristles; v is the relative velocity between two contact surfaces; v s is the Stribeck velocity; F c is the Coulomb friction; F s is the level of stiction force; s 0 is the stiffness; s 1 is the damping coefficient; s 2 is the viscous coefficient. Note that the g(v) defined by Equation (7) is a positive function and is used to describe the Stribeck effect.
With the values of friction model parameters given, one can use Equations (6)- (8) to estimate the friction force.

Fuzzy logic-based reinforcement ILC
Iterative Learning Control (ILC) is one of the most commonly used approaches in suppressing external periodic disturbances [13,[27][28][29][30][31][32]. In many motion control applications, the servomechanism is required to perform the same contour-following tasks multiple times. In such a case, it is reasonable to exploit the idea of ILC in suppressing external periodic disturbances so as to improve contouring-following accuracy. Figure 3 illustrates the block diagram of a control scheme consisting of a control-law-based ILC and a feedback controller. According to Figure 3, the system output in the jth iteration is described by (9) In Equation (9), u j is the control output in the jth iteration and is described by where e j+1 (k) is the tracking error in the (j+1)th iteration.
In Equation (10), u FB(j+1) (k) = CÁe j+1 (k) represents the feedback term, while the feedforward term provided by the ILC is described by u ILCðjþ1Þ ðkÞ ¼ Q½u ILCðjÞ ðkÞ þ L j e j ðkÞ (11) where Q is a low-pass filter and L j is the learning function.
In practice, the value of the learning function L j in the jth iteration has a significant impact on the performance of ILC. If the value of L j is too large, the tracking error may diverge eventually. In contrast, if the value of L j is too small, the tracking error may converge in a very slow fashion so that it will take a much longer time to achieve satisfactory contour-following accuracy. In addition, if unknown disturbance is introduced in the system during the operation process, an ILC with a fixed learning rate may not be able to react quickly enough to effectively suppress the adverse effects caused by non-periodic disturbance.
In order to overcome the aforementioned difficulty, by exploiting the paradigm of reinforcement learning [33,34], this paper employs a deep learning strategy [35] to adjust the value of the learning function in the jth iteration. The proposed control scheme is called "fuzzy logic-based reinforcement-ILC" in this paper. In general, the main idea of reinforcement learning is to exploit interactions between "state measurement", "agent" and "environment" to learn and find solutions.
In this paper, "position feedback" is regarded as "state sensing", "learning function L" is the "agent" and "motion accuracy" is the "environment". Most importantly, the idea of reinforcement is exploited in this paper to adjust the value of the learning function L so as to improve contour-following accuracy. Figure 4 illustrates the block diagram of the control scheme consisting of the proposed RILC and a feedback controller. As indicated in Figure 4, the ILC is aimed at reducing the tracking error of each axis so as to improve contour-following accuracy. In each iteration, the root mean square (RMS) value of the tracking error of the entire contour-following task is calculated. The RMS value of the tracking error of the jth iteration is compared with that of the (j¡1)th iteration as described by where e (j¡1)_RMS (described by Equation (13)) is the RMS value of the tracking error of the entire contourfollowing task in the (j¡1)th iteration; e (j¡2)_RMS denotes the RMS value of the tracking error of the entire contour-following task in the (j¡2)th iteration; De (j)_RMS is the difference between e (j¡1)_RMS and e (j¡2)_RMS .
The negative value of De (j)_RMS indicates that the ILC is effective in reducing tracking error. Therefore, there is no need to change the value of the learning function L j . In contrast, if the value of De (j)_RMS is positive or zero, it indicates that the ILC is ineffective in reducing tracking error and the performance of ILC deteriorates. As a result, a reinforcement signal R (j+1) is generated to invoke a change in the value of the learning function L j . In this paper, fuzzy logic [36]   exploited to adjust the value of the learning function L j upon receiving a reinforcement signal. In particular, the reinforcement signal R (j) is described by where R (j) is the reward in the jth iteration; e limit is the pre-defined threshold; K et is the tuning gain.
In particular, De (j)_RMS and R (j) are employed as the two inputs to the fuzzy logic-based RILC learning rate regulator. The membership functions of the fuzzy term sets of De (j)_RMS , R (j) and DL (j) are shown in Figure 5(ac), respectively.
In addition to the periodic disturbance, the bi-axial motion platform also endures non-periodic disturbance. In general, the over-learning phenomenon of ILC is due to the high-frequency non-periodic disturbance. In the first several iterations of learning, the adverse effect caused by the high-frequency nonperiodic disturbance is not significant since the periodic disturbance is yet to be effectively suppressed by the ILC. After several iterations, the periodic disturbance can be suppressed by the ILC, while the adverse effect due to the high-frequency non-periodic disturbance will not be effectively suppressed if a large learning rate is used. As a result, if both the values of e (j)_RMS and R (j) are "positive big", one can conclude that the learning process transpires well enough that a larger learning rate can be adopted. In contrast, if both the values of e (j)_RMS and R (j) are very small and close to "zero", it suggests that an "over-learning phenomenon" may occur. In this case, a smaller learning rate should be used. Based on the above observations and discussions, a rule table for the proposed fuzzy logic-based RILC regulator is constructed as shown in Table 1.
In this paper, the output of the fuzzy logic-based RILC learning rate regulator DL (j) in the jth iteration is calculated using the weighted average defuzzification method [37].
After obtaining the value of DL (j) , one can update the learning function of RILC for the (j + 1)th iteration using L ðjþ1Þ ¼ DL ðjÞ L initial (16) In general, to speed up the convergence process, a larger value of the learning function for the first iteration is chosen. In addition, the value of L j in all iterations is limited to between zero and one.

Proposed control scheme for contour-
following of an X-Y motion stage Figure 6 illustrates the block diagram of the proposed fuzzy logic-based RIL-CCCMAC with a LuGre friction model-based compensation term. In addition to a commonly used feedback controller and a command feedforward compensation term, the proposed fuzzy logic-based RIL-CCCMAC also exploits the advantages of ILC, cross-coupled control and CMAC. In particular, a control law-based ILC is employed in the outer position loop to reduce the tracking error of each axis caused by external periodic disturbance resulting from repetitive contour-following motions.  In addition, based on the contour error information provided by the parameter-based contour error estimation algorithm, a CCC is exploited in the position loop to reduce contour-following errors caused by incompatibility among different axes. Moreover, a CMAC is employed to replace the CCC K c in the original CCC structure to adjust the amount of compensation to each axis. As for the inner velocity loop, in addition to the commonly used Proportional-Integral (PI)-type feedback controller, a LuGre friction model-based compensation term is adopted to compensate for the adverse effects due to friction. Suppose that the modified CCC (i.e. a CCC used in conjunction with the parameter-based contour error estimation algorithm as shown in Figure 1) indicated by the dotted-line rectangle in Figure 6 is inactive at first. According to Figure 6, one will have v cx ¼ sK fx R x þ Q½u ILCjx ðkÞ þ LE jx ðkÞ þ K px E ðjþ1Þx ðkÞ (17) v cy ¼ sK fy R y þ Q½u ILCjy ðkÞ þ LE jy ðkÞ þ K py E ðjþ1Þy ðkÞ (18) where v cx and v cy are the velocity commands provided by the outer position loop controller for x-axis and yaxis, respectively; E x and E y represent the tracking errors of x-axis and y-axis, respectively. Now, consider the modified CCC indicated by the dotted-line rectangle in Figure 6. Based on the contour errorE 0 c estimated using the parameter-based contour error estimation algorithm, Equation (19) is used to update the weights of CMAC [18].
In addition, the amount of compensation force provided by the CCCMAC to the motions in x-axis and y-axis is calculated using Equations (21) and (22), respectively.
Suppose that the modified CCC in Figure 6 is activated. Then, v cx and v cy in Figure 6 can be rewritten as where the position loop control laws for the x-axis are In addition where the position loop control laws for the y-axis are V ff y ¼ sK fy R y (28) U ilcy ¼ Q½u ILCjy ðkÞ þ LE jy ðkÞ (30)

Experimental set-up
In order to verify the effectiveness of the proposed approach, several contour-following tasks have been conducted on a bi-axial motion platform as shown in Figure 7. This bi-axial motion platform consists of two ball screws actuated by two Panasonic 400 W AC servomotors (MSDA041A1A) with built-in 2500 £ 4 pulses/rev encoders. In addition, the pitch of the ball screw is 1 cm/rev. The servo drive of each AC servomotor is set to torque mode throughout all the contour-following experiments and the sampling time is set to 1 ms.

Contour-following experiment of bi-axial motion stage
The desired circle-shaped contour used in each contour-following experiment is represented in Non- Uniform Rational B-Splines (NURBS) form as shown in Figure 8(a). A NURBS interpolator with the consideration of s-curve acceleration/deceleration motion planning is used to generate position commands for the x-axis and y-axis as shown in Figure 8(b,c) [38]. A total of three different control schemes for the velocity loop are tested in each contour-following experiment.  Tables 2 and 3. In particular, from Figure 9(a), one can find that the maximum tracking error on the x-axis for the case of control scheme #1 is around §0.325 mm, while Figure 9(b) shows that the maximum tracking error on the x-axis for the case of control scheme #2 is around §0.13 mm. Experimental results shown in Figure 9(a,b) indicates that control scheme #2 outperforms control scheme #1. These findings are not surprising, since in this paper the input to the CMAC is the velocity command; namely, CMAC possesses the ability of friction compensation as well as command feedforward. However, control scheme #2 suffers from the aforementioned over-learning phenomenon since the tracking errors start rising after several iterations. Figure 9(c,d) shows the tracking error and their RMS values on the x-axis for control scheme #3, respectively. In particular, as shown in Figure 9(d,e), the RMS value of tracking error and the value of learning function keep decreasing through iterations. In addition, Tables 2 and 3 summarize the performance comparison of contour-following among the three tested control schemes. Clearly, Tables 2 and 3 and the tracking errors shown in Figure 9 all indicate that the proposed fuzzy logic-based RIL-CCCMAC combined with a PI-type velocity loop feedback controller outperforms the other two control schemes also tested in the experiments. In order to verify the effectiveness of the proposed fuzzy logic-based RIL-CCCMAC scheme  in coping with non-periodic disturbance, a signal that emulates the external disturbance is intentionally added to the bi-axial motion platform in the sixth iteration.
Experimental results shown in Figure 10(a,b) reveal that the RMS value of tracking error in the seventh iteration is larger than that of the sixth iteration so that a reinforcement signal is generated to invoke a change in the value of the learning function. In this case, the proposed fuzzy logic-based RILC learning rate regulator is exploited to adjust the value of the learning function as shown in Figure 10(c). Experimental results shown in Figure 10(b,c) indicate that the RMS value of tracking error starts decreasing after the value of the  learning function is adjusted. The above observations suggest that the proposed fuzzy logic-based RIL-CCCMAC scheme is indeed capable of suppressing external disturbance. Note that the formulas used to compute the tracking error and performance indices listed in Tables 2 and 3 are described by Equations (13), (31) and (32).

Conclusion
The contour-following control scheme proposed in this paper consists of a position outer loop controller and a velocity inner loop controller. In particular, the position loop is composed of a P-type feedback loop controller, a velocity feedforward term, a fuzzy logicbased RILC and a CCCMAC. The fuzzy logic-based RILC is mainly used to compensate for external periodic and non-periodic disturbance, while the CCCMAC is proposed to deal with the problem of dynamics incompatibility between different axes by exploiting an accurate contour error estimation algorithm. The velocity loop consists of a PI-type feedback loop controller and a LuGre model-based friction compensation scheme. Experimental results of several contour-following tasks indicate that the proposed approach outperforms several existing motion control schemes also tested in the contour-following experiments. In particular, by adjusting the learning rate online, the proposed RILC is able to effectively suppress external periodic and non-periodic disturbance so as to improve contour-following accuracy.