The relaxed gradient based iterative algorithm for solving the generalized coupled complex conjugate and transpose Sylvester matrix equations

Inspired by the idea of Ma et al. (Journal of the Franklin Institute, 2018), we adopt relaxation technique and introduce relaxation factors into the gradient based iterative (GI) algorithm, and the relaxed based iterative (RGI) algorithm is established to solve the generalized coupled complex conjugate and transpose Sylvester matrix equations. By applying the real representation and straighten operation, we contain the sufficient and necessary condition for convergence of the RGI method. In order to effectively utilize this algorithm, we further derive the optimal convergence parameter and some related conclusions. Moreover, to overcome the high dimension calculation problem, a sufficient condition for convergence with less computational complexity is determined. Finally, numerical examples are reported to demonstrate the availability and superiority of the constructed iterative algorithm.


Introduction
Solving matrix equations is one of the research focuses of computational mathematics [1][2][3][4].The Sylvester matrix equation is an important type of matrix equations, which has a wide range of applications in control and system theory, pole assignment, model reduction and so further [5][6][7].Therefore, finding feasible and effective algorithms for the Sylvester matrix equation has important theoretical significance and practical application value.
In this paper, we aim to find the solution of the generalized coupled complex conjugate and transpose Sylvester matrix equations where are the known matrices and Y j ∈ C r j ×s j are unknown matrices to be solved.Equation ( 1) is involved in both science and engineering.Besides, its form includes many other special matrix equations, such as complex conjugate Sylvester matrix equations, complex transpose Sylvester matrix equations and complex conjugate and transpose Sylvester matrix equations [8][9][10].Therefore, it is meaningful to research efficient methods for solving Equation (1).
At present, the methods for solving the Sylvester matrix equation mainly include direct methods and iterative methods.However, when solving high-dimensional matrix equations, the direct methods may lead to lengthy computation time.In order to efficiently solve these matrix equations, we prefer to apply iterative methods.In the past few decades, many scholars are devoted to establishing iterative methods to solve various types of Sylvester matrix equations [7,[11][12][13][14].
From the previous works [15,16], we know that it is difficult to calculate the exact solution of matrix equations, which consumes large computing costs.In the field of systems and control, calculating approximate solutions is sufficient.So iterative solutions have been widely concerned by researchers, and many researchers paid attention to iterative methods and got excellent results.Ding and Chen developed various iterative algorithms to solve Ax = b, AXB = F and other Sylvester matrix equations [17][18][19].Subsequently, many effective iterative methods were proposed.In [10], the least squares based iterative method has been applied to find the solutions of the Sylvester transpose matrix equation AXB + CX T D = F. Xie and Ding constructed the gradient based iterative (GI) methods for the matrix equations AXB + CXD = F [9]. Wu et al. also investigated the GI method for solving the Sylvester conjugate matrix equation AXB + CXD = F [20].Owing to the availability of the GI algorithm, the GI algorithm has been extended to solve general Sylvester matrix equations by researchers.For instance, Wu et al. proposed the GI algorithm to find the solution of coupled Sylvester-conjugate matrix equations [21] l i=1 where ) are the known matrices.Song et al. applied the GI method to the coupled Sylvestertranspose matrix equations [22] l i=1 where ) are the known matrices.
The above two matrix equations are important types of matrix equations, which are frequently involved in the fields of systems and control.Not only that, they are also the generalized forms of matrix equations in [10,20], respectively.The convergence properties and the optimal convergence parameters of the GI algorithm for Equations ( 2) and ( 3) have been investigated.Subsequently, Beik et al. proposed the GI algorithm for solving the generalized coupled Sylvester-transpose and conjugate matrix equations [23] T where A viμ , B viμ , C viμ , D viμ , M viμ , N viμ , H viμ , G viμ , F v (v = 1, 2, . . ., N) are known matrices with proper dimensions.The form of the above matrix equations is quite general.When p and s 1 , s 2 , s 3 , s 4 , are taken to be some special values, Equation ( 4) can be transformed into other matrix equations.
Except for the above classical Sylvester matrix equations, the GI algorithm has also been applied to solve periodic matrix equations [24,25].Li et al. established the GI method for the forward periodic Sylvester matrix equations and backward forward periodic Sylvester matrix equations [25] and where Numerical experiments have also showed the effectiveness of the GI algorithm.
Although the theory of the GI algorithm has been systematically proposed by researchers, this algorithm still has some drawbacks.In [26], Fan et al. pointed out that the GI algorithm costs large computation time and storage space when encountering ill-posed problems.In order to further optimize the convergence performance of the GI algorithm, the relaxed gradient based iterative (RGI) algorithm has been proposed by introducing the relaxed factor to adjust the weight of the iteration sequences.Niu et al. developed the RGI algorithm to solve the Sylvester matrix equations [27].Numerical experiments have shown that relaxation techniques can effectively reduce computation time and storage space, and improve the convergence rate of the GI algorithm.
Due to the superiority of the RGI algorithm, many scholars have extended this algorithm to solve more general matrix equations.Recently, in [28,29], Huang et al. applied the RGI algorithm to solve coupled Sylvester-conjugate matrix equation ( 2) and coupled Sylvester-transpose matrix equation (3).And the experiments results illustrate that the convergence rate of the RGI algorithm is faster than the GI one.Then Wang et al. consider the solution of the complex conjugate and transpose matrix equations (7) where A i , B i , E ∈ C n×n (i ∈ I [1,4]) are the known matrices.By introducing relaxation factors and applying the hierarchical identification principle [30], Wang et al. presented the RGI method to solve Equation (7).However, Wang et al. didn't discuss the generalized form of Equation (7).Based on the ideas of [30], we extend the RGI algorithm to the generalized coupled complex conjugate and transpose Sylvester matrix equations.
Inspired by the idea of [28], we construct the RGI algorithm for solving Equation (1).Its form can be specific written as The form of the above matrix equations is quite general, which contains several classic Sylverster matrix equations.Especially, Equations ( 2)-( 3) are special cases of Equation (1).If i = j = p = q = 1, Equation (1) will reduce to Equation (7).Therefore, finding faster algorithms to solve Equation ( 1) is of great significance.
To accelerate convergence rate of the GI algorithm for Equation (1), we combine relaxation technology with hierarchical identification principle, and we derive the relaxed gradient based iterative (RGI) algorithm to solve Equation (1).This principle regards the unknown matrix as the system parameter matrix to be solved, then it builds a recursive formula to approach the unknown solution [27,28,30,31].Furthermore, we can effectively control the weight of the iteration sequence by introducing relaxation factors.In theory, we exploit the real representation and the straightening operator to prove the convergence properties of the constructed algorithm.Meanwhile, the sufficient and necessary condition for convergence is presented.Finally, numerical experiments further demonstrate the effectiveness and superiority of the RGI algorithm.The main motivation and contribution of this paper are summarized as follows: • In order to accelerate the convergence rate of the GI algorithm [23], we combine the GI algorithm with relaxation technique.By introducing l relaxation factors, we construct the RGI algorithm for Equation (1).Due to that Equation (1) extremely is general, the algorithm constructed in this paper is also more general.It is meaningful to promote the development of the field of solving matrix equations.• To optimize convergence theory, we utilize real representation and straighten operation as tool, and present the sufficient and necessary condition for convergence of the RGI method.To overcome high-dimensional computing problems, the sufficient condition for convergence and some related results are proposed.Besides, we use numerical experiments to fully demonstrate the effectiveness and superiority of the RGI algorithm.
The remainder of this paper is structured as follows.In Section 2, we list several useful notations and definitions.Moreover, we construct the relaxed gradient based iterative (RGI) algorithm to find the iterative solution of Equation (1) in Section 3. In Section 4, we deduce the convergence properties of the proposed method, including the sufficient and necessary condition for convergence, the optimal convergence factor and the related corollary.In Section 5, two numerical experiments are reported to validate the superior of convergence for the new algorithm.In the end, Section 6 proposes the some conclusions.

Preliminaries
For the sake of convenience, we provide several main notations and lemmas which are used throughout this paper.The set of m × n complex matrix is denoted by C m×n .For A ∈ C m×n , there are some related notations as follows: • A indicates the conjugate of the matrix A; • A T represents the transpose of the matrix A; • A H stands for the conjugate transpose of the matrix A; • σ max (A) stands for the maximal singular of the matrix A; • σ min (A) stands for the minimal singular of the matrix A; • cond(A) = σ max (A)/σ min (A) is defined as the condition number of A; • λ max (A) represents the maximal eigenvalue of the matrix A; • λ min (A) indicates the minimal eigenvalue of the matrix A; • A 2 is defined as the the spectral norm of the matrix A. • A indicates the Frobenius norm of the matrix A.
• ρ(A) represents the spectral radius of the matrix A; Then, some significant definitions and lemmas are listed below.

Definition 2.1 ([28]):
Let A ∈ C m×n , then A can be uniquely expressed as

Definition 2.3 ([28]
): Let e in denote an n-dimensional column vector which has 1 in the ith position and 0's elsewhere.The vec-permutation matrix P mn can be defined as . . .
If X, A ∈ C m×n and B ∈ C p×q , we have and Next, we review several lemmas which are used to prove the convergence property.

Lemma 2.3 ([28]):
For A ∈ C m×r , B ∈ C s×n , F ∈ C m×n , if the matrix equation AXB = F has unique solution, then the iterative sequences {X(k)} converges to the exact solution X * for any initial matrix X(0) by the following algorithm and the algorithm is convergent if and only if Meanwhile, the optimal convergence factor is Proof: Define error matrix According to the expression (17), it has Repeatedly applying the relationship of the above expression leads to If the convergence parameter μ is selected to satisfy the following inequality holds This means that lim i→∞ Z(i) 2 = 0. Due to that the matrix equation AXB = F has unique solution, then it has lim k→∞ X(k) = 0.The proof of Equation ( 18) is completed.
Taking the vec-operator of both sides of the expression (17) and applying Lemma 2.2, it can get The above equation implies that I − μBB H ⊗ A H A is the iterative matrix of the algorithm.Thus, the optimal convergence parameter satisfies the following equation By simple deductions, Expression ( 19) can be obtained.

Lemma 2.4 ([28]): The properties of real representation (.) are as follows:
For two complex matrices A ∈ C m×n , B ∈ C n×r , then Here, unitary matrices E n is defined as Furthermore, based on the definition of matrix Frobenius norm and real representation, then then, the optimal convergence parameter μ is selected as Proof: Build function y = max 1≤i≤n |1 − μm i |, and then Equation ( 24) has been obtained by drawing graph in [29].Besides, The optimal convergence factor μ satisfies

The above equation indicates that
Thus, the proof is completed.

The relaxed gradient-based iterative algorithm
In this section, we mainly propose the relaxed gradient based iterative (RGI) algorithm to solve the generalized coupled complex conjugate and transpose matrix equation.The main idea of this algorithm is to use the hierarchical identification principle to divide Equation ( 1) into several subsystems.The unknown matrixes Y j are regarded as the identified parameters matrices.Meanwhile, we construct intermediate matrices and adopt an average strategy.Then, the relaxation factors ω l , l ∈ I[1, q] are introduced, which are utilized to adjust the weights of matrix schemes.The construction process of the RGI algorithm is as follows.
Firstly, define the following intermediate matrices, From the expression of Equation ( 1), some subsystems are given below, According to the above fictitious subsystems and Lemma 2.3, we can put forward the iterative schemes as follows, For the sake of convenience, we provide the following notations, s ∈ I [1,4], Combining Equations ( 25)- (28) with Equations ( 33)-(36) and utilizing the hierarchical identification principle, the recursive systems are established.Due to that the unknown matrices Y j are included in the expressions, we replace Y j in ( 25)- (28) with Then, by taking the average value of Inspire by the idea of the RGI method in [28], we introduce the relaxation factors ω l (l ∈ I[1, q]) into the above recursive systems.Based on the previous analysis process, the relaxed gradient-based iterative (RGI) algorithm for Equation ( 1) is presented as follows.
In the RGI algorithm μ indicates the convergence parameter.The relaxation factors ω l (l ∈ I [1, q]) are used to control the weight of iterative sequences, and it can effectively improve the convergence rate of the GI method.In particular, if the relaxed factors are selected as ω l = 1  2 for all l, Algorithm 1 will reduce to the Algorithm 1 The relaxed gradient-based iterative algorithm Step 1: and given any positive constants number ε and μ.Let 0 Step 3: Update the sequence Y (1)  l (k Calculate Step 4: Set k = k + 1; return to Step 2.
GI algorithm [23].Besides, Algorithm 1 with ω l = 1 2 and p = q = i = j = 1 will change into the iterative method in [30].Compared with the RGI algorithm in [28], the new algorithm is more general which includes many kind of iterative formulas.
In what follows, the convergence properties of the RGI method are analysed below.At the same time, we also provide the detailed proof of convergence theory.

Convergence analysis of the RGI algorithm
The section presents the sufficient and necessary condition for convergence of the RGI algorithm.Furthermore, to overcome the high-dimensional calculation problem of the iterative matrix, we further discuss the sufficient condition for convergence.Theorem 4.1: Assume that the generalized coupled complex conjugate and transpose matrix Equation ( 1) has a unique solution, then the iterative sequences Y l (k) l ∈ I[1, q] converge to the exact solution Y * l by the RGI algorithm for any initial matrices Y l (0) for all l ∈ I[1, q] if and only if the convergence factor μ is selected to satisfy where Proof: Denote To facilitate our statement, the expressions of Z i (k) (i ∈ I [1, p]) are defined as follows From the definition of error matrices and the expression of Y (1)  l (k + 1) in the RGI algorithm, we derive that for l ∈ I[1, q] Y (1)  l (k It follows from the expression of Y (2)  l (k + 1) in the RGI method that Y (2)  l (k By the expression of Y (3) It follows from the expression of Y ( 4) Combing ( 48)-( 51) with Line 5 of the RGI algorithm leads to According to Lemma 2.4, taking the real representation on both sides of (52) results in Using straightening operator in (53) and applying Definition 2.3, for l ∈ I[1, q] we have Furthermore, by applying the real representation on two sides of (47), we get Then, utilizing the vec-operator in (55) can deduce Finally, substituting (56) into (54) results in Thus, Equation (57) can be written as the following expression The matrices M and Q are given in ( 43) and (44).It follows from Equation (59) that the matrix [I − μMQ T Q] is the iterative matrix of Algorithm 1.So the sufficient and necessary condition for convergence of the RGI algorithm is Due to the fact that the iterative matrix Finally, the range of convergence parameter μ making the RGI algorithm convergent is Here, we complete the proof of Theorem 4.1.
In order to further effectively utilize the RGI algorithm, we should get the optimal convergence parameter μ of this method.When ρ(I − M reaches minimum value, the convergence behaviour of the RGI method achieves the optimum.According to Lemma 2.5, the necessary and sufficient condition for By simple calculations, the optimal convergence parameter is obtained as Then, we will further discuss the convergence properties of the RGI method with the relaxation parameters ω l = ω for l ∈ I [1, q].Some relevant conclusions are proposed below.
represent the initial value and unique solution of the RGI algorithm, respectively.Based on the conditions of Theorem 4.1, if the relaxation factor are selected as ω l = ω for l ∈ I [1, q], it holds that And the optimal convergence parameter is Under this situation, the following inequality holds Proof: According to the fact that Combining Expression (59) with the properties of matrix norms, we derive with Based on Lemma 2.4 and A 2 = 2 A 2 , it holds By the definition of the error matrix and Inequality (70), we derive that Moreover, when ρ(I − 1 4 μω(1 − ω)Q T Q) is minimized, the convergence performance of the RGI algorithm can achieve optimal.So we should choose the optimal parameter μ opt to minimize ρ(I has a non-trivial solution.By simple derivations, the best convergence parameter is If the convergence parameter μ is selected as the one in (75), it will lead to Then Equation ( 68) can be derived by substituting Equation ( 76) into (73).2 involves the calculation of real representation and Kronecker product, which leads to high-dimensional problems.In order to overcome this drawback and develop computational efficiency, we further derive sufficient condition for the convergence with less computational complexity.

Corollary 4.1:
Assume that the conditions of Theorem 4.1 are satisfied, then Algorithm 1 is convergent for any initial matrix if the parameters ω and μ are selected to satisfy the following inequality Proof: By the properties of Frobenius norm of matrix, one has Notice the fact that A 2 , and we have the following inequality By combining (79) with (42), the conclusion of Corollary 4.1 is correct.

Numerical experimental results
In this section, we present two numerical examples to testify the effectiveness and feasibility of the RGI algorithm proposed in this paper.All experiments are performed on a personal computer with AMD Ryzen 5 5600U with Radeon Graphics 2.30 GHz, 16.0 GB.The programming language is computed in MATLAB R2021b.In our experiment, we compare the convergence behaviour of the RGI algorithm with the GI one in terms of the iterative number (IT), calculation time (CPU) in seconds and the relative error (ERR).

Example 5.1 ([34]):
We consider the generalized coupled complex conjugate and transpose Sylvester matrix Equation (1) in the case of p = q = 4, and its form is as follows with This matrix equation has the following exact solution In this example, the initial iterative matrices are taken as Y i (0) = 10 × I 3 , i ∈ I [1,4], and the relative iterative error is defined as (81) where Y i (k)(i ∈ I [1,4]) stand for the kth iteration solution.The iteration is terminated if the relative iterative error is less than δ or the number of the prescribed iteration step k max = 30000 is exceeded.Here, δ is a positive number.By some calculations, we find that Example 5.1 satisfies the condition of Theorem 4.1.Then the optimal parameter of the RGI algorithm is obtained as μ = 5.2559e − 06 when relaxation factors are chosen as ω 1 = 0.25, ω 2 = 0.52, ω 3 = 0.32, ω 4 = 0.48.However, there are errors in the experiment, and the convergence rate of the RGI method is not the fastest if μ is chosen to be 5.2559e−06.Thus, we try to find the optimal experimental parameter near the value μ = 5.2559e − 06.In Figure 1, we compare the convergence performance of the RGI algorithm under μ 1 = 5.1499e − 06, μ 2 = 5.2559e − 06, μ 3 = 5.0499e − 06 and μ opt = 5.2499e − 06 , respectively.As shown in Figure 1, if the convergence parameter μ is selected as different values, the convergence curve also has corresponding change.In order to more intuitively observe the performance of the RGI algorithm under different convergence parameters, we list the IT of the RGI algorithm in Table 1.It is evident that the convergence performance is the best when parameter μ is chosen to be μ RGI = 5.2499e − 06.
Moreover, the RGI algorithm with ω 1 = ω 2 = ω 3 = ω 4 = 0.5 reduces to the GI algorithm.Similarly, we adopt the method of experiment debugging to find the optimal experimental parameter of the GI algorithm.Finally, the IT of the GI algorithm with μ GI = 4.5503e − 06 is the least.
In Figures 2-3, we present that the convergence curves of the RGI and GI algorithms with different δ.In this experiment, we compare the convergence curves of two algorithms under the optimal experimental parameters.It follows from Figures 2-3 that the ERR decreases as the IT increases and gradually approaches 0, which indicates that the tested algorithms are effective and convergent.In addition, Figures 2-3 clearly show that the IT of the RGI and GI algorithms are decreasing as the increasing of δ.Besides, we can find that the convergence speed of the RGI algorithm is always faster than the GI one in the above four situations.
In order to more specifically verify the advantages of the RGI algorithm, we list detailed numerical results of the RGI and GI algorithms in Table 2, which includes IT, CPU and ERR.According to Table 2, the IT and CPU of the tested algorithms are gradually increase with the decreasing of the parameter δ.Moreover, it can be seen that the IT and CPU of the RGI method are always less than the GI one.Therefore, we can conclude that the convergence performance of the RGI algorithm proposed in this paper is better than the GI algorithm [23].     1) in the special case of p = q = 2, and it has with the following parametric matrices It has the exact solution The initial iterative matrices are taken to be Y i (0) = 10 −6 × I 3 , i ∈ I [1,2].Then, we denote the relative iterative error by In this example, all runs are stopped once ERR is less than ξ or k reaches the maximal iterative steps k max = 50,000.Here, ξ is a positive number.For Example 5.2, we also compare the convergence performance of the RGI and GI algorithms.The optimal convergence parameters involved in two algorithms are determined by the following method.If relaxation factors are selected as ω 1 = 0.07, ω 2 = 0.18, the optimal convergence factor of the RGI algorithm is adopted as μ RGI = 1.0821e − 04 by Theorem 4.1.Moreover, the RGI algorithm with ω 1 = ω 2 = 0.5 reduce to the GI algorithm.By some calculations, the best convergence parameter of the GI algorithm is μ GI = 4.5361e − 05.
In Figures 4-5, we plot the graphs of ERR(log10) versus the IT of the RGI and GI algorithms with different ξ .According to the convergence curves, we observe  that the two algorithms are both convergent and efficient.It is obvious that the convergence rate of the RGI method (ω 1 = 0.07, ω 2 = 0.18) is always faster than GI one (ω 1 = ω 2 = 0.5) for the four cases of ξ .In addition, it follows from Figures 3-4 that the IT and CPU of the tested algorithms are increasing with the decreasing of ξ .In particular, the convergence advantage of the RGI algorithm is more obvious when ξ is smaller.The results illustrate that the RGI algorithm is superior to the GI algorithm if the relaxation parameters are chosen appropriately.
In order to further verify the advantages of the proposed algorithm, we clearly report the numerical results of the RGI and GI methods for Example 5.2 in Table 3. From Table 3, it is easy to discover that the IT of the algorithms is increasing with the decreasing of relative error.Furthermore, the IT and CPU of the RGI method are less than those of the GI one.As a whole, the proposed algorithm has better convergence behaviours than the GI method.This means that the relaxation technique can effectively improve the convergence rate of the GI algorithm.

Concluding remarks
In this paper, by adopting the relaxation technique into the GI algorithm, we establish the relaxed gradientbased iterative (RGI) algorithm to solve the generalized coupled complex conjugate and transpose Sylvester matrix equations.The main idea of the algorithm is introducing relaxation parameter to control the weights of iterative sequences.Applying straighten operation and real representation of complex matrices, we derive the necessary and sufficient condition for convergence of the RGI algorithm.Besides, the optimal convergence parameter and some related conclusions are given.To overcome high-dimensional computational problems, we propose sufficient condition for convergence with smaller computational complexity.Finally, numerical experiments verify that the RGI algorithm has more excellent convergence performance than the GI one.Note that in our experiment, the relaxation factors ω l (l ∈ I[1, q]) are obtained through experimental debugging.The selection criteria for the optimal relaxation factors are not provided.The future research direction is to further develop the theory of selecting the optimal relaxation factor.Besides, the value of the convergence parameter μ in the RGI algorithm is fixed.To optimize the convergence performance of the RGI algorithm, we will consider to introduce different step size factors into the RGI algorithm.
n×n are the known matrices.In theory, Li et al. proposed the sufficient and necessary conditions for the convergence of the GI algorithm.

Remark 4 . 1 :
In Theorem 4.1, the sufficient and necessary condition for convergence of the RGI method is obtained.However,

Figure 1 .
Figure 1.Comparison of convergence performance of RGI with different parameters μ for Example 5.1.

Figure 5 .
Figure 5.The convergence curves of the tested methods with ξ = 0.001 (left) and ξ = 0.0001 (right) for of Example 5.2.
If m i , i ∈ I[1, n]are any given positive number, denote the maximum and minimum values of m i as m max = max 1≤i≤n m i and m min = min 1≤i≤n m i , respectively.It has Lemma 2.5 ([29]):

Table 2 .
Numerical results of the tested methods with different δ for Example 5.1.

Table 3 .
Numerical results of the tested methods with different ξ for Example 5.2.