Sparse Feedback Controller: From Open-loop Solution to Closed-loop Realization

In this paper, we explore the discrete time sparse feedback control for a linear invariant system, where the proposed optimal feedback controller enjoys input sparsity by using a dynamic linear compensator, i.e., the components of feedback control signal having the smallest possible nonzero values. The resulting augmented dynamics ensures closed-loop stability, which infers sparse feedback controller from open-loop solution to closed-loop realization. In particular, we show that the implemented sparse feedback (closed-loop) control solution is equivalent to the original sparse (open-loop) control solution under a specified basis. We then extend the dynamic compensator to a feedforward tracking control problem. Finally, numerical examples demonstrate the effectiveness of proposed control approach.


Introduction
Sparse control is closely related to sparse optimization [1], which penalizes sparsity on controller so as to schedule resource-aware allocation. In control design, the sparsitypromoting idea has thrived in various directions to distributed control [2][3][4], tracking control [5], and predictive control [6][7][8]. When the sparsity is imposed on the control structure (i.e., structured sparsity) whose controller depends on a static state feedback [9][10][11], then the sparse control is recast into a distributed control, which attempts to reduce the number of communication links [2] in networked control system. Another appealing alternative is to penalize sparsity on control signal (i.e., input sparsity) to implement a sparse control that maximizes the time duration over which the control value is exactly zero [12]. In this paper, we are interested in the generation of control signals, which emphasizes on the latter sparse control, namely, reducing control effort or fuel consumption, also known as "ℓ 1 optimal control" or "maximum hands-off control" [6,13].
The optimal control design of sparse signals is of great significance for control system and directly affects the dynamic process of the system. Many practical systems of interest are dependent on a feedback mechanism to achieve closed-loop stability. However, closed-loop realization is more challenging to optimal control problem because determining the feedback gain (matrix) is a non-trivial task [14]. Indeed, in closed-loop sparse control scenarios, almost all existing literature has been focused on discussing "structured sparsity" [2][3][4] by optimizing linear quadratic state feedback cost [9][10][11] rather than pursuing our expected sparse control inputs (i.e., ℓ 1 optimal control) [6,15]. Furthermore, most successful stories on sparse control taking ℓ 1 cost of discrete (resp., L 1 cost of continuous) systems have been extensively treated in open-loop solutions [6,12,13,16]. These recent advances motivate us to study the closed-loop ℓ 1 optimal control problem.
Although "real-time control" bridges the gap between the open-loop and closedloop solutions, schemes such as self-triggered sparse control [12,Sec. VI] and sparse predictive control [6][7][8]17] can, and often do, emerge feedback solutions. On the other hand, these iterative feedback algorithms perform online optimization, leading to the computationally burden, especially when the decision variable is high dimension. Neither exploring sparsity on the structure of feedback gain matrix or exploiting real-time control, we immediately promotes sparsity on the control inputs with a closed-loop response, called sparse feedback control realization. Inspired by seminal works [14,18], a relatively optimal control technique paves the way towards the open-loop solution to the closed-loop solution by means of linear implementation.
In this paper, we focus on closed-loop realization for sparse optimal control design for a discrete linear time invariant system, where the state feedback controller is linear dynamic, as well as enjoys input sparsity. We become aware of the result [19] that induces a desired sparse input by taking a row-sparsity on the static state feedback gain, which implies a structured sparsity on channels. In contrast, we here leverage a dynamic state feedback controller and keep the standard optimal control framework. We believe that such a direct sparse optimization of control signal is more convenient to synthesizing sparse feedback control. One of its benefits is that it relies solely on offline optimization. Thus, the proposed controller can often offer a significant computational saving and control effort minimization. Encouraged by simple theoretical and numerical results by a position paper [20] of the present authors, we extend the results to a more comprehensive version, including problem setup, proofs, tracking control, and simulations. The main contributions of this article are summarized as follows.
• This paper gives feedback realization of sparse optimal control via a dynamic linear compensator. In other words, the sparse feedback controller is derived from open-loop solutions of sparse optimal control. Besides, we propose the computationally tractable analytical and explicit feedback solution for the sparse control problem (Problems 1 and 2). • We provide the stability, optimaility, and sparsity for the closed-loop augmented system, and display that the designed sparse feedback control is essentially a deadbeat control ( This paper is organized as follows. In Section 2, we introduce the problem formulation and the basic preliminaries. Section 3 gives the main result for stabilizing sparse feedback control using a dynamic linear compensator, which can be divided into two steps, that is, sparse optimization and feedback realization, respectively. Section 4 extends the result to a tracking problem by devising a dynamic tracking controller. The numerical examples are illustrated in Section 5. Section 6 concludes this paper. Notation. Throughout this paper, let R, R n , and R n×m denote the sets of real numbers, n dimension of real vectors, and n × m size real matrices, respectively. We use I n (or 0 n ) to denote the identity (or zero) matrix of size n × n, 0 n×m to denote the zero matrix of size n × m; and for brevity, we sometimes abbreviate I (or 0) to represent the identify (or zero) matrix with appropriate size. Let 1 n = [1 · · · 1] ⊤ ∈ R n be the all one vector, and e i ∈ R q stands for an q-tuple basis vector whose all entries equal to 0, except the ith, which is 1. Given a vector x = [x 1 x 2 · · · x n ] ⊤ ∈ R n , we define the ℓ 1 and ℓ 2 norms, respectively, by

Review of sparse optimal control
Consider a discrete linear time invariant (LTI) system described by where u(t) ∈ R m is the control input with m ≤ n, x(t) ∈ R n is the state with an initial value x 0 , y(t) ∈ R p is the output, and A, B, C, and D are real constant matrices of appropriate sizes. Throughout this paper, we assume that the pair (A, B) is reachable.
In this paper, we are interested in sparse optimal control problem, and the control objective is to seek a control sequence {u(t)} N −1 t=0 such that it drives the resultant state x(t) from an initial state x(0) = x 0 to the origin in a finite N steps (i.e., x(N ) = 0) with minimum or sparse control effort.
As indicated in [1], an exact sparsity is achieved by penalizing an ℓ 0 "quasi-norm" on decision variables. However, computing the ℓ 0 norm precisely is challenging due to its non-convex and non-smooth nature, often resulting in an NP-hard problem. It suggests replacing the ℓ 0 cost with convex relaxation using an ℓ 1 norm, which still generates the sparse solution. In particular, the restricted isometry property reveals an equivalence between ℓ 0 (sparse) optimal control and ℓ 1 optimal control for discrete systems [6]. In this context, we shift the idea from compressed sensing to sparse control problem, which seeks an "open-loop" ℓ 1 optimal control action u * in control system [6,13] defined as where u = u ⊤ (0) u ⊤ (1) · · · u ⊤ (N − 1) ⊤ ∈ R mN , and u 1 indicates the ℓ 1 norm of input vector u that sums the absolute values of its elements. Meanwhile, a feasible control set is described by ∈ R n×mN is an N step reachability matrix and satisfies full row rank, i.e., rank(Φ N ) = n. Occasionally, the state and input constraints are necessarily taken into account, for instance, y(t) ∈ Y, where Y is a convex and closed set. Besides, the horizon N should be sufficiently long so that the admissible set of u is nonempty.

Dynamic linear compensator
Before proceeding with the "feedback realization", a compensator K which we demand to design is a dynamic and linear state feedback controller, depending on the evolution K : where z(t) is the state of the compensator and F , G, H, and K are real matrices of compatible sizes. Note that we set the initial state z(0) of the compensator as zero (i.e., z(0) = 0). The advantages of such a dynamic compensator K of (3) are threefold. First, it brings a linear dynamic fashion to the sparse feedback control realization, which allows for computationally tractable compensator gain matrices. Second, it promotes input/temporal sparsity [6,12,13], rather than structured/spatial sparsity for the controller [2][3][4][9][10][11]. Lastly, it ensures internal stability for the closed-loop system.
Notice that the requirement z(0) = 0 is not overly restrictive in our context. In fact, in the following section, we further impose z(N ) = 0, along with x(N ) = 0, as part of the sparse control implementation. This means that we consider the closedloop system through deadbeat control. In this case, once the stable closed-loop reaches the zero state within a finite time, the condition z(0) = 0 is automatically fulfilled whenever we have another x(0) = 0 due to a new disturbance.

Sparse feedback control realization
In this section, we focus on optimal sparse feedback control synthesis from open-loop solution to closed-loop realization. Determining the explicit matrices of F , G, H, K for dynamic compensator (3) is of primary interest in designing sparse feedback controller.
Let us formally state the constrained sparse optimal control problem with a general initial condition x 0 ∈ X 0 , where X 0 . = {e 1 , e 2 , . . . , e n } is used to generate all n possible input-state trajectories, and e i ∈ R n represents the standard basis vector, e.g., e 1 = [1 0 · · · 0] ⊤ ∈ R n . The problem is as follows: Find F , G, H, and K of (3) such that (i) the dynamic compensator (3) stabilizes the plant (1) and (ii) for any x 0 ∈ X 0 with z(0) = 0, the controller (3) generates an input sequence {u(t)} N −1 t=0 , which minimizes N −1 t=0 u(t) 1 subject to the terminal constraints x(N ) = 0 and z(N ) = 0 for a positive integer N , as well as the state and input constraints where s ∈ R p is a given positive vector.
Remark 1. The input sparsity can be easily performed by minimizing the convex ℓ 1 norm instead of the non-convex ℓ 0 norm, as seen in the previous section. Moreover, although we select only a subset of the initial states of the plant, i.e., x 0 ∈ X 0 , which is sufficient for our purposes. In fact, suppose that the given constrained sparse control problem is solved. Since the resultant closed-loop system composed of (1) and (3) is linear, it means that for any x 0 ∈ R n with z(0) = 0, the controller (3) generates a linear combination of the input sequences corresponding to x 0 = e 1 , x 0 = e 2 , . . ., x 0 = e n , thereby achieving sparsity and satisfying x(N ) = 0 and z(N ) = 0.

Closed-loop augmented system
In the celebrated works [14,18], the authors performed the linear implementation built from the relatively optimal technique. This oracle suggests us to investigate an augmented closed-loop system composed of the discrete dynamics (1) and the dynamic compensator (3) of the form where the corresponding state and the gain matrices are given by To represent the closed-loop system dynamics in a compact way, we accordingly introduce a stable matrix P , which is an N -Jordan block associated with 0 eigenvalue, defined by Based on the previous problem setup, we formulate the constrained sparse optimal control problem as the following sparse optimization.
Problem 1 (Sparse Optimization). Find the matrices X ∈ R n×nN and U ∈ R m×nN such that the obtained U is sparse, which amounts to solve a constrained ℓ 1 norm input matrix optimization where e 1 = [1 0 · · · 0] ⊤ ∈ R N and abs(·) returns the absolute value of each element in a matrix.
It is mentioned that Problem 1 is a convex optimization, and hence the solution is computationally tractable by means of the off-the-shelf packages, such as CVX [21] or YALMIP [22]

in MATLAB.
Once the open-loop optimal solution (X, U ) of Problem 1 is attained, we then proceed the second step, that is to say, we move on to tackling the following sparse feedback realization problem.
Problem 2 (Feedback Realization). Based on the solution (X, U ) of Problem 1, solve a linear equation with respect to (K, H, G, F ) and determine the compensator's gain matrices, where

Stability analysis
The approach to sparse feedback control design makes use of the above discussed sparse optimization (Problem 1) and feedback realization (Problem 2), where the dynamic compensator ensures the internally stability of the closed-loop augmented system (5).
The result is summarized in the following theorem and corollary.
Theorem 3.1 (Sparse Feedback Control Realization). Suppose that Problem 1 has the minimizer (X, U ). Then the equation (7) has the unique solution (K, H, G, F ) and the resulting compensator Proof. We first describe the matrices X and U as where X t ∈ R n×n , U t ∈ R m×n , and t = 0, 1, . . . , N − 1. Notice that checking the second constraint of Problem 1 gives rise to the result X 0 = I n . With this fact, it admits that the matrix Ψ is non-singular. In other words, we claim that Since the matrix Ψ is invertible, it follows that the equation (7) has the unique solution (K, H, G, F ) associated with the dynamic compensator (3). Meanwhile, the first constraint of Problem 1 with (7) and (8) asserts that This implies that the closed-loop matrix A + BK = Ψ(P ⊗ I n )Ψ −1 , so that it is similar to a nilpotent matrix (P ⊗ I n ), hence, the closed-loop system (5) is internally stable and the zero terminal state x(N ) = 0 is achieved for any initial state ψ(0) of the system. Moreover, we see that the sequences x(t) = X(e t+1 ⊗ x 0 ) = X t x 0 , u(t) = U (e t+1 ⊗ x 0 ) = U t x 0 , and z(t) = Z(e t+1 ⊗ x 0 ) = Z t x 0 are indeed generated by the system (1) with x(0) = x 0 and the controller (3) with z(0) = 0. As a matter of fact, apparently x(0) = X(e 1 ⊗ x 0 ) = X 0 x 0 = x 0 and z(0) = Z(e 1 ⊗ x 0 ) = 0. Furthermore, based on the fact that (P ⊗ I n )(e t+1 ⊗ x 0 ) = e t+2 ⊗ x 0 , we have We next consider the third constraint of Problem 1, whose validity can be inspected by assessing the inequality −abs(CX + DU ) ≤ CX + DU ≤ abs(CX + DU ) holds true. Therefore, for a given positive vector s ∈ R p and x 0 ∈ X 0 , we have Notice that Cx(t) + Du(t) = (CX + DU )(e t+1 ⊗ x 0 ) = y(t), then the output constraint of the form {−s ≤ y(t) ≤ s} in (4) is verified. According to the above arguments, we claim that the sequences (10), (11), (12), and (13) indeed satisfy the input-state trajectories of LTI dynamics (1) under output constraint (4), which proves the theorem for realizing sparse feedback control.
Remark 2 (Offline vs. Online). It is clear that realizing sparse feedback control in Theorem 3.1 is based on offline computation, and hence the computational complexity is low. Compared with sparse predictive control [6,8,17], a real-time feedback iterations is employed to ensure closed-loop dynamics and online optimization is repeatedly performed as a feedback controller to calculate sparse solutions. Beyond all doubt, predictive feedback control naturally leads to computational burden when the sizes of controlled system is high (e.g., the curse of dimensionality), even for using a fast ADMM (alternating direction method of multipliers) algorithm [6,7].
Based on the proposed Theorem 3.1, we can directly give a corollary that establishes the connection between the open-loop sparse optimal control solution and the closedloop sparse optimal control solution.
Corollary 3.2 (Equivalence). Suppose that Problem 1 has the minimizer (X, U ). Let u * K be optimal sparse feedback control (i.e., closed-loop ℓ 1 optimal control) solution using a dynamic linear compensator K (3), and u * be open-loop ℓ 1 optimal control solution u * of program (2) with output constraint (4), respectively. Then, for x 0 ∈ X 0 , it holds that Corollary 3.3 (Deadbeat Control). Suppose that Problems 1 and 2 have solved, then the implemented sparse feedback controller u * K = Hz +Kx * (i.e., closed-loop ℓ 1 optimal control) of discrete LTI plant (1) is essentially an N -step deadbeat controller.
Remark 3. Regarding the deadbeat control, since the designed compensator K brings the state x(t) to the origin in N steps (satisfying x(N ) = 0), which places all of the eigenvalues of the augmented closed-loop system matrix A + BK at the origin in the complex plane.

Extension: Tracking problem
In this section, we extend the above result of sparse feedback control to tracking control problem [23,Chapter 8].
We start by giving a step-type reference signal r(t) ∈ R p as where r − ∈ R p and r + ∈ R p are constant vectors. The purpose of tracking problem is to design a dynamic tracking compensator such that the performance output tracks a reference input with zero steady-state error by using additional feedforward gains. For this reason, we define the tracking error by e(t) = y(t) − r(t), where y(t) is a performance output signal stated in LTI plant (1). Meanwhile, we make the following assumption before giving an effective tracking controller.
Assumption 1. For a tracking problem, assume that the performance output signal y(t) ∈ R p and the control signal u(t) ∈ R m in LTI dynamics (1) be of the same size (i.e., p = m) and take the matrix D = 0 m .
It is known that the performance output y(t) ∈ R m can track any reference signal r(t) ∈ R m of (15) in the steady-state if As already reported in Section 3, an analogous dynamic tracking compensator K r can be applied to the discrete LTI plant (1) by adding a prescribed reference input r(t) ∈ R m to the control actuator (3). To this end, a dynamic tracking compensator Figure 1. Feedforward tracking control system: r(t) is reference signal; y(t) is the performance output which must track a specified reference input r(t); "Comp" represents a dynamic tracking compensator (17) applied to a discrete LTI plant (1).
K r for the plant can be designed as where L and M represent the feedforward gain matrices with suitable sizes, and r(t) ∈ R m is a specific reference input (15). Notice that here the initial value z r (0) of tracking compensator K r is set to zero (i.e., z r (0) = 0). Figure 1 shows the closed-loop system composed of the plant (1) and the controller (17). For a preferable reference input tracking, we employ the difference or variation of the control inputs N −1 t=0 u(t + 1) − u(t) 1 as the performance index, which is referred to as minimum attention control [24][25][26]. We slightly relax the constraints in the previous sections by removing the state and input constraints (4). As a result, we formulate the following tracking (minimum attention) control problem that we aim to solve here: Find F , G, H, and K of (17) such that Then, determine L and M of (17) such that the steady state gain of the closed-loop system from r(t) to y(t) is the identity and that from r(t) to z r (t) is zero.
Remark 4. When we have a solution to the above problem, we see that y(t) tracks r(t) without steady state error owing to the selected steady state gain. We also observe that u(t) achieves minimum attention for any x 0 ∈ R n due to linearity of the system. Moreover, since z(N ) = 0 is achieved in the steady state for any r + , the condition z(0) = 0 is automatically satisfied whenever we have another r + as a new reference signal.
The closed-loop behavior can be described in an augmented description where and the other matrices A, B, K, and C have been defined for (5). Based on the problem setup above, we first consider the following minimum attention control problem.
Problem 3 (Minimum Attention Control). Find the matrices X ∈ R n×nN and U ∈ R m×nN such that the obtained U solves a minimum attention control problem min X,U Looking for the solution (X, U ) of Problem 3 is always accessible because, the above problem is a convex program. Then, with (K, H, G, F ) of (7) and (8), the resultant closed-loop system (18) always assures internal stability, that is, the condition A + BK = Ψ(P ⊗ I n )Ψ −1 holds, as discussed in Section 3.
We next deal with tracking problem. Due to the fact that the closed-loop augmented system (18) is internally stable with matrices (K, H, G, F ), it admits a unique steadystate ψ ∞ = [x ⊤ ∞ z ⊤ ∞ ] ⊤ for a desired reference input r + = lim t→∞ r(t). More precisely, we have the following matrix equation If y ∞ = Cx ∞ = r + for any reference r + , the output y(t) tracks reference r(t) with no steady-state tracking error. If z ∞ = 0 for any r + , we can have z(0) = 0 whenever the reference signal changes again after a steady-state is achieved.
In what follows, we are going to achieve tracking error elimination with z ∞ = 0 by assigning feedforward tracking gains M and L, leading to the following lemma.
Based on the obtained matrices (K, H, G, F, M, L), for all initial state x 0 ∈ R n and any reference r + ∈ R m , there exist the unique steady-state values (x ∞ , z ∞ ) such that y ∞ = r + and z ∞ = 0 are achieved in the steady-state.
Proof. Since the gain matrices (K, H, G, F ) can previously be calculated by (7) and (8), the augmented closed-loop system (18) is internally stable, as reported in Theo-rem 3.1. We here reformulate the augmented system (19) as Suppose that the zero steady-state z ∞ = 0, we then have This implies that, if the matrix I − (A + BK) is invertible, Therefore, we see the result y ∞ = Cx ∞ = r + if M is selected as (20).
On the other hand, the steady-state of the tracking compensator z ∞ in (22) satisfies Consequently, we derive when the matrix (I − F ) is invertible. Thus, we see that z ∞ = 0 if the feedforward gain matrix L meets (21). Therefore, the feedforward gains (20) and (21) make error cancellation for achieving tracking.   (17), where the tracking control input realizes minimum attention.

Numerical simulations
In this section, we perform several numerical examples to illustrate the effectiveness of the designed sparse feedback control, which gives a closed-loop optimal solution by using a dynamic linear compensator.

Single input
At first, we consider a single-input control system modeled as a linearized cart-pole system, and the parameters benchmark are similar to [ Meanwhile, the output matrices with respect to state-input constraints (4) are set to , which means that the enforced constrained state is only third component of the state x 3 (t) and the imposed constrained input is u(t).
By selecting the suitable variable s, it gives state and input constraints as follows Next, we simulate the discrete-time controlled system, the target is to drive the cart state from a non-zero initial state . In order to realize the sparse feedback controller, we need to solve Problems 1 and 2 to seek the closed-loop ℓ 1 optimal solution. By computing, the total CPU time in PC is 0.38 [s] in MATLAB R2020b using CVX [21], and the found optimal value U * 1 = 6.4204. Figure 2 illustrates the related closed-loop optimal input and state trajectories, respectively. It reflects the input sparsity on sparse feedback control, in which the control sequence is with less active components, and the optimal control meets constraint |u(t)| ≤ 1. In addition, it plots the optimal state trajectories, where the pole angle x 3 fluctuates between the bounds −1 and 1, satisfying the prescribed state constraint |x 3 (t)| ≤ 1. Meanwhile, the trajectories of four different states start from an initial state x 0 and eventually converge to zero state as time tends to a fixed steps under the dynamic compensator (3), this implies that the closed-loop stabilization is achieved.

Multiple inputs numerical benchmark
As a second numerical simulation, we show the result that the synthesized dynamic controller (3) is useful for sparse feedback control of multi-input control system. We here consider a discretized version of third-order system with two control inputs. Using   After the state-input matrices X and U were calculated (see (A.1) in Appendix), we obtained the optimal value U * 1 = 24.1544. Due to the fact that the augmented matrix Ψ is invertible (9), then the controller K with real matrices (K, H, G, F ) is computationally efficient. Figure 3 reveals the pattern of the compensator K, in which the color-bar reports the level of real values of the correlation elements in matrix K. Theorem 3.1 implies that we require the knowledge of the matrices H and K (see (A.2) in Appendix) to synthesize the sparse feedback control, as follows u * K = 1.8798 0.0000 −0.0000 2.8111 0.0000 0.0000 −2.7970 0.0000 −0.0000 1.7122 −0.0000 0.0000 . Figure 4, the optimal feedback control signals contain two components, where both control inputs are along the input constraints |u K,i (t)| ≤ 10, ∀i = 1, 2. Clearly, the inferred feedback control sequences are sparse as desired. From this figure, it appears that the optimal state trajectories converge to zeros with minimum control effort.

Tracking problem
Finally, we show a numerical example to illustrate the effectiveness of our extended dynamic tracking compensator (17) for tracking problem, in Section 4. By taking a can be easily applied to the discrete LTI plant so as to achieve tracking. Figure 5 depicts the evolution of minimum attention tracking control (top) and the corresponding tracking trajectories. It can be see that the performance output signal y(t) gradually tracks a step reference signal r(t) = 1 under the specified time steps.

Conclusion
In this paper, we have proposed a sparse feedback controller from open-loop solution to closed-loop realization. By means of implementing a dynamic linear compensator, the stability, optimality, and sparsity of the closed-loop ℓ 1 optimal control are ensured. Besides, we extended the result to a tracking problem to achieve the minimum attention control. Finally, the simulations illustrated the effectiveness of the proposed sparse feedback control. Future work will focus on investigating sparse feedback control for model-free systems, exploiting input-state/output data to design a data-driven sparse controller. Additionally, an intriguing aspect to explore is sparse output feedback control. However, directly replicating the current results may be challenging due to the influence of generic initial conditions and the initialization of the dynamic output compensator, which are associated with the model matching problem.