A new result on recovery sparse signals using orthogonal matching pursuit

Orthogonal matching pursuit (OMP) algorithm is a classical greedy algorithm widely used in compressed sensing. In this paper, by exploiting the Wielandt inequality and some properties of orthogonal projection matrix, we obtained a new number of iterations required for the OMP algorithm to perform exact recovery of sparse signals, which improves significantly upon the latest results as we know.


Introduction
Orthogonal matching pursuit (OMP) has received growing attention due to its simplicity and competitive reconstruction performance recently. Consider the following compressed linear model: where x ∈ C n is a K-sparse signal (i.e., x 0 ≤ K), = [φ 1 , φ 2 , . . . , φ n ] ∈ C m×n is a known measurement matrix with m n and y ∈ C m is the observation signal. It has been demonstrated that under some appropriate conditions on , OMP can reliably recover the signal x based on a set of compressive observations y by iteratively identifying the support of the sparse signal according to the maximum correlation between columns of measurement matrix and the current residual. See Table 1 for a detailed description of the OMP algorithm (Cai & Wang, 2011;Chang & Wu, 2014;Tropp & Gilbert, 2007;Wang & Shim, 2016;Wen et al., 2020Wen et al., , 2017Wu et al., 2013). In Table 1, supp(x) is the set of nonzero positions in x. r k denotes the residual after the kth iteration of OMP and T k the estimated support set within kth iteration of OMP.
In compressed sensing, a commonly used framework for analysing the recovery performance is the restricted isometry property (RIP) (Cai et al., 2010;Candes & Tao, 2005;Chang & Wu, 2014). A matrix is said to satisfy the RIP of order K if there exists a constant δ ∈ [0, 1) such that for all K-sparse signal x. In particular, the minimum of all constants δ satisfying (2) is called the K-order Restricted Isometry Constant (RIC) and denoted by δ K . Over the years, many RIP-based conditions have been proposed to guarantee exact recovery of any K-sparse signals via OMP in K iterations. It has been shown in Davenport and Wakin (2010) Huang and Zhu (2011). Mo (2015) demonstrated that δ K+1 < ( √ K + 1) −1 is a sharp condition for exact recovery of any K-sparse signal with OMP in K iterations. Our recent work Liu et al. (2017) provides some sufficient conditions for recovering restricted classes of K-sparse signals with a more relaxed bound on RIC.
x k = arg min u∈C n :supp(u)⊂T k y − u 2 , Output: T k and x k In this paper, we present a new result on how many iterations of OMP would be enough to guarantee exact recovery of sparse signals: which improves significantly upon the results proposed in Wang and Shim (2016). We first give some notation. Let N = {0, 1, 2, . . .}, N + = {1, 2, . . .} and n = {1, 2, . . . , n}.
[·] and · denote floor and ceiling function, respectively. For any two sets and , let \ = {i : i ∈ , i / ∈ }, and | | is the cardinality of . For ⊂ n and = ∅, denotes the submatrix of that contains only the columns indexed by and x denotes the subvector of x that contains only the entries indexed by , and span( ) represents the span of columns in . Let P = ( * ) + * stand for an orthogonal projection matrix onto span( ), where * is the conjugate transpose of the matrix , and ( * ) + is Moore-Penrose pseudo inverse of * . P ⊥ = I m − P is an orthogonal projection matrix onto the orthogonal complement of span( ), where I m denotes the identity matrix. In particular, if = ∅, then x ∅ is a 0-by-1 empty vector, ∅ is an m-by-0 empty matrix, ∅ x ∅ is an m-by-1 zero matrix and span( ∅ ) := {0}. For further details on empty matrices, see, e.g., Bernstein (2005).

Main results
For notational simplicity, we denote k = T \ T k .

Remark 2.1 (Performance of Theorem 2.1): From
In the following, we compare the lower bound in (4) with the result of Wang and Shim (2016), which has been showed in (3). We first establish an upper bound for the ratio of (4) to (3) by using monotonicity property of the RIC (δ 1 ≤ δ K+κ ) as It is easy to check that R(δ) < 1 for 0 < δ < 1, which means the lower bound of c in this paper is uniformly smaller than the one proposed in Wang and Shim (2016). See Figure 1, for example, we have R(3 −1 ) = 0.57 and R(2 −1 ) = 0.58. For 0.015 < δ < 0.993, we have 0.57 < R(δ) < 0.8. That is, nearly 98% of the values of the bounds proposed in this paper is less than 0.8 times of the one in (3).

Remark 2.2 (Main differences with the previous work):
Our methods have several key distinctions in construction of the more efficient lower and upper bounds of resident in OMP. First, Wang and Shim (2016) obtained the upper bound of residual in OMP algorithm based on the following fundamental set: We modified the above set to which leads to a more efficient upper bound of resident in OMP algorithm (see inequality (17) of Section 3.3). Second, Livshitz and Temlyakov (2014), Wang and Shim (2016), and Zhang (2011) only use RIP to obtain the lower bound of resident. However, in this paper, we not only used RIP, but also utilized the Wielandt inequality (Wang & Ip, 1999) to derive more efficient lower bound of resident (see inequality (16) of Section 3.3). The details can be found in the following sections.
Let σ = (1 − δ) −1 , thus σ > 1. It can be verified that there exists the maximum value of σ i Hence, we give the following definition.
Definition 3.2: Let k = ∅, integer L(k) is said to be the k − σ character of x if it satisfies the following condition: By Definition 3.2, it is easy to check that Recalling that Based on k − σ character L(k), we define the following integer sequence {κ i } as Obviously, the sequence {κ i } is monotonically nondecreasing, i.e., κ 0 ≤ κ 1 ≤ · · · .

Main idea
Inspired by Wang and Shim (2016) and Zhang (2011), we show that after k iterations, OMP can select a substantial amount of indices in k for a specified number of additional iterations, and the rate of the number of additional iterations to the number of chosen indices is upper bounded by the constant α. More precisely, we give the following important proposition, which is the basis of Theorem 2.1.

Proposition 3.1:
where L(k) is the k − σ character of x. Now, we can prove Theorem 2.1 based on Proposition 3.1. The proof can be divided into two steps.

Sketch of proof of Proposition 3.1
We here give a sketch of the proof of Proposition 3.1, the remaining details of (16) and (17) can be found in the Appendix. For notational simplicity, let k + [α2 L(k) ] = n 1 . From k = ∅, k + [α| k |] ≤ κ, and (8), we have and (10) can be rewritten as By (P.2), a sufficient condition of the above inequality is Now, what remains is the proof of (15), which is based on the analysis of the residual of OMP. First, by exploiting Wielandt inequality and RIP, we construct a lower bound for r n 1 2 2 , that is, r n 1 2 2 ≥ (1 − δ 2 ) n 1 x n 1 2 2 .
Next, by exploiting some properties of orthogonal projection matrix and RIP, we construct an upper bound for r n 1 2 2 , r n 1 2 From (16) and (17), it is easy to verify (15) holds. Hence the remains is the proof of (16) and (17)

Conclusion
In this paper, we analyse the number of iterations required for OMP to exactly recover sparse signals. Our analysis shows that OMP can recover any Ksparse signals in [cK] iterations (c ≥ − 4(1+δ 1 ) 1−δ ln 1−δ 2 ), which is uniformly smaller than the one proposed in Wang and Shim (2016). For example, to accurately recover K-sparse signals, it has been shown in Wang and Shim (2016) that OMP requires 30 K iterations with δ 31 K ≤ 2 −1 while our result shows that OMP requires only [16.8 K] iterations with δ [17.8 K] ≤ 2 −1 . In practical application, a large number of iterations often lead to a high computational complexity. Our result provides a theoretical basis for the reduction of the number of iterations required for OMP.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
The corresponding author gratefully acknowledges support from the National Natural Science  (7) Appendices Appendix 1. Proof of (16) The proof of (16) is based on the Wielandt inequality (Wang & Ip, 1999), which is presented in the following Lemma.

Lemma A.1 (Wielandt inequality): Let A = A 11 A 12
A 21 A 22 be an n-order positive-definite matrix with aI n ≤ A ≤ bI n , (a > 0). Then we have We now proceed to the proof of (16). The conclusion holds naturally if n 1 = ∅. In the following, we prove that (16) still holds under n 1 = ∅. Without loss of generality, we assume that T n 1 = {1, 2, . . . , n 1 }, and n 1 = {n 1 + 1, n 1 + 2, . . . , n 1 + n 2 }, where n 2 = | n 1 |. Notice that (A8) can be proved as follows. It is easy to check that where (a), (b), (c), (d) above can be obtained as follows: (a) is from that fact that P ⊥ T i+1 r i 2 2 + P T i+1 r i 2 2 = r i 2 2 ; (b) is due to t i+1 ∈ T i+1 ; (c) is because is followed from the definition of RIP. Now we construct a lower bound for | r i , φ t i+1 | 2 below. Notice that P ⊥ T i x = 0 for ⊂ T i and T \ k ⊂ T k ⊂ T i . Thus we have From (A10), it follows that Notice that P ⊥ T i r i = r i . From (A11), we have 2Re 2 . (A12) Hence by (A7) and (A12), we have and \ T i = ∅. Now we prove In fact, if i = 0, then T 0 = ∅ and P ⊥ T 0 = I m . From RIP, we have Then from (A12), (A14) and the arithmetic-geometric mean inequality, it can be verified that