Maximum Correntropy Criterion Based l1-Iterative Wiener Filter for Sparse Channel Estimation Robust to Impulsive Noise
Maximum Correntropy Criterion Based l1-Iterative Wiener Filter for Sparse Channel Estimation...
Lim, Junseok
2020-01-21 00:00:00
applied sciences Article Maximum Correntropy Criterion Based l -Iterative Wiener Filter for Sparse Channel Estimation Robust to Impulsive Noise Junseok Lim Department of Electrical Engineering, Sejong University, Seoul 143-747, Korea; jslim@sejong.ac.kr; Tel.: +82-2-3408-3299 Received: 20 December 2019; Accepted: 20 January 2020; Published: 21 January 2020 Abstract: In this paper, we propose a new sparse channel estimator robust to impulsive noise environments. For this kind of estimator, the convex regularized recursive maximum correntropy (CR-RMC) algorithm has been proposed. However, this method requires information about the true sparse channel to find the regularization coecient for the convex regularization penalty term. In addition, the CR-RMC has a numerical instability in the finite-precision cases that is linked to the inversion of the auto-covariance matrix. We propose a new method for sparse channel estimation robust to impulsive noise environments using an iterative Wiener filter. The proposed algorithm does not need information about the true sparse channel to obtain the regularization coecient for the convex regularization penalty term. It is also numerically more robust, because it does not require the inverse of the auto-covariance matrix. Keywords: mathematical models of digital signal processing; digital filtering; maximum correntropy; impulsive noise; sparse channel estimation 1. Introduction In many signal processing applications [1–4], we find various sparse channels in which most of the impulse responses are close to zero and only some of them are large. In recent years, many kinds of sparse adaptive filtering algorithms have been proposed for sparse system estimation, including recursive least squares (RLS)-based [5–9] and least mean square (LMS)-based algorithms [10–14]. It is generally known that RLS-based algorithms have faster convergence and less error after convergence than LMS-based algorithms [15]. However, there are fewer RLS-based than LMS-based algorithms. Among these, the convex regularized recursive least squares (CR-RLS) proposed by Eksioglu [6] is a full recursive convex regularized RLS like a typical RLS. While the aforementioned algorithms typically show good performance in a Gaussian noise environment, their performance deteriorates in a nonGaussian noise environment such as an impulsive noise environment. Recently, the maximum correntropy criterion (MCC) [16–19] has been successfully applied to various adaptive algorithms robust to impulsive noise. Current studies in robust sparse adaptive methods have resulted in the development of CR-RLS-based algorithms with MCC [20,21], and showed strong robustness under impulsive noise. However, CR-RLS used in [20,21] is not practical when determining the regularization coecient for the sparse regularization term because CR-RLS [6] needs information about the true channel when calculating the regularization coecients. In addition, MCC CR-RLS algorithms (so called convex regularized recursive maximum correntropy (CR-RMC)) [20,21] include the inversion of the auto-covariance matrix, which is linked to the numerical instability in finite-precision environments [15]. The recursive inverse (RI) algorithm [22,23] and the iterative Wiener filter (IWF) algorithm [24] have recently been proposed. RI and IWF have the same structure besides a step size calculation. They Appl. Sci. 2020, 10, 743; doi:10.3390/app10030743 www.mdpi.com/journal/applsci Appl. Sci. 2020, 10, 743 2 of 8 perform similarly to the conventional RLS algorithm in terms of convergence and mean squared error, without using the inverse of the auto-covariance matrix. Therefore, RI [22,23] and IWF [24] can be considered algorithms without the numerical instability of RLS. This paper proposes a sparse channel estimation algorithm robust to impulse noise using IWF and maximum correntropy criterion with l -norm regularization. The proposed algorithm includes a new regularization coecient calculation method for l -norm regularization that does not require information about true channels. In addition, the proposed algorithm has numerical stability because it does not include inverse matrix calculation. In Section 2 of this paper, we derive the new algorithm using IWF. In Section 3, we provide simulation results that show the performance of the proposed algorithm. In Section 4, we note our conclusions. 2. MCC l -IWF Formulation In the channel estimation problem, we assume that at time instant n the observed signal y(n) is the result of the input signal x(k) sequence passing through the system w = [w , , w ] in o 0 M 1 the M-dimensional finite impulse response (FIR) format. Especially, in the sparse channel estimation problem, we assume that the system response w is sparse. In the adaptive channel estimation, we apply an M dimensional channel w(k) to the same dimensional signal vector x(k), estimate an output yˆ(k) = x (k)w(k), and calculate the error signal e(k) = y(k) + n(k) yˆ(k) = e y(k) yˆ(k), where y(k) is the output of the actual system, yˆ(k) is the estimated output, and n(k) is the measurement noise. Especially, the measurement noise is nonGaussian. To estimate the channel in nonGaussian noise, we define an MCC cost function with exponential forgetting factor shown in (1) [20,21] and minimize it adaptively. 8 9 0 1 > > > B C > B y(m) w ˆ (n) x(m) C > > B C < = n m B C B C minimize> exp ,s.t. w ˆ (n) c>, (1) B C > > B 2 C 1 > > w ˆ (n) @ 2 A : ; m = 0 T T where w ˆ (n) = [wˆ (n), , wˆ (n)] , x(m) = [x(m), x(m 1), , x(m M + 1)] , is a forgetting M 1 M 1 factor, and
w ˆ (n)
, wˆ (n) . The Lagrangian for (1) becomes k = 0 J(w ˆ (n),
(n)) = (w ˆ (n)) +
(n)
w ˆ (n)
c , (2) y(m) w ˆ (n) x(m) ( ) n m where (w ˆ (n)) = exp , and
(n) is a real-valued Lagrangian multiplier. m = 0 We minimize the regularized cost function to find the optimal vector in the same way that IWF was derived [24]. The regularized cost function is convex and nondierentiable; therefore, subgradient analysis replaces the gradient. When denoting a subgradient vector of f at w ˆ with r f(w ˆ ), the subgradient vector of J(w ˆ (n),
(n)) with respect to w ˆ (n) can be written as follows: s s ( ˆ ) ( ˆ )
ˆ r J w(n),
(n) = r w(n) +
(n)r w(n) . (3) Hence, for the optimal w ˆ (n) minimizing J(w ˆ (n),
(n)), we set the subgradient of J(w ˆ (n),
(n)) to 0 at the optimal point. When evaluating the gradientr(w ˆ (n)), we can derive a gradient vector as (4). r J(w ˆ (n),
(n)) = (F(n)w ˆ (n) r(n)) +
(n)sgn(w ˆ (n)) = g , (4) Appl. Sci. 2020, 10, 743 3 of 8 n 2 e(n) T T T n m where e(n) = y(n) w(n) x(n),F(n) = x(m)x(m) = F(n 1) + exp x(n)x(n) , m = 0 n 2 e(n) n m s ˆ
( ˆ ) r(n) = y(m)x(m) = r(n 1) + exp y(n)x(n), andr w(n) = sgn w(n) [6]. 2 1 m = 0 Using Equation (4), we can obtain the update expression for w ˆ (n) as (5). w ˆ (n + 1) = w ˆ (n) r J(w ˆ (n),
(n)) = w ˆ (n) g (5) n n To get the step size , we find the that minimizes exponentially averaged a posteriori error n n energy, J(w ˆ (n + 1),
(n)), where a posteriori error is e(n) = y(n) w ˆ (n + 1) x(n). T T 1 1 s r J(w ˆ (n + 1),
(n)) = w ˆ (n + 1) F(n)g + r(n) g
(n)r w ˆ (n + 1) g 2 2 n n n T T 1 1 s (6) w ˆ (n + 1) F(n)g + r(n) g
(n)r w ˆ (n) g 2 2 n n n T T T 1 1 = w ˆ (n + 1) F(n)g + r(n) g
(n)sgn(w ˆ (n)) g . 2 2 n n n Substituting Equation (5) into Equation (6), we get 1 1 1 T T T r J(w ˆ (n + 1),
(n)) = w ˆ (n) F(n)g + g F(n)g + r(n) g
(n)sgn(w ˆ (n)) g . (7) n n n n n 2 2 2 To find , we setr J(w ˆ (n),
(n)) = 0, and 1 T T w ˆ (n) F(n) r(n) g +
(n)sgn(w ˆ (n)) g g g 2 n n n n = = (8) 1 T T g F(n)g g F(n)g 2 n n n We have to derive regularization coecient
ˆ(n) such that w ˆ (n + 1) = c, i.e., the l -norm of vector w ˆ (n + 1) is preserved at all time steps of n. This can be represented by a flow equation in continuous time-domain in [25]. 0 1 @ w ˆ (t) B@ w ˆ (t) C B C @w ˆ @w ˆ 1 1 B C s B C = = r w ˆ (t) = 0. (9) B C @ A 1 @t @w ˆ @t @t Using a suciently small interval , the time derivative in (9) can be approximated as T @w ˆ T (w ˆ (n + 1) w ˆ (n)) s s r
w ˆ (t)
r
w ˆ (n)
= 0. (10) 1 1 @t Using (5) and (6), (10) becomes T T sgn(w ˆ (n)) (w ˆ (n + 1) w ˆ (n)) = sgn(w ˆ (n)) ( g ) = 0. (11) and 1 1 sgn(w ˆ (n)) F(n)w(n) r(n) +
(n)sgn(w ˆ (n)) = 0. (12) 2 2 The regularization coecient
ˆ(n) obtained from Equation (12) is as follows. sgn(w ˆ (n)) (F(n)w(n) r(n)) (n) = . (13) sgn(w ˆ (n)) sgn(w ˆ (n)) Appl. Sci. 2020, 10, 743 4 of 8 On the contrary, CR-RMC algorithm in [20] uses the same regularization coecient as that in [6]. The regularization coecient is shown in (14) tr F (n) ( ) w ˆ (n) + +sgn(w ˆ (n)) F (n)"(n) ˆ(n) = 2 , (14) F (n)sgn(w ˆ (n)) where "(n) = w e(n) w ˆ (n) and w e(n) is the solution to the normal equation, F(n)w e(n) = r(n). In (14), the regularization coecient has the parameter, . In [6] and [20], the parameter was set as = f(w ) = kw k , with w indicating the impulse response of the true channel. There was true true 1 true no further discussion about how to set . We summarize the algorithm in Table 1. Table 1. Summary of the l - iterative Wiener filter (IWF). Initialization: F(0),r(0),w ˆ (0), . For n = 1 ::: e(n) F(n) = F(n 1) + exp x(n)x(n) e(n) r(n) = r(n 1) + exp y(n)x(n) sgn(w ˆ (n)) (F(n)w(n) r(n)) (n) = ( ) ( ) sgn w ˆ (n) sgn w ˆ (n) g = F(n)w ˆ (n) r(n) +
(n)sgn(w ˆ (n)) g g n 1 n 1 n T g F(n)g n 1 n 1 w ˆ (n + 1) = w ˆ (n) g end 3. Simulation Results In this section, we compare the sparse channel estimation performance between the proposed algorithm and the convex regularized recursive maximum correntropy (CR-RMC) [20]. In addition, the numerical robustness of the proposed algorithm is compared with that of CR-RMC in the finite-precision environments. 3.1. Estimation of Sparse Channels In this experiment, we showed the sparse system estimation results. The simulation was performed under the same experimental conditions in [6]. The true system parameter w had an order of M = 64. Out of the 64 coecients, there were S nonzero coecients. The nonzero coecients were placed randomly, and the values of the coecients were drawn from a N(0, 1/S) distribution. The impulsive noise is generated according to the Gaussian mixture model [26] 2 2 p = (1 p )N 0, + p N 0, (15) v r r 1 2 2 2 ( ) where N 0, i = 1, 2 denote the Gaussian distribution with zero-mean and variance . The p i i denotes the occurrence probability of the Gaussian distribution with variance , which usually is much larger than so as to generate the impulsive noises. The zero-mean Gaussian distribution with variance generated the background noise, and the zero-mean Gaussian distribution with variance 2 2 2 (usually ) generated the impulsive noise with the probability p . In this experiment, we set 2 2 1 the variance of to 0.01 and generate the input signal so that SNR keeps 20dB. The other parameters were set as = 500 and p = 0.01. We compare CR-RMC [20] using the true system response information and the proposed algorithm using a regularization factor that did not use the true system response. It also included the results of Appl. Sci. 2020, 10, 743 5 of 8 the MCC-RLS [27] and the conventional RLS without considering impulsive noise and sparsity. For the performance evaluation, we simulated the algorithms in the sparse impulse response for S = 4, 8, 16, 32. Figure 1 illustrates the mean standard deviation (MSD) curves. The results show that the estimation performance of the proposed algorithm is similar to that of CR-RMC using the regularization factor referring to the true system impulse response. As expected, the conventional RLS produced the worst MSD in all cases. Figure 1. Steady state MSD for S = 4, 8, 16, 32 (--: the proposed algorithm, --: convex regularized recursive maximum correntropy (CR-RMC), --: maximum correntropy criterion (MCC)- recursive least squares (RLS), solid line: conventional RLS without considering impulsive noise and sparsity): (a) S = 4, (b) S = 8, (c) S = 16, (d) S = 32. Figure 1 confirms that, without a priori information about the true system impulse response, the proposed regularization factor works similarly to that of the regularization factor in CR-RMC using the true system impulse response information. 3.2. Numerical Robustness Experiment In this experiment, we showed the proposed algorithm to be numerically more robust than CR-RMC in the finite-precision environments. We performed channel estimation with finite precision by quantization to show the numerical robustness [28]. The round-o error from the quantization with finite bits was accumulated and propagated through the inverse matrix operation of F(n), and, finally, explosive divergence occurred [15,28]. To illustrate this, we repeated numerical stability experiments, decreasing the quantization bit from 32 bits by 1 bit to find the quantization bits that started numerical instability in each algorithm while comparing and verifying the performance for the case of S = 4 and S = 16. In addition, the rest of the setup for the experiment was the same as Experiment 3.1. Figure 2 shows the results of comparing the performance of the proposed algorithm and CR-RMC in terms of MSD with dierent numbers of quantization bits. Figure 2a,b shows the results when quantized to 32 bits. In this case, we can observe that the proposed algorithm as well as CR-RMC converges normally as Figure 1a,b. Figure 2 shows the results of comparing the performance of the proposed algorithm and CR-RMC in terms of MSD with dierent numbers of quantization bits. Figure 2a,b shows the results when quantized to 32 bits. In this case, we can observe that the proposed algorithm as well as CR-RMC converges normally as Figure 1a,b. Figure 2c,d shows the quantization results for 16 bits. In 16 bits, CR-RMC started numerical instability. Compared with the results of Figure 2a,b, it can be observed that quantized CR-RMC diverges due to the cumulative eect of the error Appl. Sci. 2020, 10, 743 6 of 8 of quantization noise. Figure 2e,f shows the quantization results for 11 bits. In 11 bits, the proposed algorithm also started numerical instability. If we consider the level of quantization error with signal to quantization noise ratio (SQNR), SQNR (dB) = 1.76 + 6.02 bits [29], CR-RMC is stable at above 98.08 dB in SQNR. The proposed algorithm is stable at above 67.98 dB in SQNR. In other word, the proposed algorithm has 30.1 dB gain in numerical stability compared to CR-RMC. Figure 2. Results of numerical robustness experiment (--: the proposed algorithm, -x-: CR-RMC): (a) S = 4 case quantized by 32 bits; (b) S = 16 case quantized by 32 bits; (c) S = 4 case quantized by 16 bits; (d) S = 16 case quantized by 16; (e) S = 4 case quantized by 11 bits; and (f) S = 16 case quantized by 11 bits. The experimental results confirm that the proposed algorithm is numerically more robust than CR-RMC. 4. Conclusions In this paper, this paper have proposed a sparse channel estimation algorithm robust to impulse noise using IWF and MCC with l -norm regularization. The proposed algorithm includes a regularization factor calculation algorithm without any requirement for a priori knowledge about the true system response. The simulation results show that the proposed algorithm works similarly to the CR-RMC algorithm with a regularization factor referring to the true system response information. In addition, simulation results show that the proposed algorithm is more robust against numerical error than the CR-RMC algorithm. Appl. Sci. 2020, 10, 743 7 of 8 Funding: This research received no external funding. Acknowledgments: This paper was supported by the Agency for Defense Development (ADD) in Korea (UD190005DD). Conflicts of Interest: The authors declare no conflict of interest. References 1. Loganathan, P.; Khong, A.W.; Naylor, P.A. A class of sparseness-controlled algorithms for echo cancellation. IEEE Trans. Audio Speech Lang. Process. 2003, 17, 1591–1601. [CrossRef] 2. Carbonelli, C.; Vedantam, S.; Mitra, U. Sparse channel estimation with zero tap detection. IEEE Trans. Wirel. Commun. 2007, 6, 1743–1763. [CrossRef] 3. Yousef, N.R.; Sayed, A.H.; Khajehnouri, N. Detection of fading overlapping multipath components. Signal Process. 2006, 86, 2407–2425. [CrossRef] 4. Singer, A.C.; Nelson, J.K.; Kozat, S.S. Signal processing for underwater acoustic communications. IEEE Commun. Mag. 2009, 47, 90–96. [CrossRef] 5. Babadi, B.; Kalouptsidis, N.; Tarokh, V. SPARLS: The sparse RLS algorithm. IEEE Trans. Signal Process. 2010, 58, 4013–4025. [CrossRef] 6. Eksioglu, E.M. RLS algorithm with convex regularization. IEEE Signal Process. Lett. 2011, 18, 470–473. [CrossRef] 7. Eksioglu, E.M. Sparsity regularised recursive least squares adaptive filtering. IET Signal Process. 2011, 5, 480–487. [CrossRef] 8. Das, B.; Chakraborty, M. Improved l -RLS adaptive filter algorithms. Electron. Lett. 2017, 53, 1650–1651. [CrossRef] 9. Liu, L.; Zhang, Y.; Sun, D. VFF l -norm penalised WL-RLS algorithm using DCD iterations for underwater acoustic communication. IET Commun. 2017, 11, 615–621. [CrossRef] 10. Li, Y.; Wang, Y.; Jiang, T. Norm-adaption penalized least mean square/fourth algorithm for sparse channel estimation. Signal Process. 2016, 128, 243–251. [CrossRef] 11. Jahromi, M.N.S.; Salman, M.S.; Hocanin, A. Convergence analysis of the zero-attracting variable step-size LMS algorithm for sparse system identification. Signal Image Video Process. 2013, 9, 1353–1356. [CrossRef] 12. Li, Y.; Wang, Y.; Jiang, T. Sparse-aware setmembership NLMS algorithms and their application for sparse channel estimation and echo cancelation. AEU-Int. J. Electron. Commun. 2016, 70, 895–902. [CrossRef] 13. Gu, Y.; Jin, J.; Mei, S. l -norm constraint LMS algorithm for sparse system identification. IEEE Signal Process. Lett. 2009, 16, 774–777. 14. Chen, Y.; Gu, Y.; Hero, A.O. Sparse LMS for system identification. In Proceedings of the IEEE International Conference on Acoustics, Speech Signal Processing, Taipei, Taiwan, 19–24 April 2009; pp. 3125–3128. 15. Haykin, S. Adaptive Filter Theory, 5th ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2014. 16. Liu, W.; Pokharel, P.P.; Principe, J.C. Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Trans Signal Process. 2007, 55, 5286–5298. 17. Chen, B.; Xing, L.; Liang, J.; Zheng, N.; Principe, J.C. Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion. IEEE Signal Process. Lett. 2014, 21, 880–884. 18. Ma, W.; Qu, H.; Gui, G.; Xu, L.; Zhao, J.; Chen, B. Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments. J. Frankl. Inst. 2015, 352, 2708–2727. [CrossRef] 19. Chen, B.; Xing, L.; Zhao, H.; Zheng, N.; Principe, J.C. Generalized correntropy for robust adaptive filtering. IEEE Trans. Signal Process. 2016, 64, 3376–3387. [CrossRef] 20. Zhang, X.; Li, K.; Wu, Z.; Fu, Y.; Zhao, H.; Chen, B. Convex regularized recursive maximum correntropy algorithm. Signal Process. 2016, 129, 12–16. [CrossRef] 21. Ma, W.; Duan, J.; Chen, B.; Gui, G.; Man, W. Recursive generalized maximum correntropy criterion algorithm with sparse penalty constraints for system identification. Asian J. Control 2017, 19, 1164–1172. [CrossRef] 22. Ahmad, M.S.; Kukrer, O.; Hocanin, A. Recursive inverse adaptive filtering algorithm. Digit. Signal Process. 2011, 21, 491–496. [CrossRef] 23. Salman, M.S.; Kukrer, O.; Hocanin, A. Recursive inverse algorithm: Mean-square-error analysis. Digit. Signal Process. 2017, 66, 10–17. [CrossRef] Appl. Sci. 2020, 10, 743 8 of 8 24. Xi, B.; Liu, Y. Iterative Wiener Filter. Electron. Lett. 2013, 49, 343–344. [CrossRef] 25. Khalid, S.; Abrar, S. Blind adaptive algorithm for sparse channel equalization using projections onto l -ball. Electron. Lett. 2015, 51, 1422–1424. [CrossRef] 26. Wang, W.; Zhao, J.; Qu, H.; Chen, B. A correntropy inspired variable step-size sign algorithm against impulsive noises. Signal Process. 2017, 141, 168–175. [CrossRef] 27. Ma, W.; Qu, H.; Zhao, J. Estimator with forgetting factor of correntropy and recursive algorithm for trac network prediction. In Proceedings of the 2013 25th Chinese Control and Decision Conference (CCDC), Guiyang, China, 25–27 May 2013; pp. 490–494. 28. Sayed, A.H. Fundamentals of Adaptive Filtering; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2003; pp. 775–803. 29. Proakis, J.G.; Manolakis, D.K. Digital Signal Processing, 4th ed.; Pearson Education Limited: London, UK, 2014; p. 35. © 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png
Applied Sciences
Multidisciplinary Digital Publishing Institute
http://www.deepdyve.com/lp/multidisciplinary-digital-publishing-institute/maximum-correntropy-criterion-based-l1-iterative-wiener-filter-for-h1dRLoCh1l