Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust Atmospheric-Light Estimation

Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust... applied sciences Article Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust Atmospheric-Light Estimation Yutaro Iwamoto, Naoaki Hashimoto and Yen-Wei Chen * College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan; yiwamoto@fc.ritsumei.ac.jp (Y.I.); naoaki111.de@gmail.com (N.H.) * Correspondence: chen@is.ritsumei.ac.jp Received: 16 December 2019; Accepted: 4 February 2020; Published: 9 February 2020 Abstract: This study proposes real-time haze removal from a single image using normalised pixel-wise dark-channel prior (DCP). DCP assumes that at least one RGB colour channel within most local patches in a haze-free image has a low-intensity value. Since the spatial resolution of the transmission map depends on the patch size and it loses the detailed structure with large patch sizes, original work refines the transmission map using an image-matting technique. However, it requires high computational cost and is not adequate for real-time application. To solve these problems, we use normalised pixel-wise haze estimation without losing the detailed structure of the transmission map. This study also proposes robust atmospheric-light estimation using a coarse-to-fine search strategy and down-sampled haze estimation for acceleration. Experiments with actual and simulated haze images showed that the proposed method achieves real-time results of visually and quantitatively acceptable quality compared with other conventional methods of haze removal. Keywords: haze removal; dark channel; atmospheric-light estimation; coarse-to-fine search strategy 1. Introduction In recent years, self-driving vehicles, underwater robots, and remote sensing have attracted attention; such applications employ fast and robust image-recognition techniques. However, images of outdoor or underwater scenes have poor image quality because of haze (Figure 1a), thus affecting image recognition. To solve this problem, many haze removal techniques were proposed, and these techniques can be classified into non-learning-based and learning-based approaches. Non-learning-based approaches use multiple haze images [1], depth information [2] and prior knowledge from a single haze image [3–5]. Methods employing prior knowledge maximise contrast within the local patch [3], assuming that surface shading and transmission are locally uncorrelated [4], and statistically observe that at least one RGB colour channel within most local patches in a haze-free image has a low-intensity value [5]. Median and guided-image filters [6,7] are used for accelerating haze removal; however, these methods could not achieve real-time processing (defined as 20 fps for our calculations herein). Learning-based approaches employ random forest [8], colour-attenuation and prior-based brightness-saturation relation [9] and deep learning [10,11]. These methods can achieve accurate and fast haze removal compared with conventional non-learning-based approaches. In deep-learning-based methods, large-scale pairs of haze images and corresponding haze-free images must be prepared and their relation must be trained. Image pairs of haze and haze-free images cannot be existed simultaneously in actual situation; therefore, haze images are generated from haze-free images by employing haze-observation model [10] and depth information from the corresponding haze-free images [11]. The haze-removal accuracy of deep-learning-based methods depends on the Appl. Sci. 2020, 10, 1165; doi:10.3390/app10031165 www.mdpi.com/journal/applsci Appl. Sci. 2020, 10, 1165 2 of 13 dataset and preparing large datasets is cumbersome. Deep-learning-based methods [10,11] are faster than conventional methods [5,6]; however, computational times of 1.5 s [10] and 0.61 s [11] are required for haze removal of a 640 480 image using 3.4 GHz CPU without GPU acceleration; these methods also could not achieve real-time processing. This study proposes a real-time haze-removal method using a normalised pixel-wise dark-channel prior (DCP) to enable real-time application (Figure 1b,c). This paper is an extended version of [12]. Contributions of the proposed method are as follows: (a) Normalised pixel-wise DCP Original patch-wise DCP method requires high computational cost to refine the transmission map using an image-matting technique. In this paper, we propose a normalised pixel-wise DCP method with no need for refinement of transmission map compared with the patch-wise method. (b) Accelerating haze removal via down-sampling We estimate the transmission map and atmospheric light using down-sampled haze image for acceleration. This idea is inspired by [13]. (c) Robust atmospheric-light estimation To reduce the computational time and improve robustness, we propose a coarse-to-fine search strategy for atmospheric-light estimation. The remainder of this paper is organized as follows. In Section 2, we introduces He et al.’s method [5] in detail because it forms the basis of the proposed method. Section 3 provides the description of proposed method. The experimental results and discussion are reported in Section 4, and conclusion is drawn in Section 5. Figure 1. Examples of original haze image and proposed haze-removal image and transmission map (g = 0.5). 2. Traditional Dark Channel Prior This section describes DCP [5], which is the basis of the proposed method. The haze-observation model [1,5] is represented by I(x) = t(x)J(x) + (1 t(x))A, (1) where I(x) is the observed RGB colour vector of haze image at coordinate x, J(x) is the ideal haze-free image at coordinate x, A is the atmospheric light, t(x) is the value of transmission map at coordinate x. To solve the haze-removal problem, some prior knowledge such as DCP must be applied. The transmission map derivation (Section 2.1), atmospheric-light estimation (Section 2.2) and haze-removal image creation (Section 2.3) are explained as follows. 2.1. Estimation of Transmission Map Medium transmission t(x) [1,5] is expressed by bd(x) t(x) = e , (2) Appl. Sci. 2020, 10, 1165 3 of 13 where b is the scattering coefficient of the atmosphere and d(x) is the depth at coordinate x. He et al. [5] used DCP, indicating that at least one RGB colour channel within most local patch has a low-intensity value DC (J(x)) = min min (J (y)) , (3) y2W(x) c2fr,g,bg where J (y) is a colour channel of haze-free image J(y) at coordinate y and DC is the dark-channel operator which extracts a mimimum RGB colour channel in a local patch W(x) centered at coordinate x. From Equations (1) and (3) can be rewritten as I(x) J(x) ˜ ˜ DC = t(x)DC + (1 t(x)) , (4) A A where t(x) is the coarse transmission map based on patch and the argument I(x)/A and J(x)/A are to be element-wise division. If W(x) is set to large patch size (e.g., 15 15), DC(J(x)/A) should tend to be zero. Finally, the transmission t(x) can be estimated by Equation (5). I(x) t(x) = 1 wDC . (5) where w is the haze removal rate which is considered to the human perception for depth scene (0.95 in ˜ ˜ the He et al. [5]). Since t(x) is calculated by each large patch to satisfy the DCP, t(x) is not smooth in edge region and the spatial resolution is lost. To solve the problem, He et al. [5] refined the transmission map t(x) using image-matting processing [14] as post-processing. However, such processing requires high computational cost and several tens of seconds to execute the haze-removal method. 2.2. Estimation of Atmospheric Light Atmospheric light A comprises pixels of the observed image for which t(x) = 0 in Equation (1); there is no direct light and the distance is infinity in Equation (2). In the outdoor image, this generally represents the intensity of the sky region. To estimate atmospheric light A, the highest luminance value is considered in the haze image I [3]. If an image contains a white object, the atmospheric light A is misestimated, and optimum atmospheric light A is estimated using dark-channel value [5]. Initially He et al. [5] determined the top 0.1% brightest pixels in the dark-channel image, and chose the highest intensity pixels from those same pixels in haze image I. Although this approach is useful because it can estimate atmospheric light A by ignoring small white object, the size is limited below the patch size. 2.3. Estimation of Haze-Removal Image Haze removal can be calculated by modifying Equation (1) as follows: I(x) A J(x) = + A, (6) max t(x), t ( ) where t(x) is the refined transmission map from patch-based transmission map t(x), A is atmospheric light and t is a parameter that is set to 0.1 to avoid division by a small value. 3. Proposed Method Computer vision tasks, such as self-driving vehicles, under-water robots and remote sensing, employ real-time haze removal to realise fast and robust image recognition. In this section, a real-time and highly accurate haze-removal algorithm is proposed. Appl. Sci. 2020, 10, 1165 4 of 13 3.1. Normalized Pixel-Wise Dark Channel Prior In the DCP, the spatial resolution of the transmission map t(x) worsens along object edges because of calculating spatial minimisation in a dark-channel image. Therefore, He et al. [5] refined the transmission map via image-matting processing [14]. However, image-matting processing requires a high computational cost and is not acceptable for real-time application. Therefore, they proposed a guided-image filter [7] as a fast image-matting technique. Other researchers also proposed a pixel-wise DCP [15–17] and a method combining original patch-wise DCP in a flat region and pixel-wise DCP around the edge region [18]. Although pixel-wise DCP can estimate the transmission map t(x) without selecting a minimum value spatially, the result tends to be darker than the haze image (Figure 2b). The histogram of medium transmission t(x) in Figure 3 shows that the pixel-wise DCP without normalisation shifts to the left side compared with the histogram of the original patch-wise DCP (He et al. [5]). This is why the DC(J(x)/A) of Equation (4) cannot be zero by setting the patch size to 1 1 instead of 15 15. Therefore, in the proposed method, the DC(J(x)/A) of Equation (4) has a small value; the value of (DC (x)) is defined by multiplying normalised dark channel of haze image I, which ranges from 0 to 1 and the ratio g in Equation (8). I(x) I (x) DC = min , (7) A A c2fr,g,bg c c I (x) I (y) min min min c c c2fr,g,bg A y2W c2fr,g,bg A DC (x) = g       , (8) c c I (y) I (y) max min min min c c y2W c2fr,g,bg A y2W c2fr,g,bg A where DC is a pixel-wise dark channel operator, W is the entire image. The transmission map t(x) of normalized pixel-wise DCP can be calculated by I(x) 1 wDC t(x) = , (9) 1 DC (x) The histogram (Figure 3) of the transmission map t(x) derived by the proposed method shifts towards the right side compared with the histogram of transmission map without normalisation. As a result, the histogram of the proposed method gets close to the original patch-wise DCP. Here, if g is set to be 0, Equation (9) corresponds to the pixel-wise DCP without normalisation (Figure 2b). Furthermore, setting g to be a small value (e.g., 0.25) results in a dark image within the yellow dotted rectangle (Figure 2c), but if g is set to be a large value (e.g., 0.75), the haze-removal effect diminishes within the dashed red rectangle (Figure 2e). Figure 2. Differences among haze-removal images with each normalisation parameter g. Appl. Sci. 2020, 10, 1165 5 of 13 Figure 3. Histogram of medium transmission with each method. 3.2. Acceleration by Down-Sampling In the haze-removal method, it is necessary to calculate the transmission map t(x) for each pixel. Therefore, the calculation time of haze removal depends on the image size, and thus larger image sizes have higher associated calculation costs. Fortunately, observation of the transmission map t(x) indicates that it is characterised by a relatively low frequency except edges between objects, particularly at different depths. Therefore, we reduced the computation time greatly by down-sampling the input image. It estimated the transmission map t(x) and atmospheric light A using down-sampled image, and then the haze-removal image J was estimated by the up-sampled transmission map t(x) using Equation (6). Figure 4 shows the haze-removal results with different down-sampling ratios. The down-sampling ratio set to 1/4 achieved visually acceptable results, but when it was set to 1/8 or 1/16, halo effects were generated along edges such as along the sides of trees and leaves within the dashed red rectangles (second row of Figure 4e,f). Also, significant aliasing occurred along edges of the bench within the yellow dotted rectangle (third row of Figure 4e,f), and uneven colour occurred in the enhancement results in second row of Figure 4e,f. We therefore set the down-sampling ratio to 1/4 in all further experiments. Also, we used box filtering in down-sampling and bicubic interpolation in up-sampling. Too, acceleration by the down-sampling approach helps with noise suppression by spatial smoothing. Figure 4. Haze image (first row), different haze-removal images (second row), transmission maps (third row) and corresponding down-sampling ratios. 3.3. Robust Atmospheric Light Estimation He et al. [5] estimated atmospheric light A using the original patch-wise DCP, which is a robust method because it ignores small white objects by using a large patch size (e.g., 15 15). In addition, Liu et al.’s method [19] segments the sky region and uses the average value of that region Appl. Sci. 2020, 10, 1165 6 of 13 as atmospheric light A. In our proposed method, because the dark-channel image is calculated for each pixel, white regions (represented by the blue ‘+’ mark in Figure 5) are misinterpreted as atmospheric light A. On the basis of Figure 5, we found that our proposed method cannot use the He et al. [5] approach directly. In addition, their method requires extra computation time for sorting the top 0.1% brightest pixels in the dark channel of haze image I. To solve this problem, we propose a method that robustly estimates atmospheric light A by using a coarse-to-fine search strategy. Figure 6 shows the flow of the coarse-to-fine search strategy. In this strategy, initially the resolution of the dark-channel image is reduced step by step and the position of the largest dark-channel value is obtained at the lowest resolution; next it recalculates the position of the largest dark-channel value in the second-lowest resolution and continues to recalculate the position of the largest dark-channel value until the original image size is attained. In Figure 5, the red ‘’ mark (coarse-to-fine search strategy) is the correctly estimated atmospheric light A. Figure 5. Effectiveness of coarse-to-fine search strategy. Blue ‘+’ mark is result of atmospheric-light estimation by pixel-wise dark-channel image without using coarse-to-fine strategy. Red ‘’ is result of pixel-wise dark-channel image using coarse-to-fine strategy. Figure 6. Flow of coarse-to-fine search strategy for estimting atmospheric light A. 4. Results and Discussion In this section, we compare our method with Tarel et al.’s method [6], He et al.’s method [5] and Cai et al.’s method [10] for qualitative visual evaluation and quantitative evaluation. Ref. [10] is used trained network provided by [20]. We used haze and haze-free images downloaded from the Flickr website [21] (all collected images are public domain or creative commons zero license) and MATLAB source codes [20,22,23]. Initially, we generated five uniform and nonuniform haze images (Figures 7b and 8c) from the haze-free image (Figure 7a) by applying Equation (1). In order to do these simulations, we had to set the transmission map, for which we experimented with setting to uniform and nonuniform medium Appl. Sci. 2020, 10, 1165 7 of 13 transmissions. In the uniform medium transmission t, it is set to 0.5 directly. On the other hand, in the nonuniform medium transmission t(x), we set depth by manually segmenting four to five classes for the each image (Figure 8b) and then fixing the depth for each class; we determined the medium transmission t(x) by applying Equation (2). Figure 7. Comparison of proposed method with conventional method using simulated haze images generated with uniform transmission map (t = 0.5). (a) Original haze-free image, (b) Simulated haze image, (c) Tarel et al. [6], (d) He et al. [5], (e) Cai et al. [10], (f) Pixel-wise DCP without normalisation, (g) Proposed method (g = 0.9). In the quantitative evaluation, peak-signal-to-noise-ratio (PSNR) and structural similarity (SSIM) [24] are calculated H1 W1 K1 MSE = (G(i, j, k) J(i, j, k)) , å å å HWK i=0 j=0 k=0 MAX PSNR = 20 log p , (10) MSE Appl. Sci. 2020, 10, 1165 8 of 13 H1 W1 K1 2m (i, j, k)m (i, j, k) + C 2s (i, j, k) + C G J 1 GJ 2 SSIM = , (11) å å å 2 2 2 2 HWK m (i, j, k) + m (i, j, k) + C s (i, j, k) + s (i, j, k) + C G J 1 G J 2 i=0 j=0 k=0 where H, W and K are image size as height and width and number of colour channel respectively; G and J are the ground-truth image and haze-removal result, respectively; MAX is maximum possible value of ground-truth image; m and m are gaussian weighted averages of G and J, respectively, G J within local patch; within local patch, s and s are standard deviations of G and J, respectively, within G J 2 2 local patch; s is a covariance of G and J within local patch; C (set to 0.01 ) and C (set to 0.03 ) are GJ 1 2 small constants. Secondly, we compared with proposed method and conventional method to actual haze image as qualitative visual evaluation. Finally, we show the comparison of computation time by each method and image size. In common of qualitative evaluation in results with setting the uniform or the nonuniform medium transmission, the results of Figures 7 and 8 show that both our proposed method (Figures 7g and 8h) and He et al.’s method [5] (Figures 7d and 8e) can obtain highly accurate haze-removal images that are indistinguishable from the original haze-free image. Cai et al.’s method [10] can also obtain highly accurate haze-removal images in outdoor scene such as cityscape and landscape images (Figures 7e and 8f). However, Cai et al.’s method [10] cannot remove the haze in underwater scene. The reason is that underwater images are not included in the training data. The results of pixel-wise DCP without normalisation (Figures 7f and 8g) are darker than their original haze-free images (Figure 7a). In the quantitative evaluation with uniform setting in Table 1, it is apparent that our proposed method can obtain the highest PSNR and SSIM values compared with conventional methods if the appropriate value for g is selected. Here, in the case of uniform medium transmission t, c c c c min (min (I (y)/A )) is close to 1 t because min (min (J (y)/A )) is close to 0, y2W y2W c2fr,g,bg c2fr,g,bg c c c c and max (min (I (y)/A )) is close to 1 because max (min (J (y)/A )) is close to y2W c2fr,g,bg y2W c2fr,g,bg 1 in Equation (8). As the result, the appropriate value of g is close to 1 when the w equals to 1. From Table 1, the proposed method can obtain the best results when g is set to a large value. On the other hand, in the quantitative evaluation with nonuniform setting shown in Table 2, some results from He et al.’s method [5] achieved better performance than the proposed method. The main reason is that it is not easy to estimate an appropriate g in the case of nonuniform medium transmission t(x) because it depends on the haze scene. How to automatically determine an appropriate value from the distribution of haze in the scene is our future work. Table 1. Quantitative evaluation with PSNR and SSIM [24] for simulated haze images generated using uniform transmission map (t = 0.5). First row is PSNR value, and second row is SSIM value in each cell. Proposed Method (g) Tarel He Cai et al. [6] et al. [5] et al. [10] 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 13.60 20.58 24.86 11.31 12.25 13.31 14.52 15.94 17.64 19.76 22.57 26.75 34.77 32.62 cityscape 0.842 0.918 0.966 0.623 0.685 0.743 0.796 0.844 0.887 0.925 0.956 0.981 0.996 0.997 10.77 27.22 12.19 15.86 16.86 17.99 19.29 20.80 22.59 24.75 27.32 29.85 30.40 25.78 crab 0.651 0.971 0.705 0.853 0.881 0.905 0.926 0.944 0.958 0.968 0.976 0.979 0.979 0.969 10.69 22.45 17.89 16.43 17.31 18.28 19.37 20.60 22.01 23.65 25.56 27.68 29.56 27.56 coral reef 0.661 0.944 0.825 0.817 0.851 0.881 0.907 0.929 0.946 0.960 0.970 0.976 0.979 0.977 11.41 26.76 23.54 14.99 16.03 17.21 18.57 20.18 22.13 24.52 27.45 30.27 30.15 25.06 landscape1 0.613 0.947 0.895 0.802 0.833 0.860 0.883 0.904 0.922 0.937 0.950 0.959 0.962 0.946 12.08 23.73 20.24 14.43 15.40 16.50 17.78 19.30 21.14 23.47 26.59 31.06 35.53 27.88 landscape2 0.718 0.933 0.884 0.805 0.840 0.870 0.897 0.921 0.941 0.959 0.973 0.982 0.986 0.978 Appl. Sci. 2020, 10, 1165 9 of 13 Figure 8. Comparison of proposed method with conventional method using simulated haze image generated with nonuniform transmission map. (a) Original haze-free image, (b) Manual segmented image, (c) Simulated haze image, (d) Tarel et al. [6], (e) He et al. [5], (f) Cai et al. [10], (g) Pixel-wise DCP without normalisation, (h)Proposed method (g = 0.5). We used the paired t-test to verify whether any performance differences between the proposed method and state-of-the-art methods are statistically significant. The test results are summarized in Table 3. The statistically significant methods (p < 0.05) are indicated by “Yes” and others are indicated by “No”. As shown in Table 3, the proposed method outperformed Tarel et al.’s method [6] and pixel-wise DCP (g = 0) method in both uniform and nonuniform medium transmission cases. On the other hand, the proposed method outperformed He et al.’s method [5] and Cai et al.’s method [10] only in the uniform setting and there are no significant difference in the nonuniform setting. Appl. Sci. 2020, 10, 1165 10 of 13 Table 2. Quantitative evaluation with PSNR and SSIM [24] for simulated haze images generated using nonuniform transmission map. First row is PSNR value, and second row is SSIM value in each cell. Proposed Method (g) Tarel He Cai et al. [6] et al. [5] et al. [10] 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 13.58 22.74 19.88 12.89 14.35 16.08 18.12 20.46 22.66 23.26 21.60 19.20 16.98 15.06 cityscape 0.774 0.941 0.904 0.723 0.799 0.859 0.903 0.934 0.953 0.958 0.948 0.920 0.868 0.789 12.10 27.19 12.99 15.29 16.28 17.37 18.59 19.95 21.43 22.97 24.32 25.01 24.64 22.64 crab 0.702 0.970 0.743 0.834 0.863 0.889 0.909 0.926 0.938 0.946 0.951 0.951 0.947 0.933 11.74 19.78 18.07 16.25 17.22 18.26 19.38 20.53 21.65 22.58 23.11 23.04 22.38 20.83 coral reef 0.691 0.925 0.837 0.807 0.846 0.877 0.901 0.918 0.929 0.935 0.937 0.934 0.927 0.914 14.02 26.47 25.35 15.29 16.63 18.15 19.88 21.74 23.38 24.01 23.14 21.41 19.53 17.60 landscape1 0.667 0.936 0.937 0.81 0.852 0.882 0.902 0.916 0.923 0.925 0.919 0.904 0.874 0.824 14.63 23.60 18.02 15.97 17.23 18.57 19.88 20.90 21.25 20.74 19.61 18.23 16.84 15.54 landscape2 0.742 0.920 0.848 0.813 0.857 0.885 0.902 0.912 0.914 0.908 0.893 0.865 0.822 0.761 Table 3. Paired t-test results between proposed method and conventional methods. The statistically significant methods (p < 0.05) are indicated by “Yes” and others are indicated by “No”. Tarel He Cai pixel-wise DCP et al. [6] et al. [5] et al. [10] w/o normalisation(g = 0 ) Proposed PSNR Yes Yes Yes Yes uniform method(g = 0.9) SSIM Yes Yes Yes Yes Proposed PSNR Yes No No Yes non-uniform method(g = 0.5) SSIM Yes No No Yes Figure 9 shows that our haze-removal method produced good results for processing actual haze images. Closer qualitative evaluation confirms that the images processed by our proposed method (Figure 9f) are visually similar to those obtained by He et al.’s method [5] (Figure 9c). We can see that the results of pixel-wise DCP without normalisation (Figure 9e) are also unnaturally darker than those obtained by He et al.’s method [5] (Figure 9c) and our proposed method (Figure 9f). Furthermore, although Tarel et al.’s method [6] obtained clearer haze-removal results in the pumpkin, bridge and townscape images compared with our proposed method results, our evaluation confirmed that the colours of the park, bridge and townscape images changed from those of the original haze images. We also noted the occurrence of halo effects in the train image. In addition, Tarel et al.’s method cannot work well in the underwater image. Cai et al.’s method [10] (Figure 9d) can remove haze more naturally than other methods. In particular, it can remove haze uniformly in the sky region and the colour is more natural. On the other hand, it cannot work well in the underwater image. Figure 10 shows computation time for each image size for each method, assuming a i7-5557U (3.1 GHz, 2 cores, 4 threads) without GPU acceleration and main memory size is 16 GB. All methods are implemented in MATLAB. Using conventional methods, it takes several tens of seconds, and they cannot achieve real-time calculation. However, our proposed method can achieve real-time calculation until image size exceeds 1024 680 pixels. Appl. Sci. 2020, 10, 1165 11 of 13 Figure 9. Comparison of haze-removal results by our proposed method with conventional methods applied to actual haze images. (a) Haze image, (b) Tarel et al. [6], (c) He et al. [5], (d) Cai et al. [10], (e) Pixel-wise DCP without normalisation, (f) Proposed method (g = 0.5). Figure 10. Comparison of computation time for each image size and each method. 5. Conclusions In this paper, we propose a haze-removal method using a normalised pixel-wise DCP method. We also propose a fast transmission map estimation by down-sampling and robust atmospheric-light estimation using a coarse-to-fine search strategy. Experimental results show that the proposed method can achieve haze removal with acceptable accuracy and greater efficiency than can conventional methods. The advantage of the proposed method is its fast computation with acceptable visual quality compared with state-of-the-art-methods. On the other hand, its disadvantage is that the user must set an appropriate g manually for each different haze scene. How to systematically determine the appropriate g value from the distribution of haze in the scene is our future work. In addition, we are going to apply the method to real applications, such as automatic-driving, underwater-robot and remotely sensed imaging [25]. Appl. Sci. 2020, 10, 1165 12 of 13 Author Contributions: Conceptualization, methodology, software and analysis, Y.I.; investigation and software, N.H.; Conceptualization and validation, Y.-W.C. All authors have read and agree to the published version of the manuscript. Funding: This research received no external funding Acknowledgments: The authors would like to thank Enago (www.enago.jp) for the English language review. Conflicts of Interest: The authors declare no conflict of interest. References 1. Narasimhan, S.G.; Nayar, S.K. Chromatic framework for vision in bad weather. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hilton Head, SC, USA, 13–15 June 2000; Volume 1, pp. 598–605. 2. Kopf, J.; Neubert, B.; Chen, B.; Cohen, M.; Cohen-Or, D.; Deussen, O.; Uyttendaele, M.; Lischinski, D. Deep photo: model-based photograph enhancement and viewing. ACM Trans. Graph. (TOG) 2008, 27, 116 . [CrossRef] 3. Tan, R.T. Visibility in Bad Weather from a Single Image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, 23–28 June 2008; pp. 1–8. 4. Fatal, R. Single Image Dehazing. ACM Trans. Graph. (TOG) 2008, 27. [CrossRef] 5. He, K.; Sun, J.; Tang, X. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33, 2341–2353. [PubMed] 6. Tarel, J.P.; Hautière, N. Fast visibility restoration from a single color or gray level image. In Proceedings of the IEEE 12th International Conference on Computer Vision (ICCV), Kyoto, Japan, 27 September–4 October 2009; pp. 2201–2208. 7. He, K.; Sun, J.; Tang, X. Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 1397–1409. [CrossRef] [PubMed] 8. Tang, K.; Yang, J.; Wang, J. Investigating Haze-relevant Features in A Learning Framework for Image Dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 2995–3000. 9. Zhu, Q.; Mai, J.; Shao, L. A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior. IEEE Trans. Image Process. 2015, 24. [CrossRef] 10. Cai, B.; Xu, X.; Jia, K.; Qing, C.; Tao, D.. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Trans. Image Process. 2016, 25, 5187–5198. [CrossRef] [PubMed] 11. Ren, W.; Liu, S.; Zhang, H.; Pan, J.; Cao, X.; Yang, M.H. Single Image Dehazing via Multi-scale Convolutional Neural Networks. In Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, 11–14 October 2016; pp. 154–169. 12. Iwamoto, Y.; Hashimoto, N.; Chen, Y.W. Fast Dark Channel Prior Based Haze Removal from a Single Image. In Proceedings of the 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD2018), Huangshan, China, 28–30 July 2018. 13. He, K.; Sun, J. Fast Guided Filter. arXiv 2015, arXiv:1505.00996. 14. Levin, A.; Lischinski, D.; Weiss, Y. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 30, 228–242. [CrossRef] [PubMed] 15. Long, J.; Shi, Z.; Tang, W. Fast Haze Removal for a Single Remote Sensing Image Using Dark Channel Prior. In Proceedings of the IEEE 2012 International Conference on Computer Vision in Remote Sensing (CVRS), Xiamen, China, 16–18 December 2012; pp. 132–135. 16. Hsieh, C.H.; Weng, Z.M.; Lin, Y.S. Single image haze removal with pixel-based transmission map estimation. In WSEAS Recent Advances in Information Science; World Scientific: Singerpore, 2016; pp. 121–126. 17. Kotera, H. A color correction for degraded scenes by air pollution. J. Color Sci. Assoc. Jpn. 2016, 40, 49–59. 18. Han, T.; Wan, Y. A fast dark channel prior-based depth map approximation method for dehazing single images. In Proceedings of the IEEE Third International Conference on Information Science and Technology (ICIST), Yangzhou, Jiangsu, China, 23–25 March 2013; pp. 1355–1359. 19. Liu, W.; Chen, X.; Chu, X.; Wu, Y.; Lv, J. Haze removal for a single inland waterway image using sky segmentation and dark channel prior. IET Image Process. 2016, 10, 996–1006. [CrossRef] Appl. Sci. 2020, 10, 1165 13 of 13 20. The Matlab Source Code of Cai’s Method. Available online: https://github.com/caibolun/DehazeNet (accessed on 9 January 2020). 21. Flickr Webpage. Available online: https://www.flickr.com/ (accessed on 25 May 2018). 22. The Matlab Source Code of Tarel’s Method. Available online: http://perso.lcpc.fr/tarel.jean-philippe/ publis/iccv09.html (accessed on 25 May 2018). 23. The Matlab Source Code of He’s Method. Available online: https://github.com/sjtrny/Dark-Channel- Haze-Removal (accessed on 25 May 2018). 24. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: from error visibility to stuctural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [CrossRef] [PubMed] 25. Ahmad, M.; Khan, A.M.; Mazzara, M.; Distefano, S. Multi-layer Extreme Learning Machine-based Autoencoder for Hyperspectral Image Classification, In Proceedings of the the 14th International Conference on Computer Vision Theory and Applications (VISAPP’ 19), Valletta, Malta, 27–29 February 2019; pp. 25–27. c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Sciences Multidisciplinary Digital Publishing Institute

Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust Atmospheric-Light Estimation

Download PDF
13 pages

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/real-time-haze-removal-using-normalised-pixel-wise-dark-channel-prior-jC0Xjulz0w

References (26)

Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2020 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer The statements, opinions and data contained in the journals are solely those of the individual authors and contributors and not of the publisher and the editor(s). Terms and Conditions Privacy Policy
ISSN
2076-3417
DOI
10.3390/app10031165
Publisher site
See Article on Publisher Site

Abstract

applied sciences Article Real-Time Haze Removal Using Normalised Pixel-Wise Dark-Channel Prior and Robust Atmospheric-Light Estimation Yutaro Iwamoto, Naoaki Hashimoto and Yen-Wei Chen * College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan; yiwamoto@fc.ritsumei.ac.jp (Y.I.); naoaki111.de@gmail.com (N.H.) * Correspondence: chen@is.ritsumei.ac.jp Received: 16 December 2019; Accepted: 4 February 2020; Published: 9 February 2020 Abstract: This study proposes real-time haze removal from a single image using normalised pixel-wise dark-channel prior (DCP). DCP assumes that at least one RGB colour channel within most local patches in a haze-free image has a low-intensity value. Since the spatial resolution of the transmission map depends on the patch size and it loses the detailed structure with large patch sizes, original work refines the transmission map using an image-matting technique. However, it requires high computational cost and is not adequate for real-time application. To solve these problems, we use normalised pixel-wise haze estimation without losing the detailed structure of the transmission map. This study also proposes robust atmospheric-light estimation using a coarse-to-fine search strategy and down-sampled haze estimation for acceleration. Experiments with actual and simulated haze images showed that the proposed method achieves real-time results of visually and quantitatively acceptable quality compared with other conventional methods of haze removal. Keywords: haze removal; dark channel; atmospheric-light estimation; coarse-to-fine search strategy 1. Introduction In recent years, self-driving vehicles, underwater robots, and remote sensing have attracted attention; such applications employ fast and robust image-recognition techniques. However, images of outdoor or underwater scenes have poor image quality because of haze (Figure 1a), thus affecting image recognition. To solve this problem, many haze removal techniques were proposed, and these techniques can be classified into non-learning-based and learning-based approaches. Non-learning-based approaches use multiple haze images [1], depth information [2] and prior knowledge from a single haze image [3–5]. Methods employing prior knowledge maximise contrast within the local patch [3], assuming that surface shading and transmission are locally uncorrelated [4], and statistically observe that at least one RGB colour channel within most local patches in a haze-free image has a low-intensity value [5]. Median and guided-image filters [6,7] are used for accelerating haze removal; however, these methods could not achieve real-time processing (defined as 20 fps for our calculations herein). Learning-based approaches employ random forest [8], colour-attenuation and prior-based brightness-saturation relation [9] and deep learning [10,11]. These methods can achieve accurate and fast haze removal compared with conventional non-learning-based approaches. In deep-learning-based methods, large-scale pairs of haze images and corresponding haze-free images must be prepared and their relation must be trained. Image pairs of haze and haze-free images cannot be existed simultaneously in actual situation; therefore, haze images are generated from haze-free images by employing haze-observation model [10] and depth information from the corresponding haze-free images [11]. The haze-removal accuracy of deep-learning-based methods depends on the Appl. Sci. 2020, 10, 1165; doi:10.3390/app10031165 www.mdpi.com/journal/applsci Appl. Sci. 2020, 10, 1165 2 of 13 dataset and preparing large datasets is cumbersome. Deep-learning-based methods [10,11] are faster than conventional methods [5,6]; however, computational times of 1.5 s [10] and 0.61 s [11] are required for haze removal of a 640 480 image using 3.4 GHz CPU without GPU acceleration; these methods also could not achieve real-time processing. This study proposes a real-time haze-removal method using a normalised pixel-wise dark-channel prior (DCP) to enable real-time application (Figure 1b,c). This paper is an extended version of [12]. Contributions of the proposed method are as follows: (a) Normalised pixel-wise DCP Original patch-wise DCP method requires high computational cost to refine the transmission map using an image-matting technique. In this paper, we propose a normalised pixel-wise DCP method with no need for refinement of transmission map compared with the patch-wise method. (b) Accelerating haze removal via down-sampling We estimate the transmission map and atmospheric light using down-sampled haze image for acceleration. This idea is inspired by [13]. (c) Robust atmospheric-light estimation To reduce the computational time and improve robustness, we propose a coarse-to-fine search strategy for atmospheric-light estimation. The remainder of this paper is organized as follows. In Section 2, we introduces He et al.’s method [5] in detail because it forms the basis of the proposed method. Section 3 provides the description of proposed method. The experimental results and discussion are reported in Section 4, and conclusion is drawn in Section 5. Figure 1. Examples of original haze image and proposed haze-removal image and transmission map (g = 0.5). 2. Traditional Dark Channel Prior This section describes DCP [5], which is the basis of the proposed method. The haze-observation model [1,5] is represented by I(x) = t(x)J(x) + (1 t(x))A, (1) where I(x) is the observed RGB colour vector of haze image at coordinate x, J(x) is the ideal haze-free image at coordinate x, A is the atmospheric light, t(x) is the value of transmission map at coordinate x. To solve the haze-removal problem, some prior knowledge such as DCP must be applied. The transmission map derivation (Section 2.1), atmospheric-light estimation (Section 2.2) and haze-removal image creation (Section 2.3) are explained as follows. 2.1. Estimation of Transmission Map Medium transmission t(x) [1,5] is expressed by bd(x) t(x) = e , (2) Appl. Sci. 2020, 10, 1165 3 of 13 where b is the scattering coefficient of the atmosphere and d(x) is the depth at coordinate x. He et al. [5] used DCP, indicating that at least one RGB colour channel within most local patch has a low-intensity value DC (J(x)) = min min (J (y)) , (3) y2W(x) c2fr,g,bg where J (y) is a colour channel of haze-free image J(y) at coordinate y and DC is the dark-channel operator which extracts a mimimum RGB colour channel in a local patch W(x) centered at coordinate x. From Equations (1) and (3) can be rewritten as I(x) J(x) ˜ ˜ DC = t(x)DC + (1 t(x)) , (4) A A where t(x) is the coarse transmission map based on patch and the argument I(x)/A and J(x)/A are to be element-wise division. If W(x) is set to large patch size (e.g., 15 15), DC(J(x)/A) should tend to be zero. Finally, the transmission t(x) can be estimated by Equation (5). I(x) t(x) = 1 wDC . (5) where w is the haze removal rate which is considered to the human perception for depth scene (0.95 in ˜ ˜ the He et al. [5]). Since t(x) is calculated by each large patch to satisfy the DCP, t(x) is not smooth in edge region and the spatial resolution is lost. To solve the problem, He et al. [5] refined the transmission map t(x) using image-matting processing [14] as post-processing. However, such processing requires high computational cost and several tens of seconds to execute the haze-removal method. 2.2. Estimation of Atmospheric Light Atmospheric light A comprises pixels of the observed image for which t(x) = 0 in Equation (1); there is no direct light and the distance is infinity in Equation (2). In the outdoor image, this generally represents the intensity of the sky region. To estimate atmospheric light A, the highest luminance value is considered in the haze image I [3]. If an image contains a white object, the atmospheric light A is misestimated, and optimum atmospheric light A is estimated using dark-channel value [5]. Initially He et al. [5] determined the top 0.1% brightest pixels in the dark-channel image, and chose the highest intensity pixels from those same pixels in haze image I. Although this approach is useful because it can estimate atmospheric light A by ignoring small white object, the size is limited below the patch size. 2.3. Estimation of Haze-Removal Image Haze removal can be calculated by modifying Equation (1) as follows: I(x) A J(x) = + A, (6) max t(x), t ( ) where t(x) is the refined transmission map from patch-based transmission map t(x), A is atmospheric light and t is a parameter that is set to 0.1 to avoid division by a small value. 3. Proposed Method Computer vision tasks, such as self-driving vehicles, under-water robots and remote sensing, employ real-time haze removal to realise fast and robust image recognition. In this section, a real-time and highly accurate haze-removal algorithm is proposed. Appl. Sci. 2020, 10, 1165 4 of 13 3.1. Normalized Pixel-Wise Dark Channel Prior In the DCP, the spatial resolution of the transmission map t(x) worsens along object edges because of calculating spatial minimisation in a dark-channel image. Therefore, He et al. [5] refined the transmission map via image-matting processing [14]. However, image-matting processing requires a high computational cost and is not acceptable for real-time application. Therefore, they proposed a guided-image filter [7] as a fast image-matting technique. Other researchers also proposed a pixel-wise DCP [15–17] and a method combining original patch-wise DCP in a flat region and pixel-wise DCP around the edge region [18]. Although pixel-wise DCP can estimate the transmission map t(x) without selecting a minimum value spatially, the result tends to be darker than the haze image (Figure 2b). The histogram of medium transmission t(x) in Figure 3 shows that the pixel-wise DCP without normalisation shifts to the left side compared with the histogram of the original patch-wise DCP (He et al. [5]). This is why the DC(J(x)/A) of Equation (4) cannot be zero by setting the patch size to 1 1 instead of 15 15. Therefore, in the proposed method, the DC(J(x)/A) of Equation (4) has a small value; the value of (DC (x)) is defined by multiplying normalised dark channel of haze image I, which ranges from 0 to 1 and the ratio g in Equation (8). I(x) I (x) DC = min , (7) A A c2fr,g,bg c c I (x) I (y) min min min c c c2fr,g,bg A y2W c2fr,g,bg A DC (x) = g       , (8) c c I (y) I (y) max min min min c c y2W c2fr,g,bg A y2W c2fr,g,bg A where DC is a pixel-wise dark channel operator, W is the entire image. The transmission map t(x) of normalized pixel-wise DCP can be calculated by I(x) 1 wDC t(x) = , (9) 1 DC (x) The histogram (Figure 3) of the transmission map t(x) derived by the proposed method shifts towards the right side compared with the histogram of transmission map without normalisation. As a result, the histogram of the proposed method gets close to the original patch-wise DCP. Here, if g is set to be 0, Equation (9) corresponds to the pixel-wise DCP without normalisation (Figure 2b). Furthermore, setting g to be a small value (e.g., 0.25) results in a dark image within the yellow dotted rectangle (Figure 2c), but if g is set to be a large value (e.g., 0.75), the haze-removal effect diminishes within the dashed red rectangle (Figure 2e). Figure 2. Differences among haze-removal images with each normalisation parameter g. Appl. Sci. 2020, 10, 1165 5 of 13 Figure 3. Histogram of medium transmission with each method. 3.2. Acceleration by Down-Sampling In the haze-removal method, it is necessary to calculate the transmission map t(x) for each pixel. Therefore, the calculation time of haze removal depends on the image size, and thus larger image sizes have higher associated calculation costs. Fortunately, observation of the transmission map t(x) indicates that it is characterised by a relatively low frequency except edges between objects, particularly at different depths. Therefore, we reduced the computation time greatly by down-sampling the input image. It estimated the transmission map t(x) and atmospheric light A using down-sampled image, and then the haze-removal image J was estimated by the up-sampled transmission map t(x) using Equation (6). Figure 4 shows the haze-removal results with different down-sampling ratios. The down-sampling ratio set to 1/4 achieved visually acceptable results, but when it was set to 1/8 or 1/16, halo effects were generated along edges such as along the sides of trees and leaves within the dashed red rectangles (second row of Figure 4e,f). Also, significant aliasing occurred along edges of the bench within the yellow dotted rectangle (third row of Figure 4e,f), and uneven colour occurred in the enhancement results in second row of Figure 4e,f. We therefore set the down-sampling ratio to 1/4 in all further experiments. Also, we used box filtering in down-sampling and bicubic interpolation in up-sampling. Too, acceleration by the down-sampling approach helps with noise suppression by spatial smoothing. Figure 4. Haze image (first row), different haze-removal images (second row), transmission maps (third row) and corresponding down-sampling ratios. 3.3. Robust Atmospheric Light Estimation He et al. [5] estimated atmospheric light A using the original patch-wise DCP, which is a robust method because it ignores small white objects by using a large patch size (e.g., 15 15). In addition, Liu et al.’s method [19] segments the sky region and uses the average value of that region Appl. Sci. 2020, 10, 1165 6 of 13 as atmospheric light A. In our proposed method, because the dark-channel image is calculated for each pixel, white regions (represented by the blue ‘+’ mark in Figure 5) are misinterpreted as atmospheric light A. On the basis of Figure 5, we found that our proposed method cannot use the He et al. [5] approach directly. In addition, their method requires extra computation time for sorting the top 0.1% brightest pixels in the dark channel of haze image I. To solve this problem, we propose a method that robustly estimates atmospheric light A by using a coarse-to-fine search strategy. Figure 6 shows the flow of the coarse-to-fine search strategy. In this strategy, initially the resolution of the dark-channel image is reduced step by step and the position of the largest dark-channel value is obtained at the lowest resolution; next it recalculates the position of the largest dark-channel value in the second-lowest resolution and continues to recalculate the position of the largest dark-channel value until the original image size is attained. In Figure 5, the red ‘’ mark (coarse-to-fine search strategy) is the correctly estimated atmospheric light A. Figure 5. Effectiveness of coarse-to-fine search strategy. Blue ‘+’ mark is result of atmospheric-light estimation by pixel-wise dark-channel image without using coarse-to-fine strategy. Red ‘’ is result of pixel-wise dark-channel image using coarse-to-fine strategy. Figure 6. Flow of coarse-to-fine search strategy for estimting atmospheric light A. 4. Results and Discussion In this section, we compare our method with Tarel et al.’s method [6], He et al.’s method [5] and Cai et al.’s method [10] for qualitative visual evaluation and quantitative evaluation. Ref. [10] is used trained network provided by [20]. We used haze and haze-free images downloaded from the Flickr website [21] (all collected images are public domain or creative commons zero license) and MATLAB source codes [20,22,23]. Initially, we generated five uniform and nonuniform haze images (Figures 7b and 8c) from the haze-free image (Figure 7a) by applying Equation (1). In order to do these simulations, we had to set the transmission map, for which we experimented with setting to uniform and nonuniform medium Appl. Sci. 2020, 10, 1165 7 of 13 transmissions. In the uniform medium transmission t, it is set to 0.5 directly. On the other hand, in the nonuniform medium transmission t(x), we set depth by manually segmenting four to five classes for the each image (Figure 8b) and then fixing the depth for each class; we determined the medium transmission t(x) by applying Equation (2). Figure 7. Comparison of proposed method with conventional method using simulated haze images generated with uniform transmission map (t = 0.5). (a) Original haze-free image, (b) Simulated haze image, (c) Tarel et al. [6], (d) He et al. [5], (e) Cai et al. [10], (f) Pixel-wise DCP without normalisation, (g) Proposed method (g = 0.9). In the quantitative evaluation, peak-signal-to-noise-ratio (PSNR) and structural similarity (SSIM) [24] are calculated H1 W1 K1 MSE = (G(i, j, k) J(i, j, k)) , å å å HWK i=0 j=0 k=0 MAX PSNR = 20 log p , (10) MSE Appl. Sci. 2020, 10, 1165 8 of 13 H1 W1 K1 2m (i, j, k)m (i, j, k) + C 2s (i, j, k) + C G J 1 GJ 2 SSIM = , (11) å å å 2 2 2 2 HWK m (i, j, k) + m (i, j, k) + C s (i, j, k) + s (i, j, k) + C G J 1 G J 2 i=0 j=0 k=0 where H, W and K are image size as height and width and number of colour channel respectively; G and J are the ground-truth image and haze-removal result, respectively; MAX is maximum possible value of ground-truth image; m and m are gaussian weighted averages of G and J, respectively, G J within local patch; within local patch, s and s are standard deviations of G and J, respectively, within G J 2 2 local patch; s is a covariance of G and J within local patch; C (set to 0.01 ) and C (set to 0.03 ) are GJ 1 2 small constants. Secondly, we compared with proposed method and conventional method to actual haze image as qualitative visual evaluation. Finally, we show the comparison of computation time by each method and image size. In common of qualitative evaluation in results with setting the uniform or the nonuniform medium transmission, the results of Figures 7 and 8 show that both our proposed method (Figures 7g and 8h) and He et al.’s method [5] (Figures 7d and 8e) can obtain highly accurate haze-removal images that are indistinguishable from the original haze-free image. Cai et al.’s method [10] can also obtain highly accurate haze-removal images in outdoor scene such as cityscape and landscape images (Figures 7e and 8f). However, Cai et al.’s method [10] cannot remove the haze in underwater scene. The reason is that underwater images are not included in the training data. The results of pixel-wise DCP without normalisation (Figures 7f and 8g) are darker than their original haze-free images (Figure 7a). In the quantitative evaluation with uniform setting in Table 1, it is apparent that our proposed method can obtain the highest PSNR and SSIM values compared with conventional methods if the appropriate value for g is selected. Here, in the case of uniform medium transmission t, c c c c min (min (I (y)/A )) is close to 1 t because min (min (J (y)/A )) is close to 0, y2W y2W c2fr,g,bg c2fr,g,bg c c c c and max (min (I (y)/A )) is close to 1 because max (min (J (y)/A )) is close to y2W c2fr,g,bg y2W c2fr,g,bg 1 in Equation (8). As the result, the appropriate value of g is close to 1 when the w equals to 1. From Table 1, the proposed method can obtain the best results when g is set to a large value. On the other hand, in the quantitative evaluation with nonuniform setting shown in Table 2, some results from He et al.’s method [5] achieved better performance than the proposed method. The main reason is that it is not easy to estimate an appropriate g in the case of nonuniform medium transmission t(x) because it depends on the haze scene. How to automatically determine an appropriate value from the distribution of haze in the scene is our future work. Table 1. Quantitative evaluation with PSNR and SSIM [24] for simulated haze images generated using uniform transmission map (t = 0.5). First row is PSNR value, and second row is SSIM value in each cell. Proposed Method (g) Tarel He Cai et al. [6] et al. [5] et al. [10] 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 13.60 20.58 24.86 11.31 12.25 13.31 14.52 15.94 17.64 19.76 22.57 26.75 34.77 32.62 cityscape 0.842 0.918 0.966 0.623 0.685 0.743 0.796 0.844 0.887 0.925 0.956 0.981 0.996 0.997 10.77 27.22 12.19 15.86 16.86 17.99 19.29 20.80 22.59 24.75 27.32 29.85 30.40 25.78 crab 0.651 0.971 0.705 0.853 0.881 0.905 0.926 0.944 0.958 0.968 0.976 0.979 0.979 0.969 10.69 22.45 17.89 16.43 17.31 18.28 19.37 20.60 22.01 23.65 25.56 27.68 29.56 27.56 coral reef 0.661 0.944 0.825 0.817 0.851 0.881 0.907 0.929 0.946 0.960 0.970 0.976 0.979 0.977 11.41 26.76 23.54 14.99 16.03 17.21 18.57 20.18 22.13 24.52 27.45 30.27 30.15 25.06 landscape1 0.613 0.947 0.895 0.802 0.833 0.860 0.883 0.904 0.922 0.937 0.950 0.959 0.962 0.946 12.08 23.73 20.24 14.43 15.40 16.50 17.78 19.30 21.14 23.47 26.59 31.06 35.53 27.88 landscape2 0.718 0.933 0.884 0.805 0.840 0.870 0.897 0.921 0.941 0.959 0.973 0.982 0.986 0.978 Appl. Sci. 2020, 10, 1165 9 of 13 Figure 8. Comparison of proposed method with conventional method using simulated haze image generated with nonuniform transmission map. (a) Original haze-free image, (b) Manual segmented image, (c) Simulated haze image, (d) Tarel et al. [6], (e) He et al. [5], (f) Cai et al. [10], (g) Pixel-wise DCP without normalisation, (h)Proposed method (g = 0.5). We used the paired t-test to verify whether any performance differences between the proposed method and state-of-the-art methods are statistically significant. The test results are summarized in Table 3. The statistically significant methods (p < 0.05) are indicated by “Yes” and others are indicated by “No”. As shown in Table 3, the proposed method outperformed Tarel et al.’s method [6] and pixel-wise DCP (g = 0) method in both uniform and nonuniform medium transmission cases. On the other hand, the proposed method outperformed He et al.’s method [5] and Cai et al.’s method [10] only in the uniform setting and there are no significant difference in the nonuniform setting. Appl. Sci. 2020, 10, 1165 10 of 13 Table 2. Quantitative evaluation with PSNR and SSIM [24] for simulated haze images generated using nonuniform transmission map. First row is PSNR value, and second row is SSIM value in each cell. Proposed Method (g) Tarel He Cai et al. [6] et al. [5] et al. [10] 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 13.58 22.74 19.88 12.89 14.35 16.08 18.12 20.46 22.66 23.26 21.60 19.20 16.98 15.06 cityscape 0.774 0.941 0.904 0.723 0.799 0.859 0.903 0.934 0.953 0.958 0.948 0.920 0.868 0.789 12.10 27.19 12.99 15.29 16.28 17.37 18.59 19.95 21.43 22.97 24.32 25.01 24.64 22.64 crab 0.702 0.970 0.743 0.834 0.863 0.889 0.909 0.926 0.938 0.946 0.951 0.951 0.947 0.933 11.74 19.78 18.07 16.25 17.22 18.26 19.38 20.53 21.65 22.58 23.11 23.04 22.38 20.83 coral reef 0.691 0.925 0.837 0.807 0.846 0.877 0.901 0.918 0.929 0.935 0.937 0.934 0.927 0.914 14.02 26.47 25.35 15.29 16.63 18.15 19.88 21.74 23.38 24.01 23.14 21.41 19.53 17.60 landscape1 0.667 0.936 0.937 0.81 0.852 0.882 0.902 0.916 0.923 0.925 0.919 0.904 0.874 0.824 14.63 23.60 18.02 15.97 17.23 18.57 19.88 20.90 21.25 20.74 19.61 18.23 16.84 15.54 landscape2 0.742 0.920 0.848 0.813 0.857 0.885 0.902 0.912 0.914 0.908 0.893 0.865 0.822 0.761 Table 3. Paired t-test results between proposed method and conventional methods. The statistically significant methods (p < 0.05) are indicated by “Yes” and others are indicated by “No”. Tarel He Cai pixel-wise DCP et al. [6] et al. [5] et al. [10] w/o normalisation(g = 0 ) Proposed PSNR Yes Yes Yes Yes uniform method(g = 0.9) SSIM Yes Yes Yes Yes Proposed PSNR Yes No No Yes non-uniform method(g = 0.5) SSIM Yes No No Yes Figure 9 shows that our haze-removal method produced good results for processing actual haze images. Closer qualitative evaluation confirms that the images processed by our proposed method (Figure 9f) are visually similar to those obtained by He et al.’s method [5] (Figure 9c). We can see that the results of pixel-wise DCP without normalisation (Figure 9e) are also unnaturally darker than those obtained by He et al.’s method [5] (Figure 9c) and our proposed method (Figure 9f). Furthermore, although Tarel et al.’s method [6] obtained clearer haze-removal results in the pumpkin, bridge and townscape images compared with our proposed method results, our evaluation confirmed that the colours of the park, bridge and townscape images changed from those of the original haze images. We also noted the occurrence of halo effects in the train image. In addition, Tarel et al.’s method cannot work well in the underwater image. Cai et al.’s method [10] (Figure 9d) can remove haze more naturally than other methods. In particular, it can remove haze uniformly in the sky region and the colour is more natural. On the other hand, it cannot work well in the underwater image. Figure 10 shows computation time for each image size for each method, assuming a i7-5557U (3.1 GHz, 2 cores, 4 threads) without GPU acceleration and main memory size is 16 GB. All methods are implemented in MATLAB. Using conventional methods, it takes several tens of seconds, and they cannot achieve real-time calculation. However, our proposed method can achieve real-time calculation until image size exceeds 1024 680 pixels. Appl. Sci. 2020, 10, 1165 11 of 13 Figure 9. Comparison of haze-removal results by our proposed method with conventional methods applied to actual haze images. (a) Haze image, (b) Tarel et al. [6], (c) He et al. [5], (d) Cai et al. [10], (e) Pixel-wise DCP without normalisation, (f) Proposed method (g = 0.5). Figure 10. Comparison of computation time for each image size and each method. 5. Conclusions In this paper, we propose a haze-removal method using a normalised pixel-wise DCP method. We also propose a fast transmission map estimation by down-sampling and robust atmospheric-light estimation using a coarse-to-fine search strategy. Experimental results show that the proposed method can achieve haze removal with acceptable accuracy and greater efficiency than can conventional methods. The advantage of the proposed method is its fast computation with acceptable visual quality compared with state-of-the-art-methods. On the other hand, its disadvantage is that the user must set an appropriate g manually for each different haze scene. How to systematically determine the appropriate g value from the distribution of haze in the scene is our future work. In addition, we are going to apply the method to real applications, such as automatic-driving, underwater-robot and remotely sensed imaging [25]. Appl. Sci. 2020, 10, 1165 12 of 13 Author Contributions: Conceptualization, methodology, software and analysis, Y.I.; investigation and software, N.H.; Conceptualization and validation, Y.-W.C. All authors have read and agree to the published version of the manuscript. Funding: This research received no external funding Acknowledgments: The authors would like to thank Enago (www.enago.jp) for the English language review. Conflicts of Interest: The authors declare no conflict of interest. References 1. Narasimhan, S.G.; Nayar, S.K. Chromatic framework for vision in bad weather. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hilton Head, SC, USA, 13–15 June 2000; Volume 1, pp. 598–605. 2. Kopf, J.; Neubert, B.; Chen, B.; Cohen, M.; Cohen-Or, D.; Deussen, O.; Uyttendaele, M.; Lischinski, D. Deep photo: model-based photograph enhancement and viewing. ACM Trans. Graph. (TOG) 2008, 27, 116 . [CrossRef] 3. Tan, R.T. Visibility in Bad Weather from a Single Image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, 23–28 June 2008; pp. 1–8. 4. Fatal, R. Single Image Dehazing. ACM Trans. Graph. (TOG) 2008, 27. [CrossRef] 5. He, K.; Sun, J.; Tang, X. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33, 2341–2353. [PubMed] 6. Tarel, J.P.; Hautière, N. Fast visibility restoration from a single color or gray level image. In Proceedings of the IEEE 12th International Conference on Computer Vision (ICCV), Kyoto, Japan, 27 September–4 October 2009; pp. 2201–2208. 7. He, K.; Sun, J.; Tang, X. Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35, 1397–1409. [CrossRef] [PubMed] 8. Tang, K.; Yang, J.; Wang, J. Investigating Haze-relevant Features in A Learning Framework for Image Dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 2995–3000. 9. Zhu, Q.; Mai, J.; Shao, L. A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior. IEEE Trans. Image Process. 2015, 24. [CrossRef] 10. Cai, B.; Xu, X.; Jia, K.; Qing, C.; Tao, D.. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Trans. Image Process. 2016, 25, 5187–5198. [CrossRef] [PubMed] 11. Ren, W.; Liu, S.; Zhang, H.; Pan, J.; Cao, X.; Yang, M.H. Single Image Dehazing via Multi-scale Convolutional Neural Networks. In Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, 11–14 October 2016; pp. 154–169. 12. Iwamoto, Y.; Hashimoto, N.; Chen, Y.W. Fast Dark Channel Prior Based Haze Removal from a Single Image. In Proceedings of the 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD2018), Huangshan, China, 28–30 July 2018. 13. He, K.; Sun, J. Fast Guided Filter. arXiv 2015, arXiv:1505.00996. 14. Levin, A.; Lischinski, D.; Weiss, Y. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 30, 228–242. [CrossRef] [PubMed] 15. Long, J.; Shi, Z.; Tang, W. Fast Haze Removal for a Single Remote Sensing Image Using Dark Channel Prior. In Proceedings of the IEEE 2012 International Conference on Computer Vision in Remote Sensing (CVRS), Xiamen, China, 16–18 December 2012; pp. 132–135. 16. Hsieh, C.H.; Weng, Z.M.; Lin, Y.S. Single image haze removal with pixel-based transmission map estimation. In WSEAS Recent Advances in Information Science; World Scientific: Singerpore, 2016; pp. 121–126. 17. Kotera, H. A color correction for degraded scenes by air pollution. J. Color Sci. Assoc. Jpn. 2016, 40, 49–59. 18. Han, T.; Wan, Y. A fast dark channel prior-based depth map approximation method for dehazing single images. In Proceedings of the IEEE Third International Conference on Information Science and Technology (ICIST), Yangzhou, Jiangsu, China, 23–25 March 2013; pp. 1355–1359. 19. Liu, W.; Chen, X.; Chu, X.; Wu, Y.; Lv, J. Haze removal for a single inland waterway image using sky segmentation and dark channel prior. IET Image Process. 2016, 10, 996–1006. [CrossRef] Appl. Sci. 2020, 10, 1165 13 of 13 20. The Matlab Source Code of Cai’s Method. Available online: https://github.com/caibolun/DehazeNet (accessed on 9 January 2020). 21. Flickr Webpage. Available online: https://www.flickr.com/ (accessed on 25 May 2018). 22. The Matlab Source Code of Tarel’s Method. Available online: http://perso.lcpc.fr/tarel.jean-philippe/ publis/iccv09.html (accessed on 25 May 2018). 23. The Matlab Source Code of He’s Method. Available online: https://github.com/sjtrny/Dark-Channel- Haze-Removal (accessed on 25 May 2018). 24. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: from error visibility to stuctural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [CrossRef] [PubMed] 25. Ahmad, M.; Khan, A.M.; Mazzara, M.; Distefano, S. Multi-layer Extreme Learning Machine-based Autoencoder for Hyperspectral Image Classification, In Proceedings of the the 14th International Conference on Computer Vision Theory and Applications (VISAPP’ 19), Valletta, Malta, 27–29 February 2019; pp. 25–27. c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Journal

Applied Sciences – Multidisciplinary Digital Publishing Institute

Published: Feb 9, 2020

There are no references for this article.