Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling

Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling hv photonics Article Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling 1 1 1 2 1 , 3 , 1 , 3 Ziheng Qiu , Xinyi Guo , Tian’ao Lu , Pan Qi , Zibang Zhang * and Jingang Zhong Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China; qiuziheng@stu2019.jnu.edu.cn (Z.Q.); sydneeguo@stu2016.jnu.edu.cn (X.G.); lutianao1994@stu2017.jnu.edu.cn (T.L.); tzjg@jnu.edu.cn (J.Z.) Department of Electronics Engineering, Guangdong Communication Polytechnic, Guangzhou 510650, China; qiqipan@gdcp.edu.cn Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Jinan University, Guangzhou 510632, China * Correspondence: tzzb@jnu.edu.cn Abstract: Fourier single-pixel imaging (FSI) is a branch of single-pixel imaging techniques. It allows any image to be reconstructed by acquiring its Fourier spectrum by using a single-pixel detector. FSI uses Fourier basis patterns for structured illumination or structured detection to acquire the Fourier spectrum of image. However, the spatial resolution of the reconstructed image mainly depends on the number of Fourier coefficients sampled. The reconstruction of a high-resolution image typically requires a number of Fourier coefficients to be sampled. Consequently, a large number of single- pixel measurements lead to a long data acquisition time, resulting in imaging of a dynamic scene challenging. Here we propose a new sampling strategy for FSI. It allows FSI to reconstruct a clear and sharp image with a reduced number of measurements. The key to the proposed sampling strategy is to perform a density-varying sampling in the Fourier space and, more importantly, the density with respect to the importance of Fourier coefficients is subject to a one-dimensional Gaussian function. The final image is reconstructed from the undersampled Fourier spectrum through compressive sensing. We experimentally demonstrate the proposed method is able to reconstruct a sharp and Citation: Qiu, Z.; Guo, X.; Lu, T.; clear image of 256  256 pixels with a sampling ratio of 10%. The proposed method enables fast Qi, P.; Zhang, Z.; Zhong, J. Efficient single-pixel imaging and provides a new approach for efficient spatial information acquisition. Fourier Single-Pixel Imaging with Gaussian Random Sampling. Keywords: computational imaging; single-pixel imaging; sampling strategy; compressive sens- Photonics 2021, 8, 319. https:// doi.org/10.3390/photonics8080319 ing; optimization Received: 30 June 2021 Accepted: 6 August 2021 Published: 9 August 2021 1. Introduction Single-pixel imaging [1–8] is a computational imaging technique that allows images to Publisher’s Note: MDPI stays neutral be acquired by using a spatially unresolvable detector, namely, single-pixel detector (such with regard to jurisdictional claims in as, photodiode, solar cell, and photomultiplier tube). Compared with typical pixelated published maps and institutional affil- detectors (such as, CCD and CMOS), single-pixel detectors can work at a wide waveband, iations. especially at the wavebands where pixelated detectors are expensive or even technically unavailable (such as infrared, deep ultraviolet, X-ray, or terahertz). Thus, single-pixel imaging has been considered as a potential solution for imaging at special wavebands and attracted a lot of attention in the last decade [9–16]. Copyright: © 2021 by the authors. The key to single-pixel imaging is spatial light modulation. Spatial light modulation Licensee MDPI, Basel, Switzerland. allows the spatial information of the target object to be encoded into a 1-D light signal This article is an open access article sequence, which is suitable for single-pixel detection. The desired object image can be distributed under the terms and retrieved by decoding the spatial information from the resulting single-pixel measurements conditions of the Creative Commons through the image reconstruction algorithm corresponding to the spatial light modula- Attribution (CC BY) license (https:// tion strategy. creativecommons.org/licenses/by/ 4.0/). Photonics 2021, 8, 319. https://doi.org/10.3390/photonics8080319 https://www.mdpi.com/journal/photonics Photonics 2021, 8, 319 2 of 12 Fourier single-pixel imaging (FSI) [17–22] is a branch of basis-scan single-pixel imaging techniques. It uses Fourier basis patterns for spatial light modulation. Fourier basis patterns are also known as sinusoidal intensity patterns. The Fourier spectrum of the desired object image can be acquired by using Fourier basis patterns for structured illumination or structured detection. Compared with other basis-scan single-pixel imaging methods (such as, Hadamard [23–27], DCT [28]), FSI has been proven more data-efficient when the differential method of measurement is employed [29]. Specifically, FSI with 3-step phase shifting can reconstruct a lossless image in a differential-measurement manner taking as many measurements as 1.5 times the number of image pixels. For other basis-scan single-pixel imaging methods, differential measurement will result in the single-pixel measurements being doubled. Moreover, the generation of Fourier basis patterns is flexible. The basis patterns of FSI can be generated by the interference of two plane waves [30], which potentially allows FSI to be implemented without using a pixelated spatial light modulator. Such a property benefits imaging at the wavebands where spatial light modulators are not available. However, as other single-pixel imaging methods do, FSI suffers from the tradeoff between imaging quality and imaging time. The spatial resolution of the image recon- structed by FSI mainly depends on the number of Fourier coefficients sampled. Specifically, it requires more spatial information to reconstruct an image with finer details. The more spatial information implies more single-pixel measurements, and consequently, longer data acquisition time. However, the data acquisition time is crucial for fast imaging, especially when imaging a dynamic scene. Thus, it is worth exploring how to improve the data efficiency in FSI. Initially, FSI was proposed with the spiral sampling strategy [17]. The sampling strat- egy exploits the prior knowledge that most information of natural images is concentrated in low-frequency bands of the Fourier space. According to the spiral sampling strategy, only low-frequency Fourier coefficients will be sampled with high-frequency coefficients discarded. However, the lack of high-frequency components could result in severe ringing artifacts in the reconstructed images, especially when the sampling ratio is low. Later, sev- eral sampling strategies were reported, such as statistical-importance [18], diamond [31], circular [31], and polynomial [32]. Different sampling strategies are referred to different sub-sets and different orderings of the Fourier basis patterns used for spatial light mod- ulation. We note that the research on basis patterns ordering is a hot spot in single-pixel imaging, because an optimal sampling strategy enables images of unchanged quality to be reconstructed from the least single-pixel measurements and therefore shortest data acquisi- tion time. For example, Russian doll [23], cake-cutting [24], origami [25], and total variation ascending orderings [26] were recently proposed for Hadamard single-pixel imaging. Here we propose a sampling strategy for FSI termed Gaussian random sampling. The core of the proposed sampling strategy is to perform a variable density sampling in the Fourier space and the density is based on the importance of Fourier coefficients. Specifically, the sampling density with respect to the importance of Fourier coefficients is subject to a 1-D Gaussian function. The importance of a Fourier coefficient is referred to the magnitude of the modulus of the coefficient. In other words, the larger the modulus of a Fourier coefficient is, the more important this coefficient is. Combined with compressive sensing (CS), the proposed method is able to reconstruct a clear and sharp image from far fewer single-pixel measurements than image pixels. We experimentally demonstrate the proposed method is able to reconstruct a high-quality image of 256  256 pixels with a sampling ratio of 10%. The proposed method enables fast single-pixel imaging and provides a new approach for efficient spatial information acquisition. Photonics 2021, 8, x FOR PEER REVIEW 3 of 13 Photonics 2021, 8, 319 3 of 12 2. Principle A schematic diagram of structured illumination-based FSI set-up is shown in 2. Principle Figure 1a. As the figure shows, a digital micro-mirror device (DMD) is used as the spa- A schematic diagram of structured illumination-based FSI set-up is shown in Figure 1a. tial light modulator to generate Fourier basis patterns. Each Fourier basis pattern can be As the figure shows, a digital micro-mirror device (DMD) is used as the spatial light expressed as modulator to generate Fourier basis patterns. Each Fourier basis pattern can be expressed as  Px,c y =+ ⋅os2πϕ f x+ f y + , () () xy (1)  1 1 P(x, y) = +  cos 2p f x + f y + j , (1) x y 2 2 where xy , denotes the coordinate in the spatial domain, ϕ denotes the initial phase, () and f and f are spatial frequency corresponding to and y direction, respec- where (xx, y) denotes y the coordinate in the spatial domain, j denotes the initial phase, and f and f are spatial frequency corresponding to x and y direction, respectively. As tively. As DMDs are capable of high-speed binary patterns generation, Fourier basis x y DMDs are capable of high-speed binary patterns generation, Fourier basis patterns are patterns are generally binarized through dithering [33], as the inset in Figure 1a shows. generally binarized through dithering [33], as the inset in Figure 1a shows. A photodiode A photodiode amplifier (PDA) is used as the single-pixel detector to collect the amplifier (PDA) is used as the single-pixel detector to collect the back-scattered light from back-scattered light from the object under structured illumination. The Fourier spectrum the object under structured illumination. The Fourier spectrum of the desired object image of the desired object image can be acquired by using the three-step phase-shifting strat- can be acquired by using the three-step phase-shifting strategy, as Figure 1b shows. Each egy, as Figure 1b shows. Each Fourier coefficient, If ,f , is acquired by using a set of () x y Fourier coefficient, I f , f , is acquired by using a set of three Fourier basis patterns of the x y three Fourier basis patterns of the same spatial frequency pair but a different initial same spatial frequency pair but a different initial phase. The initial phase, j , of the i-th phase. The initial phase, ϕ , of the i-th step pattern is 21 i − π 3 . The Fourier coefficient e () step pattern is 2(i 1)p/3. The Fourier coefficient I associated with the spatial frequency f , f can be calculated through I associated with the spatial frequency f , f can be calculated through x y () x y e  I f , f = (2D D D ) + 3j(D D ), (2) If,2 f =− D D−D + 3j D−D , x y() 1() 2 3 () 2 3 (2) xy 12 3 2 3 where j is the imaginary unit, and D denotes the single-pixel measurement correspond- where j is the imaginary unit, and D denotes the single-pixel measurement corre- ing to the i-th step pattern. As Figure 1b shows, the Fourier spectrum of a real-valued sponding to the i-th step pattern. As Figure 1b shows, the Fourier spectrum of a re- image is conjugate symmetric. Thus, the symmetric coefficients need not be sampled. To al-valued image is conjugate symmetric. Thus, the symmetric coefficients need not be reconstruct a lossless image by FSI, the number of Fourier coefficients acquired is one half sampled. To reconstruct a lossless image by FSI, the number of Fourier coefficients ac- of the number of image pixels. If the three-step phase shifting strategy is employed for quired is one half of the number of image pixels. If the three-step phase shifting strategy differential measurement, the number of single-pixel measurements will be 1.5-fold the is employed for differential measurement, the number of single-pixel measurements will number of image pixels. The object image can be reconstructed from the Fourier spectrum be 1.5-fold the number of image pixels. The object image can be reconstructed from the acquired through a 2-D inversed Fourier transform or CS. The proposed method uses CS Fourier spectrum acquired through a 2-D inversed Fourier transform or CS. The pro- for image reconstruction. posed method uses CS for image reconstruction. Figure 1. Illustration of three-step phase shifting FSI. (a) In a structured illumination-based setup, Figure 1. Illustration of three-step phase shifting FSI. (a) In a structured illumination-based setup, the the object is under illumination of Fourier basis patterns generated by a DMD. The Fourier basis object is under illumination of Fourier basis patterns generated by a DMD. The Fourier basis patterns patterns are dithered. (b) The object image is retrieved by acquiring the Fourier spectrum of the are dithered. (b) The object image is retrieved by acquiring the Fourier spectrum of the image. Each image. Each complex-valued Fourier coefficient can be acquired by using a set of three-step complex-valued Fourier coefficient can be acquired by using a set of three-step phase-shifting Fourier phase-shifting Fourier basis patterns where the phase shift is 23 π . The conjugate symmetry of basis patterns where the phase shift is 2p/3. The conjugate symmetry of the Fourier spectrum allows the Fourier spectrum allows a lossless image to be retrieved with only one half of the Fourier coef- a lossless image to be retrieved with only one half of the Fourier coefficients acquired. ficients acquired. The reconstruction of a large-size image requires a large number of single-pixel mea- The reconstruction of a large-size image requires a large number of single-pixel surements, resulting in a long data acquisition time. Undersampling is a typically used measurements, resulting in a long data acquisition time. Undersampling is a typically strategy to reconstruct an image of satisfactory quality with a reduced number of measure- ments. In the context of FSI, undersampling means only a portion of the Fourier spectrum Photonics 2021, 8, 319 4 of 12 is sampled. Inspired by the work by W. Meng et al. [32], we propose a Gaussian random sampling strategy for FSI. The key to the proposed sampling strategy is to perform a variable density sampling in the Fourier space and, more importantly, the density with respect to the importance of Fourier coefficients is subject to a 1-D Gaussian function. In other words, the more important the Fourier coefficient is, the higher probability the coefficient is to be sampled with. The final image is reconstructed from the under-sampled Fourier spectrum through CS. CS is referred to algorithms that can recover certain sparse or compressible signals or images (the length of the signals is M N) from far fewer samples or measurements (the length of the measurements is n and n << M N) than traditional methods (according with Shannon’s theorem) use. How to retrieve the signals from a far small number of sampled data is an ill-posed and under-determined problem. However, CS algorithms can recover sparse solutions by imposing a series of convex-optimization constraints, such as l –norm minimization, greedy algorithm, minimum total variation, etc. However, it is difficult to predict which Fourier coefficients are important for any object or scene to be imaged. Here we adopt a statistics method reported by Bian et al. [18] to derive the importance distribution of coefficients in the Fourier space for reference. Specifically, we use DIV2K database [34], which provides hundreds of high-quality natural images. As Figure 2a shows, we use all 800 natural images from the database and each high- resolution full-color image first is converted into grayscale and segmented to a number of M N-pixel sub images, where M N is the size of the reconstructed image. Then we apply a 2-D Fourier transform to every single sub image and sum up the moduli at the corresponding locations of all resulting Fourier spectra. Lastly, the Fourier coefficients of the summed Fourier spectrum in a size of M N pixels are sorted in a descending order of magnitude. Please note that the conjugate symmetric coefficients are discarded. In our case, M = 256 and N = 256. The number of sorted coefficients is 32,770. Each sorted coefficient has its own index, k, which indicates the importance of the coefficient. The smaller the index is, the higher the importance of the coefficient. Next, as Figure 2b shows, we generate a uniformly distributed random function r(k) whose range is from 0 to 1. We also generate a Gaussian function n o g(k) = exp [(k 1)/k ] /s , (3) max where k is a positive integer denoting the index with descending order of importance, k = 32, 770 when M = 256 and N = 256, and s is the standard deviation of the max Gaussian function. The value of s depends on the sampling ratio h. Here, the sampling ratio is defined as twice the number of sampled Fourier coefficients to the number of total Fourier coefficient in the Fourier spectrum, where “twice” is for the conjugate symmetry. When h < 0.5, there is a simple relation between the standard deviation and the sampling ratio, that is, s = (2h) /p. As indicated by the red lines in Figure 2b, the maxima of the 1-D Gaussian function is at k = 1. If g(k) > r(k), then the k-th Fourier coefficient is marked to be sampled, and vice versa. The resulting sampling masks for different sampling ratios are shown in Figure 2c. Please note that white pixels in the masks indicate the Fourier coefficients at the corresponding locations are to be sampled. We note that such a sampling strategy would result in a few high-importance coef- ficients not being sampled, but adopting a CS algorithm for image reconstruction allows those un-sampled high-importance coefficients to be recovered through optimization. It is because high-importance coefficients are sampled with a high density, which im- poses a strong constraint to find the globally optimized solution for the un-sampled high-importance coefficients. As such, more single-pixel measurements can be spent in sampling the remaining low-importance coefficients and those low-importance coefficients mainly contribute to high-frequency information. Consequently, the spatial resolution of the resulting image is improved. Photonics 2021, 8, 319 5 of 12 Photonics 2021, 8, x FOR PEER REVIEW 5 of 13 Figure 2. The generation of Gaussian random sampling mask by the proposed method. In the first Figure 2. The generation of Gaussian random sampling mask by the proposed method. In the first step (a), the important distribution of coefficients in the Fourier domain is obtained through statis- step (a), the important distribution of coefficients in the Fourier domain is obtained through statistics. tics. Images from a database are decolorized and segmented to M × N pixels. Fourier transform is Images from a database are decolorized and segmented to M N pixels. Fourier transform is applied applied to each segmented sub image and the moduli at the corresponding locations of the result- to each segmented sub image and the moduli at the corresponding locations of the resulting Fourier ing Fourier spectra are summed up. Discarding the conjugate symmetric coefficients in the spectra are summed up. Discarding the conjugate symmetric coefficients in the summed spectrum, summed spectrum, the remaining coefficients are sorted in a descending order. Each sorted coeffi- the remaining coefficients are sorted in a descending order. Each sorted coefficient has its own cient has its own index k , which indicates the importance of the coefficient. The smaller the index index k, which indicates the importance of the coefficient. The smaller the index is, the higher the is, the higher the importance of the coefficient. In the second step (b), a uniformly distributed importance of the coefficient. In the second step (b), a uniformly distributed random function r(k) random function rk and a Gaussian function g k with a specific sampling ratio, η , are () () and a Gaussian function g(k) with a specific sampling ratio, h, are generated. In the third step (c), all generated. In the third step (c), all Fourier coefficients, whose index k satisfies g kr > k , are () () Fourier coefficients, whose index k satisfies g(k) > r(k), are marked as ‘to be sampled’ (white pixel) marked as ‘to be sampled’ (white pixel) in the sampling mask. Filling factor is defined as the ratio in the sampling mask. Filling factor is defined as the ratio of marked coefficients to all coefficients in of marked coefficients to all coefficients in the Fourier space, which is also one half of the sampling the Fourier space, which is also one half of the sampling ratio (due to the conjugate symmetry). ratio (due to the conjugate symmetry). 3. Simulation Each sorted coefficient has its own index, k , which indicates the importance of the The proposed method is first validated by numerical simulations. The simulations coefficient. The smaller the index is, the higher the importance of the coefficient. Next, as are conducted on a desktop computer equipped with an Intel(R) Core(TM) i7-8700K CPU, Figure 2b shows, we generate a uniformly distributed random function whose rk() 16 GB RAM, Windows 10 operating system, and MATLAB 2019a. The CS algorithm we range is from 0 to 1. We also generate a Gaussian function employ for image reconstruction is L1-Magic [35]. To demonstrate the advantages of the proposed sampling strategy, we compare it gk =− exp k− 1 k σ , () ( ) (3) {}  max with another three methods, including radial sampling strategy [36], circular sampling strategy [31], and polynomial sampling [32]. The methods in comparison are either typ- Photonics 2021, 8, 319 6 of 12 ically used or recently proposed. Figure 3 shows the sampling masks generated by the aforementioned strategies for different sampling ratios. In particular, the polynomial sampling strategy requires two user-defined parameters, r and R. The combination of the two parameters, (r, R) is set to be (18, 0.05), (9, 0.05), (7, 0.1), and (5, 0.18) for sampling Photonics 2021, 8, x FOR PEER REVIEW 7 of 13 ratios 1%, 3%, 5%, and 10%, respectively. These parameters settings guarantee the best performance of the polynomial sampling strategy in our case. Figure 3. Sampling masks used in the simulations and the experiments. Figure 3. Sampling masks used in the simulations and the experiments. In the first simulation, a USAF-1951 resolution chart pattern is used as the test im- In the first simulation, a USAF-1951 resolution chart pattern is used as the test image. age. The image is with 256 × 256 pixels. As the results show in Figure 4, the radial sam- The image is with 256  256 pixels. As the results show in Figure 4, the radial sampling pling strategy is not able to reconstruct any bars, when the sampling ratio is below 10%. strategy is not able to reconstruct any bars, when the sampling ratio is below 10%. Even Even when the sampling ratio is 10%, the finest resolvable bars are Group-2 Element 5. In when the sampling ratio is 10%, the finest resolvable bars are Group-2 Element 5. In addition, the circular sampling strategy can successfully reconstruct Group-2 Element 6, addition, the circular sampling strategy can successfully reconstruct Group-2 Element when the sampling ratio is 3%. The polynomial sampling strategy and the proposed 6, when the sampling ratio is 3%. The polynomial sampling strategy and the proposed sampling strategy can even reconstruct Group-1 Element 1 when the sampling ratio is 3%, sampling strategy can even reconstruct Group-1 Element 1 when the sampling ratio is but the image reconstructed by the polynomial sampling strategy appears blurred and 3%, but the image reconstructed by the polynomial sampling strategy appears blurred and smeare smear d. Wh ed. en the sampling r When the sampling atio ratio is 5%, th is 5%, e circular samplin the circular sampling g strategy can strategy on can ly recon- only reconstr struct Group uct Gr- oup-1 1 Element 2, while the Element 2, while the po polynomial lynomial sampling str sampling strategy ategy and and the the pr proposed oposed sampling strategy can well reconstruct Group-1 Element 4, but the image reconstructed sampling strategy can well reconstruct Group-1 Element 4, but the image reconstructed by the polynomial sampling strategy appears a little bit noisy. When the sampling ratio is by the polynomial sampling strategy appears a little bit noisy. When the sampling ratio 10%, is 10the %, the circula circular sampling r sampli strategy ng strategy ca can reconstr n reconstruct Group- uct Group-1 Element 1 Element 5 5. The. polynomial The polyno- sampling strategy and the proposed sampling strategy can well reconstruct Group 0 mial sampling strategy and the proposed sampling strategy can well reconstruct Group 0 Element 2. Element 2. Photonics 2021, 8, 319 7 of 12 Photonics 2021, 8, x FOR PEER REVIEW 8 of 13 Figure 4. Comparison of the reconstruction results of USAF-1951 resolution test chart for different Figure 4. Comparison of the reconstruction results of USAF-1951 resolution test chart for different sampling strategies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted sampling strategies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each reconstruction. by S, R, and T, respectively) are given in the inset of each reconstruction. We also quantitatively evaluate the reconstruction quality by using structural simi- We also quantitatively evaluate the reconstruction quality by using structural similar- larity index (SSIM) [37] and root-mean-square error (RMSE). The SSIM, RMSE, and im- ity index (SSIM) [37] and root-mean-square error (RMSE). The SSIM, RMSE, and image age reconstruction time (denoted by S, R, and T, respectively) are given in the inset of reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each re- each reconstruction. The quantitative measures also demonstrate the proposed sampling construction. The quantitative measures also demonstrate the proposed sampling strategy strategy has better performance than the other sampling strategies in comparison. has better performance than the other sampling strategies in comparison. In the second simulation, we use a natural image—“Cameraman”—for testing. The In the second simulation, we use a natural image—“Cameraman”—for testing. The size of the test image is also 256 × 256 pixels. As the results show in Figure 5, the images size of the test image is also 256  256 pixels. As the results show in Figure 5, the images reconstructed by the radial sampling strategy turn out to be the worst, especially when reconstructed by the radial sampling strategy turn out to be the worst, especially when the the sampling ratio is lower than 10%. The polynomial, the circular, and the proposed sampling ratio is lower than 10%. The polynomial, the circular, and the proposed sampling sampling strategies are capable of reconstructing recognizable contents when the sam- strategies are capable of reconstructing recognizable contents when the sampling ratio is 3%. pling Therat image io is 3%. reconstr The im uctedage by the reconst proposed ructed method by the proposed m appears clearer ethod , which appears clea is evident by rer, the which is ev details (the ident cameraman’s by the details (t face, he c the ame camera, raman’s and fac the e, the camer buildings) a, that and the build are reconstr ings) ucted. that Both are recon the SSIMs structed. Both and RMSEs the SSIM demonstrate s and the RM advantage SEs demoof nstrate the the proposed advantag method. e of the pro- posed method. Photonics 2021, 8, 319 8 of 12 Photonics 2021, 8, x FOR PEER REVIEW 9 of 13 Figure 5. Comparison of the reconstruction results of “Cameraman” for different sampling strate- Figure 5. Comparison of the reconstruction results of “Cameraman” for different sampling strategies gies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each reconstruction. respectively) are given in the inset of each reconstruction. 4. Experiment 4. Experiment The proposed method is also demonstrated with real experiments. The schematic The proposed method is also demonstrated with real experiments. The schematic diagram of the experimental set-up is shown in Figure 1a. The set-up consists of a 12-watt diagram of the experimental set-up is shown in Figure 1a. The set-up consists of a 12-watt white-light LED, a DMD (ViALUX V-7001), an imaging lens, a target object, a collecting white-light LED, a DMD (ViALUX V-7001), an imaging lens, a target object, a collecting lens, and a PDA (Thorlabs PDA101A). Note that we binarize the Fourier basis patterns lens, and a PDA (Thorlabs PDA101A). Note that we binarize the Fourier basis patterns with with the upsample-and-dither strategy [29], so as to take advantage of the high-speed the upsample-and-dither strategy [29], so as to take advantage of the high-speed binary pattern binary pa generation ttern genera offer tion ed of byfe the red by the DMD. The DMD. The patternspa artterns e initially are ini with tial256 ly with 256 256 pixels. × 256 The pixel patterns s. The patterns a are upsampled re upsampl with ed wi a ratio th a of ra 2ti thr o of ough 2 through the bi the bicubic interpolation cubic interpoland ation a then nd binarized using the Floyd–Steinberg algorithm [33]. We use two different scenes for the then binarized using the Floyd–Steinberg algorithm [33]. We use two different scenes for experiment. the experime The nt. The one scene is one scene is a USAF-1951 a USAF-1951 re resolution solution targ target printed et printed on a p on a piece of iA4 ece of A paper4 . The other scene consists of a pair of china dolls as foreground and the printed resolution paper. The other scene consists of a pair of china dolls as foreground and the printed target pattern as background. resolution target pattern as background. Similarly, we compare the proposed Gaussian random sampling strategy with the Similarly, we compare the proposed Gaussian random sampling strategy with the radial, the circular, and the polynomial sampling strategies in the experiments. As the radial, the circular, and the polynomial sampling strategies in the experiments. As the reconstructed images show in Figure 6, the experimental results coincide with the simula- reconstructed images show in Figure 6, the experimental results coincide with the simu- tion results, demonstrating that the proposed sampling strategy allows for better imaging lation results, demonstrating that the proposed sampling strategy allows for better im- quality especially when the sampling ratio is low. Please note that the reconstructed images aging quality especially when the sampling ratio is low. Please note that the recon- presented in Figure 6 are acquired at the DMD rate of 50 Hz. structed images presented in Figure 6 are acquired at the DMD rate of 50 Hz. Photonics 2021, 8, x FOR PEER REVIEW 10 of 13 Photonics 2021, 8, 319 9 of 12 Photonics 2021, 8, x FOR PEER REVIEW 10 of 13 Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing rate is 50 Hz. Scale bars = 5 cm. rate is 50 Hz. Scale bars = 5 cm. rate is 50 Hz. Scale bars = 5 cm. In order to demonstrate the proposed fast single-pixel imaging, we test the pro- In order to demonstrate the proposed fast single-pixel imaging, we test the pro- In order to demonstrate the proposed fast single-pixel imaging, we test the proposed posed method with different DMD refreshing rates. In this test, the sampling ratio is set posed method with different DMD refreshing rates. In this test, the sampling ratio is set method with different DMD refreshing rates. In this test, the sampling ratio is set to 10%, to 10%, and therefore, the number of single-pixel measurements is 9831. As the results to 10%, and therefore, the number of single-pixel measurements is 9831. As the results and therefore, the number of single-pixel measurements is 9831. As the results show show in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without no- show in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without no- in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without noticeable ticeable noise. In other words, the proposed method is able to capture a high-quality ticeable noise. In other words, the proposed method is able to capture a high-quality noise. In other words, the proposed method is able to capture a high-quality image of image of 256 × 256 pixels within 50 s. As the DMD refreshing rate increases, the noise image of 256 × 256 pixels within 50 s. As the DMD refreshing rate increases, the noise 256  256 pixels within 50 s. As the DMD refreshing rate increases, the noise becomes becomes obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is becomes obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is slightly noisy, slightly noisy, but the data acquisition time can be reduced down to 5 s. When the DMD slightly noisy, but the data acquisition time can be reduced down to 5 s. When the DMD but the data acquisition time can be reduced down to 5 s. When the DMD refreshing rate is refreshing rate is 20,000 Hz, the image appears noisy, but the objects in the image are still refreshing rate is 20,000 Hz, the image appears noisy, but the objects in the image are still 20,000 Hz, the image appears noisy, but the objects in the image are still recognizable. recognizable. recognizable. Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing rates. Sampling ratio = 10%. rates. Sampling ratio = 10%. rates. Sampling ratio = 10%. Photonics 2021, 8, 319 10 of 12 5. Discussion The total imaging time in single-pixel imaging includes data acquisition time and the image reconstruction time. This work aims at improving the sampling efficiency in FSI to reduce the data acquisition time. For the purpose of imaging a dynamic scene, a short data acquisition time is desirable, because the data acquisition time in single-pixel imaging is like the exposure time in conventional photography. Severe blur might be caused by the motion of objects if the data acquisition time is long. We note that the proposed sampling strategy requires CS for image reconstruction and CS algorithms are commonly computationally exhausted. In our future work, we consider using deep learning [38–45] to reconstruct the final image from the undersampled Fourier spectrum so as to reduce the image reconstruction time. In this paper, we demonstrate that the proposed Gaussian random sampling strategy can effectively improve the sampling efficiency of FSI. We consider that the proposed sampling strategy is applicable to other basis-scan single-pixel methods. The core of the proposed sampling strategy is to conduct density-varying sampling in an orthogonal transform domain so as to improve sampling efficiency for fast single-pixel imaging. The proposed sampling strategy utilizes the fact that the energy of any natural image concen- trates at the low-frequency band of a certain transform domain. It is such ununiformity of energy distribution that enables density varying sampling. Natural images have sparse rep- resentation in DCT, Hadamard transform, and wavelet transform domains. We therefore consider that the reported sampling strategy can be applied in DCT single-pixel imaging, Hadamard single-pixel imaging, and other basis scan single-pixel imaging methods. In comparison with the polynomial sampling strategy, the proposed method can reproduce better images when the sampling ratio is low. In addition, the proposed method requires no user-defined parameters, which adds flexibility to the method in practical use. 6. Conclusions We propose the Gaussian random sampling strategy to achieve efficient FSI. The key to the proposed sampling strategy is to conduct density-varying sampling in the Fourier domain so as to improve sampling efficiency for fast single-pixel imaging. As is demonstrated by the simulations and the experiments, the proposed method is able to reproduce a sharp and clear image of 256  256 pixels with a sampling ratio of 10%. This work benefits fast single-pixel imaging and provides a new approach for efficient spatial information acquisition. Author Contributions: Conceptualization, Z.Z. and J.Z.; validation Z.Q., X.G., T.L. and P.Q.; funding acquisition, Z.Z. and J.Z.; writing, Z.Z. and Z.Q.; supervision, J.Z. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by National Natural Science Foundation of China (NSFC), (61905098 and 61875074) and Guangzhou Basic and Applied Basic Research Foundation (202002030319 and 202102080589). Data Availability Statement: The data presented in this study are available on request from the corresponding author. Conflicts of Interest: The authors declare no conflict of interest. References 1. Decker, J.A. Hadamard-Transform Image Scanning. Appl. Opt. 1970, 9, 1392–1395. [CrossRef] 2. Edgar, M.; Gibson, G.; Padgett, M. Principles and prospects for single-pixel imaging. Nat. Photonics 2019, 13, 13–20. [CrossRef] 3. Sun, M.; Zhang, J. Single-pixel imaging and its application in three-dimensional reconstruction: A brief review. Sensors 2019, 19, 732. [CrossRef] [PubMed] 4. Gibson, G.; Johnson, S.; Padgett, M. Single-pixel imaging 12 years on: A review. Opt. Express 2020, 28, 28190–28208. [CrossRef] [PubMed] 5. Sun, M.J.; Edgar, M.P.; Gibson, G.M.; Sun, B.Q.; Radwell, N.; Lamb, R.; Padgett, M.J. Single-pixel three-dimensional imaging with time-based depth resolution. Nat. Commun. 2016, 7, 12010. [CrossRef] [PubMed] Photonics 2021, 8, 319 11 of 12 6. Phillips, D.B.; Sun, M.J.; Taylor, J.M.; Edgar, M.P.; Barnett, S.M.; Gibson, G.M.; Padgett, M.J. Adaptive foveated single-pixel imaging with dynamic supersampling. Sci. Adv. 2017, 3, e1601782. [CrossRef] 7. Sun, M.J.; Edgar, M.P.; Phillips, D.B.; Gibson, G.M.; Padgett, M.J. Improving the signal-to-noise ratio of single-pixel imaging using digital microscanning. Opt. Express 2016, 24, 10476–10485. [CrossRef] 8. Xu, Z.H.; Chen, W.; Penuelas, J.; Padgett, M.; Sun, M.J. 1000 fps computational ghost imaging using LED-based structured illumination. Opt. Express 2018, 26, 2427–2434. [CrossRef] 9. Greenberg, J.; Krishnamurthy, K.; Brady, D. Compressive single-pixel snapshot x-ray diffraction imaging. Opt. Lett. 2014, 39, 111–114. [CrossRef] 10. Yu, H.; Lu, R.; Han, S.; Xie, H.; Du, G.; Xiao, T.; Zhu, D. Fourier-transform ghost imaging with hard X rays. Phys. Rev. Lett. 2016, 117, 113901. [CrossRef] [PubMed] 11. Zhang, A.; He, Y.; Wu, L.; Chen, L.; Wang, B. Tabletop x-ray ghost imaging with ultra-low radiation. Optica 2018, 5, 374–377. [CrossRef] 12. Chan, W.; Charan, K.; Takhar, D.; Kelly, K.; Baraniuk, R.; Mittleman, D. A single-pixel terahertz imaging system based on compressed sensing. Appl. Phys. Lett. 2008, 93, 121105. [CrossRef] 13. Shrekenhamer, D.; Watts, C.; Padilla, W. Terahertz single pixel imaging with an optically controlled dynamic spatial light modulator. Opt. Express 2013, 21, 12507–12518. [CrossRef] 14. Hornett, S.; Stantchev, R.; Vardaki, M.; Beckerleg, C.; Hendry, E. Subwavelength terahertz imaging of graphene photoconductivity. Nano Lett. 2016, 16, 7019–7024. [CrossRef] 15. Yu, W.; Liu, X.; Yao, X.; Wang, C.; Zhai, Y.; Zhai, G. Complementary compressive imaging for the telescopic system. Sci. Rep. 2015, 4, 5834. [CrossRef] [PubMed] 16. Studer, V.; Bobin, J.; Chahid, M.; Mousavi, H.; Candes, E.; Dahan, M. Compressive fluorescence microscopy for biological and hyperspectral imaging. Proc. Natl. Acad. Sci. USA 2012, 109, E1679–E1687. [CrossRef] 17. Zhang, Z.; Ma, X.; Zhong, J. Single-pixel imaging by means of Fourier spectrum acquisition. Nat. Commun. 2015, 6, 6225. [CrossRef] 18. Bian, L.; Suo, J.; Hu, X.; Chen, F.; Dai, Q. Efficient single pixel imaging in Fourier space. J. Opt.-UK 2016, 18, 085704. [CrossRef] 19. Jiang, H.; Zhu, S.; Zhao, H.; Xu, B.; Li, X. Adaptive regional single-pixel imaging based on the Fourier slice theorem. Opt. Express 2017, 25, 15118–15130. [CrossRef] 20. Huang, J.; Shi, D.; Yuan, K.; Hu, S.; Wang, Y. Computational-weighted Fourier single-pixel imaging via binary illumination. Opt. Express 2018, 26, 16547–16559. [CrossRef] 21. Xu, B.; Jiang, H.; Zhao, H.; Li, X.; Zhu, S. Projector-defocusing rectification for Fourier single-pixel imaging. Opt. Express 2018, 26, 5005–5017. [CrossRef] 22. Zhang, Z.; Liu, S.; Peng, J.; Yao, M.; Zheng, G.; Zhong, J. Simultaneous spatial, spectral, and 3D compressive imaging via efficient Fourier single-pixel measurements. Optica 2018, 5, 315–319. [CrossRef] 23. Sun, M.; Meng, L.; Edgar, M.; Padgett, M.; Radwell, N. A Russian dolls ordering of the Hadamard basis for compressive single-pixel imaging. Sci. Rep. 2017, 7, 1–7. [CrossRef] 24. Yu, W. Super sub-Nyquist single-pixel imaging by means of cake-cutting Hadamard basis sort. Sensors 2019, 19, 4122. [CrossRef] 25. Yu, W.; Liu, Y. Single-pixel imaging with origami pattern construction. Sensors 2019, 19, 5135. [CrossRef] [PubMed] 26. Yu, X.; Stantchev, R.; Yang, F.; Pickwell-MacPherson, E. Super sub-Nyquist single-pixel imaging by total variation ascending ordering of the Hadamard basis. Sci. Rep. 2020, 10, 9338. [CrossRef] 27. Vaz, P.; Amaral, D.; Requicha Ferreira, L.; Morgado, M.; Cardoso, J. Image quality of compressive single-pixel imaging using different Hadamard orderings. Opt. Express 2020, 28, 11666–11681. [CrossRef] [PubMed] 28. Liu, B.; Yang, Z.; Liu, X.; Wu, L. Coloured computational imaging with single-pixel detectors based on a 2D discrete cosine transform. J. Mod. Opt. 2017, 64, 259–264. [CrossRef] 29. Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Fast Fourier single-pixel imaging via binary illumination. Sci. Rep. 2017, 7, 12029. [CrossRef] [PubMed] 30. Yang, J.; Gong, L.; Xu, X.; Hai, P.; Shen, Y.; Suzuki, Y.; Wang, L. Motionless volumetric photoacoustic microscopy with spatially invariant resolution. Nat. Commun. 2017, 8, 780. [CrossRef] 31. Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Hadamard single-pixel imaging versus Fourier single-pixel imaging. Opt. Express 2017, 25, 19619–19639. [CrossRef] [PubMed] 32. Meng, W.; Shi, D.; Huang, J.; Yuan, K.; Wang, Y.; Fan, C. Sparse Fourier single-pixel imaging. Opt. Express 2019, 7, 31490–31503. 33. Floyd, R.; Steinberg, L. An adaptive algorithm for spatial grey. Proc. Soc. Inf. Disp. 1976, 17, 75–77. 34. Agustsson, E.; Timofte, R. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 21–26 July 2017; pp. 126–135. 35. Candès, E.; Romberg, J. l1-magic: Recovery of Sparse Signals via Convex Programming. Available online: https://statweb. stanford.edu/~{}candes/software/l1magic/downloads/l1magic.pdf (accessed on 1 June 2021). 36. Candès, E.; Romberg, J.; Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 2006, 52, 489–509. [CrossRef] 37. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2014, 13, 600–612. [CrossRef] Photonics 2021, 8, 319 12 of 12 38. Yang, X.; Jiang, P.; Jiang, M.; Xu, L.; Wu, L.; Yang, X.; Zhang, W.; Zhang, J.; Zhang, Y. High imaging quality of Fourier single pixel imaging based on generative adversarial networks at low sampling rate. Opt. Lasers Eng. 2021, 140, 106533. [CrossRef] 39. Higham, C.; Murray-Smith, R.; Padgett, M.; Edgar, M. Deep learning for real-time single-pixel video. Sci. Rep. 2018, 8, 2369. [CrossRef] 40. Jiang, W.; Li, X.; Peng, X.; Sun, B. Imaging high-speed moving targets with a single-pixel detector. Opt. Express 2020, 28, 7889–7897. [CrossRef] [PubMed] 41. Li, F.; Zhao, M.; Tian, Z.; Willomitzer, F.; Cossairt, O. Compressive ghost imaging through scattering media with deep learning. Opt. Express 2020, 28, 17395–17408. [CrossRef] [PubMed] 42. Rizvi, S.; Cao, J.; Hao, Q. Deep learning based projector defocus compensation in single-pixel imaging. Opt. Express 2020, 28, 25134–25148. [CrossRef] 43. Rizvi, S.; Cao, J.; Zhang, K.; Hao, Q. DeepGhost: Real-time computational ghost imaging via deep learning. Sci. Rep. 2020, 10, 11400. [CrossRef] [PubMed] 44. Rizvi, S.; Cao, J.; Zhang, K.; Hao, Q. Improving imaging quality of real-time Fourier single-pixel imaging via deep learning. Sensors 2019, 19, 4190. [CrossRef] [PubMed] 45. Radwell, N.; Johnson, S.; Edgar, M.; Higham, C.; Murray-Smith, R.; Padgett, M. Deep learning optimized single-pixel LiDAR. Appl. Phys. Lett. 2019, 115, 231101. [CrossRef] http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Photonics Multidisciplinary Digital Publishing Institute

Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/efficient-fourier-single-pixel-imaging-with-gaussian-random-sampling-cBuQozQSye
Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2021 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer The statements, opinions and data contained in the journals are solely those of the individual authors and contributors and not of the publisher and the editor(s). MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Terms and Conditions Privacy Policy
ISSN
2304-6732
DOI
10.3390/photonics8080319
Publisher site
See Article on Publisher Site

Abstract

hv photonics Article Efficient Fourier Single-Pixel Imaging with Gaussian Random Sampling 1 1 1 2 1 , 3 , 1 , 3 Ziheng Qiu , Xinyi Guo , Tian’ao Lu , Pan Qi , Zibang Zhang * and Jingang Zhong Department of Optoelectronic Engineering, Jinan University, Guangzhou 510632, China; qiuziheng@stu2019.jnu.edu.cn (Z.Q.); sydneeguo@stu2016.jnu.edu.cn (X.G.); lutianao1994@stu2017.jnu.edu.cn (T.L.); tzjg@jnu.edu.cn (J.Z.) Department of Electronics Engineering, Guangdong Communication Polytechnic, Guangzhou 510650, China; qiqipan@gdcp.edu.cn Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Jinan University, Guangzhou 510632, China * Correspondence: tzzb@jnu.edu.cn Abstract: Fourier single-pixel imaging (FSI) is a branch of single-pixel imaging techniques. It allows any image to be reconstructed by acquiring its Fourier spectrum by using a single-pixel detector. FSI uses Fourier basis patterns for structured illumination or structured detection to acquire the Fourier spectrum of image. However, the spatial resolution of the reconstructed image mainly depends on the number of Fourier coefficients sampled. The reconstruction of a high-resolution image typically requires a number of Fourier coefficients to be sampled. Consequently, a large number of single- pixel measurements lead to a long data acquisition time, resulting in imaging of a dynamic scene challenging. Here we propose a new sampling strategy for FSI. It allows FSI to reconstruct a clear and sharp image with a reduced number of measurements. The key to the proposed sampling strategy is to perform a density-varying sampling in the Fourier space and, more importantly, the density with respect to the importance of Fourier coefficients is subject to a one-dimensional Gaussian function. The final image is reconstructed from the undersampled Fourier spectrum through compressive sensing. We experimentally demonstrate the proposed method is able to reconstruct a sharp and Citation: Qiu, Z.; Guo, X.; Lu, T.; clear image of 256  256 pixels with a sampling ratio of 10%. The proposed method enables fast Qi, P.; Zhang, Z.; Zhong, J. Efficient single-pixel imaging and provides a new approach for efficient spatial information acquisition. Fourier Single-Pixel Imaging with Gaussian Random Sampling. Keywords: computational imaging; single-pixel imaging; sampling strategy; compressive sens- Photonics 2021, 8, 319. https:// doi.org/10.3390/photonics8080319 ing; optimization Received: 30 June 2021 Accepted: 6 August 2021 Published: 9 August 2021 1. Introduction Single-pixel imaging [1–8] is a computational imaging technique that allows images to Publisher’s Note: MDPI stays neutral be acquired by using a spatially unresolvable detector, namely, single-pixel detector (such with regard to jurisdictional claims in as, photodiode, solar cell, and photomultiplier tube). Compared with typical pixelated published maps and institutional affil- detectors (such as, CCD and CMOS), single-pixel detectors can work at a wide waveband, iations. especially at the wavebands where pixelated detectors are expensive or even technically unavailable (such as infrared, deep ultraviolet, X-ray, or terahertz). Thus, single-pixel imaging has been considered as a potential solution for imaging at special wavebands and attracted a lot of attention in the last decade [9–16]. Copyright: © 2021 by the authors. The key to single-pixel imaging is spatial light modulation. Spatial light modulation Licensee MDPI, Basel, Switzerland. allows the spatial information of the target object to be encoded into a 1-D light signal This article is an open access article sequence, which is suitable for single-pixel detection. The desired object image can be distributed under the terms and retrieved by decoding the spatial information from the resulting single-pixel measurements conditions of the Creative Commons through the image reconstruction algorithm corresponding to the spatial light modula- Attribution (CC BY) license (https:// tion strategy. creativecommons.org/licenses/by/ 4.0/). Photonics 2021, 8, 319. https://doi.org/10.3390/photonics8080319 https://www.mdpi.com/journal/photonics Photonics 2021, 8, 319 2 of 12 Fourier single-pixel imaging (FSI) [17–22] is a branch of basis-scan single-pixel imaging techniques. It uses Fourier basis patterns for spatial light modulation. Fourier basis patterns are also known as sinusoidal intensity patterns. The Fourier spectrum of the desired object image can be acquired by using Fourier basis patterns for structured illumination or structured detection. Compared with other basis-scan single-pixel imaging methods (such as, Hadamard [23–27], DCT [28]), FSI has been proven more data-efficient when the differential method of measurement is employed [29]. Specifically, FSI with 3-step phase shifting can reconstruct a lossless image in a differential-measurement manner taking as many measurements as 1.5 times the number of image pixels. For other basis-scan single-pixel imaging methods, differential measurement will result in the single-pixel measurements being doubled. Moreover, the generation of Fourier basis patterns is flexible. The basis patterns of FSI can be generated by the interference of two plane waves [30], which potentially allows FSI to be implemented without using a pixelated spatial light modulator. Such a property benefits imaging at the wavebands where spatial light modulators are not available. However, as other single-pixel imaging methods do, FSI suffers from the tradeoff between imaging quality and imaging time. The spatial resolution of the image recon- structed by FSI mainly depends on the number of Fourier coefficients sampled. Specifically, it requires more spatial information to reconstruct an image with finer details. The more spatial information implies more single-pixel measurements, and consequently, longer data acquisition time. However, the data acquisition time is crucial for fast imaging, especially when imaging a dynamic scene. Thus, it is worth exploring how to improve the data efficiency in FSI. Initially, FSI was proposed with the spiral sampling strategy [17]. The sampling strat- egy exploits the prior knowledge that most information of natural images is concentrated in low-frequency bands of the Fourier space. According to the spiral sampling strategy, only low-frequency Fourier coefficients will be sampled with high-frequency coefficients discarded. However, the lack of high-frequency components could result in severe ringing artifacts in the reconstructed images, especially when the sampling ratio is low. Later, sev- eral sampling strategies were reported, such as statistical-importance [18], diamond [31], circular [31], and polynomial [32]. Different sampling strategies are referred to different sub-sets and different orderings of the Fourier basis patterns used for spatial light mod- ulation. We note that the research on basis patterns ordering is a hot spot in single-pixel imaging, because an optimal sampling strategy enables images of unchanged quality to be reconstructed from the least single-pixel measurements and therefore shortest data acquisi- tion time. For example, Russian doll [23], cake-cutting [24], origami [25], and total variation ascending orderings [26] were recently proposed for Hadamard single-pixel imaging. Here we propose a sampling strategy for FSI termed Gaussian random sampling. The core of the proposed sampling strategy is to perform a variable density sampling in the Fourier space and the density is based on the importance of Fourier coefficients. Specifically, the sampling density with respect to the importance of Fourier coefficients is subject to a 1-D Gaussian function. The importance of a Fourier coefficient is referred to the magnitude of the modulus of the coefficient. In other words, the larger the modulus of a Fourier coefficient is, the more important this coefficient is. Combined with compressive sensing (CS), the proposed method is able to reconstruct a clear and sharp image from far fewer single-pixel measurements than image pixels. We experimentally demonstrate the proposed method is able to reconstruct a high-quality image of 256  256 pixels with a sampling ratio of 10%. The proposed method enables fast single-pixel imaging and provides a new approach for efficient spatial information acquisition. Photonics 2021, 8, x FOR PEER REVIEW 3 of 13 Photonics 2021, 8, 319 3 of 12 2. Principle A schematic diagram of structured illumination-based FSI set-up is shown in 2. Principle Figure 1a. As the figure shows, a digital micro-mirror device (DMD) is used as the spa- A schematic diagram of structured illumination-based FSI set-up is shown in Figure 1a. tial light modulator to generate Fourier basis patterns. Each Fourier basis pattern can be As the figure shows, a digital micro-mirror device (DMD) is used as the spatial light expressed as modulator to generate Fourier basis patterns. Each Fourier basis pattern can be expressed as  Px,c y =+ ⋅os2πϕ f x+ f y + , () () xy (1)  1 1 P(x, y) = +  cos 2p f x + f y + j , (1) x y 2 2 where xy , denotes the coordinate in the spatial domain, ϕ denotes the initial phase, () and f and f are spatial frequency corresponding to and y direction, respec- where (xx, y) denotes y the coordinate in the spatial domain, j denotes the initial phase, and f and f are spatial frequency corresponding to x and y direction, respectively. As tively. As DMDs are capable of high-speed binary patterns generation, Fourier basis x y DMDs are capable of high-speed binary patterns generation, Fourier basis patterns are patterns are generally binarized through dithering [33], as the inset in Figure 1a shows. generally binarized through dithering [33], as the inset in Figure 1a shows. A photodiode A photodiode amplifier (PDA) is used as the single-pixel detector to collect the amplifier (PDA) is used as the single-pixel detector to collect the back-scattered light from back-scattered light from the object under structured illumination. The Fourier spectrum the object under structured illumination. The Fourier spectrum of the desired object image of the desired object image can be acquired by using the three-step phase-shifting strat- can be acquired by using the three-step phase-shifting strategy, as Figure 1b shows. Each egy, as Figure 1b shows. Each Fourier coefficient, If ,f , is acquired by using a set of () x y Fourier coefficient, I f , f , is acquired by using a set of three Fourier basis patterns of the x y three Fourier basis patterns of the same spatial frequency pair but a different initial same spatial frequency pair but a different initial phase. The initial phase, j , of the i-th phase. The initial phase, ϕ , of the i-th step pattern is 21 i − π 3 . The Fourier coefficient e () step pattern is 2(i 1)p/3. The Fourier coefficient I associated with the spatial frequency f , f can be calculated through I associated with the spatial frequency f , f can be calculated through x y () x y e  I f , f = (2D D D ) + 3j(D D ), (2) If,2 f =− D D−D + 3j D−D , x y() 1() 2 3 () 2 3 (2) xy 12 3 2 3 where j is the imaginary unit, and D denotes the single-pixel measurement correspond- where j is the imaginary unit, and D denotes the single-pixel measurement corre- ing to the i-th step pattern. As Figure 1b shows, the Fourier spectrum of a real-valued sponding to the i-th step pattern. As Figure 1b shows, the Fourier spectrum of a re- image is conjugate symmetric. Thus, the symmetric coefficients need not be sampled. To al-valued image is conjugate symmetric. Thus, the symmetric coefficients need not be reconstruct a lossless image by FSI, the number of Fourier coefficients acquired is one half sampled. To reconstruct a lossless image by FSI, the number of Fourier coefficients ac- of the number of image pixels. If the three-step phase shifting strategy is employed for quired is one half of the number of image pixels. If the three-step phase shifting strategy differential measurement, the number of single-pixel measurements will be 1.5-fold the is employed for differential measurement, the number of single-pixel measurements will number of image pixels. The object image can be reconstructed from the Fourier spectrum be 1.5-fold the number of image pixels. The object image can be reconstructed from the acquired through a 2-D inversed Fourier transform or CS. The proposed method uses CS Fourier spectrum acquired through a 2-D inversed Fourier transform or CS. The pro- for image reconstruction. posed method uses CS for image reconstruction. Figure 1. Illustration of three-step phase shifting FSI. (a) In a structured illumination-based setup, Figure 1. Illustration of three-step phase shifting FSI. (a) In a structured illumination-based setup, the the object is under illumination of Fourier basis patterns generated by a DMD. The Fourier basis object is under illumination of Fourier basis patterns generated by a DMD. The Fourier basis patterns patterns are dithered. (b) The object image is retrieved by acquiring the Fourier spectrum of the are dithered. (b) The object image is retrieved by acquiring the Fourier spectrum of the image. Each image. Each complex-valued Fourier coefficient can be acquired by using a set of three-step complex-valued Fourier coefficient can be acquired by using a set of three-step phase-shifting Fourier phase-shifting Fourier basis patterns where the phase shift is 23 π . The conjugate symmetry of basis patterns where the phase shift is 2p/3. The conjugate symmetry of the Fourier spectrum allows the Fourier spectrum allows a lossless image to be retrieved with only one half of the Fourier coef- a lossless image to be retrieved with only one half of the Fourier coefficients acquired. ficients acquired. The reconstruction of a large-size image requires a large number of single-pixel mea- The reconstruction of a large-size image requires a large number of single-pixel surements, resulting in a long data acquisition time. Undersampling is a typically used measurements, resulting in a long data acquisition time. Undersampling is a typically strategy to reconstruct an image of satisfactory quality with a reduced number of measure- ments. In the context of FSI, undersampling means only a portion of the Fourier spectrum Photonics 2021, 8, 319 4 of 12 is sampled. Inspired by the work by W. Meng et al. [32], we propose a Gaussian random sampling strategy for FSI. The key to the proposed sampling strategy is to perform a variable density sampling in the Fourier space and, more importantly, the density with respect to the importance of Fourier coefficients is subject to a 1-D Gaussian function. In other words, the more important the Fourier coefficient is, the higher probability the coefficient is to be sampled with. The final image is reconstructed from the under-sampled Fourier spectrum through CS. CS is referred to algorithms that can recover certain sparse or compressible signals or images (the length of the signals is M N) from far fewer samples or measurements (the length of the measurements is n and n << M N) than traditional methods (according with Shannon’s theorem) use. How to retrieve the signals from a far small number of sampled data is an ill-posed and under-determined problem. However, CS algorithms can recover sparse solutions by imposing a series of convex-optimization constraints, such as l –norm minimization, greedy algorithm, minimum total variation, etc. However, it is difficult to predict which Fourier coefficients are important for any object or scene to be imaged. Here we adopt a statistics method reported by Bian et al. [18] to derive the importance distribution of coefficients in the Fourier space for reference. Specifically, we use DIV2K database [34], which provides hundreds of high-quality natural images. As Figure 2a shows, we use all 800 natural images from the database and each high- resolution full-color image first is converted into grayscale and segmented to a number of M N-pixel sub images, where M N is the size of the reconstructed image. Then we apply a 2-D Fourier transform to every single sub image and sum up the moduli at the corresponding locations of all resulting Fourier spectra. Lastly, the Fourier coefficients of the summed Fourier spectrum in a size of M N pixels are sorted in a descending order of magnitude. Please note that the conjugate symmetric coefficients are discarded. In our case, M = 256 and N = 256. The number of sorted coefficients is 32,770. Each sorted coefficient has its own index, k, which indicates the importance of the coefficient. The smaller the index is, the higher the importance of the coefficient. Next, as Figure 2b shows, we generate a uniformly distributed random function r(k) whose range is from 0 to 1. We also generate a Gaussian function n o g(k) = exp [(k 1)/k ] /s , (3) max where k is a positive integer denoting the index with descending order of importance, k = 32, 770 when M = 256 and N = 256, and s is the standard deviation of the max Gaussian function. The value of s depends on the sampling ratio h. Here, the sampling ratio is defined as twice the number of sampled Fourier coefficients to the number of total Fourier coefficient in the Fourier spectrum, where “twice” is for the conjugate symmetry. When h < 0.5, there is a simple relation between the standard deviation and the sampling ratio, that is, s = (2h) /p. As indicated by the red lines in Figure 2b, the maxima of the 1-D Gaussian function is at k = 1. If g(k) > r(k), then the k-th Fourier coefficient is marked to be sampled, and vice versa. The resulting sampling masks for different sampling ratios are shown in Figure 2c. Please note that white pixels in the masks indicate the Fourier coefficients at the corresponding locations are to be sampled. We note that such a sampling strategy would result in a few high-importance coef- ficients not being sampled, but adopting a CS algorithm for image reconstruction allows those un-sampled high-importance coefficients to be recovered through optimization. It is because high-importance coefficients are sampled with a high density, which im- poses a strong constraint to find the globally optimized solution for the un-sampled high-importance coefficients. As such, more single-pixel measurements can be spent in sampling the remaining low-importance coefficients and those low-importance coefficients mainly contribute to high-frequency information. Consequently, the spatial resolution of the resulting image is improved. Photonics 2021, 8, 319 5 of 12 Photonics 2021, 8, x FOR PEER REVIEW 5 of 13 Figure 2. The generation of Gaussian random sampling mask by the proposed method. In the first Figure 2. The generation of Gaussian random sampling mask by the proposed method. In the first step (a), the important distribution of coefficients in the Fourier domain is obtained through statis- step (a), the important distribution of coefficients in the Fourier domain is obtained through statistics. tics. Images from a database are decolorized and segmented to M × N pixels. Fourier transform is Images from a database are decolorized and segmented to M N pixels. Fourier transform is applied applied to each segmented sub image and the moduli at the corresponding locations of the result- to each segmented sub image and the moduli at the corresponding locations of the resulting Fourier ing Fourier spectra are summed up. Discarding the conjugate symmetric coefficients in the spectra are summed up. Discarding the conjugate symmetric coefficients in the summed spectrum, summed spectrum, the remaining coefficients are sorted in a descending order. Each sorted coeffi- the remaining coefficients are sorted in a descending order. Each sorted coefficient has its own cient has its own index k , which indicates the importance of the coefficient. The smaller the index index k, which indicates the importance of the coefficient. The smaller the index is, the higher the is, the higher the importance of the coefficient. In the second step (b), a uniformly distributed importance of the coefficient. In the second step (b), a uniformly distributed random function r(k) random function rk and a Gaussian function g k with a specific sampling ratio, η , are () () and a Gaussian function g(k) with a specific sampling ratio, h, are generated. In the third step (c), all generated. In the third step (c), all Fourier coefficients, whose index k satisfies g kr > k , are () () Fourier coefficients, whose index k satisfies g(k) > r(k), are marked as ‘to be sampled’ (white pixel) marked as ‘to be sampled’ (white pixel) in the sampling mask. Filling factor is defined as the ratio in the sampling mask. Filling factor is defined as the ratio of marked coefficients to all coefficients in of marked coefficients to all coefficients in the Fourier space, which is also one half of the sampling the Fourier space, which is also one half of the sampling ratio (due to the conjugate symmetry). ratio (due to the conjugate symmetry). 3. Simulation Each sorted coefficient has its own index, k , which indicates the importance of the The proposed method is first validated by numerical simulations. The simulations coefficient. The smaller the index is, the higher the importance of the coefficient. Next, as are conducted on a desktop computer equipped with an Intel(R) Core(TM) i7-8700K CPU, Figure 2b shows, we generate a uniformly distributed random function whose rk() 16 GB RAM, Windows 10 operating system, and MATLAB 2019a. The CS algorithm we range is from 0 to 1. We also generate a Gaussian function employ for image reconstruction is L1-Magic [35]. To demonstrate the advantages of the proposed sampling strategy, we compare it gk =− exp k− 1 k σ , () ( ) (3) {}  max with another three methods, including radial sampling strategy [36], circular sampling strategy [31], and polynomial sampling [32]. The methods in comparison are either typ- Photonics 2021, 8, 319 6 of 12 ically used or recently proposed. Figure 3 shows the sampling masks generated by the aforementioned strategies for different sampling ratios. In particular, the polynomial sampling strategy requires two user-defined parameters, r and R. The combination of the two parameters, (r, R) is set to be (18, 0.05), (9, 0.05), (7, 0.1), and (5, 0.18) for sampling Photonics 2021, 8, x FOR PEER REVIEW 7 of 13 ratios 1%, 3%, 5%, and 10%, respectively. These parameters settings guarantee the best performance of the polynomial sampling strategy in our case. Figure 3. Sampling masks used in the simulations and the experiments. Figure 3. Sampling masks used in the simulations and the experiments. In the first simulation, a USAF-1951 resolution chart pattern is used as the test im- In the first simulation, a USAF-1951 resolution chart pattern is used as the test image. age. The image is with 256 × 256 pixels. As the results show in Figure 4, the radial sam- The image is with 256  256 pixels. As the results show in Figure 4, the radial sampling pling strategy is not able to reconstruct any bars, when the sampling ratio is below 10%. strategy is not able to reconstruct any bars, when the sampling ratio is below 10%. Even Even when the sampling ratio is 10%, the finest resolvable bars are Group-2 Element 5. In when the sampling ratio is 10%, the finest resolvable bars are Group-2 Element 5. In addition, the circular sampling strategy can successfully reconstruct Group-2 Element 6, addition, the circular sampling strategy can successfully reconstruct Group-2 Element when the sampling ratio is 3%. The polynomial sampling strategy and the proposed 6, when the sampling ratio is 3%. The polynomial sampling strategy and the proposed sampling strategy can even reconstruct Group-1 Element 1 when the sampling ratio is 3%, sampling strategy can even reconstruct Group-1 Element 1 when the sampling ratio is but the image reconstructed by the polynomial sampling strategy appears blurred and 3%, but the image reconstructed by the polynomial sampling strategy appears blurred and smeare smear d. Wh ed. en the sampling r When the sampling atio ratio is 5%, th is 5%, e circular samplin the circular sampling g strategy can strategy on can ly recon- only reconstr struct Group uct Gr- oup-1 1 Element 2, while the Element 2, while the po polynomial lynomial sampling str sampling strategy ategy and and the the pr proposed oposed sampling strategy can well reconstruct Group-1 Element 4, but the image reconstructed sampling strategy can well reconstruct Group-1 Element 4, but the image reconstructed by the polynomial sampling strategy appears a little bit noisy. When the sampling ratio is by the polynomial sampling strategy appears a little bit noisy. When the sampling ratio 10%, is 10the %, the circula circular sampling r sampli strategy ng strategy ca can reconstr n reconstruct Group- uct Group-1 Element 1 Element 5 5. The. polynomial The polyno- sampling strategy and the proposed sampling strategy can well reconstruct Group 0 mial sampling strategy and the proposed sampling strategy can well reconstruct Group 0 Element 2. Element 2. Photonics 2021, 8, 319 7 of 12 Photonics 2021, 8, x FOR PEER REVIEW 8 of 13 Figure 4. Comparison of the reconstruction results of USAF-1951 resolution test chart for different Figure 4. Comparison of the reconstruction results of USAF-1951 resolution test chart for different sampling strategies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted sampling strategies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each reconstruction. by S, R, and T, respectively) are given in the inset of each reconstruction. We also quantitatively evaluate the reconstruction quality by using structural simi- We also quantitatively evaluate the reconstruction quality by using structural similar- larity index (SSIM) [37] and root-mean-square error (RMSE). The SSIM, RMSE, and im- ity index (SSIM) [37] and root-mean-square error (RMSE). The SSIM, RMSE, and image age reconstruction time (denoted by S, R, and T, respectively) are given in the inset of reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each re- each reconstruction. The quantitative measures also demonstrate the proposed sampling construction. The quantitative measures also demonstrate the proposed sampling strategy strategy has better performance than the other sampling strategies in comparison. has better performance than the other sampling strategies in comparison. In the second simulation, we use a natural image—“Cameraman”—for testing. The In the second simulation, we use a natural image—“Cameraman”—for testing. The size of the test image is also 256 × 256 pixels. As the results show in Figure 5, the images size of the test image is also 256  256 pixels. As the results show in Figure 5, the images reconstructed by the radial sampling strategy turn out to be the worst, especially when reconstructed by the radial sampling strategy turn out to be the worst, especially when the the sampling ratio is lower than 10%. The polynomial, the circular, and the proposed sampling ratio is lower than 10%. The polynomial, the circular, and the proposed sampling sampling strategies are capable of reconstructing recognizable contents when the sam- strategies are capable of reconstructing recognizable contents when the sampling ratio is 3%. pling Therat image io is 3%. reconstr The im uctedage by the reconst proposed ructed method by the proposed m appears clearer ethod , which appears clea is evident by rer, the which is ev details (the ident cameraman’s by the details (t face, he c the ame camera, raman’s and fac the e, the camer buildings) a, that and the build are reconstr ings) ucted. that Both are recon the SSIMs structed. Both and RMSEs the SSIM demonstrate s and the RM advantage SEs demoof nstrate the the proposed advantag method. e of the pro- posed method. Photonics 2021, 8, 319 8 of 12 Photonics 2021, 8, x FOR PEER REVIEW 9 of 13 Figure 5. Comparison of the reconstruction results of “Cameraman” for different sampling strate- Figure 5. Comparison of the reconstruction results of “Cameraman” for different sampling strategies gies and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, and sampling ratios. The SSIM, RMSE, and image reconstruction time (denoted by S, R, and T, respectively) are given in the inset of each reconstruction. respectively) are given in the inset of each reconstruction. 4. Experiment 4. Experiment The proposed method is also demonstrated with real experiments. The schematic The proposed method is also demonstrated with real experiments. The schematic diagram of the experimental set-up is shown in Figure 1a. The set-up consists of a 12-watt diagram of the experimental set-up is shown in Figure 1a. The set-up consists of a 12-watt white-light LED, a DMD (ViALUX V-7001), an imaging lens, a target object, a collecting white-light LED, a DMD (ViALUX V-7001), an imaging lens, a target object, a collecting lens, and a PDA (Thorlabs PDA101A). Note that we binarize the Fourier basis patterns lens, and a PDA (Thorlabs PDA101A). Note that we binarize the Fourier basis patterns with with the upsample-and-dither strategy [29], so as to take advantage of the high-speed the upsample-and-dither strategy [29], so as to take advantage of the high-speed binary pattern binary pa generation ttern genera offer tion ed of byfe the red by the DMD. The DMD. The patternspa artterns e initially are ini with tial256 ly with 256 256 pixels. × 256 The pixel patterns s. The patterns a are upsampled re upsampl with ed wi a ratio th a of ra 2ti thr o of ough 2 through the bi the bicubic interpolation cubic interpoland ation a then nd binarized using the Floyd–Steinberg algorithm [33]. We use two different scenes for the then binarized using the Floyd–Steinberg algorithm [33]. We use two different scenes for experiment. the experime The nt. The one scene is one scene is a USAF-1951 a USAF-1951 re resolution solution targ target printed et printed on a p on a piece of iA4 ece of A paper4 . The other scene consists of a pair of china dolls as foreground and the printed resolution paper. The other scene consists of a pair of china dolls as foreground and the printed target pattern as background. resolution target pattern as background. Similarly, we compare the proposed Gaussian random sampling strategy with the Similarly, we compare the proposed Gaussian random sampling strategy with the radial, the circular, and the polynomial sampling strategies in the experiments. As the radial, the circular, and the polynomial sampling strategies in the experiments. As the reconstructed images show in Figure 6, the experimental results coincide with the simula- reconstructed images show in Figure 6, the experimental results coincide with the simu- tion results, demonstrating that the proposed sampling strategy allows for better imaging lation results, demonstrating that the proposed sampling strategy allows for better im- quality especially when the sampling ratio is low. Please note that the reconstructed images aging quality especially when the sampling ratio is low. Please note that the recon- presented in Figure 6 are acquired at the DMD rate of 50 Hz. structed images presented in Figure 6 are acquired at the DMD rate of 50 Hz. Photonics 2021, 8, x FOR PEER REVIEW 10 of 13 Photonics 2021, 8, 319 9 of 12 Photonics 2021, 8, x FOR PEER REVIEW 10 of 13 Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing Figure 6. Experiment results for (a) USAF-1951 resolution test chart pattern and (b) a pair of china dolls. DMD refreshing rate is 50 Hz. Scale bars = 5 cm. rate is 50 Hz. Scale bars = 5 cm. rate is 50 Hz. Scale bars = 5 cm. In order to demonstrate the proposed fast single-pixel imaging, we test the pro- In order to demonstrate the proposed fast single-pixel imaging, we test the pro- In order to demonstrate the proposed fast single-pixel imaging, we test the proposed posed method with different DMD refreshing rates. In this test, the sampling ratio is set posed method with different DMD refreshing rates. In this test, the sampling ratio is set method with different DMD refreshing rates. In this test, the sampling ratio is set to 10%, to 10%, and therefore, the number of single-pixel measurements is 9831. As the results to 10%, and therefore, the number of single-pixel measurements is 9831. As the results and therefore, the number of single-pixel measurements is 9831. As the results show show in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without no- show in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without no- in Figure 7, the reconstructions for 50 Hz and 200 Hz are clear and without noticeable ticeable noise. In other words, the proposed method is able to capture a high-quality ticeable noise. In other words, the proposed method is able to capture a high-quality noise. In other words, the proposed method is able to capture a high-quality image of image of 256 × 256 pixels within 50 s. As the DMD refreshing rate increases, the noise image of 256 × 256 pixels within 50 s. As the DMD refreshing rate increases, the noise 256  256 pixels within 50 s. As the DMD refreshing rate increases, the noise becomes becomes obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is becomes obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is obvious and the signal-noise ratio (SNR) decreases. The image for 2000 Hz is slightly noisy, slightly noisy, but the data acquisition time can be reduced down to 5 s. When the DMD slightly noisy, but the data acquisition time can be reduced down to 5 s. When the DMD but the data acquisition time can be reduced down to 5 s. When the DMD refreshing rate is refreshing rate is 20,000 Hz, the image appears noisy, but the objects in the image are still refreshing rate is 20,000 Hz, the image appears noisy, but the objects in the image are still 20,000 Hz, the image appears noisy, but the objects in the image are still recognizable. recognizable. recognizable. Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing Figure 7. Experiment results reconstructed by the proposed method for different DMD refreshing rates. Sampling ratio = 10%. rates. Sampling ratio = 10%. rates. Sampling ratio = 10%. Photonics 2021, 8, 319 10 of 12 5. Discussion The total imaging time in single-pixel imaging includes data acquisition time and the image reconstruction time. This work aims at improving the sampling efficiency in FSI to reduce the data acquisition time. For the purpose of imaging a dynamic scene, a short data acquisition time is desirable, because the data acquisition time in single-pixel imaging is like the exposure time in conventional photography. Severe blur might be caused by the motion of objects if the data acquisition time is long. We note that the proposed sampling strategy requires CS for image reconstruction and CS algorithms are commonly computationally exhausted. In our future work, we consider using deep learning [38–45] to reconstruct the final image from the undersampled Fourier spectrum so as to reduce the image reconstruction time. In this paper, we demonstrate that the proposed Gaussian random sampling strategy can effectively improve the sampling efficiency of FSI. We consider that the proposed sampling strategy is applicable to other basis-scan single-pixel methods. The core of the proposed sampling strategy is to conduct density-varying sampling in an orthogonal transform domain so as to improve sampling efficiency for fast single-pixel imaging. The proposed sampling strategy utilizes the fact that the energy of any natural image concen- trates at the low-frequency band of a certain transform domain. It is such ununiformity of energy distribution that enables density varying sampling. Natural images have sparse rep- resentation in DCT, Hadamard transform, and wavelet transform domains. We therefore consider that the reported sampling strategy can be applied in DCT single-pixel imaging, Hadamard single-pixel imaging, and other basis scan single-pixel imaging methods. In comparison with the polynomial sampling strategy, the proposed method can reproduce better images when the sampling ratio is low. In addition, the proposed method requires no user-defined parameters, which adds flexibility to the method in practical use. 6. Conclusions We propose the Gaussian random sampling strategy to achieve efficient FSI. The key to the proposed sampling strategy is to conduct density-varying sampling in the Fourier domain so as to improve sampling efficiency for fast single-pixel imaging. As is demonstrated by the simulations and the experiments, the proposed method is able to reproduce a sharp and clear image of 256  256 pixels with a sampling ratio of 10%. This work benefits fast single-pixel imaging and provides a new approach for efficient spatial information acquisition. Author Contributions: Conceptualization, Z.Z. and J.Z.; validation Z.Q., X.G., T.L. and P.Q.; funding acquisition, Z.Z. and J.Z.; writing, Z.Z. and Z.Q.; supervision, J.Z. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by National Natural Science Foundation of China (NSFC), (61905098 and 61875074) and Guangzhou Basic and Applied Basic Research Foundation (202002030319 and 202102080589). Data Availability Statement: The data presented in this study are available on request from the corresponding author. Conflicts of Interest: The authors declare no conflict of interest. References 1. Decker, J.A. Hadamard-Transform Image Scanning. Appl. Opt. 1970, 9, 1392–1395. [CrossRef] 2. Edgar, M.; Gibson, G.; Padgett, M. Principles and prospects for single-pixel imaging. Nat. Photonics 2019, 13, 13–20. [CrossRef] 3. Sun, M.; Zhang, J. Single-pixel imaging and its application in three-dimensional reconstruction: A brief review. Sensors 2019, 19, 732. [CrossRef] [PubMed] 4. Gibson, G.; Johnson, S.; Padgett, M. Single-pixel imaging 12 years on: A review. Opt. Express 2020, 28, 28190–28208. [CrossRef] [PubMed] 5. Sun, M.J.; Edgar, M.P.; Gibson, G.M.; Sun, B.Q.; Radwell, N.; Lamb, R.; Padgett, M.J. Single-pixel three-dimensional imaging with time-based depth resolution. Nat. Commun. 2016, 7, 12010. [CrossRef] [PubMed] Photonics 2021, 8, 319 11 of 12 6. Phillips, D.B.; Sun, M.J.; Taylor, J.M.; Edgar, M.P.; Barnett, S.M.; Gibson, G.M.; Padgett, M.J. Adaptive foveated single-pixel imaging with dynamic supersampling. Sci. Adv. 2017, 3, e1601782. [CrossRef] 7. Sun, M.J.; Edgar, M.P.; Phillips, D.B.; Gibson, G.M.; Padgett, M.J. Improving the signal-to-noise ratio of single-pixel imaging using digital microscanning. Opt. Express 2016, 24, 10476–10485. [CrossRef] 8. Xu, Z.H.; Chen, W.; Penuelas, J.; Padgett, M.; Sun, M.J. 1000 fps computational ghost imaging using LED-based structured illumination. Opt. Express 2018, 26, 2427–2434. [CrossRef] 9. Greenberg, J.; Krishnamurthy, K.; Brady, D. Compressive single-pixel snapshot x-ray diffraction imaging. Opt. Lett. 2014, 39, 111–114. [CrossRef] 10. Yu, H.; Lu, R.; Han, S.; Xie, H.; Du, G.; Xiao, T.; Zhu, D. Fourier-transform ghost imaging with hard X rays. Phys. Rev. Lett. 2016, 117, 113901. [CrossRef] [PubMed] 11. Zhang, A.; He, Y.; Wu, L.; Chen, L.; Wang, B. Tabletop x-ray ghost imaging with ultra-low radiation. Optica 2018, 5, 374–377. [CrossRef] 12. Chan, W.; Charan, K.; Takhar, D.; Kelly, K.; Baraniuk, R.; Mittleman, D. A single-pixel terahertz imaging system based on compressed sensing. Appl. Phys. Lett. 2008, 93, 121105. [CrossRef] 13. Shrekenhamer, D.; Watts, C.; Padilla, W. Terahertz single pixel imaging with an optically controlled dynamic spatial light modulator. Opt. Express 2013, 21, 12507–12518. [CrossRef] 14. Hornett, S.; Stantchev, R.; Vardaki, M.; Beckerleg, C.; Hendry, E. Subwavelength terahertz imaging of graphene photoconductivity. Nano Lett. 2016, 16, 7019–7024. [CrossRef] 15. Yu, W.; Liu, X.; Yao, X.; Wang, C.; Zhai, Y.; Zhai, G. Complementary compressive imaging for the telescopic system. Sci. Rep. 2015, 4, 5834. [CrossRef] [PubMed] 16. Studer, V.; Bobin, J.; Chahid, M.; Mousavi, H.; Candes, E.; Dahan, M. Compressive fluorescence microscopy for biological and hyperspectral imaging. Proc. Natl. Acad. Sci. USA 2012, 109, E1679–E1687. [CrossRef] 17. Zhang, Z.; Ma, X.; Zhong, J. Single-pixel imaging by means of Fourier spectrum acquisition. Nat. Commun. 2015, 6, 6225. [CrossRef] 18. Bian, L.; Suo, J.; Hu, X.; Chen, F.; Dai, Q. Efficient single pixel imaging in Fourier space. J. Opt.-UK 2016, 18, 085704. [CrossRef] 19. Jiang, H.; Zhu, S.; Zhao, H.; Xu, B.; Li, X. Adaptive regional single-pixel imaging based on the Fourier slice theorem. Opt. Express 2017, 25, 15118–15130. [CrossRef] 20. Huang, J.; Shi, D.; Yuan, K.; Hu, S.; Wang, Y. Computational-weighted Fourier single-pixel imaging via binary illumination. Opt. Express 2018, 26, 16547–16559. [CrossRef] 21. Xu, B.; Jiang, H.; Zhao, H.; Li, X.; Zhu, S. Projector-defocusing rectification for Fourier single-pixel imaging. Opt. Express 2018, 26, 5005–5017. [CrossRef] 22. Zhang, Z.; Liu, S.; Peng, J.; Yao, M.; Zheng, G.; Zhong, J. Simultaneous spatial, spectral, and 3D compressive imaging via efficient Fourier single-pixel measurements. Optica 2018, 5, 315–319. [CrossRef] 23. Sun, M.; Meng, L.; Edgar, M.; Padgett, M.; Radwell, N. A Russian dolls ordering of the Hadamard basis for compressive single-pixel imaging. Sci. Rep. 2017, 7, 1–7. [CrossRef] 24. Yu, W. Super sub-Nyquist single-pixel imaging by means of cake-cutting Hadamard basis sort. Sensors 2019, 19, 4122. [CrossRef] 25. Yu, W.; Liu, Y. Single-pixel imaging with origami pattern construction. Sensors 2019, 19, 5135. [CrossRef] [PubMed] 26. Yu, X.; Stantchev, R.; Yang, F.; Pickwell-MacPherson, E. Super sub-Nyquist single-pixel imaging by total variation ascending ordering of the Hadamard basis. Sci. Rep. 2020, 10, 9338. [CrossRef] 27. Vaz, P.; Amaral, D.; Requicha Ferreira, L.; Morgado, M.; Cardoso, J. Image quality of compressive single-pixel imaging using different Hadamard orderings. Opt. Express 2020, 28, 11666–11681. [CrossRef] [PubMed] 28. Liu, B.; Yang, Z.; Liu, X.; Wu, L. Coloured computational imaging with single-pixel detectors based on a 2D discrete cosine transform. J. Mod. Opt. 2017, 64, 259–264. [CrossRef] 29. Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Fast Fourier single-pixel imaging via binary illumination. Sci. Rep. 2017, 7, 12029. [CrossRef] [PubMed] 30. Yang, J.; Gong, L.; Xu, X.; Hai, P.; Shen, Y.; Suzuki, Y.; Wang, L. Motionless volumetric photoacoustic microscopy with spatially invariant resolution. Nat. Commun. 2017, 8, 780. [CrossRef] 31. Zhang, Z.; Wang, X.; Zheng, G.; Zhong, J. Hadamard single-pixel imaging versus Fourier single-pixel imaging. Opt. Express 2017, 25, 19619–19639. [CrossRef] [PubMed] 32. Meng, W.; Shi, D.; Huang, J.; Yuan, K.; Wang, Y.; Fan, C. Sparse Fourier single-pixel imaging. Opt. Express 2019, 7, 31490–31503. 33. Floyd, R.; Steinberg, L. An adaptive algorithm for spatial grey. Proc. Soc. Inf. Disp. 1976, 17, 75–77. 34. Agustsson, E.; Timofte, R. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 21–26 July 2017; pp. 126–135. 35. Candès, E.; Romberg, J. l1-magic: Recovery of Sparse Signals via Convex Programming. Available online: https://statweb. stanford.edu/~{}candes/software/l1magic/downloads/l1magic.pdf (accessed on 1 June 2021). 36. Candès, E.; Romberg, J.; Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory 2006, 52, 489–509. [CrossRef] 37. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2014, 13, 600–612. [CrossRef] Photonics 2021, 8, 319 12 of 12 38. Yang, X.; Jiang, P.; Jiang, M.; Xu, L.; Wu, L.; Yang, X.; Zhang, W.; Zhang, J.; Zhang, Y. High imaging quality of Fourier single pixel imaging based on generative adversarial networks at low sampling rate. Opt. Lasers Eng. 2021, 140, 106533. [CrossRef] 39. Higham, C.; Murray-Smith, R.; Padgett, M.; Edgar, M. Deep learning for real-time single-pixel video. Sci. Rep. 2018, 8, 2369. [CrossRef] 40. Jiang, W.; Li, X.; Peng, X.; Sun, B. Imaging high-speed moving targets with a single-pixel detector. Opt. Express 2020, 28, 7889–7897. [CrossRef] [PubMed] 41. Li, F.; Zhao, M.; Tian, Z.; Willomitzer, F.; Cossairt, O. Compressive ghost imaging through scattering media with deep learning. Opt. Express 2020, 28, 17395–17408. [CrossRef] [PubMed] 42. Rizvi, S.; Cao, J.; Hao, Q. Deep learning based projector defocus compensation in single-pixel imaging. Opt. Express 2020, 28, 25134–25148. [CrossRef] 43. Rizvi, S.; Cao, J.; Zhang, K.; Hao, Q. DeepGhost: Real-time computational ghost imaging via deep learning. Sci. Rep. 2020, 10, 11400. [CrossRef] [PubMed] 44. Rizvi, S.; Cao, J.; Zhang, K.; Hao, Q. Improving imaging quality of real-time Fourier single-pixel imaging via deep learning. Sensors 2019, 19, 4190. [CrossRef] [PubMed] 45. Radwell, N.; Johnson, S.; Edgar, M.; Higham, C.; Murray-Smith, R.; Padgett, M. Deep learning optimized single-pixel LiDAR. Appl. Phys. Lett. 2019, 115, 231101. [CrossRef]

Journal

PhotonicsMultidisciplinary Digital Publishing Institute

Published: Aug 9, 2021

Keywords: computational imaging; single-pixel imaging; sampling strategy; compressive sensing; optimization

There are no references for this article.