Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Occlusion Culling for Wide-Angle Computer-Generated Holograms Using Phase Added Stereogram Technique

Occlusion Culling for Wide-Angle Computer-Generated Holograms Using Phase Added Stereogram Technique hv photonics Article Occlusion Culling for Wide-Angle Computer-Generated Holograms Using Phase Added Stereogram Technique Juan Martinez-Carranza *, Tomasz Kozacki, Rafał Kukołowicz , Maksymilian Chlipala and Moncy Sajeev Idicula Faculty of Mechatronics, Warsaw University of Technology, 8 Sw. A. Boboli Street, 02-525 Warsaw, Poland; t.kozacki@mchtr.pw.edu.pl (T.K.); rafal.kukolowicz.dokt@pw.edu.pl (R.K.); M.Chlipala@mchtr.pw.edu.pl (M.C.); moncy.idicula.dokt@pw.edu.pl (M.S.I.) * Correspondence: j.martinez@mchtr.pw.edu.pl Abstract: A computer-generated hologram (CGH) allows synthetizing view of 3D scene of real or virtual objects. Additionally, CGH with wide-angle view offers the possibility of having a 3D experience for large objects. An important feature to consider in the calculation of CGHs is occlusion between surfaces because it provides correct perception of encoded 3D scenes. Although there is a vast family of occlusion culling algorithms, none of these, at the best of our knowledge, consider occlusion when calculating CGHs with wide-angle view. For that reason, in this work we propose an occlusion culling algorithm for wide-angle CGHs that uses the Fourier-type phase added stereogram (PAS). It is shown that segmentation properties of the PAS can be used for setting efficient conditions for occlusion culling of hidden areas. The method is efficient because it enables processing of dense cloud of points. The investigated case has 24 million of point sources. Moreover, quality of the occluded wide-angle CGHs is tested by two propagation methods. The first propagation technique quantifies quality of point reproduction of calculated CGH, while the second method enables the quality assessment of the occlusion culling operation over an object of complex shape. Finally, the applicability of proposed occlusion PAS algorithm is tested by synthetizing wide-angle CGHs that are numerically and optically reconstructed. Citation: Martinez-Carranza, J.; Kozacki, T.; Kukołowicz, R.; Chlipala, M.; Idicula, M.S. Occlusion Culling Keywords: computer-generated hologram; phase added stereogram algorithm; occlusion; frequency for Wide-Angle Computer-Generated domain; Fourier transform; propagation method; wide-angle view; direct integration method; tiling Holograms Using Phase Added Stereogram Technique. Photonics 2021, 8, 298. https://doi.org/10.3390/ photonics8080298 1. Introduction The aim of 3D display technology is to reproduce the depth perception of surrounding Received: 2 July 2021 world. An important feature that must be considered in this technology is the ability to Accepted: 23 July 2021 reproduce image providing a complete 3D experience to the viewers [1], i.e., full-parallax. Published: 27 July 2021 Among several solutions that have been proposed in the last decades for developing 3D display technology, digital holography constitutes the best-known framework due Publisher’s Note: MDPI stays neutral to the ability of truly reconstructing the 3D wavefront [2]. The possibility of complex with regard to jurisdictional claims in wavefront reproduction makes digital holography popular choice in several areas as mi- published maps and institutional affil- croscopy [3], metrology [4], non-destructive sensing [5], security [6], design of diffractive iations. optical elements [7]. In the realm of display technology, digital holography enables accurate reproduction of physiological cues [8,9], which is important for photorealistic 3-D images. However, digital holographic systems cannot capture scenes for wide-angle viewing dis- play. With the development of computing technology, holograms can be registered without Copyright: © 2021 by the authors. the necessity of a physical holographic setup. Computer-generated hologram (CGH) does Licensee MDPI, Basel, Switzerland. not have limitations of physical systems [10] such as optical aberrations, extension, noise, This article is an open access article or sensitivity to vibrations. Nevertheless, CGHs have limitations for providing the truly 3D distributed under the terms and experience. The first limitation is related with the pixel pitch of the spatial light modulators conditions of the Creative Commons (SLMs) that are used for displaying CGHs. To have a full-parallax, pixel pitches below one Attribution (CC BY) license (https:// micrometer must be used. High-end SLM technologies can offer SLMs with pixel pitch up creativecommons.org/licenses/by/ to 3.74 m and 4K resolution. Devices in research stage have shown that the pixel pitch 4.0/). Photonics 2021, 8, 298. https://doi.org/10.3390/photonics8080298 https://www.mdpi.com/journal/photonics Photonics 2021, 8, 298 2 of 16 can be reduced up to one micrometer [11,12]. However, this type of technology is still developing. Fortunately, pixel pitch of an SLM can be reduced indirectly by employing optical systems [13], which solves the problem of wide-angle view. The second limitation of CGHs is related to generation of the necessary visual cues for reconstructing photorealistic 3-D images, such as shading, occlusions, transparency, texture mapping, light reflectance properties [14–18]. An important visual cue that any CGH technique must consider is oc- clusion because it provides information about perception of depth [16]. Implementation of occlusion is a challenging task in CGH since the whole information of a 3D object is coded in 2D hologram. A large variety of occlusion algorithms have been designed depending on the employed CGH technique [16,17,19–22]. Propagation-based CGH techniques can apply occlusion culling of unseen regions [13,17,21,23,24]. A popular approach for obtaining CGHs is employing point clouds of 3D objects. Each point in the cloud act as a point source that is well localized in space with a defined amplitude. Point sources emit a spherical wavefront that propagates to the hologram plane. Superposition of all propagated wavefronts generates the CGH [25–28]. However, occlusion processing for cloud point methods is not a simple task. There are several solutions that can solve this problem. For example, reference [21] employs inter-wave- recording-plane light propagation and masks, which has proved to be efficient. Other researches use the stereogram holographic approach to cull hidden back points [8,29]. A simple technique, called spatial frequency index (SFI), is presented in reference [29]. This approach calculates the instantaneous frequencies of the points with respect to the corresponding hologram tile. Instantaneous frequencies are group in sets of the same frequency, which provides the corresponding occlusion relation with the object points. Finally, the point in the set that is closer to the hologram plane is the only candidate to apport holographic information to the CGH. The SFI method can be understood as a phase added stereogram (PAS) algorithm [30–32] that enables occlusion culling. However, the SFI strategy is time consuming, because it does not employ Fourier transformation (FT). Instead, it uses tables for grouping and selecting the proper frequencies, and what is more important, the size of the tiling is small. All mentioned approaches enable occlusion culling of unseen areas. However, none of these techniques consider the wide projection angle. For example, propagation-based occlusion techniques are not optimal for CGHs with wide-angle, since they do not usually consider geometry where the object is much larger than the hologram [25]. This geomet- rical feature is challenging for rigorous propagation algorithms that must be used. For point-occlusion based methods, the wide-angle problem is not considered. In this work, we propose a methodology for occlusion culling based on the Fourier type-PAS algorithm. Unlike the SFI strategy, our approach does not require tables for grouping the correspond- ing frequencies, and we provide the condition for selecting the sub-hologram tile size for carrying out proper occlusion culling. This allows reducing the processing time of the CGH. The proposed occlusion-PAS-based algorithm enables fast calculation of wide-angle CGHs with the removal of hidden surfaces, which has not been shown until now. The developed PAS method is tested for wide-angle CGH, i.e., small pixel pitch. Moreover, two propagation solutions are proposed for testing the accuracy and quality of the occlusion- based PAS algorithm. The first, based on the direct integration method [33], quantifies quality of point reproduction of calculated CGH. The second, which is a generalization of the multi Fast Fourier Transform Angular Spectrum (MFFT-AS) method [10], assesses the quality of the occlusion culling operation over a 3D object assembled of 24 million of point sources. Theoretical capabilities of our solutions are tested for proving accuracy of computations. Finally, the applicability of our occlusion culling method is carried out numerically and experimentally. Photonics 2021, 8, x FOR PEER REVIEW 3 of 16 Photonics 2021, 8, 298 3 of 16 2. Synthetizing CGH with the PAS 2. Synthetizing A hologram CGH H is obt with the ained PAS as the result of measuring the intensity that results from the linear superposition of a diffracted object wavefield O and a reference wavefield R. A hologram H is obtained as the result of measuring the intensity that results from The detected intensity can be mathematically expressed as follows the linear superposition of a diffracted object wavefield O and a reference wavefield R. The detected intensity can be mathematically expressed as follows HO=+R +OR**+O R. (1) 2 2 H = jOj +jRj + OR +O R. (1) The hologram is a set of dark and bright fringes that encode amplitude and phase The hologram is a set of dark and bright fringes that encode amplitude and phase information of the object. When using a digital camera for registering H, a digital holo- information of the object. When using a digital camera for registering H, a digital hologram gram (DH) is obtained. DH can be processed numerically for reconstructing the encoded (DH) is obtained. DH can be processed numerically for reconstructing the encoded object object information. However, capturing of DHs requires interferometric systems that are information. However, capturing of DHs requires interferometric systems that are sensitive sensitive to optical aberrations, environmental and mechanical vibrations, or noise. to optical aberrations, environmental and mechanical vibrations, or noise. With the development of computing technology, holograms can be registered with- With the development of computing technology, holograms can be registered without out the necessity of a physical holographic setup. When synthetizing CGH, only the object the necessity of a physical holographic setup. When synthetizing CGH, only the object wave field O is calculated. Solutions for calculating this wavefield have been studied for wave field O is calculated. Solutions for calculating this wavefield have been studied for more than three decades [30,34–37]. The most popular approach consists in decomposing more than three decades [30,34–37]. The most popular approach consists in decomposing the 3D object into a set of illuminating point sources that are distributed in space. Each the 3D object into a set of illuminating point sources that are distributed in space. Each point point source emits a spherical wavefront that propagates to the hologram plane. Super- source emits a spherical wavefront that propagates to the hologram plane. Superposition position of all propagated wavefronts generate the CGH as follows of all propagated wavefronts generate the CGH as follows p a ikD p pikD O = e , (2) Oe = , (2) p=1 D p=1 2 2 where D = (x x ) + (y y ) + z , (x , y , z ) are the spatial position and a 22 2 p 2 p 2 p p p p p where Dx=−()x +(y−y)+z , (xp, yp, zp) are the spatial position and ap amplitudes p 22pp p amplitudes of the point sources, respectively. Nevertheless, this solution is time consum- of the point sources, respectively. Nevertheless, this solution is time consuming [26,28]. ing [26,28]. Efficient techniques based on propagation have been proposed for calculating Efficient techniques based on propagation have been proposed for calculating O O [21,25,36,38,39]. With the aim to give the viewer a complete 3-D experience with full- [21,25,36,38,39]. With the aim to give the viewer a complete 3-D experience with full-par- parallax, wide-angle CGHs are required. The scheme of geometry for recording wide-angle allax, wide-angle CGHs are required. The scheme of geometry for recording wide-angle CGHs is presented in Figure 1. The distance between hologram plane (x , y ) and object 1 1 CGHs is presented in Figure 1. The distance between hologram plane ( 1x1, y1) and object plane (x , y ) is z. Field of View (FoV) in a CGH is calculated as FoV = 2sin (/2D), where 2 2 −1 plane (x2, y2) is z. Field of View (FoV) in a CGH is calculated as FoV = 2sin (λ/2Δ), where and D are the wavelength and pixel pitch of the hologram, respectively. Thus, wide-angle λ and Δ are the wavelength and pixel pitch of the hologram, respectively. Thus, wide- CGHs are obtained by employing small pixel pitch at the hologram plane. For example, for angle CGHs are obtained by employing small pixel pitch at the hologram plane. For ex- given  = 0.64 m and D = 0.62 m the FoV 62 . Large FoV allows synthetizing CGHs of ample, for given λ = 0.64 µm and Δ = 0.62 µm the FoV ≈ 62°. Large FoV allows synthetizing objects with spatial extension B = B = 2z tan(FoV). These spatial extensions are much x 2 y 2 CGHs of objects with spatial extension Bx2 = By2 = 2z tan(FoV). These spatial extensions are larger than the CGH size; for example, when using z = 1000 mm, B = B = 1209 mm x 2 y 2 much larger than the CGH size; for example, when using z = 1000 mm, Bx2 = By2 = 1209 mm while an a 4K CGH (4120 2060) will have the dimensions B = 2.6 mm and B = 1.3 mm. x 1 y 2 while an a 4K CGH (4120 × 2060) will have the dimensions Bx1 = 2.6 mm and By2 = 1.3 mm. Figure 1. Geometry for recording wide-angle CGHs. Figure 1. Geometry for recording wide-angle CGHs. Photonics 2021, 8, x FOR PEER REVIEW 4 of 16 Unfortunately, propagation-based techniques are not an optimal approach for syn- thetizing CGHs of wide-angle, since they do not consider geometry, where the object is much larger than the hologram [25]. This geometrical feature is challenging for rigorous propagation algorithms. Alternatives for efficient synthesis of CGHs are PAS-based solu- tions. These solutions do not have the restrictions of propagation-based methods since they consider the basic idea of summing point source responses at the hologram plane [28,31,32]. The PAS algorithm divides the CGH into smaller segments of size N s × Ns. The number of segments within the CGH is N × M, where N is the number of segments in the Photonics 2021, 8, 298 4 of 16 horizontal direction and M in the vertical direction. At each segment, PAS approximates the spherical wavefront that comes from the p-th point source of the object with a local tilted plane wave [31], as shown in Figure 2a. The direction of the plane wave is calculated Unfortunately, propagation-based techniques are not an optimal approach for syn- according to the value of the local instantaneous frequency at the center of the segment thetizing CGHs of wide-angle, since they do not consider geometry, where the object is cc much larger than the hologram [25]. This geometrical feature is challenging for rigorous x , y , where n = 1, …, N, and m = 1, …, M indicates segment position over the hori- () nm propagation algorithms. Alternatives for efficient synthesis of CGHs are PAS-based solu- tions. These solutions do not have the restrictions of propagation-based methods since they zontal and vertical direction, respectively. Since the frequency information of each seg- consider the basic idea of summing point source responses at the hologram plane [28,31,32]. ment is coded according to the local approximation of the tilted plane, the PAS can calcu- The PAS algorithm divides the CGH into smaller segments of size N  N . The number of s s late the diffracted field of objects larger than the hologram size. The term O for a given segments within the CGH is N  M, where N is the number of segments in the horizontal direction and M in the vertical direction. At each segment, PAS approximates the spherical segment (m,n) is calculated with the PAS as follows wavefront that comes from the p-th point source of the object with a local tilted plane wave [31], as shown in Figure 2a. The direction of the plane wave is calculated according to c c the value of the local instantaneous frequency at the center of the segment (x , y ), where p ikD n m cc pmn  Oe=− expi2πf (xx )+f (y−y ) , (3) n = 1, . . . , N, and m = 1, . . . , M indicates segment position over the horizontal and vertical mn () pxnm11 n pymn m  pmn direction, respectively. Since the frequency information of each segment is coded according p=1 to the local approximation of the tilted plane, the PAS can calculate the diffracted field of objects larger than the hologram size. The term O for a given segment (m,n) is calculated where Dpmn is the distance from p-th point source to the center of the segment (m,n), and with the PAS as follows ikD c c pmn sinθθ − sin O = e exp i2p f (x x ) + f (y y ) , (3) pxnm 1 pymn 1 mn å pxnm rxnm n m pmn p=1f = , (4) pxnm where D is the distance from p-th point source to the center of the segment (m,n), and pmn sin q sin q pxnm rxnm sinθθ − sin pynm rynm f = , (4) pxnm f = l , (5) pynm sin q sin q pynm rynm f = , (5) pynm are the local instantaneous frequencies. The angles θxpnm, θypnm, θrxnm, and θrynm are the inci- are the local instantaneous frequencies. The angles q , q , q , and q are the xpnm ypnm rxnm rynm dent angles of object and reference beams on x-axis and y-axis in relation to the center of incident angles of object and reference beams on x-axis and y-axis in relation to the center the segment, respectively. Notably, the hologram in the corresponding segment is a su- of the segment, respectively. Notably, the hologram in the corresponding segment is a superposition of plane waves. When the reference beam has a constant incident angle, perposition of plane waves. When the reference beam has a constant incident angle, spa- spatial frequencies depend on the object beam [31]. tial frequencies depend on the object beam [31]. Figure 2. CGH PAS scheme. (a) Spherical wavefront decomposition. (b) CGH synthesis. Figure 2. CGH PAS scheme. (a) Spherical wavefront decomposition. (b) CGH synthesis. Photonics 2021, 8, 298 5 of 16 Direct computation of Equation (3) can still be slow for a dense cloud of points. Nev- ertheless, this process can be significantly accelerated when employing the FT. When representing Equation (3) in frequency domain, the plane waves in the segment are trans- formed into a 2D distribution of Dirac deltas. The positions of these deltas need to be set to a discrete value that is specified by the segment size. Hence, Equations (4) and (5) are rewritten as: sin q sin q pxnm rxnm f = D f , (6) pxnm D f l 1 sin q sin q pynm rynm f = D f , (7) pynm D f l where Df = 1/N D is the frequency sampling, and b  c is the rounding operation. When employing Equations (6) and (7), the term O is expressed in the frequency domain as mn ikD ˆ ˆ ˆ pmn O = d f f , f f e . (8) x pxnm y pynm pmn p=1 After placing all the Dirac deltas for the 3D point distribution into the segment, the inverse FT is applied, and then the complex field for a given segment is obtained. This procedure is repeated for all segments for obtaining the CGH, as shown in Figure 2b. 3. Occlusion Culling of Back Points with the PAS Representation of 3D object for point-based approach offers a simple and flexible way of depicting its shape. Moreover, this representation allows associating the sources with additional information such as colors, textures, and normals [40]. Each point source generates a complex field from the scene towards the hologram plane, and the final CGH is the sum of all corresponding spherical waves. In the point-based representation, point sources cannot occlude one another, unless they accidentally fall along the same direction from the viewpoint. The wavefields from back and front points are encoded in the CGH. This prevents correct reproduction of depth cues of the 3D object in holographic image. In order to eliminate the back points, occlusion techniques are needed. Occlusion techniques can rely on geometric considerations [40], light shielding [17], or hybrid [15,23]. Since the CGH must provide motion parallax, the chosen occlusion technique must consider that the visibility of object changes according to the movement of the viewer. Occlusion can be introduced by employing the segmentation property of the PAS [29]. As shown in Equation (3), the term O for a given segment (m,n) is a superposition of plane waves having directional wave vectors calculated according to the center of the segment. When applying the FT to the segment, the coordinates of 3D point sources are mapped into the sets of 2D Dirac deltas in frequency domain. The frequency support of this segment is given by D . Since frequency allocation depends on the size of the segment, the gaps between adjacent Dirac deltas can be controlled in the frequency domain by the value of N . Figure 3 illustrates this effect for three point sources when using different segments sizes. The point sources x , and x are adjacent front points, while x is a back point. The angles p p +1 m of corresponding plane waves are q , q , and q , respectively. In this example, the angles p p +1 m fulfill the relation q < q < q . Figure 3a presents the case of a small segment. In this p m p +1 case, the wavefront information for all point sources is located in the same frequency pixel ˆ ˆ ˆ since f = f = f . For carrying out the occlusion, information related with the point p p+1 m source x needs to be disregarded because it has the farthest separation from the hologram plane. When enlarging the segment, as shown in Figure 3b, wavefront information of ˆ ˆ front points x and x is allocated in frequency pixels f and f , respectively. In this p p p+1 p+1 case, the information related to the source x will be placed to the nearest neighbor f . m p+1 Similarly to Figure 3a, the information related to the source x needs to be disregarded for the occlusion culling. Finally, for large segment the wavefront information of each point source is stored in independent pixels, as presented in Figure 3c. As seen in Figure 3, the Photonics 2021, 8, x FOR PEER REVIEW 6 of 16 separation from the hologram plane. When enlarging the segment, as shown in Figure 3b, wavefront information of front points xp and xp+1 is allocated in frequency pixels and , respectively. In this case, the information related to the source xm will be placed to p+1 the nearest neighbor . Similarly to Figure 3a, the information related to the source xm p+1 needs to be disregarded for the occlusion culling. Finally, for large segment the wavefront Photonics 2021, 8, 298 6 of 16 information of each point source is stored in independent pixels, as presented in Figure 3c. As seen in Figure 3, the key point of occlusion culling when using PAS is the allocation of frequency information of the sources that are closer to the hologram plane first, which key point of occlusion culling when using PAS is the allocation of frequency information of are labeled as front sources. These sources contribute the proper object information to the the sources that are closer to the hologram plane first, which are labeled as front sources. CGH. Other point sources of the same frequency but farther from the hologram plane are These sources contribute the proper object information to the CGH. Other point sources of labeled as back sources, which must be disregarded. Thus, sorting of the cloud in respect the same frequency but farther from the hologram plane are labeled as back sources, which must be disregarded. Thus, sorting of the cloud in respect to the z-axis allows identifying to the z-axis allows identifying points from the nearest to the farthest in respect to the points from the nearest to the farthest in respect to the hologram plane. hologram plane. Figure 3. Allocation of discrete frequencies in the PAS algorithm: (a) small segment, (b) medium Figure 3. Allocation of discrete frequencies in the PAS algorithm: (a) small segment, (b) medium segment, and (c) large segment. segment, and (c) large segment. With the aim to calculate the maximum segment size that enables occlusion, let’s consider a set of points that are distributed in the plane (x ,z). Then, adjacent points x 2 p With the aimand to ca x lcand ulaan te arbitrary the maxi hologram mum segment siz segment with center e thaat t ena x arb eles occlusi chosen. Theo dif n, fer le ence t’s p+1 between corresponding local frequencies is given by consider a set of points that are distributed in the plane (x2,z). Then, adjacent points xp and sin q sin q xp+1 and an arbitrary hologram segment with center at x are chosen. The difference be- pxn (p+1)xn f f = . (9) pxn (p+1)xn tween corresponding local frequencies is given by Note that  =  + D, where D is the angular difference between adjacent (p+1)nx pnx sinθθ − sin sources. To eliminate any gap between adjacent frequencies, the right-hand side of Equation (9) (1px + ) n pxn ff−= . (9) must be equal or smaller than the frequency sampling of the segment. This can be expressed as (1px + ) n pxn 2 sin(Dq/2) cos(q + Dq/2) pxn . (10) Note that θ(p+1)nx = θpnx + Δθ, where Δθ is the angular difference b etween adjacent N D l sources. To eliminate any gap between adjacent frequencies, the right-hand side of Equa- The denominator on the right-hand side of Equation (10) has been obtained by using tion (9) must be equal or smaller than the frequency sampling of the segment. This can be the sum-to-product identity. When meeting this condition, the two wavefronts corre- expressed as sponding to the discrete frequencies f and f are occupying two adjacent discrete pxn (p+1)xn locations in the frequency matrix. When considering a back point x , which view angle q m m lies between q and q , the holographic information associated with the frequency f p p+1 mxn 2sin(Δ+ θθ / 2) cos( Δθ / 2) pxn can be accommodated into the nearest neighbor f or f . Nevertheless, the contri- pxn (p+1)xn ≥ . (10) bution from f can be disregarded since the frequency positions are already occupied mxn N Δ λ by the front point sources, as shown in Figure 3b. Finally, maximum segment size can be found by solving Equation (10) for N , which yields The denominator on the right-hand side of Equation (10) has been obtained by using N < sin(Dq/2) cos(q + Dq/2) . (11) s pxn the sum-to-product identity. When meeting this condition, the two wavefronts corre- 2D sponding to the discrete frequencies f and f are occupying two adjacent dis- Notably, Equation (11) pxnallows calculating (1px + ) n the segment size that enables occlusion culling of back points. Once the occlusion culling of back points is complete, inverse FT is crete locations in the frequency matrix. When considering a back point xm, which view applied, and thus, the hologram segment with the proper depth perspective is obtained. angle θm lies between θp and θp+1, the holographic information associated with the fre- It is important to differentiate two cases in the occlusion culling. The first case can be exemplified by Figure 3a. In this case, a small segment is employed for the occlusion. As depicted in this figure, the holographic information of two sources (or more) is allocated in Photonics 2021, 8, x FOR PEER REVIEW 7 of 16 quency can be accommodated into the nearest neighbor f or f . Never- mxn pxn (1px + ) n theless, the contribution from can be disregarded since the frequency positions are mxn already occupied by the front point sources, as shown in Figure 3b. Finally, maximum segment size can be found by solving Equation (10) for Ns, which yields −1 N<Δ sin(θθ / 2) cos( +Δθ / 2) . (11) spxn 2Δ Notably, Equation (11) allows calculating the segment size that enables occlusion culling of back points. Once the occlusion culling of back points is complete, inverse FT is applied, and thus, the hologram segment with the proper depth perspective is obtained. It is important to differentiate two cases in the occlusion culling. The first case can be exemplified by Figure 3a. In this case, a small segment is employed for the occlusion. As depicted in this figure, the holographic information of two sources (or more) is allocated in the same frequency pixel. This means that resolution between the sources is lost. Thus, the smaller the segment is, the smaller the resolution in the CGH. On the other hand, when the chosen segment size is closer to Ns, wavefront information of the sources is placed in different frequency pixels, as presented in Figure 3b, which allows preserving resolution Photonics 2021, 8, 298 7 of 16 in the reconstructed image. Finally, it is important to note that position of the segment shifts allocation of the discrete frequencies but not the condition imposed by Equation (11). For segments that are too far from the center of the CGH, the shifting causes discrete the same frequency pixel. This means that resolution between the sources is lost. Thus, the −1 frequencies to fall outside from the frequency support Δ . smaller the segment is, the smaller the resolution in the CGH. On the other hand, when the chosen segment size is closer to N , wavefront information of the sources is placed in Visualization of the occlusion culling for a segment in the frequency domain is pre- different frequency pixels, as presented in Figure 3b, which allows preserving resolution sented in Figure 4. The employed object is a point cloud of a gargoyle figurine with di- in the reconstructed image. Finally, it is important to note that position of the segment mensions 1055 mm (height) × 971 mm (width) × 485 mm (depth) composed of approxi- shifts allocation of the discrete frequencies but not the condition imposed by Equation (11). For segments that are too far from the center of the CGH, the shifting causes discrete mately 24M points with a density of 9 points/mm . The distance from the center of the frequencies to fall outside from the frequency support D . object to the hologram plane is one meter. Employed features allow calculating an average Visualization of the occlusion culling for a segment in the frequency domain is pre- angular spacing of 0.038°. Employed features allow calculating an average angular spac- sented in Figure 4. The employed object is a point cloud of a gargoyle figurine with ing of 0.035°. When employing, Δ = 0.62 µm and λ = 640 nm, which enables maximum dimensions 1055 mm (height)  971 mm (width)  485 mm (depth) composed of approxi- mately 24M points with a density of 9 points/mm . The distance from the center of the angle view of 62.5°, it is found that Ns must be smaller than 1550 pixels. Thus, segment object to the hologram plane is one meter. Employed features allow calculating an average size is selected as Ns = 1024. The size of the calculated CGH is chosen as 4096 × 4096 pixels. angular spacing of 0.038 . Employed features allow calculating an average angular spacing In terms of the PAS technique, the CGH consists of 4 segments of size Ns × Ns, and thus N, of 0.035 . When employing, D = 0.62 m and  = 640 nm, which enables maximum angle view of 62.5 , it is found that N must be smaller than 1550 pixels. Thus, segment size is M = 4. Figure 4 shows the amplitudes of the segment m = 1 and n = 1 in the frequency selected as N = 1024. The size of the calculated CGH is chosen as 4096  4096 pixels. In domain. Figure 4a depicts the cloud of points without occlusion. Image shows that back terms of the PAS technique, the CGH consists of 4 segments of size N  N , and thus N, s s and front points contribute to the holographic information of the segment. Thus, the depth M = 4. Figure 4 shows the amplitudes of the segment m = 1 and n = 1 in the frequency cues are not correctly displayed. Figure 4b shows the cloud of points when applying the domain. Figure 4a depicts the cloud of points without occlusion. Image shows that back and front points contribute to the holographic information of the segment. Thus, the depth proposed occlusion culling based PAS. After the occlusion culling of back points, only the cues are not correctly displayed. Figure 4b shows the cloud of points when applying the front of the gargoyle is seen, which is the correct observable surface. Later, the cloud of proposed occlusion culling based PAS. After the occlusion culling of back points, only the points is rotated 180° and PAS with culling occlusion is applied again. Hence, the back of front of the gargoyle is seen, which is the correct observable surface. Later, the cloud of the gargoyle is obtpoints ainedis . T rotated he res 180 ult is and sh Pown AS with in F culling igure occlusion 4c. These is applied examples show again. Hence, that our the back of the gargoyle is obtained. The result is shown in Figure 4c. These examples show that method can eliminate unwanted back points by using the segmentation properties of the our method can eliminate unwanted back points by using the segmentation properties of PAS. the PAS. Figure 4. Frequency mapping of the point cloud when using PAS for the segment m = 1, n = 1. (a) PAS without occlusion. (b) Front and (c) back of the gargoyle when applying PAS-based occlusion algorithm. In its basic configuration, the PAS algorithm introduces a truncation error when discretizing the instantaneous local frequencies (Equations (6) and (7)) [32], which decreases the reconstruction quality of the CGH. This truncation error depends on the segment size and can be easily reduced by using a large segment. In the general case of CGH calculation, arbitrarily large segment cannot be selected since the wavefront at the hologram plane may not be well represented. For the proposed occlusion culling PAS, the restriction of the segment becomes stricter due to the limitation imposed with Equation (11). Accurate PAS (A-PAS) can be employed for solving the discretization problem while fulfilling the N constraint and sampling theory [31] at the same time. The A-PAS considers that the s Photonics 2021, 8, 298 8 of 16 segment is enlarged in the discrete frequency domain by an integer factor r. In this way, the frequency sampling is increased r-times, which allows a better approximation of the discrete values of the frequencies to their respective continuous values. After applying the inverse FT to the enlarged segment, the original segment is taken. The A-PAS is implemented in the occluded PAS algorithm by considering that the enlarged segment in frequency domain is not larger than N . This means that we must find a hologram segment of size N and extension factor r such that N > N r. When selecting proper N and r, the so s so so accuracy of the PAS algorithm can be reached while preserving the occlusion culling of unwanted back points. 4. Numerical Reconstruction Methods for Wide View-Angle CGHs With the aim to measure accuracy and quality of the holographic image encoded in the CGH, propagation methods [33,41,42] must be employed. In this section, we investigate two propagation techniques. The first, which is based on the direct integration method [33] is used to assess the accuracy of the CGH calculated by the PAS. The second, that is a generalization of the MFFT-AS technique [10], allows for reconstructing the whole object and visualizing quality of the PAS based occlusion culling. 4.1. Off Axis Direct Integration Method Recovering the full complex field from wide-angle CGH with classical propagation techniques requires a huge amount of zero padding in x and y directions. Thus, an enormous amount of computational power is required for storing and operating such information. Moreover, the time that is necessary for obtaining the reconstruction will be on the scale of hours or even days. Therefore, it is proposed to employ the integration method [33] for reconstructing a very small off-axis window of the entire complex field. This is done with the aim to carry out a fast quantification of point reproduction quality of calculated CGH. Off-axis wavefield recovery can be carried out using the Fourier-based solver of the Rayleigh-Sommerfeld integral [43]: u (x , y , 0) = IFFT[FFT[u (x , y , z )]FFT[g(x + x x , y + y y ,z)]], (12) out 2 2 in 1 1 R 2 OA 1 2 OA 1 where x = x x , y = y y , and (x , y ) are the coordinates of transversal shift, u is 2 1 oa 2 1 oa oa oa in the complex field from the hologram, u is the reconstructed field, the subindexes 1 and 2 out point out the input and output planes, respectively, and exp(ikD) 1 g = ik (13) D D 2 2 2 where D = x + y + z . Application of the FFT in Equation (12) generates a circular convolution affecting accuracy of reconstruction. Influence of circular convolution is avoided by increasing twice size of the input matrix by using zero-padding. When solving Equation (12) with these considerations, the desired field is given by the N  N lower x y right submatrix, where N is the size of the input field. This method preserves the pixel size between input at output planes. Thus, the size of the integrated window is x 2 (N D/2, 2 x N D/2), y 2 (N D/2, N D/2). This off-axis direct integration (OADI) method enables fast x x x wavefield reconstruction from any point of view with small amount of memory [33,41,44] but for very small window. 4.2. Multi Fast Fourier Transform Angular Spectrum Method For the reconstruction of full holographic data from the wide-angle CGH, a general- ization of the MFFT-AS method is proposed [10]. The MFFT-AS was initially employed for the reconstruction of Fourier horizontal parallax-DHs (FHPO-DHs) [45], which are small in vertical and large in horizontal directions, respectively. This method applies (q+1)N  N x y FFTs instead of two large FFTs from the classical angular spectrum (AS) method, where N , x Photonics 2021, 8, 298 9 of 16 N are the dimensions of the FHPO-DH and q = bB /B c is the compression factor for y y y2 y1 direction (B is vertical dimension of the hologram). A critical step in the MFFT-AS is the y1 summation in form of “tiles” in y direction. This makes possible to significantly reduce the computational load. Hence, reconstruction of FHPO-DH with AS is possible. In this work, we extend the MFFT AS application to two dimensions, which allows obtaining accurate reconstruction of full parallax CGH. Implementation of the MFFT-AS can be described as follows: let O be the 2D object wave of N  N samples and spatial dimensions B  B . We wish to find its diffracted x y x1 y1 field at plane (x , y ). The MFFT-AS approach accomplishes this task by calculation of 2 2 p  q FFTs as N 1 N 1 x Y 2pi(na/pN +mb/qN ) 2pi(nl/N +mj/N ) x y x x O (lp + a, jq + b) = O(n, m)e e , (14) 1 å å n=0 m=0 where p = bB /B c is the compression factor in horizontal direction, lp+a, jq+b represent x2 x1 the frequency coordinates separated by 1/pB and 1/qB , respectively. Equation (14) is x1 y1 multiplied by a shifted propagation kernel, which yields 2 2 (lp+a) (jq+b) 2piz 2 2 2 l (pB ) (qB ) x1 y1 e e O (lp + a, jq + b) = O (lp + a, jq + b)e . (15) 2 1 It is worth noting that the Equation (15) has size of N p  N q; however, this large x y matrix is never allocated. Sequentially, calculated vectors of size N  N with Equation (15) x y are assembled as matrices of the same length N  N by next part of the MFFT-AS method. x y This element of the MFFT-AS is based on the property of FFT that was proposed in references [46] and allows efficient calculations of high-resolution focused wave of small spatial extend. The property enables reduction of bandwidth of FFT without the need of computing FFT of the full-sized vector. To explain the property, let assume a 1D signal A of q1 length qN . The method starts with summing up “tiles” as A (n) = A(tq + n), where y s t=0 n 2 0, . . . , N 1 and then computes FFT. Notably, the A is a vector, which contains q-th y s frequency of A . In the 2D MFFT-AS method, the tiling is applied in both directions and the introduced property enables reduction of the sampling rate of the calculated diffracted field. Thus, the result is calculated using at first ‘tiles’ summation method, and then FT as N 1 p1 y q1 N 1 2pi(nl/N +mj/N ) x y O(n, m) = O (sp + l, tq + j)e . (16) å å å å 2 l=0 s=0 j=0 t=0 Notably, full field reconstruction with the 2D MFFT-AS AS requires q  p + 1 FFTs of size N  N , and pixel size at the output plane is determined by Dx = p Dx , and x y 2 1 Dy = q Dy . 2 1 4.3. Comparison of Full and Small off Axis Window Reconstruction The strategies based on off axis window and full image reconstruction are here com- pared from the computational point of view. For this, the set of holograms for reconstruction distance z = 1000 mm, pixel count 4096  4096, and several pixel pitches were generated and reconstructed. Table 1 presents the image size and the obtained computation times for both recon- struction methods when using selected pixel pitches. Notably, the MFFT-AS enables to obtain image sizes ranging from one quarter of meter up to one meter. However, the price to pay is a large processing time. For example, reconstruction of the smallest pixel pitch (D = 0.6 m) requires almost 12 h of processing. On the other hand, the OADI method provides results in a few seconds for reconstructing the complex field but of window size of a few millimeters. Photonics 2021, 8, 298 10 of 16 Table 1. Calculated Field of View and computation time for the off-axis direct integration method and the MFFT-AS,  = 640 nm, z = 1000 mm. Image Size Calculation Speed D/FoV OADI MFFT-AS OADI MFFT-AS 0.6 m/62 2.5 mm  2.5 mm 1.2 m  1.2 m 10.5 s 38,300 s 1.2 m/31 5 mm  5 mm 0.54 m  0.54 m 10.5 s 1900 s 1.8 m/20 7.3 mm  7.3 mm 0.36 m  0.36 m 10.5 s 304 s 2.4 m/15 9.9 mm  9.9 mm 0.26 m  0.26 m 10.5 s 104 s 5. Diffraction Efficiency Based on Assessment of CGH Quality In this section, assessment of CGH quality based on diffraction efficiency measure is carried out. This is done with the aim to quantify quality of point reproduction of calculated CGH. Reconstruction of wide-angle CGHs require, see first row of Table 1, a large amount of memory and processing time. Nevertheless, for measuring the diffraction efficiency of CGHs of this work, we retrieve a small region where a point source is reconstructed. Hence, the OADI method is suitable for measuring the diffraction efficiency of CGHs calculated with the PAS algorithm. The first diffraction efficiency-based test is evaluated with two CGHs of ten point sources distributed linearly along the diagonal from the center of the image P (0, 0, z) up to corner point P = (0.475B , 0.475B z), where z = 1000 mm. Pixel pitch of the CGHs 10 x 2 y 2, is D = 0.62 m. Theoretical CGH, which is reference one, is calculated using Equation (2), while the second CGH with the PAS algorithm employing N = 256 and r = 4. Calculated so CGHs are processed with Equation (12) and the corresponding off-axis coordinate. As a result, the single point source with its nearest neighborhood is reconstructed. Figure 5a,b present the reconstruction of point P of theoretical and PAS CGHs, respectively. Visually, reconstructed point sources have identical profiles and almost the same peak value (P-V). Figure 5c presents the accuracy test in relation to point source position along the diagonal of image space. Maximum amplitude values are normalized with respect to the on-axis point P from theoretical CGH. Regardless, the CGH is made with PAS method or with Equation (2), the linear loss of amplitude due to point source position is present. Moreover, it is clearly seen in the plot that accuracy of reconstructed point sources is strongly related to the selection of the parameter r, e.g., for r 3, the maximum amplitude of each point starts to behave stable and is close to the result of theoretical CGH. Accordingly, CGHs of high accuracy made with PAS can be achieved by using a value N and large r. Nevertheless, so the constraint provided by Equation (11) must be fulfilled all the time if occlusion culling is desired. For this reason, we perform a test that is aimed to measure the accuracy of calculated CGHs with the PAS with different parameters N and r. As a measure of so accuracy, the ratio of maximum amplitude of reconstructed point P between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. The obtained ratios demonstrate that when using original segment size (r = 1), the diffraction efficiency is low and unstable, which means that calculated CGH with PAS is not optimal. When using large values of r (r > 2), the maximum amplitude increases its value independently of the N , which is pointed out by large dash box into the plot. so Photonics 2021, 8, x FOR PEER REVIEW 11 of 16 Photonics 2021, 8, x FOR PEER REVIEW 11 of 16 of each point starts to behave stable and is close to the result of theoretical CGH. Accord- ingly, CGHs of high accuracy made with PAS can be achieved by using a value Nso and of each point starts to behave stable and is close to the result of theoretical CGH. Accord- large r. Nevertheless, the constraint provided by Equation (11) must be fulfilled all the ingly, CGHs of high accuracy made with PAS can be achieved by using a value Nso and time if occlusion culling is desired. For this reason, we perform a test that is aimed to large r. Nevertheless, the constraint provided by Equation (11) must be fulfilled all the measure the accuracy of calculated CGHs with the PAS with different parameters Nso and time if occlusion culling is desired. For this reason, we perform a test that is aimed to r. As a measure of accuracy, the ratio of maximum amplitude of reconstructed point P10 measure the accuracy of calculated CGHs with the PAS with different parameters Nso and between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. r. As a measure of accuracy, the ratio of maximum amplitude of reconstructed point P10 The obtained ratios demonstrate that when using original segment size (r = 1), the diffrac- between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. tion efficiency is low and unstable, which means that calculated CGH with PAS is not The obtained ratios demonstrate that when using original segment size (r = 1), the diffrac- optimal. When using large values of r (r > 2), the maximum amplitude increases its value Photonics 2021 tion ef , 8, 298ficiency is low and unstable, which means that calculated CGH with PAS is not 11 of 16 independently of the Nso, which is pointed out by large dash box into the plot. optimal. When using large values of r (r > 2), the maximum amplitude increases its value independently of the Nso, which is pointed out by large dash box into the plot. Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Recon- structed amplitude of the point source P10 from theoretical CGH; (b) Reconstructed amplitude of Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Recon- Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Reconstructed amplitude of the point the point source P10 from PAS CGH; and (c) normalized reconstructed amplitudes for several Nso structed amplitude of the point source P10 from theoretical CGH; (b) Reconstructed amplitude of source P from theoretical CGH; (b) Reconstructed amplitude of the point source P from PAS CGH; and (c) normalized 10 10 and r. reconstr the point ucted amplitudes source P10 for from several PAS N CGH; an and r. d (c) normalized reconstructed amplitudes for several Nso so and r. Figure 6. (a) Normalized reconstructed amplitude of P10 for different Nso and r; (b) Reconstruction Figure 6. (a) Normalized reconstructed amplitude of P for different N and r; (b) Reconstruction time for a CGH of 10 so time for a CGH of 1 million points for different segment Nso and factors r without (PAS) and with 1 million points for different segment N and factors r without (PAS) and with (OC-PAS) occlusion culling. so Figure 6. (a) Normalized reconstructed amplitude of P10 for different Nso and r; (b) Reconstruction (OC-PAS) occlusion culling. time for a CGH of 1 million points for different segment Nso and factors r without (PAS) and with The second test is carried out for measuring the computation time of calculated (OC-PAS) occlusion culling. CGH with a point cloud of one million points. The cloud has a similar point density The second test is carried out for measuring the computation time of calculated CGH to the gargoyle figurine presented in Section 3. Different N and r values without and so with a point cloud of one million points. The cloud has a similar point density to the gar- with occlusion culling routine are employed. The results are presented in Figure 6b. The second test is carried out for measuring the computation time of calculated CGH goyle figurine presented in Section Note that PAS calculation 3. Different of a segment Nso and consists r va of lues twowi parts: thout a allocation nd wi of th occl holographic u- with a point cloud of one million points. The cloud has a similar point density to the gar- information according to the local instantaneous discrete frequencies and performing FFT. sion culling routine are employed. The results are presented in Figure 6b. Note that PAS goyle figurine presented in Section 3. Different Nso and r values without and with occlu- Time consumption for allocation depends on the number of points, and thus the larger the calculation of a segment consists of two parts: allocation of holographic information ac- sion culling routine are employed. The results are presented in Figure 6b. Note that PAS cloud is, the larger the computational effort. Time consumption is particularly demanding cording to the local instantaneous discrete frequencies and performing FFT. Time con- calculation of a segment consists of two parts: allocation of holographic information ac- when the number of points within the cloud is counted in millions. In comparison, the sumption for allocation depends on the number of points, and thus the larger the cloud computational effort of the FFT only depends on the size of the frequency matrix (N  r), so cording to the local instantaneous discrete frequencies and performing FFT. Time con- where the holographic information is stored. Since the computational complexity of is, the larger the computational effort. Time consumption is particularly demanding when sumption for allocation depends on the number of points, and thus the larger the cloud FFT is given by (N  r)  log(N  r), then a small product of N and r will result so so so the number of points within the cloud is counted in millions. In comparison, the compu- is, the larger the computational effort. Time consumption is particularly demanding when in a short computation time, which is almost negligible, while a large product between tational effort of the FFT only depends on the size of the frequency matrix (Nso × r), where the number of points within the cloud is counted in millions. In comparison, the compu- N and r, will increase it. However, this FFT-related time is still shorter than for point so the holographic information is stored. Since the computational complexity of FFT is given processing. A benefit of using large segments N is that less segments are computed for tational effort of the FFT only depends on the size of the frequency matrix (Nso × r), where so hologram generation, which speeds up the CGH computation. When measuring time the holographic information is stored. Since the computational complexity of FFT is given for the occlusion culling, it is seen that the occlusion PAS is faster than normal PAS. For example, generation of the CGH without occlusion for N = 32 and r = 3, and N = 512 and so so Photonics 2021, 8, 298 12 of 16 r = 3 took around 674 and 4.6 s, respectively. On the other hand, calculation of the CGH with occlusion for previous parameters took around 297 s and 3.9 s, respectively. Notably, for segments of size N  r that do not fulfill Equation (11) the growth of computation so time is higher for the occlusion culling than for the computation without occlusion. This is because of the finer frequency sampling of the segment, which means that back points are allowed to store their holographic information. Therefore, the number of processed points grows, and thus the computation time as well. As discussed in former paragraph, selection of N and r for ensuring high accuracy so and fast execution time must consider the constraint imposed by Equation (11). However, Equation (11) relies in the angular spacing D. The parameter D depends on the geometry of the object, the dimension of the hologram, and the separation between object and hologram planes. Due to the mentioned features, the estimation of the angular spacing is a non-trivial issue. Here, it is decided to estimate the local angular spacing with the fitting of a Gaussian distribution to the computed local density of points. Also, considering the standard deviations, we estimate the quasi-minimal angular spacing value for the points clouds that statistically work properly for over 99% of the object surface. Table 2 presents the connection between the average angular spacing of the point cloud and the proper segment size N as it is stated in Equation (11). Four point clouds with the same geometry so from gargoyle figure from Section 3 are employed. These clouds have 200 thousand, 1 million, 5 million, and 24 million of points. Investigation of these exemplary point clouds presents the proper choice of the segment size due to the point cloud angular spacing and their respective CGH time calculation. Table 2. Calculated segment size respect to the average point cloud angular spacing for neighboring points and computation time of CGH-PAS without and with occlusion culling. 200 Thousand 1 Million 5 Million 24 Million Local angular spacing 0.4051 0.1806 0.0821 0.035 N —Equation (11) 146 329 723 1563 N (r = 1) 128 256 512 1024 so PAS without OC 7.7 s 9.4 s 14.0 s 31.0 s PAS with OC 4.5 s 6.1 s 8.9 s 20.5 s 6. Reconstruction of Wide-Angle CGHs Calculated with the Occlusion Culling Based-PAS Algorithm In this section, numerical and optical reconstructions of wide-angle CGHs obtained with the occlusion culling based PAS are carried out. PAS algorithm is implemented in Matlab software, which is installed on a PC equipped with Intel i9-9900K processor, 32 GB of RAM, and run without support of GPU. Selected size of the CGHs is 4160  2160 pixels, which corresponds to the size of the 4K SLM Holoeye GAEA 2,0, pixel pitch D = 3.74 m. SLM CGHs are calculated from the cloud of 24 million points defined in Section 3 and PAS parameters: N = 512, r = 3, and D = 0.62 m, which according to Table 2 fulfills N limit. so s Calculation time for each CGH is around 190 s, which can be reduced up to 30 s when using the Matlab parallel toolbox. When comparing this time with Table 1 from reference [29], it can be seen that our approach improves the calculation time by a factor of 3000. Figure 7a–c show numerical reconstructions for three perspectives angles: 0 , 45 , and 180 of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of the occlusion culling based PAS algorithm allows eliminating unwanted back points, and hence, reconstruction of the object with proper depth cues. It is worth noting that SFI method is not able to provide any of the presented results since it is a method designed under the constraints of the paraxial regime. Video S1 presents the numerical reconstruction of the gargoyle view angles from 0 to 360 . The cloud of points was rotated 8 between each frame. Photonics 2021, 8, x FOR PEER REVIEW 13 of 16 Photonics 2021, 8, x FOR PEER REVIEW 13 of 16 Figure 7a–c show numerical reconstructions for three perspectives angles: 0°, 45°, and 180° of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of the occlusion culling based PAS algorithm allows eliminating unwanted back points, and Figure 7a–c show numerical reconstructions for three perspectives angles: 0°, 45°, and hence, reconstruction of the object with proper depth cues. It is worth noting that SFI 180° of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of method is not able to provide any of the presented results since it is a method designed the occlusion culling based PAS algorithm allows eliminating unwanted back points, and under the constraints of the paraxial regime. Visualization 1 presents the numerical recon- hence, reconstruction of the object with proper depth cues. It is worth noting that SFI struction of the gargoyle view angles from 0° to 360°. The cloud of points was rotated 8° method is not able to provide any of the presented results since it is a method designed between each frame. under the constraints of the paraxial regime. Visualization 1 presents the numerical recon- Photonics 2021, 8, 298 13 of 16 struction of the gargoyle view angles from 0° to 360°. The cloud of points was rotated 8° between each frame. Figure 7. 2D MFFT AS reconstructions for different perspective angles: (a) 0°, (b) 45°, and (c) 180°. Optical reconstruction of CGH is carried out in the display setup presented in Figure Figure 7. 2D MFFT AS reconstructions for different perspective angles:  (a) 0°,  (b) 45°, and  (c) 180°. Figure 7. 2D MFFT AS reconstructions for different perspective angles: (a) 0 , (b) 45 , and (c) 180 . 8. Illumination laser source (λR = 640 nm) hits half-wave plate (HWP), which controls po- Optical reconstruction of CGH is carried out in the display setup presented in Figure 8. larization state. Next, the beam passes through microscopic objective (MO) and a pinhole Optical reconstruction of CGH is carried out in the display setup presented in Figure Illumination laser source ( = 640 nm) hits half-wave plate (HWP), which controls polar- (PH), generating a spherical wavefront. A collimator lens C (Fc = 300 mm) is placed in front 8. Illumination laser source (λR = 640 nm) hits half-wave plate (HWP), which controls po- ization state. Next, the beam passes through microscopic objective (MO) and a pinhole of the pinhole to obtain an extended plane wave. Plane wave illumination is directed to larization state. Ne (PH), xt, th generating e beam pa asse spherical s through m wavefront. icros Acop collimator ic object lens ive (MO C (F =)300 and mm) a pis inhole placed in the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal front of the pinhole to obtain an extended plane wave. Plane wave illumination is directed (PH), generating a spherical wavefront. A collimator lens C (Fc = 300 mm) is placed in front demagnifying system composed of an objective (F1 = 200 mm, Ø1 = 50 mm) and an eyepiece to the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal of the pinhole to obtain an extended plane wave. Plane wave illumination is directed to demagnifying system composed of an objective (F = 200 mm, Ø = 50 mm) and an eyepiece 1 1 (F2 = 33 mm, Ø2 = 25 mm). At the viewing window (VW) plane pixel pitch of the image of the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal (F = 33 mm, Ø = 25 mm). At the viewing window (VW) plane pixel pitch of the image of 2 2 SLM is Δ’SLM = F2/F1 × ΔSLM = 0.62 µm. The small pixel pitch allows for a wide viewing angle demagnifying system composed of an objective (F1 = 200 mm, Ø1 = 50 mm) and an eyepiece SLM is D’ = F /F  D = 0.62 m. The small pixel pitch allows for a wide viewing SLM 2 1 SLM (62.5°). However, the physical aperture of the eyepiece limits it to 36°. angle (62.5 ). However, the physical aperture of the eyepiece limits it to 36 . (F2 = 33 mm, Ø2 = 25 mm). At the viewing window (VW) plane pixel pitch of the image of SLM is Δ’SLM = F2/F1 × ΔSLM = 0.62 µm. The small pixel pitch allows for a wide viewing angle (62.5°). However, the physical aperture of the eyepiece limits it to 36°. Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. Figure 9 shows optical reconstructions of CGH generated and reconstructed for four Figure 9 shows optical reconstructions of CGH generated and reconstructed for four Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. view angles: 0 , 45 , 180 and 275 . To fit the viewing angle of the display system, gargoyle view angles: 0°, 45°, 180° and 275°. To fit the viewing angle of the display system, gargoyle dimensions are resized to 559  527  279 [mm ]. Since the dimension of the cloud is dimensions are resized to 559 × 527 × 279 [mm ]. Since the dimen  sion of the cloud is re- reduced, the average angular spacing of the point cloud decreases to 0.030 . For this value, Figure 9 shows optical reconstructions of CGH generated and reconstructed for four duced, the average angular spacing of the point cloud decreases to 0.030°. For this value, the N must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy view angles: 0°, 45°, 180° and 275°. To fit the viewing angle of the display system, gargoyle and speed for CGH calculation as well. Obtained results prove that the occlusion culling the NS must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy dimensions are resized to 559 × 527 × 279 [mm ]. Since the dimension of the cloud is re- based-PAS can eliminate unwanted back points for each angle. Video S2 presents the and speed for CGH calculation as well. Obtained results prove that the occlusion culling duced, the average angular spacing of the point cloud decreases to 0.030°. For this value, optical reconstruction of the gargoyle from 0 to 360 . Here, the cloud of points was rotated based-PAS can eliminate unwanted back points for each angle. Visualization 2 presents 1 per frame. the NS must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy and speed for CGH calculation as well. Obtained results prove that the occlusion culling based-PAS can eliminate unwanted back points for each angle. Visualization 2 presents Photonics 2021, 8, x FOR PEER REVIEW 14 of 16 Photonics 2021, 8, 298 14 of 16 the optical reconstruction of the gargoyle from 0° to 360°. Here, the cloud of points was rotated 1° per frame. Figure 9. Optical reconstructions of CGHs generated with culling-occlusion-PAS-based algorithm Figure 9. Optical reconstructions of CGHs generated with culling-occlusion-PAS-based algorithm for different view angles: for different view angles: (a) 0°, (b) 45°, (c) 180°, and (d) 275°. (a) 0 , (b) 45 , (c) 180 , and (d) 275 . 7. Conclusions 7. Conclusions In this work, we proposed a simple, effective and fast occlusion culling strategy for In this work, we proposed a simple, effective and fast occlusion culling strategy for wide-angle view holograms. Our approach takes advantage of PAS algorithm, which is based on mapping of 3D object information from spatial to frequency domain. It is shown wide-angle view holograms. Our approach takes advantage of PAS algorithm, which is that optimal segment size that enables occlusion of back points can be calculated according based on mapping of 3D object information from spatial to frequency domain. It is shown to the average angular spacing of the point cloud. Moreover, sorting of the cloud of points that optimal segment size that enables occlusion of back points can be calculated accord- in the z-axis allows ordering the point sources from the nearest to the farthest respect to the ing to the average angular spacing of the point cloud. Moreover, sorting of the cloud of hologram plane. In this way, sources that are closer to the hologram plane are labeled as points in the z-axi fr s ont allones ows and orderi allocate ng the poi their holographic nt sources f information rom the nearest to the f into the frequency matrix arthest re- first. Other point sources of the same frequency but farther from the hologram plane are labeled as spect to the hologram plane. In this way, sources that are closer to the hologram plane are back ones, and thus, their information into the frequency matrix is omitted. Moreover, two labeled as front ones and allocate their holographic information into the frequency matrix methods for numerical reconstruction of wide-angle view CGH were studied. The first first. Other point sources of the same frequency but farther from the hologram plane are was the OADI method, which allows reconstructing a small window from the whole image labeled as back ones, and thus, their information into the frequency matrix is omitted. space. In this way, efficient numerical technique is obtained, which enables measuring the diffraction efficiency of the PAS algorithm. The second was the 2D MFFT-AS, which Moreover, two methods for numerical reconstruction of wide-angle view CGH were stud- allows reconstructing the full image space. It was employed for reconstructing the whole ied. The first was the OADI method, which allows reconstructing a small window from object and visualize the quality of the occlusion culling that was carried out with the the whole image space. In this way, efficient numerical technique is obtained, which ena- PAS. Finally, optical reconstruction of occluded CGHs in wide-angle holographic display bles measuring the diffraction efficiency of the PAS algorithm. The second was the 2D verified the validity of our occlusion culling strategy. Notably, our methodology enables fast computation time for obtaining occluded CGHs from very large point clouds. MFFT-AS, which allows reconstructing the full image space. It was employed for recon- structing the whole object and visualize the quality of the occlusion culling that was car- Supplementary Materials: The following are available online at https://www.mdpi.com/article/ ried out with the PAS. Finally, optical reconstruction of occluded CGHs in wide-angle 10.3390/photonics8080298/s1, Video S1: The numerical reconstruction of the gargoyle view angles holographic display verified the validity of our occlusion culling strategy. Notably, our from 0 to 360 ; Video S2: The optical reconstruction of the gargoyle from 0 to 360 . methodology enables fast computation time for obtaining occluded CGHs from very large Author Contributions: Conceptualization and methodology, J.M.-C.; software, R.K.; experimental point clouds. validation, M.C. and M.S.I.; supervision, T.K. All authors have read and agreed to the published version of the manuscript. Author Contributions: Conceptualization and methodology, J.M.-C.; software, R.K.; experimental Funding: This research was funded by National Science Centre, Poland grant number UMO- 2018/31/B/ST7/02980. validation, M.C. and M.S.I.; supervision, T.K. All authors have read and agreed to the published version of the manuscript. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Funding: This research was funded by National Science Centre, Poland grant number UMO- 2018/31/B/ST7/02980. Data Availability Statement: Data underlying the results presented in this paper are publicly available at this time but may be obtained from the authors upon request. Institutional Review Board Statement: Not applicable Acknowledgments: The authors would like to thank to the Warsaw University of Technology for the Informed Consent Statement: Not applicable. statutory funds provided to this research. Conflicts of Interest: The authors declare no conflict of interest. Data Availability Statement: Data underlying the results presented in this paper are publicly avail- able at this time but may be obtained from the authors upon request. Acknowledgments: The authors would like to thank to the Warsaw University of Technology for the statutory funds provided to this research. Conflicts of Interest: The authors declare no conflict of interest. References 1. Yamaguchi, M. Full-Parallax Holographic Light-Field 3-D Displays and Interactive 3-D Touch. Proc. IEEE 2017, 105, 947–959. 2. Bove, V.M. Display holography’s digital second act. Proc. IEEE 2012, 100, 918–928. Photonics 2021, 8, 298 15 of 16 References 1. Yamaguchi, M. Full-Parallax Holographic Light-Field 3-D Displays and Interactive 3-D Touch. Proc. IEEE 2017, 105, 947–959. [CrossRef] 2. Bove, V.M. Display holography’s digital second act. Proc. IEEE 2012, 100, 918–928. [CrossRef] 3. Kim, M.K. Principles and techniques of digital holographic microscopy. SPIE Rev. 2010, 1, 1–50. [CrossRef] 4. Dong, J.; Jiang, C.; Jia, S. Digital holographic metrology based on multi-angle interferometry. Opt. Lett. 2016, 41, 4301. [CrossRef] 5. Grujic, ´ D.; Vasiljevic, ´ D.; Pantelic, ´ D.; Tomic, ´ L.; Stamenkovic, ´ Z.; Jelenkovic, ´ B.; Zhang, W.; Zhang, D.; Vukmirovic, ´ N.; Stepanenko, D. Naturally safe: Cellular noise for document security. J. Biophotonics 2019, 12, 1–9. 6. Khan, G.S.; Mantel, K.; Harder, I.; Lindlein, N.; Schwider, J. Design considerations for the absolute testing approach of aspherics using combined diffractive optical elements. Appl. Opt. 2007, 46, 7040–7048. [CrossRef] [PubMed] 7. Zhang, H.; Zhao, Y.; Cao, L.; Jin, G. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Opt. Express 2015, 23, 3901–3913. [CrossRef] 8. Lee, S.; Jo, Y.; Yoo, D.; Cho, J.; Lee, D.; Lee, B. Tomographic near-eye displays. Nat. Commun. 2019, 10, 1–10. [CrossRef] 9. Kozacki, T.; Martinez-Carranza, J.; Kukolowicz, R.; Finke, W. Accurate reconstruction of horizontal parallax-only holograms by angular spectrum and efficient zero-padding. Appl. Opt. 2020, 59, 8450–8458. [CrossRef] 10. Isomae, Y.; Shibata, Y.; Ishinabe, T.; Fujikake, H. Design of 1-m-pitch liquid crystal spatial light modulators having dielectric shield wall structure for holographic display with wide field of view. Opt. Rev. 2017, 24, 165–176. [CrossRef] 11. Higashida, R.; Funabashi, N.; Aoshima, K.; Miura, M.; Machida, K. Diffraction of light using high-density magneto-optical light modulator array. Opt. Eng. 2020, 59, 1–13. [CrossRef] 12. Buckley, E. Holographic projector with one lens. Opt. Lett. 2010, 35, 3399–3401. [CrossRef] 13. Blinder, D.; Chlipala, M.; Kozacki, T.; Schelkens, P. Photorealistic computer generated holography with global illumination and path tracing. Opt. Lett. 2021, 46, 2188–2191. [CrossRef] 14. Wakunami, K.; Yamashita, H.; Yamaguchi, M. Occlusion culling for computer generated hologram based on ray-wavefront conversion. Opt. Express 2013, 21, 21811–21822. [CrossRef] 15. Matsushima, K.; Nakahara, S. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method. Appl. Opt. 2009, 48, H54–H63. [CrossRef] [PubMed] 16. Zhang, H.; Cao, L.; Jin, G. Computer-generated hologram with occlusion effect using layer-based processing. Appl. Opt. 2017, 56, F138–F143. [CrossRef] 17. Yeom, H.-J.; Park, J.-H. Calculation of reflectance distribution using angular spectrum convolution in mesh-based computer generated hologram. Opt. Express 2016, 24, 19801–19813. [CrossRef] 18. Chen, R.H.Y.; Wilkinson, T.D. Computer generated hologram from point cloud using graphics processor. Appl. Opt. 2009, 48, 6841–6850. [CrossRef] 19. Chen, R.H.Y.; Wilkinson, T.D. Computer generated hologram with geometric occlusion using GPU-accelerated depth buffer rasterization for three-dimensional display. Appl. Opt. 2009, 48, 4246–4255. [CrossRef] [PubMed] 20. Symeonidou, A.; Blinder, D.; Munteanu, A.; Schelkens, P. Computer-generated holograms by multiple wavefront recording plane method with occlusion culling. Opt. Express 2015, 23, 22149–22161. [CrossRef] 21. Hong, J.; Kim, Y.; Choi, H.J.; Hahn, J.; Park, J.H.; Kim, H.; Min, S.-W.; Chen, M. Three-dimensional display technologies of recent interest: Principles, status, and issues. Appl. Opt. 2011, 50, 87–115. [CrossRef] [PubMed] 22. Gilles, A.; Gioia, P.; Cozot, R.; Morin, L. Complex modulation computer-generated hologram with occlusion effect by a fast hybrid point-source/wave-field approach. In Proceedings of the 2015 Conference: Pacific Conference on Graphics and Applications, Beiging, China, 7–9 October 2015; pp. 1–9. 23. Askari, M.; Kim, S.-B.; Shin, K.-S.; Ko, S.-B.; Kim, S.-H.; Park, D.-Y.; Yu, Y.-G.; Park, J.-H. Occlusion handling using angular spectrum convolution in fully analytical mesh-based computer generated hologram. Opt. Express 2017, 25, 25867–25878. [CrossRef] 24. Chen, J.-S.; Chu, D.P. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. Opt. Express 2015, 23, 18143–18155. [CrossRef] [PubMed] 25. Kang, H.; Stoykova, E.; Yoshikawa, H. Fast phase-added stereogram algorithm for generation of photorealistic 3D content. Appl. Opt. 2016, 55, A135–A143. [CrossRef] 26. Shimobaba, T.; Masuda, N.; Ito, T. Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane. Opt. Lett. 2009, 34, 3133–3136. [CrossRef] 27. Liu, S.; Wei, H.; Li, N.; Liu, Z.; Zhang, J. Occlusion calculation algorithm for computer generated hologram based on ray tracing. Opt. Commun. 2019, 443, 76–85. [CrossRef] 28. Zhao, K.; Huang, Y.; Yan, X.; Jiang, X. Occlusion culling and calculation for a computer generated hologram using spatial frequency index method. J. Opt. 2015, 17, 1–9. [CrossRef] 29. Yamaguchi, M.; Hoshino, H.; Honda, T.; Ohyama, N. Phase-Added Stereogram: Calculation of Hologram Using Computer Graphics Technique. In Proceedings of the 1993 Conference: Practical Holography VII: Imaging and Materials, San Jose, CA, USA, 17 September 1993; pp. 25–31. 30. Kang, H.; Yamaguchi, T.; Yoshikawa, H. Accurate phase-added stereogram to improve the coherent stereogram. Appl. Opt. 2008, 47, D44–D54. [CrossRef] Photonics 2021, 8, 298 16 of 16 31. Kang, H. Compensated phase-added stereogram for real-time holographic display. Opt. Eng. 2007, 46, 1–11. [CrossRef] 32. Shen, F.; Wang, A. Fast-Fourier-transform based numerical integration method for the Rayleigh-Sommerfeld diffraction formula. Appl. Opt. 2006, 45, 1102–1110. [CrossRef] 33. Leseberg, D.; Frère, C. Computer-generated holograms of 3-D objects composed of tilted planar segments. Appl. Opt. 1988, 27, 3020–3024. [CrossRef] [PubMed] 34. Yatagai, T. Stereoscopic approach to 3-D display using computer-generated holograms. Appl. Opt. 1976, 15, 2722–2729. [CrossRef] 35. Tang, C.; Yao, B. Fast computation for generating CGH of a 3D object by employing connections between layers. J. Mod. Opt. 2012, 59, 1406–1409. [CrossRef] 36. Shimobaba, T.; Kakue, T.; Ito, T. Review of Fast Algorithms and Hardware Implementations on Computer Holography. IEEE Trans. Ind. Inform. 2016, 12, 1611–1622. [CrossRef] 37. Zhao, Y.; Cao, L.; Zhang, H.; Kong, D.; Jin, G. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Opt. Express 2015, 23, 25440–25449. [CrossRef] [PubMed] 38. Ju, Y.-G.; Park, J.-H. Foveated computer-generated hologram and its progressive update using triangular mesh scene model for near-eye displays. Opt. Express 2019, 27, 23725–23738. [CrossRef] [PubMed] 39. Katz, S.; Tal, A.; Basri, R. Direct visibility of point sets. ACM Trans. Graph. 2007, 26, 1–11. [CrossRef] 40. Veerman, J.A.C.; Rusch, J.J.; Urbach, H.P. Calculation of the Rayleigh–Sommerfeld diffraction integral by exact integration of the fast oscillating factor. J. Opt. Soc. Am. A 2005, 22, 636–646. [CrossRef] 41. Kozacki, T.; Falaggis, K.; Kujawinska, M. Computation of diffracted fields for the case of high numerical aperture using the angular spectrum method. Appl. Opt. 2012, 51, 7080–7088. [CrossRef] [PubMed] 42. Matsushima, K. Shifted angular spectrum method for off-axis numerical propagation. Opt. Express 2010, 18, 18453–18463. [CrossRef] [PubMed] 43. Zhang, W.; Zhang, H.; Matsushima, K.; Jin, G. Shifted band-extended angular spectrum method for off-axis diffraction calculation. Opt. Express 2021, 29, 10089–10103. [CrossRef] 44. Kozacki, T.; Finke, G.; Garbat, P.; Zaperty, W.; Kujawinska, ´ M. Wide angle holographic display system with spatiotemporal multiplexing. Opt. Express 2012, 20, 27473–27481. [CrossRef] [PubMed] 45. Kanka, M.; Wuttig, A.; Graulig, C.; Riesenberg, R. Fast exact scalar propagation for an in-line holographic microscopy on the diffraction limit. Opt. Lett. 2010, 35, 217–219. [CrossRef] [PubMed] 46. Hillenbrand, M.; Hoffmann, A.; Kelly, D.P.; Sinzinger, S. Fast nonparaxial scalar focal field calculations. J. Opt. Soc. Am. A 2014, 31, 1206–1214. [CrossRef] http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Photonics Multidisciplinary Digital Publishing Institute

Occlusion Culling for Wide-Angle Computer-Generated Holograms Using Phase Added Stereogram Technique

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/occlusion-culling-for-wide-angle-computer-generated-holograms-using-gmBHQG0ZrA
Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2021 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer The statements, opinions and data contained in the journals are solely those of the individual authors and contributors and not of the publisher and the editor(s). MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Terms and Conditions Privacy Policy
ISSN
2304-6732
DOI
10.3390/photonics8080298
Publisher site
See Article on Publisher Site

Abstract

hv photonics Article Occlusion Culling for Wide-Angle Computer-Generated Holograms Using Phase Added Stereogram Technique Juan Martinez-Carranza *, Tomasz Kozacki, Rafał Kukołowicz , Maksymilian Chlipala and Moncy Sajeev Idicula Faculty of Mechatronics, Warsaw University of Technology, 8 Sw. A. Boboli Street, 02-525 Warsaw, Poland; t.kozacki@mchtr.pw.edu.pl (T.K.); rafal.kukolowicz.dokt@pw.edu.pl (R.K.); M.Chlipala@mchtr.pw.edu.pl (M.C.); moncy.idicula.dokt@pw.edu.pl (M.S.I.) * Correspondence: j.martinez@mchtr.pw.edu.pl Abstract: A computer-generated hologram (CGH) allows synthetizing view of 3D scene of real or virtual objects. Additionally, CGH with wide-angle view offers the possibility of having a 3D experience for large objects. An important feature to consider in the calculation of CGHs is occlusion between surfaces because it provides correct perception of encoded 3D scenes. Although there is a vast family of occlusion culling algorithms, none of these, at the best of our knowledge, consider occlusion when calculating CGHs with wide-angle view. For that reason, in this work we propose an occlusion culling algorithm for wide-angle CGHs that uses the Fourier-type phase added stereogram (PAS). It is shown that segmentation properties of the PAS can be used for setting efficient conditions for occlusion culling of hidden areas. The method is efficient because it enables processing of dense cloud of points. The investigated case has 24 million of point sources. Moreover, quality of the occluded wide-angle CGHs is tested by two propagation methods. The first propagation technique quantifies quality of point reproduction of calculated CGH, while the second method enables the quality assessment of the occlusion culling operation over an object of complex shape. Finally, the applicability of proposed occlusion PAS algorithm is tested by synthetizing wide-angle CGHs that are numerically and optically reconstructed. Citation: Martinez-Carranza, J.; Kozacki, T.; Kukołowicz, R.; Chlipala, M.; Idicula, M.S. Occlusion Culling Keywords: computer-generated hologram; phase added stereogram algorithm; occlusion; frequency for Wide-Angle Computer-Generated domain; Fourier transform; propagation method; wide-angle view; direct integration method; tiling Holograms Using Phase Added Stereogram Technique. Photonics 2021, 8, 298. https://doi.org/10.3390/ photonics8080298 1. Introduction The aim of 3D display technology is to reproduce the depth perception of surrounding Received: 2 July 2021 world. An important feature that must be considered in this technology is the ability to Accepted: 23 July 2021 reproduce image providing a complete 3D experience to the viewers [1], i.e., full-parallax. Published: 27 July 2021 Among several solutions that have been proposed in the last decades for developing 3D display technology, digital holography constitutes the best-known framework due Publisher’s Note: MDPI stays neutral to the ability of truly reconstructing the 3D wavefront [2]. The possibility of complex with regard to jurisdictional claims in wavefront reproduction makes digital holography popular choice in several areas as mi- published maps and institutional affil- croscopy [3], metrology [4], non-destructive sensing [5], security [6], design of diffractive iations. optical elements [7]. In the realm of display technology, digital holography enables accurate reproduction of physiological cues [8,9], which is important for photorealistic 3-D images. However, digital holographic systems cannot capture scenes for wide-angle viewing dis- play. With the development of computing technology, holograms can be registered without Copyright: © 2021 by the authors. the necessity of a physical holographic setup. Computer-generated hologram (CGH) does Licensee MDPI, Basel, Switzerland. not have limitations of physical systems [10] such as optical aberrations, extension, noise, This article is an open access article or sensitivity to vibrations. Nevertheless, CGHs have limitations for providing the truly 3D distributed under the terms and experience. The first limitation is related with the pixel pitch of the spatial light modulators conditions of the Creative Commons (SLMs) that are used for displaying CGHs. To have a full-parallax, pixel pitches below one Attribution (CC BY) license (https:// micrometer must be used. High-end SLM technologies can offer SLMs with pixel pitch up creativecommons.org/licenses/by/ to 3.74 m and 4K resolution. Devices in research stage have shown that the pixel pitch 4.0/). Photonics 2021, 8, 298. https://doi.org/10.3390/photonics8080298 https://www.mdpi.com/journal/photonics Photonics 2021, 8, 298 2 of 16 can be reduced up to one micrometer [11,12]. However, this type of technology is still developing. Fortunately, pixel pitch of an SLM can be reduced indirectly by employing optical systems [13], which solves the problem of wide-angle view. The second limitation of CGHs is related to generation of the necessary visual cues for reconstructing photorealistic 3-D images, such as shading, occlusions, transparency, texture mapping, light reflectance properties [14–18]. An important visual cue that any CGH technique must consider is oc- clusion because it provides information about perception of depth [16]. Implementation of occlusion is a challenging task in CGH since the whole information of a 3D object is coded in 2D hologram. A large variety of occlusion algorithms have been designed depending on the employed CGH technique [16,17,19–22]. Propagation-based CGH techniques can apply occlusion culling of unseen regions [13,17,21,23,24]. A popular approach for obtaining CGHs is employing point clouds of 3D objects. Each point in the cloud act as a point source that is well localized in space with a defined amplitude. Point sources emit a spherical wavefront that propagates to the hologram plane. Superposition of all propagated wavefronts generates the CGH [25–28]. However, occlusion processing for cloud point methods is not a simple task. There are several solutions that can solve this problem. For example, reference [21] employs inter-wave- recording-plane light propagation and masks, which has proved to be efficient. Other researches use the stereogram holographic approach to cull hidden back points [8,29]. A simple technique, called spatial frequency index (SFI), is presented in reference [29]. This approach calculates the instantaneous frequencies of the points with respect to the corresponding hologram tile. Instantaneous frequencies are group in sets of the same frequency, which provides the corresponding occlusion relation with the object points. Finally, the point in the set that is closer to the hologram plane is the only candidate to apport holographic information to the CGH. The SFI method can be understood as a phase added stereogram (PAS) algorithm [30–32] that enables occlusion culling. However, the SFI strategy is time consuming, because it does not employ Fourier transformation (FT). Instead, it uses tables for grouping and selecting the proper frequencies, and what is more important, the size of the tiling is small. All mentioned approaches enable occlusion culling of unseen areas. However, none of these techniques consider the wide projection angle. For example, propagation-based occlusion techniques are not optimal for CGHs with wide-angle, since they do not usually consider geometry where the object is much larger than the hologram [25]. This geomet- rical feature is challenging for rigorous propagation algorithms that must be used. For point-occlusion based methods, the wide-angle problem is not considered. In this work, we propose a methodology for occlusion culling based on the Fourier type-PAS algorithm. Unlike the SFI strategy, our approach does not require tables for grouping the correspond- ing frequencies, and we provide the condition for selecting the sub-hologram tile size for carrying out proper occlusion culling. This allows reducing the processing time of the CGH. The proposed occlusion-PAS-based algorithm enables fast calculation of wide-angle CGHs with the removal of hidden surfaces, which has not been shown until now. The developed PAS method is tested for wide-angle CGH, i.e., small pixel pitch. Moreover, two propagation solutions are proposed for testing the accuracy and quality of the occlusion- based PAS algorithm. The first, based on the direct integration method [33], quantifies quality of point reproduction of calculated CGH. The second, which is a generalization of the multi Fast Fourier Transform Angular Spectrum (MFFT-AS) method [10], assesses the quality of the occlusion culling operation over a 3D object assembled of 24 million of point sources. Theoretical capabilities of our solutions are tested for proving accuracy of computations. Finally, the applicability of our occlusion culling method is carried out numerically and experimentally. Photonics 2021, 8, x FOR PEER REVIEW 3 of 16 Photonics 2021, 8, 298 3 of 16 2. Synthetizing CGH with the PAS 2. Synthetizing A hologram CGH H is obt with the ained PAS as the result of measuring the intensity that results from the linear superposition of a diffracted object wavefield O and a reference wavefield R. A hologram H is obtained as the result of measuring the intensity that results from The detected intensity can be mathematically expressed as follows the linear superposition of a diffracted object wavefield O and a reference wavefield R. The detected intensity can be mathematically expressed as follows HO=+R +OR**+O R. (1) 2 2 H = jOj +jRj + OR +O R. (1) The hologram is a set of dark and bright fringes that encode amplitude and phase The hologram is a set of dark and bright fringes that encode amplitude and phase information of the object. When using a digital camera for registering H, a digital holo- information of the object. When using a digital camera for registering H, a digital hologram gram (DH) is obtained. DH can be processed numerically for reconstructing the encoded (DH) is obtained. DH can be processed numerically for reconstructing the encoded object object information. However, capturing of DHs requires interferometric systems that are information. However, capturing of DHs requires interferometric systems that are sensitive sensitive to optical aberrations, environmental and mechanical vibrations, or noise. to optical aberrations, environmental and mechanical vibrations, or noise. With the development of computing technology, holograms can be registered with- With the development of computing technology, holograms can be registered without out the necessity of a physical holographic setup. When synthetizing CGH, only the object the necessity of a physical holographic setup. When synthetizing CGH, only the object wave field O is calculated. Solutions for calculating this wavefield have been studied for wave field O is calculated. Solutions for calculating this wavefield have been studied for more than three decades [30,34–37]. The most popular approach consists in decomposing more than three decades [30,34–37]. The most popular approach consists in decomposing the 3D object into a set of illuminating point sources that are distributed in space. Each the 3D object into a set of illuminating point sources that are distributed in space. Each point point source emits a spherical wavefront that propagates to the hologram plane. Super- source emits a spherical wavefront that propagates to the hologram plane. Superposition position of all propagated wavefronts generate the CGH as follows of all propagated wavefronts generate the CGH as follows p a ikD p pikD O = e , (2) Oe = , (2) p=1 D p=1 2 2 where D = (x x ) + (y y ) + z , (x , y , z ) are the spatial position and a 22 2 p 2 p 2 p p p p p where Dx=−()x +(y−y)+z , (xp, yp, zp) are the spatial position and ap amplitudes p 22pp p amplitudes of the point sources, respectively. Nevertheless, this solution is time consum- of the point sources, respectively. Nevertheless, this solution is time consuming [26,28]. ing [26,28]. Efficient techniques based on propagation have been proposed for calculating Efficient techniques based on propagation have been proposed for calculating O O [21,25,36,38,39]. With the aim to give the viewer a complete 3-D experience with full- [21,25,36,38,39]. With the aim to give the viewer a complete 3-D experience with full-par- parallax, wide-angle CGHs are required. The scheme of geometry for recording wide-angle allax, wide-angle CGHs are required. The scheme of geometry for recording wide-angle CGHs is presented in Figure 1. The distance between hologram plane (x , y ) and object 1 1 CGHs is presented in Figure 1. The distance between hologram plane ( 1x1, y1) and object plane (x , y ) is z. Field of View (FoV) in a CGH is calculated as FoV = 2sin (/2D), where 2 2 −1 plane (x2, y2) is z. Field of View (FoV) in a CGH is calculated as FoV = 2sin (λ/2Δ), where and D are the wavelength and pixel pitch of the hologram, respectively. Thus, wide-angle λ and Δ are the wavelength and pixel pitch of the hologram, respectively. Thus, wide- CGHs are obtained by employing small pixel pitch at the hologram plane. For example, for angle CGHs are obtained by employing small pixel pitch at the hologram plane. For ex- given  = 0.64 m and D = 0.62 m the FoV 62 . Large FoV allows synthetizing CGHs of ample, for given λ = 0.64 µm and Δ = 0.62 µm the FoV ≈ 62°. Large FoV allows synthetizing objects with spatial extension B = B = 2z tan(FoV). These spatial extensions are much x 2 y 2 CGHs of objects with spatial extension Bx2 = By2 = 2z tan(FoV). These spatial extensions are larger than the CGH size; for example, when using z = 1000 mm, B = B = 1209 mm x 2 y 2 much larger than the CGH size; for example, when using z = 1000 mm, Bx2 = By2 = 1209 mm while an a 4K CGH (4120 2060) will have the dimensions B = 2.6 mm and B = 1.3 mm. x 1 y 2 while an a 4K CGH (4120 × 2060) will have the dimensions Bx1 = 2.6 mm and By2 = 1.3 mm. Figure 1. Geometry for recording wide-angle CGHs. Figure 1. Geometry for recording wide-angle CGHs. Photonics 2021, 8, x FOR PEER REVIEW 4 of 16 Unfortunately, propagation-based techniques are not an optimal approach for syn- thetizing CGHs of wide-angle, since they do not consider geometry, where the object is much larger than the hologram [25]. This geometrical feature is challenging for rigorous propagation algorithms. Alternatives for efficient synthesis of CGHs are PAS-based solu- tions. These solutions do not have the restrictions of propagation-based methods since they consider the basic idea of summing point source responses at the hologram plane [28,31,32]. The PAS algorithm divides the CGH into smaller segments of size N s × Ns. The number of segments within the CGH is N × M, where N is the number of segments in the Photonics 2021, 8, 298 4 of 16 horizontal direction and M in the vertical direction. At each segment, PAS approximates the spherical wavefront that comes from the p-th point source of the object with a local tilted plane wave [31], as shown in Figure 2a. The direction of the plane wave is calculated Unfortunately, propagation-based techniques are not an optimal approach for syn- according to the value of the local instantaneous frequency at the center of the segment thetizing CGHs of wide-angle, since they do not consider geometry, where the object is cc much larger than the hologram [25]. This geometrical feature is challenging for rigorous x , y , where n = 1, …, N, and m = 1, …, M indicates segment position over the hori- () nm propagation algorithms. Alternatives for efficient synthesis of CGHs are PAS-based solu- tions. These solutions do not have the restrictions of propagation-based methods since they zontal and vertical direction, respectively. Since the frequency information of each seg- consider the basic idea of summing point source responses at the hologram plane [28,31,32]. ment is coded according to the local approximation of the tilted plane, the PAS can calcu- The PAS algorithm divides the CGH into smaller segments of size N  N . The number of s s late the diffracted field of objects larger than the hologram size. The term O for a given segments within the CGH is N  M, where N is the number of segments in the horizontal direction and M in the vertical direction. At each segment, PAS approximates the spherical segment (m,n) is calculated with the PAS as follows wavefront that comes from the p-th point source of the object with a local tilted plane wave [31], as shown in Figure 2a. The direction of the plane wave is calculated according to c c the value of the local instantaneous frequency at the center of the segment (x , y ), where p ikD n m cc pmn  Oe=− expi2πf (xx )+f (y−y ) , (3) n = 1, . . . , N, and m = 1, . . . , M indicates segment position over the horizontal and vertical mn () pxnm11 n pymn m  pmn direction, respectively. Since the frequency information of each segment is coded according p=1 to the local approximation of the tilted plane, the PAS can calculate the diffracted field of objects larger than the hologram size. The term O for a given segment (m,n) is calculated where Dpmn is the distance from p-th point source to the center of the segment (m,n), and with the PAS as follows ikD c c pmn sinθθ − sin O = e exp i2p f (x x ) + f (y y ) , (3) pxnm 1 pymn 1 mn å pxnm rxnm n m pmn p=1f = , (4) pxnm where D is the distance from p-th point source to the center of the segment (m,n), and pmn sin q sin q pxnm rxnm sinθθ − sin pynm rynm f = , (4) pxnm f = l , (5) pynm sin q sin q pynm rynm f = , (5) pynm are the local instantaneous frequencies. The angles θxpnm, θypnm, θrxnm, and θrynm are the inci- are the local instantaneous frequencies. The angles q , q , q , and q are the xpnm ypnm rxnm rynm dent angles of object and reference beams on x-axis and y-axis in relation to the center of incident angles of object and reference beams on x-axis and y-axis in relation to the center the segment, respectively. Notably, the hologram in the corresponding segment is a su- of the segment, respectively. Notably, the hologram in the corresponding segment is a superposition of plane waves. When the reference beam has a constant incident angle, perposition of plane waves. When the reference beam has a constant incident angle, spa- spatial frequencies depend on the object beam [31]. tial frequencies depend on the object beam [31]. Figure 2. CGH PAS scheme. (a) Spherical wavefront decomposition. (b) CGH synthesis. Figure 2. CGH PAS scheme. (a) Spherical wavefront decomposition. (b) CGH synthesis. Photonics 2021, 8, 298 5 of 16 Direct computation of Equation (3) can still be slow for a dense cloud of points. Nev- ertheless, this process can be significantly accelerated when employing the FT. When representing Equation (3) in frequency domain, the plane waves in the segment are trans- formed into a 2D distribution of Dirac deltas. The positions of these deltas need to be set to a discrete value that is specified by the segment size. Hence, Equations (4) and (5) are rewritten as: sin q sin q pxnm rxnm f = D f , (6) pxnm D f l 1 sin q sin q pynm rynm f = D f , (7) pynm D f l where Df = 1/N D is the frequency sampling, and b  c is the rounding operation. When employing Equations (6) and (7), the term O is expressed in the frequency domain as mn ikD ˆ ˆ ˆ pmn O = d f f , f f e . (8) x pxnm y pynm pmn p=1 After placing all the Dirac deltas for the 3D point distribution into the segment, the inverse FT is applied, and then the complex field for a given segment is obtained. This procedure is repeated for all segments for obtaining the CGH, as shown in Figure 2b. 3. Occlusion Culling of Back Points with the PAS Representation of 3D object for point-based approach offers a simple and flexible way of depicting its shape. Moreover, this representation allows associating the sources with additional information such as colors, textures, and normals [40]. Each point source generates a complex field from the scene towards the hologram plane, and the final CGH is the sum of all corresponding spherical waves. In the point-based representation, point sources cannot occlude one another, unless they accidentally fall along the same direction from the viewpoint. The wavefields from back and front points are encoded in the CGH. This prevents correct reproduction of depth cues of the 3D object in holographic image. In order to eliminate the back points, occlusion techniques are needed. Occlusion techniques can rely on geometric considerations [40], light shielding [17], or hybrid [15,23]. Since the CGH must provide motion parallax, the chosen occlusion technique must consider that the visibility of object changes according to the movement of the viewer. Occlusion can be introduced by employing the segmentation property of the PAS [29]. As shown in Equation (3), the term O for a given segment (m,n) is a superposition of plane waves having directional wave vectors calculated according to the center of the segment. When applying the FT to the segment, the coordinates of 3D point sources are mapped into the sets of 2D Dirac deltas in frequency domain. The frequency support of this segment is given by D . Since frequency allocation depends on the size of the segment, the gaps between adjacent Dirac deltas can be controlled in the frequency domain by the value of N . Figure 3 illustrates this effect for three point sources when using different segments sizes. The point sources x , and x are adjacent front points, while x is a back point. The angles p p +1 m of corresponding plane waves are q , q , and q , respectively. In this example, the angles p p +1 m fulfill the relation q < q < q . Figure 3a presents the case of a small segment. In this p m p +1 case, the wavefront information for all point sources is located in the same frequency pixel ˆ ˆ ˆ since f = f = f . For carrying out the occlusion, information related with the point p p+1 m source x needs to be disregarded because it has the farthest separation from the hologram plane. When enlarging the segment, as shown in Figure 3b, wavefront information of ˆ ˆ front points x and x is allocated in frequency pixels f and f , respectively. In this p p p+1 p+1 case, the information related to the source x will be placed to the nearest neighbor f . m p+1 Similarly to Figure 3a, the information related to the source x needs to be disregarded for the occlusion culling. Finally, for large segment the wavefront information of each point source is stored in independent pixels, as presented in Figure 3c. As seen in Figure 3, the Photonics 2021, 8, x FOR PEER REVIEW 6 of 16 separation from the hologram plane. When enlarging the segment, as shown in Figure 3b, wavefront information of front points xp and xp+1 is allocated in frequency pixels and , respectively. In this case, the information related to the source xm will be placed to p+1 the nearest neighbor . Similarly to Figure 3a, the information related to the source xm p+1 needs to be disregarded for the occlusion culling. Finally, for large segment the wavefront Photonics 2021, 8, 298 6 of 16 information of each point source is stored in independent pixels, as presented in Figure 3c. As seen in Figure 3, the key point of occlusion culling when using PAS is the allocation of frequency information of the sources that are closer to the hologram plane first, which key point of occlusion culling when using PAS is the allocation of frequency information of are labeled as front sources. These sources contribute the proper object information to the the sources that are closer to the hologram plane first, which are labeled as front sources. CGH. Other point sources of the same frequency but farther from the hologram plane are These sources contribute the proper object information to the CGH. Other point sources of labeled as back sources, which must be disregarded. Thus, sorting of the cloud in respect the same frequency but farther from the hologram plane are labeled as back sources, which must be disregarded. Thus, sorting of the cloud in respect to the z-axis allows identifying to the z-axis allows identifying points from the nearest to the farthest in respect to the points from the nearest to the farthest in respect to the hologram plane. hologram plane. Figure 3. Allocation of discrete frequencies in the PAS algorithm: (a) small segment, (b) medium Figure 3. Allocation of discrete frequencies in the PAS algorithm: (a) small segment, (b) medium segment, and (c) large segment. segment, and (c) large segment. With the aim to calculate the maximum segment size that enables occlusion, let’s consider a set of points that are distributed in the plane (x ,z). Then, adjacent points x 2 p With the aimand to ca x lcand ulaan te arbitrary the maxi hologram mum segment siz segment with center e thaat t ena x arb eles occlusi chosen. Theo dif n, fer le ence t’s p+1 between corresponding local frequencies is given by consider a set of points that are distributed in the plane (x2,z). Then, adjacent points xp and sin q sin q xp+1 and an arbitrary hologram segment with center at x are chosen. The difference be- pxn (p+1)xn f f = . (9) pxn (p+1)xn tween corresponding local frequencies is given by Note that  =  + D, where D is the angular difference between adjacent (p+1)nx pnx sinθθ − sin sources. To eliminate any gap between adjacent frequencies, the right-hand side of Equation (9) (1px + ) n pxn ff−= . (9) must be equal or smaller than the frequency sampling of the segment. This can be expressed as (1px + ) n pxn 2 sin(Dq/2) cos(q + Dq/2) pxn . (10) Note that θ(p+1)nx = θpnx + Δθ, where Δθ is the angular difference b etween adjacent N D l sources. To eliminate any gap between adjacent frequencies, the right-hand side of Equa- The denominator on the right-hand side of Equation (10) has been obtained by using tion (9) must be equal or smaller than the frequency sampling of the segment. This can be the sum-to-product identity. When meeting this condition, the two wavefronts corre- expressed as sponding to the discrete frequencies f and f are occupying two adjacent discrete pxn (p+1)xn locations in the frequency matrix. When considering a back point x , which view angle q m m lies between q and q , the holographic information associated with the frequency f p p+1 mxn 2sin(Δ+ θθ / 2) cos( Δθ / 2) pxn can be accommodated into the nearest neighbor f or f . Nevertheless, the contri- pxn (p+1)xn ≥ . (10) bution from f can be disregarded since the frequency positions are already occupied mxn N Δ λ by the front point sources, as shown in Figure 3b. Finally, maximum segment size can be found by solving Equation (10) for N , which yields The denominator on the right-hand side of Equation (10) has been obtained by using N < sin(Dq/2) cos(q + Dq/2) . (11) s pxn the sum-to-product identity. When meeting this condition, the two wavefronts corre- 2D sponding to the discrete frequencies f and f are occupying two adjacent dis- Notably, Equation (11) pxnallows calculating (1px + ) n the segment size that enables occlusion culling of back points. Once the occlusion culling of back points is complete, inverse FT is crete locations in the frequency matrix. When considering a back point xm, which view applied, and thus, the hologram segment with the proper depth perspective is obtained. angle θm lies between θp and θp+1, the holographic information associated with the fre- It is important to differentiate two cases in the occlusion culling. The first case can be exemplified by Figure 3a. In this case, a small segment is employed for the occlusion. As depicted in this figure, the holographic information of two sources (or more) is allocated in Photonics 2021, 8, x FOR PEER REVIEW 7 of 16 quency can be accommodated into the nearest neighbor f or f . Never- mxn pxn (1px + ) n theless, the contribution from can be disregarded since the frequency positions are mxn already occupied by the front point sources, as shown in Figure 3b. Finally, maximum segment size can be found by solving Equation (10) for Ns, which yields −1 N<Δ sin(θθ / 2) cos( +Δθ / 2) . (11) spxn 2Δ Notably, Equation (11) allows calculating the segment size that enables occlusion culling of back points. Once the occlusion culling of back points is complete, inverse FT is applied, and thus, the hologram segment with the proper depth perspective is obtained. It is important to differentiate two cases in the occlusion culling. The first case can be exemplified by Figure 3a. In this case, a small segment is employed for the occlusion. As depicted in this figure, the holographic information of two sources (or more) is allocated in the same frequency pixel. This means that resolution between the sources is lost. Thus, the smaller the segment is, the smaller the resolution in the CGH. On the other hand, when the chosen segment size is closer to Ns, wavefront information of the sources is placed in different frequency pixels, as presented in Figure 3b, which allows preserving resolution Photonics 2021, 8, 298 7 of 16 in the reconstructed image. Finally, it is important to note that position of the segment shifts allocation of the discrete frequencies but not the condition imposed by Equation (11). For segments that are too far from the center of the CGH, the shifting causes discrete the same frequency pixel. This means that resolution between the sources is lost. Thus, the −1 frequencies to fall outside from the frequency support Δ . smaller the segment is, the smaller the resolution in the CGH. On the other hand, when the chosen segment size is closer to N , wavefront information of the sources is placed in Visualization of the occlusion culling for a segment in the frequency domain is pre- different frequency pixels, as presented in Figure 3b, which allows preserving resolution sented in Figure 4. The employed object is a point cloud of a gargoyle figurine with di- in the reconstructed image. Finally, it is important to note that position of the segment mensions 1055 mm (height) × 971 mm (width) × 485 mm (depth) composed of approxi- shifts allocation of the discrete frequencies but not the condition imposed by Equation (11). For segments that are too far from the center of the CGH, the shifting causes discrete mately 24M points with a density of 9 points/mm . The distance from the center of the frequencies to fall outside from the frequency support D . object to the hologram plane is one meter. Employed features allow calculating an average Visualization of the occlusion culling for a segment in the frequency domain is pre- angular spacing of 0.038°. Employed features allow calculating an average angular spac- sented in Figure 4. The employed object is a point cloud of a gargoyle figurine with ing of 0.035°. When employing, Δ = 0.62 µm and λ = 640 nm, which enables maximum dimensions 1055 mm (height)  971 mm (width)  485 mm (depth) composed of approxi- mately 24M points with a density of 9 points/mm . The distance from the center of the angle view of 62.5°, it is found that Ns must be smaller than 1550 pixels. Thus, segment object to the hologram plane is one meter. Employed features allow calculating an average size is selected as Ns = 1024. The size of the calculated CGH is chosen as 4096 × 4096 pixels. angular spacing of 0.038 . Employed features allow calculating an average angular spacing In terms of the PAS technique, the CGH consists of 4 segments of size Ns × Ns, and thus N, of 0.035 . When employing, D = 0.62 m and  = 640 nm, which enables maximum angle view of 62.5 , it is found that N must be smaller than 1550 pixels. Thus, segment size is M = 4. Figure 4 shows the amplitudes of the segment m = 1 and n = 1 in the frequency selected as N = 1024. The size of the calculated CGH is chosen as 4096  4096 pixels. In domain. Figure 4a depicts the cloud of points without occlusion. Image shows that back terms of the PAS technique, the CGH consists of 4 segments of size N  N , and thus N, s s and front points contribute to the holographic information of the segment. Thus, the depth M = 4. Figure 4 shows the amplitudes of the segment m = 1 and n = 1 in the frequency cues are not correctly displayed. Figure 4b shows the cloud of points when applying the domain. Figure 4a depicts the cloud of points without occlusion. Image shows that back and front points contribute to the holographic information of the segment. Thus, the depth proposed occlusion culling based PAS. After the occlusion culling of back points, only the cues are not correctly displayed. Figure 4b shows the cloud of points when applying the front of the gargoyle is seen, which is the correct observable surface. Later, the cloud of proposed occlusion culling based PAS. After the occlusion culling of back points, only the points is rotated 180° and PAS with culling occlusion is applied again. Hence, the back of front of the gargoyle is seen, which is the correct observable surface. Later, the cloud of the gargoyle is obtpoints ainedis . T rotated he res 180 ult is and sh Pown AS with in F culling igure occlusion 4c. These is applied examples show again. Hence, that our the back of the gargoyle is obtained. The result is shown in Figure 4c. These examples show that method can eliminate unwanted back points by using the segmentation properties of the our method can eliminate unwanted back points by using the segmentation properties of PAS. the PAS. Figure 4. Frequency mapping of the point cloud when using PAS for the segment m = 1, n = 1. (a) PAS without occlusion. (b) Front and (c) back of the gargoyle when applying PAS-based occlusion algorithm. In its basic configuration, the PAS algorithm introduces a truncation error when discretizing the instantaneous local frequencies (Equations (6) and (7)) [32], which decreases the reconstruction quality of the CGH. This truncation error depends on the segment size and can be easily reduced by using a large segment. In the general case of CGH calculation, arbitrarily large segment cannot be selected since the wavefront at the hologram plane may not be well represented. For the proposed occlusion culling PAS, the restriction of the segment becomes stricter due to the limitation imposed with Equation (11). Accurate PAS (A-PAS) can be employed for solving the discretization problem while fulfilling the N constraint and sampling theory [31] at the same time. The A-PAS considers that the s Photonics 2021, 8, 298 8 of 16 segment is enlarged in the discrete frequency domain by an integer factor r. In this way, the frequency sampling is increased r-times, which allows a better approximation of the discrete values of the frequencies to their respective continuous values. After applying the inverse FT to the enlarged segment, the original segment is taken. The A-PAS is implemented in the occluded PAS algorithm by considering that the enlarged segment in frequency domain is not larger than N . This means that we must find a hologram segment of size N and extension factor r such that N > N r. When selecting proper N and r, the so s so so accuracy of the PAS algorithm can be reached while preserving the occlusion culling of unwanted back points. 4. Numerical Reconstruction Methods for Wide View-Angle CGHs With the aim to measure accuracy and quality of the holographic image encoded in the CGH, propagation methods [33,41,42] must be employed. In this section, we investigate two propagation techniques. The first, which is based on the direct integration method [33] is used to assess the accuracy of the CGH calculated by the PAS. The second, that is a generalization of the MFFT-AS technique [10], allows for reconstructing the whole object and visualizing quality of the PAS based occlusion culling. 4.1. Off Axis Direct Integration Method Recovering the full complex field from wide-angle CGH with classical propagation techniques requires a huge amount of zero padding in x and y directions. Thus, an enormous amount of computational power is required for storing and operating such information. Moreover, the time that is necessary for obtaining the reconstruction will be on the scale of hours or even days. Therefore, it is proposed to employ the integration method [33] for reconstructing a very small off-axis window of the entire complex field. This is done with the aim to carry out a fast quantification of point reproduction quality of calculated CGH. Off-axis wavefield recovery can be carried out using the Fourier-based solver of the Rayleigh-Sommerfeld integral [43]: u (x , y , 0) = IFFT[FFT[u (x , y , z )]FFT[g(x + x x , y + y y ,z)]], (12) out 2 2 in 1 1 R 2 OA 1 2 OA 1 where x = x x , y = y y , and (x , y ) are the coordinates of transversal shift, u is 2 1 oa 2 1 oa oa oa in the complex field from the hologram, u is the reconstructed field, the subindexes 1 and 2 out point out the input and output planes, respectively, and exp(ikD) 1 g = ik (13) D D 2 2 2 where D = x + y + z . Application of the FFT in Equation (12) generates a circular convolution affecting accuracy of reconstruction. Influence of circular convolution is avoided by increasing twice size of the input matrix by using zero-padding. When solving Equation (12) with these considerations, the desired field is given by the N  N lower x y right submatrix, where N is the size of the input field. This method preserves the pixel size between input at output planes. Thus, the size of the integrated window is x 2 (N D/2, 2 x N D/2), y 2 (N D/2, N D/2). This off-axis direct integration (OADI) method enables fast x x x wavefield reconstruction from any point of view with small amount of memory [33,41,44] but for very small window. 4.2. Multi Fast Fourier Transform Angular Spectrum Method For the reconstruction of full holographic data from the wide-angle CGH, a general- ization of the MFFT-AS method is proposed [10]. The MFFT-AS was initially employed for the reconstruction of Fourier horizontal parallax-DHs (FHPO-DHs) [45], which are small in vertical and large in horizontal directions, respectively. This method applies (q+1)N  N x y FFTs instead of two large FFTs from the classical angular spectrum (AS) method, where N , x Photonics 2021, 8, 298 9 of 16 N are the dimensions of the FHPO-DH and q = bB /B c is the compression factor for y y y2 y1 direction (B is vertical dimension of the hologram). A critical step in the MFFT-AS is the y1 summation in form of “tiles” in y direction. This makes possible to significantly reduce the computational load. Hence, reconstruction of FHPO-DH with AS is possible. In this work, we extend the MFFT AS application to two dimensions, which allows obtaining accurate reconstruction of full parallax CGH. Implementation of the MFFT-AS can be described as follows: let O be the 2D object wave of N  N samples and spatial dimensions B  B . We wish to find its diffracted x y x1 y1 field at plane (x , y ). The MFFT-AS approach accomplishes this task by calculation of 2 2 p  q FFTs as N 1 N 1 x Y 2pi(na/pN +mb/qN ) 2pi(nl/N +mj/N ) x y x x O (lp + a, jq + b) = O(n, m)e e , (14) 1 å å n=0 m=0 where p = bB /B c is the compression factor in horizontal direction, lp+a, jq+b represent x2 x1 the frequency coordinates separated by 1/pB and 1/qB , respectively. Equation (14) is x1 y1 multiplied by a shifted propagation kernel, which yields 2 2 (lp+a) (jq+b) 2piz 2 2 2 l (pB ) (qB ) x1 y1 e e O (lp + a, jq + b) = O (lp + a, jq + b)e . (15) 2 1 It is worth noting that the Equation (15) has size of N p  N q; however, this large x y matrix is never allocated. Sequentially, calculated vectors of size N  N with Equation (15) x y are assembled as matrices of the same length N  N by next part of the MFFT-AS method. x y This element of the MFFT-AS is based on the property of FFT that was proposed in references [46] and allows efficient calculations of high-resolution focused wave of small spatial extend. The property enables reduction of bandwidth of FFT without the need of computing FFT of the full-sized vector. To explain the property, let assume a 1D signal A of q1 length qN . The method starts with summing up “tiles” as A (n) = A(tq + n), where y s t=0 n 2 0, . . . , N 1 and then computes FFT. Notably, the A is a vector, which contains q-th y s frequency of A . In the 2D MFFT-AS method, the tiling is applied in both directions and the introduced property enables reduction of the sampling rate of the calculated diffracted field. Thus, the result is calculated using at first ‘tiles’ summation method, and then FT as N 1 p1 y q1 N 1 2pi(nl/N +mj/N ) x y O(n, m) = O (sp + l, tq + j)e . (16) å å å å 2 l=0 s=0 j=0 t=0 Notably, full field reconstruction with the 2D MFFT-AS AS requires q  p + 1 FFTs of size N  N , and pixel size at the output plane is determined by Dx = p Dx , and x y 2 1 Dy = q Dy . 2 1 4.3. Comparison of Full and Small off Axis Window Reconstruction The strategies based on off axis window and full image reconstruction are here com- pared from the computational point of view. For this, the set of holograms for reconstruction distance z = 1000 mm, pixel count 4096  4096, and several pixel pitches were generated and reconstructed. Table 1 presents the image size and the obtained computation times for both recon- struction methods when using selected pixel pitches. Notably, the MFFT-AS enables to obtain image sizes ranging from one quarter of meter up to one meter. However, the price to pay is a large processing time. For example, reconstruction of the smallest pixel pitch (D = 0.6 m) requires almost 12 h of processing. On the other hand, the OADI method provides results in a few seconds for reconstructing the complex field but of window size of a few millimeters. Photonics 2021, 8, 298 10 of 16 Table 1. Calculated Field of View and computation time for the off-axis direct integration method and the MFFT-AS,  = 640 nm, z = 1000 mm. Image Size Calculation Speed D/FoV OADI MFFT-AS OADI MFFT-AS 0.6 m/62 2.5 mm  2.5 mm 1.2 m  1.2 m 10.5 s 38,300 s 1.2 m/31 5 mm  5 mm 0.54 m  0.54 m 10.5 s 1900 s 1.8 m/20 7.3 mm  7.3 mm 0.36 m  0.36 m 10.5 s 304 s 2.4 m/15 9.9 mm  9.9 mm 0.26 m  0.26 m 10.5 s 104 s 5. Diffraction Efficiency Based on Assessment of CGH Quality In this section, assessment of CGH quality based on diffraction efficiency measure is carried out. This is done with the aim to quantify quality of point reproduction of calculated CGH. Reconstruction of wide-angle CGHs require, see first row of Table 1, a large amount of memory and processing time. Nevertheless, for measuring the diffraction efficiency of CGHs of this work, we retrieve a small region where a point source is reconstructed. Hence, the OADI method is suitable for measuring the diffraction efficiency of CGHs calculated with the PAS algorithm. The first diffraction efficiency-based test is evaluated with two CGHs of ten point sources distributed linearly along the diagonal from the center of the image P (0, 0, z) up to corner point P = (0.475B , 0.475B z), where z = 1000 mm. Pixel pitch of the CGHs 10 x 2 y 2, is D = 0.62 m. Theoretical CGH, which is reference one, is calculated using Equation (2), while the second CGH with the PAS algorithm employing N = 256 and r = 4. Calculated so CGHs are processed with Equation (12) and the corresponding off-axis coordinate. As a result, the single point source with its nearest neighborhood is reconstructed. Figure 5a,b present the reconstruction of point P of theoretical and PAS CGHs, respectively. Visually, reconstructed point sources have identical profiles and almost the same peak value (P-V). Figure 5c presents the accuracy test in relation to point source position along the diagonal of image space. Maximum amplitude values are normalized with respect to the on-axis point P from theoretical CGH. Regardless, the CGH is made with PAS method or with Equation (2), the linear loss of amplitude due to point source position is present. Moreover, it is clearly seen in the plot that accuracy of reconstructed point sources is strongly related to the selection of the parameter r, e.g., for r 3, the maximum amplitude of each point starts to behave stable and is close to the result of theoretical CGH. Accordingly, CGHs of high accuracy made with PAS can be achieved by using a value N and large r. Nevertheless, so the constraint provided by Equation (11) must be fulfilled all the time if occlusion culling is desired. For this reason, we perform a test that is aimed to measure the accuracy of calculated CGHs with the PAS with different parameters N and r. As a measure of so accuracy, the ratio of maximum amplitude of reconstructed point P between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. The obtained ratios demonstrate that when using original segment size (r = 1), the diffraction efficiency is low and unstable, which means that calculated CGH with PAS is not optimal. When using large values of r (r > 2), the maximum amplitude increases its value independently of the N , which is pointed out by large dash box into the plot. so Photonics 2021, 8, x FOR PEER REVIEW 11 of 16 Photonics 2021, 8, x FOR PEER REVIEW 11 of 16 of each point starts to behave stable and is close to the result of theoretical CGH. Accord- ingly, CGHs of high accuracy made with PAS can be achieved by using a value Nso and of each point starts to behave stable and is close to the result of theoretical CGH. Accord- large r. Nevertheless, the constraint provided by Equation (11) must be fulfilled all the ingly, CGHs of high accuracy made with PAS can be achieved by using a value Nso and time if occlusion culling is desired. For this reason, we perform a test that is aimed to large r. Nevertheless, the constraint provided by Equation (11) must be fulfilled all the measure the accuracy of calculated CGHs with the PAS with different parameters Nso and time if occlusion culling is desired. For this reason, we perform a test that is aimed to r. As a measure of accuracy, the ratio of maximum amplitude of reconstructed point P10 measure the accuracy of calculated CGHs with the PAS with different parameters Nso and between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. r. As a measure of accuracy, the ratio of maximum amplitude of reconstructed point P10 The obtained ratios demonstrate that when using original segment size (r = 1), the diffrac- between PAS and theoretical CGH is used. Corresponding results are shown in Figure 6a. tion efficiency is low and unstable, which means that calculated CGH with PAS is not The obtained ratios demonstrate that when using original segment size (r = 1), the diffrac- optimal. When using large values of r (r > 2), the maximum amplitude increases its value Photonics 2021 tion ef , 8, 298ficiency is low and unstable, which means that calculated CGH with PAS is not 11 of 16 independently of the Nso, which is pointed out by large dash box into the plot. optimal. When using large values of r (r > 2), the maximum amplitude increases its value independently of the Nso, which is pointed out by large dash box into the plot. Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Recon- structed amplitude of the point source P10 from theoretical CGH; (b) Reconstructed amplitude of Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Recon- Figure 5. Evaluation of off-axis single point reconstructions with the OADI method; (a) Reconstructed amplitude of the point the point source P10 from PAS CGH; and (c) normalized reconstructed amplitudes for several Nso structed amplitude of the point source P10 from theoretical CGH; (b) Reconstructed amplitude of source P from theoretical CGH; (b) Reconstructed amplitude of the point source P from PAS CGH; and (c) normalized 10 10 and r. reconstr the point ucted amplitudes source P10 for from several PAS N CGH; an and r. d (c) normalized reconstructed amplitudes for several Nso so and r. Figure 6. (a) Normalized reconstructed amplitude of P10 for different Nso and r; (b) Reconstruction Figure 6. (a) Normalized reconstructed amplitude of P for different N and r; (b) Reconstruction time for a CGH of 10 so time for a CGH of 1 million points for different segment Nso and factors r without (PAS) and with 1 million points for different segment N and factors r without (PAS) and with (OC-PAS) occlusion culling. so Figure 6. (a) Normalized reconstructed amplitude of P10 for different Nso and r; (b) Reconstruction (OC-PAS) occlusion culling. time for a CGH of 1 million points for different segment Nso and factors r without (PAS) and with The second test is carried out for measuring the computation time of calculated (OC-PAS) occlusion culling. CGH with a point cloud of one million points. The cloud has a similar point density The second test is carried out for measuring the computation time of calculated CGH to the gargoyle figurine presented in Section 3. Different N and r values without and so with a point cloud of one million points. The cloud has a similar point density to the gar- with occlusion culling routine are employed. The results are presented in Figure 6b. The second test is carried out for measuring the computation time of calculated CGH goyle figurine presented in Section Note that PAS calculation 3. Different of a segment Nso and consists r va of lues twowi parts: thout a allocation nd wi of th occl holographic u- with a point cloud of one million points. The cloud has a similar point density to the gar- information according to the local instantaneous discrete frequencies and performing FFT. sion culling routine are employed. The results are presented in Figure 6b. Note that PAS goyle figurine presented in Section 3. Different Nso and r values without and with occlu- Time consumption for allocation depends on the number of points, and thus the larger the calculation of a segment consists of two parts: allocation of holographic information ac- sion culling routine are employed. The results are presented in Figure 6b. Note that PAS cloud is, the larger the computational effort. Time consumption is particularly demanding cording to the local instantaneous discrete frequencies and performing FFT. Time con- calculation of a segment consists of two parts: allocation of holographic information ac- when the number of points within the cloud is counted in millions. In comparison, the sumption for allocation depends on the number of points, and thus the larger the cloud computational effort of the FFT only depends on the size of the frequency matrix (N  r), so cording to the local instantaneous discrete frequencies and performing FFT. Time con- where the holographic information is stored. Since the computational complexity of is, the larger the computational effort. Time consumption is particularly demanding when sumption for allocation depends on the number of points, and thus the larger the cloud FFT is given by (N  r)  log(N  r), then a small product of N and r will result so so so the number of points within the cloud is counted in millions. In comparison, the compu- is, the larger the computational effort. Time consumption is particularly demanding when in a short computation time, which is almost negligible, while a large product between tational effort of the FFT only depends on the size of the frequency matrix (Nso × r), where the number of points within the cloud is counted in millions. In comparison, the compu- N and r, will increase it. However, this FFT-related time is still shorter than for point so the holographic information is stored. Since the computational complexity of FFT is given processing. A benefit of using large segments N is that less segments are computed for tational effort of the FFT only depends on the size of the frequency matrix (Nso × r), where so hologram generation, which speeds up the CGH computation. When measuring time the holographic information is stored. Since the computational complexity of FFT is given for the occlusion culling, it is seen that the occlusion PAS is faster than normal PAS. For example, generation of the CGH without occlusion for N = 32 and r = 3, and N = 512 and so so Photonics 2021, 8, 298 12 of 16 r = 3 took around 674 and 4.6 s, respectively. On the other hand, calculation of the CGH with occlusion for previous parameters took around 297 s and 3.9 s, respectively. Notably, for segments of size N  r that do not fulfill Equation (11) the growth of computation so time is higher for the occlusion culling than for the computation without occlusion. This is because of the finer frequency sampling of the segment, which means that back points are allowed to store their holographic information. Therefore, the number of processed points grows, and thus the computation time as well. As discussed in former paragraph, selection of N and r for ensuring high accuracy so and fast execution time must consider the constraint imposed by Equation (11). However, Equation (11) relies in the angular spacing D. The parameter D depends on the geometry of the object, the dimension of the hologram, and the separation between object and hologram planes. Due to the mentioned features, the estimation of the angular spacing is a non-trivial issue. Here, it is decided to estimate the local angular spacing with the fitting of a Gaussian distribution to the computed local density of points. Also, considering the standard deviations, we estimate the quasi-minimal angular spacing value for the points clouds that statistically work properly for over 99% of the object surface. Table 2 presents the connection between the average angular spacing of the point cloud and the proper segment size N as it is stated in Equation (11). Four point clouds with the same geometry so from gargoyle figure from Section 3 are employed. These clouds have 200 thousand, 1 million, 5 million, and 24 million of points. Investigation of these exemplary point clouds presents the proper choice of the segment size due to the point cloud angular spacing and their respective CGH time calculation. Table 2. Calculated segment size respect to the average point cloud angular spacing for neighboring points and computation time of CGH-PAS without and with occlusion culling. 200 Thousand 1 Million 5 Million 24 Million Local angular spacing 0.4051 0.1806 0.0821 0.035 N —Equation (11) 146 329 723 1563 N (r = 1) 128 256 512 1024 so PAS without OC 7.7 s 9.4 s 14.0 s 31.0 s PAS with OC 4.5 s 6.1 s 8.9 s 20.5 s 6. Reconstruction of Wide-Angle CGHs Calculated with the Occlusion Culling Based-PAS Algorithm In this section, numerical and optical reconstructions of wide-angle CGHs obtained with the occlusion culling based PAS are carried out. PAS algorithm is implemented in Matlab software, which is installed on a PC equipped with Intel i9-9900K processor, 32 GB of RAM, and run without support of GPU. Selected size of the CGHs is 4160  2160 pixels, which corresponds to the size of the 4K SLM Holoeye GAEA 2,0, pixel pitch D = 3.74 m. SLM CGHs are calculated from the cloud of 24 million points defined in Section 3 and PAS parameters: N = 512, r = 3, and D = 0.62 m, which according to Table 2 fulfills N limit. so s Calculation time for each CGH is around 190 s, which can be reduced up to 30 s when using the Matlab parallel toolbox. When comparing this time with Table 1 from reference [29], it can be seen that our approach improves the calculation time by a factor of 3000. Figure 7a–c show numerical reconstructions for three perspectives angles: 0 , 45 , and 180 of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of the occlusion culling based PAS algorithm allows eliminating unwanted back points, and hence, reconstruction of the object with proper depth cues. It is worth noting that SFI method is not able to provide any of the presented results since it is a method designed under the constraints of the paraxial regime. Video S1 presents the numerical reconstruction of the gargoyle view angles from 0 to 360 . The cloud of points was rotated 8 between each frame. Photonics 2021, 8, x FOR PEER REVIEW 13 of 16 Photonics 2021, 8, x FOR PEER REVIEW 13 of 16 Figure 7a–c show numerical reconstructions for three perspectives angles: 0°, 45°, and 180° of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of the occlusion culling based PAS algorithm allows eliminating unwanted back points, and Figure 7a–c show numerical reconstructions for three perspectives angles: 0°, 45°, and hence, reconstruction of the object with proper depth cues. It is worth noting that SFI 180° of CGH calculated with the 2D MFFT-AS, respectively. Notably, the application of method is not able to provide any of the presented results since it is a method designed the occlusion culling based PAS algorithm allows eliminating unwanted back points, and under the constraints of the paraxial regime. Visualization 1 presents the numerical recon- hence, reconstruction of the object with proper depth cues. It is worth noting that SFI struction of the gargoyle view angles from 0° to 360°. The cloud of points was rotated 8° method is not able to provide any of the presented results since it is a method designed between each frame. under the constraints of the paraxial regime. Visualization 1 presents the numerical recon- Photonics 2021, 8, 298 13 of 16 struction of the gargoyle view angles from 0° to 360°. The cloud of points was rotated 8° between each frame. Figure 7. 2D MFFT AS reconstructions for different perspective angles: (a) 0°, (b) 45°, and (c) 180°. Optical reconstruction of CGH is carried out in the display setup presented in Figure Figure 7. 2D MFFT AS reconstructions for different perspective angles:  (a) 0°,  (b) 45°, and  (c) 180°. Figure 7. 2D MFFT AS reconstructions for different perspective angles: (a) 0 , (b) 45 , and (c) 180 . 8. Illumination laser source (λR = 640 nm) hits half-wave plate (HWP), which controls po- Optical reconstruction of CGH is carried out in the display setup presented in Figure 8. larization state. Next, the beam passes through microscopic objective (MO) and a pinhole Optical reconstruction of CGH is carried out in the display setup presented in Figure Illumination laser source ( = 640 nm) hits half-wave plate (HWP), which controls polar- (PH), generating a spherical wavefront. A collimator lens C (Fc = 300 mm) is placed in front 8. Illumination laser source (λR = 640 nm) hits half-wave plate (HWP), which controls po- ization state. Next, the beam passes through microscopic objective (MO) and a pinhole of the pinhole to obtain an extended plane wave. Plane wave illumination is directed to larization state. Ne (PH), xt, th generating e beam pa asse spherical s through m wavefront. icros Acop collimator ic object lens ive (MO C (F =)300 and mm) a pis inhole placed in the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal front of the pinhole to obtain an extended plane wave. Plane wave illumination is directed (PH), generating a spherical wavefront. A collimator lens C (Fc = 300 mm) is placed in front demagnifying system composed of an objective (F1 = 200 mm, Ø1 = 50 mm) and an eyepiece to the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal of the pinhole to obtain an extended plane wave. Plane wave illumination is directed to demagnifying system composed of an objective (F = 200 mm, Ø = 50 mm) and an eyepiece 1 1 (F2 = 33 mm, Ø2 = 25 mm). At the viewing window (VW) plane pixel pitch of the image of the phase only 4K SLM by a non-polarized beam splitter. SLM is imaged by an afocal (F = 33 mm, Ø = 25 mm). At the viewing window (VW) plane pixel pitch of the image of 2 2 SLM is Δ’SLM = F2/F1 × ΔSLM = 0.62 µm. The small pixel pitch allows for a wide viewing angle demagnifying system composed of an objective (F1 = 200 mm, Ø1 = 50 mm) and an eyepiece SLM is D’ = F /F  D = 0.62 m. The small pixel pitch allows for a wide viewing SLM 2 1 SLM (62.5°). However, the physical aperture of the eyepiece limits it to 36°. angle (62.5 ). However, the physical aperture of the eyepiece limits it to 36 . (F2 = 33 mm, Ø2 = 25 mm). At the viewing window (VW) plane pixel pitch of the image of SLM is Δ’SLM = F2/F1 × ΔSLM = 0.62 µm. The small pixel pitch allows for a wide viewing angle (62.5°). However, the physical aperture of the eyepiece limits it to 36°. Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. Figure 9 shows optical reconstructions of CGH generated and reconstructed for four Figure 9 shows optical reconstructions of CGH generated and reconstructed for four Figure 8. Scheme of wide-angle holographic display used for CGH optical reconstruction. view angles: 0 , 45 , 180 and 275 . To fit the viewing angle of the display system, gargoyle view angles: 0°, 45°, 180° and 275°. To fit the viewing angle of the display system, gargoyle dimensions are resized to 559  527  279 [mm ]. Since the dimension of the cloud is dimensions are resized to 559 × 527 × 279 [mm ]. Since the dimen  sion of the cloud is re- reduced, the average angular spacing of the point cloud decreases to 0.030 . For this value, Figure 9 shows optical reconstructions of CGH generated and reconstructed for four duced, the average angular spacing of the point cloud decreases to 0.030°. For this value, the N must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy view angles: 0°, 45°, 180° and 275°. To fit the viewing angle of the display system, gargoyle and speed for CGH calculation as well. Obtained results prove that the occlusion culling the NS must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy dimensions are resized to 559 × 527 × 279 [mm ]. Since the dimension of the cloud is re- based-PAS can eliminate unwanted back points for each angle. Video S2 presents the and speed for CGH calculation as well. Obtained results prove that the occlusion culling duced, the average angular spacing of the point cloud decreases to 0.030°. For this value, optical reconstruction of the gargoyle from 0 to 360 . Here, the cloud of points was rotated based-PAS can eliminate unwanted back points for each angle. Visualization 2 presents 1 per frame. the NS must be smaller than 1900 pixels. Hence, former PAS parameters fulfill accuracy and speed for CGH calculation as well. Obtained results prove that the occlusion culling based-PAS can eliminate unwanted back points for each angle. Visualization 2 presents Photonics 2021, 8, x FOR PEER REVIEW 14 of 16 Photonics 2021, 8, 298 14 of 16 the optical reconstruction of the gargoyle from 0° to 360°. Here, the cloud of points was rotated 1° per frame. Figure 9. Optical reconstructions of CGHs generated with culling-occlusion-PAS-based algorithm Figure 9. Optical reconstructions of CGHs generated with culling-occlusion-PAS-based algorithm for different view angles: for different view angles: (a) 0°, (b) 45°, (c) 180°, and (d) 275°. (a) 0 , (b) 45 , (c) 180 , and (d) 275 . 7. Conclusions 7. Conclusions In this work, we proposed a simple, effective and fast occlusion culling strategy for In this work, we proposed a simple, effective and fast occlusion culling strategy for wide-angle view holograms. Our approach takes advantage of PAS algorithm, which is based on mapping of 3D object information from spatial to frequency domain. It is shown wide-angle view holograms. Our approach takes advantage of PAS algorithm, which is that optimal segment size that enables occlusion of back points can be calculated according based on mapping of 3D object information from spatial to frequency domain. It is shown to the average angular spacing of the point cloud. Moreover, sorting of the cloud of points that optimal segment size that enables occlusion of back points can be calculated accord- in the z-axis allows ordering the point sources from the nearest to the farthest respect to the ing to the average angular spacing of the point cloud. Moreover, sorting of the cloud of hologram plane. In this way, sources that are closer to the hologram plane are labeled as points in the z-axi fr s ont allones ows and orderi allocate ng the poi their holographic nt sources f information rom the nearest to the f into the frequency matrix arthest re- first. Other point sources of the same frequency but farther from the hologram plane are labeled as spect to the hologram plane. In this way, sources that are closer to the hologram plane are back ones, and thus, their information into the frequency matrix is omitted. Moreover, two labeled as front ones and allocate their holographic information into the frequency matrix methods for numerical reconstruction of wide-angle view CGH were studied. The first first. Other point sources of the same frequency but farther from the hologram plane are was the OADI method, which allows reconstructing a small window from the whole image labeled as back ones, and thus, their information into the frequency matrix is omitted. space. In this way, efficient numerical technique is obtained, which enables measuring the diffraction efficiency of the PAS algorithm. The second was the 2D MFFT-AS, which Moreover, two methods for numerical reconstruction of wide-angle view CGH were stud- allows reconstructing the full image space. It was employed for reconstructing the whole ied. The first was the OADI method, which allows reconstructing a small window from object and visualize the quality of the occlusion culling that was carried out with the the whole image space. In this way, efficient numerical technique is obtained, which ena- PAS. Finally, optical reconstruction of occluded CGHs in wide-angle holographic display bles measuring the diffraction efficiency of the PAS algorithm. The second was the 2D verified the validity of our occlusion culling strategy. Notably, our methodology enables fast computation time for obtaining occluded CGHs from very large point clouds. MFFT-AS, which allows reconstructing the full image space. It was employed for recon- structing the whole object and visualize the quality of the occlusion culling that was car- Supplementary Materials: The following are available online at https://www.mdpi.com/article/ ried out with the PAS. Finally, optical reconstruction of occluded CGHs in wide-angle 10.3390/photonics8080298/s1, Video S1: The numerical reconstruction of the gargoyle view angles holographic display verified the validity of our occlusion culling strategy. Notably, our from 0 to 360 ; Video S2: The optical reconstruction of the gargoyle from 0 to 360 . methodology enables fast computation time for obtaining occluded CGHs from very large Author Contributions: Conceptualization and methodology, J.M.-C.; software, R.K.; experimental point clouds. validation, M.C. and M.S.I.; supervision, T.K. All authors have read and agreed to the published version of the manuscript. Author Contributions: Conceptualization and methodology, J.M.-C.; software, R.K.; experimental Funding: This research was funded by National Science Centre, Poland grant number UMO- 2018/31/B/ST7/02980. validation, M.C. and M.S.I.; supervision, T.K. All authors have read and agreed to the published version of the manuscript. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Funding: This research was funded by National Science Centre, Poland grant number UMO- 2018/31/B/ST7/02980. Data Availability Statement: Data underlying the results presented in this paper are publicly available at this time but may be obtained from the authors upon request. Institutional Review Board Statement: Not applicable Acknowledgments: The authors would like to thank to the Warsaw University of Technology for the Informed Consent Statement: Not applicable. statutory funds provided to this research. Conflicts of Interest: The authors declare no conflict of interest. Data Availability Statement: Data underlying the results presented in this paper are publicly avail- able at this time but may be obtained from the authors upon request. Acknowledgments: The authors would like to thank to the Warsaw University of Technology for the statutory funds provided to this research. Conflicts of Interest: The authors declare no conflict of interest. References 1. Yamaguchi, M. Full-Parallax Holographic Light-Field 3-D Displays and Interactive 3-D Touch. Proc. IEEE 2017, 105, 947–959. 2. Bove, V.M. Display holography’s digital second act. Proc. IEEE 2012, 100, 918–928. Photonics 2021, 8, 298 15 of 16 References 1. Yamaguchi, M. Full-Parallax Holographic Light-Field 3-D Displays and Interactive 3-D Touch. Proc. IEEE 2017, 105, 947–959. [CrossRef] 2. Bove, V.M. Display holography’s digital second act. Proc. IEEE 2012, 100, 918–928. [CrossRef] 3. Kim, M.K. Principles and techniques of digital holographic microscopy. SPIE Rev. 2010, 1, 1–50. [CrossRef] 4. Dong, J.; Jiang, C.; Jia, S. Digital holographic metrology based on multi-angle interferometry. Opt. Lett. 2016, 41, 4301. [CrossRef] 5. Grujic, ´ D.; Vasiljevic, ´ D.; Pantelic, ´ D.; Tomic, ´ L.; Stamenkovic, ´ Z.; Jelenkovic, ´ B.; Zhang, W.; Zhang, D.; Vukmirovic, ´ N.; Stepanenko, D. Naturally safe: Cellular noise for document security. J. Biophotonics 2019, 12, 1–9. 6. Khan, G.S.; Mantel, K.; Harder, I.; Lindlein, N.; Schwider, J. Design considerations for the absolute testing approach of aspherics using combined diffractive optical elements. Appl. Opt. 2007, 46, 7040–7048. [CrossRef] [PubMed] 7. Zhang, H.; Zhao, Y.; Cao, L.; Jin, G. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Opt. Express 2015, 23, 3901–3913. [CrossRef] 8. Lee, S.; Jo, Y.; Yoo, D.; Cho, J.; Lee, D.; Lee, B. Tomographic near-eye displays. Nat. Commun. 2019, 10, 1–10. [CrossRef] 9. Kozacki, T.; Martinez-Carranza, J.; Kukolowicz, R.; Finke, W. Accurate reconstruction of horizontal parallax-only holograms by angular spectrum and efficient zero-padding. Appl. Opt. 2020, 59, 8450–8458. [CrossRef] 10. Isomae, Y.; Shibata, Y.; Ishinabe, T.; Fujikake, H. Design of 1-m-pitch liquid crystal spatial light modulators having dielectric shield wall structure for holographic display with wide field of view. Opt. Rev. 2017, 24, 165–176. [CrossRef] 11. Higashida, R.; Funabashi, N.; Aoshima, K.; Miura, M.; Machida, K. Diffraction of light using high-density magneto-optical light modulator array. Opt. Eng. 2020, 59, 1–13. [CrossRef] 12. Buckley, E. Holographic projector with one lens. Opt. Lett. 2010, 35, 3399–3401. [CrossRef] 13. Blinder, D.; Chlipala, M.; Kozacki, T.; Schelkens, P. Photorealistic computer generated holography with global illumination and path tracing. Opt. Lett. 2021, 46, 2188–2191. [CrossRef] 14. Wakunami, K.; Yamashita, H.; Yamaguchi, M. Occlusion culling for computer generated hologram based on ray-wavefront conversion. Opt. Express 2013, 21, 21811–21822. [CrossRef] 15. Matsushima, K.; Nakahara, S. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method. Appl. Opt. 2009, 48, H54–H63. [CrossRef] [PubMed] 16. Zhang, H.; Cao, L.; Jin, G. Computer-generated hologram with occlusion effect using layer-based processing. Appl. Opt. 2017, 56, F138–F143. [CrossRef] 17. Yeom, H.-J.; Park, J.-H. Calculation of reflectance distribution using angular spectrum convolution in mesh-based computer generated hologram. Opt. Express 2016, 24, 19801–19813. [CrossRef] 18. Chen, R.H.Y.; Wilkinson, T.D. Computer generated hologram from point cloud using graphics processor. Appl. Opt. 2009, 48, 6841–6850. [CrossRef] 19. Chen, R.H.Y.; Wilkinson, T.D. Computer generated hologram with geometric occlusion using GPU-accelerated depth buffer rasterization for three-dimensional display. Appl. Opt. 2009, 48, 4246–4255. [CrossRef] [PubMed] 20. Symeonidou, A.; Blinder, D.; Munteanu, A.; Schelkens, P. Computer-generated holograms by multiple wavefront recording plane method with occlusion culling. Opt. Express 2015, 23, 22149–22161. [CrossRef] 21. Hong, J.; Kim, Y.; Choi, H.J.; Hahn, J.; Park, J.H.; Kim, H.; Min, S.-W.; Chen, M. Three-dimensional display technologies of recent interest: Principles, status, and issues. Appl. Opt. 2011, 50, 87–115. [CrossRef] [PubMed] 22. Gilles, A.; Gioia, P.; Cozot, R.; Morin, L. Complex modulation computer-generated hologram with occlusion effect by a fast hybrid point-source/wave-field approach. In Proceedings of the 2015 Conference: Pacific Conference on Graphics and Applications, Beiging, China, 7–9 October 2015; pp. 1–9. 23. Askari, M.; Kim, S.-B.; Shin, K.-S.; Ko, S.-B.; Kim, S.-H.; Park, D.-Y.; Yu, Y.-G.; Park, J.-H. Occlusion handling using angular spectrum convolution in fully analytical mesh-based computer generated hologram. Opt. Express 2017, 25, 25867–25878. [CrossRef] 24. Chen, J.-S.; Chu, D.P. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. Opt. Express 2015, 23, 18143–18155. [CrossRef] [PubMed] 25. Kang, H.; Stoykova, E.; Yoshikawa, H. Fast phase-added stereogram algorithm for generation of photorealistic 3D content. Appl. Opt. 2016, 55, A135–A143. [CrossRef] 26. Shimobaba, T.; Masuda, N.; Ito, T. Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane. Opt. Lett. 2009, 34, 3133–3136. [CrossRef] 27. Liu, S.; Wei, H.; Li, N.; Liu, Z.; Zhang, J. Occlusion calculation algorithm for computer generated hologram based on ray tracing. Opt. Commun. 2019, 443, 76–85. [CrossRef] 28. Zhao, K.; Huang, Y.; Yan, X.; Jiang, X. Occlusion culling and calculation for a computer generated hologram using spatial frequency index method. J. Opt. 2015, 17, 1–9. [CrossRef] 29. Yamaguchi, M.; Hoshino, H.; Honda, T.; Ohyama, N. Phase-Added Stereogram: Calculation of Hologram Using Computer Graphics Technique. In Proceedings of the 1993 Conference: Practical Holography VII: Imaging and Materials, San Jose, CA, USA, 17 September 1993; pp. 25–31. 30. Kang, H.; Yamaguchi, T.; Yoshikawa, H. Accurate phase-added stereogram to improve the coherent stereogram. Appl. Opt. 2008, 47, D44–D54. [CrossRef] Photonics 2021, 8, 298 16 of 16 31. Kang, H. Compensated phase-added stereogram for real-time holographic display. Opt. Eng. 2007, 46, 1–11. [CrossRef] 32. Shen, F.; Wang, A. Fast-Fourier-transform based numerical integration method for the Rayleigh-Sommerfeld diffraction formula. Appl. Opt. 2006, 45, 1102–1110. [CrossRef] 33. Leseberg, D.; Frère, C. Computer-generated holograms of 3-D objects composed of tilted planar segments. Appl. Opt. 1988, 27, 3020–3024. [CrossRef] [PubMed] 34. Yatagai, T. Stereoscopic approach to 3-D display using computer-generated holograms. Appl. Opt. 1976, 15, 2722–2729. [CrossRef] 35. Tang, C.; Yao, B. Fast computation for generating CGH of a 3D object by employing connections between layers. J. Mod. Opt. 2012, 59, 1406–1409. [CrossRef] 36. Shimobaba, T.; Kakue, T.; Ito, T. Review of Fast Algorithms and Hardware Implementations on Computer Holography. IEEE Trans. Ind. Inform. 2016, 12, 1611–1622. [CrossRef] 37. Zhao, Y.; Cao, L.; Zhang, H.; Kong, D.; Jin, G. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Opt. Express 2015, 23, 25440–25449. [CrossRef] [PubMed] 38. Ju, Y.-G.; Park, J.-H. Foveated computer-generated hologram and its progressive update using triangular mesh scene model for near-eye displays. Opt. Express 2019, 27, 23725–23738. [CrossRef] [PubMed] 39. Katz, S.; Tal, A.; Basri, R. Direct visibility of point sets. ACM Trans. Graph. 2007, 26, 1–11. [CrossRef] 40. Veerman, J.A.C.; Rusch, J.J.; Urbach, H.P. Calculation of the Rayleigh–Sommerfeld diffraction integral by exact integration of the fast oscillating factor. J. Opt. Soc. Am. A 2005, 22, 636–646. [CrossRef] 41. Kozacki, T.; Falaggis, K.; Kujawinska, M. Computation of diffracted fields for the case of high numerical aperture using the angular spectrum method. Appl. Opt. 2012, 51, 7080–7088. [CrossRef] [PubMed] 42. Matsushima, K. Shifted angular spectrum method for off-axis numerical propagation. Opt. Express 2010, 18, 18453–18463. [CrossRef] [PubMed] 43. Zhang, W.; Zhang, H.; Matsushima, K.; Jin, G. Shifted band-extended angular spectrum method for off-axis diffraction calculation. Opt. Express 2021, 29, 10089–10103. [CrossRef] 44. Kozacki, T.; Finke, G.; Garbat, P.; Zaperty, W.; Kujawinska, ´ M. Wide angle holographic display system with spatiotemporal multiplexing. Opt. Express 2012, 20, 27473–27481. [CrossRef] [PubMed] 45. Kanka, M.; Wuttig, A.; Graulig, C.; Riesenberg, R. Fast exact scalar propagation for an in-line holographic microscopy on the diffraction limit. Opt. Lett. 2010, 35, 217–219. [CrossRef] [PubMed] 46. Hillenbrand, M.; Hoffmann, A.; Kelly, D.P.; Sinzinger, S. Fast nonparaxial scalar focal field calculations. J. Opt. Soc. Am. A 2014, 31, 1206–1214. [CrossRef]

Journal

PhotonicsMultidisciplinary Digital Publishing Institute

Published: Jul 27, 2021

Keywords: computer-generated hologram; phase added stereogram algorithm; occlusion; frequency domain; Fourier transform; propagation method; wide-angle view; direct integration method; tiling

There are no references for this article.