Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns

Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge... Article Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns 1 1 1,2, Ernst Polnau , Don L. N. Hettiarachchi and Mikhail A. Vorontsov * Intelligent Optics Laboratory, School of Engineering, University of Dayton, 300 College Park, Dayton, OH 45469, USA Optonica LLC, Spring Valley, OH 45370, USA * Correspondence: mvorontsov1@udayton.edu Abstract: This study introduces electro-optical (EO) sensors (TurbNet sensors) that utilize a remote laser beacon (either coherent or incoherent) and an optical receiver with CCD camera and embed- ded edge AI computer (Jetson Xavier Nx) for in situ evaluation of the path-averaged atmospheric 2 2 turbulence refractive index structure parameter C at a high temporal rate. Evaluation of C n n values was performed using deep neural network (DNN)-based real-time processing of short-expo- sure laser-beacon light intensity scintillation patterns (images) captured by a TurbNet sensor optical receiver. Several pre-trained DNN models were loaded onto the AI computer and used for TurbNet sensor performance evaluation in a set of atmospheric propagation inference trials under diverse turbulence and meteorological conditions. DNN model training, validation, and testing were performed using datasets comprised of a large number of instances of scintillation frames and corresponding reference (“true”) C values that were measured side-by-side with a commercial scintillometer (BLS 2000). Generation of datasets and inference trials was performed at the Citation: Polnau, E.; Hettiarachchi, University of Dayton’s (UD) 7-km atmospheric propagation test range. The results demonstrated a D.L.N.; Vorontsov, M.A. 70–90% correlation between C values obtained with the TurbNet sensors and those measured Electro-Optical Sensors for side-by-side with the scintillometer. Atmospheric Turbulence Strength Characterization with Embedded Keywords: atmospheric turbulence; deep neural network; electro-optics sensor; embedded edge AI Edge AI Processing of Scintillation computing; NVIDIA Jetson Xavier Nx; real-time sensing Patterns. Photonics 2022, 9, 789. https://doi.org/10.3390/ photonics9110789 Received: 31 August 2022 1. Introduction Accepted: 20 October 2022 Published: 24 October 2022 Performance of atmospheric electro-optical (EO) systems, such as free-space laser communication, remote sensing, active imaging, directed energy, and optical surveillance Publisher’s Note: MDPI stays neu- can be significantly degraded by atmospheric effects (e.g., optical turbulence, refractivity tral with regard to jurisdictional and absorption) [1–5]. Atmospheric turbulence causes the most detrimental impact on la- claims in published maps and institu- ser-beam and image characteristics, especially in the deep turbulence conditions typical tional affiliations. for slant and/or extended-range propagation scenarios [6]. In contrast with refractivity and absorption, atmospheric turbulence strength, as characterized by the refractive index structure parameter C , can strongly fluctuate during only a few seconds for a stationary Copyright: © 2022 by the authors. Li- target [7,8] and by an order of magnitude for high-velocity targets when the line-of-site censee MDPI, Basel, Switzerland. rapidly sweeps across a large volume of turbulence. This article is an open access article To evaluate and mitigate the negative impact of atmospheric effects on the perfor- distributed under the terms and con- mance of EO systems, it is necessary for these effects to be accurately characterized and ditions of the Creative Commons At- potentially forecast along the line-of-site to the target (including moving targets) at a tem- tribution (CC BY) license (https://cre- poral resolution that is significantly higher (in situ) than in today’s available atmospheric ativecommons.org/licenses/by/4.0/). turbulence characterization EO sensors. In situ turbulence strength characterization can Photonics 2022, 9, 789. https://doi.org/10.3390/photonics9110789 www.mdpi.com/journal/photonics Photonics 2022, 9, 789 2 of 15 be applied for real-time parameter adjustment in wavefront sensing, beam control and adaptive optics systems [5,9,10], for turbulence effects mitigation in atmospheric imaging [11], and to reduce the bit error rate in laser communication systems [12–14]. Conventional electro-optics (EO) sensors currently used for C measurements are based on estimation of the statistical characteristics of the received optical field, e.g., the laser beam scintillation index, focal spot wander and/or widening, image sharpness, etc. [6,15,16]. Time-averaging is used to calculate these statistical characteristics based on sensing data collected over a relatively long time (typically a few minutes), resulting in low temporal resolution for C evaluation. A novel approach for path-integrated C parameter estimation at high temporal resolution was introduced in [8], where deep neural network (DNN)-based signal pro- cessing models (Cn^2Net DNN models) were developed for C value prediction based on the processing of short-exposure laser beam intensity scintillation patterns. Prelimi- nary results that describe the hardware implementation of this approach using EO sensor configurations (TurbNet sensors) with coherent and incoherent laser beacons and deep- machine-learning-based signal processing have been reported in conference presentations [17,18]. In this study we further extend these early results. The application of deep-machine-learning approaches to turbulence strength charac- terization is a rapidly growing research area. The major emphasis in recent publications in this field has been on the development of DNN architectures for C prediction by uti- lizing pre-recorded or simulated input data obtained using meteorological sensors [19], numerical weather prediction simulations [12], or wave-optics numerical simulations of turbulence-degraded imagery [11,20]. The goal of the research presented here was to ex- perimentally demonstrate and compare the performance of DNN-based TurbNet sensors over a 7 km distance for path-integrated parameter evaluation at a high temporal rate (approximately 1.5 sec. per C measurement) under a wide range of turbulence, envi- ronmental and meteorological conditions. This paper provides an in-depth description of TurbNet sensors, including the EO hardware (Section 2), the pre-processing of scintillation images (Section 3), the generation of datasets during the set of atmospheric measurement trials (Section 4), and DNN model training, validation, and testing (Section 5). The performance evaluation of TurbNet sen- sors in atmospheric inference experiments for C value prediction using pre-trained DNN models under diverse atmospheric turbulence and weather conditions is described in Sec- tion 6. The results of TurbNet sensor development and evaluation are summarized in Sec- tion 7. 2. TurbNet Sensor: Hardware Implementation and Experimental Setting A TurbNet sensor is comprised of optical receiver modules, a laser beacon located at opposite ends of an atmospheric propagation path, and DNN signal-processing hardware (Figure 1). In the experiments described here, we used two laser-beacon types: coherent, based on a single-mode laser source, and incoherent, utilizing a laser-emitting diode (LED). The corresponding TurbNet sensor configurations are referred to here as TurbNet- LB and TurbNet-LED for convenience. The laser-beacon module of the TurbNet-LB sensor in Figure 2 (top right) is based on a single-mode fiber-collimator with an aperture diameter of 50 mm emitting a collimated Gaussian beam of 30 mm width at 1064 nm wavelength (about 5 mW power). The fiber collimator was mounted on a gimbal platform used for angular alignment of the beacon beam, directing it towards the optical receiver module located at the opposite side of the 7 km propagation path at the UD atmospheric test range, as illustrated in Figure 1. The characteristic laser-beam footprint at the receiver plane was of the order of 60 cm. Photonics 2022, 9, 789 3 of 15 Figure 1. Schematic of the experimental setup used for TurbNet sensor development and evaluation. The laser/LED beacon modules and scintillometer transmitter were located inside an instrumental installation on the 40-m-high roof of the VA Medical Center (VAMC) in Dayton, Ohio. Optical re- ceiver modules of both the TurbNet sensor and the scintillometer were installed directly behind the window inside the UD Intelligent Optics Laboratory (UD/IOL). Locations of EO modules are indi- cated by arrows. Insert shows the propagation path altitude profile. This experimental setting is described in more detail in [8]. Both data collection and inference experiments were performed us- ing side-by-side C measurements with a commercial (Scintec BLS 2000 [21]) scintillometer serving as “ground truth” reference. The Jetson Xavier Nx AI computer was used for DNN-based C value prediction. Figure 2. Receiver and laser-beacon modules of the TurbNet-LB (top row) and TurbNet-LED (bot- tom row) sensors. Due to the impact of environmental factors (e.g., weather conditions, position of the sun and clouds) and atmospheric refractivity, the laser-beacon footprint slowly drifted with respect to the optical receiver during the experimental measurement trials. The char- acteristic range of these drifts was of the order of the beam footprint size over a 60 to 90 min timescale. To exclude the influence of laser-beam footprint centroid drift on measure- ments, the laser beacon was frequently (approximately every 10–15 min) re-aligned. This circumstance precluded uninterrupted multi-hour data collection, which is desirable for the generation of large datasets of data instances (scintillation images and C values) Photonics 2022, 9, 789 4 of 15 that are representative of a wide range of turbulence and weather conditions. These da- tasets were used for DNN model training, validation, and testing, as described below (see Sections 5 and 6). Note that laser-beam divergence of the TurbNet-LB beacon can, in principle, be in- tentionally increased to enlarge the laser-beam footprint at the optical receiver plane, thus mitigating the impact of laser-beacon misalignment on measurements. Nevertheless, to maintain high contrast in the scintillation images captured by the CCD camera, increasing the divergence would require a corresponding increase in laser power, which was unde- sirable for eye-safety reasons. The laser-beam footprint drift issue was addressed in the TurbNet-LED sensor by utilizing an LED (940 nm center wavelength, 1.0 W output power) as a light source. To reduce the emitted light divergence, the LED was placed in the focal plane of a collimating Fresnel lens with a 30 cm aperture and 30 cm focal length, as shown in Figure 2 (bottom right). The laser-beam footprint diameter at the optical receiver plane for the LED-based beacon was about 20 m. Significantly enlarging the beam footprint dramatically reduced the impact of environmental factors, thus allowing continuous measurements over 24 h without the need for laser-beacon re-alignment. The optical receivers of both TurbNet sensors were comprised of a telescope lens (L1) and imaging lens (L2) that were utilized for re-imaging the telescope pupil into a CCD camera, as illustrated in Figure 1. A diaphragm (D1) in the focal plane of the telescope was used to reduce the impact of ambient light. The TurbNet-LB receiver module in Figure 2 (top left) was implemented based on a refractive telescope of aperture diameter D = 11 cm and focal distance F = 77 cm. The light- gathering power of the telescope was not sufficient to obtain high-contrast scintillation images with the LED beacon. For this reason, in the TurbNet-LED sensor receiver module in Figure 2 (bottom left), the refractive telescope was replaced by a reflective Schmidt– Cassegrain type telescope having an approximately 2.8-times larger aperture diameter (D = 30.48 cm, F = 3.048 m). Note that the LED-based sensor necessitated a more expensive and bulkier receiver telescope in comparison with the receiver telescope of the TurbNet- LB sensor. Both TurbNet sensors utilized an identical CCD camera (12-bit Allied Vision sensor with 808 × 608 pixel resolution) to capture short-exposure laser-light-intensity distribu- tions (scintillation frames). The camera integration time (exposure time) τCCD was adjusted based on the received laser-light-intensity level: shorter (τCCD = 0.1 ms) for the TurbNet- LB sensor with the coherent laser beacon, and significantly longer (τCCD = 3 ms) for the TurbNet-LED sensor due to the relatively low received laser-light intensity. A characteristic example of scintillation frames captured by the TurbNet sensors un- der different atmospheric turbulence conditions is illustrated in Figure 3. From visual as- sessment, the spatial structures of the scintillation images are noticeably different for var- ious turbulence strengths. Photonics 2022, 9, 789 5 of 15 Figure 3. Example of scintillation patterns acquired from the TurbNet-LB sensor (top row) and TurbNet-LED sensor (bottom row) for weak-to-strong turbulence conditions, as characterized by values measured by the BLS 2000 scintillometer. Scintillation images obtained with the TurbNet-LED sensor have a characteristic black circular area due to laser-light obscuration by the Schmidt–Cassegrain telescope secondary mirror. 3. Pre-Processing of Scintillation Images and Dataset Generation Pre-processing of the input scintillation images for assestment by the TurbNet sensors was performed using an embedded-edge AI computer (Jetson Xavier Nx) that was synchronized with the scintillometer. The C values sequentially measured by the scin- tillometer were considered as “ground truth” for DNN model optimization, training, test- ing, and performance evaluation in the inference experiments. The scintillometer was pre- set to provide a single measurement per minute—the shortest measurement rate available with the BLS 2000 instrument. During each t = 60 s time interval between scin sequential C measurements, the CCD camera captured 300 frames, corresponding to a frame rate of 5 frames/sec. The GStreamer multimedia framework was used to create a pipeline from the CCD camera to the OpenCV application of the AI computer, as illustrated in Figure 4. The pipe- line was used to pre-set the camera frame rate and exposure time, convert the camera raw video stream into AVI format, and downscale from a 10-bit to 8-bit grayscale video stream. Image pre-processing steps were carried out for each frame before saving it to the dataset or inputting to the DNN models and included resizing frames to 128 × 128 pixel resolution, masking the region of interest (ROI), excluding low-contrast frames, and performing in- tensity normalization on the maximum pixel value within each frame. Corresponding to the receiver telescope aperture, the ROI was defined as an inscribed circle for the TurbNet- LB sensor and as two inscribed concentric circles for the TurbNet-LED (see Figure 3). Pre- processed frames were tagged with the corresponding “true” C values to create a da- taset composed of instances (normalized C values and the corresponding 300 scintilla- tion frames) obtained during the data-collection trials. The same media pipeline and pre- processing steps were applied to real-time inference experiments. A description of soft- ware and driver versions used for implementation of the GStreamer framework and AI processor can be found in [18]. Photonics 2022, 9, 789 6 of 15 Figure 4. Flow-diagram visualizing the sequence of steps performed in scintillation frames pre-pro- cessing in the TurbNet sensors. 4. Data Collection Experimental Trials The experimental setup described in Section 2 (see Figure 1) was utilized for synchro- nous recording of short-exposure scintillation images and corresponding “true” val- ues under diverse turbulence and environmental conditions using both the TurbNet-LB and TurbNet-LED sensors. Summaries of the data collection atmospheric trials are pre- sented in Tables 1 and 2, which include information about date and local time (EDT) of the measurement trials, the number of recorded scintillation frames, and weather condi- tions. During these trials a total of approximately 0.2510 (TurbNet-LB dataset) and 1.310 (TurbNet-LED dataset) data instances were acquired with the TurbNet-LB and TurbNet-LED sensors, respectively. Note that the 300 scintillation frames captured be- tween sequential scintillometer measurements were associated with a single C value (see Section 3). Data collection with the TurbNet-LB sensor (Table 1) was performed during 12 rela- tively short (lasting less than six hours) measurement trials conducted between 15 July 2021 and 2 September 2021. During each trial the laser beacon required frequent re-align- ment due to laser-beacon footprint drift (see Section 2), which made it difficult to pursue data collection representing day and night turbulence variability. Enduring data-collec- tion trials were conducted using the TurbNet-LED sensor, which did not require LED bea- con realignment and, thus, allowed continuous measurements over durations exceeding 24 h. Correspondingly, with the TurbNet-LED sensor, a large dataset (5.2-times larger than with the TurbNet-LB sensor) was obtained from only five measurement trials (lasting many hours), as summarized in Table 2. Table 1. Log of experimental trials conducted with the TurbNet-LB sensor. Date Start Time End Time # Frames Weather Condition 2021-07-14 16:03 20:19 37,800 Partly sunny 2021-07-15 12:48 13:35 9900 Sunny and passing clouds 2021-07-19 13:05 13:49 12,300 Sunny 2021-07-21 13:49 17:29 49,800 Sunny and broken clouds 2021-07-22 14:08 14:48 12,000 Sunny 2021-08-10 15:14 17:02 19,800 Partly sunny 2021-08-11 13:52 14:45 15,300 Sunny and broken clouds 2021-08-18 11:56 14:33 16,500 Partly sunny 2021-08-19 16:07 16:33 6900 Partly sunny 2021-08-23 10:27 15:40 48,600 Partly sunny 2021-08-24 13:58 16:18 44,700 Sunny and scattered clouds 2021-09-02 18:01 18:59 15,300 Sunny Photonics 2022, 9, 789 7 of 15 Table 2. Log of experimental trials conducted with the TurbNet-LED sensor. Date Start Time End Time # Frames Weather Condition Partly sunny, cloudy, and 2021-11-09 14:28 23:59 171,600 passing clouds Passing clouds, clear, fog, and 2021-11-10 00:00 23:59 432,000 scattered clouds Passing clouds, clear, and 2021-11-11 00:00 13:59 252,000 partly sunny Partly sunny and passing 2021-11-18 12:56 23:59 199,200 clouds 2021-11-19 00:00 17:33 316,200 Clear, overcast, and sunny The distributions of scintillation frames obtained under different turbulence condi- tions in the TurbNet datasets are illustrated in Figure 5 by histograms showing the num- 2 2 15 2/ 3 C  C  1.010 m ber of scintillation frames within the interval (histogram bin n n,0 width). Note that the first bin acounts for all scintillation frames aquired under weak 2 2 turbulence conditions corresponding to C  C . n n,0 It is apparent that the distribution of scintillation frames in the TurbNet-LB dataset in Figure 5 (left) is more balanced (more uniform across C values) than the correspond- ing TurbNet-LED dataset (Figure 5 (right)), which shows that a predominantly large num- 2 2 2 ber of frames were recorded under medium-strength turbulence (C  C  2C ) . This n,0 n n,0 was not a surprise, as data collection with the TurbNet-LED sensor was performed con- tinuously for multiple days under wide-ranging turbulence conditions. This non-uni- formity in the distribution of instances in the TurbNet-LED dataset is not desirable for DNN model training, as it leads to higher error in prediction for both low- and strong-turbulence conditions due to the obvious bias in training data volume towards me- dium-strength turbulence. In the case of the TurbNet-LED dataset containing a large num- ber of instances, the desired uniformity in data distribution (dataset balancing) was achieved by selective partial removal of data instances belonging to the medium-strength turbulence range. The modified (balanced) TurbNet-LED dataset, comprised of approxi- mately an equal number of instances ( 0.2510 ) as in the TurbNet-LB dataset, was used for DNN model training as described in Section 5. In general, removal of instances from a dataset is not the optimal method to balance it. Nevertheless, from a practical viewpoint, it was convenient to conduct a single “unsupervised” atmospheric data collection trial for 24 h and to further “balance” this dataset by selective and random removal of data in- stances recorded under medium-strength turbulence. An alternative was to conduct many additional “supervised” atmospheric data-collection trials by selecting days and times when was either low or high. Photonics 2022, 9, 789 8 of 15 Figure 5. Distribution of scintillation frames in the TurbNet-LB (left) and TurbNet-LED (right) da- 2 15 2 / 3 tasets obtained under different turbulence conditions ( C  1.010 m ). n , 0 For data collection using the TurbNet-LB sensor, uniformity of data distribution across the C span was achieved without data removal by intentionally increasing the number of “supervised” measurement trials performed under both low- and high- turbulence conditions. 5. DNN Model Training, Validation, and Testing Processing of scintillation images in the TurbNet sensors was performed using the DNN (Cn^2Net) model architecture illustrated in Figure 6 [8]. A set of 16 pre-processed consecutive scintillation frames from the CCD camera were used as the input for MFEB = 16 feature-extraction blocks (FEBs) with identical topology and trainable weights. The in- put frames entered the DNN models with a sliding window shift of mshift = 8 frames. The parameter mshift (1≤ mshift ≤16) controls the frequency of each frame reprocessing within the 16-input frame sequence. This parameter (DNN hyper-parameter) was used for prediction accuracy optimization and stabilization of the DNN training process. Input frames were simultaneously processed by the FEBs using three convolutional and max- pooling layers, followed by perceptron and fully connected layers. Additional details of the Cn^2Net DNN model can be found in [8]. The Cn^2Net software code was used to generate several (about 5–15) pretrained DNN models obtained using different realiza- tions of the initial random weights. Figure 6. Cn^2Net DNN model architecture used for C prediction via processing of scintillation images (see [8] for detailed description). Both the TurbNet-LP and modified (balanced) TurbNet-LED datasets were subdi- vided into three subsets: training (70%), validation (20%), and testing (10%). The DNN models were trained with identical training and validation subsets at a learning rate of Photonics 2022, 9, 789 9 of 15 0.0001 using 100 epochs and applying an early stop to the validation phase using 10% of the number of epochs as the validation patience (number of epochs allowed to improve performance without error decrease). The mean squared error (MSE) in C prediction was selected as the cost function for performance evaluation of the DNN model-training process. The DNN models that predicted C with less than 0.1 MSE in the training process were selected to evaluate the “never seen” data from the corresponding validation subsets. The three DNN models that demonstrated the best performance for each TurbNet-LB and TurbNet-LED data subsets were saved and loaded to the AI processor for real-time C prediction in the inference experiments. The outputs of these models were averaged to increase prediction accuracy. A characteristic example of Cn^2Net model optimization is presented in Figure 7 by scat- ter plots that compare values predicted by a DNN model to the “ground truth” meas- urements obtained for the training, validation, and testing subsets of the TurbNet-LED dataset. As shown in Figure 7, the highest prediction errors were observed for data in- stances corresponding to strong-turbulence conditions. Note that most of the prediction errors were in the 20% range for both TurbNet sensors. Figure 7. Exemplary scatterplots illustrating predicted C values for DNN model training (left), validation (middle), and testing (right) vs. DNN input sequence number MB in the corresponding subset of the TurbNet-LED dataset (MB = 1.0K corresponds to 10 input sequences and 2 15 2/ 3 C  1.010 m ). The corresponding results for DNN training, validation, and testing on the n, 0 TurbNet-LB dataset are similar and, therefore, are not presented. 6. Performance Evaluation of TurbNet Sensors in Real-time Inference Experiments Performance evaluation (inference) trials of the TurbNet sensors were conducted us- ing the experimental setup shown in Figure 1. The sensors’ CCD cameras and AI comput- ers were synchronized with the scintillometer that provided “ground-truth” C values at a t  60 second time rate. The AI computer received scintillation frames at a time scin interval oft  200 ms . Each scintillation frame was pre-processed and added to the frame image buffer to create an input sequence containing 16 sequentially captured and pre- processed scintillation frames. Each of the Nmodel pre-trained DNN models (Nmodel varied from one to five) loaded into the AI computers received identical input sequences of 16 pre-processed scintillation frames in sequential order. For a given scintillation frame input C t sequence, prediction of a value by a single DNN model required about = 100 n model ms. The overall processing time t depended on the number of DNN models M DNN model used—a parameter that is contigent on such factors as the characteristics of the training dataset (e.g., volume, distribution of data instances, etc.), the DNN training approach, the selection of DNN models for inference experiments, and a compromise between the t C processing time and prediction accuracy. In the atmospheric inference trials DNN n conducted, the C prediction rate was varied from approximately t = 1.4 sec (Nmodel n DNN t = 1) to 1.8 sec (Nmodel = 5), which was from 33 to 43 times faster than the DNN Photonics 2022, 9, 789 10 of 15 corresponding C sensing rate of the commercial scintillometer used to obtain “ground- truth” C values. The output data provided by the AI computer consisted of C (t ) (j = 1,…, Nmodel) n, j m predictions independently computed by the DNN models, their average (model-average) value C (t ) , and the moving-average (also referred to as the rolling-average) prediction n m 2 2 C (t ) C (t ) , which was obtained based on ten sequential model average outputs. Here n m n m t is the timestamp with time intervalt . m DNN TurbNet sensor performance was evaluated in a set of atmospheric interference trials repeated several weeks (from one to 24) after the last day of the training dataset collection trials to ensure that performance was not affected by environmental factors, including diurnal and seasonal changes in the weather. Exemplary results of the atmospheric inference trials are presented in Figures 8 and 9, where the “true” values measured by the scintillometer are compared to both model-average C (t ) and moving-average n m TurbNet sensor outputs for the entire range of turbulence conditions observed C (t ) n m during the inference trials, which lasted approximately two hours (Figure 8 top and Figure 9) and 24 hours (Figure 8 bottom). Note that the last inference trial (see Figure 9) was performed in the middle of a sunny, warm day in early May 2022, six months after recording the corresponding DNN training dataset during several days of relatively cold weather in November 2021 (see Table 2). Figure 8. Comparison of values measured using the BLS2000 scintillometer (red curve) with C (t ) n k ˆ 2 the DNN model-average (green curve) and the moving-average (blue curve) ob- C (t ) C (t ) n m n m tained with the TurbNet-LB (top) and TurbNet-LED (bottom) sensors during atmospheric inference 2 15 2/ 3 trials performed on 9 September 2021 and Dec. 9-10 2021 (C  1.010 m ) . The moving average, n,0 computed using ten sequential model-average outputs, corresponding to 18 seconds for the TurbNet-LB and 16 seconds for the TurbNet-LED sensor. Photonics 2022, 9, 789 11 of 15 Figure 9. “Measurement-vs.-prediction” dataplots (see Fig. 8 caption) obtained during the atmospheric inference trial with the TurbNet-LED sensor performed six months after collection of the dataset used for DNN model training. The plots in Figures 8 and 9 show a close match between the sequence of datapoints [ C (t ) and C (t ) ] obtained with the TurbNet sensors and the corresponding n m n m measurements [ C (t ) ] obtained with the scintillometer. The corresponding values of the n k correlation coefficient γ computed for the moving-average DNN output and scintillometer datapoints were γ = 0.7 for the inference trial in Figure 8 (top), γ = 0.77 for the trials in Figure 8 (bottom), and γ = 0.9 for the inference trial in Figure 9. Note that a similar span of the correlation coefficient γ was observed in several more recent field experiments. Because of the high C sensing rate, the TurbNet sensors can be used to monitor rapid changes in atmospheric turbulence dynamics. This unique capability is illustrated in Figure 10. The measured vs. DNN-predicted time evolution plots in this figure represent exemplary segments of the corresponding dependences in Figure 8. The span of model-average C (t ) data provided by the TurbNet-LB and TurbNet-LED sensors n m clearly indicates the significant changes that occurred in turbulence strength during the 60 s time interval between sequential C measurements by the scintillometer. The DNN- predicted C values show about a two-fold change in Figure 10 (left) and nearly a five- fold change in Figure 10 (right). Photonics 2022, 9, 789 12 of 15 Figure 10. Deviation in predicted C values obtained with TurbNet-LB (left) and TurbNet-LED (right) sensors during an exemplary t = 60 second interval corresponding to sequential C scin n measurements by the scintillometer (horizontal red line), where DNN model-average C (t ) and n m moving-average C (t ) predictions are shown by green and blue curves, respectively. The charac- n m teristic grey-scale scintillation images below correspond to the highest (A), (C) and lowest (B), (D) 2 15 2 / 3 points of the model-averaged curves. Here, C  1.010 m and tm and tk are timestamps of data n, 0 received from the scintillometer [ C (t ) ] and TurbNet sensors, respectively. The characteristic Fried n k parameter values 𝑟 = (0.423𝑘 𝐶 𝐿) [22] corresponding to points A-D are: r0 = 5 cm and 8.2 cm for A and B, and r0 = 1.6 cm and 4.2 cm for C and D. Here k = 2π/λ, λ = 940 nm (TurbNet-LED), λ = 1064 nm (TurbNet-LB), L = 7 km. The selected scintillation images shown in Figure 10 (bottom) correspond to the high- est and lowest C values (points A through D in Figure 10 (top)). The scintillation pattern spatial features in these images corroborate the TurbNet sensor measurements in Figure 10 (top). These results show that the TurbNet sensors enabled detection of changes in turbulence strength at high temporal resolution (in real-time), which cannot be achieved with conventional EO instruments based on collection and time-averaging of sensing data. 7. Summary and Concluding Remarks In this paper, we discuss the implementation and field evaluation of novel EO sen- sors (TurbNet sensors) that enable atmospheric turbulence strength characterization at high temporal resolution via DNN-based processing of scintillation patterns originating from a remote laser or LED beacon, using an embedded-edge AI computer (Jetson Xavier Nx). The DNN models were developed and trained with datasets composed of a large number of instances (scintillation images and their corresponding values) collected during several atmospheric measurement trials over a 7 km propagation path. TurbNet sensor performance was validated in a set of atmospheric inference trials performed side- by-side with a conventional scintillometer. The results of the inference trials demonstrated good correspondence (correlation coefficient γ ranging from 0.7 to 0.9) between the scin- tillometer measurements and turbulence strength assessment with the TurbNet sensors. It was shown that, due to the high temporal rate (between 1.4 and 1.8 s per measurement, depending on the number of DNN models used), the TurbNet sensors enabled real-time evaluation of rapid changes in atmospheric turbulence dynamics. Photonics 2022, 9, 789 13 of 15 For practical applications of EO sensors utilizing DNN-based signal processing for C evaluation, one can potentially avoid atmospheric field trials used for the collection of DNN training datasets, which require side-by-side installation of both a DNN-based EO sensor (TurbNet) and a reference scintillometer, as described in Section 3. The training dataset can be generated using wave-optics numerical simulations that provide accurate modeling of laser-beam propagation over any selected turbulence characterization site and, thus, can be utilized for the generation of sufficiently large datasets (SIM datasets) for DNN training. A DNN that is trained with this dataset can then be further used for real-time processing of scintillation images and C prediction at the selected site, as de- scribed in [8]. In the case of DNN training using a simulated dataset, a scintillometer for collection of the training data in the field is not required. Furthermore, the EO sensor with a DNN model trained using a SIM dataset could be utilized at other propagation sites simply by computing a new SIM dataset specific for that site, with corresponding DNN retraining [8]. In conclusion, it should be noted that since DNN models of TurbNet sensors are “trained” using a dataset (training dataset) obtained for specific system parameters and propagation lengths, deviation from these pre-set conditions results in a corresponding decline in C prediction accuracy. As recently shown, this problem can be addressed via 2 2 scaling of the TurbNet sensor output C values [23]. The required C scaling factors n n can be obtained separately for each system parameter or propagation length using either an analytical expression derived from the classical Kolmogorov turbulence theory (the- ory-based scaling), or through wave-optics numerical modeling and simulations (M&S- based scaling) mimicking sensor operation. For the TurbNet sensor described in this pa- per, the theory-based scaling factor α(L) that minimizes C prediction error for the prop- agation distance L is given by the simple expression α(L) = L0 ⁄ L, where L0 is the propaga- tion length (L0 = 7 km) used for generation of the training dataset. A similar scaling (ad- justment) of the EO sensor output signal to address changes in path length or wavelength is used in conventional sensing systems. Typically, these adjustments are also based on either theory (available analytical expressions), simulations, or experimental measure- ments performed during sensor calibration [15,21]. Furthermore, conventional EO atmospheric turbulence sensing systems (e.g., scintil- lometers [15], differential image motion monitor (DIMM) sensors [16], etc.) are based on analytical expressions derived from Kolmogorov turbulence theory, and, for this reason, are restrained by the theoretical assumptions and simplifications made in the derivations of these formulas. This significantly limits the utilization of these sensing systems outside the provisions prescribed by the theory, which are specific for each sensor type, e.g., weak scintillations, quasi-monochromatic, spatially coherent laser-beacon beam, point-like in- coherent light source, etc. Violation of these prescribed requirements can lead to low ac- 2 2 C C curacy or even incorrect sensing data. This limitation of conventional sensors n n can be overcome in the DNN-based sensing approach described here, which can be ex- ploited for various EO sensing modalities (e.g., turbulence strength characterization using a partially coherent optical wave generated by the LED beacon of the TurbNet-LED sen- sor). The only requirement is that instantaneously measured output sensing data in such systems distinctly change in response to variations in atmospheric turbulence conditions. Author Contributions: Problem statement and TurbNet sensing concept: M.A.V.; experimental setup and sensor implementation: E.P. and D.L.N.H.; experimental data collection: E.P. and D.L.N.H.; signal processing and DNN model development: D.L.N.H.; writing paper: M.A.V.; anal- ysis and discussion of obtained results: M.A.V., E.P. and D.L.N.H.; preparation of the manuscript for submission: M.A.V., D.L.N.H. and E.P. All authors have read and agreed to the published ver- sion of the manuscript. Photonics 2022, 9, 789 14 of 15 Funding: This work was supported by the US Office of Naval Research through the contract N00014-17-1-2535 and the US Air Force Phase I STTR through the contract FA864922P0048. Data Availability Statement: The data that support the presented results are available from the corresponding author upon reasonable request. Acknowledgments: The authors would like to thank Artem Vorontsov for providing valuable in- sights into the development of the DNN models utilized in this paper. Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manu- script, or in the decision to publish the results. References 1. Majumdar, A.K.; Ricklin, J.C. Free-Space Laser Communications; Springer: New York, NY, USA, 2008. 2. Miller, N.; Widiker, J.; McManamon, P.; Haus, J. Active Multi-Aperture Imaging through Turbulence. In Proceedings of the SPIE 8395, Acquisition, Tracking, Pointing, and Laser Systems Technologies XXVI, Baltimore, MD, USA, 15 May 2012; p. 839504. https://doi.org/10.1117/12.921160. 3. Perram, G.P.; Cusumano, S.J.; Hengehold, R.L.; Fiorino, S.T. An Introduction to Laser Weapon Systems; Directed Energy Profes- sional Society: Albuquerque, NM, USA, 2010. 4. Weiner, M.M. Atmospheric Turbulence in Optical Surveillance Systems. Appl. Opt. 1967, 6, 1984–1991. https://doi.org/10.1364/AO.6.001984. 5. Hirose, M.; Miyamura, N.; Sato, S. Deviation-based wavefront correction using the SPGD algorithm for high-resolution optical remote sensing. Appl. Opt. 2022, 61, 6722–6728. https://doi.org/10.1364/AO.461222. 6. Andrews, L.C.; Phillips, R.L. Laser Beam Propagation through Random Media; SPIE Publications: Bellingham, WA, USA, 1998. https://doi.org/10.1117/3.626196. 7. Tunick, A. Optical turbulence parameters characterized via optical measurements over a 2.33 km free-space laser path. Opt. Exp. 2008, 16, 14646. https://doi.org/10.1364/oe.16.014645. 8. Vorontsov, A.M.; Vorontsov, M.A.; Filimonov, G.A.; Polnau, E. Atmospheric Turbulence Study with Deep Machine Learning of Intensity Scintillation Patterns. Appl. Sci. 2020, 10, 8136. https://doi.org/10.3390/app10228136. 9. Lionis, A.; Tsigopoulos, A.; Cohn, K. An Application of Artificial Neural Networks to Estimate the Performance of High-Energy Laser Weapons in Maritime Environments. Technologies 2022, 10, 71. https://doi.org/10.3390/technologies10030071. 10. Bolbasova, L.A.; Lukin, V.P. Atmospheric Research for Adaptive Optics. Atmos. Ocean Opt. 2022, 35, 288–302. https://doi.org/10.1134/S1024856022030022. 11. LeMaster, D.A.; Leung, S.; Mendoza-Schrock, O.L. Joint object classification and turbulence strength estimation using convolu- tional neural networks. Appl. Opt. 2021, 60, G40–G48. https://doi.org/10.1364/AO.425119. 12. Ma, S.; Hao, S.; Zhao, Q.; Xu, C.; Xiao, J. Prediction of Atmospheric Turbulence Refractive Index Structure Constant based on Deep Learning. In Proceedings of the SPIE 11717 24th National Laser Conference & Fifteenth National Conference on Laser Technology and Opto-Electronics, Shanghai, China, 2 December 2020; p. 1171709. https://doi.org/10.1117/12.2583944. 13. Pasupathi, T.; Selvi, A.V.J. Real time experimental investigation of adaptive optics compensation technique for free space optical communication. Photon. Netw. Commun. 2022. https://doi.org/10.1007/s11107-022-00973-z. 14. Bart, M.P.; Savino, N.J.; Regmi, P.; Cohen, L.; Safavi, H.; Shaw, H.C.; Lohani, S.; Searles, T.A.; Kirby, B.T.; Lee, H.; et al. Deep learning for enhanced free-space optical communications. arXiv 2022, arXiv:2208.07712. 15. Wang, T.; Ochs, G.R.; Clifford, S.F. A saturation-resistant optical scintillometer to measure Cn2. J. Opt. Soc. Am. 1978, 68, 334– 338. https://doi.org/10.1364/JOSA.68.000334. 16. Belen’kii, M.S.; Roberts, D.W.; Stewart, J.M.; Gimmestad, G.G.; Dagle, W.R. Experimental validation of the differential image motion lidar concept. Opt. Lett. 2000, 25, 518–520. https://doi.org/10.1364/ol.25.000518. 17. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. High Temporal Resolution Sensing of Atmospheric Turbulence Refractive Index Structure Parameter (Cn^2) based on Embedded Edge AI-Processing of Scintillation Images. In Proceedings of the SPIE 12102, Real-Time Image Processing and Deep Learning, Orlando, FL, USA, 27 May 2022; p. 121020B. https://doi.org/10.1117/12.2618431. 18. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. Electro-Optics Sensor with Embedded Edge AI-Computing for Atmospheric Turbulence Refractive Index Structure Parameter (Cn2) Sensing at High Temporal Resolution. In Proceedings of the SPIE 11993, Free-Space Laser Communications XXXIV, San Francisco, CA, USA, 4 March 2022; p. 119930X. https://doi.org/10.1117/12.2618110. 19. Wang, Y.; Basu, S. Using an artificial neural network approach to estimate surface-layer optical turbulence at Mauna Loa, Ha- waii. Opt. Lett. 2016, 41, 2334–2337. https://doi.org/10.1364/OL.41.002334. 20. Nair, G.N.; Mei, K.; Patel, V.M. AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models. arXiv 2022, arXiv:2208.11284. 21. Large Aperture Scintillometer BLS2000 User’s Manual. Available online: https://www.scintec.com (accessed on 27 August 2020). Photonics 2022, 9, 789 15 of 15 22. Tyson, R.K.; Frazier, B.W. Field Guide to Adaptive Optics; SPIE Press: Bellingham, WA, USA, 2012. https://doi.org/10.1117/3.923078. 23. Filimonov, G.; Vorontsov, M. Towards Artificial-Intelligence-Based Optical Scintillometer: Scaling Issue. In Proceedings of the IEEE International Symposium on Antennas and Propagation and USNC-URSI Radio Science, Denver, CO, USA, 10–15 July http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Photonics Multidisciplinary Digital Publishing Institute

Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/electro-optical-sensors-for-atmospheric-turbulence-strength-6hlTb7tNvB

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2022 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. Terms and Conditions Privacy Policy
ISSN
2304-6732
DOI
10.3390/photonics9110789
Publisher site
See Article on Publisher Site

Abstract

Article Electro-Optical Sensors for Atmospheric Turbulence Strength Characterization with Embedded Edge AI Processing of Scintillation Patterns 1 1 1,2, Ernst Polnau , Don L. N. Hettiarachchi and Mikhail A. Vorontsov * Intelligent Optics Laboratory, School of Engineering, University of Dayton, 300 College Park, Dayton, OH 45469, USA Optonica LLC, Spring Valley, OH 45370, USA * Correspondence: mvorontsov1@udayton.edu Abstract: This study introduces electro-optical (EO) sensors (TurbNet sensors) that utilize a remote laser beacon (either coherent or incoherent) and an optical receiver with CCD camera and embed- ded edge AI computer (Jetson Xavier Nx) for in situ evaluation of the path-averaged atmospheric 2 2 turbulence refractive index structure parameter C at a high temporal rate. Evaluation of C n n values was performed using deep neural network (DNN)-based real-time processing of short-expo- sure laser-beacon light intensity scintillation patterns (images) captured by a TurbNet sensor optical receiver. Several pre-trained DNN models were loaded onto the AI computer and used for TurbNet sensor performance evaluation in a set of atmospheric propagation inference trials under diverse turbulence and meteorological conditions. DNN model training, validation, and testing were performed using datasets comprised of a large number of instances of scintillation frames and corresponding reference (“true”) C values that were measured side-by-side with a commercial scintillometer (BLS 2000). Generation of datasets and inference trials was performed at the Citation: Polnau, E.; Hettiarachchi, University of Dayton’s (UD) 7-km atmospheric propagation test range. The results demonstrated a D.L.N.; Vorontsov, M.A. 70–90% correlation between C values obtained with the TurbNet sensors and those measured Electro-Optical Sensors for side-by-side with the scintillometer. Atmospheric Turbulence Strength Characterization with Embedded Keywords: atmospheric turbulence; deep neural network; electro-optics sensor; embedded edge AI Edge AI Processing of Scintillation computing; NVIDIA Jetson Xavier Nx; real-time sensing Patterns. Photonics 2022, 9, 789. https://doi.org/10.3390/ photonics9110789 Received: 31 August 2022 1. Introduction Accepted: 20 October 2022 Published: 24 October 2022 Performance of atmospheric electro-optical (EO) systems, such as free-space laser communication, remote sensing, active imaging, directed energy, and optical surveillance Publisher’s Note: MDPI stays neu- can be significantly degraded by atmospheric effects (e.g., optical turbulence, refractivity tral with regard to jurisdictional and absorption) [1–5]. Atmospheric turbulence causes the most detrimental impact on la- claims in published maps and institu- ser-beam and image characteristics, especially in the deep turbulence conditions typical tional affiliations. for slant and/or extended-range propagation scenarios [6]. In contrast with refractivity and absorption, atmospheric turbulence strength, as characterized by the refractive index structure parameter C , can strongly fluctuate during only a few seconds for a stationary Copyright: © 2022 by the authors. Li- target [7,8] and by an order of magnitude for high-velocity targets when the line-of-site censee MDPI, Basel, Switzerland. rapidly sweeps across a large volume of turbulence. This article is an open access article To evaluate and mitigate the negative impact of atmospheric effects on the perfor- distributed under the terms and con- mance of EO systems, it is necessary for these effects to be accurately characterized and ditions of the Creative Commons At- potentially forecast along the line-of-site to the target (including moving targets) at a tem- tribution (CC BY) license (https://cre- poral resolution that is significantly higher (in situ) than in today’s available atmospheric ativecommons.org/licenses/by/4.0/). turbulence characterization EO sensors. In situ turbulence strength characterization can Photonics 2022, 9, 789. https://doi.org/10.3390/photonics9110789 www.mdpi.com/journal/photonics Photonics 2022, 9, 789 2 of 15 be applied for real-time parameter adjustment in wavefront sensing, beam control and adaptive optics systems [5,9,10], for turbulence effects mitigation in atmospheric imaging [11], and to reduce the bit error rate in laser communication systems [12–14]. Conventional electro-optics (EO) sensors currently used for C measurements are based on estimation of the statistical characteristics of the received optical field, e.g., the laser beam scintillation index, focal spot wander and/or widening, image sharpness, etc. [6,15,16]. Time-averaging is used to calculate these statistical characteristics based on sensing data collected over a relatively long time (typically a few minutes), resulting in low temporal resolution for C evaluation. A novel approach for path-integrated C parameter estimation at high temporal resolution was introduced in [8], where deep neural network (DNN)-based signal pro- cessing models (Cn^2Net DNN models) were developed for C value prediction based on the processing of short-exposure laser beam intensity scintillation patterns. Prelimi- nary results that describe the hardware implementation of this approach using EO sensor configurations (TurbNet sensors) with coherent and incoherent laser beacons and deep- machine-learning-based signal processing have been reported in conference presentations [17,18]. In this study we further extend these early results. The application of deep-machine-learning approaches to turbulence strength charac- terization is a rapidly growing research area. The major emphasis in recent publications in this field has been on the development of DNN architectures for C prediction by uti- lizing pre-recorded or simulated input data obtained using meteorological sensors [19], numerical weather prediction simulations [12], or wave-optics numerical simulations of turbulence-degraded imagery [11,20]. The goal of the research presented here was to ex- perimentally demonstrate and compare the performance of DNN-based TurbNet sensors over a 7 km distance for path-integrated parameter evaluation at a high temporal rate (approximately 1.5 sec. per C measurement) under a wide range of turbulence, envi- ronmental and meteorological conditions. This paper provides an in-depth description of TurbNet sensors, including the EO hardware (Section 2), the pre-processing of scintillation images (Section 3), the generation of datasets during the set of atmospheric measurement trials (Section 4), and DNN model training, validation, and testing (Section 5). The performance evaluation of TurbNet sen- sors in atmospheric inference experiments for C value prediction using pre-trained DNN models under diverse atmospheric turbulence and weather conditions is described in Sec- tion 6. The results of TurbNet sensor development and evaluation are summarized in Sec- tion 7. 2. TurbNet Sensor: Hardware Implementation and Experimental Setting A TurbNet sensor is comprised of optical receiver modules, a laser beacon located at opposite ends of an atmospheric propagation path, and DNN signal-processing hardware (Figure 1). In the experiments described here, we used two laser-beacon types: coherent, based on a single-mode laser source, and incoherent, utilizing a laser-emitting diode (LED). The corresponding TurbNet sensor configurations are referred to here as TurbNet- LB and TurbNet-LED for convenience. The laser-beacon module of the TurbNet-LB sensor in Figure 2 (top right) is based on a single-mode fiber-collimator with an aperture diameter of 50 mm emitting a collimated Gaussian beam of 30 mm width at 1064 nm wavelength (about 5 mW power). The fiber collimator was mounted on a gimbal platform used for angular alignment of the beacon beam, directing it towards the optical receiver module located at the opposite side of the 7 km propagation path at the UD atmospheric test range, as illustrated in Figure 1. The characteristic laser-beam footprint at the receiver plane was of the order of 60 cm. Photonics 2022, 9, 789 3 of 15 Figure 1. Schematic of the experimental setup used for TurbNet sensor development and evaluation. The laser/LED beacon modules and scintillometer transmitter were located inside an instrumental installation on the 40-m-high roof of the VA Medical Center (VAMC) in Dayton, Ohio. Optical re- ceiver modules of both the TurbNet sensor and the scintillometer were installed directly behind the window inside the UD Intelligent Optics Laboratory (UD/IOL). Locations of EO modules are indi- cated by arrows. Insert shows the propagation path altitude profile. This experimental setting is described in more detail in [8]. Both data collection and inference experiments were performed us- ing side-by-side C measurements with a commercial (Scintec BLS 2000 [21]) scintillometer serving as “ground truth” reference. The Jetson Xavier Nx AI computer was used for DNN-based C value prediction. Figure 2. Receiver and laser-beacon modules of the TurbNet-LB (top row) and TurbNet-LED (bot- tom row) sensors. Due to the impact of environmental factors (e.g., weather conditions, position of the sun and clouds) and atmospheric refractivity, the laser-beacon footprint slowly drifted with respect to the optical receiver during the experimental measurement trials. The char- acteristic range of these drifts was of the order of the beam footprint size over a 60 to 90 min timescale. To exclude the influence of laser-beam footprint centroid drift on measure- ments, the laser beacon was frequently (approximately every 10–15 min) re-aligned. This circumstance precluded uninterrupted multi-hour data collection, which is desirable for the generation of large datasets of data instances (scintillation images and C values) Photonics 2022, 9, 789 4 of 15 that are representative of a wide range of turbulence and weather conditions. These da- tasets were used for DNN model training, validation, and testing, as described below (see Sections 5 and 6). Note that laser-beam divergence of the TurbNet-LB beacon can, in principle, be in- tentionally increased to enlarge the laser-beam footprint at the optical receiver plane, thus mitigating the impact of laser-beacon misalignment on measurements. Nevertheless, to maintain high contrast in the scintillation images captured by the CCD camera, increasing the divergence would require a corresponding increase in laser power, which was unde- sirable for eye-safety reasons. The laser-beam footprint drift issue was addressed in the TurbNet-LED sensor by utilizing an LED (940 nm center wavelength, 1.0 W output power) as a light source. To reduce the emitted light divergence, the LED was placed in the focal plane of a collimating Fresnel lens with a 30 cm aperture and 30 cm focal length, as shown in Figure 2 (bottom right). The laser-beam footprint diameter at the optical receiver plane for the LED-based beacon was about 20 m. Significantly enlarging the beam footprint dramatically reduced the impact of environmental factors, thus allowing continuous measurements over 24 h without the need for laser-beacon re-alignment. The optical receivers of both TurbNet sensors were comprised of a telescope lens (L1) and imaging lens (L2) that were utilized for re-imaging the telescope pupil into a CCD camera, as illustrated in Figure 1. A diaphragm (D1) in the focal plane of the telescope was used to reduce the impact of ambient light. The TurbNet-LB receiver module in Figure 2 (top left) was implemented based on a refractive telescope of aperture diameter D = 11 cm and focal distance F = 77 cm. The light- gathering power of the telescope was not sufficient to obtain high-contrast scintillation images with the LED beacon. For this reason, in the TurbNet-LED sensor receiver module in Figure 2 (bottom left), the refractive telescope was replaced by a reflective Schmidt– Cassegrain type telescope having an approximately 2.8-times larger aperture diameter (D = 30.48 cm, F = 3.048 m). Note that the LED-based sensor necessitated a more expensive and bulkier receiver telescope in comparison with the receiver telescope of the TurbNet- LB sensor. Both TurbNet sensors utilized an identical CCD camera (12-bit Allied Vision sensor with 808 × 608 pixel resolution) to capture short-exposure laser-light-intensity distribu- tions (scintillation frames). The camera integration time (exposure time) τCCD was adjusted based on the received laser-light-intensity level: shorter (τCCD = 0.1 ms) for the TurbNet- LB sensor with the coherent laser beacon, and significantly longer (τCCD = 3 ms) for the TurbNet-LED sensor due to the relatively low received laser-light intensity. A characteristic example of scintillation frames captured by the TurbNet sensors un- der different atmospheric turbulence conditions is illustrated in Figure 3. From visual as- sessment, the spatial structures of the scintillation images are noticeably different for var- ious turbulence strengths. Photonics 2022, 9, 789 5 of 15 Figure 3. Example of scintillation patterns acquired from the TurbNet-LB sensor (top row) and TurbNet-LED sensor (bottom row) for weak-to-strong turbulence conditions, as characterized by values measured by the BLS 2000 scintillometer. Scintillation images obtained with the TurbNet-LED sensor have a characteristic black circular area due to laser-light obscuration by the Schmidt–Cassegrain telescope secondary mirror. 3. Pre-Processing of Scintillation Images and Dataset Generation Pre-processing of the input scintillation images for assestment by the TurbNet sensors was performed using an embedded-edge AI computer (Jetson Xavier Nx) that was synchronized with the scintillometer. The C values sequentially measured by the scin- tillometer were considered as “ground truth” for DNN model optimization, training, test- ing, and performance evaluation in the inference experiments. The scintillometer was pre- set to provide a single measurement per minute—the shortest measurement rate available with the BLS 2000 instrument. During each t = 60 s time interval between scin sequential C measurements, the CCD camera captured 300 frames, corresponding to a frame rate of 5 frames/sec. The GStreamer multimedia framework was used to create a pipeline from the CCD camera to the OpenCV application of the AI computer, as illustrated in Figure 4. The pipe- line was used to pre-set the camera frame rate and exposure time, convert the camera raw video stream into AVI format, and downscale from a 10-bit to 8-bit grayscale video stream. Image pre-processing steps were carried out for each frame before saving it to the dataset or inputting to the DNN models and included resizing frames to 128 × 128 pixel resolution, masking the region of interest (ROI), excluding low-contrast frames, and performing in- tensity normalization on the maximum pixel value within each frame. Corresponding to the receiver telescope aperture, the ROI was defined as an inscribed circle for the TurbNet- LB sensor and as two inscribed concentric circles for the TurbNet-LED (see Figure 3). Pre- processed frames were tagged with the corresponding “true” C values to create a da- taset composed of instances (normalized C values and the corresponding 300 scintilla- tion frames) obtained during the data-collection trials. The same media pipeline and pre- processing steps were applied to real-time inference experiments. A description of soft- ware and driver versions used for implementation of the GStreamer framework and AI processor can be found in [18]. Photonics 2022, 9, 789 6 of 15 Figure 4. Flow-diagram visualizing the sequence of steps performed in scintillation frames pre-pro- cessing in the TurbNet sensors. 4. Data Collection Experimental Trials The experimental setup described in Section 2 (see Figure 1) was utilized for synchro- nous recording of short-exposure scintillation images and corresponding “true” val- ues under diverse turbulence and environmental conditions using both the TurbNet-LB and TurbNet-LED sensors. Summaries of the data collection atmospheric trials are pre- sented in Tables 1 and 2, which include information about date and local time (EDT) of the measurement trials, the number of recorded scintillation frames, and weather condi- tions. During these trials a total of approximately 0.2510 (TurbNet-LB dataset) and 1.310 (TurbNet-LED dataset) data instances were acquired with the TurbNet-LB and TurbNet-LED sensors, respectively. Note that the 300 scintillation frames captured be- tween sequential scintillometer measurements were associated with a single C value (see Section 3). Data collection with the TurbNet-LB sensor (Table 1) was performed during 12 rela- tively short (lasting less than six hours) measurement trials conducted between 15 July 2021 and 2 September 2021. During each trial the laser beacon required frequent re-align- ment due to laser-beacon footprint drift (see Section 2), which made it difficult to pursue data collection representing day and night turbulence variability. Enduring data-collec- tion trials were conducted using the TurbNet-LED sensor, which did not require LED bea- con realignment and, thus, allowed continuous measurements over durations exceeding 24 h. Correspondingly, with the TurbNet-LED sensor, a large dataset (5.2-times larger than with the TurbNet-LB sensor) was obtained from only five measurement trials (lasting many hours), as summarized in Table 2. Table 1. Log of experimental trials conducted with the TurbNet-LB sensor. Date Start Time End Time # Frames Weather Condition 2021-07-14 16:03 20:19 37,800 Partly sunny 2021-07-15 12:48 13:35 9900 Sunny and passing clouds 2021-07-19 13:05 13:49 12,300 Sunny 2021-07-21 13:49 17:29 49,800 Sunny and broken clouds 2021-07-22 14:08 14:48 12,000 Sunny 2021-08-10 15:14 17:02 19,800 Partly sunny 2021-08-11 13:52 14:45 15,300 Sunny and broken clouds 2021-08-18 11:56 14:33 16,500 Partly sunny 2021-08-19 16:07 16:33 6900 Partly sunny 2021-08-23 10:27 15:40 48,600 Partly sunny 2021-08-24 13:58 16:18 44,700 Sunny and scattered clouds 2021-09-02 18:01 18:59 15,300 Sunny Photonics 2022, 9, 789 7 of 15 Table 2. Log of experimental trials conducted with the TurbNet-LED sensor. Date Start Time End Time # Frames Weather Condition Partly sunny, cloudy, and 2021-11-09 14:28 23:59 171,600 passing clouds Passing clouds, clear, fog, and 2021-11-10 00:00 23:59 432,000 scattered clouds Passing clouds, clear, and 2021-11-11 00:00 13:59 252,000 partly sunny Partly sunny and passing 2021-11-18 12:56 23:59 199,200 clouds 2021-11-19 00:00 17:33 316,200 Clear, overcast, and sunny The distributions of scintillation frames obtained under different turbulence condi- tions in the TurbNet datasets are illustrated in Figure 5 by histograms showing the num- 2 2 15 2/ 3 C  C  1.010 m ber of scintillation frames within the interval (histogram bin n n,0 width). Note that the first bin acounts for all scintillation frames aquired under weak 2 2 turbulence conditions corresponding to C  C . n n,0 It is apparent that the distribution of scintillation frames in the TurbNet-LB dataset in Figure 5 (left) is more balanced (more uniform across C values) than the correspond- ing TurbNet-LED dataset (Figure 5 (right)), which shows that a predominantly large num- 2 2 2 ber of frames were recorded under medium-strength turbulence (C  C  2C ) . This n,0 n n,0 was not a surprise, as data collection with the TurbNet-LED sensor was performed con- tinuously for multiple days under wide-ranging turbulence conditions. This non-uni- formity in the distribution of instances in the TurbNet-LED dataset is not desirable for DNN model training, as it leads to higher error in prediction for both low- and strong-turbulence conditions due to the obvious bias in training data volume towards me- dium-strength turbulence. In the case of the TurbNet-LED dataset containing a large num- ber of instances, the desired uniformity in data distribution (dataset balancing) was achieved by selective partial removal of data instances belonging to the medium-strength turbulence range. The modified (balanced) TurbNet-LED dataset, comprised of approxi- mately an equal number of instances ( 0.2510 ) as in the TurbNet-LB dataset, was used for DNN model training as described in Section 5. In general, removal of instances from a dataset is not the optimal method to balance it. Nevertheless, from a practical viewpoint, it was convenient to conduct a single “unsupervised” atmospheric data collection trial for 24 h and to further “balance” this dataset by selective and random removal of data in- stances recorded under medium-strength turbulence. An alternative was to conduct many additional “supervised” atmospheric data-collection trials by selecting days and times when was either low or high. Photonics 2022, 9, 789 8 of 15 Figure 5. Distribution of scintillation frames in the TurbNet-LB (left) and TurbNet-LED (right) da- 2 15 2 / 3 tasets obtained under different turbulence conditions ( C  1.010 m ). n , 0 For data collection using the TurbNet-LB sensor, uniformity of data distribution across the C span was achieved without data removal by intentionally increasing the number of “supervised” measurement trials performed under both low- and high- turbulence conditions. 5. DNN Model Training, Validation, and Testing Processing of scintillation images in the TurbNet sensors was performed using the DNN (Cn^2Net) model architecture illustrated in Figure 6 [8]. A set of 16 pre-processed consecutive scintillation frames from the CCD camera were used as the input for MFEB = 16 feature-extraction blocks (FEBs) with identical topology and trainable weights. The in- put frames entered the DNN models with a sliding window shift of mshift = 8 frames. The parameter mshift (1≤ mshift ≤16) controls the frequency of each frame reprocessing within the 16-input frame sequence. This parameter (DNN hyper-parameter) was used for prediction accuracy optimization and stabilization of the DNN training process. Input frames were simultaneously processed by the FEBs using three convolutional and max- pooling layers, followed by perceptron and fully connected layers. Additional details of the Cn^2Net DNN model can be found in [8]. The Cn^2Net software code was used to generate several (about 5–15) pretrained DNN models obtained using different realiza- tions of the initial random weights. Figure 6. Cn^2Net DNN model architecture used for C prediction via processing of scintillation images (see [8] for detailed description). Both the TurbNet-LP and modified (balanced) TurbNet-LED datasets were subdi- vided into three subsets: training (70%), validation (20%), and testing (10%). The DNN models were trained with identical training and validation subsets at a learning rate of Photonics 2022, 9, 789 9 of 15 0.0001 using 100 epochs and applying an early stop to the validation phase using 10% of the number of epochs as the validation patience (number of epochs allowed to improve performance without error decrease). The mean squared error (MSE) in C prediction was selected as the cost function for performance evaluation of the DNN model-training process. The DNN models that predicted C with less than 0.1 MSE in the training process were selected to evaluate the “never seen” data from the corresponding validation subsets. The three DNN models that demonstrated the best performance for each TurbNet-LB and TurbNet-LED data subsets were saved and loaded to the AI processor for real-time C prediction in the inference experiments. The outputs of these models were averaged to increase prediction accuracy. A characteristic example of Cn^2Net model optimization is presented in Figure 7 by scat- ter plots that compare values predicted by a DNN model to the “ground truth” meas- urements obtained for the training, validation, and testing subsets of the TurbNet-LED dataset. As shown in Figure 7, the highest prediction errors were observed for data in- stances corresponding to strong-turbulence conditions. Note that most of the prediction errors were in the 20% range for both TurbNet sensors. Figure 7. Exemplary scatterplots illustrating predicted C values for DNN model training (left), validation (middle), and testing (right) vs. DNN input sequence number MB in the corresponding subset of the TurbNet-LED dataset (MB = 1.0K corresponds to 10 input sequences and 2 15 2/ 3 C  1.010 m ). The corresponding results for DNN training, validation, and testing on the n, 0 TurbNet-LB dataset are similar and, therefore, are not presented. 6. Performance Evaluation of TurbNet Sensors in Real-time Inference Experiments Performance evaluation (inference) trials of the TurbNet sensors were conducted us- ing the experimental setup shown in Figure 1. The sensors’ CCD cameras and AI comput- ers were synchronized with the scintillometer that provided “ground-truth” C values at a t  60 second time rate. The AI computer received scintillation frames at a time scin interval oft  200 ms . Each scintillation frame was pre-processed and added to the frame image buffer to create an input sequence containing 16 sequentially captured and pre- processed scintillation frames. Each of the Nmodel pre-trained DNN models (Nmodel varied from one to five) loaded into the AI computers received identical input sequences of 16 pre-processed scintillation frames in sequential order. For a given scintillation frame input C t sequence, prediction of a value by a single DNN model required about = 100 n model ms. The overall processing time t depended on the number of DNN models M DNN model used—a parameter that is contigent on such factors as the characteristics of the training dataset (e.g., volume, distribution of data instances, etc.), the DNN training approach, the selection of DNN models for inference experiments, and a compromise between the t C processing time and prediction accuracy. In the atmospheric inference trials DNN n conducted, the C prediction rate was varied from approximately t = 1.4 sec (Nmodel n DNN t = 1) to 1.8 sec (Nmodel = 5), which was from 33 to 43 times faster than the DNN Photonics 2022, 9, 789 10 of 15 corresponding C sensing rate of the commercial scintillometer used to obtain “ground- truth” C values. The output data provided by the AI computer consisted of C (t ) (j = 1,…, Nmodel) n, j m predictions independently computed by the DNN models, their average (model-average) value C (t ) , and the moving-average (also referred to as the rolling-average) prediction n m 2 2 C (t ) C (t ) , which was obtained based on ten sequential model average outputs. Here n m n m t is the timestamp with time intervalt . m DNN TurbNet sensor performance was evaluated in a set of atmospheric interference trials repeated several weeks (from one to 24) after the last day of the training dataset collection trials to ensure that performance was not affected by environmental factors, including diurnal and seasonal changes in the weather. Exemplary results of the atmospheric inference trials are presented in Figures 8 and 9, where the “true” values measured by the scintillometer are compared to both model-average C (t ) and moving-average n m TurbNet sensor outputs for the entire range of turbulence conditions observed C (t ) n m during the inference trials, which lasted approximately two hours (Figure 8 top and Figure 9) and 24 hours (Figure 8 bottom). Note that the last inference trial (see Figure 9) was performed in the middle of a sunny, warm day in early May 2022, six months after recording the corresponding DNN training dataset during several days of relatively cold weather in November 2021 (see Table 2). Figure 8. Comparison of values measured using the BLS2000 scintillometer (red curve) with C (t ) n k ˆ 2 the DNN model-average (green curve) and the moving-average (blue curve) ob- C (t ) C (t ) n m n m tained with the TurbNet-LB (top) and TurbNet-LED (bottom) sensors during atmospheric inference 2 15 2/ 3 trials performed on 9 September 2021 and Dec. 9-10 2021 (C  1.010 m ) . The moving average, n,0 computed using ten sequential model-average outputs, corresponding to 18 seconds for the TurbNet-LB and 16 seconds for the TurbNet-LED sensor. Photonics 2022, 9, 789 11 of 15 Figure 9. “Measurement-vs.-prediction” dataplots (see Fig. 8 caption) obtained during the atmospheric inference trial with the TurbNet-LED sensor performed six months after collection of the dataset used for DNN model training. The plots in Figures 8 and 9 show a close match between the sequence of datapoints [ C (t ) and C (t ) ] obtained with the TurbNet sensors and the corresponding n m n m measurements [ C (t ) ] obtained with the scintillometer. The corresponding values of the n k correlation coefficient γ computed for the moving-average DNN output and scintillometer datapoints were γ = 0.7 for the inference trial in Figure 8 (top), γ = 0.77 for the trials in Figure 8 (bottom), and γ = 0.9 for the inference trial in Figure 9. Note that a similar span of the correlation coefficient γ was observed in several more recent field experiments. Because of the high C sensing rate, the TurbNet sensors can be used to monitor rapid changes in atmospheric turbulence dynamics. This unique capability is illustrated in Figure 10. The measured vs. DNN-predicted time evolution plots in this figure represent exemplary segments of the corresponding dependences in Figure 8. The span of model-average C (t ) data provided by the TurbNet-LB and TurbNet-LED sensors n m clearly indicates the significant changes that occurred in turbulence strength during the 60 s time interval between sequential C measurements by the scintillometer. The DNN- predicted C values show about a two-fold change in Figure 10 (left) and nearly a five- fold change in Figure 10 (right). Photonics 2022, 9, 789 12 of 15 Figure 10. Deviation in predicted C values obtained with TurbNet-LB (left) and TurbNet-LED (right) sensors during an exemplary t = 60 second interval corresponding to sequential C scin n measurements by the scintillometer (horizontal red line), where DNN model-average C (t ) and n m moving-average C (t ) predictions are shown by green and blue curves, respectively. The charac- n m teristic grey-scale scintillation images below correspond to the highest (A), (C) and lowest (B), (D) 2 15 2 / 3 points of the model-averaged curves. Here, C  1.010 m and tm and tk are timestamps of data n, 0 received from the scintillometer [ C (t ) ] and TurbNet sensors, respectively. The characteristic Fried n k parameter values 𝑟 = (0.423𝑘 𝐶 𝐿) [22] corresponding to points A-D are: r0 = 5 cm and 8.2 cm for A and B, and r0 = 1.6 cm and 4.2 cm for C and D. Here k = 2π/λ, λ = 940 nm (TurbNet-LED), λ = 1064 nm (TurbNet-LB), L = 7 km. The selected scintillation images shown in Figure 10 (bottom) correspond to the high- est and lowest C values (points A through D in Figure 10 (top)). The scintillation pattern spatial features in these images corroborate the TurbNet sensor measurements in Figure 10 (top). These results show that the TurbNet sensors enabled detection of changes in turbulence strength at high temporal resolution (in real-time), which cannot be achieved with conventional EO instruments based on collection and time-averaging of sensing data. 7. Summary and Concluding Remarks In this paper, we discuss the implementation and field evaluation of novel EO sen- sors (TurbNet sensors) that enable atmospheric turbulence strength characterization at high temporal resolution via DNN-based processing of scintillation patterns originating from a remote laser or LED beacon, using an embedded-edge AI computer (Jetson Xavier Nx). The DNN models were developed and trained with datasets composed of a large number of instances (scintillation images and their corresponding values) collected during several atmospheric measurement trials over a 7 km propagation path. TurbNet sensor performance was validated in a set of atmospheric inference trials performed side- by-side with a conventional scintillometer. The results of the inference trials demonstrated good correspondence (correlation coefficient γ ranging from 0.7 to 0.9) between the scin- tillometer measurements and turbulence strength assessment with the TurbNet sensors. It was shown that, due to the high temporal rate (between 1.4 and 1.8 s per measurement, depending on the number of DNN models used), the TurbNet sensors enabled real-time evaluation of rapid changes in atmospheric turbulence dynamics. Photonics 2022, 9, 789 13 of 15 For practical applications of EO sensors utilizing DNN-based signal processing for C evaluation, one can potentially avoid atmospheric field trials used for the collection of DNN training datasets, which require side-by-side installation of both a DNN-based EO sensor (TurbNet) and a reference scintillometer, as described in Section 3. The training dataset can be generated using wave-optics numerical simulations that provide accurate modeling of laser-beam propagation over any selected turbulence characterization site and, thus, can be utilized for the generation of sufficiently large datasets (SIM datasets) for DNN training. A DNN that is trained with this dataset can then be further used for real-time processing of scintillation images and C prediction at the selected site, as de- scribed in [8]. In the case of DNN training using a simulated dataset, a scintillometer for collection of the training data in the field is not required. Furthermore, the EO sensor with a DNN model trained using a SIM dataset could be utilized at other propagation sites simply by computing a new SIM dataset specific for that site, with corresponding DNN retraining [8]. In conclusion, it should be noted that since DNN models of TurbNet sensors are “trained” using a dataset (training dataset) obtained for specific system parameters and propagation lengths, deviation from these pre-set conditions results in a corresponding decline in C prediction accuracy. As recently shown, this problem can be addressed via 2 2 scaling of the TurbNet sensor output C values [23]. The required C scaling factors n n can be obtained separately for each system parameter or propagation length using either an analytical expression derived from the classical Kolmogorov turbulence theory (the- ory-based scaling), or through wave-optics numerical modeling and simulations (M&S- based scaling) mimicking sensor operation. For the TurbNet sensor described in this pa- per, the theory-based scaling factor α(L) that minimizes C prediction error for the prop- agation distance L is given by the simple expression α(L) = L0 ⁄ L, where L0 is the propaga- tion length (L0 = 7 km) used for generation of the training dataset. A similar scaling (ad- justment) of the EO sensor output signal to address changes in path length or wavelength is used in conventional sensing systems. Typically, these adjustments are also based on either theory (available analytical expressions), simulations, or experimental measure- ments performed during sensor calibration [15,21]. Furthermore, conventional EO atmospheric turbulence sensing systems (e.g., scintil- lometers [15], differential image motion monitor (DIMM) sensors [16], etc.) are based on analytical expressions derived from Kolmogorov turbulence theory, and, for this reason, are restrained by the theoretical assumptions and simplifications made in the derivations of these formulas. This significantly limits the utilization of these sensing systems outside the provisions prescribed by the theory, which are specific for each sensor type, e.g., weak scintillations, quasi-monochromatic, spatially coherent laser-beacon beam, point-like in- coherent light source, etc. Violation of these prescribed requirements can lead to low ac- 2 2 C C curacy or even incorrect sensing data. This limitation of conventional sensors n n can be overcome in the DNN-based sensing approach described here, which can be ex- ploited for various EO sensing modalities (e.g., turbulence strength characterization using a partially coherent optical wave generated by the LED beacon of the TurbNet-LED sen- sor). The only requirement is that instantaneously measured output sensing data in such systems distinctly change in response to variations in atmospheric turbulence conditions. Author Contributions: Problem statement and TurbNet sensing concept: M.A.V.; experimental setup and sensor implementation: E.P. and D.L.N.H.; experimental data collection: E.P. and D.L.N.H.; signal processing and DNN model development: D.L.N.H.; writing paper: M.A.V.; anal- ysis and discussion of obtained results: M.A.V., E.P. and D.L.N.H.; preparation of the manuscript for submission: M.A.V., D.L.N.H. and E.P. All authors have read and agreed to the published ver- sion of the manuscript. Photonics 2022, 9, 789 14 of 15 Funding: This work was supported by the US Office of Naval Research through the contract N00014-17-1-2535 and the US Air Force Phase I STTR through the contract FA864922P0048. Data Availability Statement: The data that support the presented results are available from the corresponding author upon reasonable request. Acknowledgments: The authors would like to thank Artem Vorontsov for providing valuable in- sights into the development of the DNN models utilized in this paper. Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manu- script, or in the decision to publish the results. References 1. Majumdar, A.K.; Ricklin, J.C. Free-Space Laser Communications; Springer: New York, NY, USA, 2008. 2. Miller, N.; Widiker, J.; McManamon, P.; Haus, J. Active Multi-Aperture Imaging through Turbulence. In Proceedings of the SPIE 8395, Acquisition, Tracking, Pointing, and Laser Systems Technologies XXVI, Baltimore, MD, USA, 15 May 2012; p. 839504. https://doi.org/10.1117/12.921160. 3. Perram, G.P.; Cusumano, S.J.; Hengehold, R.L.; Fiorino, S.T. An Introduction to Laser Weapon Systems; Directed Energy Profes- sional Society: Albuquerque, NM, USA, 2010. 4. Weiner, M.M. Atmospheric Turbulence in Optical Surveillance Systems. Appl. Opt. 1967, 6, 1984–1991. https://doi.org/10.1364/AO.6.001984. 5. Hirose, M.; Miyamura, N.; Sato, S. Deviation-based wavefront correction using the SPGD algorithm for high-resolution optical remote sensing. Appl. Opt. 2022, 61, 6722–6728. https://doi.org/10.1364/AO.461222. 6. Andrews, L.C.; Phillips, R.L. Laser Beam Propagation through Random Media; SPIE Publications: Bellingham, WA, USA, 1998. https://doi.org/10.1117/3.626196. 7. Tunick, A. Optical turbulence parameters characterized via optical measurements over a 2.33 km free-space laser path. Opt. Exp. 2008, 16, 14646. https://doi.org/10.1364/oe.16.014645. 8. Vorontsov, A.M.; Vorontsov, M.A.; Filimonov, G.A.; Polnau, E. Atmospheric Turbulence Study with Deep Machine Learning of Intensity Scintillation Patterns. Appl. Sci. 2020, 10, 8136. https://doi.org/10.3390/app10228136. 9. Lionis, A.; Tsigopoulos, A.; Cohn, K. An Application of Artificial Neural Networks to Estimate the Performance of High-Energy Laser Weapons in Maritime Environments. Technologies 2022, 10, 71. https://doi.org/10.3390/technologies10030071. 10. Bolbasova, L.A.; Lukin, V.P. Atmospheric Research for Adaptive Optics. Atmos. Ocean Opt. 2022, 35, 288–302. https://doi.org/10.1134/S1024856022030022. 11. LeMaster, D.A.; Leung, S.; Mendoza-Schrock, O.L. Joint object classification and turbulence strength estimation using convolu- tional neural networks. Appl. Opt. 2021, 60, G40–G48. https://doi.org/10.1364/AO.425119. 12. Ma, S.; Hao, S.; Zhao, Q.; Xu, C.; Xiao, J. Prediction of Atmospheric Turbulence Refractive Index Structure Constant based on Deep Learning. In Proceedings of the SPIE 11717 24th National Laser Conference & Fifteenth National Conference on Laser Technology and Opto-Electronics, Shanghai, China, 2 December 2020; p. 1171709. https://doi.org/10.1117/12.2583944. 13. Pasupathi, T.; Selvi, A.V.J. Real time experimental investigation of adaptive optics compensation technique for free space optical communication. Photon. Netw. Commun. 2022. https://doi.org/10.1007/s11107-022-00973-z. 14. Bart, M.P.; Savino, N.J.; Regmi, P.; Cohen, L.; Safavi, H.; Shaw, H.C.; Lohani, S.; Searles, T.A.; Kirby, B.T.; Lee, H.; et al. Deep learning for enhanced free-space optical communications. arXiv 2022, arXiv:2208.07712. 15. Wang, T.; Ochs, G.R.; Clifford, S.F. A saturation-resistant optical scintillometer to measure Cn2. J. Opt. Soc. Am. 1978, 68, 334– 338. https://doi.org/10.1364/JOSA.68.000334. 16. Belen’kii, M.S.; Roberts, D.W.; Stewart, J.M.; Gimmestad, G.G.; Dagle, W.R. Experimental validation of the differential image motion lidar concept. Opt. Lett. 2000, 25, 518–520. https://doi.org/10.1364/ol.25.000518. 17. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. High Temporal Resolution Sensing of Atmospheric Turbulence Refractive Index Structure Parameter (Cn^2) based on Embedded Edge AI-Processing of Scintillation Images. In Proceedings of the SPIE 12102, Real-Time Image Processing and Deep Learning, Orlando, FL, USA, 27 May 2022; p. 121020B. https://doi.org/10.1117/12.2618431. 18. Hettiarachchi, D.L.N.; Polnau, E.; Vorontsov, M.A. Electro-Optics Sensor with Embedded Edge AI-Computing for Atmospheric Turbulence Refractive Index Structure Parameter (Cn2) Sensing at High Temporal Resolution. In Proceedings of the SPIE 11993, Free-Space Laser Communications XXXIV, San Francisco, CA, USA, 4 March 2022; p. 119930X. https://doi.org/10.1117/12.2618110. 19. Wang, Y.; Basu, S. Using an artificial neural network approach to estimate surface-layer optical turbulence at Mauna Loa, Ha- waii. Opt. Lett. 2016, 41, 2334–2337. https://doi.org/10.1364/OL.41.002334. 20. Nair, G.N.; Mei, K.; Patel, V.M. AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models. arXiv 2022, arXiv:2208.11284. 21. Large Aperture Scintillometer BLS2000 User’s Manual. Available online: https://www.scintec.com (accessed on 27 August 2020). Photonics 2022, 9, 789 15 of 15 22. Tyson, R.K.; Frazier, B.W. Field Guide to Adaptive Optics; SPIE Press: Bellingham, WA, USA, 2012. https://doi.org/10.1117/3.923078. 23. Filimonov, G.; Vorontsov, M. Towards Artificial-Intelligence-Based Optical Scintillometer: Scaling Issue. In Proceedings of the IEEE International Symposium on Antennas and Propagation and USNC-URSI Radio Science, Denver, CO, USA, 10–15 July

Journal

PhotonicsMultidisciplinary Digital Publishing Institute

Published: Oct 24, 2022

Keywords: atmospheric turbulence; deep neural network; electro-optics sensor; embedded edge AI computing; NVIDIA Jetson Xavier Nx; real-time sensing

There are no references for this article.