Access the full text.

Sign up today, get DeepDyve free for 14 days.

Photonics
, Volume 9 (7) – Jun 28, 2022

/lp/multidisciplinary-digital-publishing-institute/study-on-the-influence-of-random-vibration-of-space-based-payload-on-ApAFOeZZuG

- Publisher
- Multidisciplinary Digital Publishing Institute
- Copyright
- © 1996-2022 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer The statements, opinions and data contained in the journals are solely those of the individual authors and contributors and not of the publisher and the editor(s). MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Terms and Conditions Privacy Policy
- ISSN
- 2304-6732
- DOI
- 10.3390/photonics9070455
- Publisher site
- See Article on Publisher Site

hv photonics Article Study on the Inﬂuence of Random Vibration of Space-Based Payload on Area-Array Camera Frame-by-Frame Imaging Ziqi Yu, Li Jiang *, Kai Ling and Zhihai Yao College of Science, Changchun University of Science and Technology, Changchun 130022, China; 2019100055@mails.cust.edu.cn (Z.Y.); 2020100097@mails.cust.edu.cn (K.L.); yaozh@cust.edu.cn (Z.Y.) * Correspondence: jiangli@cust.edu.cn Abstract: Satellite vibration is an important factor that can seriously reduce the image quality of remote sensing imaging. In this paper, the inﬂuence of the random vibration of the payload on the frame-by-frame imaging quality of the area-array CMOS camera is studied. Firstly, the mode of the camera’s dynamic frame-by-frame imaging is established; secondly, the curvature mapping function between the ground and camera focal planes is derived based on the invariance of the ﬁeld of view (FOV). The frequency domain-ﬁltered template of random vibration is constructed. Then, the distortion and degradation images, of single-frame images at different attitude angles, are inversed; ﬁnally, the inﬂuence of attitude angle, exposure time, and the error of velocity, caused by random vibrations on dynamic frame-by-frame imaging, are analyzed. The simulation results show that the degree of image edge distortion gets larger when the attitude angle increases from 0 to 50 . At the same time, the MTF decreases rapidly with the increase of the error of velocity and the attitude angle. Meanwhile, when the output-image SSIM decreases by 0.1, the MSE increases by 18.5. The experimental results show that the ﬁeld of view (FOV) of dynamic imaging should be reasonably set, and the error of velocity should be effectively reduced to obtain high-quality remote sensing images. Keywords: remote sensing; satellite vibration; frequency domain analysis; image distortion Citation: Yu, Z.; Jiang, L.; Ling, K.; Yao, Z. Study on the Inﬂuence of Random Vibration of Space-Based 1. Introduction Payload on Area-Array Camera Space-based remote sensing technology has important signiﬁcance and application in Frame-by-Frame Imaging. Photonics national construction and urban planning [1–3]. Coverage has always been an important 2022, 9, 455. https://doi.org/ indicator for evaluating high-resolution aerospace images. Technologies, such as multi- 10.3390/photonics9070455 satellite networking, multi-camera stitching [4], and agile satellite imaging, have effectively Received: 19 April 2022 increased the coverage of remote sensing images [5]. However, the burden of the space Accepted: 16 June 2022 payload has also increased. In order to reduce the weight of payload, a dynamic frame-by- Published: 28 June 2022 frame imaging mode, in which an optical camera and an oscillating mirror cooperate with each other, has been proposed by scholars [6,7]. In this mode, the mutual restriction between Publisher’s Note: MDPI stays neutral the aperture and focal length of the optical system, which can realize high-resolution and with regard to jurisdictional claims in large-width imaging, has been broken. However, the multi-mechanism cooperative motion published maps and institutional afﬁl- will have more random interference factors, which can affect the stability of the dynamic iations. velocity and reduce the quality of the image in this dynamic imaging mode [8]. Therefore, it is necessary to further systematically analyze the law of image degradation in the dynamic imaging process. Copyright: © 2022 by the authors. In the process of remote sensing imaging, mechanical vibrations lead to the lower Licensee MDPI, Basel, Switzerland. precision of the accurate alignment system and the instability of the tracking system, which This article is an open access article seriously affect the quality of satellite imaging [9]. So far, researchers have analyzed distributed under the terms and the effect of satellite vibration on imaging quality in traditional imaging mode. The conditions of the Creative Commons optical system function (OTF) of linear vibration and sine vibration on the image plane Attribution (CC BY) license (https:// were derived by Hadar et al. [10,11]. Adrian et al. [12] further proposed a method for creativecommons.org/licenses/by/ calculating OTF with no limitation of any speciﬁc satellite vibration type, and the formula 4.0/). Photonics 2022, 9, 455. https://doi.org/10.3390/photonics9070455 https://www.mdpi.com/journal/photonics Photonics 2022, 9, 455 2 of 14 of the modulation transfer function (MTF) was extended to the full frequency range by Du et al. [13]. Since the total MTF was reduced by spatial vibration, a frequency domain ﬁltering system, which included various vibration types, had been mentioned by Deng [14] and Li et al. [15] to simulate the image degradation phenomenon caused by the multi- source vibration of the payload. In recent years, many new modes of dynamic imaging have been proposed in order to improve the imaging width and resolution. A circular-scanning imaging model was established by Song et al. [16]. By setting a reasonable scanning speed, the overlapping output of multi-frame images was completed to maximize the imaging coverage of the camera. Zhong et al. [17] further described the optical rotation phenomenon between the object and the image, in the circular scanning imaging model, by establishing different coordinate systems, so the imaging model of the circular scanning sensor tended to be mature. Liu [18] et al. proposed a scanning mirror system, which consisted of an objective lens, relay lens, and three-stage imaging lens, to simulate the relationship between objects and images in a two-dimensional scanning motion. It provided a theoretical basis for the structural design of the scanning mirror. Meanwhile, Tianqing et al. [19] established an experimental dynamic scanning imaging system with fast steering mirrors, which was used to monitor hot targets and scout hot areas in real time. With the increase of the scanning attitude angle, the output image was seriously distorted. Ng et al. [20] conﬁrmed that imaging distortion could be corrected by measuring the four polarization directions of polarization imaging. In the scanning imaging system, using a camera’s front mirror, Lu [21] further expounded on the design scheme from two aspects: structure and the control circuit. In this system, the encoder read the payload attitude information when the mirror of the camera periodically scanned. With the development of dynamic optical imaging technology, the vibration analysis of traditional imaging can’t meet the actual requirements. Therefore, it is necessary to further study the law of image degradation, caused by vibration, in the new dynamic imaging mode. During the frame-by-frame imaging process, the internal disturbance source of the payload randomly vibrates irregularly. When random vibration is coupled with the disturbance characteristics of the dynamic frame-by-frame imaging model, new problems will appear. In order to solve the above problems, a periodic dynamic frame-by-frame imaging model of the CMOS camera is established in this paper. The pixels’ curvature mapping function, between the payload CMOS camera and the ground imaging range, is derived according to the invariance of the ﬁeld of view. The error of velocity, caused by random vibration, is introduced into the camera focal plane to invert the image degradation phenomenon in the process of dynamic imaging and to quantitatively analyze the quality of space remote-sensing images in a complex space environment. 2. Model of Dynamic Imaging 2.1. Dynamic Frame-by-Frame Imaging Models In order to expand the ground imaging range, the SiC mirror rotates periodically in front of the CMOS camera scans, at a certain velocity, in the dynamic imaging mode [22]. In this process, the dynamic frame-by-frame imaging schematic is established, as shown in Figure 1. When the satellite ﬂies to the observation area along the orbit, the mirror rotates with a certain angular speed at the initial position, and the initial sub-frame image is exposed at T The exposure time interval is calculated by satellite, according to the single frame width, and then, the single-frame images along the vertical track direction are obtained according to the time interval T2, T3 . . . Tn. Finally, the extended-amplitude image can be gained after splicing all the frame images according to the time sequence. However, the ground area, covered by CMOS pixels, changes from a square to an approximate trapezoidal image, with different sizes, as the imaging attitude angle increases in the dynamic imaging mode. At the same time, the anisotropic motion between pixels leads to serious distortion of the image on the CMOS focal plane. In addition, the internal rigid structure, when subjected to Photonics 2022, 9, x FOR PEER REVIEW 3 of 15 Photonics 2022, 9, 455 3 of 14 dynamic disturbance, produces serious random vibration. It makes the CMOS focal plane deviate from the target image by a certain angle, resulting in the degradation of image quality. For the sake of effectively analyzing the distorted output image and the degraded Photonics 2022, 9, x FOR PEER REVIEW 3 of 15 image caused by random vibration in the dynamic frame-by-frame imaging process, the imaging simulation link is established, as shown in Figure 2. Figure 1. Schematic diagram of dynamic frame-by-frame imaging. When the satellite flies to the observation area along the orbit, the mirror rotates with a certain angular speed at the initial position, and the initial sub-frame image is exposed at T1. The exposure time interval is calculated by satellite, according to the single frame width, and then, the single-frame images along the vertical track direction are obtained according to the time interval T2, T3…Tn. Finally, the extended-amplitude image can be gained after splicing all the frame images according to the time sequence. However, the ground area, covered by CMOS pixels, changes from a square to an approximate trapezoidal image, with different sizes, as the imaging attitude angle increases in the dynamic imaging mode. At the same time, the anisotropic motion between pixels leads to serious distortion of the image on the CMOS focal plane. In addition, the internal rigid structure, when subjected to dynamic disturbance, produces serious random vibration. It makes the CMOS focal plane deviate from the target image by a certain angle, resulting in the degradation of image quality. For the sake of effectively analyzing the distorted output image and the degraded image caused by random vibration in the dynamic frame- by-frame imaging process, the imaging simulation link is established, as shown in Figure Figure 1. Schematic diagram of dynamic frame-by-frame imaging. Figure 1. Schematic diagram of dynamic frame-by-frame imaging. When the satellite flies to the observation area along the orbit, the mirror rotates with a certain angular speed at the initial position, and the initial sub-frame image is exposed at T1. The exposure time interval is calculated by satellite, according to the single frame width, and then, the single-frame images along the vertical track direction are obtained according to the time interval T2, T3…Tn. Finally, the extended-amplitude image can be gained after splicing all the frame images according to the time sequence. However, the ground area, covered by CMOS pixels, changes from a square to an approximate trapezoidal image, with different sizes, as the imaging attitude angle increases in the dynamic imaging mode. At the same time, the anisotropic motion between pixels leads to serious distortion of the image on the CMOS focal plane. In addition, the internal rigid structure, when subjected to dynamic disturbance, produces serious random vibration. It makes the CMOS focal plane deviate from the target image by a certain angle, resulting in the degradation of image quality. For the sake of effectively analyzing the distorted output image and the degraded image caused by random vibration in the dynamic frame- by-frame imaging process, the imaging simulation link is established, as shown in Figure Figure 2. Simulation link for dynamic imaging. Figure 2. Simulation link for dynamic imaging. The ground target image is obtained by one exposure of the camera, as shown in Figure 2 1 , where the pixel’s coordinate function is g(x, y). According to the invariance of the ﬁeld of view, distortion image on the camera’s focal plane is obtained, as shown in Figure 2 2 , where the pixel’s coordinate function is p(x, y). The generalized function xfg is expressed as the transformation operation between the ground target and the distorted image on the camera focal plane, so the mapping relationship can be expressed as: p(x, y) = xfg(x, y)g (1) The degraded image can be obtained by the convolution of the distorted image function and the line spread function h(x, y), which is caused by satellite vibration, as shown in Figure 2 3 . Figure 2. Simulation link for dynamic imaging. Photonics 2022, 9, 455 4 of 14 o(x, y) = h(x, y) xfg(x, y)g (2) In the formula, o(x, y) is the degraded image interfered by random vibration. With the transformation of the Fourier transform, the frequency domain calculation formula is expressed as: O f , f = MTF f , f F x g x, y (3) f [ ( )]g x y x y where, O f , f is the output-image spectrum; Ffx[g(x, y)]g is the input image spectrum; x y MTF f , f is called the modulation transfer function, which is the Fourier transform of x y the line spread function h(x, y) caused by random vibration. MTF f , f = h(x, y) exp j2p f x + f y dxdy (4) x y x y It represents that the different frequency components are modulated by the ﬁltering system. The distortion degraded image is obtained by the inverse Fourier transform of the output-image spectrum O f , f , as shown in Figure 2 4 . x y 2.2. The Exposure Equation of CMOS Camera The synchronization between the ground imaging efﬁciency and the reading efﬁciency of the pixels on the camera focal plane is the premise of high-resolution imaging [23]. According to the object–image mapping relationship, the exposure time of the CMOS camera in the dynamic frame-by-frame imaging process is deﬁned as the function of the ground resolution GSD and the ground imaging velocity V , which can be expressed as follows: t = GSD/V (5) As shown in Figure 3, an instantaneous imaging model is established by taking the single-frame image at the moment T in Figure 1 as an example. The expression of the geocentric angle b and the camera object distance L can be calculated from the trigonometric function: R + H b = p q arcsin sin q (6) l = (R + H) + R 2(R + H)R cos d (7) e e e e where, R is the radius of the earth and H is the height of the satellite orbit. According to the relationship among camera object distance L, focal length f , and pixel size a, the ground resolution calculation formula can be expressed as: L a n h io GSD = (8) R +H f cos p arcsin sin q Since the imaging velocity of the camera on the ground is combined velocity V , which consists of camera scanning velocity V and satellite ﬂight velocity V = GM/(R + H), x y e it can be expressed as: V = (V L) + GM/(R + H) (9) c x e In the formula, G is the gravitational constant and M is the mass of the earth. It can be supposed that the number of the imaging line of the area-array CMOS camera is N, while the exposure time of the area-array camera, at different attitude angles, can be expressed as: GSD t(q) = (10) t=1 Photonics 2022, 9, x FOR PEER REVIEW 5 of 15 (7) lR=+H +R2− R+HRcosδ () () ee e e where, R is the radius of the earth and is the height of the satellite orbit. According to the relationship among camera object distance , focal length , and pixel size , the ground resolution calculation formula can be expressed as: La × GSD = RH + (8) f×− cosπθ arcsin sin Since the imaging velocity of the camera on the ground is combined velocity , which consists of camera scanning velocity and satellite flight velocity VG=+ M() R H ye , it can be expressed as: VV=+L GM R+H (9) () ( ) cx e In the formula, G is the gravitational constant and is the mass of the earth. It can be supposed that the number of the imaging line of the area-array CMOS camera is , while the exposure time of the area-array camera, at different attitude angles, can be expressed as: Photonics 2022, 9, 455 GSD 5 of 14 t θ = () (10) τ =1 Figure 3. Schematic diagram of overall dynamic frame by frame imaging. Figure 3. Schematic diagram of overall dynamic frame by frame imaging. 2.3. Curvature Distortion Mapping Function of Ground Pixels 2.3. Curvature Distortion Mapping Function of Ground Pixels For explaining the curvature mapping relationship between the ground target image For explaining the curvature mapping relationship between the ground target image Photonics 2022, 9, x FOR PEER REVIEW 6 of 15 and camera focal plane image, the local frame-by-frame imaging model between the object and camera focal plane image, the local frame-by-frame imaging model between the object and the image is established, as shown in Figure 4. and the image is established, as shown in Figure 4. Figure 4. Schematic diagram of local dynamic frame-by-frame imaging. Figure 4. Schematic diagram of local dynamic frame-by-frame imaging. Firstly, the phase coordinate system is set, in which the center of the focal plane of Firstly, the phase coordinate system is set, in which the center of the focal plane of ij × the CMOS camera is the coordinate origin . There are imaging units on the phase the CMOS camera is the coordinate origin O. There are i j imaging units on the phase surface surface (t (the he n number umber of of imaging imaging units units in in the the X direct X direction ion is is i,i, and and t the henumber number of of imaging imaging units in the Y direction is j). Secondly, the object plane coordinate system is set, in which units in the Y direction is ). Secondly, the object plane coordinate system is set, in which the center point of the ground imaging area is the coordinate origin Q. Then, the ﬁeld angle the center point of the ground imaging area is the coordinate origin . Then, the field d and m δ of any pixel point in the X direction and the Y direction on the camera focal plane angle and of any pixel point in the X direction and the Y direction on the camera can be expressed as: focal plane can be expressed as: 2p i tan d = a (11) 2pi − 2 f tan δ= ⋅ a (11) 2 f 2pj − tanμ= ⋅ a (12) 2 f In the formula, f is the focal length of the camera, a is the pixel size, while p and are the X and Y components of the target pixel in the phase coordinate system, respectively. Then, any pixel point in the object plane coordinate system can be expressed as: H ⋅tan δ g = x (13) cos() θ + μ gH =⋅tan θ+ μ () (14) g g In the formula, and are X and Y components of target pixel point in the object x y plane, respectively. Combining Formulas (13) and (14), the curvature mapping function ξ g xy, , which represents the relationship between the ground target image and the { ()} phase surface distortion image, can be expressed as: Ha 2 p − i () g = 2 f cos θ+− arctan ap 2 j 2f () {} ξ gx,= y {} () (15) g=+Ha tan θ arctan 2p−j 2f () {} yy Photonics 2022, 9, 455 6 of 14 2p j tan m = a (12) 2 f In the formula, f is the focal length of the camera, a is the pixel size, while p and p x y are the X and Y components of the target pixel in the phase coordinate system, respectively. Then, any pixel point in the object plane coordinate system can be expressed as: H tan d g = (13) cos(q + m) g = H tan(q + m) (14) In the formula, g and g are X and Y components of target pixel point in the object x y plane, respectively. Combining Formulas (13) and (14), the curvature mapping function xfg(x, y)g, which represents the relationship between the ground target image and the phase surface distortion image, can be expressed as: Ha(2p i) g = 2 f cos q+arctan a 2p j /2 f f [ ( y ) ]g xfg(x, y)g = (15) g = H tan q + arctan a 2p j /2 f y y 2.4. Frequency Domain Image Filtering of Random Vibration In the process of frame-by-frame imaging, a certain error of velocity V of the optical camera is caused by the multi-source vibration of the aerospace payload, which is trans- mitted to the camera focal plane through the mechanical structure. After camera exposure, such error image shifts d can sharply destroy the output image sharpness. If the input is image is a gaussian image, the line spread function LSF, caused by random vibration, can be expressed as: 1 x LSF = p exp (16) 2pd 2d is is The MTF of the ﬁlter system is obtained by the Fourier transform of the LSF: 2 2 2 MTF(u) = exp 2p d u (17) is In a linear system, the function of the MTF is approximated to be a digital ﬁlter, which is called a ﬁlter template. It realizes the response to various frequency components of the input spectrum in an image ﬁltering system, and the output image spectrum O f , f can x y be obtained from Formula (3) 2 2 2 O f , f = F x g x, y exp 2p d u (18) f [ ( )]g x y is The degraded image o(x, y) can be obtained by the inverse Fourier transform of the output image spectrum: m1 n1 o(x, y) = O f , f exp j2p x f /m + y f /n (19) x y x y å å f =0 f =0 x y where x = 0, 1, . . . , m 1 and Y = 0, 1, . . . , n 1 indicate that the size of the input image is m n. 2.5. Image Sharpness Evaluation SSIM and MSE are common image sharpness assessment tools that can compare the difference between two images of the same size. After the degraded image of the frequency domain ﬁltering is obtained, SSIM and MSE are used to analyze the regularity of image quality degradation caused by satellite vibration. Photonics 2022, 9, 455 7 of 14 The formula for SSIM is expressed as: t t t 1 2 3 SSIM(x , y ) = [l(x , y )] [c(x , y )] [s(x , y )] (20) i i i i i i i i t t t 1 2 3 where [l(x , y )] is brightness distortion; [c(x , y )] is contrast distortion; [s(x , y )] is i i i i i i structural distortion. t , t , and t are the corresponding weight coefﬁcients, respectively. 1 2 3 The value range of SSIM is [0, 1]. The larger the value of SSIM is, the smaller the image distortion is. The formula for MSE is expressed as: M N MSE = [o(x , y ) x[g(x , y )]] (21) å å i i i i MN x=1 y=1 where o(x , y ) and x[g(x , y )] are the pixel values of the reference image, and the evaluated i i i i image at M and N are also the image size. The higher the value of MSE is, the greater the degree of the distortion is. 3. Results Based on the above theoretical model, the degradation phenomenon of the output image is simulated and analyzed in the dynamic frame-by-frame imaging process. The parameters of the dynamic imaging model are set, as shown in Table 1: Table 1. Simulation parameters. Serial Number Simulation Parameters Value 1 height of the track 500 km 2 camera focal length 0.5 m 3 earth radius 6400 km 4 dynamic scan velocity 10 /s 5 pixel size of camera focal plane 3.5 m 6 attitude angle –50 50 3.1. Ground Vibration Test In the dynamic frame-by-frame imaging process, the certain error of velocity, caused by random vibration, has occurred in the mechanical structure. The error of velocity leads to the imaging shift on the camera focal plane within the exposure time, which directly causes the image blur. Therefore, it is necessary to analyze the inﬂuence of the error of velocity caused by satellite vibration on dynamic imaging. In the ground test platform, the multi-cycle tracking velocity is obtained, as shown in Figure 5a, where the blue curve is the actual tracking velocity, and the red curve is the ideal tracking velocity. It can be seen that there is a certain error between the actual velocity and the ideal velocity from Figure 5a. Based on the error analysis of the ground tracking velocity, the curve of the error of velocity is obtained, as shown in Figure 5b, where the error of velocity peaks at different frequencies have been marked. Five pieces of data are extracted in the actual error of the velocity range (0.0007577 /s, 0.001146 /s); meanwhile, the inﬂuences of different error velocities and attitude angles on the MTF of the system are analyzed in the following part. Photonics 2022, 9, x FOR PEER REVIEW 8 of 15 3. Results Based on the above theoretical model, the degradation phenomenon of the output image is simulated and analyzed in the dynamic frame-by-frame imaging process. The parameters of the dynamic imaging model are set, as shown in Table 1: Table 1. Simulation parameters. Serial Number Simulation Parameters Value 1 height of the track 500 km 2 camera focal length 0.5 m 3 earth radius 6400 km 4 dynamic scan velocity 10°/s 5 pixel size of camera focal plane 3.5 μm 6 attitude angle –50° ≤ θ ≤ 50° 3.1. Ground Vibration Test In the dynamic frame-by-frame imaging process, the certain error of velocity, caused by random vibration, has occurred in the mechanical structure. The error of velocity leads to the imaging shift on the camera focal plane within the exposure time, which directly causes the image blur. Therefore, it is necessary to analyze the influence of the error of velocity caused by satellite vibration on dynamic imaging. In the ground test platform, the multi-cycle tracking velocity is obtained, as shown in Figure 5a, where the blue curve is the actual tracking velocity, and the red curve is the ideal tracking velocity. It can be seen that there is a certain error between the actual velocity and the ideal velocity from Figure 5a. Based on the error analysis of the ground tracking velocity, the curve of the error of velocity is obtained, as shown in Figure 5b, where the error of velocity peaks at different frequencies have been marked. Five pieces of data are extracted in the actual error of the velocity range (0.0007577°/s, Photonics 2022, 9, 455 8 of 14 0.001146°/s); meanwhile, the influences of different error velocities and attitude angles on the MTF of the system are analyzed in the following part. Photonics 2022, 9, x FOR PEER REVIEW 9 of 15 (a) (b) Figure 5. Ground platform test data. (a) Ground tracking velocity test curves, (b) The error of Figure 5. Ground platform test data. (a) Ground tracking velocity test curves, (b) The error of velocity image. velocity image. 3.2. Analysis of Ground Distortion Imaging 3.2. Analysis of Ground Distortion Imaging During the imaging process of the dynamic scanning mirror, the payload’s attitude During the imaging process of the dynamic scanning mirror, the payload’s attitude angle changes periodically. When the angle between the optical axis of the camera and the angle changes periodically. When the angle between the optical axis of the camera and satellite nadir gets larger, the ground imaging range, corresponding to CMOS pixels, the satellite nadir gets larger, the ground imaging range, corresponding to CMOS pixels, gradually increases. It leads to the anisotropy of motion between pixels and the distortion gradually increases. It leads to the anisotropy of motion between pixels and the distortion of images projected onto the focal plane of the camera. In order to simulate this image, a of images projected onto the focal plane of the camera. In order to simulate this image, a unit pixel is set in every 30 pixels, which represents the pixel-number of the ground image. unit pixel is set in every 30 pixels, which represents the pixel-number of the ground image. The arrows represent the direction of movement of the ground image during the imaging The arrows represent the direction of movement of the ground image during the imaging process. The distortion diagram of the ground pixels, at attitude angles of 0°, 30°/−30°, and process. The distortion diagram of the ground pixels, at attitude angles of 0 , 30 /30 , 50°/−50°, can be calculated by the curvature mapping function (15), as shown in the Figure and 50 /50 , can be calculated by the curvature mapping function (15), as shown in the Figure 6. (a) (b) (c) Figure 6. Schematic diagram of distorted image. (a) Attitude angle 0°, (b) Attitude angle 30°/−30°, Figure 6. Schematic diagram of distorted image. (a) Attitude angle 0 , (b) Attitude angle 30 /30 , (c) Attitude angle 50°/−50°. (c) Attitude angle 50 /50 . It can be seen that, when the attitude angle of satellite is 0 , the ground imaging It can be seen that, when the attitude angle of satellite is 0 °, the ground imaging range is about 300 pixels 250 pixels. It indicates that the ground imaging range is nar- range is about 300 pixels × 250 pixels. It indicates that the ground imaging range is narrow, row, and there is almost no image distortion, as shown in Figure 6a. When the satellite and there is almost no image distortion, as shown in Figure 6a. When the satellite attitude attitude angle gradually increases to 30 or 50 , the ground imaging range becomes angle gradually increases to 30 ° or 50 °, the ground imaging range becomes 400 pixels × 400 pixels 350 pixels or 700 pixels 450 pixels. It can be seen that, when the attitude an- 350 pixels or 700 pixels × 450 pixels. It can be seen that, when the attitude angle increases, gle increases, the ground imaging range becomes wider, and the image distortion deepens, the ground imaging range becomes wider, and the image distortion deepens, as shown in as shown in Figure 6b,c. Figure 6b,c. 3.3. Analysis the MTF of Sub-Satellite Point Imaging Interfered by Random Vibration When the attitude angle of satellite is 0°, both the ground resolution GSD and the ground scanning velocity Vc are fixed values. Based on Formula (10), the exposure time is t = 0.0051 s. The MTF curves of different error velocities are simulated, as shown in Figure Photonics 2022, 9, x FOR PEER REVIEW 9 of 15 3.2. Analysis of Ground Distortion Imaging During the imaging process of the dynamic scanning mirror, the payload’s attitude angle changes periodically. When the angle between the optical axis of the camera and the satellite nadir gets larger, the ground imaging range, corresponding to CMOS pixels, gradually increases. It leads to the anisotropy of motion between pixels and the distortion of images projected onto the focal plane of the camera. In order to simulate this image, a unit pixel is set in every 30 pixels, which represents the pixel-number of the ground image. The arrows represent the direction of movement of the ground image during the imaging process. The distortion diagram of the ground pixels, at attitude angles of 0°, 30°/−30°, and 50°/−50°, can be calculated by the curvature mapping function (15), as shown in the Figure (a) (b) (c) Figure 6. Schematic diagram of distorted image. (a) Attitude angle 0°, (b) Attitude angle 30°/−30°, (c) Attitude angle 50°/−50°. It can be seen that, when the attitude angle of satellite is 0 °, the ground imaging range is about 300 pixels × 250 pixels. It indicates that the ground imaging range is narrow, and there is almost no image distortion, as shown in Figure 6a. When the satellite attitude angle gradually increases to 30 ° or 50 °, the ground imaging range becomes 400 pixels × 350 pixels or 700 pixels × 450 pixels. It can be seen that, when the attitude angle increases, the ground imaging range becomes wider, and the image distortion deepens, as shown in Photonics 2022, 9, 455 9 of 14 Figure 6b,c. 3.3. Analysis the MTF of Sub-Satellite Point Imaging Interfered by Random Vibration 3.3. Analysis the MTF of Sub-Satellite Point Imaging Interfered by Random Vibration When the attitude angle of satellite is 0°, both the ground resolution GSD and the When the attitude angle of satellite is 0 , both the ground resolution GSD and the ground scanning velocity Vc are fixed values. Based on Formula (10), the exposure time is ground scanning velocity V are ﬁxed values. Based on Formula (10), the exposure time t = 0.0051 s. The MTF curves of different error velocities are simulated, as shown in Figure is t = 0.0051 s. The MTF curves of different error velocities are simulated, as shown in Figure 7. Photonics 2022, 9, x FOR PEER REVIEW 10 of 15 Figure 7. MTF curves at different error velocities. Figure 7. MTF curves at different error velocities. It can be seen that, when the velocity error is caused by random vibration increases, the error image shift on the focal plane of the camera gets larger. Therefore, the MTF curve It can be seen that, when the velocity error is caused by random vibration increases, shows a significant downward trend, which means that the clarity of the output image is the error image shift on the focal plane of the camera gets larger. Therefore, the MTF curve reduced. shows a signiﬁcant downward trend, which means that the clarity of the output image is reduced. 3.4. Analysis the MTF of Dynamic Imaging Interfered by Random Vibration 3.4. Analysis the MTF of Dynamic Imaging Interfered by Random Vibration In the dynamic frame by frame imaging system, the attitude angle of the camera will In the dynamic frame by frame imaging system, the attitude angle of the camera will change periodically. In this process, the exposure time of the camera is different when the change periodically. In this process, the exposure time of the camera is different when attitude angle of satellite varies at different velocities. Therefore, it is necessary to calculate the attitude angle of satellite varies at different velocities. Therefore, it is necessary to the exposure time of subframes with different attitude angles. According to Equation (10) calculate the exposure time of subframes with different attitude angles. According to and the parameters in Table 1, the exposure time of the camera at different angles is Equation (10) and the parameters in Table 1, the exposure time of the camera at different simulated, as shown in Figure 8. angles is simulated, as shown in Figure 8. Figure 8. Camera exposure function. Figure 8. Camera exposure function. It can be seen that the camera exposure time becomes larger as the satellite attitude It can be seen that the camera exposure time becomes larger as the satellite attitude angle increases during the single-frame imaging in Figure 8. angle increases during the single-frame imaging in Figure 8. In order to analyze the influence of the error of velocity V and attitude angle θ on MTF, spatial frequency is set as the Nyquist frequency υ =1/2a , and then, the MTF is transformed into: 22 2 −⋅2tπθ fVant () e MTF θ =exp () (22) 4a The MTF curves with different error velocities and attitude angles are obtained, as shown in Figure 9. Photonics 2022, 9, 455 10 of 14 In order to analyze the inﬂuence of the error of velocity V and attitude angle q on MTF, spatial frequency is set as the Nyquist frequency u = 1/2a, and then, the MTF is transformed into: 2 2 2 2p f tan [V t(q)] MTF(q) = exp (22) 4a Photonics 2022, 9, x FOR PEER REVIEW 11 of 15 The MTF curves with different error velocities and attitude angles are obtained, as shown in Figure 9. Figure Figure 9. 9. MTF MTFccurves urves of d of dynamic ynamic frame-by-frame imaging. frame-by-frame imaging. Comparing Figures 7 and 9, it can be seen that there are similar conclusions in the Comparing Figures 7 and 9, it can be seen that there are similar conclusions in the dynamic imaging model and the satellite nadir imaging model: When the camera’s attitude dynamic imaging model and the satellite nadir imaging model: When the camera’s angle attitude is set angle is set as as a ﬁxed value, a fixed val the MTF ue, t decr he eases MTFwith decrthe easeincr s weasing ith the incr erroreasing of velocity error o . f However, when the error of velocity is set as a ﬁxed value, the MTF value achieves its velocity. maximum However, wh at the attitude en the error angle of0 ve . locity When is se the t as optical a fixed axis value, the of the camera MTF value deviates ach friom evesthe satellite nadir, MTF decreases signiﬁcantly, so it appears to be an approximate “arch” in the its maximum at the attitude angle 0°. When the optical axis of the camera deviates from MTF the sat curves. ellite nadir, MTF decreases significantly, so it appears to be an approximate “arch” It can be explained that the exposure time of the camera at the satellite nadir is the in the MTF curves. shortest. It can be expla With the iincr ned tha easing t the exposure time of attitude angle, the the camera exposure time at the sa of the telli camer te nadi ar becomes is the longer shortest. Wi . When th the i the camera ncreasi isng disturbed attitude by angle, the the exposure ti error of velocityme , caused of the ca by random mera becomes vibration, the longer. Wh error image en the camer shift onathe is disturbe camerad by focal thplane e error o is limited f velocity, bycaused the short by r exposur andom v e itime. bration, With the error image shift on the camera focal plane is limited by the short exposure time. With the increasing attitude angle, the error image shift in the camera focal plane becomes larger due the ito ncreasi the long ng atti exposur tude aengl time. e, the error i The more merr age sh or image ift in t shifts he camer are intr a foc oduced al plane into becomes the focal larger due to the long exposure time. The more error image shifts are introduced into the plane of the camera, the more signiﬁcantly the MTF drops. Therefore, the image quality of the focal p sub-frames, lane of the c farther amer fr aom , the more the satellite significantly the nadir, is degraded MTF drops. more obviously Therefore, th in e im a group age of quality of the sub-frames, farther from the satellite nadir, is degraded more obviously in single-frame wide-frame images. a group of single-frame wide-frame images. 3.5. Remote Sensing Image Simulation 3.5. Remote Sensing Image Simulation In this section, the mechanism of image distortion of the curvature map and image degradation, caused by satellite random vibration, have been explained. Firstly, the high- In this section, the mechanism of image distortion of the curvature map and image r degr esolution adation, ground caused by image sais tell converted ite random by vibrat the gr ion, h ound ave been exp curvature mapping lained. Firstly Formula , the high (15), and - the distorted image on the camera focal plane is obtained; secondly, the distorted image is resolution ground image is converted by the ground curvature mapping Formula (15), ﬁlter and the di ed in the storted i frequency mage on the ca domain bymera f the ﬁlter ocal template plane is (18) obtai and, ned; secondl then, by the y, the di Fourier storted inverse transform image is fi(19). ltered i Finally n the f , the req degraded uency dom image ain bis y the fi obtained. lter template (18) and, then, by the Fourier In the inver coaxial se tran plane sform optical (19). Fin system, ally, th the e degr image adedegradati d image is obtained. on result, in the direction along the track In thand e coaxi theal p vertical lane op track, tical sy is the stem same. , the im Then, age de the gr system adation r ﬁlter esultemplate t, in the d (18) irect can ion be expr along the tra essed as: ck and the vertical track, is the same. Then, the system filter template (18) can be expressed as: 2 2 2 2 2 2 MTF(u) = exp 2p d u exp 2p d u (23) is x is y 22 2 2 2 2 MTFυπ =− exp 2dd υ ⋅exp−2π υ () () ( ) (23) is x is y where the error of velocity, caused by random vibration, is set to 0.00076°/s. Then, the frequency domain-filtered template images, at attitude angles of 0°, 30°/−30°, and 50°/−50°, are simulated, as shown in Figure 10. It can be seen that the image filtering has the same effect when the attitude angle is symmetrical about the position of the satellite nadir. Photonics 2022, 9, 455 11 of 14 where the error of velocity, caused by random vibration, is set to 0.00076 /s. Then, the fre- quency domain-ﬁltered template images, at attitude angles of 0 , 30 /30 , and 50 /50 , Photonics 2022, 9, x FOR PEER REVIEW 12 of 15 are simulated, as shown in Figure 10. It can be seen that the image ﬁltering has the same effect when the attitude angle is symmetrical about the position of the satellite nadir. (a) (b) (c) Figure 10. Image filter template: (a) attitude angle 0°; ( b) attitude angle 30°/ −30°; and ( c) attitude Figure 10. Image ﬁlter template: (a) attitude angle 0 ; (b) attitude angle 30 /30 ; and (c) attitude angle 50°/−50°. angle 50 /50 . In order to intuitively describe the actual image degradation phenomenon, the high- In order to intuitively describe the actual image degradation phenomenon, the high- resolution ground target image, with a size of 2200 pixel × 400 pixel, is used as the original resolution ground target image, with a size of 2200 pixel 400 pixel, is used as the original imag image. e. The sub-fr The sub-frame ame imag images esat atattitude attitude a angles ngles ofof − 50 50°, ,−30 30°, 0°, 30 , 0 , 30 °, a , and nd 50 50° are assumed are assumed tto o be t be the he st stitching itching unit units s f for or wid wide-format e-format im imaging. aging. After After be being ing f ﬁlter iltered ed by t by the hetemplate template shown in Figure 10, the images interfered by random vibration at different attitude angles shown in Figure 10, the images interfered by random vibration at different attitude angles can be obtained, as shown in Figure 11. The ﬁrst line is the original image, the second line can be obtained, as shown in Figure 11. The first line is the original image, the second line is the distorted output image on the camera focal plane, and the third line is the degraded is the distorted output image on the camera focal plane, and the third line is the degraded image caused by the satellite vibration. image caused by the satellite vibration. From the ﬁrst line of Figure 11, it can be seen that the imaging range of the satellite nadir is the smallest. As the angle between the optical axis of the payload and the position of the satellite nadir increases, the imaging range of the area–array camera on the ground becomes wider. From the second line of Figure 11, it can be seen that the degree of image distortion far from the satellite nadir is greater, while the size and scale of the target image get bigger, and the disturbance factor has more inﬂuence on the image clarity. From the third line of Figure 11, it can be seen that satellite vibration seriously affects the quality of remote sensing images. Image sharpness evaluation indicators, SSIM (structural similarity) and MSE (mean square error), are usually used to evaluate the degree of image degradation. The higher the value of MSE is, the greater the degree of distortion is. The value range of SSIM is [0, 1]. The larger the value of SSIM is, the smaller the image distortion is. The data of SSIM and MSE, at different attitude angles, are listed in Table 2. When the attitude angle is 0 , the SSIM value is 0.3136, and the MSE value is 76.7977. With the attitude angle increasing to 30 and 50 , SSIM decreases, and the MSE increases, which indicates that, when the error of velocity is the same, the image sharpness decreases with the increasing of the deviation angle between the payload optical axis and the satellite nadir position. Table 2. The results of image sharpness evaluation. Attitude Angle SSIM MSE 50 0.2125 89.3903 30 0.2958 79.9711 0 0.3136 76.7977 30 0.2509 84.1601 50 0.2041 94.5861 Figure 11. The simulation process of remote sensing image degradation. Photonics 2022, 9, x FOR PEER REVIEW 12 of 15 (a) (b) (c) Figure 10. Image filter template: (a) attitude angle 0°; (b) attitude angle 30°/−30°; and (c) attitude angle 50°/−50°. In order to intuitively describe the actual image degradation phenomenon, the high- resolution ground target image, with a size of 2200 pixel × 400 pixel, is used as the original image. The sub-frame images at attitude angles of −50°, −30°, 0°, 30°, and 50° are assumed to be the stitching units for wide-format imaging. After being filtered by the template shown in Figure 10, the images interfered by random vibration at different attitude angles can be obtained, as shown in Figure 11. The first line is the original image, the second line Photonics 2022, 9, 455 12 of 14 is the distorted output image on the camera focal plane, and the third line is the degraded image caused by the satellite vibration. Figure 11. The simulation process of remote sensing image degradation. Figure 11. The simulation process of remote sensing image degradation. 4. Discussion Different from traditional push-broom imaging modes, the degraded image of dy- namic scanning mirror imaging, caused by satellite vibration, is analyzed in this paper. In this dynamic imaging model, the image quality will be inﬂuenced by the linkage between the satellite and the payload. Therefore, the vibration mechanism of the frame-by-frame imaging model is more complex. In order to simplify the complex relationship of the model, the disturbance of the satellite is displayed by the random vibration factors, and the disturbance of the payload is displayed by the periodically changing attitude angle factors. Then, the frequency domain-ﬁltered template of random vibration is constructed. The following results can be obtained by simulating the dynamic imaging process with the orbital height 500km and the scanning velocity 10 /s of the mirrors in front of the space camera: 1. In the process of dynamic imaging, the ground image information becomes larger, and the distortion degree of the output image on the focal plane grows larger with the increasing angle between the optical axis of the camera and the position of the satellite nadir. 2. When the attitude angle is a certain value, the MTF drops rapidly with the increasing error of velocity. 3. The camera exposure time is determined by the periodically changing attitude angle. When the scanning mirrors scan at the satellite nadir, the camera exposure time is the shortest. With the increasing angle between the optical axis and the satellite nadir, the camera exposure time is extended. When the error of velocity is a certain value, the MTF drops with the axis of symmetry of the satellite nadir. Photonics 2022, 9, 455 13 of 14 4. In a group of wide, single-frame output images, the error of velocity is caused by the random vibration of the payload mechanism, resulting in an error image shift on the focal plane of the camera. In a set of single-frame images that constitute wide-format imaging, the distortion effect and degradation degree of each single-frame image are different. The single-frame image far away from the satellite nadir has the largest distortion, and the image degradation phenomenon is obvious. 5. Conclusions To summarize, the wide image output by the CMOS camera with a scanning mirror is simulated in this paper. The degradation law of high-resolution large-width imaging, interfered with by satellite random vibrations, is analyzed. The simulation result shows that the camera exposure time is extended when the payload attitude angle is increasing during dynamic imaging. Further, the degree of image distortion on the camera focal plane becomes larger at locations far away from the satellite nadir. The error of velocity, caused by random vibration, increases the shift amount of an image on the camera focal plane, which directly reduces the image quality. Therefore, the ﬁeld of view (FOV) should be reasonably set, and the damping mechanism should be installed to obtain a high resolution large-width image. Author Contributions: Conceptualization, L.J.; methodology, L.J.; software, L.J.; validation, Z.Y. (Ziqi Yu); formal analysis: L.J. and Z.Y. (Zhihai Yao); investigation, Z.Y. (Zhihai Yao); resources, L.J.; data curation, Z.Y. (Ziqi Yu); writing—original draft preparation, Z.Y. (Ziqi Yu); writing—review and editing, L.J., Z.Y. (Ziqi Yu) and K.L.; visualization, Z.Y. (Ziqi Yu). All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the National Natural Science Foundation of China (NSFC), grant number 62101071 and 62171430. This research was funded by the Natural Science Foundation of Jilin Province, grant number 20210101099JC and 20210101186JC. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Not applicable. Acknowledgments: The authors would like to thank National Natural Science Foundation of China. Conﬂicts of Interest: The authors declare no conﬂict of interest. References 1. Zhu, Z.; Zhou, Y.; Seto, K.C.; Stokes, E.C.; Deng, C.; Pickett, S.T.A.; Taubenböck, H. Understanding an urbanizing planet: Strategic directions for remote sensing. Remote Sens. Environ. 2019, 228, 164–182. [CrossRef] 2. Tomás, R.; Li, Z. Earth Observations for Geohazards: Present and Future Challenges. Remote Sens. 2017, 9, 194. [CrossRef] 3. Wang, Y.; Sun, G.; Guo, S. Target Detection Method for Low-Resolution Remote Sensing Image Based on ESRGAN and ReDet. Photonics 2021, 8, 431. [CrossRef] 4. Xie, Y.; Han, J.; Gu, X.; Liu, Q. On-orbit radiometric calibration for a space-borne multi-camera mosaic imaging sensor. Remote Sens. 2017, 9, 1248. [CrossRef] 5. Chen, S.; Liu, J.; Huang, W.; Chen, R. Wide Swath Stereo Mapping from Gaofen-1 Wide-Field-View (WFV) Images Using Calibration. Sensors 2018, 18, 739. [CrossRef] 6. Yang, X.; Wang, J.; Jin, G. Calculation of Line Frequency for Rotational Pendulum Imaging of the TDI Camera Vertical Track; Changchun Zhongbang Jinghua Intellectual Property Agency Co., Ltd.: Changchun, China, 2019. 7. Du, B.; Li, S.; She, Y.; Li, W.; Wang, H. Area targets observation mission planning of agile satellite considering the drift angle constraint. J. Astron. Telesc. Instrum. Syst. 2018, 4, 047002. [CrossRef] 8. Wang, S.; Yang, X.; Feng, R.; Gao, S.; Han, J. Dynamic Disturbance Analysis of Whiskbroom Area Array Imaging of Aerospace Optical Camera. IEEE Access 2021, 9, 137099–137106. [CrossRef] 9. Zaman, I.U.; Boyraz, O. Impact of receiver architecture on small satellite optical link in the presence of pointing jitter. Appl. Opt. 2020, 59, 10177–10184. [CrossRef] 10. Hadar, O.; Fisher, M.; Kopeika, N.S. Image resolution limits resulting from mechanical vibrations. Part III: Numerical calculation of modulation transfer function. Opt. Eng. 1992, 31, 581–589. [CrossRef] Photonics 2022, 9, 455 14 of 14 11. Hadar, O.; Dror, I.; Kopeika, N.S. Image resolution limits resulting from mechanical vibrations. Part IV: Real-time numerical calculation of optical transfer functions and experimental veriﬁcation. Opt. Eng. 1994, 33, 566–578. [CrossRef] 12. Stern, A.; Kopeika, N.S. Analytical method to calculate optical transfer functions for image motion using moments and its implementation in image restoration. Proc. SPIE Int. Soc. Opt. Eng. 1996, 2827, 190–202. [CrossRef] 13. Du, Y.; Ding, Y.; Xu, Y.; Sun, C. Dynamic modulation transfer function analysis of images blurred by sinusoidal vibration. J. Opt. Soc. Korea 2016, 20, 762–769. [CrossRef] 14. Deng, C.; Mu, D.; An, Y.; Yan, Y.; Li, Z. Reduction of satellite ﬂywheel microvibration using rubber shock absorbers. J. Vibroeng. 2017, 19, 1465–1478. [CrossRef] 15. Li, J.; Liu, Z.; Si, L. Suppressing the image smear of the vibration modulation transfer function for remote-sensing optical cameras. Appl. Opt. 2017, 56, 1616–1624. [CrossRef] 16. Song, M.; Qu, H.; Zhang, G.; Jin, G. Design of aerospace camera circular scanning imaging model. Infrared Laser Eng. 2018, 47, 195–202. [CrossRef] 17. Zhong, L.; Wu, X.; Wang, P. Construction and Veriﬁcation of Rigorous Imaging Model of Spaceborne Linear Array Vertical Orbit Ring Scanning Sensor. In Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering, Xiamen, China, 6–8 November 2020; pp. 94–98. 18. Liu, Z.; Gao, L.; Wang, J.; Yu, T. Research on the inﬂuence of swing mirror of infrared imaging system with image-space scanning. Infrared Phys. Technol. 2018, 92, 459–465. [CrossRef] 19. Chang, T.; Wang, Q.; Zhang, L.; Hao, N.; Dai, W. Battleﬁeld dynamic scanning and staring imaging system based on fast steering mirror. J. Syst. Eng. Electron. 2019, 30, 37–56. [CrossRef] 20. Ng, S.H.; Allan, B.; Ierodiaconou, D.; Anand, V.; Babanin, A.; Juodkazis, S. Drone polariscopy—Towards remote sensing applications. Photonics 2021, 11, 46. [CrossRef] 21. Yue-Lin, L.; Wang, Y.; Fu-Qi, S.; Xue, H.; Chen, J.; Jiang, Y.; Liu, X.; Chen, Z. Design of scanning mirror control system for satellite-borne DOAS spectrometer. Opt. Precis. Eng. 2019, 27, 630–636. [CrossRef] 22. Jiang, P.; Zhou, P. The Optimization of a Convex Aspheric Lightweight SiC Mirror and Its Optical Metrology. Photonics 2022, 9, 210. [CrossRef] 23. Wei, Q.; Chao, X. Attitude maneuver planning of agile satellites for time delay integration imaging. J. Guid. Control Dyn. 2019, 43, 46–59. [CrossRef]

Photonics – Multidisciplinary Digital Publishing Institute

**Published: ** Jun 28, 2022

**Keywords: **remote sensing; satellite vibration; frequency domain analysis; image distortion

Loading...

You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!

Read and print from thousands of top scholarly journals.

System error. Please try again!

Already have an account? Log in

Bookmark this article. You can see your Bookmarks on your DeepDyve Library.

To save an article, **log in** first, or **sign up** for a DeepDyve account if you don’t already have one.

Copy and paste the desired citation format or use the link below to download a file formatted for EndNote

Access the full text.

Sign up today, get DeepDyve free for 14 days.

All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.