Access the full text.
Sign up today, get DeepDyve free for 14 days.
Article Vision-Based Structural FE Model Updating Using Genetic Algorithm Gun Park, Ki-Nam Hong and Hyungchul Yoon * School of Civil Engineering, Chungbuk National University, Chungdae-ro 1, Seowon-Gu, Cheongju, Chungbuk 28644, Korea; email@example.com (G.P.); firstname.lastname@example.org (K.-N.H.) * Correspondence: email@example.com; Tel.: +82-432612404 Abstract: Structural members can be damaged from earthquakes or deterioration. The finite element (FE) model of a structure should be updated to reflect the damage conditions. If the stiffness reduc- tion is ignored, the analysis results will be unreliable. Conventional FE model updating techniques measure the structure response with accelerometers to update the FE model. However, accelerom- eters can measure the response only where the sensor is installed. This paper introduces a new com- puter-vision based method for structural FE model updating using genetic algorithm. The system measures the displacement of the structure using seven different object tracking algorithms, and optimizes the structural parameters using genetic algorithm. To validate the performance, a lab- scale test with a three-story building was conducted. The displacement of each story of the building was measured before and after reducing the stiffness of one column. Genetic algorithm automati- cally optimized the non-damaged state of the FE model to the damaged state. The proposed method successfully updated the FE model to the damaged state. The proposed method is expected to re- duce the time and cost of FE model updating. Keywords: structural health monitoring; computer vision; model updating; genetic algorithm Citation: Park, G.; Hong, K.-N.; 1. Introduction Yoon, H. Vision-Based Structural FE Recent earthquakes around the world have shown how earthquake disasters can Model Updating Using Genetic wreak havoc in society. For example, an earthquake in Taiwan in 1999 resulted in over Algorithm. Appl. Sci. 2021, 11, 1622. 10,000 casualties and economic losses of over USD 10 billion. The magnitude 7 earthquake https://doi.org/10.3390/app11041622 in Haiti in 2010 caused tens of thousands of casualties, and the collapse of social infra- Received: 15 January 2021 structure resulted in huge economic losses. The earthquake and tsunami that occurred in Accepted: 8 February 2021 Japan in 2011 not only killed more than 10,000 people, they also caused a major radioac- Published: 10 February 2021 tive leak owing to the explosion of the Daiichi nuclear power plant. The earthquake in Mexico in 2017 caused more than 1000 casualties, and a number of skyscrapers in Mexico Publisher’s Note: MDPI stays neu- City, the capital city of Mexico, were severely damaged . tral with regard to jurisdictional Structures can be damaged through seismic loadings, and the conditions of such claims in published maps and insti- structures need to be determined to ensure their safety. The current practice for inspecting tutional affiliations. damaged structures requires engineers to directly evaluate the damage by viewing the structure by sight. However, this assessment can be subjective and might expose the en- gineers to risk. In recent years, vibration-based Structural Health Monitoring (SHM), a method for monitoring the health of such structures by measuring the response of the Copyright: © 2021 by the authors. Li- structures, has recently entered the spotlight. SHM aims to ensure the safety of civil infra- censee MDPI, Basel, Switzerland. structure by measuring its dynamic response. This article is an open access article The traditional SHM system measures the structural response by installing wired distributed under the terms and con- sensors such as accelerometers and displacement transducers, e.g., the Alamosa Canyon ditions of the Creative Commons At- Bridge, I-10 Bridge, Hakucho Bridge in Japan, Bill Emerson Memorial Bridge in Missouri, tribution (CC BY) license (http://crea- tivecommons.org/licenses/by/4.0/). and the Tsing Ma Bridge in Hong Kong [2–6]. However, the wired SHM system cannot Appl. Sci. 2021, 11, 1622. https://doi.org/10.3390/app11041622 www.mdpi.com/journal/applsci Appl. Sci. 2021, 11, 1622 2 of 16 be easily deployed to existing structures, and is costly. To overcome these limitations, wireless sensor technology has been introduced. Westermo and Thompson presented a sensor that can evaluate the condition of the structure, and Pines and Lovell discussed a sensor and wireless communication technology that remotely monitors the health of large civil structures. Krantz et al. developed a micro-sensor that can retrieve data from an em- bedded strain gauge, and Oshima et al. proposed a monitoring system using a mobile phone . Although enhancements in wireless sensor technology could reduce the cost of SHM systems, these methods still require manual installation of the sensors. Computer vision technology has recently been applied to SHM, overcoming the limitations of wired and wireless systems. Jiang et al.  concluded that close-range photogrammetry technol- ogy is a powerful measurement method that can provide unique solutions for a variety of bridge engineering applications. Kwiatkowski et al.  compared and evaluated the use- fulness of various measurement techniques (terrestrial laser scanning (TLS), tachymetry, photogrammetry) applied to establish the behavior of suspension bridges under various loads. Gawronek et al.  measures the vertical displacement of railroad bridges using terrestrial laser scanning (TLS) technology, and confirmed that this technology can pro- vide reasonable results for quick assessment of railway infrastructure. Kwiatkowski et al.  confirmed that it is possible to obtain a 3D model of a structure with satisfactory ac- curacy through TLS and photogrammetry. Porras-Amores et al.  presented the most efficient method for generating 3D models of underground structures using TLS tech- niques. Shih and Sung  conducted a study to extract the displacement of the structure and characterize the vibration mode of the cantilever beam using digital image correlation technology. Feng et al.  and Yoon et al.  confirmed the reliability of a vision-based system for system identification by comparing vision-based and wired sensor systems. Furthermore, a method used to measure the structural displacement from an unmanned aerial vehicle (UAV) was proposed by Yoon et al. . In addition, Lim and Yoon  presented a motion sensing technique using vision-based motion sensing to acquire the location information of the real-time dynamic load for pedestrian suspension bridges. Alt- hough these vision-based methods have made significant improvements in measuring the displacement or conducting system identification, a study for applying computer vision technology to FE model updating has yet to be conducted. This study aims to develop a framework for computer-vision based structural finite element (FE) model updating. Various computer vision algorithms, including Kanade– Lucas–Tomasi (KLT), multiple instance learning (MIL), kernelized correlation filter (KCF), and minimum output sum of squared error (MOSSE) are applied to measure the seismic loads and the corresponding response (displacement) of the structures. Genetic algorithm was applied to optimize the FE model of the structure into an as-is damaged model, and to predict the stiffness reductions for all members. To validate the proposed system, a lab- scale test using a three-story building model was conducted. In Section 4.4, the validation results showed that the proposed method can update the FE model to an as-is damage state, within an accuracy of approximately 80%. The proposed method is expected to re- duce the time and cost of FE model updating . 2. Background In this study, vision-based structural model updating was performed based on the following three topics. Tracking algorithm. FE model updating. Genetic algorithm. In this chapter, brief concepts of various tracking algorithms, finite element model updating, and genetic algorithms are described. Appl. Sci. 2021, 11, 1622 3 of 16 2.1. Tracking Algorithm The KLT tracker was introduced by Lucas and Kanade and extended by Tomasi and Kanade. The KLT tracker tracks an object in two simple steps. First, it finds the traceable feature points in the first frame, and then uses the calculated displacement to track the features detected in the next frames. The feature points of an object are tracked using an optical flow tracker . The channel and spatial reliability (CSRT) tracker was proposed by LuNežič et al. , and improves the discriminative correlation filter (DCF) tracker by introducing spatial and channel reliability. The spatial reliability map was used to resize the filters to find optimal filter sizes by taking advantage of non-rectangular targets. The KCF tracker proposed by Henriques et al.  is a tracker that extends the MOSSE tracker. The KCF tracker improves the tracking accuracy by extracting the histogram of gradient (HoG) function, but requires more data. Therefore, the data are calculated in the Fourier domain to speed up the data processing procedure. The MedianFlow tracker proposed by Kalal et al.  tracks the point of each object forward and backward, and compares the results of the trajectory. If the object is tracked correctly, the trajectory forward and back- ward will yield the same results; otherwise, a tracking error will indicate the opposite value. The error was used to track points inside the bounding box and classify the inliers and outliers. The MIL tracker proposed by Babenko et al.  conducted object tracking similar to the online boosting approach. The MIL used a set of image patches (called bags) instead of a single sample for training. For classification, the MIL uses a multi-instance learning approach that considers object bags by grouping similar samples into such bags. A bag containing more than one positive example was called a positive bag, which is oth- erwise called a negative bag. This technique not only prevents the MIL tracker from losing important information, it also prevents mislabeling problems. The MOSSE tracker pro- posed by Bolme et al.  achieved high efficiency by calculating the correlation in the time domain. The correlation filter improved the efficiency for scaling, rotation, defor- mation, and occlusion problems from the traditional approach. The MOSSE is more effi- cient than the other correlation filter-based trackers because the target does not need to be at the center of the image initially. The TLD tracker proposed by Kalal et al.  was a technique for long-term tracking and consisted of three parts: tracker, learner, and detec- tor. The tracker aimed to track the objects in successive frames, and the learner used the P-expert and N-expert to estimate false positives and false alarms to update the detector. In addition, the detector finds targets based on the appearance model, provides an output to the learner, and modifies the tracker if necessary. The goal of the TLD is to improve the robustness of the tracking by disabling the online learning when an object is out of the frame or completely obscured by another object, preventing learning through misinfor- mation. In addition, the detection component allows the TLD to redetect the object if it later appears in the video again. 2.2. FE Model Updating The finite element method (FEM) is the most widely used method for civil engineer- ing and other applications. A finite element method is characterized by a variation formu- lation, a discretization strategy, one or more solution algorithms, and post-processing pro- cedures [25-27]. In this study, model updating was applied that the concept of reverse engineering to the finite element method. FE model updating is the process of ensuring that finite element analysis results in models that better reflect the measured data than the initial models. It is part of verification and validation of numerical models. In order to facilitate model updating, a method to find the properties of the member must be pre- sented. Genetic algorithm is one of the most popular and powerful optimization algo- rithms, and thus is being applied to various applications including FE model updating. Appl. Sci. 2021, 11, 1622 4 of 16 2.3. Genetic Algorithm The genetic algorithm is based on the biogenetics of the natural world, and is a par- allel and global search algorithm, based on Darwin’s theory of survival of the fittest. Ge- netic algorithms produce better and better solutions by expressing the possible solutions to the problem to be solved in a data structure of a fixed form and then gradually trans- forming them. Here, the data structure representing the solutions is genes, and the process of making better and better solutions by modifying them can be expressed as evolution. Genetic algorithms are more of an approach to solving a problem than an algorithm to solve a specific problem, and can be applied to any problem that can be expressed in a format that can be used in genetic algorithms. In general, when a problem is too complex to be computed, it can be approached through a genetic algorithm as a way to obtain an answer close to the optimal solution, even though the actual optimal solution cannot be obtained. In this case, it does not perform as well as an algorithm optimized to solve the problem, but it can provide an acceptable level of solution in most cases. 3. System Development Figure 1 shows an overview of the proposed FE model updating method described in this study. The FE model updating method can be composed of two main components: (i) vision-based displacement measurements and (ii) FE model updating, and each chapter explains the main components in detail. Figure 1. Overview of the vision-based finite element (FE) model updating. 3.1. Vision-Based Dis placement Measurement This process demanded four consecutive steps, shown in Figure 1: (i) region of inter- est (ROI) selection, (ii) feature detection, (iii) detected feature tracking, and (iv) tracking outlier removal. Here, the ROI represents the location of the object that needs to be tracked. Each ROI contains several functions that are typically used for displacement measurement purposes. The ROI is selected manually by drawing a box in the first frame of the video. Object tracking is a computer vision technique used to track the motion of an object in an image captured by a general camera through real-time or post-processing . Nu- merous object tracking algorithms have been introduced to track an object including KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD. Among these tracking algorithms, a few methods have been applied for structural health monitoring purposes; CSRT has been applied to human motion tracking , and KLT has been applied to track the dynamic behavior of various structures . This study followed the vision-based displacement measurement system proposed by Yoon et. al. , expanded the system to be applied for FE model updating, instead of system identification. In addition, this study applied six additional tracking algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD) in addition to the KLT tracker, Appl. Sci. 2021, 11, 1622 5 of 16 which was used in the original system. To measure the displacement of a structure using tracking algorithms, the first step is to calibrate the camera. There are two main reasons for camera calibration: (1) the removal of the radial distortion and tangential distortion, and (2) to compensate the camera ego motion. Camera lenses have tended to use very wide-angle lenses. However, these lenses intentionally introduce radial distortion to in- crease the field of view. In addition, the camera can move during the measurement, which can result in significant errors for displacement measurements . The extrinsic camera matrix, which contains the translation and rotation information of the camera, needs to be estimated to compensate for the ego-motion of the camera. Therefore, the camera must be calibrated to minimize these distortions and achieve accurate displacement measure- ments. Once the camera calibration is conducted, the video is analyzed frame by frame to determine the dynamic response of the structure. This process can be summarized in four steps : (1) selecting a region of interest (ROI), (2) detecting the features, (3) tracking the detected features, and (4) removing tracking outliers, as shown in Figure 2. Here, the re- gion of interest (ROI) indicates the object that needs to be tracked and manually selected in the first frame of the video. Figure 2. Overview of the vision-based displacement technique . 3.2. Structural FE Model Updating Using Genetic Algorithm An FE analysis has been considered a powerful tool for decades, and is capable of simulating the behavior of the structures. However, creating accurate FE models is not an easy task. Various modeling techniques and guidelines are used to make the appropriate selection of the element type (beam, plate, or solid), degrees of freedom, and analysis methods. These modeling techniques depend on the skills and experience of the engineer, and the purpose of the model (e.g., a static FE analysis and a dynamic FE analysis require different FE models for the same accuracy). The initial FE model, created on the basis of the techniques described above, can accurately predict the initial (undamaged) state of the structure. However, as the structure ages, the FE model created in the design stage does not reflect the characteristics of the as-is structure. Therefore, the FE model updating tech- nique is used to reflect the as-is (damaged) state of the structure. The FE model updating procedure involves three steps: (1) measuring the response as the reference data, (2) se- lecting the parameters to update, and (3) tuning the model. Prior experience has shown that measuring the accurate response and selecting the model parameters are the key com- ponents for successfully updating the FE model . The genetic algorithm is an optimization algorithm rooted in the principles of biolog- ical evolution, applying the natural law that individuals with superior traits produce su- perior offspring. The genetic algorithm is quite different from classical optimization algo- rithms. Traditional optimization methods use derivatives and search for the objective function of all points in a limited space one at a time, or select an arbitrary point to start the search. However, the genetic algorithm conducts a directional search and a probability Appl. Sci. 2021, 11, 1622 6 of 16 search without using the concept of derivatives. This leads to an advantage of solving optimization problems, including continuous–discontinuous mixing, discontinuity, and non-convex regions. The genetic algorithm optimizes the problem by taking a population of individuals consisting of a combination of binary numbers and performing three pro- cesses: selection, crossover, and mutation as shown in figure 3 . Figure 3. Genetic selection method using genetic algorithm. In this study, the natural frequency of the structure was used as the objective func- tion, which can best describe the dynamic characteristics of the structure. Decision varia- bles were considered as the cross-sectional area and the thickness of the structure mem- bers, and the objective function, decision variables, and constraint set for the discrete op- timization problem are shown in Equations (1)–(3). (1) 𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 ∶ 𝑊 = ∑ 𝑡𝑒𝑐 𝑡𝑜 ∶ 0< 𝐴 ≤ 𝐴 (2) 0< 𝑡 ≤ 𝑡 (3) In Equations (1)–(3), 𝜔 is the ith natural frequency of the structure of the FE model, and 𝜔 is the ith natural frequency of the structure obtained from the experiment. In ad- dition, 𝐴 is the cross-sectional area of the undamaged column, 𝐴 is the cross-sec- tional area of the damaged column, 𝑡 is the thickness of the damaged column, and 𝑡 is the thickness of the undamaged column. Crossover and mutation are as important items as the objective function in the ge- netic algorithm (GA). Crossover specifies how the genetic algorithm combines two par- ents to form a crossover son for the next generation and mutation specifies how the genetic algorithm randomly alters individuals in a population to create a mutant son. Mutation provides genetic diversity and enables the genetic algorithm to search a broader space. In this study, because the wall stiffness was set as an input parameter, individuals of each generation were set for six. The number of individuals constituting the first genera- tion was 200 for each of the wall stiffness values, and after each individual from generation “0” was simulated, the objective function W for the natural frequency of the structure was evaluated. The best individuals of this generation “0” were those with the lowest value of the objective function 𝑊 , and the next generations (Generation 1 and subsequent gener- ation) were generated by crossover and mutation as described below. In order to compose the parents of the new generation, 5% of individuals with the lowest objective function W value of the previous generation were selected, and 80% of individuals were generated by crossover of the selected parents for a new generation. At 𝑆𝑢𝑏𝑗 Appl. Sci. 2021, 11, 1622 7 of 16 this time, the crossover was made by the following method. For example, if p1 and p2 are parents, then p1 and p2 can be expressed as follows: p1 = [433,111; 353,778; 380,222; 327,333; 433,111; 10000] p2 = [504,749; 36,379; 417,040; 10,000; 142,329; 596,282] and the binary vector is [1 1 0 1 0 0], the function returns the following son: Son = [433,111; 353,778; 417,040; 327,333; 142,329; 596282] Excluding the 5% with the lowest objective function W value and the 80% obtained through crossover, the remaining 5% were obtained through mutation. The values of in- dividuals generated by mutations should not excessively exceed the range of individual values for each generation. Thus, the mutation adds a random number taken from a Gaussian distribution with mean 0 to each entry of the parent vector. The reason for gen- erating individuals by mutation was to find new individuals in areas that have not been previously applied. This procedure was individually repeated for the wall stiffness for several generations until the objective function W reached a value within error tolerance or the number of generations reached a set value [30–32]. Figure 4 shows how the fitness of the objective function changes through the genetic algorithm used in this study. Ini- tially, it can be seen that the objective function rapidly decreases, but it can be seen that the value changes little after about 150 generations. This means that after the 150th gener- ation, the individual constituting the wall stiffness has found an appropriate range to sat- isfy the objective function. Figure 4. Fitness values at each generation. 4. Validation Test In order to verify the proposed method in this paper, shaking table test of the lab- scale model and model updating of the FE model were performed. The displacement of the structure was measured using a vision-based technique to obtain the result of the shaking table test of lab-scale model, and the natural frequency of the structure was cal- culated based on the test result. Model updating was performed by applying the natural frequency of the structure calculated through the test as the objective function of the ge- netic algorithm. The validation of the proposed method was confirmed by comparing the estimated stiffness of the wall using FE model updating with the theoretical stiffness of the wall. 4.1. Test Setup To validate whether the proposed method can apply FE model updating accurately, a lab-scale validation test was conducted. A three-story building model with equally dis- tributed masses connected by aluminum columns was chosen as the validation structure. The total height of the structure is 920 mm and the width is 240 mm. In order to generate large displacement of the structure when excitation load is applied, the thickness of the wall is 1.2mm. Figure 5 shows an overview of the lab-scale validation test. To validate the Appl. Sci. 2021, 11, 1622 8 of 16 performance of the FE model updating, this study conducted an experiment twice: (1) using an undamaged structure, and (2) by changing one of the aluminum columns to a damaged column. The column between the first and second floors was assumed to be damaged by drilling five holes into the column. The model was attached to a unidirec- tional shaking table with a maximum displacement of 100 mm. Band-limited white noise (BLWN) was adopted as an input motion to provide energy within a wide range of fre- quencies. Consumer-grade cameras are used to record the displacement output of the model and the shaking table. These cameras are oriented perpendicular to the motion axis as shown in Figure 5. A smartphone camera (Samsung GALAXY S8) was used to record the video with a frame rate of 60 fps and a resolution of 1080p (1920 pixels × 1080 pixels). In order to improve the accuracy in detecting features necessary for displacement meas- urements, a fixed white background is used behind the experiment setup. The recorded video was converted into grayscale to improve the accuracy of detecting the features re- quired for displacement measurements. Figure 5. Setup for shaking table test. 4.2. FE Model The FE model for the validation test is as shown in Figure 6. A simple frame model was used [33–36] to validate the proposed vision-based FE model updating method. The element used in the FE model is the beam type, and the material properties are assumed to be elastic, and the basement and slab were applied steel (Unite Weight: 7.85 t/m3, E: 210 GPa, Poisson’s Ratio: 0.3), and the wall was applied stainless steel (Unite Weight: 7.90 t/m3, E: 201 GPa, Poisson’s Ratio: 0.3). The boundary condition of the FE model was as- sumed to be fixed at the lower part of the structure, and Cholesky factorization was used to calculate the natural frequency of the structure. Appl. Sci. 2021, 11, 1622 9 of 16 (a) Specification (Units: mm) (b) FE Model Figure 6. FE model used for model updating. In finite element analysis, the size of the element is an important factor affecting the results. Íñiguez-Macedo et al.  analyzed the sensitivity of the element size applied to the finite element analysis to the analysis in a study proposing a methodology for auto- matically determining the most appropriate Ci constants for modeling the behavior of a group of elastomers. Therefore, this study also confirmed the effect of element size on the calculation of the natural frequency of the structure. The size of the element was calculated by applying the sizes of 1/2, 1/4, 1/8, and 1/16 based on the FE model shown in Figure 6 and the results are as shown in Table 1. Table 1. Mesh sensitivity analysis result. Original Size ½ Mesh Size ¼ Mesh Size 1/8 Mesh Size 1/16 Mesh Mesh Value Value Accuracy Value Accuracy Value Accuracy Value Accuracy Mode (Hz) (Hz) (%) (Hz) (%) (Hz) (%) (Hz) (%) 1st 1.543 1.544 99.884 1.544 99.881 1.544 99.881 1.544 99.881 2nd 4.532 4.574 99.092 4.575 99.067 4.575 99.066 4.575 99.066 3rd 6.724 6.785 99.109 6.788 99.058 6.789 99.056 6.789 99.056 As a result of the sensitivity analysis, it was found that the FE model applied in this study did not significantly affect the natural frequency of the structure even if the struc- tural analysis was performed while reducing the size of the element. Therefore, it was confirmed that the FE model shown in Figure 6 has a sufficient element size to perform model updating. 4.3. Object Tracking Results The shaking table test was conducted based on the described test setup, and recorded videos were analyzed using seven different algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD). The tracking video with the region of interest (ROI) area is shown in Figure 7a. As a result, KLT (Figure 7b), KCF (Figure 7d), MIL (Figure 7f), and MOSSE (Figure 7g) successfully tracked the object during the shaking table test. However, the Appl. Sci. 2021, 11, 1622 10 of 16 CSRT (Figure 7c), MedianFlow (Figure 7e), and TLD (Figure 7h) algorithms showed that the ROI deviated from the initially designated object. (a) (b) (c) (d) (e) (f) (g) (h) Figure 7. Vision-based displacement measurement: (a) region of interest (ROI) at the initial frame, (b–h) ROI at the final frame using different tracking algorithms. Figure 8 shows the time–displacement curve of the KLT algorithm that succeeded in tracking (Figure 8a) and the TLD algorithm in which the ROI deviated from the initially designated object (Figure 8b). The time–displacement curve of the CSRT algorithm in which the ROI deviated from the initial designated object can confirm that the displace- ment was measured abnormally. Appl. Sci. 2021, 11, 1622 11 of 16 (a) KLT (b) TLD Figure 8. Time-displacement curve using different object tracking algorithms. The frequency response function (FRF) was calculated, as shown in Figure 9, using the displacement measured by the four object tracking algorithms that successfully tracked the object. Based on the FRF calculated using the four object tracking algorithms, the dynamic characteristics of the structure were confirmed, the results of which are shown in Table 2. The dynamic characteristics of the structure shown in Table 2 are used as the fitness function in the FE model updating, and the details are described in the next section. (a) KLT (b) MIL (c) KCF (d) MOSSE Figure 9. Comparison of the transfer function using different object tracking algorithms. Table 2. The natural frequency for different object tracking algorithms. Natural Frequency (Hz) Mode KLT MIL KCF MOSSE 1st 1.543 1.550 1.547 1.541 2nd 4.539 4.529 4.531 4.539 3rd 6.726 6.722 6.724 6.726 4.4. FE Model Updating In this section, the FE model updating technique is applied to a three-story model to which BLWN input motion is applied. The natural frequency of the structure analyzed by the four object tracking algorithms based on a recording using a smartphone (Samsung GALAXY S8) was adopted for the fitness function in the FE model updating. The FE Appl. Sci. 2021, 11, 1622 12 of 16 model applied to the model updating is shown in Figure 9a, and the stiffness of the struc- ture at the design stage and the stiffness of the damaged structure are shown in Figure 10 (b) and Figure 10 (c), respectively. The FE model was created based on the structure used in the lab-scale test, and it was composed of 10 nodes and 12 frame elements under a fixed boundary condition. Model updating was conducted by applying the results of Section 3.2 and the FE model described above. The stiffness of each floor of the structure was estimated through model updating, and the stiffness of each floor of the structure estimated for each algo- rithm is shown in Table 1 and Figure 9d–g. As a result of the FE model updating, the four object tracking algorithms were confirmed to be able to search for the damaged location of the structure. This can be confirmed through the stiffness of the second floor of the structure, which was estimated to be relatively small compared to the stiffness of the other floors. For the estimated stiffness of the undamaged floor, it was confirmed that the four object tracking algorithms showed an error of approximately 5–7%. In addition, for the estimated stiffness of the damaged floor, it was confirmed that the four object tracking algorithms showed an error of approximately 18–21%. It was confirmed that the damaged floor of the structure showed a relatively low accuracy compared to the undamaged floor. However, owing to the characteristics of the finite element model, the lab-scale model cannot be fully simulated; therefore, an error of approximately 20% is considered to be a fairly reliable result. It was confirmed that the estimated stiffness of the structure using the four object tracking algorithms showed a similar accuracy. This is because the value of the natural frequency (Table 3), which are the dynamic characteristics of the structure estimated by the four object tracking algorithms, have similar values. However, when only the estimated stiffness of the damaged structure was considered, it was confirmed that the most appropriate algorithm to be applied to the FE model updating is the KLT algorithm. Table 3. Comparison of column stiffness between experiment and prediction model using different tracking algorithms. Design Analysis (N/mm) Error (%) Damaged Stage (N/mm) KLT MIL KCF MOSSE KLT MIL KCF MOSSE (N/mm) Floor 1 1,286,400 1,286,400 1,194,896 1,211,896 1,211,896 1,217,792 6.833 5.792 5.792 5.333 Floor 2 1,286,400 1,270,320 1,031,800 995,888 993,208 983,024 18.776 21.603 21.814 22.616 Floor 3 1,286,400 1,286,400 1,189,384 1,194,744 1,199,568 1,188,848 7.542 7.125 6.75 7.583 Average - - - - - - 11.050 11.507 11.452 11.844 Appl. Sci. 2021, 11, 1622 13 of 16 (a) (b) (c) (d) (e) (f) (g) Figure 10. Comparison of the estimated stiffness using different object tracking algorithms: (a) FE model, (b) column stiffness of design stage, (c) damaged column stiffness, and (d–g) estimated column stiffness using different tracking algorithms. Although the natural frequency (Table 2) of the structure predicted by the four object tracking algorithms showed similar values, it was confirmed that the accuracy of the esti- mated stiffness of the structure differs by floor. Based on the above results, it was con- firmed that when FE model updating will be performed using the natural frequency of the structure as the fitness function, the accuracy of the FE model updating may be low- ered if the highly accurate natural frequency of the structure is not applied. 5. Discussion This paper presented a new technique to perform the FE model updating of the struc- tures using vision-based displacement measurements. Although the proposed method achieved reliable results for updating the FE model into a damaged state, there were some errors owing to the following reasons. (1) One of the main sources of the error is a modeling error. The FE model used in this study was a frame structure. Detailed information such as bolt connections was not considered in this model. Therefore, the response owing to these connections was not Appl. Sci. 2021, 11, 1622 14 of 16 fully reflected in the updated model. The accuracy of the FE model updating is ex- pected to be improved by developing a more detailed FE model. (2) Another source of error is inaccurate damping. The damping ratio of the structure is an important factor influencing the response of the structure. In practice, the damp- ing ratio is assumed to be 5% for reinforced concrete structures and 2% for welded steel structures but can be varied by the material and structural conditions. Kudu et al.  compared the modal damping ratio obtained by considering the measurement duration, frequency range, and sampling rate, with the method used to identify the modal parameters as variable parameters. As a result, it was observed that the change in natural frequency was extremely small whereas the modal damping ratio changed significantly depending on the selected parameters. In this study, the damping ratio of the structure was assumed to be 2%, which caused an error in the FE model up- dating. (3) Imperfection in vision-based tracking algorithms can also be a source of error. In this study, displacement was measured using four tracking methods, and FE model up- dating was performed using the results. However, the results of the model updating showed differences according to each tracking method. Each tracking algorithm has its own tracking mechanisms that can cause errors during the tracking process. Some common phenomena in computer vision, such as radial distortion and a rolling shut- ter effect, have also reduced the accuracy. In this study, these issues were minimized using the method proposed by Yoon et al. , but these distortions could not be fully removed. 6. Conclusions and Future Studies In this study, a vision-based finite element model updating using the genetic algo- rithm was proposed. The proposed system automatically measured the displacement of the structure from a video and updated the parameters of the FE model by using the ge- netic algorithm. To validate the performance of the proposed method, a lab-scale test with a three-story building was conducted. The displacement of each story of the building was measured before and after reducing the stiffness of one of the columns. The genetic algo- rithm was used to automatically optimize a non-damaged state of the FE model to a dam- aged state. Seven different object tracking algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD) were used and the FE model updating results were compared. As a result, proposed method using KLT, KCF, MIL, and MOSSE algorithms successfully up- dated the FE model to the as-is damaged condition, whereas CSRT, MedianFlow, and TLD were not able to update the FE model appropriately. Proposed method using KLT tracker showed the most reliable result with 89% accuracy for estimating the member stiffness. While some limitations are remaining as a future study, the proposed method is expected to reduce the time and effort for FE model updating by using computer vision. Author Contributions: Conceptualization, H.Y.; methodology, H.Y.; software, G.P.; validation, G.P.; formal analysis, G.P.; investigation, G.P.; resources, H.Y.; data curation, G.P.; writing—origi- nal draft preparation, G.P.; writing—review and editing, H.Y, K.H.; visualization, G.P.; supervi- sion, H.Y., K.H.; project administration, H.Y., K.H.; funding acquisition, H.Y. All authors have read and agreed to the published version of the manuscript. Funding: This work was supported by the Korea Agency for Infrastructure Technology Advance- ment (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 20CTAP- C153021-02). Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Not applicable. Conflicts of Interest: The authors declare no conflicts of interest. Appl. Sci. 2021, 11, 1622 15 of 16 References 1. Park, G.; Yoon, H.; Hong, K. Proposed Equations for Calculating Dynamic Hydraulic Pressure in a Rectangular Structure. Appl. Sci. 2020, 10, 8406, doi:10.3390/app10238406. 2. Doebling, S.W.; Farrar, C.R.; Cornwell, P. A Statistical Comparison of Impact and Ambient Testing Results from the Alamosa Can-yon Bridge; In Proc. of the 15th Int. Modal Analysis Conference, pp. 264-269. Orlando, FL: Society of Experimental Me- chanics, 1997. 3. Todd, M.; Johnson, G.; Vohra, S.; Chen-Chang, C.; Danver, B.; Malsawma, L., Civil infrastructure monitoring with fiber Bragg grating sensor arrays. In Proceedings of the 2nd International Workshop on Structural Health Monitoring, pp. 359-368, Oct. 17- 19, Qingdao, China, 2018. 4. Abe, M.; Fujino, Y.; Yanagihara, M.; Sato, M. Monitoring of Hakucho Suspension Bridge by ambient vibration measurement. In Proceedings of the SPIE’s 5th Annual International Symposium on Nondestructive Evaluation and Health Monitoring of Aging Infrastructure, Newport Beach, CA, USA, 6–8 March 2000; Volume 3995, pp. 237–244, doi:10.1117/12.387815. 5. Celebi, M.; Purvis, R.; Hartnagel, B.; Gupta, S.; Clogston, P.; Yen, P.; O’Connor, J.; Franke, M. Seismic instrumentation of the Bill Emerson Memorial Mississippi River Bridge at Cape Girardeau (MO): A cooperative effort. In Proceedings of the 4th In-terna- tional Seismic Highway Conference, Memphis, TN, USA, 9–11 February 2004. 6. Wong, K.-Y. Instrumentation and health monitoring of cable-supported bridges. Struct. Control. Health Monit. 2004, 11, 91– 124, doi:10.1002/stc.33. 7. Yilmaz, A.; Javed, O.; Shah, M. Object tracking: A survey. ACM Comput. Surv. (CSUR) 2006, 38, 13-es. 8. Jiang, R.; Jáuregui, D.V.; White, K.R. Close-range photogrammetry applications in bridge measurement: Literature review. Measurement 2008, 41, 823–834, doi:10.1016/j.measurement.2007.12.005. 9. Kwiatkowski, J.; Anigacz, W.; Beben, D. Comparison of Non-Destructive Techniques for Technological Bridge Deflection Test- ing. Materials 2020, 13, 1908, doi:10.3390/ma13081908. 10. Gawronek, P.; Makuch, M.; Mitka, B.; Gargula, T. Measurements of the Vertical Displacements of a Railway Bridge Using TLS Technology in the Context of the Upgrade of the Polish Railway Transport. Sensors 2019, 19, 4275, doi:10.3390/s19194275. 11. Kwiatkowski, J.; Anigacz, W.; Beben, D. A Case Study on the Noncontact Inventory of the Oldest European Cast-iron Bridge Using Terrestrial Laser Scanning and Photogrammetric Techniques. Remote. Sens. 2020, 12, 2745, doi:10.3390/rs12172745. 12. Porras-Amores, C.; Mazarrón, F.R.; Cañas, I.; Villoria Sáez, P. Terrestrial Laser Scanning Digitalization in Underground Con- structions. J. Cult. Herit. 2019, 38, 213–220. 13. Shih, M.-H.; Sung, W.-P. Developing Dynamic Digital Image Techniques with Continuous Parameters to Detect Structural Damage. Sci. World J. 2013, 2013, 1–7, doi:10.1155/2013/453468. 14. Feng, M.Q.; Fukuda, Y.; Feng, D.; Mizuta, M. Nontarget Vision Sensor for Remote Measurement of Bridge Dynamic Response. J. Bridg. Eng. 2015, 20, 04015023, doi:10.1061/(asce)be.1943-5592.0000747. 15. Yoon, H.; Elanwar, H.; Choi, H.; Golparvar-Fard, M.; Spencer, B.F., Jr. Target-free approach for vision-based structural sys-tem identification using consumer-grade cameras. Struct. Control Health Monit. 2016, 23, 1405–1416. 16. Yoon, H.; Shin, J.; Spencer, B.F., Jr. Structural displacement measurement using an unmanned aerial system. Comput. Aided Civ. Infrastruct. Eng. 2018, 33, 183–192. 17. Lim, J.; Yoon, H. Real-time Pedestrian Dynamic-load Localization using Vision-based Motion Sensing. J. Korean Soc. Hazard Mitig. 2019, 19, 323–330, doi:10.9798/kosham.2019.19.7.323. 18. Boda, R.; Priyadarsini, M.J.P.; Pemeena, J. Face detection and tracking using KLT and Viola Jones. ARPN J. Eng. Appl. Sci. 2016, 11, 13472–13476. 19. LuNežič, A.; Vojíř, T.; Čehovin Zajc, L.; Matas, J.; Kristan, M. Discriminative correlation filter TracNer with channel and spatial reliability. Int. J. Comput. Vis. 2018, 126, 671–688. 20. Henriques, J.F.; Caseiro, R.; Martins, P.; Batista, J. High-Speed Tracking with Kernelized Correlation Filters. IEEE Trans. Pat- tern Anal. Mach. Intell. 2014, 37, 583–596, doi:10.1109/tpami.2014.2345390. 21. Kalal, Z.; Mikolajczyk, K.; Matas, J. Forward-Backward Error: Automatic Detection of Tracking Failures. In Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, 23–26 August 2010; Institute of Electrical and Elec- tronics Engineers (IEEE): Piscataway Township, NJ, USA, 2010; pp. 2756–2759. 22. Babenko, B.; Yang, M.-H.; Belongie, S. Visual tracking with online multiple instance learning. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; IEEE: Piscataway Township, NJ, USA, 2009; pp. 983–990. 23. Bolme, D.S.; Beveridge, J.R.; Draper, B.A.; Lui, Y.M. Visual object tracking using adaptive correlation filters. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 13–18 June 2010; Institute of Electrical and Electronics Engineers (IEEE): Piscataway Township, NJ, USA, 2010; pp. 2544–2550. 24. Kalal, Z.; Mikolajczyk, K.; Matas, J. Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 34, 1409–1422. 25. Bathe, K.-J. Finite Element Procedures; Prentice-Hall, Inc., Englewood Cliffs, New Jersey: 2007; ISBN 0-9790049-0-X. 26. Ferreira, A.J. Matlab Codes for Finite Element Analysis: Solids and Structures; Springer Science & Business Media: Ber-lin/Hei- delberg, Germany, 2008; Volome 157; ISBN 1-4020-9200-8. 27. Kattan, P.I. Matlab Guide to Finite Elements: An Interactive Approach; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2010; ISBN 3-540-70698-4. Appl. Sci. 2021, 11, 1622 16 of 16 28. Brownjohn, J.W.; Xia, P. Finite element model updating of a damaged structure. 17th International Modal Analysis Confer-ence, SEM, Kissimmee, Florida, 25 February 1999; pp. 457–462. 29. Lee, J.-Y.; Goh, J.-Y. Selection of the principal genotype with genetic algorithm. J. Korean Data Inf. Sci. Soc. 2009, 20, 639–647. 30. Lorza, R.L.; García, R.E.; Martinez, R.F.; Calvo, M. Ángeles M. Using Genetic Algorithms with Multi-Objective Optimization to Adjust Finite Element Models of Welded Joints. Metals 2018, 8, 230, doi:10.3390/met8040230. 31. Lostado-Lorza, R.; Martinez, R.F.; Mac Donald, B.; Villanueva, P. Combining soft computing techniques and the finite ele-ment method to design and optimize complex welded products. Integr. Comput. Eng. 2015, 22, 153–170, doi:10.3233/ica-150484. 32. Lostado-Lorza, R.; Escribano-Garcia, R.; Fernandez-Martinez, R.; Illera-Cueva, M.; Mac Donald, B.J. Using the finite element method and data mining techniques as an alternative method to determine the maximum load capacity in tapered roller bear- ings. J. Appl. Log. 2017, 24, 4–14, doi:10.1016/j.jal.2016.11.009. 33. Wu, J.; Li, Q. Finite element model updating for a high-rise structure based on ambient vibration measurements. Eng. Struct. 2004, 26, 979–990, doi:10.1016/j.engstruct.2004.03.002. 34. Brownjohn, J.M.; Xia, P.-Q.; Hao, H.; Xia, Y. Civil Structure Condition Assessment by FE Model Updating: Methodology and Case Studies. Finite Elem. Anal. Des. 2001, 37, 761–775. 35. Goktepe, F.; Celebi, E.; Omid, A.J. Numerical and experimental study on scaled soil-structure model for small shaking table tests. Soil Dyn. Earthq. Eng. 2019, 119, 308–319, doi:10.1016/j.soildyn.2019.01.016. 36. Hokmabadi, A.S.; Fatahi, B.; Samali, B. Assessment of soil–pile–structure interaction influencing seismic response of mid-rise buildings sitting on floating pile foundations. Comput. Geotech. 2014, 55, 172–186, doi:10.1016/j.compgeo.2013.08.011. 37. Íñiguez-Macedo, S.; Lostado-Lorza, R.; Escribano-García, R.; Martínez-Calvo, M. Ángeles Finite Element Model Updating Com- bined with Multi-Response Optimization for Hyper-Elastic Materials Characterization. Materials 2019, 12, 1019, doi:10.3390/ma12071019. 38. Kudu, F.N.; Uçak, Ş.; Osmancikli, G.; Türker, T.; Bayraktar, A. Estimation of damping ratios of steel structures by operational modal analysis method. J. Constr. Steel Res. 2015, 112, 61–68. 39. Yoon, H.; Hoskere, V.; Park, J.W.; Spencer, J.B.F. Cross-Correlation-Based Structural System Identification Using Unmanned Aerial Vehicles. Sensors 2017, 17, 2075, doi:10.3390/s17092075.
Applied Sciences – Multidisciplinary Digital Publishing Institute
Published: Feb 10, 2021
Access the full text.
Sign up today, get DeepDyve free for 14 days.