Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

Chih-Chiang Wei

doi:10.1155/2020/5462040

Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

Wei, Chih-Chiang 2020-07-23 00:00:00 Hindawi Advances in Meteorology Volume 2020, Article ID 5462040, 18 pages https://doi.org/10.1155/2020/5462040 Research Article Development of Stacked Long Short-Term Memory Neural NetworkswithNumericalSolutionsforWindVelocityPredictions Chih-Chiang Wei Department of Marine Environmental Informatics and Center of Excellence for Ocean Engineering, National Taiwan Ocean University, Keelung City, Taiwan Correspondence should be addressed to Chih-Chiang Wei; ccwei@ntou.edu.tw Received 27 December 2019; Accepted 8 July 2020; Published 23 July 2020 Academic Editor: 'eodore Karacostas Copyright © 2020 Chih-Chiang Wei. 'is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Taiwan, being located on a path in the west Paciﬁc Ocean where typhoons often strike, is often aﬀected by typhoons. 'e accompanying strong winds and torrential rains make typhoons particularly damaging in Taiwan. 'erefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better preparation to mitigate a typhoon’s toll on life and property. For more accurate wind speed predictions during a typhoon episode, we used cutting-edge machine learning techniques to construct a wind speed prediction model. To ensure model accuracy, we used, as variable input, simulated values from the Weather Research and Forecasting model of the numerical weather prediction system in addition to adopting deeper neural networks that can deepen neural network structures in the construction of estimation models. Our deeper neural networks comprise multilayer perceptron (MLP), deep recurrent neural networks (DRNNs), and stacked long short-term memory (LSTM). 'ese three model-structure types diﬀer by their memory capacity: MLPs are model networks with no memory capacity, whereas DRNNs and stacked LSTM are model networks with memory capacity. A model structure with memory capacity can analyze time-series data and continue memorizing and learning along the time axis. 'e study area is northeastern Taiwan. Results showed that MLP, DRNN, and stacked LSTM prediction error rates increased with prediction time (1–6 hours). Comparing the three models revealed that model networks with memory capacity (DRNN and stacked LSTM) were more accurate than those without memory capacity. A further comparison of model networks with memory capacity revealed that stacked LSTM yielded slightly more accurate results than did DRNN. Additionally, we determined that in the construction of the wind speed prediction model, the use of numerically simulated values reduced the error rate approximately by 30%. 'ese results indicate that the inclusion of numerically simulated values in wind speed prediction models enhanced their prediction accuracy. Taiwan even if the typhoon itself does not hit Taiwan. 1. Introduction 'erefore, typhoons constitute a serious natural disaster in A typhoon is a severe natural disaster that aﬀects tropical Taiwan. and subtropical coastal countries, and it occurs most fre- For example, the 2015 Typhoon Soudelor was the most quently in the northwestern Paciﬁc Ocean. Taiwan is to the destructive typhoon that occurred in Taiwan in recent east of the Eurasian Continent and at the western side of the history, with gust intensity exceeding 12 on the Beaufort Paciﬁc Ocean; its climate is intermediate between tropical wind force scale (32.7 m/s). Its strong gusts caused wide- and subtropical climate. 'us, typhoons frequently occur in spread damage to infrastructure, aﬀecting gas supply, power Taiwan, generally in summer and fall. Typhoons aﬀecting and utilities, transportation and communication, and Taiwan typically develop at the sea surface southeast of weather radar stations. Electricity was cut oﬀ in approxi- Taiwan, and most typhoons are accompanied by torrential mately 4.5 million households simultaneously during Ty- rains and strong winds [1]. Such rain and wind add to the phoon Soudelor—the greatest recorded number in recent damage from typhoons, posing a great threat to the trans- history. 'e economic loss from the typhoon was estimated portation, economic, agricultural, and ﬁshery activities in to be as high as US$76 million [2]. 2 Advances in Meteorology results to be explained in terms of physical relationships. 'erefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better NWP models simulate the atmosphere on user-deﬁned grid scales as a moving ﬂuid. 'rough several types of param- preparation to mitigate a typhoon’s toll on life and property. In this study, cutting-edge machine learning (ML) tech- eterizations, NWP models also account for the inﬂuence of niques were used to improve predictive accuracy. In general, subgrid physical processes on grid-resolved motions ML algorithms learn from a huge dataset, improving their [31, 32]. NWP models, such as the Weather Research and ability to identify patterns in the data. Speciﬁcally, ML in- Forecasting model (WRF), have been increasingly popular as volves creating algorithms to make prediction from sets of a low-cost alternative source of data for such assessments of unknown data. Given their ability to perform parameter climate parameters [33]. WRF oﬀers a wide variety of physical and dynamical elements to choose from; these el- adjustment and achieve optimization through self-learning, neural network algorithms in ML are particularly powerful ements must be put together to form model conﬁgurations, with which the model can be run [34]. However, because of [3]. Such algorithms have been extensively used in recent wind speed prediction models, and these models are in- imperfect models and uncertain initial boundary atmo- spheric conditions, errors exist in the NWP output [35, 36]. creasingly data-driven due to developments in ML [4–11]. In the development of neural networks, multilayer perceptron Recent studies have applied the NWP model to typhoons (MLP) networks are a classic approach that are often used and tropical cyclones [37–39]. However, the prediction of and compared with other neural network models. For ex- severe meteorological phenomena (such as typhoons), ample, Wei [12] compared the accuracy of MLP with that of which result from a multiplicity of mutually interacting adaptive network-based fuzzy-inference-system neural multiscale processes, remains a major challenge for NWP networks in the construction of typhoon wind speed pre- systems [25, 40, 41]. 'ese models are typically unable to predict wind intensity with satisfactory accuracy, even at diction models. Deep learning has become possible due to the expo- short forecast times and a high horizontal resolution [42, 43]. Some studies have tried combining NWP with ML nential increase in computing power in recent years. 'is approach is the further derivation of multiple neural layers models to develop an integrated climate prediction model. For example, Zhao et al. [44] evaluated the performance and from the original neural layers of a model. Such derivation improves an algorithm’s ability to learn, better approxi- enhanced accuracy of a day-ahead wind-power forecasting mating the complex neural network structure of a human system. 'e system comprised artiﬁcial neural networks and being. For example, Hu et al. [13] formulated multilayer an NWP model. deep neural networks that were trained using data from Furthermore, to enhance the predictive accuracy of data-rich wind plants. 'ese networks extracted wind speed constructed ML models, other researchers have used nu- patterns, and the mapping was ﬁnely tuned using data from merically simulated results as input data for the construction of ML models. For example, Zhao et al. [45] developed the newly constructed wind plants. Tiancheng et al. [14] pro- posed a sandstorm prediction method that considered both ARIMAX model, where wind speed results from the WRF simulation were chosen as an exogenous input variable. the eﬀect of atmospheric movement and ground factors on sandstorm occurrence, called improved naive Bayesian However, to the best of our knowledge, in the literature on convolutional neural network classiﬁcation algorithm. short-term wind speed prediction for typhoons (or tropical In the ﬁeld of neural networks, recurrent neural networks cyclones), numerically simulated values have seldom been (RNNs), which can analyze sequential (or time-series) data, used as an input variable for ML prediction models. have recently been developed [15–20]. RNNs are con- 'erefore, in relation to the construction of a ML-based nectionist models with the ability to selectively pass infor- wind speed prediction model, we evaluated the improve- mation across sequence steps while processing sequential data ments to predictive accuracy aﬀorded by the use of nu- one element at a time [21]. 'erefore, RNNs are important, merically simulated values (by comparing between its use and nonuse). especially in the analysis of sequential data. A particular type of RNNs is long short-term memory (LSTM), a class of ar- 'erefore, our study has two primary aims: (1) develop an ML- and neural network-based wind speed prediction tiﬁcial neural networks where connections between units form a directed cycle [22] introduced the LSTM primarily to model and compare the predictive accuracy of various overcome the problem of diminishing gradients. LSTM neural network-based algorithms and (2) evaluate the im- creates an internal network state that allows it to exhibit provements to predictive accuracy aﬀorded by the use of dynamic temporal behavior. Unlike feedforward neural numerical solutions obtained from NWP models (by networks, RNNs can use their internal memory to process comparing between its use and nonuse) in a typhoon-surface arbitrary sequences of input [23]. To model the time series of wind-speed prediction model. wind speed data, Byeon et al. [24] developed the LSTM for the prediction of typhoon wind speeds. Hence, feature en- 2. Methodology and Algorithms hancement from RNNs has been explored in wind prediction. 'e widespread application of ensembles in numerical Figure 1 illustrates the ﬂow of the construction, involving weather prediction (NWP) has helped researchers improve NWP numerical solutions, of our neural network-based weather forecasts [25–30]. Numerical models can be used to typhoon wind-velocity prediction model. In the ﬁrst stage, calculate all climate parameters through atmospheric dy- we collected data on the typhoon characteristics and ground namics and numerical methods, allowing for simulated wind speed of the research area’s historical typhoon events. Advances in Meteorology 3 Collect data associated with historical typhoon events Perform the wind field simulation of typhoons with an NWP numerical model Refine data of typhoon characteristics and surface wind Generate wind-speed speed observations simulation solutions Preprocess the datasets and split data into training-validation and testing subsets Conduct the neural networks-based wind speed prediction model using MLP, DRNN, and stacked LSTM Predict wind speed using different neural networks-based prediction models Evaluate the forecast accuracy in ML models using different neural networks-based algorithms Evaluate of forecast accuracy in ML models using inputs with and without NWP numercial solutions Verify the optimal neural networks-based wind velocity prediction models Figure 1: Flowchart of neural network-based typhoon wind velocity prediction model using NWP numerical solutions. In the second step, wind speed solutions were obtained from forecast accuracy of the stacked LSTM model was evaluated a wind ﬁeld simulation of typhoons, involving an NWP against other neural networks. We also evaluated whether numerical model. In our study, we also employed a WRF forecast accuracy in ML-based models improved when NWP numerical model to simulate circulation distribution, thus numerical solutions were used as input. obtaining the wind speed values in the research area. Two datasets could be built, one comprising typhoon data and the 2.1. Frameworks Underlying the Proposed Neural Networks. other comprising wind simulation results from the NWP model. 'e datasets were split into testing and training- In this section, we describe the neural network-based ar- chitectures, which used the MLP, DRNN, and stacked LSTM validation subsets. 'e training-validation set was used for algorithms, that were adopted for model construction. As the learning of several ML-based wind velocity prediction illustrated in Figure 2(a), the MLP is a typical type of models, and the testing set was used for the identiﬁcation of feedforward backpropagation neural network that uses the optimal prediction model among these models. 'e best model among a set of ML neural network-based processing units placed in the input, hidden, and output layers [47–49]. Each unit (with an associated weight) in a models, involving MLP, DRNN, and stacked LSTM, was determined. According to [46], real-time dynamics consti- layer is connected to the units in adjacent layers [50, 51]. In the study, to enhance learning eﬃcacy—and by implication, tute the most challenging aspect of wind speed forecasting. We determined the stacked LSTM model to be the best for approximation, and prediction accuracy—we added hidden layers to a simple MLP neural network; the MLP was trained wind speed forecasting due to its appropriate handling of long- and short-term time dependency. In the ﬁnal stage, the through backpropagation. Generally, the weight updates 4 Advances in Meteorology Predicted V Predicted V Predicted H t+i t+i t+i Output layer Output layer Output layer Dropout layer Dropout layer Hidden layer Activation fun Summing fun Context Hidden layer units t f Hidden layer Activation fun Summing fun Context Hidden layer units Input layer Input layer t Inputs for wind velocity prediction Input layer Inputs for wind velocity prediction Inputs for wind velocity prediction (a) (b) (c) Figure 2: Architecture of (a) MLP, (b) DRNN, and (c) stacked LSTM neural networks. between layers are calculated in terms of the stochastic the ﬁrst LSTM layer produces sequence vectors used as the gradient descent [52]. Speciﬁcally, input of the subsequent LSTM layer. Moreover, the LSTM layer receives feedback from its previous time step, thus zE Δw (t + 1) � βΔw (t) + η , (1) allowing for the capturing of data patterns. 'e dropout ij ij zw ij layer also excludes 10% of the neurons to avoid overﬁtting. 'e basic structure of LSTM, as illustrated in the LSTM where w (t) is the weight set connecting layers i and j at ij layer in Figure 2(c), comprises an input gate i , output gate o , t t time t, Δw is the weight correction, η is the learning rate, β is forget gate f , and memory cell c . A single LSTM layer has a t t a momentum coeﬃcient, and E is a cost function that in- second-order RNN architecture that excels at storing se- dicates the diﬀerence between the target and predicted quential short-term memories and retrieving them at many values. In particular, η and β are hyperparameters for time steps later [55]. An LSTM network is identical to a adjusting the spacing of weight correction. standard RNN, with the exception of the summation units in As illustrated in Figure 2(b), the multilayers of the RNN the hidden layer being replaced by memory blocks [56]. structure comprise an input layer, multiple recurrent layers, Equations (2)–(6) describe how output values are updated at and an output layer. When the length of recurrent layer is 1, each step [22, 46, 57]. Speciﬁcally, the framework is a simple RNN. Here, the recurrent network is based on the networks developed by [53]. In the RNN, the f � σ􏼐W · x + U · h + b 􏼑, (2) t f t f t−1 f hidden units are connected to context units; in the successive time step, the units feed back into the hidden units. 'e i � σ W · x + U · h + b , (3) t i t i t−1 i hidden state at any time step can contain information from an (almost) arbitrarily long context window [21]. 'e DRNN o � σ W · x + U · h + b , (4) model framework has multiple recurrent layers before the t o t o t−1 o forwarding to a dropout layer and output layer at the ﬁnal c � f · c + i · σ W · x + U · h + b 􏼁, (5) output. In this paper, the dropout layer excludes 10% of the t t t−1 t c t c t−1 c neurons to avoid overﬁtting. A stacked LSTM architecture is deﬁned as an LSTM h � o · σ c􏼁 , (6) t t t model that comprises multiple LSTM layers (Figure 2(c)). where x is the input vector; σ is the activation function; W , 'e stacked LSTM, also known as deep LSTM, was ﬁrst t f W , W , W , U , U , U , and U are the weight vector terms; b , formulated by [54] and was applied to speech recognition i o c f i o c f b , b , and b are the corresponding bias terms; and h and h problems. Similar to the framework underpinning the i o c t −1 are the current and previous hidden vectors, respectively. DRNN model, the stacked LSTM model uses multiple LSTM In this study, because the function converges quickly and layers that are stacked before the forwarding to a dropout has no problems with a vanishing gradient, we used the layer and output layer at the ﬁnal output. In a stacked LSTM, Recurrent Recurrent layer layer LSTM layer LSTM layer Advances in Meteorology 5 ° ° grid spacing for the coarser domain (118 ReLU activation function in the middle layers (the hidden, E–126 E, ° ° recurrent, and LSTM layers) of the aforementioned three 21 N–29 N) was 15 km, and that of the ﬁner domain ° ° ° ° models. 'e ReLU is deﬁned as the positive part of its ar- (121 E–123 E, 24 N–26 N) was 3 km. Both domains had 32 gument; that is, f(x) � max (0, x), where x is the input to a vertical levels. 'is study used the WRF physical param- neuron. 'e ReLU function has been recently used to replace eters recommended by [59, 60]. 'ey studied the tracks and the sigmoid function in neural networks, resulting in good rainfall of typhoons that have invaded Taiwan in addition performance and fast training times [58]. Moreover, in these to other physical parameters, and they conducted wind proposed models, the input layers receive the observed and forecasts using the WRF model. 'ey determined the simulated meteorological values as their input. following physical parameters to be suitable for wind speed forecasts: with regard to the planetary layer, those from the Yonsei University (YSU) scheme; with regard to micro- 3. Study Area and Data physics, those from the WRF Single-Moment 5-Class Taiwan is located at the western Paciﬁc Ocean, a region (WSM5) scheme; for cumulus parameterization, those frequently in the path of typhoons. Typhoons in Taiwan are from the Kain–Fritsch scheme; and with regard to long- often accompanied by torrential rains, which severely en- wave radiation, those from the Rapid Radiative Transfer danger life and property. 'us, more accurate regional wind Model scheme. speed prediction is needed to improve typhoon protection. After our simulation of a large number of typhoon In this study, we chose northeastern Taiwan as the study area events, the WRF model generated wind velocity simulations (Figure 3), wherein Taipei City and Yilan City are the two at eight ground gauge stations. In the veriﬁcation of sub- major cities. sequent WRF wind outcomes, the output objective for the WRF model was the simulated wind value at an altitude of 10 m, because the ground gauge-station wind speed reached 3.1. Data from Gauge and Typhoon Weather Stations. the observation value at an altitude of 10 m. Figure 5 displays According to typhoon data from the Central Weather Bu- the scatter plot of the WRF model simulation values and reau (CWB), between 2000 and 2018, 29 typhoons have observed gauge-station values. According to Figure 5, sev- directly struck Taiwan (Table 1; 2003, 2011, and 2018 were eral observation stations (Keelung, Anbu, Yilan, and Su-ao) excluded from the table because typhoons did not directly have consistently higher wind speeds. 'ese higher wind aﬀect Taiwan in those years). 'e tracks of these historical speeds are caused by their surrounding topographies—which typhoons are illustrated in Figure 4. 'ese typhoons can be cause them to face windwards—in conjunction with the classiﬁed according to the Saﬃr–Simpson wind scale anticlockwise circulatory ﬂow characteristic of typhoons. depending on the intensity of their maximum sustained Pengjiayu Station is similar to the four stations. However, winds. A tropical storm has the wind speed range of being located on an island on the sea around northeastern 18–33 m/s, and category 1, 2, and 3 typhoons have the wind Taiwan, the station is directly aﬀected only by a typhoon’s speed ranges of 33–43 m/s, 43–50 m/s, and 50–58 m/s, re- circulatory ﬂow and not the station’s surrounding topog- spectively. In these collected typhoons, the numbers for the raphy; this station also records higher wind speed values. 'e category 1, 2, and 3 typhoons were 9, 5, and 8, respectively, Taipei, Banqiao, and Tamsui Stations are located within the and the number for the tropical storm was 7. Tamsui Basin and are thus fully enclosed by mountains. 'e data included two types of datasets: one on typhoon Because the circulatory ﬂow of typhoons can be easily dynamic characteristics and another (comprising 2621 disrupted by the surrounding topography, the wind speed hourly records) on surface wind speed observations. Surface values observed at these stations are slightly lower. wind speed data were released by the CWB and measured For the four windward observation stations, wind speed from eight gauge stations: Anbu, Banqiao, Keelung, Pen- simulation values were lower and approximately equal to gjiayu, Su-ao, Taipei, Tamsui, and Yilan (Figure 3). In ad- actual values for high and low wind speeds, respectively. For dition, data on typhoon dynamics were released by the the three observation stations in the Tamsui Basin, wind CWB. 'e data comprised six variables: pressure at the speed simulation values were approximately equal to the typhoon center, latitude of the typhoon center, longitude of actual values, for all wind speeds. the typhoon center, typhoon radius (i.e., the distance from To evaluate the simulation outcomes, this study used the typhoon center for points with winds greater than 15.5 m/s), mean absolute error (MAE) and root mean squared error moving speed of the typhoon, and maximum wind speed at (RMSE) that are deﬁned as follows: the typhoon center. 􏼌 􏼌 􏼌 􏼌 􏼌 􏼌 MAE � 􏽘 O − Y , 􏼌 􏼌 i i i�1 3.2. Data from Numerical Solutions. WRF model simula- tions using typhoon data were used to generate wind speed 􏽶�� (7) values at each meteorological station. To set the initial ﬁeld n and boundary conditions, we used data from the Final RMSE � 􏽘 O − Y􏼁 , i i i�1 Operational Global Analysis dataset. 'e dataset is a part of the Global Data Assimilation System of the US Govern- where Y is the estimated value of record i, O is the ob- ment’s National Center for Environmental Protection. 'e i i servation of record i, and n is the total number of records. grid was set to be a two-domain nested grid. 'e horizontal 6 Advances in Meteorology Figure 3: Map of Taiwan’s northeastern region and the meteorological stations therein. Table 1: Typhoons aﬀecting the study area from 2000 to 2017 (29 120°E 130°E 140°E 150°E 160°E 40°N incidents). Year Typhoon Period (UTC) 2000 Kai-Tak 6–10 Jul 2001 Toraji 28–31 Jul 30°N 2001 Nari 10–20 Sep 2002 Nakri 9–11 Jul 2004 Mindulle 29 Jun–3 Jul 2004 Aere 23–26 Aug 20°N 2004 Haima 12-13 Sep 2004 Nock-Ten 24–26 Oct 2005 Haitang 17–19 Jul 2005 Longwang 1-2 Oct 10°N 2006 Bilis 12–14 Jul 2007 Sepat 16–19 Aug 2007 Krosa 5–8 Oct 2008 Kalmaegi 16–19 Jul Figure 4: Tracks of historical typhoon incidents that aﬀected 2008 Fung-Wong 27–29 Jul Taiwan. 2008 Sinlaku 10–16 Sep 2008 Jangmi 27–30 Sep 2009 Morakot 6–9 Aug 2010 Namtheun 30-31 Aug lowest error values; their MAE and RMSE values 2012 Saola 30 Jul–3 Aug had errors in the ranges of 1.092–1.362 m/s and 2013 Soulik 10–14 Jul 1.422–1.654 m/s, respectively. 2013 Trami 16–22 Aug 2013 Kong-Rey 27–30 Aug 2014 Matmo 21–23 Jul 4. Modeling 2014 Fung-Wong 19–22 Sep 2015 Soudelor 6–9 Aug We used ML-based models to construct our wind speed 2015 Dujuan 25–29 Sep prediction model for our study area. Observation stations 2016 Megi 25–28 Sep at Taipei and Yilan were chosen as the test location. When 2017 Nesat 26–30 Jul constructing the model, the attribute data entered into the model included data on typhoon dynamics, data from ground meteorological observation stations, and data 'e performance of the estimation indicators (MAE and obtained from the aforementioned simulation. In this RMSE) in terms of errors is presented in Figure 6. 'e study, we performed data splitting for all typhoon epi- estimation indicators for the Pengjiayu Station had the sodes. Our training-validation set comprised data on 23 highest error values (MAE � 3.547 m/s and RMSE � 5.139 m/s), typhoon episodes between 2000 and 2013 (2093 records in followed by those of the four observation stations (Keel- total); our testing set comprised data on six typhoon ung, Anbu, Yilan, and Su-ao) facing windward—their episodes between 2014 and 2017 (528 records in total). MAE and RMSE values had errors in the ranges Model training and validation were performed through 1.623–2.020 m/s and 2.405–2.728 m/s, respectively. 'e 10-fold cross-validation, in which the training set was estimation indicators for the three observation stations divided into 10 subsamples, one of which was retained for (Taipei, Banqiao, and Tamsui) in the Tamsui Basin had the model veriﬁcation, and the other nine were used for model Advances in Meteorology 7 10 45 8 18 4 20 y = 0.653x + 2.0423 R = 0.5337 10 4 y = 0.4348x + 2.7207 y = 0.4401x + 3.7208 2 2 2 5 R = 0.5456 R = 0.7781 0123456789 10 11 12 13 Pengjiayu 02468 10 12 14 16 18 20 22 24 26 28 0 5 10 15 20 25 30 35 40 45 50 55 –1 –1 Observation (m·s ) –1 Observation (m·s ) Observation (m·s ) (g) (h) (a) 13 24 y = 0.626x + 1.5413 R = 0.6214 1 4 y = 0.4039x + 2.5521 0 2 R = 0.6099 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 –1 Observation (m·s ) 0 24 6 810 12 14 16 18 20 22 24 26 –1 (f ) Observation (m·s ) (b) y = 0.5563x + 1.7758 2 35 R = 0.5677 0 30 4 y = 0.4428x + 1.84 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 –1 R = 0.6311 Observation (m·s ) 25 (e) 0 24 6 810 12 14 16 18 20 22 24 26 28 –1 Observation (m·s ) (c) 5 y = 0.3963x + 2.3769 R = 0.6873 0 5 10 15 20 25 30 35 –1 Observation (m·s ) (d) Figure 5: Scatterplots depicting observations vs. WRF simulations at eight stations: (a) Pengjiayu, (b) Keelung, (c) Yilan, (d) Su-ao, (e) Banqiao, (f) Taipei, (g) Tamsui, and (h) Anbu. training. In the veriﬁcation process, each subsample must 'ese hyperparameters were evaluated through trial be veriﬁed. and error. First, the number of neurons in a middle layer As mentioned in Section 2.1, multilayer neural networks was adjusted from 10 to 100 until the curves of the RMSE were used to construct our ML neural network-based of the RMSE errors were approximated. 'e middle layers models. We used the adaptive moment estimation optimi- of the MLP, DRNN, and stacked LSTM models are the zation algorithm (also known as the Adam optimizer) to hidden, recurrent, and LSTM layers, respectively. Sub- sequently, the prediction errors corresponding to the optimize the momentum and learning rate [61]. Generally, the Adam optimizer is more broadly applied in neural minimum RMSE were obtained in the form of the optimal networks [62–64]. 'e Adam optimizer can be used instead solution for the number of neurons in the middle layer. of the classical stochastic gradient descent procedure to When the lead time was 1 h, for the Taipei Station, the iteratively update network weights based on training data. optimal numbers of neurons were 30, 50, and 50 for the We also calibrated the hyperparameters, speciﬁcally, the MLP, DRNN, and LSTM models, respectively; for the number of neurons in a middle layer and the length of the Yilan Station, those numbers for the three models were 40, middle layers in the MLP, DRNN, and stacked LSTM 30, and 30, respectively (Figures 7(a) and 8(a)). Subse- models. quently, we calibrated the length of the middle layers in –1 –1 Simulation (m·s ) Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) 8 Advances in Meteorology Pengjiayu Keelung Anbu Yilan Su-ao Taipei Banqiao Tamsui Station MAE RMSE Figure 6: Simulation performance in terms of prediction errors. 1.30 1.30 1.25 1.25 1.20 1.20 1.15 1.15 1.10 1.10 1.05 1.05 10 20 30 40 50 60 70 80 90 100 1 2 34 5 6 79 8 10 MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 7: Sensitivity of model parameters on the Taipei gauge: (a) number of neurons in a middle layer and (b) length of middle layers. 2.10 2.10 2.00 2.00 1.90 1.90 1.80 1.80 1.70 1.70 1.60 1.60 1.50 1.50 10 20 30 40 50 60 70 80 90 100 1 2 34 5 6 79 8 10 MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 8: Sensitivity of model parameters on the Yilan gauge: (a) number of neurons in a middle layer and (b) length of middle layers. the networks, adjusting them to be between 1 and 10 the three models were 7, 6, and 5, respectively layers. For the Taipei Station, the optimal lengths of the (Figures 7(b) and 8(b)). middle layers were 7, 5, and 4 for the MLP, DRNN, and Using the aforementioned method, we conducted pa- LSTM models, respectively; for the Yilan Station, those for rameter testing for the prediction models. Tests were RMSE (m/s) RMSE (m/s) Errors (m/s) RMSE (m/s) RMSE (m/s) Advances in Meteorology 9 3.0 1.8 2.5 1.5 2.0 1.2 1.5 0.9 1.0 0.6 0.5 0.3 0.0 0.0 123456 123456 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 9: Performance levels of neural network-based predictions for lead times between 1 and 6 h using a training-validation set for the (a) Taipei Station and (b) Yilan Station. Soudelor Typhoon Megi Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (a) Soudelor Typhoon Megi 16 Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (b) Figure 10: Continued. RMSE (m/s) Wind speed (m/s) Wind speed (m/s) RMSE (m/s) 10 Advances in Meteorology Soudelor Typhoon Megi Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (c) Figure 10: Predicted results of typhoons (2014–2017) in the testing set on the Taipei gauge for the lead times of (a) 1 h, (b) 3 h, and (c) 6 h. Dujuan Soudelor Typhoon Megi 28 Nesat 24 Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (a) Dujuan Soudelor 30 Typhoon Megi Nesat Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (b) Figure 11: Continued. Wind speed (m/s) Wind speed (m/s) Wind speed (m/s) Advances in Meteorology 11 30 Dujuan Soudelor Typhoon Megi 28 Nesat Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (c) Figure 11: Predicted results of testing typhoons (2014–2017) on the Yilan gauge for the lead times of (a) 1 h, (b) 3 h, and (c) 6 h. 2.0 3.0 1.8 2.5 1.6 1.4 2.0 1.2 1.0 1.5 0.8 1.0 0.6 0.4 0.5 0.2 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) 3.5 1.2 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (c) (d) Figure 12: Performance measures at lead times from 1 to 6 h on the Taipei gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. MAPE MAE (m/s) Wind speed (m/s) RMSPE RMSE (m/s) 12 Advances in Meteorology 2.7 3.5 2.4 3.0 2.1 1.8 2.5 1.5 2.0 1.2 1.5 0.9 1.0 0.6 0.5 0.3 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) 1.6 5.0 4.5 1.4 4.0 1.2 3.5 1.0 3.0 0.8 2.5 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (c) (d) Figure 13: Performance measures at lead times from 1 to 6 h on the Yilan gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. conducted using the training-validation set at six hourly lead the Yilan Station were between 9.9 m/s (Fung-Wong) and times between 1 and 6 h. 'e RMSE performance values (for 26.8 m/s (Soudelor). As mentioned, the windward Yilan Station is more susceptible to the circular ﬂow of the all lead times) for the three models are presented in Figure 9. 'e stacked LSTM model outperformed the MLP and typhoon, unlike the Taipei Station, which is shielded by DRNN models for all lead times and for both the Taipei and the mountains of the Tamsui Basin. 'erefore, wind speed Yilan Stations. We further tested and ﬁne-tuned these was consistently higher at the Yilan station. models using the testing set to conﬁrm the accuracy and Figures 10(a)–10(c) illustrate the prediction results for the feasibility of each model. Taipei Station at the lead times of 1, 3, and 6 h, respec- tively; for all prediction models, prediction accuracy was inversely related to the lead time. 'e data for the Yilan 5. Evaluation Station exhibited a similar trend (Figures 11(a)−11(c)). Such a situation is not atypical for prediction models. 5.1. Predictions of Forecasting Horizons. We tested and Intuitively, the further a model predicts into the future, evaluated the MLP, DRNN, and stacked LSTM models the harder it is to obtain useful, real-time features for using the testing set. 'e testing set comprised data prediction. 'erefore, to evaluate predictive accuracy, we (measured at the Yilin and Taipei Stations) on six typhoon used the error evaluation indicators to quantify each episodes that occurred between 2014 and 2017. Figures 10 model’s rate of error in predicting the subsequent hour’s and 11 illustrate the time-series charts of the simulated wind speed. and observed wind speed values for these typhoon epi- To compute term-by-term comparisons of the relative sodes. 'e six typhoon episodes were Matmo in 2014, error in the prediction with respect to the actual value of the Fung-Wong in 2014, Soudelor in 2015, Dujuan in 2015, variable, the mean absolute percentage error (MAPE) and Megi in 2016, and Nesat in 2017. According to root mean square percentage error (RMSPE) were calculated. Figure 10(a), the maximum observed wind speeds for the MAPE and RMSPE can be calculated using the following Taipei Station were between 7.8 m/s (Fung-Wong) and formulae: 14.9 m/s (Soudelor). According to Figure 11(a), those for MAPE MAE (m/s) RMSPE RMSE (m/s) Advances in Meteorology 13 Table 2: Average performance measures of absolute error terms (MAE and RMSE) and relative error terms (MAPE and RMSPE) for 1–6 h predictions. Station Measure MLP DRNN Stacked LSTM MAE (m/s) 1.241 1.070 0.928 RMSE (m/s) 1.666 1.439 1.228 Taipei MAPE 0.730 0.599 0.560 RMSPE 2.129 1.764 1.593 MAE (m/s) 1.863 1.714 1.528 RMSE (m/s) 2.680 2.489 2.185 Yilan MAPE 1.151 0.974 0.927 RMSPE 3.643 3.030 2.847 2.0 3.0 1.8 2.5 1.6 1.4 2.0 1.2 1.0 1.5 0.8 1.0 0.6 0.4 0.5 0.2 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (a) (b) 3.5 1.2 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (c) (d) Figure 14: Performance measures of predictive accuracy for the use and nonuse of WRF simulation values on the Taipei gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. 􏼌 􏼌 n 􏼌 􏼌 Generally, MAPE is used to express the MAE as a 􏼌 􏼌 1 O − Y i i 􏼌 􏼌 MAPE � 􏽘􏼌 􏼌, percentage of the observations. MAPE is an unbiased sta- 􏼌 􏼌 􏼌 􏼌 n O i�1 tistic for measuring the predictive capability of a model [65]. 􏽶�� RMSPE has the same properties as the RMSE but is (8) expressed as a percentage [66]. 1 O − Y i i RMSPE � 􏽘 􏼠 􏼡 . Figures 12 and 13 illustrate the performance of each n O i�1 model in terms of the four error indicators (MAE, RMSE, MAPE MAE (m/s) RMSPE RMSE (m/s) 14 Advances in Meteorology 3.5 4.5 3.0 4.0 2.5 3.5 3.0 2.0 2.5 1.5 2.0 1.5 1.0 1.0 0.5 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (a) (b) 2.0 5.5 1.8 5.0 1.6 4.5 1.4 4.0 1.2 3.5 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (c) (d) Figure 15: Performance measures of predictive accuracy for the use and nonuse of WRF simulation values on the Yilan gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. MAPE, and RMSPE) for all lead times and for both Taipei network unit that can memorize numerical values of and Yilan Stations. For the MLP model, the error rate was diﬀerent time lengths (to determine the quantity of useful steeply and positively related to the lead time. However, for information). Additionally, a gate in the LSTM blocks can the DRNN and stacked LSTM models, this relation was help determine if the input is important enough to be remembered and if the input can be exported as outputs. positive but slighter. To explain the results using the RMSPE error as an example, the RMSPE error increased as the lead Because the data involved in wind speed prediction are time increased. For the Taipei Station and for any given lead sequential, the decision on how far back the retention of time, the MLP model exhibited the steepest increase in error, memory data should go becomes consequential for pre- followed by the DRNN and RMSPE models. Speciﬁcally, dictive accuracy. By contrast, because the MLP model has when the lead time increased from 1 to 6 h, RMSPE in- no memory capacity, its predictions are based on limited creased from 1.494 to 2.616, from 1.386 to 2.181, and from and currently available information, thus decreasing its 1.205 to 2.030 for the MLP, DRNN, and stacked LSTM predictive accuracy. For the DRNN model, although it can models, respectively. Similar results were obtained for the receive memory information from long ago, the lack of a gate ﬁlter system can cause the receiving of too much (i.e., Yilan Station. When the lead time increased from 1 to 6 h, RMSPE increased from 3.391 to 4.142, from 2.409 to 3.613, overly long) memory information, potentially under- and from 2.127 to 3.457 for the MLP, DRNN, and stacked mining predictive accuracy. LSTM models, respectively. Table 2 presents the average measurement values of the 5.2. Evaluation of Forecast Eﬃciency with and without Nu- four error indicators for all lead times. In terms of the two unbiased statistical indicators, MAPE and RMSPE, the merical Solutions. We evaluated whether the use of WRF simulation values as input aﬀects the model’s accuracy in stacked LSTM and MLP models exhibited the most and least favorable performance, respectively. Stacked LSTM’s predicting wind speed. We focused on the stacked LSTM model because it was the best performing model. Figures 14 superiority is attributable to its model structure. Specif- ically, LSTM is a neural network that contains LSTM and 15 illustrate the performance of the stacked LSTM model in terms of the four error indicators (MAE, RMSE, blocks. LSTM blocks can be described as a type of smart MAPE MAE (m/s) RMSPE RMSE (m/s) Advances in Meteorology 15 45 45 40 40 35 35 30 30 25 25 20 20 15 15 10 10 5 5 0 0 123456 123456 Lead time (h) Lead time (h) Taipei Station Taipei Station Yilan Station Yilan Station (a) (b) Figure 16: Improvement rate of predictive accuracy in terms of MAE and RMSE, comparing the use and nonuse of WRF simulation values for the Taipei and Yilan Stations. MAPE, and RMSPE) for all lead times. We compared 6. Conclusions forecast eﬃciency with and without WRF simulated values. To accurately predict the wind speed of future typhoons, we Data for the Taipei and Yilan stations are presented in constructed a typhoon wind speed prediction model using Figures 14 and 15, respectively. cutting-edge ML techniques. RNNs have been recently We determined that the use of WRF simulation values as developed as a type of neural network that can analyze input increased the model’s predictive accuracy. 'erefore, sequential data. 'e structure of such networks facilitates the the use of numerically simulated values as a part of the input eﬀective processing of wind speed-relevant climate data over data aids in the reduction of predictive error. an extended period. 'at is, the structure imbues RNN We deﬁne the improvement rates for the MAE and models with long-term memory capacity. 'erefore, such RMSE (denoted IR and IR , respectively) as follows: MAE RMSE networks are suitable for predicting typhoon wind speeds. MAE − MAE 􏼁 LSTM is a type of RNN that allows the user to decide the with without IR � , MAE memory time’s length. Additionally, LSTM gives users the MAE without option to ﬁlter output results, increasing LSTM’s predictive (9) RMSE − RMSE 􏼁 accuracy. According to current developments in deep with without IR � , RMSE learning, learning performance is enhanced when the layers RMSE without of neural networks are deepened. 'erefore, we used deep where MAE and MAE are the MAE results on the learning neural networks in this study. Additionally, we with without use and nonuse of WRF simulation values, respectively, and compared the performance of three types of RNNs—MLP, RMSE and RMSE are the RMSE results on the use DRNN, and stacked LSTM—in predicting wind speed with without and nonuse of the WRF simulation values, respectively. values. 'ese three types of model structure diﬀer by their Figure 16 details the improvement rates for the Taipei memory capacity: MLPs are model networks with no and Yilan Stations for all lead times. For the Taipei Station, memory capacity, whereas DRNNs and stacked LSTM are IR ranged between 25.5% and 29.7%, and IR ranged MAE RMSE model networks with memory capacity. between 27.0% and 30.8%. For the Yilan Station, IR We chose northeastern Taiwan as the study area, and the MAE ranged between 26.4% and 36.3%, and IR ranged be- RMSE observation stations at Taipei and Yilan were selected as the tween 28.4% and 35.7%. Generally, the average IR (for all study subjects. 'e results indicated that for both the Taipei MAE lead times) was 27.3% and 30.3% for the Taipei and Yilan and Yilan Stations and for the MLP, DRNN, and stacked Stations, respectively, and the average IR (for all lead LSTM models, prediction error is positively related to the RMSE times) was 28.7% and 31.1% for the Taipei and Yilan Stations, prediction lead time (of which there were six, one for each respectively. For both the Taipei and Yilan Stations, im- hour between 1 and 6 h). In other words, the lead time is provement was demonstrated in predictive accuracy, al- inversely related to predictive accuracy. Model networks though the improvement rate was higher for the Yilan with memory capacity (DRNN and stacked LSTM) are more Station. In summary, although wind speed was diﬃcult to accurate than those without memory capacity (MLP). predict accurately at the Yilan Station, the use of WRF Stacked LSTM was also discovered to be more accurate than simulation values increased the predictive accuracy of the DRNN. Stacked LSTM is better than the traditional RNN model at the Yilan Station. method. Because stacking the LSTM’s hidden layers deepens IR (%) MAE IR (%) MAE 16 Advances in Meteorology [5] M. Kim, M. Park, J. Im, S. Park, and M. I. Lee, “Machine the model, model accuracy is increased; LSTM is closer to learning approaches for detecting tropical cyclone formation being a deep learning technique than its counterparts. 'us, using satellite data,” Remote Sensing, vol. 11, 2019. stacked LSTMs is a stable technique for overcoming [6] M. E. Kozar, V. Misra, and M. D. Powell, “Hindcasts of in- problems in wind velocity prediction. tegrated kinetic energy in Atlantic tropical cyclones: a neural Finally, we examined if model accuracy is increased by network prediction scheme,” Monthly Weather Review, the use of WRF simulation values as an input variable in the vol. 144, no. 12, pp. 4591–4603, 2016. wind speed prediction model. 'e stacked LSTM model was [7] M. Madhiarasan and S. N. Deepa, “A novel criterion to select used in the evaluation, given its best performance in most hidden neuron numbers in improved back propagation evaluation tests of the current study. Evaluating predictive networks for wind speed forecasting,” Applied Intelligence, accuracy in terms of the four error indicators, namely, MAE, vol. 44, no. 4, pp. 878–893, 2016. RMSE, MAPE, and RMSPE, veriﬁed that the inclusion of the [8] J. Naik, P. Satapathy, and P. K. Dash, “Short-term wind speed numerically simulated input greatly increased model ac- and wind power prediction using hybrid empirical mode decomposition and kernel ridge regression,” Applied Soft curacy. Speciﬁcally, for all lead times, the use of numerically Computing, vol. 70, pp. 1167–1188, 2018. simulated values reduced the MAE and RMSE error rates of [9] I. P. Panapakidis, C. Michailides, and D. C. Angelides, “A the Taipei Station by 27.3% and 28.7%, respectively. Simi- data-driven short-term forecasting model for oﬀshore wind larly, the MAE and RMSE rates for the Yilan Station were speed prediction based on computational intelligence,” reduced by 30.3% and 31.1%, respectively. 'erefore, the Electronics, vol. 8, 2019. inclusion of numerically simulated values in wind speed [10] C.-C. Wei, “Conceptual weather environmental forecasting prediction models enhanced their prediction accuracy. system for identifying potential failure of under-construction structures during typhoons,” Journal of Wind Engineering and Industrial Aerodynamics, vol. 168, pp. 48–59, 2017. Data Availability [11] G. Pang, J. He, Y. Huang, and L. Zhang, “A binary logistic 'e Final Operational Global Analysis data were obtained regression model for severe convective weather with nu- merical model data,” Advances in Meteorology, vol. 2019, from the Global Data Assimilation System of the US Article ID 6127281, 15 pages, 2019. Government’s National Center for Environmental Protec- [12] C.-C. Wei, “Surface wind nowcasting in the Penghu Islands tion, which is available at https://rda.ucar.edu/datasets/ based on classiﬁed typhoon tracks and the eﬀects of the ds083.2/. 'e typhoon data were obtained from the Cen- Central Mountain Range of Taiwan,” Weather and Fore- tral Weather Bureau of Taiwan, which are available at casting, vol. 29, no. 6, pp. 1425–1450, 2014. https://rdc28.cwb.gov.tw/and https://e-service.cwb.gov.tw/ [13] Q. Hu, R. Zhang, and Y. Zhou, “Transfer learning for short- wdps/. term wind speed prediction with deep neural networks,” Renewable Energy, vol. 85, pp. 83–95, 2016. [14] L. Tiancheng, R. Qing-dao-er-ji, and Q. Ying, “Application of Conflicts of Interest improved naive Bayesian-CNN classiﬁcation algorithm in 'e author declares that there are no conﬂicts of interest sandstorm prediction in inner Mongolia,” Advances in Me- regarding the publication of this paper. teorology, vol. 2019, Article ID 5176576, 13 pages, 2019. [15] B. Cortez, B. Carrera, Y.-J. Kim, and J.-Y. Jung, “An archi- tecture for emergency event prediction using LSTM recurrent Acknowledgments neural networks,” Expert Systems with Applications, vol. 97, pp. 315–324, 2018. 'is study was supported by the Ministry of Science and [16] L. Guo, N. Li, F. Jia, Y. Lei, and J. Lin, “A recurrent neural Technology, Taiwan, under Grant no. MOST107-2111-M- network based health indicator for remaining useful life pre- 019-003. 'e author acknowledges the data provided by the diction of bearings,” Neurocomputing, vol. 240, pp. 98–109, 2017. Central Weather Bureau of Taiwan and the Research Data [17] T. Liu, S. Yu, B. Xu, and H. Yin, “Recurrent networks with Archive (RDA) at NCAR. attention and convolutional networks for sentence repre- sentation and classiﬁcation,” Applied Intelligence, vol. 48, no. 10, pp. 3797–3806, 2018. References [18] A. S. Qureshi, A. Khan, A. Zameer, and A. Usman, “Wind power prediction using deep neural network based meta [1] C.-C. Wei, “Soft computing techniques in ensemble precip- regression and transfer learning,” Applied Soft Computing, itation nowcast,” Applied Soft Computing, vol. 13, no. 2, vol. 58, pp. 742–755, 2017. pp. 793–805, 2013. [19] X. Yuan, C. Chen, M. Jiang, and Y. Yuan, “Prediction interval [2] C. H. Chang, U. T. Wang, H. C. Fu, and U. C. Lin, Disaster of wind power using parameter optimized Beta distribution Survey of Typhoon Soudelor in 2015, pp. 1–48, National Science and Technology Center for Disaster Reduction, Investigation based LSTM model,” Applied Soft Computing, vol. 82, 2019. [20] Y. Zhang, X. Wang, and H. Tang, “An improved Elman neural Report, Taipei, Taiwan, 2015. [3] C. C. Wei, P. C. Peng, C. H. Tsai, and C. L. Huang, “Regional network with piecewise weighted gradient for time series prediction,” Neurocomputing, vol. 359, pp. 199–208, 2019. forecasting of wind speeds during typhoon landfall in Taiwan: a case study of westward-moving typhoons,” Atmosphere, [21] Z. C. Lipton, J. Berkowitz, and C. Elkan, “A critical review of recurrent neural networks for sequence learning,” 2015, vol. 9, 2018. [4] E. Cadenas, W. Rivera, R. Campos-Amezcua, and C. Heard, http://arxiv.org/abs/1506.00019. [22] S. Hochreiter and J. Schmidhuber, “Long short-term mem- “Wind speed prediction using a univariate ARIMA model and a multivariate NARX model,” Energies, vol. 9, 2016. ory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. Advances in Meteorology 17 [23] Q. Zhang, H. Wang, J. Dong, G. Zhong, and X. Sun, “Pre- [39] Z. Wu, C. Jiang, B. Deng, J. Chen, and X. Liu, “Sensitivity of diction of sea surface temperature using long short-term WRF simulated typhoon track and intensity over the South memory,” IEEE Geoscience and Remote Sensing Letters, China Sea to horizontal and vertical resolutions,” Acta Oce- vol. 14, no. 10, pp. 1745–1749, 2017. anologica Sinica, vol. 38, no. 7, pp. 74–83, 2019. [24] W. Byeon, M. Liwicki, and T. M. Breuel, “Scene analysis by [40] F. Cassola, F. Ferrari, and A. Mazzino, “Numerical simula- mid-level attribute learning using 2D LSTM networks and an tions of Mediterranean heavy precipitation events with the application to web-image tagging,” Pattern Recognition Let- WRF model: a veriﬁcation exercise using diﬀerent ap- ters, vol. 63, pp. 23–29, 2015. proaches,” Atmospheric Research, vol. 164-165, pp. 210–225, [25] E. Fiori, A. Parodi, and F. Siccardi, “Uncertainty in prediction of deep moist convective processes: turbulence parameteri- [41] S. Davolio, D. Mastrangelo, M. M. Miglietta, O. Drofa, zations, microphysics and grid-scale eﬀects,” Atmospheric A. Buzzi, and P. Malguzzi, “High resolution simulations of a Research, vol. 100, no. 4, pp. 447–456, 2011. ﬂash ﬂood near Venice,” Natural Hazards and Earth System [26] F. Molteni, R. Buizza, T. N. Palmer, and T. Petroliagis, “'e Science, vol. 9, no. 5, pp. 1671–1678, 2009. ECMWF ensemble prediction system: methodology and [42] M. L. Weisman, C. Davis, W. Wang, K. W. Manning, and validation,” Quarterly Journal of the Royal Meteorological J. B. Klemp, “Experiences with 0-36-h explicit convective Society, vol. 122, no. 529, pp. 73–119, 1996. forecasts with the WRF-ARW model,” Weather and Fore- [27] A. H. Murphy, “What is a good forecast? An essay on the casting, vol. 23, no. 3, pp. 407–437, 2008. nature of goodness in weather forecasting,” Weather and [43] M. M. Miglietta and R. Rotunno, “Application of theory to Forecasting, vol. 8, no. 2, pp. 281–293, 1993. simulations of observed cases of orographically forced con- [28] X. Shen, B. Jiang, Q. Cao, W. Lin, and L. Zhang, “Sensitivity of vective rainfall,” Monthly Weather Review, vol. 140, no. 9, precipitation and structure of typhoon Hato to bulk and pp. 3039–3053, 2012. explicit spectral bin microphysics schemes,” Advances in [44] P. Zhao, J. Wang, J. Xia, Y. Dai, Y. Sheng, and J. Yue, Meteorology, vol. 2019, Article ID 2536413, 2019. “Performance evaluation and accuracy enhancement of a day- [29] B. Storm and S. Basu, “'e WRF model forecast-derived low- ahead wind power forecasting system in China,” Renewable level wind shear climatology over the United States Great Energy, vol. 43, pp. 234–241, 2012. Plains,” Energies, vol. 3, no. 2, pp. 258–276, 2010. [45] E. Zhao, J. Zhao, L. Liu, Z. Su, and N. An, “Hybrid wind speed [30] K. Trachte, “Atmospheric moisture pathways to the highlands prediction based on a self-adaptive ARIMAX model with an of the tropical Andes: analyzing the eﬀects of spectral nudging exogenous WRF simulation,” Energies, vol. 9, 2016. on diﬀerent driving ﬁelds for regional climate modeling,” [46] S. Liang, L. Nguyen, and F. Jin, “A multi-variable stacked Atmosphere, vol. 9, no. 11, p. 456, 2018. long-short term memory network for wind speed forecast- [31] D. Carvalho, A. Rocha, M. Gomez-Gesteira, and C. Silva Santos, ing,” in Proceedings of the 2018 IEEE International Conference “WRF wind simulation and wind energy production estimates on Big Data, pp. 4561–4564, Seattle, WA, USA, December forced by diﬀerent reanalyses: comparison with observed data for Portugal,” Applied Energy, vol. 117, pp. 116–126, 2014. [47] A. Rostami, A. Baghban, A. H. Mohammadi, A. Hemmati- [32] X. T. Chadee, N. R. Seegobin, and R. M. Clarke, “Optimizing Sarapardeh, and S. Habibzadeh, “Rigorous prognostication of the weather research and forecasting (WRF) model for permeability of heterogeneous carbonate oil reservoirs: smart mapping the near-surface wind resources over the south- modeling and correlation development,” Fuel, vol. 236, ernmost caribbean islands of Trinidad and Tobago,” Energies, pp. 110–123, 2019. vol. 10, 2017. [48] H. R. Saghaﬁ, A. Rostami, and M. Arabloo, “Evolving new [33] N. K. Awan, H. Truhetz, and A. Gobiet, “Parameterization- strategies to estimate reservoir oil formation volume factor: induced error characteristics of MM5 and WRF operated in smart modeling and correlation development,” Journal of climate mode over the Alpine region: an ensemble-based Petroleum Science and Engineering, vol. 181, p. 106180, 2019. analysis,” Journal of Climate, vol. 24, no. 12, pp. 3107–3123, [49] A. Rostami, M. Arabloo, A. Kamari, and A. H. Mohammadi, “Modeling of CO solubility in crude oil during carbon di- [34] D. E. K. Dzebre, A. A. Acheampong, J. Ampofo, and oxide enhanced oil recovery using gene expression pro- M. S. Adaramola, “A sensitivity study of surface wind sim- gramming,” Fuel, vol. 210, pp. 768–782, 2017. ulations over coastal Ghana to selected time control and [50] S. S. Behzadi, C. Prakasvudhisarn, J. Klocker, P. Wolschann, nudging options in the weather research and forecasting and H. Viernstein, “Comparison between two types of arti- model,” Heliyon, vol. 5, 2019. ﬁcial neural networks used for validation of pharmaceutical [35] E. Kalney, Atmospheric Modeling, Data Assimilation, and processes,” Powder Technology, vol. 195, no. 2, pp. 150–157, Predictability, Cambridge University Press, Cambridge, UK, [51] C.-C. Wei, “Comparing lazy and eager learning models for [36] Y. Chu, C. Li, Y. Wang, J. Li, and J. Li, “A long-term wind water level forecasting in river-reservoir basins of inundation speed ensemble forecasting system with weather adapted regions,” Environmental Modelling & Software, vol. 63, correction,” Energies, vol. 9, no. 11, p. 894, 2016. pp. 137–155, 2015. [37] Z. Di, W. Gong, Y. Gan, C. Shen, and Q. Duan, “Combi- [52] D.-C. Lo, C.-C. Wei, and E.-P. Tsai, “Parameter automatic natorial optimization for WRF physical parameterization calibration approach for neural-network-based cyclonic schemes: a case study of three-day typhoon simulations over the Northwest Paciﬁc Ocean,” Atmosphere, vol. 10, no. 5, precipitation forecast models,” Water, vol. 7, no. 12, p. 233, 2019. pp. 3963–3977, 2015. [38] C.-C. Wei, “Wavelet support vector machines for forecasting [53] J. L. Elman, “Finding structure in time,” Cognitive Science, precipitation in tropical cyclones: comparisons with GSVM, vol. 14, no. 2, pp. 179–211, 1990. regression, and MM5,” Weather and Forecasting, vol. 27, [54] A. Graves, “Generating sequences with recurrent neural no. 2, pp. 438–450, 2012. networks,” 2013, http://arxiv.org/abs/1308.0850. 18 Advances in Meteorology [55] F. Weninger, J. Geiger, M. Wollmer, ¨ B. Schuller, and G. Rigoll, “Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments,” Computer Speech & Language, vol. 28, no. 4, pp. 888–902, 2014. [56] A. Graves, Supervised Sequence Labelling with Recurrent Neural Networks, Springer, Berlin, Germany, 2012. [57] F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: continual prediction with LSTM,” Neural Computa- tion, vol. 12, no. 10, pp. 2451–2471, 2000. [58] V. Nair and G. Hinton, “Rectiﬁed linear units improve re- stricted Boltzmann machines,” in Proceedings of Machine Learning Research, pp. 807–814, Sardinia, Italy, May 2010. [59] T. C. Yeh, S. H. Chen, and J. S. Hong, @e Forecast Technique Development Studies on the Typhoon Track, Rainfall and Winds Forecast Over Taiwan Area: A Study on the Imple- mentation of WRF Typhoon Forecasting Component in the Operational Environment of CWB, Funded by Ministry of Science and Technology of Taiwan, Taipei, Taiwan, No. NSC96-2625-Z-052-003 in Chinese, 2008. [60] T. C. Yeh, C. T. Terng, C. S. Lee, and M. J. Yang, “'e forecast technique development studies on the typhoon track, rainfall and winds forecast over taiwan area: a study on the imple- mentation of WRF typhoon forecasting component in the operational environment of CWB (II),” Funded by Ministry of Science and Technology of Taiwan, No. NSC97-2625-M-052- 002 (in Chinese), 2009. [61] D. P. Kingma and J. L. Ba, “Adam: a method for stochastic optimization,” in Proceedings of the International Conference on Learning Representations, pp. 1–13, San Diego, CA, USA, May 2015. [62] F. Han, J. Jiang, Q.-H. Ling, and B.-Y. Su, “A survey on metaheuristic optimization for random single-hidden layer feedforward neural network,” Neurocomputing, vol. 335, pp. 261–273, 2019. [63] A. Sharma, “Guided stochastic gradient descent algorithm for inconsistent datasets,” Applied Soft Computing, vol. 73, pp. 1068–1080, 2018. [64] C. Yao, D. Cai, J. Bu, and G. Chen, “Pre-training the deep generative models with adaptive hyperparameter optimiza- tion,” Neurocomputing, vol. 247, pp. 144–155, 2017. [65] W.-C. Wang, K.-W. Chau, C.-T. Cheng, and L. Qiu, “A comparison of performance of several artiﬁcial intelligence methods for forecasting monthly discharge time series,” Journal of Hydrology, vol. 374, no. 3-4, pp. 294–306, 2009. [66] D. A. Swanson, J. Tayman, and T. M. Bryan, “MAPE-R: a rescaled measure of accuracy for cross-sectional subnational population forecasts,” Journal of Population Research, vol. 28, no. 2-3, pp. 225–243, 2011. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Meteorology Hindawi Publishing Corporation http://www.deepdyve.com/lp/hindawi-publishing-corporation/development-of-stacked-long-short-term-memory-neural-networks-with-ktFGn8ta3J

Loading next page...

References (66)

Ying Zhang, Xiaoping Wang, Huiming Tang (2019)
An improved Elman neural network with piecewise weighted gradient for time series prediction
Neurocomputing, 359
Aqsa Qureshi, Asifullah Khan, Aneela Zameer, A. Usman (2017)
Wind power prediction using deep neural network based meta regression and transfer learning
Appl. Soft Comput., 58
J. Elman (1990)
Finding Structure in Time
Cogn. Sci., 14
Anuraganand Sharma (2018)
Guided Stochastic Gradient Descent Algorithm for inconsistent datasets
Appl. Soft Comput., 73
Alex Graves (2012)
Supervised Sequence Labelling with Recurrent Neural Networks
, 385
Tengfei Liu, Shuangyuan Yu, Baomin Xu, Hongfeng Yin (2018)
Recurrent networks with attention and convolutional networks for sentence representation and classification
Applied Intelligence, 48
D. Carvalho, A. Rocha, M. Gómez-Gesteira, C. Santos (2014)
WRF wind simulation and wind energy production estimates forced by different reanalyses: Comparison with observed data for Portugal
Applied Energy, 117
Chih-Chiang Wei (2012)
Wavelet Support Vector Machines for Forecasting Precipitation in Tropical Cyclones: Comparisons with GSVM, Regression, and MM5
Weather and Forecasting, 27
K. Trachte (2018)
Atmospheric Moisture Pathways to the Highlands of the Tropical Andes: Analyzing the Effects of Spectral Nudging on Different Driving Fields for Regional Climate Modeling
Atmosphere
E. Fiori, A. Parodi, F. Siccardi (2011)
Uncertainty in prediction of deep moist convective processes: Turbulence parameterizations, microphysics and grid-scale effects
Atmospheric Research, 100
Zachary Lipton (2015)
A Critical Review of Recurrent Neural Networks for Sequence Learning
ArXiv, abs/1506.00019
Felix Gers, J. Schmidhuber, Fred Cummins (2000)
Learning to Forget: Continual Prediction with LSTM
Neural Computation, 12
Guqian Pang, Jianchuan He, Yuming Huang, Liuhong Zhang (2019)
A Binary Logistic Regression Model for Severe Convective Weather with Numerical Model Data
Advances in Meteorology
Alex Graves (2013)
Generating Sequences With Recurrent Neural Networks
ArXiv, abs/1308.0850
M. Madhiarasan, S. Deepa (2016)
A novel criterion to select hidden neuron numbers in improved back propagation networks for wind speed forecasting
Applied Intelligence, 44
H. Saghafi, A. Rostami, M. Arabloo (2019)
Evolving new strategies to estimate reservoir oil formation volume factor: Smart modeling and correlation development
Journal of Petroleum Science and Engineering
Q. Zhang, Hui Wang, Junyu Dong, G. Zhong, Xin Sun (2017)
Prediction of Sea Surface Temperature Using Long Short-Term Memory
IEEE Geoscience and Remote Sensing Letters, 14
E. Zhao, Jing Zhao, Liwei Liu, Zhongyue Su, Ning An (2015)
Hybrid Wind Speed Prediction Based on a Self-Adaptive ARIMAX Model with an Exogenous WRF Simulation
Energies, 9
Xiaohui Yuan, Chen Chen, M. Jiang, Yanbin Yuan (2019)
Prediction interval of wind power using parameter optimized Beta distribution based LSTM model
Appl. Soft Comput., 82
C. T. Terng T. C. Yeh
The forecast technique development studies on the typhoon track, rainfall and winds forecast over taiwan area: a study on the implementation of WRF typhoon forecasting component in the operational environment of CWB (II),
D. Lo, Chih-Chiang Wei, En-Ping Tsai (2015)
Parameter Automatic Calibration Approach for Neural-Network-Based Cyclonic Precipitation Forecast Models
Water, 7
F. Molteni, R. Buizza, T. Palmer, T. Petroliagis (1996)
The ECMWF Ensemble Prediction System: Methodology and validation
Quarterly Journal of the Royal Meteorological Society, 122
Michael Kozar, V. Misra, M. Powell (2016)
Hindcasts of Integrated Kinetic Energy in Atlantic Tropical Cyclones: A Neural Network Prediction Scheme
Monthly Weather Review, 144
Liu Tiancheng, Ren Qing-dao-er-ji, Qiu Ying (2019)
Application of Improved Naive Bayesian-CNN Classification Algorithm in Sandstorm Prediction in Inner Mongolia
Advances in Meteorology, 2019
A. Murphy (1993)
What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting
Weather and Forecasting, 8
F. Cassola, F. Ferrari, A. Mazzino (2015)
Numerical simulations of Mediterranean heavy precipitation events with the WRF model: A verification exercise using different approaches
Atmospheric Research, 164
S. Hochreiter, J. Schmidhuber (1997)
Long Short-Term Memory
Neural Computation, 9
Chih-Chiang Wei (2014)
Surface Wind Nowcasting in the Penghu Islands Based on Classified Typhoon Tracks and the Effects of the Central Mountain Range of Taiwan
Weather and Forecasting, 29
Chih-Chiang Wei (2015)
Comparing lazy and eager learning models for water level forecasting in river-reservoir basins of inundation regions
Environ. Model. Softw., 63
S. Davolio, D. Mastrangelo, M. Miglietta, O. Drofa, A. Buzzi, P. Malguzzi (2009)
High resolution simulations of a flash flood near Venice.
Natural Hazards and Earth System Sciences, 9
Yiqi Chu, LI Chengcai, Yefang Wang, Jing Li, Jian Li (2016)
A Long-Term Wind Speed Ensemble Forecasting System with Weather Adapted Correction
Energies, 9
Chih-Chiang Wei (2013)
Soft computing techniques in ensemble precipitation nowcast
Appl. Soft Comput., 13
Sisheng Liang, L. Nguyen, Fang Jin (2018)
A Multi-variable Stacked Long-Short Term Memory Network for Wind Speed Forecasting
2018 IEEE International Conference on Big Data (Big Data)
Minsang Kim, Myung‐Sook Park, J. Im, Seonyoung Park, Myong-in Lee (2019)
Machine Learning Approaches for Detecting Tropical Cyclone Formation Using Satellite Data
Remote. Sens., 11
Chengwei Yao, Deng Cai, Jiajun Bu, Gencai Chen (2017)
Pre-training the deep generative models with adaptive hyperparameter optimization
Neurocomputing, 247
Z. Di, W. Gong, Y. Gan, Chenwei Shen, Q. Duan (2019)
Combinatorial Optimization for WRF Physical Parameterization Schemes: A Case Study of Three-Day Typhoon Simulations over the Northwest Pacific Ocean
Atmosphere
D. Dzebre, Akwasi Acheampong, J. Ampofo, M. Adaramola (2019)
A sensitivity study of Surface Wind simulations over Coastal Ghana to selected Time Control and Nudging options in the Weather Research and Forecasting Model
Heliyon, 5
Fei Han, Jing Jiang, Qinghua Ling, Benyue Su (2019)
A survey on metaheuristic optimization for random single-hidden layer feedforward neural network
Neurocomputing, 335
(2015)
Disaster Survey of Typhoon Soudelor in 2015
Pan Zhao, Jiangfeng Wang, Junrong Xia, Yiping Dai, Yingxin Sheng, Jie-shun Yue (2012)
Performance evaluation and accuracy enhancement of a day-ahead wind power forecasting system in China
Renewable Energy, 43
B. Storm, S. Basu (2010)
The WRF Model Forecast-Derived Low-Level Wind Shear Climatology over the United States Great Plains
Energies, 3
J. Naik, P. Satapathy, P. Dash (2017)
Short-term wind speed and wind power prediction using hybrid empirical mode decomposition and kernel ridge regression
Appl. Soft Comput., 70
A. Rostami, A. Baghban, A. Mohammadi, A. Hemmati-Sarapardeh, S. Habibzadeh (2019)
Rigorous prognostication of permeability of heterogeneous carbonate oil reservoirs: Smart modeling and correlation development
Fuel
Wenchao Wang, K. Chau, Chun-tian Cheng, Lin Qiu (2009)
A comparison of performance of several artificial intelligence
A. Rostami, M. Arabloo, A. Kamari, A. Mohammadi (2017)
Modeling of CO2 solubility in crude oil during carbon dioxide enhanced oil recovery using gene expression programming
Fuel, 210
Liang Guo, Naipeng Li, Feng Jia, Y. Lei, Jing Lin (2017)
A recurrent neural network based health indicator for remaining useful life prediction of bearings
Neurocomputing, 240
Vinod Nair, Geoffrey Hinton (2010)
Rectified Linear Units Improve Restricted Boltzmann Machines
Diederik Kingma, Jimmy Ba (2014)
Adam: A Method for Stochastic Optimization
CoRR, abs/1412.6980
Eugenia Kalnay (2002)
Atmospheric Modeling, Data Assimilation and Predictability
M. Weisman, C. Davis, Wei Wang, Kevin Manning, J. Klemp (2008)
Experiences with 0–36-h Explicit Convective Forecasts with the WRF-ARW Model
Weather and Forecasting, 23
Zhiyuan Wu, Changbo Jiang, B. Deng, Jie Chen, Xiaojian Liu (2019)
Sensitivity of WRF simulated typhoon track and intensity over the South China Sea to horizontal and vertical resolutions
Acta Oceanologica Sinica, 38
E. Cadenas, W. Rivera, R. Campos-Amezcua, Christopher Heard (2016)
Wind Speed Prediction Using a Univariate ARIMA Model and a Multivariate NARX Model
Energies, 9
I. Panapakidis, C. Michailides, Demos Angelides (2019)
A Data-Driven Short-Term Forecasting Model for Offshore Wind Speed Prediction Based on Computational Intelligence
Electronics
Wonmin Byeon, M. Liwicki, T. Breuel (2015)
Scene analysis by mid-level attribute learning using 2D LSTM networks and an application to web-image tagging
Pattern Recognit. Lett., 63
M. Miglietta, R. Rotunno (2012)
Application of Theory to Simulations of Observed Cases of Orographically Forced Convective Rainfall
Monthly Weather Review, 140
F. Weninger, Jürgen Geiger, M. Wöllmer, Björn Schuller, G. Rigoll (2014)
Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments
Comput. Speech Lang., 28
X. Chadee, N. Seegobin, R. Clarke (2017)
Optimizing the Weather Research and Forecasting (WRF) Model for Mapping the Near-Surface Wind Resources over the Southernmost Caribbean Islands of Trinidad and Tobago
Energies, 10
S. Behzadi, C. Prakasvudhisarn, J. Klocker, P. Wolschann, H. Viernstein (2009)
Comparison between two types of Artificial Neural Networks used for validation of pharmaceutical processes
Powder Technology, 195
(2015)
Disaster Survey of Typhoon Soudelor
Q. Hu, Rujia Zhang, Yucan Zhou (2016)
Transfer learning for short-term wind speed prediction with deep neural networks
Renewable Energy, 85
Bitzel Cortez, Berny Carrera, Young-jin Kim, Jae-Yoon Jung (2018)
An architecture for emergency event prediction using LSTM recurrent neural networks
Expert Syst. Appl., 97
Xiaodian Shen, Baolin Jiang, Qimin Cao, Wenshi Lin, Lan Zhang (2019)
Sensitivity of precipitation and structure of Typhoon Hato to bulk and explicit spectral bin microphysics schemes
Advances in Meteorology
Chih-Chiang Wei, P. Peng, Cheng-Han Tsai, Chien-Lin Huang (2018)
Regional Forecasting of Wind Speeds during Typhoon Landfall in Taiwan: A Case Study of Westward-Moving Typhoons
, 9
N. Awan, H. Truhetz, A. Gobiet (2011)
Parameterization-Induced Error Characteristics of MM5 and WRF Operated in Climate Mode over the Alpine Region: An Ensemble-Based Analysis
Journal of Climate, 24
David Swanson, J. Tayman, T. Bryan (2011)
MAPE-R: a rescaled measure of accuracy for cross-sectional subnational population forecasts
Journal of Population Research, 28
Chih-Chiang Wei (2017)
Conceptual weather environmental forecasting system for identifying potential failure of under-construction structures during typhoons
Journal of Wind Engineering and Industrial Aerodynamics, 168

Publisher: Hindawi Publishing Corporation
Copyright: Copyright © 2020 Chih-Chiang Wei. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN: 1687-9309
eISSN: 1687-9317
DOI: 10.1155/2020/5462040
Publisher site: See Article on Publisher Site

Abstract

Hindawi Advances in Meteorology Volume 2020, Article ID 5462040, 18 pages https://doi.org/10.1155/2020/5462040 Research Article Development of Stacked Long Short-Term Memory Neural NetworkswithNumericalSolutionsforWindVelocityPredictions Chih-Chiang Wei Department of Marine Environmental Informatics and Center of Excellence for Ocean Engineering, National Taiwan Ocean University, Keelung City, Taiwan Correspondence should be addressed to Chih-Chiang Wei; ccwei@ntou.edu.tw Received 27 December 2019; Accepted 8 July 2020; Published 23 July 2020 Academic Editor: 'eodore Karacostas Copyright © 2020 Chih-Chiang Wei. 'is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Taiwan, being located on a path in the west Paciﬁc Ocean where typhoons often strike, is often aﬀected by typhoons. 'e accompanying strong winds and torrential rains make typhoons particularly damaging in Taiwan. 'erefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better preparation to mitigate a typhoon’s toll on life and property. For more accurate wind speed predictions during a typhoon episode, we used cutting-edge machine learning techniques to construct a wind speed prediction model. To ensure model accuracy, we used, as variable input, simulated values from the Weather Research and Forecasting model of the numerical weather prediction system in addition to adopting deeper neural networks that can deepen neural network structures in the construction of estimation models. Our deeper neural networks comprise multilayer perceptron (MLP), deep recurrent neural networks (DRNNs), and stacked long short-term memory (LSTM). 'ese three model-structure types diﬀer by their memory capacity: MLPs are model networks with no memory capacity, whereas DRNNs and stacked LSTM are model networks with memory capacity. A model structure with memory capacity can analyze time-series data and continue memorizing and learning along the time axis. 'e study area is northeastern Taiwan. Results showed that MLP, DRNN, and stacked LSTM prediction error rates increased with prediction time (1–6 hours). Comparing the three models revealed that model networks with memory capacity (DRNN and stacked LSTM) were more accurate than those without memory capacity. A further comparison of model networks with memory capacity revealed that stacked LSTM yielded slightly more accurate results than did DRNN. Additionally, we determined that in the construction of the wind speed prediction model, the use of numerically simulated values reduced the error rate approximately by 30%. 'ese results indicate that the inclusion of numerically simulated values in wind speed prediction models enhanced their prediction accuracy. Taiwan even if the typhoon itself does not hit Taiwan. 1. Introduction 'erefore, typhoons constitute a serious natural disaster in A typhoon is a severe natural disaster that aﬀects tropical Taiwan. and subtropical coastal countries, and it occurs most fre- For example, the 2015 Typhoon Soudelor was the most quently in the northwestern Paciﬁc Ocean. Taiwan is to the destructive typhoon that occurred in Taiwan in recent east of the Eurasian Continent and at the western side of the history, with gust intensity exceeding 12 on the Beaufort Paciﬁc Ocean; its climate is intermediate between tropical wind force scale (32.7 m/s). Its strong gusts caused wide- and subtropical climate. 'us, typhoons frequently occur in spread damage to infrastructure, aﬀecting gas supply, power Taiwan, generally in summer and fall. Typhoons aﬀecting and utilities, transportation and communication, and Taiwan typically develop at the sea surface southeast of weather radar stations. Electricity was cut oﬀ in approxi- Taiwan, and most typhoons are accompanied by torrential mately 4.5 million households simultaneously during Ty- rains and strong winds [1]. Such rain and wind add to the phoon Soudelor—the greatest recorded number in recent damage from typhoons, posing a great threat to the trans- history. 'e economic loss from the typhoon was estimated portation, economic, agricultural, and ﬁshery activities in to be as high as US$76 million [2]. 2 Advances in Meteorology results to be explained in terms of physical relationships. 'erefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better NWP models simulate the atmosphere on user-deﬁned grid scales as a moving ﬂuid. 'rough several types of param- preparation to mitigate a typhoon’s toll on life and property. In this study, cutting-edge machine learning (ML) tech- eterizations, NWP models also account for the inﬂuence of niques were used to improve predictive accuracy. In general, subgrid physical processes on grid-resolved motions ML algorithms learn from a huge dataset, improving their [31, 32]. NWP models, such as the Weather Research and ability to identify patterns in the data. Speciﬁcally, ML in- Forecasting model (WRF), have been increasingly popular as volves creating algorithms to make prediction from sets of a low-cost alternative source of data for such assessments of unknown data. Given their ability to perform parameter climate parameters [33]. WRF oﬀers a wide variety of physical and dynamical elements to choose from; these el- adjustment and achieve optimization through self-learning, neural network algorithms in ML are particularly powerful ements must be put together to form model conﬁgurations, with which the model can be run [34]. However, because of [3]. Such algorithms have been extensively used in recent wind speed prediction models, and these models are in- imperfect models and uncertain initial boundary atmo- spheric conditions, errors exist in the NWP output [35, 36]. creasingly data-driven due to developments in ML [4–11]. In the development of neural networks, multilayer perceptron Recent studies have applied the NWP model to typhoons (MLP) networks are a classic approach that are often used and tropical cyclones [37–39]. However, the prediction of and compared with other neural network models. For ex- severe meteorological phenomena (such as typhoons), ample, Wei [12] compared the accuracy of MLP with that of which result from a multiplicity of mutually interacting adaptive network-based fuzzy-inference-system neural multiscale processes, remains a major challenge for NWP networks in the construction of typhoon wind speed pre- systems [25, 40, 41]. 'ese models are typically unable to predict wind intensity with satisfactory accuracy, even at diction models. Deep learning has become possible due to the expo- short forecast times and a high horizontal resolution [42, 43]. Some studies have tried combining NWP with ML nential increase in computing power in recent years. 'is approach is the further derivation of multiple neural layers models to develop an integrated climate prediction model. For example, Zhao et al. [44] evaluated the performance and from the original neural layers of a model. Such derivation improves an algorithm’s ability to learn, better approxi- enhanced accuracy of a day-ahead wind-power forecasting mating the complex neural network structure of a human system. 'e system comprised artiﬁcial neural networks and being. For example, Hu et al. [13] formulated multilayer an NWP model. deep neural networks that were trained using data from Furthermore, to enhance the predictive accuracy of data-rich wind plants. 'ese networks extracted wind speed constructed ML models, other researchers have used nu- patterns, and the mapping was ﬁnely tuned using data from merically simulated results as input data for the construction of ML models. For example, Zhao et al. [45] developed the newly constructed wind plants. Tiancheng et al. [14] pro- posed a sandstorm prediction method that considered both ARIMAX model, where wind speed results from the WRF simulation were chosen as an exogenous input variable. the eﬀect of atmospheric movement and ground factors on sandstorm occurrence, called improved naive Bayesian However, to the best of our knowledge, in the literature on convolutional neural network classiﬁcation algorithm. short-term wind speed prediction for typhoons (or tropical In the ﬁeld of neural networks, recurrent neural networks cyclones), numerically simulated values have seldom been (RNNs), which can analyze sequential (or time-series) data, used as an input variable for ML prediction models. have recently been developed [15–20]. RNNs are con- 'erefore, in relation to the construction of a ML-based nectionist models with the ability to selectively pass infor- wind speed prediction model, we evaluated the improve- mation across sequence steps while processing sequential data ments to predictive accuracy aﬀorded by the use of nu- one element at a time [21]. 'erefore, RNNs are important, merically simulated values (by comparing between its use and nonuse). especially in the analysis of sequential data. A particular type of RNNs is long short-term memory (LSTM), a class of ar- 'erefore, our study has two primary aims: (1) develop an ML- and neural network-based wind speed prediction tiﬁcial neural networks where connections between units form a directed cycle [22] introduced the LSTM primarily to model and compare the predictive accuracy of various overcome the problem of diminishing gradients. LSTM neural network-based algorithms and (2) evaluate the im- creates an internal network state that allows it to exhibit provements to predictive accuracy aﬀorded by the use of dynamic temporal behavior. Unlike feedforward neural numerical solutions obtained from NWP models (by networks, RNNs can use their internal memory to process comparing between its use and nonuse) in a typhoon-surface arbitrary sequences of input [23]. To model the time series of wind-speed prediction model. wind speed data, Byeon et al. [24] developed the LSTM for the prediction of typhoon wind speeds. Hence, feature en- 2. Methodology and Algorithms hancement from RNNs has been explored in wind prediction. 'e widespread application of ensembles in numerical Figure 1 illustrates the ﬂow of the construction, involving weather prediction (NWP) has helped researchers improve NWP numerical solutions, of our neural network-based weather forecasts [25–30]. Numerical models can be used to typhoon wind-velocity prediction model. In the ﬁrst stage, calculate all climate parameters through atmospheric dy- we collected data on the typhoon characteristics and ground namics and numerical methods, allowing for simulated wind speed of the research area’s historical typhoon events. Advances in Meteorology 3 Collect data associated with historical typhoon events Perform the wind field simulation of typhoons with an NWP numerical model Refine data of typhoon characteristics and surface wind Generate wind-speed speed observations simulation solutions Preprocess the datasets and split data into training-validation and testing subsets Conduct the neural networks-based wind speed prediction model using MLP, DRNN, and stacked LSTM Predict wind speed using different neural networks-based prediction models Evaluate the forecast accuracy in ML models using different neural networks-based algorithms Evaluate of forecast accuracy in ML models using inputs with and without NWP numercial solutions Verify the optimal neural networks-based wind velocity prediction models Figure 1: Flowchart of neural network-based typhoon wind velocity prediction model using NWP numerical solutions. In the second step, wind speed solutions were obtained from forecast accuracy of the stacked LSTM model was evaluated a wind ﬁeld simulation of typhoons, involving an NWP against other neural networks. We also evaluated whether numerical model. In our study, we also employed a WRF forecast accuracy in ML-based models improved when NWP numerical model to simulate circulation distribution, thus numerical solutions were used as input. obtaining the wind speed values in the research area. Two datasets could be built, one comprising typhoon data and the 2.1. Frameworks Underlying the Proposed Neural Networks. other comprising wind simulation results from the NWP model. 'e datasets were split into testing and training- In this section, we describe the neural network-based ar- chitectures, which used the MLP, DRNN, and stacked LSTM validation subsets. 'e training-validation set was used for algorithms, that were adopted for model construction. As the learning of several ML-based wind velocity prediction illustrated in Figure 2(a), the MLP is a typical type of models, and the testing set was used for the identiﬁcation of feedforward backpropagation neural network that uses the optimal prediction model among these models. 'e best model among a set of ML neural network-based processing units placed in the input, hidden, and output layers [47–49]. Each unit (with an associated weight) in a models, involving MLP, DRNN, and stacked LSTM, was determined. According to [46], real-time dynamics consti- layer is connected to the units in adjacent layers [50, 51]. In the study, to enhance learning eﬃcacy—and by implication, tute the most challenging aspect of wind speed forecasting. We determined the stacked LSTM model to be the best for approximation, and prediction accuracy—we added hidden layers to a simple MLP neural network; the MLP was trained wind speed forecasting due to its appropriate handling of long- and short-term time dependency. In the ﬁnal stage, the through backpropagation. Generally, the weight updates 4 Advances in Meteorology Predicted V Predicted V Predicted H t+i t+i t+i Output layer Output layer Output layer Dropout layer Dropout layer Hidden layer Activation fun Summing fun Context Hidden layer units t f Hidden layer Activation fun Summing fun Context Hidden layer units Input layer Input layer t Inputs for wind velocity prediction Input layer Inputs for wind velocity prediction Inputs for wind velocity prediction (a) (b) (c) Figure 2: Architecture of (a) MLP, (b) DRNN, and (c) stacked LSTM neural networks. between layers are calculated in terms of the stochastic the ﬁrst LSTM layer produces sequence vectors used as the gradient descent [52]. Speciﬁcally, input of the subsequent LSTM layer. Moreover, the LSTM layer receives feedback from its previous time step, thus zE Δw (t + 1) � βΔw (t) + η , (1) allowing for the capturing of data patterns. 'e dropout ij ij zw ij layer also excludes 10% of the neurons to avoid overﬁtting. 'e basic structure of LSTM, as illustrated in the LSTM where w (t) is the weight set connecting layers i and j at ij layer in Figure 2(c), comprises an input gate i , output gate o , t t time t, Δw is the weight correction, η is the learning rate, β is forget gate f , and memory cell c . A single LSTM layer has a t t a momentum coeﬃcient, and E is a cost function that in- second-order RNN architecture that excels at storing se- dicates the diﬀerence between the target and predicted quential short-term memories and retrieving them at many values. In particular, η and β are hyperparameters for time steps later [55]. An LSTM network is identical to a adjusting the spacing of weight correction. standard RNN, with the exception of the summation units in As illustrated in Figure 2(b), the multilayers of the RNN the hidden layer being replaced by memory blocks [56]. structure comprise an input layer, multiple recurrent layers, Equations (2)–(6) describe how output values are updated at and an output layer. When the length of recurrent layer is 1, each step [22, 46, 57]. Speciﬁcally, the framework is a simple RNN. Here, the recurrent network is based on the networks developed by [53]. In the RNN, the f � σ􏼐W · x + U · h + b 􏼑, (2) t f t f t−1 f hidden units are connected to context units; in the successive time step, the units feed back into the hidden units. 'e i � σ W · x + U · h + b , (3) t i t i t−1 i hidden state at any time step can contain information from an (almost) arbitrarily long context window [21]. 'e DRNN o � σ W · x + U · h + b , (4) model framework has multiple recurrent layers before the t o t o t−1 o forwarding to a dropout layer and output layer at the ﬁnal c � f · c + i · σ W · x + U · h + b 􏼁, (5) output. In this paper, the dropout layer excludes 10% of the t t t−1 t c t c t−1 c neurons to avoid overﬁtting. A stacked LSTM architecture is deﬁned as an LSTM h � o · σ c􏼁 , (6) t t t model that comprises multiple LSTM layers (Figure 2(c)). where x is the input vector; σ is the activation function; W , 'e stacked LSTM, also known as deep LSTM, was ﬁrst t f W , W , W , U , U , U , and U are the weight vector terms; b , formulated by [54] and was applied to speech recognition i o c f i o c f b , b , and b are the corresponding bias terms; and h and h problems. Similar to the framework underpinning the i o c t −1 are the current and previous hidden vectors, respectively. DRNN model, the stacked LSTM model uses multiple LSTM In this study, because the function converges quickly and layers that are stacked before the forwarding to a dropout has no problems with a vanishing gradient, we used the layer and output layer at the ﬁnal output. In a stacked LSTM, Recurrent Recurrent layer layer LSTM layer LSTM layer Advances in Meteorology 5 ° ° grid spacing for the coarser domain (118 ReLU activation function in the middle layers (the hidden, E–126 E, ° ° recurrent, and LSTM layers) of the aforementioned three 21 N–29 N) was 15 km, and that of the ﬁner domain ° ° ° ° models. 'e ReLU is deﬁned as the positive part of its ar- (121 E–123 E, 24 N–26 N) was 3 km. Both domains had 32 gument; that is, f(x) � max (0, x), where x is the input to a vertical levels. 'is study used the WRF physical param- neuron. 'e ReLU function has been recently used to replace eters recommended by [59, 60]. 'ey studied the tracks and the sigmoid function in neural networks, resulting in good rainfall of typhoons that have invaded Taiwan in addition performance and fast training times [58]. Moreover, in these to other physical parameters, and they conducted wind proposed models, the input layers receive the observed and forecasts using the WRF model. 'ey determined the simulated meteorological values as their input. following physical parameters to be suitable for wind speed forecasts: with regard to the planetary layer, those from the Yonsei University (YSU) scheme; with regard to micro- 3. Study Area and Data physics, those from the WRF Single-Moment 5-Class Taiwan is located at the western Paciﬁc Ocean, a region (WSM5) scheme; for cumulus parameterization, those frequently in the path of typhoons. Typhoons in Taiwan are from the Kain–Fritsch scheme; and with regard to long- often accompanied by torrential rains, which severely en- wave radiation, those from the Rapid Radiative Transfer danger life and property. 'us, more accurate regional wind Model scheme. speed prediction is needed to improve typhoon protection. After our simulation of a large number of typhoon In this study, we chose northeastern Taiwan as the study area events, the WRF model generated wind velocity simulations (Figure 3), wherein Taipei City and Yilan City are the two at eight ground gauge stations. In the veriﬁcation of sub- major cities. sequent WRF wind outcomes, the output objective for the WRF model was the simulated wind value at an altitude of 10 m, because the ground gauge-station wind speed reached 3.1. Data from Gauge and Typhoon Weather Stations. the observation value at an altitude of 10 m. Figure 5 displays According to typhoon data from the Central Weather Bu- the scatter plot of the WRF model simulation values and reau (CWB), between 2000 and 2018, 29 typhoons have observed gauge-station values. According to Figure 5, sev- directly struck Taiwan (Table 1; 2003, 2011, and 2018 were eral observation stations (Keelung, Anbu, Yilan, and Su-ao) excluded from the table because typhoons did not directly have consistently higher wind speeds. 'ese higher wind aﬀect Taiwan in those years). 'e tracks of these historical speeds are caused by their surrounding topographies—which typhoons are illustrated in Figure 4. 'ese typhoons can be cause them to face windwards—in conjunction with the classiﬁed according to the Saﬃr–Simpson wind scale anticlockwise circulatory ﬂow characteristic of typhoons. depending on the intensity of their maximum sustained Pengjiayu Station is similar to the four stations. However, winds. A tropical storm has the wind speed range of being located on an island on the sea around northeastern 18–33 m/s, and category 1, 2, and 3 typhoons have the wind Taiwan, the station is directly aﬀected only by a typhoon’s speed ranges of 33–43 m/s, 43–50 m/s, and 50–58 m/s, re- circulatory ﬂow and not the station’s surrounding topog- spectively. In these collected typhoons, the numbers for the raphy; this station also records higher wind speed values. 'e category 1, 2, and 3 typhoons were 9, 5, and 8, respectively, Taipei, Banqiao, and Tamsui Stations are located within the and the number for the tropical storm was 7. Tamsui Basin and are thus fully enclosed by mountains. 'e data included two types of datasets: one on typhoon Because the circulatory ﬂow of typhoons can be easily dynamic characteristics and another (comprising 2621 disrupted by the surrounding topography, the wind speed hourly records) on surface wind speed observations. Surface values observed at these stations are slightly lower. wind speed data were released by the CWB and measured For the four windward observation stations, wind speed from eight gauge stations: Anbu, Banqiao, Keelung, Pen- simulation values were lower and approximately equal to gjiayu, Su-ao, Taipei, Tamsui, and Yilan (Figure 3). In ad- actual values for high and low wind speeds, respectively. For dition, data on typhoon dynamics were released by the the three observation stations in the Tamsui Basin, wind CWB. 'e data comprised six variables: pressure at the speed simulation values were approximately equal to the typhoon center, latitude of the typhoon center, longitude of actual values, for all wind speeds. the typhoon center, typhoon radius (i.e., the distance from To evaluate the simulation outcomes, this study used the typhoon center for points with winds greater than 15.5 m/s), mean absolute error (MAE) and root mean squared error moving speed of the typhoon, and maximum wind speed at (RMSE) that are deﬁned as follows: the typhoon center. 􏼌 􏼌 􏼌 􏼌 􏼌 􏼌 MAE � 􏽘 O − Y , 􏼌 􏼌 i i i�1 3.2. Data from Numerical Solutions. WRF model simula- tions using typhoon data were used to generate wind speed 􏽶�� (7) values at each meteorological station. To set the initial ﬁeld n and boundary conditions, we used data from the Final RMSE � 􏽘 O − Y􏼁 , i i i�1 Operational Global Analysis dataset. 'e dataset is a part of the Global Data Assimilation System of the US Govern- where Y is the estimated value of record i, O is the ob- ment’s National Center for Environmental Protection. 'e i i servation of record i, and n is the total number of records. grid was set to be a two-domain nested grid. 'e horizontal 6 Advances in Meteorology Figure 3: Map of Taiwan’s northeastern region and the meteorological stations therein. Table 1: Typhoons aﬀecting the study area from 2000 to 2017 (29 120°E 130°E 140°E 150°E 160°E 40°N incidents). Year Typhoon Period (UTC) 2000 Kai-Tak 6–10 Jul 2001 Toraji 28–31 Jul 30°N 2001 Nari 10–20 Sep 2002 Nakri 9–11 Jul 2004 Mindulle 29 Jun–3 Jul 2004 Aere 23–26 Aug 20°N 2004 Haima 12-13 Sep 2004 Nock-Ten 24–26 Oct 2005 Haitang 17–19 Jul 2005 Longwang 1-2 Oct 10°N 2006 Bilis 12–14 Jul 2007 Sepat 16–19 Aug 2007 Krosa 5–8 Oct 2008 Kalmaegi 16–19 Jul Figure 4: Tracks of historical typhoon incidents that aﬀected 2008 Fung-Wong 27–29 Jul Taiwan. 2008 Sinlaku 10–16 Sep 2008 Jangmi 27–30 Sep 2009 Morakot 6–9 Aug 2010 Namtheun 30-31 Aug lowest error values; their MAE and RMSE values 2012 Saola 30 Jul–3 Aug had errors in the ranges of 1.092–1.362 m/s and 2013 Soulik 10–14 Jul 1.422–1.654 m/s, respectively. 2013 Trami 16–22 Aug 2013 Kong-Rey 27–30 Aug 2014 Matmo 21–23 Jul 4. Modeling 2014 Fung-Wong 19–22 Sep 2015 Soudelor 6–9 Aug We used ML-based models to construct our wind speed 2015 Dujuan 25–29 Sep prediction model for our study area. Observation stations 2016 Megi 25–28 Sep at Taipei and Yilan were chosen as the test location. When 2017 Nesat 26–30 Jul constructing the model, the attribute data entered into the model included data on typhoon dynamics, data from ground meteorological observation stations, and data 'e performance of the estimation indicators (MAE and obtained from the aforementioned simulation. In this RMSE) in terms of errors is presented in Figure 6. 'e study, we performed data splitting for all typhoon epi- estimation indicators for the Pengjiayu Station had the sodes. Our training-validation set comprised data on 23 highest error values (MAE � 3.547 m/s and RMSE � 5.139 m/s), typhoon episodes between 2000 and 2013 (2093 records in followed by those of the four observation stations (Keel- total); our testing set comprised data on six typhoon ung, Anbu, Yilan, and Su-ao) facing windward—their episodes between 2014 and 2017 (528 records in total). MAE and RMSE values had errors in the ranges Model training and validation were performed through 1.623–2.020 m/s and 2.405–2.728 m/s, respectively. 'e 10-fold cross-validation, in which the training set was estimation indicators for the three observation stations divided into 10 subsamples, one of which was retained for (Taipei, Banqiao, and Tamsui) in the Tamsui Basin had the model veriﬁcation, and the other nine were used for model Advances in Meteorology 7 10 45 8 18 4 20 y = 0.653x + 2.0423 R = 0.5337 10 4 y = 0.4348x + 2.7207 y = 0.4401x + 3.7208 2 2 2 5 R = 0.5456 R = 0.7781 0123456789 10 11 12 13 Pengjiayu 02468 10 12 14 16 18 20 22 24 26 28 0 5 10 15 20 25 30 35 40 45 50 55 –1 –1 Observation (m·s ) –1 Observation (m·s ) Observation (m·s ) (g) (h) (a) 13 24 y = 0.626x + 1.5413 R = 0.6214 1 4 y = 0.4039x + 2.5521 0 2 R = 0.6099 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 –1 Observation (m·s ) 0 24 6 810 12 14 16 18 20 22 24 26 –1 (f ) Observation (m·s ) (b) y = 0.5563x + 1.7758 2 35 R = 0.5677 0 30 4 y = 0.4428x + 1.84 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 –1 R = 0.6311 Observation (m·s ) 25 (e) 0 24 6 810 12 14 16 18 20 22 24 26 28 –1 Observation (m·s ) (c) 5 y = 0.3963x + 2.3769 R = 0.6873 0 5 10 15 20 25 30 35 –1 Observation (m·s ) (d) Figure 5: Scatterplots depicting observations vs. WRF simulations at eight stations: (a) Pengjiayu, (b) Keelung, (c) Yilan, (d) Su-ao, (e) Banqiao, (f) Taipei, (g) Tamsui, and (h) Anbu. training. In the veriﬁcation process, each subsample must 'ese hyperparameters were evaluated through trial be veriﬁed. and error. First, the number of neurons in a middle layer As mentioned in Section 2.1, multilayer neural networks was adjusted from 10 to 100 until the curves of the RMSE were used to construct our ML neural network-based of the RMSE errors were approximated. 'e middle layers models. We used the adaptive moment estimation optimi- of the MLP, DRNN, and stacked LSTM models are the zation algorithm (also known as the Adam optimizer) to hidden, recurrent, and LSTM layers, respectively. Sub- sequently, the prediction errors corresponding to the optimize the momentum and learning rate [61]. Generally, the Adam optimizer is more broadly applied in neural minimum RMSE were obtained in the form of the optimal networks [62–64]. 'e Adam optimizer can be used instead solution for the number of neurons in the middle layer. of the classical stochastic gradient descent procedure to When the lead time was 1 h, for the Taipei Station, the iteratively update network weights based on training data. optimal numbers of neurons were 30, 50, and 50 for the We also calibrated the hyperparameters, speciﬁcally, the MLP, DRNN, and LSTM models, respectively; for the number of neurons in a middle layer and the length of the Yilan Station, those numbers for the three models were 40, middle layers in the MLP, DRNN, and stacked LSTM 30, and 30, respectively (Figures 7(a) and 8(a)). Subse- models. quently, we calibrated the length of the middle layers in –1 –1 Simulation (m·s ) Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) –1 Simulation (m·s ) 8 Advances in Meteorology Pengjiayu Keelung Anbu Yilan Su-ao Taipei Banqiao Tamsui Station MAE RMSE Figure 6: Simulation performance in terms of prediction errors. 1.30 1.30 1.25 1.25 1.20 1.20 1.15 1.15 1.10 1.10 1.05 1.05 10 20 30 40 50 60 70 80 90 100 1 2 34 5 6 79 8 10 MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 7: Sensitivity of model parameters on the Taipei gauge: (a) number of neurons in a middle layer and (b) length of middle layers. 2.10 2.10 2.00 2.00 1.90 1.90 1.80 1.80 1.70 1.70 1.60 1.60 1.50 1.50 10 20 30 40 50 60 70 80 90 100 1 2 34 5 6 79 8 10 MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 8: Sensitivity of model parameters on the Yilan gauge: (a) number of neurons in a middle layer and (b) length of middle layers. the networks, adjusting them to be between 1 and 10 the three models were 7, 6, and 5, respectively layers. For the Taipei Station, the optimal lengths of the (Figures 7(b) and 8(b)). middle layers were 7, 5, and 4 for the MLP, DRNN, and Using the aforementioned method, we conducted pa- LSTM models, respectively; for the Yilan Station, those for rameter testing for the prediction models. Tests were RMSE (m/s) RMSE (m/s) Errors (m/s) RMSE (m/s) RMSE (m/s) Advances in Meteorology 9 3.0 1.8 2.5 1.5 2.0 1.2 1.5 0.9 1.0 0.6 0.5 0.3 0.0 0.0 123456 123456 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) Figure 9: Performance levels of neural network-based predictions for lead times between 1 and 6 h using a training-validation set for the (a) Taipei Station and (b) Yilan Station. Soudelor Typhoon Megi Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (a) Soudelor Typhoon Megi 16 Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (b) Figure 10: Continued. RMSE (m/s) Wind speed (m/s) Wind speed (m/s) RMSE (m/s) 10 Advances in Meteorology Soudelor Typhoon Megi Nesat Dujuan Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (c) Figure 10: Predicted results of typhoons (2014–2017) in the testing set on the Taipei gauge for the lead times of (a) 1 h, (b) 3 h, and (c) 6 h. Dujuan Soudelor Typhoon Megi 28 Nesat 24 Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (a) Dujuan Soudelor 30 Typhoon Megi Nesat Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (b) Figure 11: Continued. Wind speed (m/s) Wind speed (m/s) Wind speed (m/s) Advances in Meteorology 11 30 Dujuan Soudelor Typhoon Megi 28 Nesat Matmo Fung-Wong 1 25 49 73 97 121 145 169 193 217 241 265 289 313 337 361 385 409 433 457 481 505 Time series (h) OBS DRNN MLP Stacked LSTM (c) Figure 11: Predicted results of testing typhoons (2014–2017) on the Yilan gauge for the lead times of (a) 1 h, (b) 3 h, and (c) 6 h. 2.0 3.0 1.8 2.5 1.6 1.4 2.0 1.2 1.0 1.5 0.8 1.0 0.6 0.4 0.5 0.2 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) 3.5 1.2 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (c) (d) Figure 12: Performance measures at lead times from 1 to 6 h on the Taipei gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. MAPE MAE (m/s) Wind speed (m/s) RMSPE RMSE (m/s) 12 Advances in Meteorology 2.7 3.5 2.4 3.0 2.1 1.8 2.5 1.5 2.0 1.2 1.5 0.9 1.0 0.6 0.5 0.3 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (a) (b) 1.6 5.0 4.5 1.4 4.0 1.2 3.5 1.0 3.0 0.8 2.5 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 1 23 4 5 6 1 23 4 5 6 Lead time (h) Lead time (h) MLP MLP DRNN DRNN Stacked LSTM Stacked LSTM (c) (d) Figure 13: Performance measures at lead times from 1 to 6 h on the Yilan gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. conducted using the training-validation set at six hourly lead the Yilan Station were between 9.9 m/s (Fung-Wong) and times between 1 and 6 h. 'e RMSE performance values (for 26.8 m/s (Soudelor). As mentioned, the windward Yilan Station is more susceptible to the circular ﬂow of the all lead times) for the three models are presented in Figure 9. 'e stacked LSTM model outperformed the MLP and typhoon, unlike the Taipei Station, which is shielded by DRNN models for all lead times and for both the Taipei and the mountains of the Tamsui Basin. 'erefore, wind speed Yilan Stations. We further tested and ﬁne-tuned these was consistently higher at the Yilan station. models using the testing set to conﬁrm the accuracy and Figures 10(a)–10(c) illustrate the prediction results for the feasibility of each model. Taipei Station at the lead times of 1, 3, and 6 h, respec- tively; for all prediction models, prediction accuracy was inversely related to the lead time. 'e data for the Yilan 5. Evaluation Station exhibited a similar trend (Figures 11(a)−11(c)). Such a situation is not atypical for prediction models. 5.1. Predictions of Forecasting Horizons. We tested and Intuitively, the further a model predicts into the future, evaluated the MLP, DRNN, and stacked LSTM models the harder it is to obtain useful, real-time features for using the testing set. 'e testing set comprised data prediction. 'erefore, to evaluate predictive accuracy, we (measured at the Yilin and Taipei Stations) on six typhoon used the error evaluation indicators to quantify each episodes that occurred between 2014 and 2017. Figures 10 model’s rate of error in predicting the subsequent hour’s and 11 illustrate the time-series charts of the simulated wind speed. and observed wind speed values for these typhoon epi- To compute term-by-term comparisons of the relative sodes. 'e six typhoon episodes were Matmo in 2014, error in the prediction with respect to the actual value of the Fung-Wong in 2014, Soudelor in 2015, Dujuan in 2015, variable, the mean absolute percentage error (MAPE) and Megi in 2016, and Nesat in 2017. According to root mean square percentage error (RMSPE) were calculated. Figure 10(a), the maximum observed wind speeds for the MAPE and RMSPE can be calculated using the following Taipei Station were between 7.8 m/s (Fung-Wong) and formulae: 14.9 m/s (Soudelor). According to Figure 11(a), those for MAPE MAE (m/s) RMSPE RMSE (m/s) Advances in Meteorology 13 Table 2: Average performance measures of absolute error terms (MAE and RMSE) and relative error terms (MAPE and RMSPE) for 1–6 h predictions. Station Measure MLP DRNN Stacked LSTM MAE (m/s) 1.241 1.070 0.928 RMSE (m/s) 1.666 1.439 1.228 Taipei MAPE 0.730 0.599 0.560 RMSPE 2.129 1.764 1.593 MAE (m/s) 1.863 1.714 1.528 RMSE (m/s) 2.680 2.489 2.185 Yilan MAPE 1.151 0.974 0.927 RMSPE 3.643 3.030 2.847 2.0 3.0 1.8 2.5 1.6 1.4 2.0 1.2 1.0 1.5 0.8 1.0 0.6 0.4 0.5 0.2 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (a) (b) 3.5 1.2 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (c) (d) Figure 14: Performance measures of predictive accuracy for the use and nonuse of WRF simulation values on the Taipei gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. 􏼌 􏼌 n 􏼌 􏼌 Generally, MAPE is used to express the MAE as a 􏼌 􏼌 1 O − Y i i 􏼌 􏼌 MAPE � 􏽘􏼌 􏼌, percentage of the observations. MAPE is an unbiased sta- 􏼌 􏼌 􏼌 􏼌 n O i�1 tistic for measuring the predictive capability of a model [65]. 􏽶�� RMSPE has the same properties as the RMSE but is (8) expressed as a percentage [66]. 1 O − Y i i RMSPE � 􏽘 􏼠 􏼡 . Figures 12 and 13 illustrate the performance of each n O i�1 model in terms of the four error indicators (MAE, RMSE, MAPE MAE (m/s) RMSPE RMSE (m/s) 14 Advances in Meteorology 3.5 4.5 3.0 4.0 2.5 3.5 3.0 2.0 2.5 1.5 2.0 1.5 1.0 1.0 0.5 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (a) (b) 2.0 5.5 1.8 5.0 1.6 4.5 1.4 4.0 1.2 3.5 3.0 1.0 2.5 0.8 2.0 0.6 1.5 0.4 1.0 0.2 0.5 0.0 0.0 123456 123456 Lead time (h) Lead time (h) With WRF simulation With WRF simulation Without WRF simulation Without WRF simulation (c) (d) Figure 15: Performance measures of predictive accuracy for the use and nonuse of WRF simulation values on the Yilan gauge: (a) MAE, (b) RMSE, (c) MAPE, and (d) RMSPE. MAPE, and RMSPE) for all lead times and for both Taipei network unit that can memorize numerical values of and Yilan Stations. For the MLP model, the error rate was diﬀerent time lengths (to determine the quantity of useful steeply and positively related to the lead time. However, for information). Additionally, a gate in the LSTM blocks can the DRNN and stacked LSTM models, this relation was help determine if the input is important enough to be remembered and if the input can be exported as outputs. positive but slighter. To explain the results using the RMSPE error as an example, the RMSPE error increased as the lead Because the data involved in wind speed prediction are time increased. For the Taipei Station and for any given lead sequential, the decision on how far back the retention of time, the MLP model exhibited the steepest increase in error, memory data should go becomes consequential for pre- followed by the DRNN and RMSPE models. Speciﬁcally, dictive accuracy. By contrast, because the MLP model has when the lead time increased from 1 to 6 h, RMSPE in- no memory capacity, its predictions are based on limited creased from 1.494 to 2.616, from 1.386 to 2.181, and from and currently available information, thus decreasing its 1.205 to 2.030 for the MLP, DRNN, and stacked LSTM predictive accuracy. For the DRNN model, although it can models, respectively. Similar results were obtained for the receive memory information from long ago, the lack of a gate ﬁlter system can cause the receiving of too much (i.e., Yilan Station. When the lead time increased from 1 to 6 h, RMSPE increased from 3.391 to 4.142, from 2.409 to 3.613, overly long) memory information, potentially under- and from 2.127 to 3.457 for the MLP, DRNN, and stacked mining predictive accuracy. LSTM models, respectively. Table 2 presents the average measurement values of the 5.2. Evaluation of Forecast Eﬃciency with and without Nu- four error indicators for all lead times. In terms of the two unbiased statistical indicators, MAPE and RMSPE, the merical Solutions. We evaluated whether the use of WRF simulation values as input aﬀects the model’s accuracy in stacked LSTM and MLP models exhibited the most and least favorable performance, respectively. Stacked LSTM’s predicting wind speed. We focused on the stacked LSTM model because it was the best performing model. Figures 14 superiority is attributable to its model structure. Specif- ically, LSTM is a neural network that contains LSTM and 15 illustrate the performance of the stacked LSTM model in terms of the four error indicators (MAE, RMSE, blocks. LSTM blocks can be described as a type of smart MAPE MAE (m/s) RMSPE RMSE (m/s) Advances in Meteorology 15 45 45 40 40 35 35 30 30 25 25 20 20 15 15 10 10 5 5 0 0 123456 123456 Lead time (h) Lead time (h) Taipei Station Taipei Station Yilan Station Yilan Station (a) (b) Figure 16: Improvement rate of predictive accuracy in terms of MAE and RMSE, comparing the use and nonuse of WRF simulation values for the Taipei and Yilan Stations. MAPE, and RMSPE) for all lead times. We compared 6. Conclusions forecast eﬃciency with and without WRF simulated values. To accurately predict the wind speed of future typhoons, we Data for the Taipei and Yilan stations are presented in constructed a typhoon wind speed prediction model using Figures 14 and 15, respectively. cutting-edge ML techniques. RNNs have been recently We determined that the use of WRF simulation values as developed as a type of neural network that can analyze input increased the model’s predictive accuracy. 'erefore, sequential data. 'e structure of such networks facilitates the the use of numerically simulated values as a part of the input eﬀective processing of wind speed-relevant climate data over data aids in the reduction of predictive error. an extended period. 'at is, the structure imbues RNN We deﬁne the improvement rates for the MAE and models with long-term memory capacity. 'erefore, such RMSE (denoted IR and IR , respectively) as follows: MAE RMSE networks are suitable for predicting typhoon wind speeds. MAE − MAE 􏼁 LSTM is a type of RNN that allows the user to decide the with without IR � , MAE memory time’s length. Additionally, LSTM gives users the MAE without option to ﬁlter output results, increasing LSTM’s predictive (9) RMSE − RMSE 􏼁 accuracy. According to current developments in deep with without IR � , RMSE learning, learning performance is enhanced when the layers RMSE without of neural networks are deepened. 'erefore, we used deep where MAE and MAE are the MAE results on the learning neural networks in this study. Additionally, we with without use and nonuse of WRF simulation values, respectively, and compared the performance of three types of RNNs—MLP, RMSE and RMSE are the RMSE results on the use DRNN, and stacked LSTM—in predicting wind speed with without and nonuse of the WRF simulation values, respectively. values. 'ese three types of model structure diﬀer by their Figure 16 details the improvement rates for the Taipei memory capacity: MLPs are model networks with no and Yilan Stations for all lead times. For the Taipei Station, memory capacity, whereas DRNNs and stacked LSTM are IR ranged between 25.5% and 29.7%, and IR ranged MAE RMSE model networks with memory capacity. between 27.0% and 30.8%. For the Yilan Station, IR We chose northeastern Taiwan as the study area, and the MAE ranged between 26.4% and 36.3%, and IR ranged be- RMSE observation stations at Taipei and Yilan were selected as the tween 28.4% and 35.7%. Generally, the average IR (for all study subjects. 'e results indicated that for both the Taipei MAE lead times) was 27.3% and 30.3% for the Taipei and Yilan and Yilan Stations and for the MLP, DRNN, and stacked Stations, respectively, and the average IR (for all lead LSTM models, prediction error is positively related to the RMSE times) was 28.7% and 31.1% for the Taipei and Yilan Stations, prediction lead time (of which there were six, one for each respectively. For both the Taipei and Yilan Stations, im- hour between 1 and 6 h). In other words, the lead time is provement was demonstrated in predictive accuracy, al- inversely related to predictive accuracy. Model networks though the improvement rate was higher for the Yilan with memory capacity (DRNN and stacked LSTM) are more Station. In summary, although wind speed was diﬃcult to accurate than those without memory capacity (MLP). predict accurately at the Yilan Station, the use of WRF Stacked LSTM was also discovered to be more accurate than simulation values increased the predictive accuracy of the DRNN. Stacked LSTM is better than the traditional RNN model at the Yilan Station. method. Because stacking the LSTM’s hidden layers deepens IR (%) MAE IR (%) MAE 16 Advances in Meteorology [5] M. Kim, M. Park, J. Im, S. Park, and M. I. Lee, “Machine the model, model accuracy is increased; LSTM is closer to learning approaches for detecting tropical cyclone formation being a deep learning technique than its counterparts. 'us, using satellite data,” Remote Sensing, vol. 11, 2019. stacked LSTMs is a stable technique for overcoming [6] M. E. Kozar, V. Misra, and M. D. Powell, “Hindcasts of in- problems in wind velocity prediction. tegrated kinetic energy in Atlantic tropical cyclones: a neural Finally, we examined if model accuracy is increased by network prediction scheme,” Monthly Weather Review, the use of WRF simulation values as an input variable in the vol. 144, no. 12, pp. 4591–4603, 2016. wind speed prediction model. 'e stacked LSTM model was [7] M. Madhiarasan and S. N. Deepa, “A novel criterion to select used in the evaluation, given its best performance in most hidden neuron numbers in improved back propagation evaluation tests of the current study. Evaluating predictive networks for wind speed forecasting,” Applied Intelligence, accuracy in terms of the four error indicators, namely, MAE, vol. 44, no. 4, pp. 878–893, 2016. RMSE, MAPE, and RMSPE, veriﬁed that the inclusion of the [8] J. Naik, P. Satapathy, and P. K. Dash, “Short-term wind speed numerically simulated input greatly increased model ac- and wind power prediction using hybrid empirical mode decomposition and kernel ridge regression,” Applied Soft curacy. Speciﬁcally, for all lead times, the use of numerically Computing, vol. 70, pp. 1167–1188, 2018. simulated values reduced the MAE and RMSE error rates of [9] I. P. Panapakidis, C. Michailides, and D. C. Angelides, “A the Taipei Station by 27.3% and 28.7%, respectively. Simi- data-driven short-term forecasting model for oﬀshore wind larly, the MAE and RMSE rates for the Yilan Station were speed prediction based on computational intelligence,” reduced by 30.3% and 31.1%, respectively. 'erefore, the Electronics, vol. 8, 2019. inclusion of numerically simulated values in wind speed [10] C.-C. Wei, “Conceptual weather environmental forecasting prediction models enhanced their prediction accuracy. system for identifying potential failure of under-construction structures during typhoons,” Journal of Wind Engineering and Industrial Aerodynamics, vol. 168, pp. 48–59, 2017. Data Availability [11] G. Pang, J. He, Y. Huang, and L. Zhang, “A binary logistic 'e Final Operational Global Analysis data were obtained regression model for severe convective weather with nu- merical model data,” Advances in Meteorology, vol. 2019, from the Global Data Assimilation System of the US Article ID 6127281, 15 pages, 2019. Government’s National Center for Environmental Protec- [12] C.-C. Wei, “Surface wind nowcasting in the Penghu Islands tion, which is available at https://rda.ucar.edu/datasets/ based on classiﬁed typhoon tracks and the eﬀects of the ds083.2/. 'e typhoon data were obtained from the Cen- Central Mountain Range of Taiwan,” Weather and Fore- tral Weather Bureau of Taiwan, which are available at casting, vol. 29, no. 6, pp. 1425–1450, 2014. https://rdc28.cwb.gov.tw/and https://e-service.cwb.gov.tw/ [13] Q. Hu, R. Zhang, and Y. Zhou, “Transfer learning for short- wdps/. term wind speed prediction with deep neural networks,” Renewable Energy, vol. 85, pp. 83–95, 2016. [14] L. Tiancheng, R. Qing-dao-er-ji, and Q. Ying, “Application of Conflicts of Interest improved naive Bayesian-CNN classiﬁcation algorithm in 'e author declares that there are no conﬂicts of interest sandstorm prediction in inner Mongolia,” Advances in Me- regarding the publication of this paper. teorology, vol. 2019, Article ID 5176576, 13 pages, 2019. [15] B. Cortez, B. Carrera, Y.-J. Kim, and J.-Y. Jung, “An archi- tecture for emergency event prediction using LSTM recurrent Acknowledgments neural networks,” Expert Systems with Applications, vol. 97, pp. 315–324, 2018. 'is study was supported by the Ministry of Science and [16] L. Guo, N. Li, F. Jia, Y. Lei, and J. Lin, “A recurrent neural Technology, Taiwan, under Grant no. MOST107-2111-M- network based health indicator for remaining useful life pre- 019-003. 'e author acknowledges the data provided by the diction of bearings,” Neurocomputing, vol. 240, pp. 98–109, 2017. Central Weather Bureau of Taiwan and the Research Data [17] T. Liu, S. Yu, B. Xu, and H. Yin, “Recurrent networks with Archive (RDA) at NCAR. attention and convolutional networks for sentence repre- sentation and classiﬁcation,” Applied Intelligence, vol. 48, no. 10, pp. 3797–3806, 2018. References [18] A. S. Qureshi, A. Khan, A. Zameer, and A. Usman, “Wind power prediction using deep neural network based meta [1] C.-C. Wei, “Soft computing techniques in ensemble precip- regression and transfer learning,” Applied Soft Computing, itation nowcast,” Applied Soft Computing, vol. 13, no. 2, vol. 58, pp. 742–755, 2017. pp. 793–805, 2013. [19] X. Yuan, C. Chen, M. Jiang, and Y. Yuan, “Prediction interval [2] C. H. Chang, U. T. Wang, H. C. Fu, and U. C. Lin, Disaster of wind power using parameter optimized Beta distribution Survey of Typhoon Soudelor in 2015, pp. 1–48, National Science and Technology Center for Disaster Reduction, Investigation based LSTM model,” Applied Soft Computing, vol. 82, 2019. [20] Y. Zhang, X. Wang, and H. Tang, “An improved Elman neural Report, Taipei, Taiwan, 2015. [3] C. C. Wei, P. C. Peng, C. H. Tsai, and C. L. Huang, “Regional network with piecewise weighted gradient for time series prediction,” Neurocomputing, vol. 359, pp. 199–208, 2019. forecasting of wind speeds during typhoon landfall in Taiwan: a case study of westward-moving typhoons,” Atmosphere, [21] Z. C. Lipton, J. Berkowitz, and C. Elkan, “A critical review of recurrent neural networks for sequence learning,” 2015, vol. 9, 2018. [4] E. Cadenas, W. Rivera, R. Campos-Amezcua, and C. Heard, http://arxiv.org/abs/1506.00019. [22] S. Hochreiter and J. Schmidhuber, “Long short-term mem- “Wind speed prediction using a univariate ARIMA model and a multivariate NARX model,” Energies, vol. 9, 2016. ory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. Advances in Meteorology 17 [23] Q. Zhang, H. Wang, J. Dong, G. Zhong, and X. Sun, “Pre- [39] Z. Wu, C. Jiang, B. Deng, J. Chen, and X. Liu, “Sensitivity of diction of sea surface temperature using long short-term WRF simulated typhoon track and intensity over the South memory,” IEEE Geoscience and Remote Sensing Letters, China Sea to horizontal and vertical resolutions,” Acta Oce- vol. 14, no. 10, pp. 1745–1749, 2017. anologica Sinica, vol. 38, no. 7, pp. 74–83, 2019. [24] W. Byeon, M. Liwicki, and T. M. Breuel, “Scene analysis by [40] F. Cassola, F. Ferrari, and A. Mazzino, “Numerical simula- mid-level attribute learning using 2D LSTM networks and an tions of Mediterranean heavy precipitation events with the application to web-image tagging,” Pattern Recognition Let- WRF model: a veriﬁcation exercise using diﬀerent ap- ters, vol. 63, pp. 23–29, 2015. proaches,” Atmospheric Research, vol. 164-165, pp. 210–225, [25] E. Fiori, A. Parodi, and F. Siccardi, “Uncertainty in prediction of deep moist convective processes: turbulence parameteri- [41] S. Davolio, D. Mastrangelo, M. M. Miglietta, O. Drofa, zations, microphysics and grid-scale eﬀects,” Atmospheric A. Buzzi, and P. Malguzzi, “High resolution simulations of a Research, vol. 100, no. 4, pp. 447–456, 2011. ﬂash ﬂood near Venice,” Natural Hazards and Earth System [26] F. Molteni, R. Buizza, T. N. Palmer, and T. Petroliagis, “'e Science, vol. 9, no. 5, pp. 1671–1678, 2009. ECMWF ensemble prediction system: methodology and [42] M. L. Weisman, C. Davis, W. Wang, K. W. Manning, and validation,” Quarterly Journal of the Royal Meteorological J. B. Klemp, “Experiences with 0-36-h explicit convective Society, vol. 122, no. 529, pp. 73–119, 1996. forecasts with the WRF-ARW model,” Weather and Fore- [27] A. H. Murphy, “What is a good forecast? An essay on the casting, vol. 23, no. 3, pp. 407–437, 2008. nature of goodness in weather forecasting,” Weather and [43] M. M. Miglietta and R. Rotunno, “Application of theory to Forecasting, vol. 8, no. 2, pp. 281–293, 1993. simulations of observed cases of orographically forced con- [28] X. Shen, B. Jiang, Q. Cao, W. Lin, and L. Zhang, “Sensitivity of vective rainfall,” Monthly Weather Review, vol. 140, no. 9, precipitation and structure of typhoon Hato to bulk and pp. 3039–3053, 2012. explicit spectral bin microphysics schemes,” Advances in [44] P. Zhao, J. Wang, J. Xia, Y. Dai, Y. Sheng, and J. Yue, Meteorology, vol. 2019, Article ID 2536413, 2019. “Performance evaluation and accuracy enhancement of a day- [29] B. Storm and S. Basu, “'e WRF model forecast-derived low- ahead wind power forecasting system in China,” Renewable level wind shear climatology over the United States Great Energy, vol. 43, pp. 234–241, 2012. Plains,” Energies, vol. 3, no. 2, pp. 258–276, 2010. [45] E. Zhao, J. Zhao, L. Liu, Z. Su, and N. An, “Hybrid wind speed [30] K. Trachte, “Atmospheric moisture pathways to the highlands prediction based on a self-adaptive ARIMAX model with an of the tropical Andes: analyzing the eﬀects of spectral nudging exogenous WRF simulation,” Energies, vol. 9, 2016. on diﬀerent driving ﬁelds for regional climate modeling,” [46] S. Liang, L. Nguyen, and F. Jin, “A multi-variable stacked Atmosphere, vol. 9, no. 11, p. 456, 2018. long-short term memory network for wind speed forecast- [31] D. Carvalho, A. Rocha, M. Gomez-Gesteira, and C. Silva Santos, ing,” in Proceedings of the 2018 IEEE International Conference “WRF wind simulation and wind energy production estimates on Big Data, pp. 4561–4564, Seattle, WA, USA, December forced by diﬀerent reanalyses: comparison with observed data for Portugal,” Applied Energy, vol. 117, pp. 116–126, 2014. [47] A. Rostami, A. Baghban, A. H. Mohammadi, A. Hemmati- [32] X. T. Chadee, N. R. Seegobin, and R. M. Clarke, “Optimizing Sarapardeh, and S. Habibzadeh, “Rigorous prognostication of the weather research and forecasting (WRF) model for permeability of heterogeneous carbonate oil reservoirs: smart mapping the near-surface wind resources over the south- modeling and correlation development,” Fuel, vol. 236, ernmost caribbean islands of Trinidad and Tobago,” Energies, pp. 110–123, 2019. vol. 10, 2017. [48] H. R. Saghaﬁ, A. Rostami, and M. Arabloo, “Evolving new [33] N. K. Awan, H. Truhetz, and A. Gobiet, “Parameterization- strategies to estimate reservoir oil formation volume factor: induced error characteristics of MM5 and WRF operated in smart modeling and correlation development,” Journal of climate mode over the Alpine region: an ensemble-based Petroleum Science and Engineering, vol. 181, p. 106180, 2019. analysis,” Journal of Climate, vol. 24, no. 12, pp. 3107–3123, [49] A. Rostami, M. Arabloo, A. Kamari, and A. H. Mohammadi, “Modeling of CO solubility in crude oil during carbon di- [34] D. E. K. Dzebre, A. A. Acheampong, J. Ampofo, and oxide enhanced oil recovery using gene expression pro- M. S. Adaramola, “A sensitivity study of surface wind sim- gramming,” Fuel, vol. 210, pp. 768–782, 2017. ulations over coastal Ghana to selected time control and [50] S. S. Behzadi, C. Prakasvudhisarn, J. Klocker, P. Wolschann, nudging options in the weather research and forecasting and H. Viernstein, “Comparison between two types of arti- model,” Heliyon, vol. 5, 2019. ﬁcial neural networks used for validation of pharmaceutical [35] E. Kalney, Atmospheric Modeling, Data Assimilation, and processes,” Powder Technology, vol. 195, no. 2, pp. 150–157, Predictability, Cambridge University Press, Cambridge, UK, [51] C.-C. Wei, “Comparing lazy and eager learning models for [36] Y. Chu, C. Li, Y. Wang, J. Li, and J. Li, “A long-term wind water level forecasting in river-reservoir basins of inundation speed ensemble forecasting system with weather adapted regions,” Environmental Modelling & Software, vol. 63, correction,” Energies, vol. 9, no. 11, p. 894, 2016. pp. 137–155, 2015. [37] Z. Di, W. Gong, Y. Gan, C. Shen, and Q. Duan, “Combi- [52] D.-C. Lo, C.-C. Wei, and E.-P. Tsai, “Parameter automatic natorial optimization for WRF physical parameterization calibration approach for neural-network-based cyclonic schemes: a case study of three-day typhoon simulations over the Northwest Paciﬁc Ocean,” Atmosphere, vol. 10, no. 5, precipitation forecast models,” Water, vol. 7, no. 12, p. 233, 2019. pp. 3963–3977, 2015. [38] C.-C. Wei, “Wavelet support vector machines for forecasting [53] J. L. Elman, “Finding structure in time,” Cognitive Science, precipitation in tropical cyclones: comparisons with GSVM, vol. 14, no. 2, pp. 179–211, 1990. regression, and MM5,” Weather and Forecasting, vol. 27, [54] A. Graves, “Generating sequences with recurrent neural no. 2, pp. 438–450, 2012. networks,” 2013, http://arxiv.org/abs/1308.0850. 18 Advances in Meteorology [55] F. Weninger, J. Geiger, M. Wollmer, ¨ B. Schuller, and G. Rigoll, “Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments,” Computer Speech & Language, vol. 28, no. 4, pp. 888–902, 2014. [56] A. Graves, Supervised Sequence Labelling with Recurrent Neural Networks, Springer, Berlin, Germany, 2012. [57] F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: continual prediction with LSTM,” Neural Computa- tion, vol. 12, no. 10, pp. 2451–2471, 2000. [58] V. Nair and G. Hinton, “Rectiﬁed linear units improve re- stricted Boltzmann machines,” in Proceedings of Machine Learning Research, pp. 807–814, Sardinia, Italy, May 2010. [59] T. C. Yeh, S. H. Chen, and J. S. Hong, @e Forecast Technique Development Studies on the Typhoon Track, Rainfall and Winds Forecast Over Taiwan Area: A Study on the Imple- mentation of WRF Typhoon Forecasting Component in the Operational Environment of CWB, Funded by Ministry of Science and Technology of Taiwan, Taipei, Taiwan, No. NSC96-2625-Z-052-003 in Chinese, 2008. [60] T. C. Yeh, C. T. Terng, C. S. Lee, and M. J. Yang, “'e forecast technique development studies on the typhoon track, rainfall and winds forecast over taiwan area: a study on the imple- mentation of WRF typhoon forecasting component in the operational environment of CWB (II),” Funded by Ministry of Science and Technology of Taiwan, No. NSC97-2625-M-052- 002 (in Chinese), 2009. [61] D. P. Kingma and J. L. Ba, “Adam: a method for stochastic optimization,” in Proceedings of the International Conference on Learning Representations, pp. 1–13, San Diego, CA, USA, May 2015. [62] F. Han, J. Jiang, Q.-H. Ling, and B.-Y. Su, “A survey on metaheuristic optimization for random single-hidden layer feedforward neural network,” Neurocomputing, vol. 335, pp. 261–273, 2019. [63] A. Sharma, “Guided stochastic gradient descent algorithm for inconsistent datasets,” Applied Soft Computing, vol. 73, pp. 1068–1080, 2018. [64] C. Yao, D. Cai, J. Bu, and G. Chen, “Pre-training the deep generative models with adaptive hyperparameter optimiza- tion,” Neurocomputing, vol. 247, pp. 144–155, 2017. [65] W.-C. Wang, K.-W. Chau, C.-T. Cheng, and L. Qiu, “A comparison of performance of several artiﬁcial intelligence methods for forecasting monthly discharge time series,” Journal of Hydrology, vol. 374, no. 3-4, pp. 294–306, 2009. [66] D. A. Swanson, J. Tayman, and T. M. Bryan, “MAPE-R: a rescaled measure of accuracy for cross-sectional subnational population forecasts,” Journal of Population Research, vol. 28, no. 2-3, pp. 225–243, 2011.

Journal

Advances in Meteorology – Hindawi Publishing Corporation

Published: Jul 23, 2020

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions

References (66)

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies