Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Metamodel-based multidisciplinary design optimization methods for aerospace system

Metamodel-based multidisciplinary design optimization methods for aerospace system Astrodynamics Vol. 5, No. 3, 185–215, 2021 https://doi.org/10.1007/s42064-021-0109-x Metamodel-based multidisciplinary design optimization methods for aerospace system 1,2 1,2 1,2 1,2 1,2 1,2 Renhe Shi , Teng Long (B), Nianhui Ye , Yufei Wu , Zhao Wei , and Zhenyu Liu 1. School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China 2. Key Laboratory of Dynamics and Control of Flight Vehicle, Ministry of Education, Beijing 100081, China ABSTRACT KEYWORDS The design of complex aerospace systems is a multidisciplinary design optimization (MDO) aerospace systems design problem involving the interaction of multiple disciplines. However, because of the necessity multidisciplinary design of evaluating expensive black-box simulations, the enormous computational cost of solving optimization (MDO) MDO problems in aerospace systems has also become a problem in practice. To resolve metamodel-based design and this, metamodel-based design optimization techniques have been applied to MDO. With optimization (MBDO) these methods, system models can be rapidly predicted using approximate metamodels to expensive black-box problems improve the optimization efficiency. This paper presents an overall survey of metamodel- based MDO for aerospace systems. From the perspective of aerospace system design, this paper introduces the fundamental methodology and technology of metamodel- based MDO, including aerospace system MDO problem formulation, metamodeling techniques, state-of-the-art metamodel-based multidisciplinary optimization strategies, and expensive black-box constraint-handling mechanisms. Moreover, various aerospace system examples are presented to illustrate the application of metamodel-based MDOs Review Article Received: 14 July 2021 to practical engineering. The conclusions derived from this work are summarized in the final section of the paper. The survey results are expected to serve as guide and reference Accepted: 23 July 2021 for designers involved in metamodel-based MDO in the field of aerospace engineering. © The Author(s) 2021 1 Introduction design of systems [3]. Moreover, MDO was first applied to the coupled aero-structural optimization problem for In practice, the development of aerospace systems in- aircraft wings [4]. The merits of MDO have been widely volves sophisticated system engineering. Aerospace sys- proven in the design practices of engineering systems, tems (e.g., satellites and spacecraft) are multidisciplinary such as aircraft [5–10], automobiles [11–13], ships [14,15], schemes consisting of several inter-coupled disciplines or and electric devices [16, 17]. In the past decade, MDO subsystems, including orbit, power, propulsion, structure, has also been applied to aerospace systems, including thermal control, attitude control, and payload. Multi- satellites [18–21], constellations [20, 22,23], and launch disciplinary design optimization (MDO) has been widely rockets [24–27], to improve the overall system perfor- employed in engineering systems design practices to im- mance. prove design performance and reduce design costs in the Although deriving the optimal design of multidisci- early development process [1]. The concept of MDO was plinary systems is advantageous, the implementation of initially developed by Sobieszczanski-Sobieski [2]. As a MDO is confronted with two crucial problems in practi- methodology, MDO focuses on the design of complex engi- cal engineering. The basic problem involves the means neering systems and subsystems by coherently exploiting by which multiple disciplines are to be coupled. The the synergy of mutually interacting phenomena [2]. It simplest approach is to implement multidisciplinary ana- comprehensively utilizes various computational analysis tools and optimization methods to determine the optimal lysis (MDA) during optimization, where the coupling B tenglong@bit.edu.cn 186 R. Shi, T. Long, N. Ye, et al. variables are directly solved via iterative analysis. In the MBDO and MDO methodologies can be referred to addition, several MDO architectures have been success- as metamodel-based MDO. Note that in practice, the fully developed to organize different analysis models and purpose of metamodel-based MDO is to overcome the computational flow for multidisciplinary optimization. burden of optimization rather than build or solve MDO These architectures can be generally classified as mono- architectures. lithic (e.g., all-at-once, individual discipline feasible, and A number of reviews regarding MDO have been re- multidisciplinary feasible) architectures and distributed ported in the literature [1, 3, 31, 32]. Many surveys on (e.g., cooperative optimization and concurrent subspace metamodel-based design and optimization have also been optimization) architectures [3], with respect to whether conducted in recent years [28, 29,33]. However, compre- sub-optimization is required. Another critical problem hensive surveys on metamodel-based MDOs are rarely involves achieving efficiency in solving MDO problems performed [34,35]. Focusing on the design of aerospace with limited computational resource. Modern aerospace systems, this paper introduces the fundamental methodo- system designs typically employ high-fidelity disciplinary logy and technology of state-of-the-art metamodel-based simulation models (e.g., finite element analysis (FEA) MDO techniques. The remainder of this paper is or- model with large grids) to enhance the quality and re- ganized as follows. Section 2 introduces the problem liability of design optimization results. However, the formulation and overall architecture of the metamodel- evaluation of expensive simulation models and iterative based MDO. The common metamodeling techniques are MDA computing processes further considerably increase described in Section 3. In Section 4, the metamodel- the associated computational cost of MDO problems in based multidisciplinary optimization strategies are dis- aerospace systems. Moreover, because of the lack of cussed in detail. The survey of expensive black-box transparency in simulation models (e.g., simulations im- constraint-handling mechanisms for metamodel-based plemented using commercial computer-aided engineering MDOs is presented in Section 5. Moreover, several real- (CAE) software), the MDA process is generally referred world aerospace system MDO examples are discussed in to as an expensive black-box function, whose gradient is Section 6. Finally, some conclusions and future works on expensive or unreliable for calculation. Thus, most exist- metamodel-based MDOs are summarized in Section 7. ing heuristic or gradient-based numerical methods (e.g., genetic algorithm and sequential quadratic programming) 2 Methodology of metamodel-based are unsuitable for solving computation-intensive black- MDO box optimization problems. 2.1 Formulation of aerospace system MDO To reduce the computational cost of expensive black- problem box optimization problems, metamodel-based design op- timization (MBDO) techniques have been developed The development of an aerospace system is a complex over the past two decades [28]. These methods are also system engineering task. For instance, a satellite system called surrogate-assisted analysis and optimization me- consists of multiple inter-coupled disciplines, including or- thods [29]. They involve the construction of a metamodel bit, propulsion, attitude control, payload, and structure, (or a surrogate) based on a set of samples to approximate as shown in Fig. 1. To improve system performance, the original expensive simulation models for analysis or optimization. These MBDO methods can also be app- lied to solve MDO problems. In this case, an MDO problem is essentially treated as a general constrained nonlinear optimization problem, where the computation- ally intensive MDA process is replaced with metamodels to reduce the number of expensive function calls. In recent years, several state-of-the-art MBDO methods have been developed [30]. Different MBDO methods are distinguished by their metamodeling techniques and de- sign space exploration strategies. The combination of Fig. 1 Various disciplines for aerospace systems. Metamodel-based multidisciplinary design optimization methods for aerospace system 187 MDO was employed to determine the optimal design (i.e., state variables) of the ith discipline, respectively; X and X are the lower and upper bounds of de- parameters for the aerospace system. LB UB sign variable, respectively; Y is the coupling state vari- In MDO, the inter-coupled relationships, data ex- ij ables from the ith discipline to the jth discipline; and change, and simulation ofl w among different disciplines D (X ,Y ,Y ) and g (X ,Y ,Y ) are the state equa- can be organized in terms of the design structure matrix, i i i ij i i i ij tions (i.e., disciplinary analysis model) and constraints as illustrated in Fig. 2. In the matrix, the diagonal ele- of the ith discipline, respectively. ments (i.e., shaded blocks) represent different disciplines Note that because of feedback coupling state vari- involved in the aerospace system. The terms above the ables, Y = {Y |i > j}, the MDA process must derive diagonal represent feed-forward variables and parame- FV ij a consistent solution at each sample point during the ters, whereas those below represent backward informa- metamodel-based optimization process. This means that tion. The design structure matrix explicitly expresses the solution (D (X ,Y ,Y ), i = 1, 2,··· , N ) must sa- the constitution of the MDO problem. Based on the con- i i i ij d tisfy all state equations. In practice, the MDA process can structed matrix, the general mathematical formulation of be organized via a fixed-point-based iteration approach, an aerospace system MDO problem can be represented as as summarized in Algorithm 1. min f(X),X = [X ,X ,··· ,X ] 1 2 N D (X ,Y ,Y ) = 0, i = 1, 2,··· , N i i i ij d 2.2 Overall procedure of metamodel-based s.t. g (X ,Y ,Y ) ⩽ 0, i = 1, 2,··· , N (1) optimization i i i ij d X ⩽ X ⩽ X LB UB Because modern aerospace system design generally in- where f(X) is the objective of the aerospace system volves computationally expensive simulation models, MDO problem (e.g., total mass and lifecycle cost); N MBDO techniques are employed to solve MDO problems is the number of disciplines; X is the vector of design to reduce the computational cost. In the metamodel- variables; X and Y are the design variables and output based MDO process, the entire MDA process is evaluated i i Fig. 2 Design structure matrix of aerospace system MDO problem. 188 R. Shi, T. Long, N. Ye, et al. to obtain a consistent design at each sample point. The Algorithm 1 Quasi-codes for fixed-point-based MDA process metamodels are constructed to approximate the MDA Input: Initial value of feedback coupling state process for multidisciplinary optimization. In this case, (0) variables (Y ); design variables (X); the responses of expensive black-box objectives and con- FV MDA convergence tolerance (ε ) MDA straints can be rapidly and inexpensively predicted by Output: Objective value (Y ); number of MDA metamodels. Note that metamodels can be gradually re- iteration (k ) MDA fined via adaptive sampling during the optimization pro- 1 Begin cess to further reduce the computational cost. Moreover, 2 exitflag ← 0 Fig. 3 illustrates the overall procedure of metamodel- 3 k ← 0 MDA 4 while exitflag == 0 do based aerospace system MDO [19]. The major steps are (k +1) (k ) MDA MDA 5 (Y ,Y ) ← MDO(X,Y ) as follows: FV FV 6 exitflag ← 1 Step 1: The aerospace system MDO problem, includ- (k +1) MDA (k +1) MDA 7 for each Y in Y FV ing the design space, objective, constraints, and MDA 8 if models to be optimized, is clearly defined. The algorithm (k +1) (k ) (k ) MDA MDA MDA |Y − Y |/|Y | ⩾ ε MDA parameters (e.g., the number of initial samples and ter- then mination criterion) of the selected MBDO methods for 9 exitflag ← 0 solving the MDO problem are configured. 10 end Step 2: A number of sample points are generated 11 end 12 k ← k + 1 MDA MDA by the design of the experiment in the design space. 13 end Then, the MDA process is invoked to obtain the objective 14 return Y , k MDA and constraint responses at each sample point. The 15 End sample points and their associated responses (referred to Fig. 3 Overall procedure of metamodel-based MDO for aerospace systems (reproduced with permission from Ref. [19], © IAA 2017). Metamodel-based multidisciplinary design optimization methods for aerospace system 189 as samples) are stored in the sample dataset. 2.3 Architecture of metamodel-based MDO Step 3: Based on existing samples in the sample techniques dataset, metamodels are constructed to approximate 2.3.1 Classification of metamodel-based MDO the objective and constraint responses from the costly methods MDA process. If the static metamodel-based optimiza- Metamodel-based MDO methods may be generally clas- tion strategy is used, then the approximation accuracy of sified into two categories in terms of whether the metamodels must be verified, as discussed in Section 2.3. metamodels are updated during optimization: static Step 4: The constructed metamodels are used to re- and dynamic metamodel-based optimization strategies place the original expensive MDA process for optimiza- (Fig. 4) [36]. tion. Global numerical optimization techniques, such In the static metamodel-based optimization strategy, as genetic algorithms, are employed to directly optimize metamodels are constructed once based on sufficient sam- the metamodels instead of the original expensive MDA ples for optimization. This is the most convenient ap- models, as given by Eq. (2): proach to implement the metamodel-based MDO pro- min f(X),X = [X ,X ,··· ,X ] 1 2 n cess. For instance, some expensive disciplinary models can be replaced with well-constructed metamodels for gˆ (X) ⩽ 0, i = 1, 2,··· , m s.t. (2) MDA and optimization. Note that to ensure the con- X ⩽ X ⩽ X LB UB fidence in their results, the approximation accuracy of where m is the number of constraints, and f(X) and the constructed metamodels should be verified before gˆ (X) are the metamodels of the objective and the ith optimization. Techniques that are widely used for ac- constraint, respectively. If the dynamic metamodel-based curacy validation include split and crossover validation optimization strategy is used, the metamodels are adap- methods. In the split validation method, the samples tively refined during optimization to efficiently search for are divided into two groups. One group of samples is the optimum, as discussed in detail in Section 4. applied to construct the metamodels, and the other is Step 5: The termination criterion is checked to deter- used to validate their accuracy. For the crossover valida- mine whether the optimization must be terminated. The tion method, several samples are randomly selected for common termination criteria in metamodel-based MDO validation, whereas the remaining samples are used for include the decrement criterion (C ) and computational metamodeling. The validation process is repeated until cost criterion (C ) [36], as given in Eq. (3). In C , if the 2 1 the prediction errors in all the samples are obtained. The error or relative error between the objective values in two most commonly used approximation accuracy indices for consecutive iterations is less than the tolerance (ε ), OPT metamodels are summarized in Table 1 [36]. For example, then the optimization terminates. In C , if the number if the complex correlation coefficient ( R ) exceeds 0.9, of existing samples (N ) exceeds the predefined maxi- then the metamodel can be assumed to be sufficiently mum number of function evaluations (NFE ), then the max accurate for optimization. optimization terminates. In contrast to constructing a static metamodel once, (k) (k−1) f(X ) − f(X ) the metamodels in the dynamic metamodel-based op- C : ⩽ ε ∥ 1 OPT (k) f(X ) timization strategy are adaptively refined according to (k) (k−1) certain criteria or mechanisms during optimization. Over- |f(X ) − f(X )| ⩽ ε (3) OPT all, the dynamic metamodel-based optimization strategy C : N > NFE 2 e max can be further divided into infill sampling criterion-based methods and space reduction-based methods. Compared Step 6: The optimized design for the aerospace system with the static metamodel-based optimization strategy, MDO problem is finally obtained when the termination the dynamic metamodel-based optimization strategy, criterion is reached. For the single-objective optimization which has become popular in recent years, is generally problem, the best feasible solution in the sample dataset more efficient in practice. This study mainly focuses on is outputted as the final solution. For the multi-objective optimization problem, the non-dominated designs in the surveying the dynamic metamodel-based optimization sample dataset are outputted as a Pareto solution. strategy, as discussed in detail in Section 4. 190 R. Shi, T. Long, N. Ye, et al. Fig. 4 Classification of MBDO methods. Table 1 Approximation accuracy indices for metamodel validation Index Symbol Formulation N N test test X X (test) (test) (test) 2 2 2 (test) 2 Complex correlation coefficient R R = 1 − (y − yˆ ) / (y − y¯ ) i i i i=1 i=1 test u X (test) (test) Root mean square error RMSE RMSE = (y − yˆ ) i i test i=1 N (test) (test) N test test |y − yˆ | 1 (test) i=1 i i t (test) 2 Relative average absolute error RAAE RAAE = ; STD = (y − y¯ ) N · STD N test test (test) (test) (test) (test) (test) (test) max(|y − y¯ |,|y − y¯ |,··· ,|y − y¯ |) 1 2 N test Relative maximum absolute error RMAE RMAE = STD (test) (test) Note: N is the number of test samples; y is the real response of the ith test sample; yˆ is the predicted response of the ith test i i (test) test sample; y¯ is the mean response of the test samples. Metamodel-based multidisciplinary design optimization methods for aerospace system 191 2.3.2 Diagram of metamodel-based MDO archi- two typical optimal Latin hypercube design methods, tecture that is, the native lhsdesign function in the MATLAB Based on the foregoing discussion, Fig. 5 illustrates the optimization toolbox [40] and ESEA-OLHD [41], are il- architecture of the metamodel-based MDO technique [36]. lustrated in Fig. 6. The lhsdesign function is implemented Before optimization, the MDA models of aerospace sys- by randomly generating several groups of sample points tems (e.g., orbit, structure, and attitude control) must and selecting the group with the best space-filling abil- be established and parameterized. Then, MBDO tech- ity. Moreover, ESEA-OLHD generates sample points by niques are employed to optimize the MDO problem by optimizing the space-filling criteria via numerical opti- evaluating the MDA process. During the optimization, mization algorithms. This shows that the random sample metamodels are constructed and refined to explore the points generated by the optimal Latin hypercube design design space. The optimization stops when the termi- methods can uniformly fill the design space, thus improv- nation criterion is satisfied. The optimized solution is ing the metamodeling performance in practice. In recent outputted for further analysis. years, some novel OLHD methods specific for metamodel- based optimization have been developed [42–44]. After the generation of sample points, the associated 3 Metamodeling techniques responses at each sample point are evaluated by calling 3.1 Design of experiments the MDA process. The samples are then utilized to construct metamodels for MDO. The design of experiments aims at generating sample points in the design space to represent the numerical 3.2 Typical metamodels characteristics of the system. Considering the limited 3.2.1 Polynomial response surface method computational resource in practice, the design of exper- The polynomial response surface method constructs a imental methods is expected to provide favorable pro- multivariate linear regression function to fit the costly jective uniformity and space-filling uniformity perfor- simulation model or MDA process [45]. A polynomial mance. In the metamodel-based aerospace system MDO response surface metamodel can be written as process, Latin hypercube is the most commonly used design among experimental methods. It can generate N (0) (i) (i) f (x) = β + β x sample sets with arbitrary quantities and dimensions. PRSM i=1 To further improve the sampling performance, in recent N N N v v v X XX years, optimal Latin hypercube design methods have been (ii) (i) 2 (ij) (i) (j) + β (x ) + β x x developed based on certain criteria, such as minimum i=1 i=1 j>1 distance [37], entropy [38], and energy [39]. For instance, (4) Fig. 5 Diagram of metamodel-based MDO architecture. 192 R. Shi, T. Long, N. Ye, et al. x1 x2 (a) lhsdesign (b) ESEA-OLHD Fig. 6 Various optimal Latin hypercube design methods. Table 2 Commonly used radial functions where N is the dimensionality of design variables; (0) (i) (ij) β , β , β are the coefficients estimated through the Function Formulation least squares method. The coefficient matrix, β , is given Linear ϕ(r ) = (r + c) Gauss ϕ(r ) = exp(−cr ) by 2 2 Spline ϕ(r ) = r log(cr ) T −1 T β = (Φ Φ) Φ y (5) Cubic ϕ(r ) = (r + c) 2 2 1/2 Multiquadratic ϕ(r ) = (r + c ) 2 2 −1/2 where y is the response vector of training sample points, Inverse multiquadratic ϕ(r ) = (r + c ) and Φ is the matrix relevant to the training sample points. 3.2.2 Radial basis functions distribution of sample points and function information. Radial basis function is an interpolation method based Typically, c can be estimated using Eq. (8) [36]: on the function value at sample points [46]. A radial c = ((max(x) − min(x))/N ) (8) basis function metamodel can be formulated as N 3.2.3 Kriging f (x) = ω ϕ (∥x − x ∥) (6) RBF i r i The Kriging model is a type of unbiased optimal estima- i=1 tion interpolation model that combines a global approxi- where N is the number of training sample points; mation model and a stochastic process. It is formulated ϕ (∥x − x ∥), i = 1, 2,··· , N , is the radial function; and r i t using Eq. (9): ω is the weight coefficient of radial function. The coeffi- f (x) = µ(x) + Z(x) (9) KRG cient vector (ω) can be calculated as follows: −1 where µ (x) is the global approximation model, which ω = A y reflects the variation trend of the expensive MDA pro- y = [y , y ,··· , y ] (7) 1 2 N cess and is usually set as a constant; Z(x) represents   ϕ(∥x − x ∥) ··· ϕ(∥x − x ∥) 1 1 1 N 2 a Gaussian process with zero mean and variance (σ ).   . . . . . A =   . . Given a set of sample points, X = {x ,x ,··· ,x }, 1 2 N ϕ(∥x − x ∥) ··· ϕ(∥x − x ∥) N 1 N N t t t the covariance matrix is given by Eq. (10): N ×N t t The typically used radial functions are listed in Cov(Z(X)) = σ R(R(x ,x )), i, j = 1, 2,··· , N (10) i j t Table 2 [36], where is the Euclidean distance is ∥x− x ∥. where R(· ) is a symmetric correlation matrix, and The approximate accuracy of this metamodel is influ- enced by shape coefficient c, which is determined by the R(x x ) is the Gaussian correlation function between i j y1 y2 Metamodel-based multidisciplinary design optimization methods for aerospace system 193 sample points x and x , as shown in Eq. (11): applied to metamodeling in practice. The representative i j method is support vector regression [50,51]. Support vec- (k) (k) tor regression is based on the principle of support vector R(x ,x ) = exp − θ |x − x | (11) i j k i j k=1 machines with slight variations and has proven to be effec- tive in solving regression problems [51]. Artificial neural where θ is the correlation parameter determined by networks are also common in metamodeling fields [52]. An maximizing the likelihood function in Eq. (12): artificial neural network is a multilayer feedforward net- N 1 L(x) = − ln(σˆ ) − ln(|R|) (12) work that approximates the targeted nonlinear function; 2 2 the network parameters are trained via numerical meth- The estimated variance (σˆ ) and mean values (µˆ) can ods (e.g., back-propagation algorithm). As a hierarchical be calculated using Eq. (13): composition of Gaussian process-based metamodels (e.g., T −1 1 R Y Kriging), deep Gaussian processes have attracted consid- µˆ = T −1 1 R 1 erable interest in recent years owing to their promising (13) T −1 (Y − 1µ)ˆ R (Y − 1µ)ˆ T T nonlinear approximation performance [53–55]. σˆ = where Y is the column vector of the responses of sample 3.3 Multi-model fusion methods points; 1 is a 1 × N unit vector. The prediction value Most conventional metamodeling techniques and and standard deviation of the unvisited point using the metamodel-based optimization processes simply utilize Kriging model are formulated in Eq. (14): high-fidelity analysis models that are accurate but com- T −1 T f (x) = µˆ + r R (Y − 1 µ)ˆ KRG T putationally intensive. Thus, the computational burden T −1 2 (14) (1 − 1 R r) 2 T −1 for solving MDO problems remains heavy because of the sˆ (x) = σˆ 1 − r R r + KRG T −1 1 R 1 excessive number of costly samples. However, real-world 3.2.4 Ensemble method aerospace system designs generally involve multi-fidelity In general, different metamodels exhibit different approxi- analysis models, such as structural FEA models with fine mation performance levels for different problems. To or coarse grids and orbital transfer models considering exploit various metamodels, different metamodels can perturbations and eclipses. Low-fidelity analysis models be combined to formulate an ensemble metamodel to are less accurate but computationally inexpensive. Multi- improve the approximation capability [47], as shown in model fusion methods (or multi-fidelity methods) have Eq. (15): been proposed to exploit multi-fidelity analysis mod- els [56–60]. In this approach, a large number of low- esm ˆ ˆ f = α f (x) (15) fidelity points are generated to capture the trends of Ensemble i i i=1 system responses, and a small number of high-fidelity ˆ points are used to calibrate the trend. The basic form where f is the ensemble of different metamodels, Ensemble ˆ of a multi-model fusion metamodel is given by Eq. (16). f (x) is the ith metamodel, N is the number of dif- i esm In terms of the method for determining the regression ferent metamodels, and α is the weight of the corre- (ρ ) and discrepancy (δ (x)) items, multi-model fusion me- sponding metamodel. In practice, the weights (α ) can thods can be divided into correction-based and Bayesian be determined with respect to the approximation error methods. of each metamodel [47,48]. If the computational cost is acceptable, the weights can be directly optimized using ˆ ˆ f (x) = ρ f (x) + δ (x) (16) HF LF numerical methods by minimizing the cross-validation 3.3.1 Correction-based method error of the ensemble metamodel [43, 49]. The correction-based method is a straightforward multi- 3.2.5 Machine-learning-based metamodeling me- thod model fusion method, where the metamodel of the low- ˆ ˆ fidelity model ( f (x)) and that of the discrepancy (δ (x)) Because the essence of metamodels in optimization is re- LF latively similar to the regression task in machine learning, are constructed. The regression item (ρ ) is incorporated some machine learning regression algorithms have been to minimize the difference between the scaled low-fidelity 194 R. Shi, T. Long, N. Ye, et al. model approximation (ρ f (x)) and high-fidelity model LF response (f (x)). Traditionally, ρ is a scalar, but mod- HF eling it as a function of x is an area of research [61,62]. Based on Ref. [59], radial basis function and Kriging are the most widely used metamodels in the correction-based multi-model fusion method. 3.3.2 Bayesian method The Bayesian multi-model fusion method, also known as the Co-Kriging method, was introduced by Kennedy and O’Hagan [63]. Co-Kriging constructs the covariance between high-fidelity and low-fidelity models to import the assistance of the low-fidelity model; in this method, Fig. 7 1D numerical example for Co-Kriging. ρ and the hyperparameters of δ (x) are both estimated. Forrester et al. applied Co-Kriging to the design opti- the design space according to a certain infill criterion. In mization field by combining it with the Bayesian model this study, two representative infill sampling criterion- update criterion to balance global exploration and local based optimization strategies are introduced: efficient exploitation [56]. In recent decades, Co-Kriging variants global optimization methods and mode-pursing sampling have become one of the most popular multi-model fu- methods. sion methods [59]. Different variants of Co-Kriging have 4.1.1 Efficient global optimization method also been proposed to improve the approximation accu- The well-known efficient global optimization algorithm racy [64–67] and reduce the metamodeling cost [66–69]. generates infill sample points in the design space [71], To illustrate the concept of multi-model fusion in- where the expected improvement is maximized to balance tuitively, a one-dimensional (1D) numerical example is the exploration and exploitation of optimization. The presented. This example has been widely reported in the formulation of the expected improvement is given by literature to demonstrate the effects of multi-model fusion Eq. (18): methods [56,57,59,66,70]. High-fidelity and low-fidelity models are formulated in Eq. (17). Here, the parame- EI(x) = E[I(x)] ters adopt A = 0.5, B = 10, and C = − 5. A Kriging y − f (x) min KRG (y − f (x))Φ  min KRG sˆ (x)  KRG model is constructed using pure high-fidelity samples. A y − f (x) min KRG +sˆ (x)ϕ , sˆ (x) > 0 KRG KRG sˆ (x)  KRG Co-Kriging model is constructed using both high-fidelity 0, sˆ (x) = 0 KRG and low-fidelity samples. As illustrated in Fig. 7, Kriging (18) inadequately approximates f , whereas the Co-Kriging HF calculation is relatively close to f with the support of HF where Φ (· ) and ϕ (· ) are the Gaussian probability dis- low-fidelity data. tribution function and probability density function, re- spectively; y is the minimum objective function value 2 min f (x) = (6x − 2) sin(12x − 4) HF , x ∈ [0, 1] (17) among existing sample points. As shown in Eq. (18), f (x) = Af + B(x − 0.5) + C LF HF if the newly added infill sample point is not the same as existing sample points, then EI(x) > 0; otherwise, EI(x) = 0. 4 Metamodel-based multidisciplinary To illustrate the efficient global optimization process optimization strategy intuitively, a 1D numerical example (Eq. (19)) is investi- 4.1 Infill sampling criterion-based optimiza- gated. tion strategy f(x) = (6x − 2) sin(12x − 4) (19) In the dynamic metamodel-based optimization strategy, one widely used method for updating metamodels is to The optimization process is illustrated in Fig. 8. The sequentially allocate the newly added sample points in figure shows that the sample point with the maximum Metamodel-based multidisciplinary design optimization methods for aerospace system 195 EI (a) Initial Kriging model (b) Initial EI(x) curve Expected improvement value Sample with maximum EI value 1.5 0.5 0 0.2 0.4 0.6 0.8 1 (c) Kriging model after the 1st iteration (d) EI(x) after the 1st iteration EI (e) Kriging model after the 2nd iteration (f) EI(x) after the 2nd iteration Fig. 8 Efficient global optimization process. 196 R. Shi, T. Long, N. Ye, et al. EI (g) Kriging model after the 3rd iteration (h) EI(x) after the 3rd iteration Fig. 8 Efficient global optimization process. (Continued) EI(x) in each iteration is selected to update the Kri- where G(i) is the cumulative distribution function, G(i) ging model. In addition, EI(x) decreases with iteration, is the cumulative sum, G is the minimum cumulative min indicating the convergence tendency of the optimization sum, r(R ) is the bias control factor determined by the process. After the 3rd iteration, the optimization process complex correlation coefficient ( R ) of the metamodel, converges to the global optimum. N is the number of inexpensive point groups, and n is cg g To further improve the optimization capacity, some the number of each group. variants of efficient global optimization have been deve- The optimization process of mode-pursing sampling is loped in recent years, especially in constraint han- illustrated in Fig. 9 [79], using a six-hump camel-back dling [72], parallel infill sampling [73], and high- problem [80], as formulated in Eq. (21): dimensional optimization [74]. Further details of the 21 1 (1) 2 (1) 4 (1) 6 (1) (2) f(x) = 4(x ) − (x ) + (x ) + x x variants of efficient global optimization are discussed in 10 3 (2) 2 (2) 4 2 Refs. [75–78]. − 4(x ) + 4(x ) ; x ∈ [−2, 2] (21) 4.1.2 Mode-pursing sampling methods The mode-pursing sampling method proposed by Wang et al. is another typical infill-criterion-based optimization strategy, where the infill criterion is implemented based on a constructed cumulative probability function [79]. In the traditional mode-pursing sampling method, the biased cumulative sum of approximated objective values of numerous inexpensive points constructs the cumulative distribution function, as given in Eq. (20): 1/r(R ) G(i) = G(i) , i = 1,··· , N cg k·n i g X X G(i) = (f − f(x ))/n max j g k=1 j=(k−1)·n +1 2 (20) Fig. 9 Mode-pursing sampling optimization procedure. r(R ) = 1, R < 0.8 In the mode-pursing sampling procedure, numerous " # 2 2 log G (R − 0.8) inexpensive points are randomly generated in the design min · 1 + 1 − , 0.8 ⩽ R ⩽ 1 log 0.75 0.2 space and sorted in ascending order with respect to their Metamodel-based multidisciplinary design optimization methods for aerospace system 197 metamodel responses, f(x ), as shown in Fig. 9(a). Then, estimation capability [98, 99]. Moreover, as an ensemble, ˆ ˆ f(x ) is subtracted from the maximum f(x ) to obtain all criteria can be used to improve robustness; this is i i a nonnegative difference (Fig. 9(b)), which is averaged called a multi-infill strategy [97, 100,101]. over a group of n inexpensive points to obtain G(i) ˆ 4.2 Space reduction-based optimization (Fig. 9(c)). Then, G(i) is normalized as G(x ), as shown ˆ strategy in Fig. 9(d). Here, G(i) satisfies the requisites of a cumu- lative distribution function, such as that monotonically The space reduction-based optimization strategy is an- increasing and varying between 0 and 1. New sample other widely used approach in practice. In this approach, points are selected from inexpensive points based on G(i), a small subspace in the design space is constructed or and the selected probability of inexpensive points in the identified. This subspace is referred to as the region of ith group is equal to G(i). The control factor, r(R ), interest, where the global optimum is located with high biases G(i) toward the inexpensive point groups with probability. During optimization, a number of newly smaller f(x ) values, as illustrated in Fig. 9(e). In other i added samples are generated in the region to gradually words, the more accurate the metamodel, the more likely improve the fitting quality of metamodels in the vicinity inexpensive points (f(x )) are selected. Because G(i) is i of the global optimum. In general, the space reduction- positive throughout the design space, and mode-pursing based optimization strategy includes three major steps: sampling adopts an elite strategy, the global convergence (1) determining the center of the region; (2) calculating property can be theoretically proved easily [79, 81]. the size of the region; and (3) trimming the region ac- Because of the potential global optimization perfor- cording to the design space. The region is identified, as mance of the mode-pursing sampling framework, a range illustrated in Fig. 10. In this paper, several representa- of variants has been proposed for different optimization tive identification methods for the ROI are introduced problems by customizing the expression or application as follows. scenario of G(i). For instance, Kazemi et al. proposed the constraint-importance mode-pursuing sampling method for constrained optimization problems by introducing Design space Sample point penalty items into G(i) [82]. Sharif et al. developed discrete variable mode-pursing sampling for discrete va- riable optimization problems by replacing a continuous objective with a discrete objective and utilizing a double- sphere strategy to improve the local exploitation abi- lity [83]. Shan and Wang proposed a Pareto set pursing method for multi-objective optimization problems by ag- Radius gregating multiple objectives into a single fitness function that reflects the dominance of sample points [84]. Other variants have also been developed for high-dimensional Center Trimmed expensive black-box optimization problems [81, 85] or Global optimum region of interest high-dimensional expensive constrained black-box opti- mization problems [86]. In addition to the foregoing algorithms, other widely Fig. 10 Space reduction-based optimization strategy. used infill sampling criteria include the minimization of surrogate prediction [87, 88], probability of improve- 4.2.1 Significant design space method ment [89, 90], mean square error [91, 92], lower confi- The significant design space method is a simple space- dence bounding [87–95], and fuzzy clustering-based crite- reduction method [94]. In this method, the region of rion [96,97]. Among the different infill sampling criteria, interest is constructed around the best sample (x ) at expected improvement, probability of improvement, mean the kth iteration. The space size (B ) is determined square error, and lower confidence bounding are generally according to the metamodel approximation accuracy in combined with Kriging or other metamodels with error the vicinity of x , as shown in Eq. (22): k 198 R. Shi, T. Long, N. Ye, et al. ∗ ∗ ∗ e = |(f − f )/f | the shrinking and amplification factors, respectively; and k k k (22) ∆ is the upper bound of the trust region radius. As ϑ = e /e k a k proved in Ref. [102], the trust region-based optimization where e is the metamodel approximation accuracy in the strategy can converge to the local optimum, which can ∗ ∗ ∗ vicinity of x ; f and f are the objective function and k k k also converge to the global optimum. In recent years, metamodel prediction values at x , respectively; ϑ is some trust region method-based adaptive optimization the size scale indicator; and e is the acceptable deviation methods have been developed to further improve the typically in the range of 0.001–0.050. Based on Eq. (22), optimization performance, as detailed in Refs. [103–105]. if the metamodel approximation accuracy is satisfied (i.e., 4.2.3 Interesting sampling region method ϑ > 1), then B is increased to B /α to explore the k k k−1 The interesting sampling region method is a machine- global optimum; otherwise, B is decreased to B · α k k−1 learning-assisted method for locating the potential region to improve the metamodel approximation accuracy and containing the global optimum [106]. In this method, the exploit the local optimum. If the length in any dimension existing samples are classified into two categories with is no less than the threshold, then the kth trial significant (L) (U) respect to their responses. If the objective value at the design space is defined as S = [B ,B ], as shown in k k k sample point is less than the predefined threshold, P , Eq. (23): thresh the sample is labeled as a superior sample, indicating (L) B = x − B best k that the sample is probably located close to the global (23) (U) B = x + B best k k optimum. A binary machine learning classifier (e.g., support vector machine [107] and Bayes classifier [108]) To prevent the significant design space from exceeding is constructed based on labeled samples. Then, a larger the entire design space, S , the kth space, S , is defined 0 k number of inexpensive geometry points are generated as the intersection between S and S , as illustrated in via Latin hypercube design in the design space in which Fig. 10. superior inexpensive points are identified by the trained 4.2.2 Trust region method classifier. The cluster center of superior inexpensive The trust region method is a widely used space reduction points is determined to depict the potential location of method for refining metamodels, where the size of the the global optimum. For instance, consider the six camel- region of interest is adjusted according to the prediction hump functions; the superior inexpensive points (i.e., dots performance of the objective improvement [102], as shown labeled with red circles) under different classification in Eq. (24): thresholds (P ) are illustrated in Fig. 11. The figure thresh ∗ ∗ ∗ ∗ ∆f = (f − f )/(f − f ) (24) k−1 k k−1 k shows that the potential location can be depicted by the cluster of superior inexpensive points using a proper where ∆ f is the prediction performance of objective P value. thresh improvement. In Eq. (24), a larger ∆ f indicates that Finally, the interesting sampling region at the kth the optimization process can lead to a better optimum. iteration is defined as a hypercube subregion, as shown A smaller ∆ f indicates that the improvement in the in Eq. (26): objective is insignificant. Note that the objective cannot (k) (k) (k) (k) (k) be improved if t is negative. ISR = [x|x − R ⩽ x ⩽ x + R ] ISR ISR The size of the region can be determined in terms of (k) (k) ∗ where R = η ∥x − x ∥ (26) pse ISR the radius of the trust region, as shown in Eq. (25): (k) ∗ ∗ where the current pseudo-optimum (x ) is the center c ∥x − x ∥, ∆f < 0.1 ISR  k k−1 ∗ ∗ of the region; x is the cluster center of superior inex- pse δ = min(c ∥x − x ∥, ∆), ∆f > 0.75 k+1 2 k k−1 pensive points (also regarded as the potential position of ∗ ∗ ∥x − x ∥, 0.10 ⩽ ∆f ⩽ 0.075 k k−1 (k) global optimum); R is the Euclidean distance between 0 < c < 1, c > 1 (25) 1 2 (k) x and x ; and η is the shrinking coefficient used to ISR pse (k) adjust the size of R . As the optimization proceeds, where δ is the radius of the trust region at the (k+1)th k+1 ∗ ∗ iteration; x and x are the best sample points at the the interesting sampling region is dynamically scaled k k−1 (k) kth and (k − 1)th iterations, respectively; c and c are based on the Euclidean distance between x and x , 1 2 ISR pse Metamodel-based multidisciplinary design optimization methods for aerospace system 199 Classification threshold = 5 Classification threshold = 0 Classification threshold =  1 Classification threshold = 2 2 2 2 1 1 1 1 0 0 0 0 1  1 1  1 2  2 2  2 2  1 0 1 2  2  1 0 1 2 2  1 0 1 2  2  1 0 1 2 Fig. 11 Identification of superior inexpensive points. considerably improving the optimization efficiency and evolutionary algorithms, metamodels are developed as global convergence. Once the current pseudo-optimum “surrogates” of expensive simulation models for stochas- (k) (x ) is far from the potential global optimum (i.e., tic evolutionary operations. This can effectively alleviate ISR (k) (k−1) ∗ ∗ ∥x − x ∥ ⩾ ∥x − x ∥ ), the exploration re- the computational complexity in engineering optimiza- 2 2 pse ISR pse ISR gion must be enlarged to include areas that may not have tion. Several potential solutions (called individuals or been covered, thus avoiding missing the true optimum. particles) are also concurrently selected as newly added (k) Additionally, if x is probably in the vicinity of the sample points to improve the surrogate approximation ISR global optimum, then the search region can be reduced to accuracy in the vicinity of the optimum. The fundamen- improve the approximation accuracy of the constructed tal architecture of the algorithm is graphically illustrated metamodels. in Fig. 12. In addition to the aforementioned methods, many other The selection of newly added infill sample points from space reduction methods have been developed. For in- the offspring population is found to be a crucial step in the stance, Wang et al. proposed an adaptive response sur- algorithm [116]. The commonly used selection criteria can face method to solve computation-intensive design prob- be generally divided into three categories: performance- lems, where regions with inferior sample points are cut based, uncertainty-based, and hybrid criteria [118]. In for sampling in the reduced design space and updat- the performance-based criterion, the infill sample points ing the metamodels [109]. In addition, some variants are selected by considering only the predicted fitness of the adaptive response surface method were investi- values [117, 119, 120]. Generally, this criterion cannot gated to further improve optimization convergence and be exclusively used because optimization probably falls robustness [110–112]. Dong et al. proposed a new space into the local optima once the surrogates fail to approxi- reduction-based optimization algorithm to solve the un- mate the simulation model well. The uncertainty-based constrained expensive black-box optimization problems, criterion chooses infill sample points with considerable where a score-based reduced subspace around the current uncertainties, effectively improving the accuracy of surro- best sample is created to accelerate the local conver- gates and leading the search to unexplored regions [121]. gence [113]. Qiu et al. proposed self-organizing maps and However, numerous simulation model evaluations are fuzzy clustering-based three-stage space reduction and performed to explore regions with sparse samples, result- metamodeling optimization methods to improve efficiency ing in a low convergence rate [118]. The hybrid crite- and robustness performance [114]. Liu et al. developed rion simultaneously incorporates performance and uncer- a Monte Carlo method and space reduction strategy to improve the efficiency of the sampling process [115]. tainty [119,122]. The primary objective of this criterion is to balance the surrogate approximation accuracy and 4.3 Surrogate-assisted evolutionary algo- population competitiveness during optimization. rithms In recent years, a number of surrogate-assisted evo- lutionary algorithms have been successfully applied to Motivated by the idea of merging surrogates into the evolutionary process, surrogate-assisted evolutionary al- solve various optimization problems, such as uncon- gorithms have received considerable attention in recent strained/constrained [119,123–130], multi-objective [131– years [116,117]. In the optimization of surrogate-assisted 137], and multi-delit fi y optimization problems [138, 139]. 200 R. Shi, T. Long, N. Ye, et al. Fig. 12 Optimization mechanism of surrogate-assisted evolutionary algorithms. 4.4 Metamodel-based optimization en- To alleviate the computational complexity caused by hanced by machine learning the “curse of dimensionality”, the dimensionality reduc- tion technique provides a promising approach for MBDO Machine learning is a well-known state-of-the-art tech- to solve high-dimensional optimization problems. One nology that has been successfully applied to big data typical approach is to conduct sensitivity analysis before mining, natural language processing, image recognition, optimization starts. Here, primary design variables are medical diagnosis, and so on. Similarly, a considerable reserved for optimization, whereas minor ones are ne- amount of research has been conducted to integrate ma- glected and set as fixed values during optimization [148]. chine learning techniques (e.g., classification learning, Although the dimensionality of the optimization problem cluster analysis, and dimensionality reduction) into en- is reduced by sensitivity analysis, the optimality of the ob- gineering optimization. Classification learning is among tained optimized solution probably decreases owing to the the most widely used machine learning techniques in irrelevance of several design variables. Another dimen- optimization and has considerable potential for identify- sionality reduction method is the utilization of manifold ing the feasibility of a solution during the optimization learning (e.g., principal component analysis and Sammon process. Motivated by this concept, several studies have mapping) to map existing sample points together with been performed to improve the optimization performance alternative infill sample points from high-dimensional by employing a support vector machine or its variants for space into low-dimensional space. A low-dimensional identifying superior inexpensive points [140–144]. Addi- surrogate is also trained to select infill sample points as tionally, other classification learning techniques, such as optimization proceeds [149,150]. decision tree [145] and k-nearest neighbor [146], are em- In modern engineering design, because new problems ployed for optimization. To capture the potential regions are generally derived from a series of previously solved effectively, classified points are further categorized based tasks, the concept of transfer optimization has been re- on a set of user-dene fi d characteristics or attributes using cently developed. It is a newly emerged methodology cluster analysis techniques. For instance, cluster analy- for improving the optimization performance of a new sis can be combined with a support vector machine to identify a potential search region by clustering numerous optimization task by mining existing knowledge [151]. superior inexpensive points, as described in Refs. [21,106]. Although numerous transfer optimization-based methods In multi-objective optimization, cluster analysis can also have been recently investigated [152–154], only a few be implemented to improve the spatial uniformity of the studies on MBDO methods using transfer optimization Pareto frontier when determining newly infilled sample have been reported [155]. Therefore, further improving points or reference vectors [135, 136,147]. the optimization performance by integrating the transfer Metamodel-based multidisciplinary design optimization methods for aerospace system 201 optimization technique into MBDO methods is promising objective function is established for optimization, and the infeasible sample point is penalized with respect to and valuable [151]. the penalty factor scale. In addition to the aforementioned data-driven machine Although penalty function methods are conveniently learning techniques, physics-informed machine learning implemented in practice, determining the proper penalty techniques, also known as physics-informed neural net- factor is difficult. If the penalty factor is extremely small, works [156, 157], have emerged as an alternative to ill- the optimization process is virtually unable to converge posed and inverse problems. Fundamental physical laws to the feasible domain. If the penalty factor is extremely and domain knowledge are embedded by exploiting ob- large, the penalty part can overlap the objective, ren- servational data [158,159], tailoring neural network archi- dering convergence difficult. Nevertheless, the penalty tecture for physics constraints [160, 161], and/or impos- function method remains widely employed in MBDO ing physics constraints into the loss function [162, 163]. methods. For instance, a predened fi penalty function The physics-informed neural network is expected to out- is employed to handle the expensive constraints [164]. perform existing machine learning methods in partially This method is further enhanced in Ref. [110], where the understood, uncertain, and high-dimensional problems. penalty factor is automatically adjusted according to the It is also another potential research topic in the area of constraint violation value. metamodel-based MDO. 5.2 Filter-based methods 5 Expensive black-box constraint-hand- In view of the difficult implementation of penalty ling mechanism function-based methods, penalty-free methods have be- In engineering practice, MDO problems in aerospace come popular in recent years; among these latter meth- systems generally involve various constraints, such as ods, filter-based methods are the most representative. the natural frequencies of a structure and end-of-life The concept of a filter was originally proposed by power in orbit. Note that because most constraints arise Fletcher [165] and was successfully applied to many from time-consuming black-box simulation models (e.g., penalty-free numerical optimization methods [166–168]. structural FEA model), directly calculating the gradient To illustrate the concept of a filter, the constraint viola- information of the constraints via numerical methods tion function of the MDO problem is defined as follows: (e.g., finite difference method) to search for the feasible h(X) = max{g (X), g (X),··· , g (X)} (28) 1 2 m domain is difficult. Thus, in engineering practice, the solution of MDO problems with numerous expensive where h(X) denotes the constraint violation function. A black-box constraints remains complex. To overcome positive h(X) represents an infeasible sample point, and this obstacle, some constraint-handling mechanisms have a larger h(X) generally indicates inadequate infeasibility. been integrated with metamodel-based MDOs to further Based on the definition of the Pareto non-domination improve the optimization capacity. set, the concept of the filter is clarified as follows: Definition 1. If and only if f(x ) ⩽ f(x ) ∩ h(x ) ⩽ i l i 5.1 Penalty function-based methods h(x ), then sample point x is considered to dominate l i Penalty function-based methods are convenient ap- another sample point pair, x ; otherwise, x and x do l i l proaches that are most commonly used for handling not dominate each other. expensive black-box constraints. Here, the objective func- Definition 2. A filter aggregates non-dominated sample tion is augmented by constraints to cope with penalty points. If a sample point (x ) is not dominated by any factors, as given in Eq. (27): point in the current filter, then it can be included in the filter; otherwise, it is rejected. F(X) = f(X) + λ max(g (X), 0) (27) According to the above definitions, the construction where λ is the penalty factor, and F(X) is the augmented of a filter basically means the creation of a Pareto front objective function. The metamodel of the augmented in terms of f(X) and h(X). A filter in the f(X)–h(X) 202 R. Shi, T. Long, N. Ye, et al. space is illustrated in Fig. 13 [21], where the dots repre- Refs. [16,169,170], all constraints are approximated by metamodels to avoid the complicated determination of sent the non-dominated sample points. If a newly added the penalty factor. Moreover, some sequential upda- sample point is located in the shaded area, then it is not ting techniques considering the confidence intervals of dominated by existing points in the filter, that is, the Kriging models for constraint approximation were deve- filter is augmented. If it is located in the hatched area, loped [171,172]. To further solve problems with numerous then the newly added point can dominate at least one constraints, the constraint aggregation method, which point in the filter (i.e., the filter is refined); otherwise, lumps numerous constraints into one or a few constraints the point is rejected by the filter. In fact, the filter gen- to significantly reduce the computational cost, has drawn erates a set of blocks for newly added sample points, as considerable interest in recent years. The Kreisselmeier– indicated by the solid lines in Fig. 13. In constrained Steinhauser function [173] is a well-known constraint optimization procedures, the filter functions as a classi- aggregation method that has been used in various appli- fier for accepting or rejecting a sample point, considering cations [86,174,175]. Moreover, based on Refs. [176,177], both optimality and feasibility. Note that a sample point Kreisselmeier–Steinhauser function-based MBDO meth- can be accepted by the filter only if it improves either the ods are efficient in solving expensive black-box constraint optimality or feasibility; hence, the feasibility and opti- problems. mality of existing sample points are gradually enhanced during the metamodel-based optimization process. 6 Metamodel-based aerospace system Region for MDO examples f ( x) augmen tin g filter 6.1 All-electric geostationary Earth orbit Region for Rejecte d by filter refining filte r satellite MDO problem ( f ( x ), h( x )) The all-electric geostationary Earth orbit satellite MDO k k Non -dominated point s problem is referred from Ref. [19]. The studied all- electric geostationary Earth orbit satellite is a cuboid with a payload module, service module, and solar array. Four Hall-effect thrusters are installed at the bottom of the satellite to execute attitude control, geosynchronous transfer, and geostationary Earth orbit station-keeping maneuvers, as shown in Fig. 14 [19]. The all-electric geo- h( x) stationary Earth orbit satellite MDO problem involves geosynchronous transfer, geostationary keeping, power, Fig. 13 Filter in f(X) = h(X) space (reproduced with thermal control, attitude control, and structure disci- permission from Ref. [21], © The Authors 2018). plines. The design structure matrix of the MDO problem A number of MBDO methods have been developed to is shown in Fig. 15 [19], and the involved analysis models solve expensive constrained black-box problems based on of disciplines are detailed in Ref. [19]. the filter concept. For instance, the filter technique is The satellite MDO problem was formulated using combined with a support vector machine to identify a Eq. (29) [19]. The design variables are steer angles potential sampling sub-region in the vicinity of the global (α, β, φ ) during low-thrust geosynchronous transfer, po- feasible optimum [21]. In Ref. [72], the probability of sitions of electric thrusters (d , d ), battery capac- T N constrained improvement is enhanced; only the sample ity (C ), area of solar arrays (A ), area of thermal s sa point with a positive probability of constrained improve- radiator (A ), angular momentum of reaction wheel ment can be accepted by the filter to enhance optimality (H ), and thickness of composite material structure and feasibility. (SH, CH, TBH, SP, CP, TBP). These are optimized to In addition to the aforementioned constraint-handling minimize the total mass of the satellite (M ) sub- satellite mechanisms, certain customized methods have also ject to the constraints of total transfer time (t ), station- been developed to handle constrained optimization. In keeping accuracy (λ and i ), available power (P max max BOL Metamodel-based multidisciplinary design optimization methods for aerospace system 203 1 GTO 2 GEO 3 Power 4 Thermal 5 Attitude 6 Structure Fig. 15 Design structure matrix of all-electric satellite (re- produced with permission from Ref. [19], © IAA 2017). Table 3 Optimization results of design variables Genetic Filter-based algorithm-based sequential radial Objective multidisciplinary basis function feasible method method Total mass (kg) 2396.6 2355.9 Number of 1200 600 simulations 6.2 Earth-observation MDO problem The Earth observation satellite MDO problem is referred from Ref. [178]. The satellite consists of payload and service cabins, as shown in Fig. 16 [178]. The payloads include a 10-band radiometer and charge-coupled device camera, which are fixed at the bottom of the payload Fig. 14 Schematic of all-electric satellite (reproduced with cabin. The solar arrays are set along the south and north permission from Ref. [19], © IAA 2017). faces of the satellite to reduce the influence of external heat flux. Other facilities, such as battery and attitude and P ), depth of discharge (DOD), steady-state tem- EOL control subsystems, are placed in the service cabin. The perature (T), angular momentum residue (c ), and AC natural frequencies (f and f ). Earth observation satellite MDO problem involves five X Y disciplines: orbit, payload, structure, power, and mass. min M = m + m + m + m satellite payload fuel power TC The design structure matrix of this problem is shown in + m + m AC others Fig. 17 [178]. The disciplinary modeling approaches are presented in Ref. [178]. where X = [α, β, φ, d , d , A , C , A , H , SH, CH, T N sa s r w TBH, SP, CP, TBP] The Earth observation satellite MDO problem also ◦ ◦ aims to minimize the total mass of the satellite, as formu- t ⩽ 180 days, λ ⩽ 0.05 , i ⩽ 0.05 f max max lated in Eq. (30) [178]. The design variables include the P ⩾ 14.15 kW, P ⩾ 11.90 kW BOL EOL s.t. aperture and focal length of payloads, area of solar arrays, 267 K < T < 328 K, c ⩾ 0, DOD ⩽ 0.8 AC capacity of storage battery, and thickness of composite f ⩾ 14 Hz, f ⩾ 14 Hz X Y material structure. The constraints include the signal-to- (29) noise ratio and resolution of payloads, noise-equivalent In this example, the filter-based sequential radial basis temperature difference, surplus power, discharge depth, function method from Ref. [21] was used to solve the and natural frequencies. MDO problem. The optimization results are summarized in Table 3 [21]. Compared with the conventional genetic min M = m + m + m satellite payload bat solar algorithm-based multidisciplinary feasible method, the + m + m str others computational cost of the filter-based sequential radial basis function method is reduced by 50%. where X = [D , D , F , F , T , T , T , A , C ] 1 2 1 2 R BH SH sa s 204 R. Shi, T. Long, N. Ye, et al. Table 4 Comparison of optimization results Number of Optimization Total mass after analysis model method optimization (kg) evaluations Sequential radial basis function 340.11 165 using support vector machine Genetic algorithm 354.97 7329 Sequential quadratic 346.15 322 programming (a) sequential radial basis function using the support vector machine is only found to be 2.25% and 51.24% of the computational costs of genetic algorithm and sequential quadratic programming, respectively. 6.3 Satellite constellation MDO problem The satellite constellation MDO problem is obtained from Ref. [20], where a small satellite constellation is estab- lished to achieve a cooperative Earth observation mission. The design structure matrix of the satellite constellation MDO problem is shown in Fig. 18 [20]. The inter-coupled relationships between the constellation configuration and design of satellite subsystems are considered to enhance (b) system performance. In this example, the constellation is based on the Walker-δ constellation. In this constellation, Fig. 16 Schematic of Earth observation satellite (reproduced the ascending nodes of the orbit planes are uniformly with permission from Ref. [178], © Taylor & Francis 2015). distributed around the equator plane, and the satellites 1 Orbit are uniformly distributed within the orbital planes at the 2 Payload same inclination [20]. 3 Structure min M = (m + m + m system payload power thermal 4 Power + m + m ) × P × S structure others 5 Mass where X = [h, i, Ω , D , f , A , C , T , T ],X = [P, S] c 0 P P s s H P d Fig. 17 Design structure matrix of Earth observation satellite MDO problem (reproduced with permission from C ⩾ 0.8  R Ref. [178], © Taylor & Francis 2015). R ⩽ 250 m, SNR ⩾ 500 s.t. (31)  DOD ⩽ 0.3, g ⩾ 0 R ⩽ 1100 m, R ⩽ 250 m w 1 2       T ⩽ 303.15 K SNR ⩾ 300, SNR ⩾ 500,  1 2   f ⩾ 20 Hz, f ⩾ 20 Hz s.t. NE∆T ⩽ 0.2 K (30) X Y g ⩾ 0, DOD ⩽ 25%, f ⩾ 20 Hz,  w x  The constellation MDO problem was formulated using f ⩾ 20 Hz, f ⩾ 50 Hz y z Eq. (31). This is a continuous-discrete mixed optimiza- tion problem. The continuous design variables (X ) for In this example, the sequential radial basis function constellation configuration include the orbital altitude using a support vector machine from Ref. [106] was used to solve the MDO problem. The optimization results are (h), inclination (i), and right ascension of the ascend- listed in Table 4 [106]. The computational cost of the ing node (Ω); those for satellite subsystems include the Metamodel-based multidisciplinary design optimization methods for aerospace system 205 Fig. 18 Design structure matrix of satellite constellation MDO problem (reproduced with permission from Ref. [20], © Springer-Verlag GmbH Germany, part of Springer Nature 2018). optical parameters of the payload, power subsystem pa- pared with the integer coding-based genetic algorithm is rameters, and thickness of structural plates. The discrete reduced by more than 85%. The optimized configurations design variables (X ) include the number of orbit planes of the constellations are shown in Fig. 19 [20]. (P) and the number of satellites in each orbit plane (S). In addition to the aforementioned metamodel-based These design variables are optimized to minimize the aerospace system MDO examples, metamodeling tech- total mass of the entire constellation system, subject niques have also been widely used in the aerospace field. to the constraints of coverage ratio, payload resolution, For instance, Xiong et al. proposed an intelligent opti- signal-to-noise ratio of payload, depth of discharge, power mization strategy based on statistical machine learning surplus, maximum temperature, and natural frequencies. for spacecraft thermal design, where a neural network In this example, the sequential radial basis function metamodel was used to approximate the spacecraft– method using a support vector machine for discrete– thermophysical model [179]. Smith et al. established a continuous mixed variables is used to solve the MDO pro- charged drag coefficient metamodel for spacecraft deorbit blem. In the optimization method, a discrete–continuous design with ionospheric drag from a low Earth orbit [180]. variable sampling method is utilized to handle the dis- Wu et al. developed a double-layer radial basis function crete variables. The optimization results, summarized metamodel-based optimization method for the liquid– in Table 5 [20], are compared with those from the inte- gas interface determination of on-orbit refueling [181]. ger coding-based genetic algorithm. This indicates that Peng and Wang proposed a dynamic metamodel-based the sequential radial basis function method considerably optimization method for spacecraft formation reconfig- decreases the system mass by approximately 28.63%. uration on liberation point orbits to achieve fast path Moreover, the computational budget of this method com- planning [182]. Feldhacker et al. investigated a sampling- based least-squares regression method to optimize the Table 5 System mass of optimized designs required velocity increments for rendezvous in spacecraft Sequential radial three-body dynamical systems [183]. Peng et al. pro- basis function Integer method for posed an adaptive metamodel-based optimization method Parameters coding-based discrete– called the sequential radial basis function for satellite genetic algorithm continuous mixed truss structure optimization [184]. Gogu et al. proposed variables a response surface approximation method to aid in the Mass of single 330.4 kg 314.5 kg satellite design of integrated thermal protection systems for space- Total mass of craft reentry [185]. From the literature, MBDO methods satellite 5287.2 kg 3773.3 kg are reported as effective and promising for solving com- constellation system plex aerospace system optimization problems. Simpson Number of et al. applied Kriging models to an aerospike nozzle MDO >2000 300 simulations problem for global approximation [186]. 206 R. Shi, T. Long, N. Ye, et al. timization strategies are discussed in detail. Generally, metamodel-based optimization strategies can be classi- fied into two categories: infill criterion-based and space reduction-based optimization strategies. Recently, novel techniques, such as evolution computation and machine learning, have also been integrated with metamodels to further improve optimization performance. Finally, some penalty-based and penalty-free constraint-handling mechanisms are discussed to illustrate how to deal with expensive black-box constraints in the metamodel-based optimization process. Additionally, several real-world aerospace system MDO examples, which can aid re- searchers in realizing the applications of metamodel-based MDO in engineering practice, are presented. Note that (a) Optimized design by integer coding-based genetic algorithm although the contents of this paper are focused on the MDO for aerospace systems, the fundamental metho- dology and technology can also be applied to the design of other engineering systems, such as ships, aircraft, and automobiles, without significant modifications. In future work, the optimization capacity of metamodel-based MDO must be further improved. For instance, due to the “curse of dimensionality”, high- dimensional expensive black-box problems remain com- plex for metamodel-based optimization. In addition, conventional metamodel-based MDOs generally consider multidisciplinary models as black-box functions. By ex- ploiting prior knowledge (e.g., designer’s experience, ex- plicit model expression, and physical laws), the black-box MDO problem can be transformed into a gray-box or (b) Design by sequential radial basis function method even a white-box optimization problem to further reduce cost and improve convergence. Moreover, the combina- Fig. 19 Comparison of constellation configurations (repro- tion of metamodeling, MDO, and artificial intelligence duced with permission from Ref. [20], © Springer-Verlag GmbH Germany, part of Springer Nature 2018). has also attracted considerable interest. Thus, with the development of advanced modeling, analysis, and com- 7 Conclusions putation technologies, the research on metamodel-based In this paper, an overview of the metamodel-based MDO MDO is anticipated to remain attractive. and its application to aerospace systems is presented. The formulation of the MDO problem for aerospace systems Acknowledgements and the overall procedure of the metamodel-based opti- This work was supported by the National Natural Science mization process are introduced. Moreover, the technical Foundation of China (Nos. 52005288 and 51675047) architecture of metamodel-based MDO is summarized. and the Aeronautical Science Foundation of China (No. Metamodeling techniques, including design of experi- 2019ZC072003). ments, metamodels, and multi-model fusion methods, are surveyed with the aim of constructing a reduced- References order numerical approximation model to inexpensively predict system responses during optimization. A number [1] Sobieszczanski-Sobieski, J., Haftka, R. T. Multidisci- of state-of-the-art metamodel-based multidisciplinary op- plinary aerospace design optimization: Survey of recent Metamodel-based multidisciplinary design optimization methods for aerospace system 207 developments. Structural Optimization, 1997, 14(1): [14] Lai, G., Lei, J., Liu, J., Cao, S., Qin, H., Zeng, F., Zhou, 1–23. R. Numerical and experimental study on comprehensive optimization for the KPIs of ship propulsion shafting [2] Sobieszczanski-Sobieski, J. Multidisciplinary design op- design based on MDO. Ocean Engineering, 2021, 222: timization: Attempt at definition. In: Proceedings of the Industry University Workshop on Multidisciplinary Aircraft Design, 1993: 23–48. [15] Liu, Q., Feng, B., Liu, Z. Ship MDO modelling based [3] Martins, J. R. R. A., Lambe, A. B. Multidisciplinary on design structure matrix. Journal of South China design optimization: A survey of architectures. AIAA University of Technology (Natural Science Edition), Journal, 2013, 51(9): 2049–2075. 2015, 43(04): 69–77. (in Chinese) [4] Grossman, B., Gurdal, Z., Strauch, G. J., Eppard, W. [16] Lei, G., Zhu, J., Guo, Y. Multidisciplinary Design Op- M., Haftka, R. T. Integrated aerodynamic/structural timization Methods for Electrical Machines and Drive design of a sailplane wing. Journal of Aircraft, 1988, Systems. Springer-Verlag Berlin Heidelberg, 2016. 25(9): 855–860. [17] Lei, G., Zhu, J., Guo, Y., Liu, C., Ma, B. A review of [5] Livne, E., Schmit, L. A., Friedmann, P. P. Towards design optimization methods for electrical machines. integrated multidisciplinary synthesis of actively con- Energies, 2017, 10(12): 1962. trolled fiber composite wings. Journal of Aircraft, 1990, [18] Wang, Y., Teng, H. Knowledge fusion design method: 27(12): 979–992. Satellite module layout. Chinese Journal of Aeronau- [6] Wang, S., He, L. The summarization multidisciplinary tics, 2009, 22(1): 32–42. design optimization for flying vehicles. Journal of As- [19] Shi, R., Liu, L., Long, T., Liu, J., Yuan, B. Surrogate tronautics, 2004, 25(6): 697–701. (in Chinese) assisted multidisciplinary design optimization for an [7] Chai, X., Yu, X., Wang, Y. Multipoint optimization all-electric GEO satellite. Acta Astronautica, 2017, on fuel efficiency in conceptual design of wide-body 138: 301–317. aircraft. Chinese Journal of Aeronautics, 2018, 31(1): [20] Shi, R., Liu, L., Long, T., Wu, Y., Wang, G. G. Multi- 99–106. disciplinary modeling and surrogate assisted optimiza- [8] Liu, Y. B., Chen, B. Y., Li, Y. H., Shen, H. D. Overview tion for satellite constellation systems. Structural and of control-centric integrated design for hypersonic ve- Multidisciplinary Optimization, 2018, 58(5): 2173– hicles. Astrodynamics, 2018, 2(4): 307–324. [9] Giannella, V., Lombardi, R., Pisani, M. M., Federico, [21] Shi, R., Liu, L., Long, T., Wu, Y., Tang, Y. Filter-based L., Barbarino, M., Citarella, R. A novel optimization sequential radial basis function method for spacecraft framework to replicate the vibro-acoustics response of multidisciplinary design optimization. AIAA Journal, an aircraft fuselage. Applied Sciences, 2020, 10(7): 2018, 57(3): 1019–1031. [22] Feng, X. Multidisciplinary design optimisation meth- [10] Wang, W., Wu, Z., Wang, D., Yang, J., Wang, P., ods and their application in the design of navigation Zhang, W. Hypersonic vehicle aerodynamic optimiza- constellations. Ph.D. Dissertation. Changsha, China: tion using field metamodel-enhanced sequential approx- University of Defence Science and Technology, 2008. imate optimization. International Journal of Aerospace [23] Guo, S., Zhou, W. M., Zhang, J., Sun, F. Y., Yu, D. T. Engineering, 2021, 2021: 8889593. Integrated constellation design and deployment method [11] Wang, H., Shan, S., Wang, G. G., Li, G. Integrating for a regional augmented navigation satellite system least square support vector regression and mode pursu- using piggyback launches. Astrodynamics, 2021, 5(1): ing sampling optimization for crashworthiness design. 49–60. Journal of Mechanical Design, 2011, 133(4): 041002. [24] Brown, N., Olds, J. Evaluation of multidisciplinary op- [12] Henric, P., Maria, R., Gerald, M., Hamid, H. Deve- timazation (MDO) techniques applied to a reusable lopment of synthesis level design model in automobile launch vehicle. In: Proceedings of the 43rd AIAA application suitable for MDO using CO approach. SAE Aerospace Sciences Meeting and Exhibit, 2005: AIAA International Journal of Materials and Manufacturing, 2005-707. 2015, 8(2): 344–356. [13] Xue, Z., Adarsh, E., Fang, J. Multidisciplinary design [25] Castellini, F., Lavagna, M. R., Riccardi, A., Busk ¨ ens, optimization of vehicle weight reduction. SAE Inter- C. Quantitative assessment of multidisciplinary de- national Journal of Materials & Manufacturing, 2016, sign models for expendable launch vehicles. Journal of 9(2): 393–399. Spacecraft and Rockets, 2013, 51(1): 343–359. 208 R. Shi, T. Long, N. Ye, et al. [26] Zhang, P., Yang, J., Luo, Z., Li, X. Collaborative op- [37] Johnson, M. E., Moore, L. M., Ylvisaker, D. Minimax timization design of suspended rocket based on re- and maximin distance designs. Journal of Statistical sponse surface. Journal of Solid Rocket Technology, Planning and Inference, 1990, 26(2): 131–148. 2017, 40(6): 793–798. (in Chinese) [38] Koehler, J. R., Owen, A. B. 9 computer experiments. [27] Liu, Z., Zhang, B. An overview of multidisciplinary Handbook of Statistics, 1996, 13: 261–308. optimization method in launch vehicle design. Astro- [39] Bates, S., Sienz, J., Toropov, V. Formulation of the nautical Systems Engineering Technology, 2017, 1(2): optimal Latin hypercube design of experiments using 1–6. a permutation genetic algorithm. In: Proceedings of [28] Wang, G. G., Shan, S. Review of metamodeling tech- the 45th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics & Materials Conference, 2004: niques in support of engineering design optimization. AIAA 2004-2011. Journal of Mechanical Design, 2007, 129(4): 370–380. [40] Tanabe, R., Fukunaga, A. S. Improving the search [29] Forrester, A. I. J., Keane, A. J. Recent advances in surrogate-based optimization. Progress in Aerospace performance of shade using linear population size re- Sciences, 2009, 45(1–3): 50–79. duction. In: Proceedings of the 2014 IEEE Congress on Evolutionary Computation New York, 2014: 1658– [30] Han, Z., Xu, C., Qiao, J., Liu, F., Chi, J., Meng, G., Zhang, K., Song, W. Recent progress of efficient global [41] Jin, R., Chen, W., Sudjianto, A. An efficient algorithm aerodynamic shape optimization using surrogate-based for constructing optimal design of computer experi- approach. Acta Aeronautica et Astronautica Sinica, ments. Journal of Statistical Planning and Inference, 2020, 41(5): 31–47. (in Chinese) 2005, 134(1): 268–287. [31] De Weck, O., Agte, J., Sobieszczanski-Sobieski, J., [42] Ye, K. Q., Li, W., Sudjianto, A. Algorithmic construc- Arendsen, P., Morris, A., Spieck, M. State-of-the-art tion of optimal symmetric Latin hypercube designs. and future trends in multidisciplinary design optimiza- Journal of Statistical Planning and Inference, 2000, tion. In: Proceedings of the 48th AIAA/ASME/ASCE/ 90(1): 145–159. AHS/ASC Structures, Structural Dynamics, and Ma- terials Conference, 2007: AIAA 2007-1905. [43] Zhu, H., Liu, L., Long, T., Peng, L. A novel algorithm of maximin Latin hypercube design using successive [32] Simpson, T. W., Toropov, V., Balabanov, V., Viana, local enumeration. Engineering Optimization, 2012, F. A. C. Design and analysis of computer experiments 44(5): 551–564. in multidisciplinary design optimization: A review of how far we have come - or not. In: Proceedings of [44] Viana, F. A. C., Venter, G., Balabanov, V. An algo- the 12th AIAA/ISSMO Multidisciplinary Analysis and rithm for fast optimal Latin hypercube design of exper- Optimization Conference, 2008: AIAA 2008-5802. iments. International Journal for Numerical Methods in Engineering, 2010, 82(2): 135–156. [33] Queipo, N. V., Haftka, R. T., Shyy, W., Goel, T., [45] Gunst, R. F. Response surface methodology: process Vaidyanathan, R., Kevin Tucker, P. Surrogate-based analysis and optimization. Progress in Aerospace Sci- and product optimization using designed experiments. ences, 2005, 41(1): 1–28. Technometrics, 1996, 38(3): 284–286. [34] Viana, F. A. C., Simpson, T. W., Balabanov, V., [46] Buhmann, M. D. Radial basis functions: Theory and implementations. Cambridge: Cambridge university Toropov, V. Special section on multidisciplinary design press, 2003. optimization: Metamodeling in multidisciplinary de- sign optimization: How far have we really come? AIAA [47] Viana, F. A. C., Haftka, R. T., Steffen, V. Multiple Journal, 2014, 52(4): 670–690. surrogates: How cross-validation errors can help us to obtain the best predictor. Structural and Multidisci- [35] Simpson, T. W., Booker, A. J., Ghosh, D., Giunta, A. plinary Optimization, 2009, 39(4): 439–457. A., Koch, P. N., Yang, R. J. Approximation methods in multidisciplinary analysis and optimization: A panel [48] Song, X., Lv, L., Li, J., Sun, W., Zhang, J. An advanced discussion. Structural and Multidisciplinary Optimiza- and robust ensemble surrogate model: Extended adap- tion, 2004, 27(5): 302–313. tive hybrid functions. Journal of Mechanical Design, 2018, 140(4): 041402. [36] Long, T., Liu, J., Wang, G. G., Liu, L., Shi, R., Guo, X. Discuss on approximate optimization strategies using [49] Acar, E. Simultaneous optimization of shape param- design of computer experiments and metamodels for eters and weight factors in ensemble of radial basis flight vehicle design. Journal of Mechanical Engineering, functions. Structural and Multidisciplinary Optimiza- 2016, 52(14): 79–105. (in Chinese) tion, 2014, 49(6): 969–978. Metamodel-based multidisciplinary design optimization methods for aerospace system 209 [50] Clarke, S. M., Griebsch, J. H., Simpson, T. W. Anal- and kriging correction. Journal of Engineering Design, ysis of support vector regression for approximation 2013, 24(8): 604–622. [63] Kennedy, M., O’Hagan, A. Predicting the output from of complex engineering analyses. Journal of Mechani- a complex computer code when fast approximations cal Design, Transactions of the ASME, 2005, 127(6): are available. Biometrika, 2000, 87(1): 1–13. 1077–1087. [64] Qian, P. Z. G., Wu, C. F. J. Bayesian hierarchical mod- [51] Awad, M., Khanna, R. Support vector regression. eling for integrating low-accuracy and high-accuracy In: Efficient Learning Machines: Theories, Concepts, experiments. Technometrics, 2008, 50(2): 192–204. and Applications for Engineers and System Designers. [65] Le Gratiet, L. Recursive co-kriging model for Design of Berkeley, CA: Apress. 2015: 67–80. Computer experiments with multiple levels of fidelity [52] Haykin, S. Neural networks: A comprehensive founda- with an application to hydrodynamic. 2012: arXiv: tion. New York: Macmillan College Publishing Com- 1210.0686[math.ST]. Available at https://arxiv.org/ pany, 1998. abs/1210.0686. [53] Damianou, A., Lawrence, N. D. Deep Gaussian pro- [66] Han, Z. H., G¨ortz, S. Hierarchical kriging model for cesses. In: Proceedings of the 16th International Con- variable-fidelity surrogate modeling. AIAA Journal, ference on Artificial Intelligence and Statistics, 2013. 2012, 50(9): 1885–1896. [54] Salimbeni, H., Deisenroth, M. P. Doubly stochastic [67] Zhang, Y., Han, Z. H., Zhang, K. S. Variable-fidelity variational inference for deep Gaussian processes. In: expected improvement method for efficient global opti- Proceedings of the 31st International Conference on mization of expensive functions. Structural and Multi- Neural Information Processing Systems, 2017. disciplinary Optimization, 2018, 58(4): 1431–1451. [55] Rajaram, D., Puranik, T. G., Ashwin Renganathan, S., [68] Han, Z. H., G¨ortz, S. Alternative cokriging method Sung, W., Fischer, O. P., Mavris, D. N., Ramamurthy, for variable-fidelity surrogate modeling. AIAA Journal, A. Empirical assessment of deep Gaussian process sur- 2012, 50(5): 1205–1210. rogate models for engineering problems. Journal of [69] Bertram, A., Zimmermann, R. Theoretical investi- Aircraft, 2020, 58(1): 182–196. gations of the new Cokriging method for variable- [56] Forrester, A. I. J., S´obester, A., Keane, A. J. Multi- fidelity surrogate modeling. Advances in Computational fidelity optimization via surrogate modelling. Proceed- Mathematics, 2018, 44(6): 1693–1716. ings of the Royal Society A: Mathematical, Physical [70] Toal, D. J. J. Some considerations regarding the use of and Engineering Science, 2007, 463: 3251–3269. multi-fidelity Kriging in the construction of surrogate [57] Park, C., Haftka, R. T., Kim, N. H. Remarks on multi- models. Structural and Multidisciplinary Optimization, fidelity surrogates. Structural and Multidisciplinary Op- 2015, 51(6): 1223–1245. timization, 2017, 55(3): 1029–1050. [71] Jones, D. R., Schonlau, M., Welch, W. J. Efficient [58] Le Gratiet, L. Multi-fidelity Gaussian process regression global optimization of expensive black-box functions. for computer experiments, Universite´ Paris-Diderot - Journal of Global Optimization, 1998, 13(4): 455–492. Paris VII, 2013. [72] Shi, R., Liu, L., Long, T., Wu, Y., Tang, Y. Filter- [59] Zhou, Q., Yang, Y., Song, X., Han, Z., Cheng, Y., Hu, based adaptive Kriging method for black-box optimiza- J., Shu, L., Jiang, P. Survey of multi-fidelity surrogate tion problems with expensive objective and constraints. models and their applications in the design and opti- Computer Methods in Applied Mechanics and Engineer- mization of engineering equipment. Journal of Mechan- ing, 2019, 347: 782–805. ical Engineering, 2020, 56(24): 219–245. (in Chinese) [73] Long, T., Wei, Z., Shi, R. H., Wu, Y. F. Parallel adap- [60] Liu, H., Ong, Y. S., Cai, J., Wang, Y. Cope with diverse tive kriging method with constraint aggregation for ex- data structures in multi-fidelity modeling: A Gaussian pensive black-box optimization problems. AIAA Jour- process method. Engineering Applications of Artificial nal, 2021: 1–15. Intelligence, 2018, 67: 211–225. [74] Amine Bouhlel, M., Bartoli, N., Regis, R. G., Otsmane, [61] Fischer, C. C., Grandhi, R. V. Utilizing an adjustment A., Morlier, J. Efficient global optimization for high- factor to scale between multiple fidelities within a design dimensional constrained problems by using the Kriging process: A stepping stone to dialable fidelity design. models combined with the partial least squares method. In: Proceedings of the 16th AIAA Non-Deterministic Engineering Optimization, 2018, 50(12): 2038–2053. Approaches Conference, 2014: AIAA 2014-1011. [75] Xu, S. G., Chen, H. Q., Zhang, J. L. A study of Nash- [62] Zheng, J., Shao, X. Y., Gao, L., Jiang, P., Li, Z. L. A EGO algorithm for aerodynamic shape design optimiza- hybrid variable-fidelity global approximation modelling tions. Structural and Multidisciplinary Optimization, method combining tuned radial basis function base 2019, 59(4): 1241–1254. 210 R. Shi, T. Long, N. Ye, et al. [76] Viana, F. A. C., Haftka, R. T., Watson, L. T. Effi- [87] Booker, A. J., Dennis, J. E., Frank, P. D., Serafini, D. cient global optimization algorithm assisted by multiple B., Torczon, V., Trosset, M. W. A rigorous framework surrogate techniques. Journal of Global Optimization, for optimization of expensive functions by surrogates. 2013, 56(2): 669–689. Structural Optimization, 1999, 17(1): 1–13. [77] He, Y., Sun, J., Song, P., Wang, X. Variable-fidelity [88] Jones, D. R. A taxonomy of global optimization meth- ods based on response surfaces. Journal of Global Op- expected improvement based efficient global optimiza- timization, 2001, 21(4): 345–383. tion of expensive problems in presence of simulation failures and its parallelization. Aerospace Science and [89] Watson, A. G., Barnes, R. J. Infill sampling criteria to Technology, 2021, 111: 106572. locate extremes. Mathematical Geology, 1995, 27(5): [78] Zhan, D. W., Xing, H. L. Expected improvement for 589–608. expensive optimization: A review. Journal of Global [90] Keane, A. J., Forrester, A. I. J., S´obester, A. Engineer- Optimization, 2020, 78(3): 507–544. ing Design via Surrogate Modelling: A Practical Guide. John Wiley & Sons, Ltd., 2008. [79] Wang, L., Shan, S., Wang, G. G. Mode-pursuing sam- pling method for global optimization on expensive [91] Martin, J., Simpson, T. W. Use of adaptive metamod- black-box functions. Engineering Optimization, 2004, eling for design optimization. In: Proceedings of the 36(4): 419–438. 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, 2002: 2002–5631. [80] Branin, F. H. J., Hoo, S. K. A method for finding multiple extrema of a function of n variables. Numerical [92] van Beers, W. C. M., Kleijnen, J. P. C. Kriging for Methods, 1972: 231–237. interpolation in random simulation. Journal of the Op- erational Research Society, 2003, 54(3): 255–262. [81] Wu, Y. F., Long, T., Shi, R. H., Wang, G. G. Mode- pursuing sampling method using discriminative coordi- [93] Cox, D. D., John, S. A statistical method for global nate perturbation for high-dimensional expensive black- optimization. In: Proceedings of the 1992 IEEE Inter- box optimization. Journal of Mechanical Design, 2021, national Conference on Systems, Man, and Cybernetics, 143(4): 041703. 1992: 1241–1246. [82] Kazemi, M., Wang, G. G., Rahnamayan, S., Gupta, [94] Long, T., Liu, L., Peng, L. Global optimization method with enhanced adaptive response surface method for K. Constraint importance mode pursuing sampling for computation-intensive design problems. Advanced Sci- continuous global optimization. In: Proceedings of the ASME 2010 International Design Engineering Tech- ence Letters, 2012, 5(2): 881–887. nical Conferences and Computers and Information in [95] Jiang, P., Cheng, J., Zhou, Q., Shu, L. S., Hu, J. X. Engineering Conference, 2011: 325–334. Variable-fidelity lower confidence bounding approach for engineering optimization problems with expensive [83] Sharif, B., Wang, G. G., Elmekkawy, T. Y. Mode pursu- simulations. AIAA Journal, 2019, 57(12): 5416–5430. ing sampling method for discrete variable optimization on expensive black-box functions. Journal of Mechani- [96] Zhu, H. G., Liu, L., Long, T., Zhao, J. F. Global opti- cal Design, 2008, 130(2): 021402. mization method using SLE and adaptive RBF based on fuzzy clustering. Chinese Journal of Mechanical En- [84] Wang, G. G., Shan, S. Q. An efficient Pareto set iden- gineering, 2012, 25(4): 768–775. tification approach for multi-objective optimization on black-box functions. In: Proceedings of the ASME 2004 [97] Tian, K., Li, Z. C., Huang, L., Du, K. F., Jiang, L. L., International Design Engineering Technical Conferences Wang, B. Enhanced variable-fidelity surrogate-based and Computers and Information in Engineering Con- optimization framework by Gaussian process regression ference, 2008: 279–291. and fuzzy clustering. Computer Methods in Applied Mechanics and Engineering, 2020, 366: 113045. [85] Cheng, G. H., Younis, A., Haji Hajikolaei, K., Wang, G. G. Trust region based mode pursuing sampling [98] Zhou, Y. C., Lu, Z. Z. An enhanced Kriging surrogate method for global optimization of high dimensional modeling technique for high-dimensional problems. Me- design problems. Journal of Mechanical Design, 2015, chanical Systems and Signal Processing, 2020, 140: 137(2): 021407. 106687. [86] Cheng, G. H., Gjernes, T., Wang, G. G. An adaptive [99] Han, Z. Kriging surrogate model and its application aggregation-based approach for expensively constrained to design optimization: A review of recent progress. black-box optimization problems. Journal of Mechanical Acta Aeronautica et Astronautica Sinica, 2016, 37(11): Design, 2018, 140(9): 091402. 3197–3225. (in Chinese) Metamodel-based multidisciplinary design optimization methods for aerospace system 211 [100] Gao, Y., Wang, X. A sequential optimization method [112] Wu, D., Long, T., Li, Y. L., Jiang, M. L., Huang, B. with multi-point sampling criterion based on Kriging Aero-structure-stealth coupled optimization for high surrogate model. Engineering Mechanics, 2012, 29(4): aspect ratio wing using adaptive metamodeling method. 90–95. (in Chinese) 15th AIAA/ISSMO Multidisciplinary Analysis and Op- [101] Song, C., Yang, X. D., Song, W. P. Multi-infill strategy timization Conference, 2014. [113] Dong, H. C., Sun, S. Q., Song, B. W., Wang, P. Multi- for kriging models used in variable fidelity optimization. surrogate-based global optimization using a score-based Chinese Journal of Aeronautics, 2018, 31(3): 448–456. infill criterion. Structural and Multidisciplinary Opti- [102] Alexandrov, N. M., Dennis, J. E., Lewis, R. M., Torc- mization, 2019, 59(2): 485–506. zon, V. A trust-region framework for managing the use [114] Qiu, H. B., Xu, Y. J., Gao, L., Li, X. K., Chi, L. of approximation models in optimization. Structural Multi-stage design space reduction and metamodeling Optimization, 1998, 15(1): 16–23. optimization method based on self-organizing maps and [103] Perez, ´ V. M., Renaud, J. E., Watson, L. T. Adaptive fuzzy clustering. Expert Systems With Applications, experimental design for construction of response surface 2016, 46: 180–195. approximations. AIAA Journal, 2002, 40(12): 2495– [115] Liu, H. T., Xu, S. L., Wang, X. F. Sequential sam- pling designs based on space reduction. Engineering [104] Gano, S. E., Renaud, J. E., Martin, J. D., Simpson, T. Optimization, 2015, 47(7): 867–884. W. Update strategies for kriging models used in variable [116] Jin, Y. C. Surrogate-assisted evolutionary computation: fidelity optimization. Structural and Multidisciplinary Recent advances and future challenges. Swarm and Optimization, 2006, 32(4): 287–298. Evolutionary Computation, 2011, 1(2): 61–70. [105] Cheng, G., Wang, G. F. Trust region based MPS [117] Jin, Y. C., Wang, H. D., Chugh, T., Guo, D., Miettinen, method for global optimization of high dimensional K. Data-driven evolutionary optimization: An overview design problems. In: Proceedings of the 53rd AIAA/ and case studies. IEEE Transactions on Evolutionary ASME/ASCE/AHS/ASC Structures, Structural Dy- Computation, 2019, 23(3): 442–458. namics and Materials Conference, 2012: AIAA 2012- [118] Li, F., Shen, W. M., Cai, X. W., Gao, L., Wang, G. G. A fast surrogate-assisted particle swarm optimiza- [106] Shi, R. H., Liu, L., Long, T., Liu, J. Sequential radial tion algorithm for computationally expensive problems. basis function using support vector machine for expen- Applied Soft Computing, 2020, 92: 106303. sive design optimization. AIAA Journal, 2016, 55(1): [119] Yang, Z., Qiu, H. B., Gao, L., Cai, X. W., Jiang, 214–227. C., Chen, L. M. Surrogate-assisted classification- [107] Suthaharan, S. Support vector machine. In: Machine collaboration differential evolution for expensive con- Learning Models and Algorithms for Big Data Classifi- strained optimization problems. Information Sciences, cation. Boston, MA: Springer US, 2016: 207–235. 2020, 508: 50–63. [108] Rish, I. An empirical study of the naive Bayes classi- [120] Wang, X. J., Wang, G. G., Song, B. W., Wang, P., fier. In: Proceedings of the IJCAI 2001 Workshop on Wang, Y. A novel evolutionary sampling assisted opti- Empirical Methods in Artificial Intelligence, 2001. mization method for high-dimensional expensive prob- [109] Wang, G. G., Dong, Z. M., Aitchison, P. Adaptive lems. IEEE Transactions on Evolutionary Computation, response surface method - a global optimization scheme 2019, 23(5): 815–827. for approximation-based design problems. Engineering [121] Tian, J., Tan, Y., Zeng, J. C., Sun, C. L., Jin, Optimization, 2001, 33(6): 707–733. Y. C. Multiobjective infill criterion driven Gaussian [110] Long, T., Wu, D., Guo, X. S., Wang, G. G., Liu, process-assisted particle swarm optimization of high- L. Efficient adaptive response surface method using dimensional expensive problems. IEEE Transactions intelligent space exploration strategy. Structural and on Evolutionary Computation, 2019, 23(3): 459–472. Multidisciplinary Optimization, 2015, 51(6): 1335– [122] Wang, H. D., Jin, Y. C., Doherty, J. Committee-based active learning for surrogate-assisted particle swarm [111] Long, T., Liu, L., Peng, L., Li, Y. Aero-structure cou- optimization of expensive problems. IEEE Transactions pled optimization of high aspect ratio wing using en- on Cybernetics, 2017, 47(9): 2664–2677. hanced adaptive response surface method. In: Proceed- [123] Glaz, B., Friedmann, P. P., Liu, L., Cajigas, J. G., ings of the 12th AIAA Aviation Technology, Integration, Bain, J., Sankar, L. N. Reduced-order dynamic stall and Operations Conference and 14th AIAA/ISSMO modeling with swept flow effects using a surrogate-based Multidisciplinary Analysis and Optimization Confer- recurrence framework. AIAA Journal, 2013, 51(4): ence, 2012: AIAA 2012-5456. 910–921. 212 R. Shi, T. Long, N. Ye, et al. [124] Sun, C. L., Zeng, J. C., Pan, J., Xue, S. D., Jin, Y. C. [136] Chugh, T., Jin, Y. C., Miettinen, K., Hakanen, J., Sind- A new fitness estimation strategy for particle swarm hya, K. A surrogate-assisted reference vector guided optimization. Information Sciences, 2013, 221: 355– evolutionary algorithm for computationally expensive 370. many-objective optimization. IEEE Transactions on [125] Vincenzi, L., Gambarelli, P. A proper infill sampling Evolutionary Computation, 2018, 22(1): 129–142. strategy for improving the speed performance of a [137] Wan, K. Z., He, C., Camacho, A., Shang, K., Cheng, Surrogate-Assisted Evolutionary Algorithm. Computers R., Ishibuchi, H. A hybrid surrogate-assisted evolu- & Structures, 2017, 178: 58–70. tionary algorithm for computationally expensive many- [126] Miranda-Varela, M. E., Mezura-Montes, E. Constraint- objective optimization. In: Proceedings of the 2019 handling techniques in surrogate-assisted evolutionary IEEE Congress on Evolutionary Computation, 2019: optimization. An empirical study. Applied Soft Com- 2018–2025. [138] Liu, B., Koziel, S., Zhang, Q. F. A multi-fidelity puting, 2018, 73: 215–229. [127] Wang, Y., Yin, D. Q., Yang, S. X., Sun, G. Y. Global surrogate-model-assisted evolutionary algorithm for and local surrogate-assisted differential evolution for computationally expensive optimization problems. expensive constrained optimization problems with in- Journal of Computational Science, 2016, 12: 28–37. equality constraints. IEEE Transactions on Cybernet- [139] Yi, J., Shen, Y. C., Shoemaker, C. A. A multi-fidelity ics, 2019, 49(5): 1642–1656. RBF surrogate-based optimization framework for com- [128] Zhao, X. G., Zhang, K., Chen, G. D., Xue, X. M., putationally expensive multi-modal problems with ap- Yao, C. J., Wang, J., Yang, Y. F., Zhao, H., Yao, J. plication to capacity planning of manufacturing sys- Surrogate-assisted differential evolution for production tems. Structural and Multidisciplinary Optimization, optimization with nonlinear state constraints. Jour- 2020, 62(4): 1787–1807. [140] Xie, T. L., Jiang, P., Zhou, Q., Shu, L. S., Zhang, Y. nal of Petroleum Science and Engineering, 2020, 194: H., Meng, X. Z., Wei, H. Advanced multi-objective [129] Zhang, K., Zhao, X. G., Chen, G. D., Zhao, M. J., robust optimization under interval uncertainty using Wang, J., Yao, C. J., Sun, H., Yao, J., Wang, W., kriging model and support vector machine. Journal of Zhang, G. D. A double-model differential evolution Computing and Information Science in Engineering, for constrained waterflooding production optimization. 2018, 18(4): 041012. Journal of Petroleum Science and Engineering, 2021, [141] Malak, R. J. Jr, Paredis, C. J. J. Using support vector machines to formalize the valid input domain of pre- 207: 109059. [130] Yu, M. Y., Li, X., Liang, J. A dynamic surrogate- dictive models in systems design problems. Journal of assisted evolutionary algorithm framework for expen- Mechanical Design, 2010, 132(10): 101001. [142] Liu, X., Wu, Y. Z., Wang, B. X., Ding, J. W., Jie, H. X. sive structural optimization. Structural and Multidisci- An adaptive local range sampling method for reliability- plinary Optimization, 2020, 61(2): 711–729. [131] Lim, D. K., Woo, D. K., Yeo, H. K., Jung, S. Y., Ro, based design optimization using support vector machine J. S., Jung, H. K. A novel surrogate-assisted multi- and Kriging model. Structural and Multidisciplinary objective optimization algorithm for an electromag- Optimization, 2017, 55(6): 2285–2304. [143] Tang, Y. F., Long, T., Shi, R. H., Wu, Y. F., Wang, G. netic machine design. IEEE Transactions on Magnetics, 2015, 51(3): 1–4. G. Sequential radial basis function-based optimization [132] Datta, R., Regis, R. G. A surrogate-assisted evolution method using virtual sample generation. Journal of strategy for constrained multi-objective optimization. Mechanical Design, 2020, 142(11): 111701. Expert Systems With Applications, 2016, 57: 270–284. [144] Basudhar, A., Dribusch, C., Lacaze, S., Missoum, S. [133] Dıaz-Manr ´ ıquez, ´ A., Toscano, G., Barron-Zambrano, Constrained efficient global optimization with support J. H., Tello-Leal, E. A review of surrogate assisted vector machines. Structural and Multidisciplinary Op- multiobjective evolutionary algorithms. Computational timization, 2012, 46(2): 201–221. [145] Lim, D., Ong, Y. S., Gupta, A., Goh, C. K., Dutta, P. Intelligence and Neuroscience, 2016, 2016: 1–14. [134] Venturelli, G., Benini, E., Laniewski-Wo l lk, L. A S. Towards a new Praxis in optinformatics targeting Kriging-assisted multiobjective evolutionary algorithm. knowledge re-use in evolutionary computation: Simulta- Applied Soft Computing, 2017, 58: 155–175. neous problem learning and optimization. Evolutionary [135] Wang, C. Z., Ding, J. L., Cheng, R., Liu, C. X., Chai, Intelligence, 2016, 9(4): 203–220. T. Y. Data-driven surrogate-assisted multi-objective [146] Park, J., Kim, K. Y. Instance variant nearest neighbor optimization of complex beneficiation operational pro- using particle swarm optimization for function approxi- cess. IFAC-PapersOnLine, 2017, 50(1): 14982–14987. mation. Applied Soft Computing, 2016, 40: 331–341. Metamodel-based multidisciplinary design optimization methods for aerospace system 213 [147] Isaacs, A., Ray, T., Smith, W. An evolutionary algo- [158] Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G. rithm with spatially distributed surrogates for multi- E. Learning nonlinear operators via DeepONet based objective optimization. In: Progress in Artificial Life . on the universal approximation theorem of operators. ACAL 2007. Lecture Notes in Computer Science, Vol. Nature Machine Intelligence, 2021, 3(3): 218–229. 4828. Randall, M., Abbass, H. A., Wiles, J., Eds. [159] Kashefi, A., Rempe, D., Guibas, L. J. A point-cloud Springer, Berlin, Heidelberg, 2007: 257–268 deep learning framework for prediction of fluid flow fields on irregular geometries. Physics of Fluids, 2021, [148] Castillo, E., Mınguez, ´ R., Castillo, C. Sensitivity anal- 33(2): 027104. ysis in optimization and reliability problems. Reliability Engineering & System Safety, 2008, 93(12): 1788–1800. [160] Kondor, R., Son, H. T., Pan, H., Anderson, B., Trivedi, S. Covariant compositional networks for learn- [149] Chen, G., Zhang, K., Xue, X., Zhang, L., Yao, J., Sun, ing graphs. 2018: arXiv:180102144. Available at https: H., Fan, L., Yang, Y. Surrogate-assisted evolution- //arxiv.org/abs/2009.11193. ary algorithm with dimensionality reduction method [161] Darbon, J., Meng, T. On some neural network architec- for water flooding production optimization. Journal of tures that can represent viscosity solutions of certain Petroleum Science and Engineering, 2020, 185: 106633. high dimensional Hamilton–Jacobi partial differential [150] Liu, B., Zhang, Q., Gielen, G. G. A Gaussian process equations. Journal of Computational Physics, 2021, surrogate model assisted evolutionary algorithm for 425: 109907. medium scale expensive optimization problems. IEEE [162] Chen, W., Chiu, K., Fuge, M. D. Airfoil design parame- Transactions on Evolutionary Computation, 2013, terization and optimization using bezier ´ generative ad- 18(2): 180–192. versarial networks. AIAA Journal, 2020, 58(11): 4723– [151] Gupta, A., Ong, Y., Feng, L. Insights on transfer opti- mization: Because experience is the best teacher. IEEE [163] Pfrommer, S., Halm, M., Posa, M. Contactnets: Learn- Transactions on Emerging Topics in Computational ing of discontinuous contact dynamics with smooth, Intelligence, 2018, 2(1): 51–64. implicit representations. 2020: arXiv:200911193. Avail- [152] Gupta, A., Ong, Y., Feng, L., Tan, K. C. Multiobjective able at https://arxiv.org/abs/2009.11193. multifactorial optimization in evolutionary multitask- [164] Wang, G. G., Simpson, T. Fuzzy clustering based hier- ing. IEEE Transactions on Cybernetics, 2017, 47(7): archical metamodeling for design space reduction and 1652–1665. optimization. Engineering Optimization, 2004, 36(3): [153] Ma, X., Chen, Q., Yu, Y., Sun, Y., Ma, L., Zhu, Z. 313–335. A two-level transfer learning algorithm for evolutiona- [165] Fletcher, R., Leyffer, S. Nonlinear programming with- ry multitasking. Frontiers in Neuroscience, 2020, 13: out a penalty function. Mathematical Programming, 2002, 91(2): 239–269. [154] Cheng, M.-Y., Gupta, A., Ong, Y.-S., Ni, Z.-W. Co- [166] Fletcher, R., Leyffer, S., Toint, P. L. On the global evolutionary multitasking for concurrent global opti- convergence of a filter-SQP algorithm. SIAM Journal mization: With case studies in complex engineering on Optimization, 2002, 13(1): 44–59. design. Engineering Applications of Artificial Intelli- [167] Audet, C., Dennis, J. E. A pattern search filter method gence, 2017, 64: 13–24. for nonlinear programming without derivatives. SIAM [155] Min, A. T. W., Ong, Y., Gupta, A., Goh, C. Multi- Journal on Optimization, 2004, 14(4): 980–1010. problem surrogates: Transfer evolutionary multiobjec- [168] Gonzaga, C. C., Karas, E., Vanti, M. A globally conver- tive optimization of computationally expensive prob- gent filter method for nonlinear programming. SIAM lems. IEEE Transactions on Evolutionary Computation, Journal on Optimization, 2004, 14(3): 646–669. 2019, 23(1): 15–28. [169] Regis, R. G. Constrained optimization by radial basis [156] Raissi, M., Perdikaris, P., Karniadakis, G. E. Physics- function interpolation for high-dimensional expensive informed neural networks: A deep learning framework black-box problems with infeasible initial points. Engi- for solving forward and inverse problems involving non- neering Optimization, 2014, 46(2): 218–243. linear partial differential equations. Journal of Compu- [170] Rashid, K., Ambani, S., Cetinkaya, E. An adaptive mul- tational Physics, 2019, 378: 686–707. tiquadric radial basis function method for expensive [157] Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, black-box mixed-integer nonlinear constrained opti- P., Wang, S., Yang, L. Physics-informed machine learn- mization. Engineering Optimization, 2013, 45(2): 185– ing. Nature Reviews Physics, 2021, 3(6): 422–440. 206. 214 R. Shi, T. Long, N. Ye, et al. [171] Qian, J. C., Yi, J. X., Cheng, Y. S., Liu, J., Zhou, Q. [184] Peng, L., Liu, L., Long, T., Yang, W. An efficient truss A sequential constraints updating approach for Krig- structure optimization framework based on CAD/CAE ing surrogate model-assisted engineering optimization integration and sequential radial basis function meta- design problem. Engineering With Computers, 2020, model. Structural and Multidisciplinary Optimization, 36(3): 993–1009. 2014, 50(2): 329–346. [172] Sasena, M. J., Papalambros, P., Goovaerts, P. Ex- [185] Gogu, C., Haftka, R. T., Bapanapalli, S. K., Sankar, ploration of metamodeling sampling criteria for con- B. V. Dimensionality reduction approach for response strained global optimization. Engineering Optimization, surface approximations: Application to thermal design. 2002, 34(3): 263–278. AIAA Journal, 2009, 47(7): 1700–1708. [173] Kreisselmeier, G., Steinhauser, R. Systematic control [186] Simpson, T. W., Mauery, T. M., Korte, J. J., Mis- design by optimizing a vector performance index. IFAC tree, F. Kriging models for global approximation in Proceedings Volumes, 1979, 12(7): 113–117. simulation-based multidisciplinary design optimization. [174] Poon, N. M. K., Martins, J. R. R. A. An adaptive AIAA Journal, 2001, 39(12): 2233–2241. approach to constraint aggregation using adjoint sensi- tivity analysis. Structural and Multidisciplinary Opti- mization, 2007, 34(1): 61–73. [175] Gao, H., Zhu, J., Zhang, W., Zhou, Y. An improved Renhe Shi received his B.S. degree adaptive constraint aggregation for integrated layout in flight vehicle design engineering and and topology optimization. Computer Methods in Ap- Ph.D. degree in aeronautical and astro- plied Mechanics and Engineering, 2015, 289: 387–408. nautical science and technology from Bei- [176] Zhang, K. S., Han, Z. H., Gao, Z. J., Wang, Y. Con- jing Institute of Technology, China, in straint aggregation for large number of constraints in 2013 and 2019, respectively. He is a wing surrogate-based optimization. Structural and Mul- postdoctoral researcher in the School of tidisciplinary Optimization, 2019, 59(2): 421–438. Aerospace Engineering, Tsinghua Uni- [177] Kennedy, G. J. Strategies for adaptive optimization versity, from 2019 to 2020. Currently, he is an asso- with aggregation constraints using interior-point meth- ciate professor in the School of Aerospace Engineering, Bei- ods. Computers & Structures, 2015, 153: 217–229. jing Institute of Technology. His research interests include [178] Shi, R., Liu, L., Long, T., Liu, J. An efficient ensemble metamodel-based design optimization, multidisciplinary de- of radial basis functions method based on quadratic sign optimization, and aerospace system engineering. E-mail: programming. Engineering Optimization, 2016, 48(7): shirenhe@bit.edu.cn. 1202–1225. [179] Xiong, Y., Guo, L., Tian, D., Zhang, Y., Liu, C. Intelli- gent optimization strategy based on statistical machine learning for spacecraft thermal design. IEEE Access, 2020, 8: 204268–204282. Teng Long received his B.S. degree in [180] Smith, B. G. A., Capon, C. J., Brown, M., Boyce, R. R. flight vehicle propulsion engineering and Ph.D. degree in aeronautical and astro- Ionospheric drag for accelerated deorbit from upper low earth orbit. Acta Astronautica, 2020, 176: 520–530. nautical science and technology from Bei- [181] Wu, Z., Huang, Y., Chen. X., Zhang, X., Yao, W. jing Institute of Technology, China, in Surrogate modeling for liquid-gas interface determina- 2004 and 2009, respectively. He is a pro- fessor and executive dean at the School of tion under microgravity. Acta Astronautica, 2018, 152: 71–77. Aerospace Engineering, Beijing Institute [182] Peng, H., Wang. W. Adaptive surrogate model-based of Technology. His research interests include multidisciplinary fast path planning for spacecraft formation reconfigu- design optimization theory and its applications to flight vehi- cle conceptual design and multi-aircraft collaborative mission ration on libration point orbits. Aerospace Science and Technology, 2016, 54: 151–163. planning and decision-making. Professor Long is a member [183] Feldhacker, J. D., Jones, B. A., Doostan, A., Hampton, of the American Institute of Aeronautics and Astronautics J. Reduced cost mission design using surrogate models. (AIAA) and the American Society of Mechanical Engineers (ASME). E-mail: tenglong@bit.edu.cn. Advances in Space Research, 2016, 57(2): 588–603. Metamodel-based multidisciplinary design optimization methods for aerospace system 215 Nianhui Ye received his B.S. degree in Zhenyu Liu received his B.S. degree in flight vehicle design and engineering from flight vehicle design and engineering from Beijing Institute of Technology, China, in Beijing Institute of Technology, China, in 2019. He is currently pursuing his Ph.D. 2020. He is currently pursuing his M.S. degree at the School of Aerospace and En- degree at the School of Aerospace and gineering, Beijing Institute of Technology. Engineering, Beijing Institute of Tech- His research interests include surrogate- nology, China. His research interests in- assisted evolutionary computation and clude surrogate-assisted multi-objective multidisciplinary modeling of cross-media flight vehicles. optimization methods and flight vehicle multidisciplinary de- E-mail: ynh1996sh@163.com. sign optimization. E-mail: liuzykm@163.com. Yufei Wu received his B.S. degree in Open Access This article is licensed under a Creative flight vehicle design engineering from Bei- Commons Attribution 4.0 International License, which jing Institute of Technology, China, in permits use, sharing, adaptation, distribution and re- 2017. He is currently pursuing his Ph.D. production in any medium or format, as long as you degree in aeronautical and astronautical give appropriate credit to the original author(s) and the science and technology at Beijing Institute source, provide a link to the Creative Commons licence, of Technology. His research interests in- and indicate if changes were made. clude metamodel-based design optimiza- tion theory and its applications to cross-domain morphing The images or other third party material in this article flight vehicle design. E-mail: wuyufei@bit.edu.cn. are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the mate- Zhao Wei received his B.S. degree in rial. If material is not included in the article’s Creative mechanical engineering and automation Commons licence and your intended use is not permitted from Zhengzhou University, China, in by statutory regulation or exceeds the permitted use, you 2016. He is currently pursuing his Ph.D. will need to obtain permission directly from the copyright degree in aeronautical and astronauti- holder. cal science and technology at Beijing In- To view a copy of this licence, visit http://creative- stitute of Technology. His research in- terests include flight vehicle multidisci- commons.org/licenses/by/4.0/. plinary design optimization and mission planning. E-mail: weizhao0123@163.com. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Astrodynamics Springer Journals

Metamodel-based multidisciplinary design optimization methods for aerospace system

Loading next page...
 
/lp/springer-journals/metamodel-based-multidisciplinary-design-optimization-methods-for-0f3x2EqbYo
Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2021
ISSN
2522-008X
eISSN
2522-0098
DOI
10.1007/s42064-021-0109-x
Publisher site
See Article on Publisher Site

Abstract

Astrodynamics Vol. 5, No. 3, 185–215, 2021 https://doi.org/10.1007/s42064-021-0109-x Metamodel-based multidisciplinary design optimization methods for aerospace system 1,2 1,2 1,2 1,2 1,2 1,2 Renhe Shi , Teng Long (B), Nianhui Ye , Yufei Wu , Zhao Wei , and Zhenyu Liu 1. School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China 2. Key Laboratory of Dynamics and Control of Flight Vehicle, Ministry of Education, Beijing 100081, China ABSTRACT KEYWORDS The design of complex aerospace systems is a multidisciplinary design optimization (MDO) aerospace systems design problem involving the interaction of multiple disciplines. However, because of the necessity multidisciplinary design of evaluating expensive black-box simulations, the enormous computational cost of solving optimization (MDO) MDO problems in aerospace systems has also become a problem in practice. To resolve metamodel-based design and this, metamodel-based design optimization techniques have been applied to MDO. With optimization (MBDO) these methods, system models can be rapidly predicted using approximate metamodels to expensive black-box problems improve the optimization efficiency. This paper presents an overall survey of metamodel- based MDO for aerospace systems. From the perspective of aerospace system design, this paper introduces the fundamental methodology and technology of metamodel- based MDO, including aerospace system MDO problem formulation, metamodeling techniques, state-of-the-art metamodel-based multidisciplinary optimization strategies, and expensive black-box constraint-handling mechanisms. Moreover, various aerospace system examples are presented to illustrate the application of metamodel-based MDOs Review Article Received: 14 July 2021 to practical engineering. The conclusions derived from this work are summarized in the final section of the paper. The survey results are expected to serve as guide and reference Accepted: 23 July 2021 for designers involved in metamodel-based MDO in the field of aerospace engineering. © The Author(s) 2021 1 Introduction design of systems [3]. Moreover, MDO was first applied to the coupled aero-structural optimization problem for In practice, the development of aerospace systems in- aircraft wings [4]. The merits of MDO have been widely volves sophisticated system engineering. Aerospace sys- proven in the design practices of engineering systems, tems (e.g., satellites and spacecraft) are multidisciplinary such as aircraft [5–10], automobiles [11–13], ships [14,15], schemes consisting of several inter-coupled disciplines or and electric devices [16, 17]. In the past decade, MDO subsystems, including orbit, power, propulsion, structure, has also been applied to aerospace systems, including thermal control, attitude control, and payload. Multi- satellites [18–21], constellations [20, 22,23], and launch disciplinary design optimization (MDO) has been widely rockets [24–27], to improve the overall system perfor- employed in engineering systems design practices to im- mance. prove design performance and reduce design costs in the Although deriving the optimal design of multidisci- early development process [1]. The concept of MDO was plinary systems is advantageous, the implementation of initially developed by Sobieszczanski-Sobieski [2]. As a MDO is confronted with two crucial problems in practi- methodology, MDO focuses on the design of complex engi- cal engineering. The basic problem involves the means neering systems and subsystems by coherently exploiting by which multiple disciplines are to be coupled. The the synergy of mutually interacting phenomena [2]. It simplest approach is to implement multidisciplinary ana- comprehensively utilizes various computational analysis tools and optimization methods to determine the optimal lysis (MDA) during optimization, where the coupling B tenglong@bit.edu.cn 186 R. Shi, T. Long, N. Ye, et al. variables are directly solved via iterative analysis. In the MBDO and MDO methodologies can be referred to addition, several MDO architectures have been success- as metamodel-based MDO. Note that in practice, the fully developed to organize different analysis models and purpose of metamodel-based MDO is to overcome the computational flow for multidisciplinary optimization. burden of optimization rather than build or solve MDO These architectures can be generally classified as mono- architectures. lithic (e.g., all-at-once, individual discipline feasible, and A number of reviews regarding MDO have been re- multidisciplinary feasible) architectures and distributed ported in the literature [1, 3, 31, 32]. Many surveys on (e.g., cooperative optimization and concurrent subspace metamodel-based design and optimization have also been optimization) architectures [3], with respect to whether conducted in recent years [28, 29,33]. However, compre- sub-optimization is required. Another critical problem hensive surveys on metamodel-based MDOs are rarely involves achieving efficiency in solving MDO problems performed [34,35]. Focusing on the design of aerospace with limited computational resource. Modern aerospace systems, this paper introduces the fundamental methodo- system designs typically employ high-fidelity disciplinary logy and technology of state-of-the-art metamodel-based simulation models (e.g., finite element analysis (FEA) MDO techniques. The remainder of this paper is or- model with large grids) to enhance the quality and re- ganized as follows. Section 2 introduces the problem liability of design optimization results. However, the formulation and overall architecture of the metamodel- evaluation of expensive simulation models and iterative based MDO. The common metamodeling techniques are MDA computing processes further considerably increase described in Section 3. In Section 4, the metamodel- the associated computational cost of MDO problems in based multidisciplinary optimization strategies are dis- aerospace systems. Moreover, because of the lack of cussed in detail. The survey of expensive black-box transparency in simulation models (e.g., simulations im- constraint-handling mechanisms for metamodel-based plemented using commercial computer-aided engineering MDOs is presented in Section 5. Moreover, several real- (CAE) software), the MDA process is generally referred world aerospace system MDO examples are discussed in to as an expensive black-box function, whose gradient is Section 6. Finally, some conclusions and future works on expensive or unreliable for calculation. Thus, most exist- metamodel-based MDOs are summarized in Section 7. ing heuristic or gradient-based numerical methods (e.g., genetic algorithm and sequential quadratic programming) 2 Methodology of metamodel-based are unsuitable for solving computation-intensive black- MDO box optimization problems. 2.1 Formulation of aerospace system MDO To reduce the computational cost of expensive black- problem box optimization problems, metamodel-based design op- timization (MBDO) techniques have been developed The development of an aerospace system is a complex over the past two decades [28]. These methods are also system engineering task. For instance, a satellite system called surrogate-assisted analysis and optimization me- consists of multiple inter-coupled disciplines, including or- thods [29]. They involve the construction of a metamodel bit, propulsion, attitude control, payload, and structure, (or a surrogate) based on a set of samples to approximate as shown in Fig. 1. To improve system performance, the original expensive simulation models for analysis or optimization. These MBDO methods can also be app- lied to solve MDO problems. In this case, an MDO problem is essentially treated as a general constrained nonlinear optimization problem, where the computation- ally intensive MDA process is replaced with metamodels to reduce the number of expensive function calls. In recent years, several state-of-the-art MBDO methods have been developed [30]. Different MBDO methods are distinguished by their metamodeling techniques and de- sign space exploration strategies. The combination of Fig. 1 Various disciplines for aerospace systems. Metamodel-based multidisciplinary design optimization methods for aerospace system 187 MDO was employed to determine the optimal design (i.e., state variables) of the ith discipline, respectively; X and X are the lower and upper bounds of de- parameters for the aerospace system. LB UB sign variable, respectively; Y is the coupling state vari- In MDO, the inter-coupled relationships, data ex- ij ables from the ith discipline to the jth discipline; and change, and simulation ofl w among different disciplines D (X ,Y ,Y ) and g (X ,Y ,Y ) are the state equa- can be organized in terms of the design structure matrix, i i i ij i i i ij tions (i.e., disciplinary analysis model) and constraints as illustrated in Fig. 2. In the matrix, the diagonal ele- of the ith discipline, respectively. ments (i.e., shaded blocks) represent different disciplines Note that because of feedback coupling state vari- involved in the aerospace system. The terms above the ables, Y = {Y |i > j}, the MDA process must derive diagonal represent feed-forward variables and parame- FV ij a consistent solution at each sample point during the ters, whereas those below represent backward informa- metamodel-based optimization process. This means that tion. The design structure matrix explicitly expresses the solution (D (X ,Y ,Y ), i = 1, 2,··· , N ) must sa- the constitution of the MDO problem. Based on the con- i i i ij d tisfy all state equations. In practice, the MDA process can structed matrix, the general mathematical formulation of be organized via a fixed-point-based iteration approach, an aerospace system MDO problem can be represented as as summarized in Algorithm 1. min f(X),X = [X ,X ,··· ,X ] 1 2 N D (X ,Y ,Y ) = 0, i = 1, 2,··· , N i i i ij d 2.2 Overall procedure of metamodel-based s.t. g (X ,Y ,Y ) ⩽ 0, i = 1, 2,··· , N (1) optimization i i i ij d X ⩽ X ⩽ X LB UB Because modern aerospace system design generally in- where f(X) is the objective of the aerospace system volves computationally expensive simulation models, MDO problem (e.g., total mass and lifecycle cost); N MBDO techniques are employed to solve MDO problems is the number of disciplines; X is the vector of design to reduce the computational cost. In the metamodel- variables; X and Y are the design variables and output based MDO process, the entire MDA process is evaluated i i Fig. 2 Design structure matrix of aerospace system MDO problem. 188 R. Shi, T. Long, N. Ye, et al. to obtain a consistent design at each sample point. The Algorithm 1 Quasi-codes for fixed-point-based MDA process metamodels are constructed to approximate the MDA Input: Initial value of feedback coupling state process for multidisciplinary optimization. In this case, (0) variables (Y ); design variables (X); the responses of expensive black-box objectives and con- FV MDA convergence tolerance (ε ) MDA straints can be rapidly and inexpensively predicted by Output: Objective value (Y ); number of MDA metamodels. Note that metamodels can be gradually re- iteration (k ) MDA fined via adaptive sampling during the optimization pro- 1 Begin cess to further reduce the computational cost. Moreover, 2 exitflag ← 0 Fig. 3 illustrates the overall procedure of metamodel- 3 k ← 0 MDA 4 while exitflag == 0 do based aerospace system MDO [19]. The major steps are (k +1) (k ) MDA MDA 5 (Y ,Y ) ← MDO(X,Y ) as follows: FV FV 6 exitflag ← 1 Step 1: The aerospace system MDO problem, includ- (k +1) MDA (k +1) MDA 7 for each Y in Y FV ing the design space, objective, constraints, and MDA 8 if models to be optimized, is clearly defined. The algorithm (k +1) (k ) (k ) MDA MDA MDA |Y − Y |/|Y | ⩾ ε MDA parameters (e.g., the number of initial samples and ter- then mination criterion) of the selected MBDO methods for 9 exitflag ← 0 solving the MDO problem are configured. 10 end Step 2: A number of sample points are generated 11 end 12 k ← k + 1 MDA MDA by the design of the experiment in the design space. 13 end Then, the MDA process is invoked to obtain the objective 14 return Y , k MDA and constraint responses at each sample point. The 15 End sample points and their associated responses (referred to Fig. 3 Overall procedure of metamodel-based MDO for aerospace systems (reproduced with permission from Ref. [19], © IAA 2017). Metamodel-based multidisciplinary design optimization methods for aerospace system 189 as samples) are stored in the sample dataset. 2.3 Architecture of metamodel-based MDO Step 3: Based on existing samples in the sample techniques dataset, metamodels are constructed to approximate 2.3.1 Classification of metamodel-based MDO the objective and constraint responses from the costly methods MDA process. If the static metamodel-based optimiza- Metamodel-based MDO methods may be generally clas- tion strategy is used, then the approximation accuracy of sified into two categories in terms of whether the metamodels must be verified, as discussed in Section 2.3. metamodels are updated during optimization: static Step 4: The constructed metamodels are used to re- and dynamic metamodel-based optimization strategies place the original expensive MDA process for optimiza- (Fig. 4) [36]. tion. Global numerical optimization techniques, such In the static metamodel-based optimization strategy, as genetic algorithms, are employed to directly optimize metamodels are constructed once based on sufficient sam- the metamodels instead of the original expensive MDA ples for optimization. This is the most convenient ap- models, as given by Eq. (2): proach to implement the metamodel-based MDO pro- min f(X),X = [X ,X ,··· ,X ] 1 2 n cess. For instance, some expensive disciplinary models can be replaced with well-constructed metamodels for gˆ (X) ⩽ 0, i = 1, 2,··· , m s.t. (2) MDA and optimization. Note that to ensure the con- X ⩽ X ⩽ X LB UB fidence in their results, the approximation accuracy of where m is the number of constraints, and f(X) and the constructed metamodels should be verified before gˆ (X) are the metamodels of the objective and the ith optimization. Techniques that are widely used for ac- constraint, respectively. If the dynamic metamodel-based curacy validation include split and crossover validation optimization strategy is used, the metamodels are adap- methods. In the split validation method, the samples tively refined during optimization to efficiently search for are divided into two groups. One group of samples is the optimum, as discussed in detail in Section 4. applied to construct the metamodels, and the other is Step 5: The termination criterion is checked to deter- used to validate their accuracy. For the crossover valida- mine whether the optimization must be terminated. The tion method, several samples are randomly selected for common termination criteria in metamodel-based MDO validation, whereas the remaining samples are used for include the decrement criterion (C ) and computational metamodeling. The validation process is repeated until cost criterion (C ) [36], as given in Eq. (3). In C , if the 2 1 the prediction errors in all the samples are obtained. The error or relative error between the objective values in two most commonly used approximation accuracy indices for consecutive iterations is less than the tolerance (ε ), OPT metamodels are summarized in Table 1 [36]. For example, then the optimization terminates. In C , if the number if the complex correlation coefficient ( R ) exceeds 0.9, of existing samples (N ) exceeds the predefined maxi- then the metamodel can be assumed to be sufficiently mum number of function evaluations (NFE ), then the max accurate for optimization. optimization terminates. In contrast to constructing a static metamodel once, (k) (k−1) f(X ) − f(X ) the metamodels in the dynamic metamodel-based op- C : ⩽ ε ∥ 1 OPT (k) f(X ) timization strategy are adaptively refined according to (k) (k−1) certain criteria or mechanisms during optimization. Over- |f(X ) − f(X )| ⩽ ε (3) OPT all, the dynamic metamodel-based optimization strategy C : N > NFE 2 e max can be further divided into infill sampling criterion-based methods and space reduction-based methods. Compared Step 6: The optimized design for the aerospace system with the static metamodel-based optimization strategy, MDO problem is finally obtained when the termination the dynamic metamodel-based optimization strategy, criterion is reached. For the single-objective optimization which has become popular in recent years, is generally problem, the best feasible solution in the sample dataset more efficient in practice. This study mainly focuses on is outputted as the final solution. For the multi-objective optimization problem, the non-dominated designs in the surveying the dynamic metamodel-based optimization sample dataset are outputted as a Pareto solution. strategy, as discussed in detail in Section 4. 190 R. Shi, T. Long, N. Ye, et al. Fig. 4 Classification of MBDO methods. Table 1 Approximation accuracy indices for metamodel validation Index Symbol Formulation N N test test X X (test) (test) (test) 2 2 2 (test) 2 Complex correlation coefficient R R = 1 − (y − yˆ ) / (y − y¯ ) i i i i=1 i=1 test u X (test) (test) Root mean square error RMSE RMSE = (y − yˆ ) i i test i=1 N (test) (test) N test test |y − yˆ | 1 (test) i=1 i i t (test) 2 Relative average absolute error RAAE RAAE = ; STD = (y − y¯ ) N · STD N test test (test) (test) (test) (test) (test) (test) max(|y − y¯ |,|y − y¯ |,··· ,|y − y¯ |) 1 2 N test Relative maximum absolute error RMAE RMAE = STD (test) (test) Note: N is the number of test samples; y is the real response of the ith test sample; yˆ is the predicted response of the ith test i i (test) test sample; y¯ is the mean response of the test samples. Metamodel-based multidisciplinary design optimization methods for aerospace system 191 2.3.2 Diagram of metamodel-based MDO archi- two typical optimal Latin hypercube design methods, tecture that is, the native lhsdesign function in the MATLAB Based on the foregoing discussion, Fig. 5 illustrates the optimization toolbox [40] and ESEA-OLHD [41], are il- architecture of the metamodel-based MDO technique [36]. lustrated in Fig. 6. The lhsdesign function is implemented Before optimization, the MDA models of aerospace sys- by randomly generating several groups of sample points tems (e.g., orbit, structure, and attitude control) must and selecting the group with the best space-filling abil- be established and parameterized. Then, MBDO tech- ity. Moreover, ESEA-OLHD generates sample points by niques are employed to optimize the MDO problem by optimizing the space-filling criteria via numerical opti- evaluating the MDA process. During the optimization, mization algorithms. This shows that the random sample metamodels are constructed and refined to explore the points generated by the optimal Latin hypercube design design space. The optimization stops when the termi- methods can uniformly fill the design space, thus improv- nation criterion is satisfied. The optimized solution is ing the metamodeling performance in practice. In recent outputted for further analysis. years, some novel OLHD methods specific for metamodel- based optimization have been developed [42–44]. After the generation of sample points, the associated 3 Metamodeling techniques responses at each sample point are evaluated by calling 3.1 Design of experiments the MDA process. The samples are then utilized to construct metamodels for MDO. The design of experiments aims at generating sample points in the design space to represent the numerical 3.2 Typical metamodels characteristics of the system. Considering the limited 3.2.1 Polynomial response surface method computational resource in practice, the design of exper- The polynomial response surface method constructs a imental methods is expected to provide favorable pro- multivariate linear regression function to fit the costly jective uniformity and space-filling uniformity perfor- simulation model or MDA process [45]. A polynomial mance. In the metamodel-based aerospace system MDO response surface metamodel can be written as process, Latin hypercube is the most commonly used design among experimental methods. It can generate N (0) (i) (i) f (x) = β + β x sample sets with arbitrary quantities and dimensions. PRSM i=1 To further improve the sampling performance, in recent N N N v v v X XX years, optimal Latin hypercube design methods have been (ii) (i) 2 (ij) (i) (j) + β (x ) + β x x developed based on certain criteria, such as minimum i=1 i=1 j>1 distance [37], entropy [38], and energy [39]. For instance, (4) Fig. 5 Diagram of metamodel-based MDO architecture. 192 R. Shi, T. Long, N. Ye, et al. x1 x2 (a) lhsdesign (b) ESEA-OLHD Fig. 6 Various optimal Latin hypercube design methods. Table 2 Commonly used radial functions where N is the dimensionality of design variables; (0) (i) (ij) β , β , β are the coefficients estimated through the Function Formulation least squares method. The coefficient matrix, β , is given Linear ϕ(r ) = (r + c) Gauss ϕ(r ) = exp(−cr ) by 2 2 Spline ϕ(r ) = r log(cr ) T −1 T β = (Φ Φ) Φ y (5) Cubic ϕ(r ) = (r + c) 2 2 1/2 Multiquadratic ϕ(r ) = (r + c ) 2 2 −1/2 where y is the response vector of training sample points, Inverse multiquadratic ϕ(r ) = (r + c ) and Φ is the matrix relevant to the training sample points. 3.2.2 Radial basis functions distribution of sample points and function information. Radial basis function is an interpolation method based Typically, c can be estimated using Eq. (8) [36]: on the function value at sample points [46]. A radial c = ((max(x) − min(x))/N ) (8) basis function metamodel can be formulated as N 3.2.3 Kriging f (x) = ω ϕ (∥x − x ∥) (6) RBF i r i The Kriging model is a type of unbiased optimal estima- i=1 tion interpolation model that combines a global approxi- where N is the number of training sample points; mation model and a stochastic process. It is formulated ϕ (∥x − x ∥), i = 1, 2,··· , N , is the radial function; and r i t using Eq. (9): ω is the weight coefficient of radial function. The coeffi- f (x) = µ(x) + Z(x) (9) KRG cient vector (ω) can be calculated as follows: −1 where µ (x) is the global approximation model, which ω = A y reflects the variation trend of the expensive MDA pro- y = [y , y ,··· , y ] (7) 1 2 N cess and is usually set as a constant; Z(x) represents   ϕ(∥x − x ∥) ··· ϕ(∥x − x ∥) 1 1 1 N 2 a Gaussian process with zero mean and variance (σ ).   . . . . . A =   . . Given a set of sample points, X = {x ,x ,··· ,x }, 1 2 N ϕ(∥x − x ∥) ··· ϕ(∥x − x ∥) N 1 N N t t t the covariance matrix is given by Eq. (10): N ×N t t The typically used radial functions are listed in Cov(Z(X)) = σ R(R(x ,x )), i, j = 1, 2,··· , N (10) i j t Table 2 [36], where is the Euclidean distance is ∥x− x ∥. where R(· ) is a symmetric correlation matrix, and The approximate accuracy of this metamodel is influ- enced by shape coefficient c, which is determined by the R(x x ) is the Gaussian correlation function between i j y1 y2 Metamodel-based multidisciplinary design optimization methods for aerospace system 193 sample points x and x , as shown in Eq. (11): applied to metamodeling in practice. The representative i j method is support vector regression [50,51]. Support vec- (k) (k) tor regression is based on the principle of support vector R(x ,x ) = exp − θ |x − x | (11) i j k i j k=1 machines with slight variations and has proven to be effec- tive in solving regression problems [51]. Artificial neural where θ is the correlation parameter determined by networks are also common in metamodeling fields [52]. An maximizing the likelihood function in Eq. (12): artificial neural network is a multilayer feedforward net- N 1 L(x) = − ln(σˆ ) − ln(|R|) (12) work that approximates the targeted nonlinear function; 2 2 the network parameters are trained via numerical meth- The estimated variance (σˆ ) and mean values (µˆ) can ods (e.g., back-propagation algorithm). As a hierarchical be calculated using Eq. (13): composition of Gaussian process-based metamodels (e.g., T −1 1 R Y Kriging), deep Gaussian processes have attracted consid- µˆ = T −1 1 R 1 erable interest in recent years owing to their promising (13) T −1 (Y − 1µ)ˆ R (Y − 1µ)ˆ T T nonlinear approximation performance [53–55]. σˆ = where Y is the column vector of the responses of sample 3.3 Multi-model fusion methods points; 1 is a 1 × N unit vector. The prediction value Most conventional metamodeling techniques and and standard deviation of the unvisited point using the metamodel-based optimization processes simply utilize Kriging model are formulated in Eq. (14): high-fidelity analysis models that are accurate but com- T −1 T f (x) = µˆ + r R (Y − 1 µ)ˆ KRG T putationally intensive. Thus, the computational burden T −1 2 (14) (1 − 1 R r) 2 T −1 for solving MDO problems remains heavy because of the sˆ (x) = σˆ 1 − r R r + KRG T −1 1 R 1 excessive number of costly samples. However, real-world 3.2.4 Ensemble method aerospace system designs generally involve multi-fidelity In general, different metamodels exhibit different approxi- analysis models, such as structural FEA models with fine mation performance levels for different problems. To or coarse grids and orbital transfer models considering exploit various metamodels, different metamodels can perturbations and eclipses. Low-fidelity analysis models be combined to formulate an ensemble metamodel to are less accurate but computationally inexpensive. Multi- improve the approximation capability [47], as shown in model fusion methods (or multi-fidelity methods) have Eq. (15): been proposed to exploit multi-fidelity analysis mod- els [56–60]. In this approach, a large number of low- esm ˆ ˆ f = α f (x) (15) fidelity points are generated to capture the trends of Ensemble i i i=1 system responses, and a small number of high-fidelity ˆ points are used to calibrate the trend. The basic form where f is the ensemble of different metamodels, Ensemble ˆ of a multi-model fusion metamodel is given by Eq. (16). f (x) is the ith metamodel, N is the number of dif- i esm In terms of the method for determining the regression ferent metamodels, and α is the weight of the corre- (ρ ) and discrepancy (δ (x)) items, multi-model fusion me- sponding metamodel. In practice, the weights (α ) can thods can be divided into correction-based and Bayesian be determined with respect to the approximation error methods. of each metamodel [47,48]. If the computational cost is acceptable, the weights can be directly optimized using ˆ ˆ f (x) = ρ f (x) + δ (x) (16) HF LF numerical methods by minimizing the cross-validation 3.3.1 Correction-based method error of the ensemble metamodel [43, 49]. The correction-based method is a straightforward multi- 3.2.5 Machine-learning-based metamodeling me- thod model fusion method, where the metamodel of the low- ˆ ˆ fidelity model ( f (x)) and that of the discrepancy (δ (x)) Because the essence of metamodels in optimization is re- LF latively similar to the regression task in machine learning, are constructed. The regression item (ρ ) is incorporated some machine learning regression algorithms have been to minimize the difference between the scaled low-fidelity 194 R. Shi, T. Long, N. Ye, et al. model approximation (ρ f (x)) and high-fidelity model LF response (f (x)). Traditionally, ρ is a scalar, but mod- HF eling it as a function of x is an area of research [61,62]. Based on Ref. [59], radial basis function and Kriging are the most widely used metamodels in the correction-based multi-model fusion method. 3.3.2 Bayesian method The Bayesian multi-model fusion method, also known as the Co-Kriging method, was introduced by Kennedy and O’Hagan [63]. Co-Kriging constructs the covariance between high-fidelity and low-fidelity models to import the assistance of the low-fidelity model; in this method, Fig. 7 1D numerical example for Co-Kriging. ρ and the hyperparameters of δ (x) are both estimated. Forrester et al. applied Co-Kriging to the design opti- the design space according to a certain infill criterion. In mization field by combining it with the Bayesian model this study, two representative infill sampling criterion- update criterion to balance global exploration and local based optimization strategies are introduced: efficient exploitation [56]. In recent decades, Co-Kriging variants global optimization methods and mode-pursing sampling have become one of the most popular multi-model fu- methods. sion methods [59]. Different variants of Co-Kriging have 4.1.1 Efficient global optimization method also been proposed to improve the approximation accu- The well-known efficient global optimization algorithm racy [64–67] and reduce the metamodeling cost [66–69]. generates infill sample points in the design space [71], To illustrate the concept of multi-model fusion in- where the expected improvement is maximized to balance tuitively, a one-dimensional (1D) numerical example is the exploration and exploitation of optimization. The presented. This example has been widely reported in the formulation of the expected improvement is given by literature to demonstrate the effects of multi-model fusion Eq. (18): methods [56,57,59,66,70]. High-fidelity and low-fidelity models are formulated in Eq. (17). Here, the parame- EI(x) = E[I(x)] ters adopt A = 0.5, B = 10, and C = − 5. A Kriging y − f (x) min KRG (y − f (x))Φ  min KRG sˆ (x)  KRG model is constructed using pure high-fidelity samples. A y − f (x) min KRG +sˆ (x)ϕ , sˆ (x) > 0 KRG KRG sˆ (x)  KRG Co-Kriging model is constructed using both high-fidelity 0, sˆ (x) = 0 KRG and low-fidelity samples. As illustrated in Fig. 7, Kriging (18) inadequately approximates f , whereas the Co-Kriging HF calculation is relatively close to f with the support of HF where Φ (· ) and ϕ (· ) are the Gaussian probability dis- low-fidelity data. tribution function and probability density function, re- spectively; y is the minimum objective function value 2 min f (x) = (6x − 2) sin(12x − 4) HF , x ∈ [0, 1] (17) among existing sample points. As shown in Eq. (18), f (x) = Af + B(x − 0.5) + C LF HF if the newly added infill sample point is not the same as existing sample points, then EI(x) > 0; otherwise, EI(x) = 0. 4 Metamodel-based multidisciplinary To illustrate the efficient global optimization process optimization strategy intuitively, a 1D numerical example (Eq. (19)) is investi- 4.1 Infill sampling criterion-based optimiza- gated. tion strategy f(x) = (6x − 2) sin(12x − 4) (19) In the dynamic metamodel-based optimization strategy, one widely used method for updating metamodels is to The optimization process is illustrated in Fig. 8. The sequentially allocate the newly added sample points in figure shows that the sample point with the maximum Metamodel-based multidisciplinary design optimization methods for aerospace system 195 EI (a) Initial Kriging model (b) Initial EI(x) curve Expected improvement value Sample with maximum EI value 1.5 0.5 0 0.2 0.4 0.6 0.8 1 (c) Kriging model after the 1st iteration (d) EI(x) after the 1st iteration EI (e) Kriging model after the 2nd iteration (f) EI(x) after the 2nd iteration Fig. 8 Efficient global optimization process. 196 R. Shi, T. Long, N. Ye, et al. EI (g) Kriging model after the 3rd iteration (h) EI(x) after the 3rd iteration Fig. 8 Efficient global optimization process. (Continued) EI(x) in each iteration is selected to update the Kri- where G(i) is the cumulative distribution function, G(i) ging model. In addition, EI(x) decreases with iteration, is the cumulative sum, G is the minimum cumulative min indicating the convergence tendency of the optimization sum, r(R ) is the bias control factor determined by the process. After the 3rd iteration, the optimization process complex correlation coefficient ( R ) of the metamodel, converges to the global optimum. N is the number of inexpensive point groups, and n is cg g To further improve the optimization capacity, some the number of each group. variants of efficient global optimization have been deve- The optimization process of mode-pursing sampling is loped in recent years, especially in constraint han- illustrated in Fig. 9 [79], using a six-hump camel-back dling [72], parallel infill sampling [73], and high- problem [80], as formulated in Eq. (21): dimensional optimization [74]. Further details of the 21 1 (1) 2 (1) 4 (1) 6 (1) (2) f(x) = 4(x ) − (x ) + (x ) + x x variants of efficient global optimization are discussed in 10 3 (2) 2 (2) 4 2 Refs. [75–78]. − 4(x ) + 4(x ) ; x ∈ [−2, 2] (21) 4.1.2 Mode-pursing sampling methods The mode-pursing sampling method proposed by Wang et al. is another typical infill-criterion-based optimization strategy, where the infill criterion is implemented based on a constructed cumulative probability function [79]. In the traditional mode-pursing sampling method, the biased cumulative sum of approximated objective values of numerous inexpensive points constructs the cumulative distribution function, as given in Eq. (20): 1/r(R ) G(i) = G(i) , i = 1,··· , N cg k·n i g X X G(i) = (f − f(x ))/n max j g k=1 j=(k−1)·n +1 2 (20) Fig. 9 Mode-pursing sampling optimization procedure. r(R ) = 1, R < 0.8 In the mode-pursing sampling procedure, numerous " # 2 2 log G (R − 0.8) inexpensive points are randomly generated in the design min · 1 + 1 − , 0.8 ⩽ R ⩽ 1 log 0.75 0.2 space and sorted in ascending order with respect to their Metamodel-based multidisciplinary design optimization methods for aerospace system 197 metamodel responses, f(x ), as shown in Fig. 9(a). Then, estimation capability [98, 99]. Moreover, as an ensemble, ˆ ˆ f(x ) is subtracted from the maximum f(x ) to obtain all criteria can be used to improve robustness; this is i i a nonnegative difference (Fig. 9(b)), which is averaged called a multi-infill strategy [97, 100,101]. over a group of n inexpensive points to obtain G(i) ˆ 4.2 Space reduction-based optimization (Fig. 9(c)). Then, G(i) is normalized as G(x ), as shown ˆ strategy in Fig. 9(d). Here, G(i) satisfies the requisites of a cumu- lative distribution function, such as that monotonically The space reduction-based optimization strategy is an- increasing and varying between 0 and 1. New sample other widely used approach in practice. In this approach, points are selected from inexpensive points based on G(i), a small subspace in the design space is constructed or and the selected probability of inexpensive points in the identified. This subspace is referred to as the region of ith group is equal to G(i). The control factor, r(R ), interest, where the global optimum is located with high biases G(i) toward the inexpensive point groups with probability. During optimization, a number of newly smaller f(x ) values, as illustrated in Fig. 9(e). In other i added samples are generated in the region to gradually words, the more accurate the metamodel, the more likely improve the fitting quality of metamodels in the vicinity inexpensive points (f(x )) are selected. Because G(i) is i of the global optimum. In general, the space reduction- positive throughout the design space, and mode-pursing based optimization strategy includes three major steps: sampling adopts an elite strategy, the global convergence (1) determining the center of the region; (2) calculating property can be theoretically proved easily [79, 81]. the size of the region; and (3) trimming the region ac- Because of the potential global optimization perfor- cording to the design space. The region is identified, as mance of the mode-pursing sampling framework, a range illustrated in Fig. 10. In this paper, several representa- of variants has been proposed for different optimization tive identification methods for the ROI are introduced problems by customizing the expression or application as follows. scenario of G(i). For instance, Kazemi et al. proposed the constraint-importance mode-pursuing sampling method for constrained optimization problems by introducing Design space Sample point penalty items into G(i) [82]. Sharif et al. developed discrete variable mode-pursing sampling for discrete va- riable optimization problems by replacing a continuous objective with a discrete objective and utilizing a double- sphere strategy to improve the local exploitation abi- lity [83]. Shan and Wang proposed a Pareto set pursing method for multi-objective optimization problems by ag- Radius gregating multiple objectives into a single fitness function that reflects the dominance of sample points [84]. Other variants have also been developed for high-dimensional Center Trimmed expensive black-box optimization problems [81, 85] or Global optimum region of interest high-dimensional expensive constrained black-box opti- mization problems [86]. In addition to the foregoing algorithms, other widely Fig. 10 Space reduction-based optimization strategy. used infill sampling criteria include the minimization of surrogate prediction [87, 88], probability of improve- 4.2.1 Significant design space method ment [89, 90], mean square error [91, 92], lower confi- The significant design space method is a simple space- dence bounding [87–95], and fuzzy clustering-based crite- reduction method [94]. In this method, the region of rion [96,97]. Among the different infill sampling criteria, interest is constructed around the best sample (x ) at expected improvement, probability of improvement, mean the kth iteration. The space size (B ) is determined square error, and lower confidence bounding are generally according to the metamodel approximation accuracy in combined with Kriging or other metamodels with error the vicinity of x , as shown in Eq. (22): k 198 R. Shi, T. Long, N. Ye, et al. ∗ ∗ ∗ e = |(f − f )/f | the shrinking and amplification factors, respectively; and k k k (22) ∆ is the upper bound of the trust region radius. As ϑ = e /e k a k proved in Ref. [102], the trust region-based optimization where e is the metamodel approximation accuracy in the strategy can converge to the local optimum, which can ∗ ∗ ∗ vicinity of x ; f and f are the objective function and k k k also converge to the global optimum. In recent years, metamodel prediction values at x , respectively; ϑ is some trust region method-based adaptive optimization the size scale indicator; and e is the acceptable deviation methods have been developed to further improve the typically in the range of 0.001–0.050. Based on Eq. (22), optimization performance, as detailed in Refs. [103–105]. if the metamodel approximation accuracy is satisfied (i.e., 4.2.3 Interesting sampling region method ϑ > 1), then B is increased to B /α to explore the k k k−1 The interesting sampling region method is a machine- global optimum; otherwise, B is decreased to B · α k k−1 learning-assisted method for locating the potential region to improve the metamodel approximation accuracy and containing the global optimum [106]. In this method, the exploit the local optimum. If the length in any dimension existing samples are classified into two categories with is no less than the threshold, then the kth trial significant (L) (U) respect to their responses. If the objective value at the design space is defined as S = [B ,B ], as shown in k k k sample point is less than the predefined threshold, P , Eq. (23): thresh the sample is labeled as a superior sample, indicating (L) B = x − B best k that the sample is probably located close to the global (23) (U) B = x + B best k k optimum. A binary machine learning classifier (e.g., support vector machine [107] and Bayes classifier [108]) To prevent the significant design space from exceeding is constructed based on labeled samples. Then, a larger the entire design space, S , the kth space, S , is defined 0 k number of inexpensive geometry points are generated as the intersection between S and S , as illustrated in via Latin hypercube design in the design space in which Fig. 10. superior inexpensive points are identified by the trained 4.2.2 Trust region method classifier. The cluster center of superior inexpensive The trust region method is a widely used space reduction points is determined to depict the potential location of method for refining metamodels, where the size of the the global optimum. For instance, consider the six camel- region of interest is adjusted according to the prediction hump functions; the superior inexpensive points (i.e., dots performance of the objective improvement [102], as shown labeled with red circles) under different classification in Eq. (24): thresholds (P ) are illustrated in Fig. 11. The figure thresh ∗ ∗ ∗ ∗ ∆f = (f − f )/(f − f ) (24) k−1 k k−1 k shows that the potential location can be depicted by the cluster of superior inexpensive points using a proper where ∆ f is the prediction performance of objective P value. thresh improvement. In Eq. (24), a larger ∆ f indicates that Finally, the interesting sampling region at the kth the optimization process can lead to a better optimum. iteration is defined as a hypercube subregion, as shown A smaller ∆ f indicates that the improvement in the in Eq. (26): objective is insignificant. Note that the objective cannot (k) (k) (k) (k) (k) be improved if t is negative. ISR = [x|x − R ⩽ x ⩽ x + R ] ISR ISR The size of the region can be determined in terms of (k) (k) ∗ where R = η ∥x − x ∥ (26) pse ISR the radius of the trust region, as shown in Eq. (25): (k) ∗ ∗ where the current pseudo-optimum (x ) is the center c ∥x − x ∥, ∆f < 0.1 ISR  k k−1 ∗ ∗ of the region; x is the cluster center of superior inex- pse δ = min(c ∥x − x ∥, ∆), ∆f > 0.75 k+1 2 k k−1 pensive points (also regarded as the potential position of ∗ ∗ ∥x − x ∥, 0.10 ⩽ ∆f ⩽ 0.075 k k−1 (k) global optimum); R is the Euclidean distance between 0 < c < 1, c > 1 (25) 1 2 (k) x and x ; and η is the shrinking coefficient used to ISR pse (k) adjust the size of R . As the optimization proceeds, where δ is the radius of the trust region at the (k+1)th k+1 ∗ ∗ iteration; x and x are the best sample points at the the interesting sampling region is dynamically scaled k k−1 (k) kth and (k − 1)th iterations, respectively; c and c are based on the Euclidean distance between x and x , 1 2 ISR pse Metamodel-based multidisciplinary design optimization methods for aerospace system 199 Classification threshold = 5 Classification threshold = 0 Classification threshold =  1 Classification threshold = 2 2 2 2 1 1 1 1 0 0 0 0 1  1 1  1 2  2 2  2 2  1 0 1 2  2  1 0 1 2 2  1 0 1 2  2  1 0 1 2 Fig. 11 Identification of superior inexpensive points. considerably improving the optimization efficiency and evolutionary algorithms, metamodels are developed as global convergence. Once the current pseudo-optimum “surrogates” of expensive simulation models for stochas- (k) (x ) is far from the potential global optimum (i.e., tic evolutionary operations. This can effectively alleviate ISR (k) (k−1) ∗ ∗ ∥x − x ∥ ⩾ ∥x − x ∥ ), the exploration re- the computational complexity in engineering optimiza- 2 2 pse ISR pse ISR gion must be enlarged to include areas that may not have tion. Several potential solutions (called individuals or been covered, thus avoiding missing the true optimum. particles) are also concurrently selected as newly added (k) Additionally, if x is probably in the vicinity of the sample points to improve the surrogate approximation ISR global optimum, then the search region can be reduced to accuracy in the vicinity of the optimum. The fundamen- improve the approximation accuracy of the constructed tal architecture of the algorithm is graphically illustrated metamodels. in Fig. 12. In addition to the aforementioned methods, many other The selection of newly added infill sample points from space reduction methods have been developed. For in- the offspring population is found to be a crucial step in the stance, Wang et al. proposed an adaptive response sur- algorithm [116]. The commonly used selection criteria can face method to solve computation-intensive design prob- be generally divided into three categories: performance- lems, where regions with inferior sample points are cut based, uncertainty-based, and hybrid criteria [118]. In for sampling in the reduced design space and updat- the performance-based criterion, the infill sample points ing the metamodels [109]. In addition, some variants are selected by considering only the predicted fitness of the adaptive response surface method were investi- values [117, 119, 120]. Generally, this criterion cannot gated to further improve optimization convergence and be exclusively used because optimization probably falls robustness [110–112]. Dong et al. proposed a new space into the local optima once the surrogates fail to approxi- reduction-based optimization algorithm to solve the un- mate the simulation model well. The uncertainty-based constrained expensive black-box optimization problems, criterion chooses infill sample points with considerable where a score-based reduced subspace around the current uncertainties, effectively improving the accuracy of surro- best sample is created to accelerate the local conver- gates and leading the search to unexplored regions [121]. gence [113]. Qiu et al. proposed self-organizing maps and However, numerous simulation model evaluations are fuzzy clustering-based three-stage space reduction and performed to explore regions with sparse samples, result- metamodeling optimization methods to improve efficiency ing in a low convergence rate [118]. The hybrid crite- and robustness performance [114]. Liu et al. developed rion simultaneously incorporates performance and uncer- a Monte Carlo method and space reduction strategy to improve the efficiency of the sampling process [115]. tainty [119,122]. The primary objective of this criterion is to balance the surrogate approximation accuracy and 4.3 Surrogate-assisted evolutionary algo- population competitiveness during optimization. rithms In recent years, a number of surrogate-assisted evo- lutionary algorithms have been successfully applied to Motivated by the idea of merging surrogates into the evolutionary process, surrogate-assisted evolutionary al- solve various optimization problems, such as uncon- gorithms have received considerable attention in recent strained/constrained [119,123–130], multi-objective [131– years [116,117]. In the optimization of surrogate-assisted 137], and multi-delit fi y optimization problems [138, 139]. 200 R. Shi, T. Long, N. Ye, et al. Fig. 12 Optimization mechanism of surrogate-assisted evolutionary algorithms. 4.4 Metamodel-based optimization en- To alleviate the computational complexity caused by hanced by machine learning the “curse of dimensionality”, the dimensionality reduc- tion technique provides a promising approach for MBDO Machine learning is a well-known state-of-the-art tech- to solve high-dimensional optimization problems. One nology that has been successfully applied to big data typical approach is to conduct sensitivity analysis before mining, natural language processing, image recognition, optimization starts. Here, primary design variables are medical diagnosis, and so on. Similarly, a considerable reserved for optimization, whereas minor ones are ne- amount of research has been conducted to integrate ma- glected and set as fixed values during optimization [148]. chine learning techniques (e.g., classification learning, Although the dimensionality of the optimization problem cluster analysis, and dimensionality reduction) into en- is reduced by sensitivity analysis, the optimality of the ob- gineering optimization. Classification learning is among tained optimized solution probably decreases owing to the the most widely used machine learning techniques in irrelevance of several design variables. Another dimen- optimization and has considerable potential for identify- sionality reduction method is the utilization of manifold ing the feasibility of a solution during the optimization learning (e.g., principal component analysis and Sammon process. Motivated by this concept, several studies have mapping) to map existing sample points together with been performed to improve the optimization performance alternative infill sample points from high-dimensional by employing a support vector machine or its variants for space into low-dimensional space. A low-dimensional identifying superior inexpensive points [140–144]. Addi- surrogate is also trained to select infill sample points as tionally, other classification learning techniques, such as optimization proceeds [149,150]. decision tree [145] and k-nearest neighbor [146], are em- In modern engineering design, because new problems ployed for optimization. To capture the potential regions are generally derived from a series of previously solved effectively, classified points are further categorized based tasks, the concept of transfer optimization has been re- on a set of user-dene fi d characteristics or attributes using cently developed. It is a newly emerged methodology cluster analysis techniques. For instance, cluster analy- for improving the optimization performance of a new sis can be combined with a support vector machine to identify a potential search region by clustering numerous optimization task by mining existing knowledge [151]. superior inexpensive points, as described in Refs. [21,106]. Although numerous transfer optimization-based methods In multi-objective optimization, cluster analysis can also have been recently investigated [152–154], only a few be implemented to improve the spatial uniformity of the studies on MBDO methods using transfer optimization Pareto frontier when determining newly infilled sample have been reported [155]. Therefore, further improving points or reference vectors [135, 136,147]. the optimization performance by integrating the transfer Metamodel-based multidisciplinary design optimization methods for aerospace system 201 optimization technique into MBDO methods is promising objective function is established for optimization, and the infeasible sample point is penalized with respect to and valuable [151]. the penalty factor scale. In addition to the aforementioned data-driven machine Although penalty function methods are conveniently learning techniques, physics-informed machine learning implemented in practice, determining the proper penalty techniques, also known as physics-informed neural net- factor is difficult. If the penalty factor is extremely small, works [156, 157], have emerged as an alternative to ill- the optimization process is virtually unable to converge posed and inverse problems. Fundamental physical laws to the feasible domain. If the penalty factor is extremely and domain knowledge are embedded by exploiting ob- large, the penalty part can overlap the objective, ren- servational data [158,159], tailoring neural network archi- dering convergence difficult. Nevertheless, the penalty tecture for physics constraints [160, 161], and/or impos- function method remains widely employed in MBDO ing physics constraints into the loss function [162, 163]. methods. For instance, a predened fi penalty function The physics-informed neural network is expected to out- is employed to handle the expensive constraints [164]. perform existing machine learning methods in partially This method is further enhanced in Ref. [110], where the understood, uncertain, and high-dimensional problems. penalty factor is automatically adjusted according to the It is also another potential research topic in the area of constraint violation value. metamodel-based MDO. 5.2 Filter-based methods 5 Expensive black-box constraint-hand- In view of the difficult implementation of penalty ling mechanism function-based methods, penalty-free methods have be- In engineering practice, MDO problems in aerospace come popular in recent years; among these latter meth- systems generally involve various constraints, such as ods, filter-based methods are the most representative. the natural frequencies of a structure and end-of-life The concept of a filter was originally proposed by power in orbit. Note that because most constraints arise Fletcher [165] and was successfully applied to many from time-consuming black-box simulation models (e.g., penalty-free numerical optimization methods [166–168]. structural FEA model), directly calculating the gradient To illustrate the concept of a filter, the constraint viola- information of the constraints via numerical methods tion function of the MDO problem is defined as follows: (e.g., finite difference method) to search for the feasible h(X) = max{g (X), g (X),··· , g (X)} (28) 1 2 m domain is difficult. Thus, in engineering practice, the solution of MDO problems with numerous expensive where h(X) denotes the constraint violation function. A black-box constraints remains complex. To overcome positive h(X) represents an infeasible sample point, and this obstacle, some constraint-handling mechanisms have a larger h(X) generally indicates inadequate infeasibility. been integrated with metamodel-based MDOs to further Based on the definition of the Pareto non-domination improve the optimization capacity. set, the concept of the filter is clarified as follows: Definition 1. If and only if f(x ) ⩽ f(x ) ∩ h(x ) ⩽ i l i 5.1 Penalty function-based methods h(x ), then sample point x is considered to dominate l i Penalty function-based methods are convenient ap- another sample point pair, x ; otherwise, x and x do l i l proaches that are most commonly used for handling not dominate each other. expensive black-box constraints. Here, the objective func- Definition 2. A filter aggregates non-dominated sample tion is augmented by constraints to cope with penalty points. If a sample point (x ) is not dominated by any factors, as given in Eq. (27): point in the current filter, then it can be included in the filter; otherwise, it is rejected. F(X) = f(X) + λ max(g (X), 0) (27) According to the above definitions, the construction where λ is the penalty factor, and F(X) is the augmented of a filter basically means the creation of a Pareto front objective function. The metamodel of the augmented in terms of f(X) and h(X). A filter in the f(X)–h(X) 202 R. Shi, T. Long, N. Ye, et al. space is illustrated in Fig. 13 [21], where the dots repre- Refs. [16,169,170], all constraints are approximated by metamodels to avoid the complicated determination of sent the non-dominated sample points. If a newly added the penalty factor. Moreover, some sequential upda- sample point is located in the shaded area, then it is not ting techniques considering the confidence intervals of dominated by existing points in the filter, that is, the Kriging models for constraint approximation were deve- filter is augmented. If it is located in the hatched area, loped [171,172]. To further solve problems with numerous then the newly added point can dominate at least one constraints, the constraint aggregation method, which point in the filter (i.e., the filter is refined); otherwise, lumps numerous constraints into one or a few constraints the point is rejected by the filter. In fact, the filter gen- to significantly reduce the computational cost, has drawn erates a set of blocks for newly added sample points, as considerable interest in recent years. The Kreisselmeier– indicated by the solid lines in Fig. 13. In constrained Steinhauser function [173] is a well-known constraint optimization procedures, the filter functions as a classi- aggregation method that has been used in various appli- fier for accepting or rejecting a sample point, considering cations [86,174,175]. Moreover, based on Refs. [176,177], both optimality and feasibility. Note that a sample point Kreisselmeier–Steinhauser function-based MBDO meth- can be accepted by the filter only if it improves either the ods are efficient in solving expensive black-box constraint optimality or feasibility; hence, the feasibility and opti- problems. mality of existing sample points are gradually enhanced during the metamodel-based optimization process. 6 Metamodel-based aerospace system Region for MDO examples f ( x) augmen tin g filter 6.1 All-electric geostationary Earth orbit Region for Rejecte d by filter refining filte r satellite MDO problem ( f ( x ), h( x )) The all-electric geostationary Earth orbit satellite MDO k k Non -dominated point s problem is referred from Ref. [19]. The studied all- electric geostationary Earth orbit satellite is a cuboid with a payload module, service module, and solar array. Four Hall-effect thrusters are installed at the bottom of the satellite to execute attitude control, geosynchronous transfer, and geostationary Earth orbit station-keeping maneuvers, as shown in Fig. 14 [19]. The all-electric geo- h( x) stationary Earth orbit satellite MDO problem involves geosynchronous transfer, geostationary keeping, power, Fig. 13 Filter in f(X) = h(X) space (reproduced with thermal control, attitude control, and structure disci- permission from Ref. [21], © The Authors 2018). plines. The design structure matrix of the MDO problem A number of MBDO methods have been developed to is shown in Fig. 15 [19], and the involved analysis models solve expensive constrained black-box problems based on of disciplines are detailed in Ref. [19]. the filter concept. For instance, the filter technique is The satellite MDO problem was formulated using combined with a support vector machine to identify a Eq. (29) [19]. The design variables are steer angles potential sampling sub-region in the vicinity of the global (α, β, φ ) during low-thrust geosynchronous transfer, po- feasible optimum [21]. In Ref. [72], the probability of sitions of electric thrusters (d , d ), battery capac- T N constrained improvement is enhanced; only the sample ity (C ), area of solar arrays (A ), area of thermal s sa point with a positive probability of constrained improve- radiator (A ), angular momentum of reaction wheel ment can be accepted by the filter to enhance optimality (H ), and thickness of composite material structure and feasibility. (SH, CH, TBH, SP, CP, TBP). These are optimized to In addition to the aforementioned constraint-handling minimize the total mass of the satellite (M ) sub- satellite mechanisms, certain customized methods have also ject to the constraints of total transfer time (t ), station- been developed to handle constrained optimization. In keeping accuracy (λ and i ), available power (P max max BOL Metamodel-based multidisciplinary design optimization methods for aerospace system 203 1 GTO 2 GEO 3 Power 4 Thermal 5 Attitude 6 Structure Fig. 15 Design structure matrix of all-electric satellite (re- produced with permission from Ref. [19], © IAA 2017). Table 3 Optimization results of design variables Genetic Filter-based algorithm-based sequential radial Objective multidisciplinary basis function feasible method method Total mass (kg) 2396.6 2355.9 Number of 1200 600 simulations 6.2 Earth-observation MDO problem The Earth observation satellite MDO problem is referred from Ref. [178]. The satellite consists of payload and service cabins, as shown in Fig. 16 [178]. The payloads include a 10-band radiometer and charge-coupled device camera, which are fixed at the bottom of the payload Fig. 14 Schematic of all-electric satellite (reproduced with cabin. The solar arrays are set along the south and north permission from Ref. [19], © IAA 2017). faces of the satellite to reduce the influence of external heat flux. Other facilities, such as battery and attitude and P ), depth of discharge (DOD), steady-state tem- EOL control subsystems, are placed in the service cabin. The perature (T), angular momentum residue (c ), and AC natural frequencies (f and f ). Earth observation satellite MDO problem involves five X Y disciplines: orbit, payload, structure, power, and mass. min M = m + m + m + m satellite payload fuel power TC The design structure matrix of this problem is shown in + m + m AC others Fig. 17 [178]. The disciplinary modeling approaches are presented in Ref. [178]. where X = [α, β, φ, d , d , A , C , A , H , SH, CH, T N sa s r w TBH, SP, CP, TBP] The Earth observation satellite MDO problem also ◦ ◦ aims to minimize the total mass of the satellite, as formu- t ⩽ 180 days, λ ⩽ 0.05 , i ⩽ 0.05 f max max lated in Eq. (30) [178]. The design variables include the P ⩾ 14.15 kW, P ⩾ 11.90 kW BOL EOL s.t. aperture and focal length of payloads, area of solar arrays, 267 K < T < 328 K, c ⩾ 0, DOD ⩽ 0.8 AC capacity of storage battery, and thickness of composite f ⩾ 14 Hz, f ⩾ 14 Hz X Y material structure. The constraints include the signal-to- (29) noise ratio and resolution of payloads, noise-equivalent In this example, the filter-based sequential radial basis temperature difference, surplus power, discharge depth, function method from Ref. [21] was used to solve the and natural frequencies. MDO problem. The optimization results are summarized in Table 3 [21]. Compared with the conventional genetic min M = m + m + m satellite payload bat solar algorithm-based multidisciplinary feasible method, the + m + m str others computational cost of the filter-based sequential radial basis function method is reduced by 50%. where X = [D , D , F , F , T , T , T , A , C ] 1 2 1 2 R BH SH sa s 204 R. Shi, T. Long, N. Ye, et al. Table 4 Comparison of optimization results Number of Optimization Total mass after analysis model method optimization (kg) evaluations Sequential radial basis function 340.11 165 using support vector machine Genetic algorithm 354.97 7329 Sequential quadratic 346.15 322 programming (a) sequential radial basis function using the support vector machine is only found to be 2.25% and 51.24% of the computational costs of genetic algorithm and sequential quadratic programming, respectively. 6.3 Satellite constellation MDO problem The satellite constellation MDO problem is obtained from Ref. [20], where a small satellite constellation is estab- lished to achieve a cooperative Earth observation mission. The design structure matrix of the satellite constellation MDO problem is shown in Fig. 18 [20]. The inter-coupled relationships between the constellation configuration and design of satellite subsystems are considered to enhance (b) system performance. In this example, the constellation is based on the Walker-δ constellation. In this constellation, Fig. 16 Schematic of Earth observation satellite (reproduced the ascending nodes of the orbit planes are uniformly with permission from Ref. [178], © Taylor & Francis 2015). distributed around the equator plane, and the satellites 1 Orbit are uniformly distributed within the orbital planes at the 2 Payload same inclination [20]. 3 Structure min M = (m + m + m system payload power thermal 4 Power + m + m ) × P × S structure others 5 Mass where X = [h, i, Ω , D , f , A , C , T , T ],X = [P, S] c 0 P P s s H P d Fig. 17 Design structure matrix of Earth observation satellite MDO problem (reproduced with permission from C ⩾ 0.8  R Ref. [178], © Taylor & Francis 2015). R ⩽ 250 m, SNR ⩾ 500 s.t. (31)  DOD ⩽ 0.3, g ⩾ 0 R ⩽ 1100 m, R ⩽ 250 m w 1 2       T ⩽ 303.15 K SNR ⩾ 300, SNR ⩾ 500,  1 2   f ⩾ 20 Hz, f ⩾ 20 Hz s.t. NE∆T ⩽ 0.2 K (30) X Y g ⩾ 0, DOD ⩽ 25%, f ⩾ 20 Hz,  w x  The constellation MDO problem was formulated using f ⩾ 20 Hz, f ⩾ 50 Hz y z Eq. (31). This is a continuous-discrete mixed optimiza- tion problem. The continuous design variables (X ) for In this example, the sequential radial basis function constellation configuration include the orbital altitude using a support vector machine from Ref. [106] was used to solve the MDO problem. The optimization results are (h), inclination (i), and right ascension of the ascend- listed in Table 4 [106]. The computational cost of the ing node (Ω); those for satellite subsystems include the Metamodel-based multidisciplinary design optimization methods for aerospace system 205 Fig. 18 Design structure matrix of satellite constellation MDO problem (reproduced with permission from Ref. [20], © Springer-Verlag GmbH Germany, part of Springer Nature 2018). optical parameters of the payload, power subsystem pa- pared with the integer coding-based genetic algorithm is rameters, and thickness of structural plates. The discrete reduced by more than 85%. The optimized configurations design variables (X ) include the number of orbit planes of the constellations are shown in Fig. 19 [20]. (P) and the number of satellites in each orbit plane (S). In addition to the aforementioned metamodel-based These design variables are optimized to minimize the aerospace system MDO examples, metamodeling tech- total mass of the entire constellation system, subject niques have also been widely used in the aerospace field. to the constraints of coverage ratio, payload resolution, For instance, Xiong et al. proposed an intelligent opti- signal-to-noise ratio of payload, depth of discharge, power mization strategy based on statistical machine learning surplus, maximum temperature, and natural frequencies. for spacecraft thermal design, where a neural network In this example, the sequential radial basis function metamodel was used to approximate the spacecraft– method using a support vector machine for discrete– thermophysical model [179]. Smith et al. established a continuous mixed variables is used to solve the MDO pro- charged drag coefficient metamodel for spacecraft deorbit blem. In the optimization method, a discrete–continuous design with ionospheric drag from a low Earth orbit [180]. variable sampling method is utilized to handle the dis- Wu et al. developed a double-layer radial basis function crete variables. The optimization results, summarized metamodel-based optimization method for the liquid– in Table 5 [20], are compared with those from the inte- gas interface determination of on-orbit refueling [181]. ger coding-based genetic algorithm. This indicates that Peng and Wang proposed a dynamic metamodel-based the sequential radial basis function method considerably optimization method for spacecraft formation reconfig- decreases the system mass by approximately 28.63%. uration on liberation point orbits to achieve fast path Moreover, the computational budget of this method com- planning [182]. Feldhacker et al. investigated a sampling- based least-squares regression method to optimize the Table 5 System mass of optimized designs required velocity increments for rendezvous in spacecraft Sequential radial three-body dynamical systems [183]. Peng et al. pro- basis function Integer method for posed an adaptive metamodel-based optimization method Parameters coding-based discrete– called the sequential radial basis function for satellite genetic algorithm continuous mixed truss structure optimization [184]. Gogu et al. proposed variables a response surface approximation method to aid in the Mass of single 330.4 kg 314.5 kg satellite design of integrated thermal protection systems for space- Total mass of craft reentry [185]. From the literature, MBDO methods satellite 5287.2 kg 3773.3 kg are reported as effective and promising for solving com- constellation system plex aerospace system optimization problems. Simpson Number of et al. applied Kriging models to an aerospike nozzle MDO >2000 300 simulations problem for global approximation [186]. 206 R. Shi, T. Long, N. Ye, et al. timization strategies are discussed in detail. Generally, metamodel-based optimization strategies can be classi- fied into two categories: infill criterion-based and space reduction-based optimization strategies. Recently, novel techniques, such as evolution computation and machine learning, have also been integrated with metamodels to further improve optimization performance. Finally, some penalty-based and penalty-free constraint-handling mechanisms are discussed to illustrate how to deal with expensive black-box constraints in the metamodel-based optimization process. Additionally, several real-world aerospace system MDO examples, which can aid re- searchers in realizing the applications of metamodel-based MDO in engineering practice, are presented. Note that (a) Optimized design by integer coding-based genetic algorithm although the contents of this paper are focused on the MDO for aerospace systems, the fundamental metho- dology and technology can also be applied to the design of other engineering systems, such as ships, aircraft, and automobiles, without significant modifications. In future work, the optimization capacity of metamodel-based MDO must be further improved. For instance, due to the “curse of dimensionality”, high- dimensional expensive black-box problems remain com- plex for metamodel-based optimization. In addition, conventional metamodel-based MDOs generally consider multidisciplinary models as black-box functions. By ex- ploiting prior knowledge (e.g., designer’s experience, ex- plicit model expression, and physical laws), the black-box MDO problem can be transformed into a gray-box or (b) Design by sequential radial basis function method even a white-box optimization problem to further reduce cost and improve convergence. Moreover, the combina- Fig. 19 Comparison of constellation configurations (repro- tion of metamodeling, MDO, and artificial intelligence duced with permission from Ref. [20], © Springer-Verlag GmbH Germany, part of Springer Nature 2018). has also attracted considerable interest. Thus, with the development of advanced modeling, analysis, and com- 7 Conclusions putation technologies, the research on metamodel-based In this paper, an overview of the metamodel-based MDO MDO is anticipated to remain attractive. and its application to aerospace systems is presented. The formulation of the MDO problem for aerospace systems Acknowledgements and the overall procedure of the metamodel-based opti- This work was supported by the National Natural Science mization process are introduced. Moreover, the technical Foundation of China (Nos. 52005288 and 51675047) architecture of metamodel-based MDO is summarized. and the Aeronautical Science Foundation of China (No. Metamodeling techniques, including design of experi- 2019ZC072003). ments, metamodels, and multi-model fusion methods, are surveyed with the aim of constructing a reduced- References order numerical approximation model to inexpensively predict system responses during optimization. A number [1] Sobieszczanski-Sobieski, J., Haftka, R. T. Multidisci- of state-of-the-art metamodel-based multidisciplinary op- plinary aerospace design optimization: Survey of recent Metamodel-based multidisciplinary design optimization methods for aerospace system 207 developments. Structural Optimization, 1997, 14(1): [14] Lai, G., Lei, J., Liu, J., Cao, S., Qin, H., Zeng, F., Zhou, 1–23. R. Numerical and experimental study on comprehensive optimization for the KPIs of ship propulsion shafting [2] Sobieszczanski-Sobieski, J. Multidisciplinary design op- design based on MDO. Ocean Engineering, 2021, 222: timization: Attempt at definition. In: Proceedings of the Industry University Workshop on Multidisciplinary Aircraft Design, 1993: 23–48. [15] Liu, Q., Feng, B., Liu, Z. Ship MDO modelling based [3] Martins, J. R. R. A., Lambe, A. B. Multidisciplinary on design structure matrix. Journal of South China design optimization: A survey of architectures. AIAA University of Technology (Natural Science Edition), Journal, 2013, 51(9): 2049–2075. 2015, 43(04): 69–77. (in Chinese) [4] Grossman, B., Gurdal, Z., Strauch, G. J., Eppard, W. [16] Lei, G., Zhu, J., Guo, Y. Multidisciplinary Design Op- M., Haftka, R. T. Integrated aerodynamic/structural timization Methods for Electrical Machines and Drive design of a sailplane wing. Journal of Aircraft, 1988, Systems. Springer-Verlag Berlin Heidelberg, 2016. 25(9): 855–860. [17] Lei, G., Zhu, J., Guo, Y., Liu, C., Ma, B. A review of [5] Livne, E., Schmit, L. A., Friedmann, P. P. Towards design optimization methods for electrical machines. integrated multidisciplinary synthesis of actively con- Energies, 2017, 10(12): 1962. trolled fiber composite wings. Journal of Aircraft, 1990, [18] Wang, Y., Teng, H. Knowledge fusion design method: 27(12): 979–992. Satellite module layout. Chinese Journal of Aeronau- [6] Wang, S., He, L. The summarization multidisciplinary tics, 2009, 22(1): 32–42. design optimization for flying vehicles. Journal of As- [19] Shi, R., Liu, L., Long, T., Liu, J., Yuan, B. Surrogate tronautics, 2004, 25(6): 697–701. (in Chinese) assisted multidisciplinary design optimization for an [7] Chai, X., Yu, X., Wang, Y. Multipoint optimization all-electric GEO satellite. Acta Astronautica, 2017, on fuel efficiency in conceptual design of wide-body 138: 301–317. aircraft. Chinese Journal of Aeronautics, 2018, 31(1): [20] Shi, R., Liu, L., Long, T., Wu, Y., Wang, G. G. Multi- 99–106. disciplinary modeling and surrogate assisted optimiza- [8] Liu, Y. B., Chen, B. Y., Li, Y. H., Shen, H. D. Overview tion for satellite constellation systems. Structural and of control-centric integrated design for hypersonic ve- Multidisciplinary Optimization, 2018, 58(5): 2173– hicles. Astrodynamics, 2018, 2(4): 307–324. [9] Giannella, V., Lombardi, R., Pisani, M. M., Federico, [21] Shi, R., Liu, L., Long, T., Wu, Y., Tang, Y. Filter-based L., Barbarino, M., Citarella, R. A novel optimization sequential radial basis function method for spacecraft framework to replicate the vibro-acoustics response of multidisciplinary design optimization. AIAA Journal, an aircraft fuselage. Applied Sciences, 2020, 10(7): 2018, 57(3): 1019–1031. [22] Feng, X. Multidisciplinary design optimisation meth- [10] Wang, W., Wu, Z., Wang, D., Yang, J., Wang, P., ods and their application in the design of navigation Zhang, W. Hypersonic vehicle aerodynamic optimiza- constellations. Ph.D. Dissertation. Changsha, China: tion using field metamodel-enhanced sequential approx- University of Defence Science and Technology, 2008. imate optimization. International Journal of Aerospace [23] Guo, S., Zhou, W. M., Zhang, J., Sun, F. Y., Yu, D. T. Engineering, 2021, 2021: 8889593. Integrated constellation design and deployment method [11] Wang, H., Shan, S., Wang, G. G., Li, G. Integrating for a regional augmented navigation satellite system least square support vector regression and mode pursu- using piggyback launches. Astrodynamics, 2021, 5(1): ing sampling optimization for crashworthiness design. 49–60. Journal of Mechanical Design, 2011, 133(4): 041002. [24] Brown, N., Olds, J. Evaluation of multidisciplinary op- [12] Henric, P., Maria, R., Gerald, M., Hamid, H. Deve- timazation (MDO) techniques applied to a reusable lopment of synthesis level design model in automobile launch vehicle. In: Proceedings of the 43rd AIAA application suitable for MDO using CO approach. SAE Aerospace Sciences Meeting and Exhibit, 2005: AIAA International Journal of Materials and Manufacturing, 2005-707. 2015, 8(2): 344–356. [13] Xue, Z., Adarsh, E., Fang, J. Multidisciplinary design [25] Castellini, F., Lavagna, M. R., Riccardi, A., Busk ¨ ens, optimization of vehicle weight reduction. SAE Inter- C. Quantitative assessment of multidisciplinary de- national Journal of Materials & Manufacturing, 2016, sign models for expendable launch vehicles. Journal of 9(2): 393–399. Spacecraft and Rockets, 2013, 51(1): 343–359. 208 R. Shi, T. Long, N. Ye, et al. [26] Zhang, P., Yang, J., Luo, Z., Li, X. Collaborative op- [37] Johnson, M. E., Moore, L. M., Ylvisaker, D. Minimax timization design of suspended rocket based on re- and maximin distance designs. Journal of Statistical sponse surface. Journal of Solid Rocket Technology, Planning and Inference, 1990, 26(2): 131–148. 2017, 40(6): 793–798. (in Chinese) [38] Koehler, J. R., Owen, A. B. 9 computer experiments. [27] Liu, Z., Zhang, B. An overview of multidisciplinary Handbook of Statistics, 1996, 13: 261–308. optimization method in launch vehicle design. Astro- [39] Bates, S., Sienz, J., Toropov, V. Formulation of the nautical Systems Engineering Technology, 2017, 1(2): optimal Latin hypercube design of experiments using 1–6. a permutation genetic algorithm. In: Proceedings of [28] Wang, G. G., Shan, S. Review of metamodeling tech- the 45th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics & Materials Conference, 2004: niques in support of engineering design optimization. AIAA 2004-2011. Journal of Mechanical Design, 2007, 129(4): 370–380. [40] Tanabe, R., Fukunaga, A. S. Improving the search [29] Forrester, A. I. J., Keane, A. J. Recent advances in surrogate-based optimization. Progress in Aerospace performance of shade using linear population size re- Sciences, 2009, 45(1–3): 50–79. duction. In: Proceedings of the 2014 IEEE Congress on Evolutionary Computation New York, 2014: 1658– [30] Han, Z., Xu, C., Qiao, J., Liu, F., Chi, J., Meng, G., Zhang, K., Song, W. Recent progress of efficient global [41] Jin, R., Chen, W., Sudjianto, A. An efficient algorithm aerodynamic shape optimization using surrogate-based for constructing optimal design of computer experi- approach. Acta Aeronautica et Astronautica Sinica, ments. Journal of Statistical Planning and Inference, 2020, 41(5): 31–47. (in Chinese) 2005, 134(1): 268–287. [31] De Weck, O., Agte, J., Sobieszczanski-Sobieski, J., [42] Ye, K. Q., Li, W., Sudjianto, A. Algorithmic construc- Arendsen, P., Morris, A., Spieck, M. State-of-the-art tion of optimal symmetric Latin hypercube designs. and future trends in multidisciplinary design optimiza- Journal of Statistical Planning and Inference, 2000, tion. In: Proceedings of the 48th AIAA/ASME/ASCE/ 90(1): 145–159. AHS/ASC Structures, Structural Dynamics, and Ma- terials Conference, 2007: AIAA 2007-1905. [43] Zhu, H., Liu, L., Long, T., Peng, L. A novel algorithm of maximin Latin hypercube design using successive [32] Simpson, T. W., Toropov, V., Balabanov, V., Viana, local enumeration. Engineering Optimization, 2012, F. A. C. Design and analysis of computer experiments 44(5): 551–564. in multidisciplinary design optimization: A review of how far we have come - or not. In: Proceedings of [44] Viana, F. A. C., Venter, G., Balabanov, V. An algo- the 12th AIAA/ISSMO Multidisciplinary Analysis and rithm for fast optimal Latin hypercube design of exper- Optimization Conference, 2008: AIAA 2008-5802. iments. International Journal for Numerical Methods in Engineering, 2010, 82(2): 135–156. [33] Queipo, N. V., Haftka, R. T., Shyy, W., Goel, T., [45] Gunst, R. F. Response surface methodology: process Vaidyanathan, R., Kevin Tucker, P. Surrogate-based analysis and optimization. Progress in Aerospace Sci- and product optimization using designed experiments. ences, 2005, 41(1): 1–28. Technometrics, 1996, 38(3): 284–286. [34] Viana, F. A. C., Simpson, T. W., Balabanov, V., [46] Buhmann, M. D. Radial basis functions: Theory and implementations. Cambridge: Cambridge university Toropov, V. Special section on multidisciplinary design press, 2003. optimization: Metamodeling in multidisciplinary de- sign optimization: How far have we really come? AIAA [47] Viana, F. A. C., Haftka, R. T., Steffen, V. Multiple Journal, 2014, 52(4): 670–690. surrogates: How cross-validation errors can help us to obtain the best predictor. Structural and Multidisci- [35] Simpson, T. W., Booker, A. J., Ghosh, D., Giunta, A. plinary Optimization, 2009, 39(4): 439–457. A., Koch, P. N., Yang, R. J. Approximation methods in multidisciplinary analysis and optimization: A panel [48] Song, X., Lv, L., Li, J., Sun, W., Zhang, J. An advanced discussion. Structural and Multidisciplinary Optimiza- and robust ensemble surrogate model: Extended adap- tion, 2004, 27(5): 302–313. tive hybrid functions. Journal of Mechanical Design, 2018, 140(4): 041402. [36] Long, T., Liu, J., Wang, G. G., Liu, L., Shi, R., Guo, X. Discuss on approximate optimization strategies using [49] Acar, E. Simultaneous optimization of shape param- design of computer experiments and metamodels for eters and weight factors in ensemble of radial basis flight vehicle design. Journal of Mechanical Engineering, functions. Structural and Multidisciplinary Optimiza- 2016, 52(14): 79–105. (in Chinese) tion, 2014, 49(6): 969–978. Metamodel-based multidisciplinary design optimization methods for aerospace system 209 [50] Clarke, S. M., Griebsch, J. H., Simpson, T. W. Anal- and kriging correction. Journal of Engineering Design, ysis of support vector regression for approximation 2013, 24(8): 604–622. [63] Kennedy, M., O’Hagan, A. Predicting the output from of complex engineering analyses. Journal of Mechani- a complex computer code when fast approximations cal Design, Transactions of the ASME, 2005, 127(6): are available. Biometrika, 2000, 87(1): 1–13. 1077–1087. [64] Qian, P. Z. G., Wu, C. F. J. Bayesian hierarchical mod- [51] Awad, M., Khanna, R. Support vector regression. eling for integrating low-accuracy and high-accuracy In: Efficient Learning Machines: Theories, Concepts, experiments. Technometrics, 2008, 50(2): 192–204. and Applications for Engineers and System Designers. [65] Le Gratiet, L. Recursive co-kriging model for Design of Berkeley, CA: Apress. 2015: 67–80. Computer experiments with multiple levels of fidelity [52] Haykin, S. Neural networks: A comprehensive founda- with an application to hydrodynamic. 2012: arXiv: tion. New York: Macmillan College Publishing Com- 1210.0686[math.ST]. Available at https://arxiv.org/ pany, 1998. abs/1210.0686. [53] Damianou, A., Lawrence, N. D. Deep Gaussian pro- [66] Han, Z. H., G¨ortz, S. Hierarchical kriging model for cesses. In: Proceedings of the 16th International Con- variable-fidelity surrogate modeling. AIAA Journal, ference on Artificial Intelligence and Statistics, 2013. 2012, 50(9): 1885–1896. [54] Salimbeni, H., Deisenroth, M. P. Doubly stochastic [67] Zhang, Y., Han, Z. H., Zhang, K. S. Variable-fidelity variational inference for deep Gaussian processes. In: expected improvement method for efficient global opti- Proceedings of the 31st International Conference on mization of expensive functions. Structural and Multi- Neural Information Processing Systems, 2017. disciplinary Optimization, 2018, 58(4): 1431–1451. [55] Rajaram, D., Puranik, T. G., Ashwin Renganathan, S., [68] Han, Z. H., G¨ortz, S. Alternative cokriging method Sung, W., Fischer, O. P., Mavris, D. N., Ramamurthy, for variable-fidelity surrogate modeling. AIAA Journal, A. Empirical assessment of deep Gaussian process sur- 2012, 50(5): 1205–1210. rogate models for engineering problems. Journal of [69] Bertram, A., Zimmermann, R. Theoretical investi- Aircraft, 2020, 58(1): 182–196. gations of the new Cokriging method for variable- [56] Forrester, A. I. J., S´obester, A., Keane, A. J. Multi- fidelity surrogate modeling. Advances in Computational fidelity optimization via surrogate modelling. Proceed- Mathematics, 2018, 44(6): 1693–1716. ings of the Royal Society A: Mathematical, Physical [70] Toal, D. J. J. Some considerations regarding the use of and Engineering Science, 2007, 463: 3251–3269. multi-fidelity Kriging in the construction of surrogate [57] Park, C., Haftka, R. T., Kim, N. H. Remarks on multi- models. Structural and Multidisciplinary Optimization, fidelity surrogates. Structural and Multidisciplinary Op- 2015, 51(6): 1223–1245. timization, 2017, 55(3): 1029–1050. [71] Jones, D. R., Schonlau, M., Welch, W. J. Efficient [58] Le Gratiet, L. Multi-fidelity Gaussian process regression global optimization of expensive black-box functions. for computer experiments, Universite´ Paris-Diderot - Journal of Global Optimization, 1998, 13(4): 455–492. Paris VII, 2013. [72] Shi, R., Liu, L., Long, T., Wu, Y., Tang, Y. Filter- [59] Zhou, Q., Yang, Y., Song, X., Han, Z., Cheng, Y., Hu, based adaptive Kriging method for black-box optimiza- J., Shu, L., Jiang, P. Survey of multi-fidelity surrogate tion problems with expensive objective and constraints. models and their applications in the design and opti- Computer Methods in Applied Mechanics and Engineer- mization of engineering equipment. Journal of Mechan- ing, 2019, 347: 782–805. ical Engineering, 2020, 56(24): 219–245. (in Chinese) [73] Long, T., Wei, Z., Shi, R. H., Wu, Y. F. Parallel adap- [60] Liu, H., Ong, Y. S., Cai, J., Wang, Y. Cope with diverse tive kriging method with constraint aggregation for ex- data structures in multi-fidelity modeling: A Gaussian pensive black-box optimization problems. AIAA Jour- process method. Engineering Applications of Artificial nal, 2021: 1–15. Intelligence, 2018, 67: 211–225. [74] Amine Bouhlel, M., Bartoli, N., Regis, R. G., Otsmane, [61] Fischer, C. C., Grandhi, R. V. Utilizing an adjustment A., Morlier, J. Efficient global optimization for high- factor to scale between multiple fidelities within a design dimensional constrained problems by using the Kriging process: A stepping stone to dialable fidelity design. models combined with the partial least squares method. In: Proceedings of the 16th AIAA Non-Deterministic Engineering Optimization, 2018, 50(12): 2038–2053. Approaches Conference, 2014: AIAA 2014-1011. [75] Xu, S. G., Chen, H. Q., Zhang, J. L. A study of Nash- [62] Zheng, J., Shao, X. Y., Gao, L., Jiang, P., Li, Z. L. A EGO algorithm for aerodynamic shape design optimiza- hybrid variable-fidelity global approximation modelling tions. Structural and Multidisciplinary Optimization, method combining tuned radial basis function base 2019, 59(4): 1241–1254. 210 R. Shi, T. Long, N. Ye, et al. [76] Viana, F. A. C., Haftka, R. T., Watson, L. T. Effi- [87] Booker, A. J., Dennis, J. E., Frank, P. D., Serafini, D. cient global optimization algorithm assisted by multiple B., Torczon, V., Trosset, M. W. A rigorous framework surrogate techniques. Journal of Global Optimization, for optimization of expensive functions by surrogates. 2013, 56(2): 669–689. Structural Optimization, 1999, 17(1): 1–13. [77] He, Y., Sun, J., Song, P., Wang, X. Variable-fidelity [88] Jones, D. R. A taxonomy of global optimization meth- ods based on response surfaces. Journal of Global Op- expected improvement based efficient global optimiza- timization, 2001, 21(4): 345–383. tion of expensive problems in presence of simulation failures and its parallelization. Aerospace Science and [89] Watson, A. G., Barnes, R. J. Infill sampling criteria to Technology, 2021, 111: 106572. locate extremes. Mathematical Geology, 1995, 27(5): [78] Zhan, D. W., Xing, H. L. Expected improvement for 589–608. expensive optimization: A review. Journal of Global [90] Keane, A. J., Forrester, A. I. J., S´obester, A. Engineer- Optimization, 2020, 78(3): 507–544. ing Design via Surrogate Modelling: A Practical Guide. John Wiley & Sons, Ltd., 2008. [79] Wang, L., Shan, S., Wang, G. G. Mode-pursuing sam- pling method for global optimization on expensive [91] Martin, J., Simpson, T. W. Use of adaptive metamod- black-box functions. Engineering Optimization, 2004, eling for design optimization. In: Proceedings of the 36(4): 419–438. 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, 2002: 2002–5631. [80] Branin, F. H. J., Hoo, S. K. A method for finding multiple extrema of a function of n variables. Numerical [92] van Beers, W. C. M., Kleijnen, J. P. C. Kriging for Methods, 1972: 231–237. interpolation in random simulation. Journal of the Op- erational Research Society, 2003, 54(3): 255–262. [81] Wu, Y. F., Long, T., Shi, R. H., Wang, G. G. Mode- pursuing sampling method using discriminative coordi- [93] Cox, D. D., John, S. A statistical method for global nate perturbation for high-dimensional expensive black- optimization. In: Proceedings of the 1992 IEEE Inter- box optimization. Journal of Mechanical Design, 2021, national Conference on Systems, Man, and Cybernetics, 143(4): 041703. 1992: 1241–1246. [82] Kazemi, M., Wang, G. G., Rahnamayan, S., Gupta, [94] Long, T., Liu, L., Peng, L. Global optimization method with enhanced adaptive response surface method for K. Constraint importance mode pursuing sampling for computation-intensive design problems. Advanced Sci- continuous global optimization. In: Proceedings of the ASME 2010 International Design Engineering Tech- ence Letters, 2012, 5(2): 881–887. nical Conferences and Computers and Information in [95] Jiang, P., Cheng, J., Zhou, Q., Shu, L. S., Hu, J. X. Engineering Conference, 2011: 325–334. Variable-fidelity lower confidence bounding approach for engineering optimization problems with expensive [83] Sharif, B., Wang, G. G., Elmekkawy, T. Y. Mode pursu- simulations. AIAA Journal, 2019, 57(12): 5416–5430. ing sampling method for discrete variable optimization on expensive black-box functions. Journal of Mechani- [96] Zhu, H. G., Liu, L., Long, T., Zhao, J. F. Global opti- cal Design, 2008, 130(2): 021402. mization method using SLE and adaptive RBF based on fuzzy clustering. Chinese Journal of Mechanical En- [84] Wang, G. G., Shan, S. Q. An efficient Pareto set iden- gineering, 2012, 25(4): 768–775. tification approach for multi-objective optimization on black-box functions. In: Proceedings of the ASME 2004 [97] Tian, K., Li, Z. C., Huang, L., Du, K. F., Jiang, L. L., International Design Engineering Technical Conferences Wang, B. Enhanced variable-fidelity surrogate-based and Computers and Information in Engineering Con- optimization framework by Gaussian process regression ference, 2008: 279–291. and fuzzy clustering. Computer Methods in Applied Mechanics and Engineering, 2020, 366: 113045. [85] Cheng, G. H., Younis, A., Haji Hajikolaei, K., Wang, G. G. Trust region based mode pursuing sampling [98] Zhou, Y. C., Lu, Z. Z. An enhanced Kriging surrogate method for global optimization of high dimensional modeling technique for high-dimensional problems. Me- design problems. Journal of Mechanical Design, 2015, chanical Systems and Signal Processing, 2020, 140: 137(2): 021407. 106687. [86] Cheng, G. H., Gjernes, T., Wang, G. G. An adaptive [99] Han, Z. Kriging surrogate model and its application aggregation-based approach for expensively constrained to design optimization: A review of recent progress. black-box optimization problems. Journal of Mechanical Acta Aeronautica et Astronautica Sinica, 2016, 37(11): Design, 2018, 140(9): 091402. 3197–3225. (in Chinese) Metamodel-based multidisciplinary design optimization methods for aerospace system 211 [100] Gao, Y., Wang, X. A sequential optimization method [112] Wu, D., Long, T., Li, Y. L., Jiang, M. L., Huang, B. with multi-point sampling criterion based on Kriging Aero-structure-stealth coupled optimization for high surrogate model. Engineering Mechanics, 2012, 29(4): aspect ratio wing using adaptive metamodeling method. 90–95. (in Chinese) 15th AIAA/ISSMO Multidisciplinary Analysis and Op- [101] Song, C., Yang, X. D., Song, W. P. Multi-infill strategy timization Conference, 2014. [113] Dong, H. C., Sun, S. Q., Song, B. W., Wang, P. Multi- for kriging models used in variable fidelity optimization. surrogate-based global optimization using a score-based Chinese Journal of Aeronautics, 2018, 31(3): 448–456. infill criterion. Structural and Multidisciplinary Opti- [102] Alexandrov, N. M., Dennis, J. E., Lewis, R. M., Torc- mization, 2019, 59(2): 485–506. zon, V. A trust-region framework for managing the use [114] Qiu, H. B., Xu, Y. J., Gao, L., Li, X. K., Chi, L. of approximation models in optimization. Structural Multi-stage design space reduction and metamodeling Optimization, 1998, 15(1): 16–23. optimization method based on self-organizing maps and [103] Perez, ´ V. M., Renaud, J. E., Watson, L. T. Adaptive fuzzy clustering. Expert Systems With Applications, experimental design for construction of response surface 2016, 46: 180–195. approximations. AIAA Journal, 2002, 40(12): 2495– [115] Liu, H. T., Xu, S. L., Wang, X. F. Sequential sam- pling designs based on space reduction. Engineering [104] Gano, S. E., Renaud, J. E., Martin, J. D., Simpson, T. Optimization, 2015, 47(7): 867–884. W. Update strategies for kriging models used in variable [116] Jin, Y. C. Surrogate-assisted evolutionary computation: fidelity optimization. Structural and Multidisciplinary Recent advances and future challenges. Swarm and Optimization, 2006, 32(4): 287–298. Evolutionary Computation, 2011, 1(2): 61–70. [105] Cheng, G., Wang, G. F. Trust region based MPS [117] Jin, Y. C., Wang, H. D., Chugh, T., Guo, D., Miettinen, method for global optimization of high dimensional K. Data-driven evolutionary optimization: An overview design problems. In: Proceedings of the 53rd AIAA/ and case studies. IEEE Transactions on Evolutionary ASME/ASCE/AHS/ASC Structures, Structural Dy- Computation, 2019, 23(3): 442–458. namics and Materials Conference, 2012: AIAA 2012- [118] Li, F., Shen, W. M., Cai, X. W., Gao, L., Wang, G. G. A fast surrogate-assisted particle swarm optimiza- [106] Shi, R. H., Liu, L., Long, T., Liu, J. Sequential radial tion algorithm for computationally expensive problems. basis function using support vector machine for expen- Applied Soft Computing, 2020, 92: 106303. sive design optimization. AIAA Journal, 2016, 55(1): [119] Yang, Z., Qiu, H. B., Gao, L., Cai, X. W., Jiang, 214–227. C., Chen, L. M. Surrogate-assisted classification- [107] Suthaharan, S. Support vector machine. In: Machine collaboration differential evolution for expensive con- Learning Models and Algorithms for Big Data Classifi- strained optimization problems. Information Sciences, cation. Boston, MA: Springer US, 2016: 207–235. 2020, 508: 50–63. [108] Rish, I. An empirical study of the naive Bayes classi- [120] Wang, X. J., Wang, G. G., Song, B. W., Wang, P., fier. In: Proceedings of the IJCAI 2001 Workshop on Wang, Y. A novel evolutionary sampling assisted opti- Empirical Methods in Artificial Intelligence, 2001. mization method for high-dimensional expensive prob- [109] Wang, G. G., Dong, Z. M., Aitchison, P. Adaptive lems. IEEE Transactions on Evolutionary Computation, response surface method - a global optimization scheme 2019, 23(5): 815–827. for approximation-based design problems. Engineering [121] Tian, J., Tan, Y., Zeng, J. C., Sun, C. L., Jin, Optimization, 2001, 33(6): 707–733. Y. C. Multiobjective infill criterion driven Gaussian [110] Long, T., Wu, D., Guo, X. S., Wang, G. G., Liu, process-assisted particle swarm optimization of high- L. Efficient adaptive response surface method using dimensional expensive problems. IEEE Transactions intelligent space exploration strategy. Structural and on Evolutionary Computation, 2019, 23(3): 459–472. Multidisciplinary Optimization, 2015, 51(6): 1335– [122] Wang, H. D., Jin, Y. C., Doherty, J. Committee-based active learning for surrogate-assisted particle swarm [111] Long, T., Liu, L., Peng, L., Li, Y. Aero-structure cou- optimization of expensive problems. IEEE Transactions pled optimization of high aspect ratio wing using en- on Cybernetics, 2017, 47(9): 2664–2677. hanced adaptive response surface method. In: Proceed- [123] Glaz, B., Friedmann, P. P., Liu, L., Cajigas, J. G., ings of the 12th AIAA Aviation Technology, Integration, Bain, J., Sankar, L. N. Reduced-order dynamic stall and Operations Conference and 14th AIAA/ISSMO modeling with swept flow effects using a surrogate-based Multidisciplinary Analysis and Optimization Confer- recurrence framework. AIAA Journal, 2013, 51(4): ence, 2012: AIAA 2012-5456. 910–921. 212 R. Shi, T. Long, N. Ye, et al. [124] Sun, C. L., Zeng, J. C., Pan, J., Xue, S. D., Jin, Y. C. [136] Chugh, T., Jin, Y. C., Miettinen, K., Hakanen, J., Sind- A new fitness estimation strategy for particle swarm hya, K. A surrogate-assisted reference vector guided optimization. Information Sciences, 2013, 221: 355– evolutionary algorithm for computationally expensive 370. many-objective optimization. IEEE Transactions on [125] Vincenzi, L., Gambarelli, P. A proper infill sampling Evolutionary Computation, 2018, 22(1): 129–142. strategy for improving the speed performance of a [137] Wan, K. Z., He, C., Camacho, A., Shang, K., Cheng, Surrogate-Assisted Evolutionary Algorithm. Computers R., Ishibuchi, H. A hybrid surrogate-assisted evolu- & Structures, 2017, 178: 58–70. tionary algorithm for computationally expensive many- [126] Miranda-Varela, M. E., Mezura-Montes, E. Constraint- objective optimization. In: Proceedings of the 2019 handling techniques in surrogate-assisted evolutionary IEEE Congress on Evolutionary Computation, 2019: optimization. An empirical study. Applied Soft Com- 2018–2025. [138] Liu, B., Koziel, S., Zhang, Q. F. A multi-fidelity puting, 2018, 73: 215–229. [127] Wang, Y., Yin, D. Q., Yang, S. X., Sun, G. Y. Global surrogate-model-assisted evolutionary algorithm for and local surrogate-assisted differential evolution for computationally expensive optimization problems. expensive constrained optimization problems with in- Journal of Computational Science, 2016, 12: 28–37. equality constraints. IEEE Transactions on Cybernet- [139] Yi, J., Shen, Y. C., Shoemaker, C. A. A multi-fidelity ics, 2019, 49(5): 1642–1656. RBF surrogate-based optimization framework for com- [128] Zhao, X. G., Zhang, K., Chen, G. D., Xue, X. M., putationally expensive multi-modal problems with ap- Yao, C. J., Wang, J., Yang, Y. F., Zhao, H., Yao, J. plication to capacity planning of manufacturing sys- Surrogate-assisted differential evolution for production tems. Structural and Multidisciplinary Optimization, optimization with nonlinear state constraints. Jour- 2020, 62(4): 1787–1807. [140] Xie, T. L., Jiang, P., Zhou, Q., Shu, L. S., Zhang, Y. nal of Petroleum Science and Engineering, 2020, 194: H., Meng, X. Z., Wei, H. Advanced multi-objective [129] Zhang, K., Zhao, X. G., Chen, G. D., Zhao, M. J., robust optimization under interval uncertainty using Wang, J., Yao, C. J., Sun, H., Yao, J., Wang, W., kriging model and support vector machine. Journal of Zhang, G. D. A double-model differential evolution Computing and Information Science in Engineering, for constrained waterflooding production optimization. 2018, 18(4): 041012. Journal of Petroleum Science and Engineering, 2021, [141] Malak, R. J. Jr, Paredis, C. J. J. Using support vector machines to formalize the valid input domain of pre- 207: 109059. [130] Yu, M. Y., Li, X., Liang, J. A dynamic surrogate- dictive models in systems design problems. Journal of assisted evolutionary algorithm framework for expen- Mechanical Design, 2010, 132(10): 101001. [142] Liu, X., Wu, Y. Z., Wang, B. X., Ding, J. W., Jie, H. X. sive structural optimization. Structural and Multidisci- An adaptive local range sampling method for reliability- plinary Optimization, 2020, 61(2): 711–729. [131] Lim, D. K., Woo, D. K., Yeo, H. K., Jung, S. Y., Ro, based design optimization using support vector machine J. S., Jung, H. K. A novel surrogate-assisted multi- and Kriging model. Structural and Multidisciplinary objective optimization algorithm for an electromag- Optimization, 2017, 55(6): 2285–2304. [143] Tang, Y. F., Long, T., Shi, R. H., Wu, Y. F., Wang, G. netic machine design. IEEE Transactions on Magnetics, 2015, 51(3): 1–4. G. Sequential radial basis function-based optimization [132] Datta, R., Regis, R. G. A surrogate-assisted evolution method using virtual sample generation. Journal of strategy for constrained multi-objective optimization. Mechanical Design, 2020, 142(11): 111701. Expert Systems With Applications, 2016, 57: 270–284. [144] Basudhar, A., Dribusch, C., Lacaze, S., Missoum, S. [133] Dıaz-Manr ´ ıquez, ´ A., Toscano, G., Barron-Zambrano, Constrained efficient global optimization with support J. H., Tello-Leal, E. A review of surrogate assisted vector machines. Structural and Multidisciplinary Op- multiobjective evolutionary algorithms. Computational timization, 2012, 46(2): 201–221. [145] Lim, D., Ong, Y. S., Gupta, A., Goh, C. K., Dutta, P. Intelligence and Neuroscience, 2016, 2016: 1–14. [134] Venturelli, G., Benini, E., Laniewski-Wo l lk, L. A S. Towards a new Praxis in optinformatics targeting Kriging-assisted multiobjective evolutionary algorithm. knowledge re-use in evolutionary computation: Simulta- Applied Soft Computing, 2017, 58: 155–175. neous problem learning and optimization. Evolutionary [135] Wang, C. Z., Ding, J. L., Cheng, R., Liu, C. X., Chai, Intelligence, 2016, 9(4): 203–220. T. Y. Data-driven surrogate-assisted multi-objective [146] Park, J., Kim, K. Y. Instance variant nearest neighbor optimization of complex beneficiation operational pro- using particle swarm optimization for function approxi- cess. IFAC-PapersOnLine, 2017, 50(1): 14982–14987. mation. Applied Soft Computing, 2016, 40: 331–341. Metamodel-based multidisciplinary design optimization methods for aerospace system 213 [147] Isaacs, A., Ray, T., Smith, W. An evolutionary algo- [158] Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G. rithm with spatially distributed surrogates for multi- E. Learning nonlinear operators via DeepONet based objective optimization. In: Progress in Artificial Life . on the universal approximation theorem of operators. ACAL 2007. Lecture Notes in Computer Science, Vol. Nature Machine Intelligence, 2021, 3(3): 218–229. 4828. Randall, M., Abbass, H. A., Wiles, J., Eds. [159] Kashefi, A., Rempe, D., Guibas, L. J. A point-cloud Springer, Berlin, Heidelberg, 2007: 257–268 deep learning framework for prediction of fluid flow fields on irregular geometries. Physics of Fluids, 2021, [148] Castillo, E., Mınguez, ´ R., Castillo, C. Sensitivity anal- 33(2): 027104. ysis in optimization and reliability problems. Reliability Engineering & System Safety, 2008, 93(12): 1788–1800. [160] Kondor, R., Son, H. T., Pan, H., Anderson, B., Trivedi, S. Covariant compositional networks for learn- [149] Chen, G., Zhang, K., Xue, X., Zhang, L., Yao, J., Sun, ing graphs. 2018: arXiv:180102144. Available at https: H., Fan, L., Yang, Y. Surrogate-assisted evolution- //arxiv.org/abs/2009.11193. ary algorithm with dimensionality reduction method [161] Darbon, J., Meng, T. On some neural network architec- for water flooding production optimization. Journal of tures that can represent viscosity solutions of certain Petroleum Science and Engineering, 2020, 185: 106633. high dimensional Hamilton–Jacobi partial differential [150] Liu, B., Zhang, Q., Gielen, G. G. A Gaussian process equations. Journal of Computational Physics, 2021, surrogate model assisted evolutionary algorithm for 425: 109907. medium scale expensive optimization problems. IEEE [162] Chen, W., Chiu, K., Fuge, M. D. Airfoil design parame- Transactions on Evolutionary Computation, 2013, terization and optimization using bezier ´ generative ad- 18(2): 180–192. versarial networks. AIAA Journal, 2020, 58(11): 4723– [151] Gupta, A., Ong, Y., Feng, L. Insights on transfer opti- mization: Because experience is the best teacher. IEEE [163] Pfrommer, S., Halm, M., Posa, M. Contactnets: Learn- Transactions on Emerging Topics in Computational ing of discontinuous contact dynamics with smooth, Intelligence, 2018, 2(1): 51–64. implicit representations. 2020: arXiv:200911193. Avail- [152] Gupta, A., Ong, Y., Feng, L., Tan, K. C. Multiobjective able at https://arxiv.org/abs/2009.11193. multifactorial optimization in evolutionary multitask- [164] Wang, G. G., Simpson, T. Fuzzy clustering based hier- ing. IEEE Transactions on Cybernetics, 2017, 47(7): archical metamodeling for design space reduction and 1652–1665. optimization. Engineering Optimization, 2004, 36(3): [153] Ma, X., Chen, Q., Yu, Y., Sun, Y., Ma, L., Zhu, Z. 313–335. A two-level transfer learning algorithm for evolutiona- [165] Fletcher, R., Leyffer, S. Nonlinear programming with- ry multitasking. Frontiers in Neuroscience, 2020, 13: out a penalty function. Mathematical Programming, 2002, 91(2): 239–269. [154] Cheng, M.-Y., Gupta, A., Ong, Y.-S., Ni, Z.-W. Co- [166] Fletcher, R., Leyffer, S., Toint, P. L. On the global evolutionary multitasking for concurrent global opti- convergence of a filter-SQP algorithm. SIAM Journal mization: With case studies in complex engineering on Optimization, 2002, 13(1): 44–59. design. Engineering Applications of Artificial Intelli- [167] Audet, C., Dennis, J. E. A pattern search filter method gence, 2017, 64: 13–24. for nonlinear programming without derivatives. SIAM [155] Min, A. T. W., Ong, Y., Gupta, A., Goh, C. Multi- Journal on Optimization, 2004, 14(4): 980–1010. problem surrogates: Transfer evolutionary multiobjec- [168] Gonzaga, C. C., Karas, E., Vanti, M. A globally conver- tive optimization of computationally expensive prob- gent filter method for nonlinear programming. SIAM lems. IEEE Transactions on Evolutionary Computation, Journal on Optimization, 2004, 14(3): 646–669. 2019, 23(1): 15–28. [169] Regis, R. G. Constrained optimization by radial basis [156] Raissi, M., Perdikaris, P., Karniadakis, G. E. Physics- function interpolation for high-dimensional expensive informed neural networks: A deep learning framework black-box problems with infeasible initial points. Engi- for solving forward and inverse problems involving non- neering Optimization, 2014, 46(2): 218–243. linear partial differential equations. Journal of Compu- [170] Rashid, K., Ambani, S., Cetinkaya, E. An adaptive mul- tational Physics, 2019, 378: 686–707. tiquadric radial basis function method for expensive [157] Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, black-box mixed-integer nonlinear constrained opti- P., Wang, S., Yang, L. Physics-informed machine learn- mization. Engineering Optimization, 2013, 45(2): 185– ing. Nature Reviews Physics, 2021, 3(6): 422–440. 206. 214 R. Shi, T. Long, N. Ye, et al. [171] Qian, J. C., Yi, J. X., Cheng, Y. S., Liu, J., Zhou, Q. [184] Peng, L., Liu, L., Long, T., Yang, W. An efficient truss A sequential constraints updating approach for Krig- structure optimization framework based on CAD/CAE ing surrogate model-assisted engineering optimization integration and sequential radial basis function meta- design problem. Engineering With Computers, 2020, model. Structural and Multidisciplinary Optimization, 36(3): 993–1009. 2014, 50(2): 329–346. [172] Sasena, M. J., Papalambros, P., Goovaerts, P. Ex- [185] Gogu, C., Haftka, R. T., Bapanapalli, S. K., Sankar, ploration of metamodeling sampling criteria for con- B. V. Dimensionality reduction approach for response strained global optimization. Engineering Optimization, surface approximations: Application to thermal design. 2002, 34(3): 263–278. AIAA Journal, 2009, 47(7): 1700–1708. [173] Kreisselmeier, G., Steinhauser, R. Systematic control [186] Simpson, T. W., Mauery, T. M., Korte, J. J., Mis- design by optimizing a vector performance index. IFAC tree, F. Kriging models for global approximation in Proceedings Volumes, 1979, 12(7): 113–117. simulation-based multidisciplinary design optimization. [174] Poon, N. M. K., Martins, J. R. R. A. An adaptive AIAA Journal, 2001, 39(12): 2233–2241. approach to constraint aggregation using adjoint sensi- tivity analysis. Structural and Multidisciplinary Opti- mization, 2007, 34(1): 61–73. [175] Gao, H., Zhu, J., Zhang, W., Zhou, Y. An improved Renhe Shi received his B.S. degree adaptive constraint aggregation for integrated layout in flight vehicle design engineering and and topology optimization. Computer Methods in Ap- Ph.D. degree in aeronautical and astro- plied Mechanics and Engineering, 2015, 289: 387–408. nautical science and technology from Bei- [176] Zhang, K. S., Han, Z. H., Gao, Z. J., Wang, Y. Con- jing Institute of Technology, China, in straint aggregation for large number of constraints in 2013 and 2019, respectively. He is a wing surrogate-based optimization. Structural and Mul- postdoctoral researcher in the School of tidisciplinary Optimization, 2019, 59(2): 421–438. Aerospace Engineering, Tsinghua Uni- [177] Kennedy, G. J. Strategies for adaptive optimization versity, from 2019 to 2020. Currently, he is an asso- with aggregation constraints using interior-point meth- ciate professor in the School of Aerospace Engineering, Bei- ods. Computers & Structures, 2015, 153: 217–229. jing Institute of Technology. His research interests include [178] Shi, R., Liu, L., Long, T., Liu, J. An efficient ensemble metamodel-based design optimization, multidisciplinary de- of radial basis functions method based on quadratic sign optimization, and aerospace system engineering. E-mail: programming. Engineering Optimization, 2016, 48(7): shirenhe@bit.edu.cn. 1202–1225. [179] Xiong, Y., Guo, L., Tian, D., Zhang, Y., Liu, C. Intelli- gent optimization strategy based on statistical machine learning for spacecraft thermal design. IEEE Access, 2020, 8: 204268–204282. Teng Long received his B.S. degree in [180] Smith, B. G. A., Capon, C. J., Brown, M., Boyce, R. R. flight vehicle propulsion engineering and Ph.D. degree in aeronautical and astro- Ionospheric drag for accelerated deorbit from upper low earth orbit. Acta Astronautica, 2020, 176: 520–530. nautical science and technology from Bei- [181] Wu, Z., Huang, Y., Chen. X., Zhang, X., Yao, W. jing Institute of Technology, China, in Surrogate modeling for liquid-gas interface determina- 2004 and 2009, respectively. He is a pro- fessor and executive dean at the School of tion under microgravity. Acta Astronautica, 2018, 152: 71–77. Aerospace Engineering, Beijing Institute [182] Peng, H., Wang. W. Adaptive surrogate model-based of Technology. His research interests include multidisciplinary fast path planning for spacecraft formation reconfigu- design optimization theory and its applications to flight vehi- cle conceptual design and multi-aircraft collaborative mission ration on libration point orbits. Aerospace Science and Technology, 2016, 54: 151–163. planning and decision-making. Professor Long is a member [183] Feldhacker, J. D., Jones, B. A., Doostan, A., Hampton, of the American Institute of Aeronautics and Astronautics J. Reduced cost mission design using surrogate models. (AIAA) and the American Society of Mechanical Engineers (ASME). E-mail: tenglong@bit.edu.cn. Advances in Space Research, 2016, 57(2): 588–603. Metamodel-based multidisciplinary design optimization methods for aerospace system 215 Nianhui Ye received his B.S. degree in Zhenyu Liu received his B.S. degree in flight vehicle design and engineering from flight vehicle design and engineering from Beijing Institute of Technology, China, in Beijing Institute of Technology, China, in 2019. He is currently pursuing his Ph.D. 2020. He is currently pursuing his M.S. degree at the School of Aerospace and En- degree at the School of Aerospace and gineering, Beijing Institute of Technology. Engineering, Beijing Institute of Tech- His research interests include surrogate- nology, China. His research interests in- assisted evolutionary computation and clude surrogate-assisted multi-objective multidisciplinary modeling of cross-media flight vehicles. optimization methods and flight vehicle multidisciplinary de- E-mail: ynh1996sh@163.com. sign optimization. E-mail: liuzykm@163.com. Yufei Wu received his B.S. degree in Open Access This article is licensed under a Creative flight vehicle design engineering from Bei- Commons Attribution 4.0 International License, which jing Institute of Technology, China, in permits use, sharing, adaptation, distribution and re- 2017. He is currently pursuing his Ph.D. production in any medium or format, as long as you degree in aeronautical and astronautical give appropriate credit to the original author(s) and the science and technology at Beijing Institute source, provide a link to the Creative Commons licence, of Technology. His research interests in- and indicate if changes were made. clude metamodel-based design optimiza- tion theory and its applications to cross-domain morphing The images or other third party material in this article flight vehicle design. E-mail: wuyufei@bit.edu.cn. are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the mate- Zhao Wei received his B.S. degree in rial. If material is not included in the article’s Creative mechanical engineering and automation Commons licence and your intended use is not permitted from Zhengzhou University, China, in by statutory regulation or exceeds the permitted use, you 2016. He is currently pursuing his Ph.D. will need to obtain permission directly from the copyright degree in aeronautical and astronauti- holder. cal science and technology at Beijing In- To view a copy of this licence, visit http://creative- stitute of Technology. His research in- terests include flight vehicle multidisci- commons.org/licenses/by/4.0/. plinary design optimization and mission planning. E-mail: weizhao0123@163.com.

Journal

AstrodynamicsSpringer Journals

Published: Sep 1, 2021

Keywords: aerospace systems design; multidisciplinary design optimization (MDO); metamodel-based design and optimization (MBDO); expensive black-box problems

References