Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Artificial intelligence for structural glass engineering applications — overview, case studies and future potentials

Artificial intelligence for structural glass engineering applications — overview, case studies... Glass Struct. Eng. (2020) 5:247–285 https://doi.org/10.1007/s40940-020-00132-8 SI: CHALLENGING GLASS Artificial intelligence for structural glass engineering applications — overview, case studies and future potentials M. A. Kraus · M. Drass Received: 31 January 2020 / Accepted: 17 August 2020 / Published online: 7 October 2020 © The Author(s) 2020 Abstract ’Big data’ and the use of ’Artificial Intelli- cation and monitoring of façades and glass structures. gence’ (AI) is currently advancing due to the increas- Finally, the current status of research as well as suc- ing and even cheaper data collection and process- cessfully conducted industry projects by the authors are ing capabilities. Social and economical change is pre- presented. The discussion of specific problems ranges dicted by numerous company leaders, politicians and from supervised ML in case of the material parame- researchers. Machine and Deep Learning (ML/DL) are ter identification of polymeric interlayers used in lami- sub-types of AI, which are gaining high interest within nated glass or the prediction of cut-edge strength based the community of data scientists and engineers world- on the process parameters of a glass cutting machine wide. Obviously, this global trend does not stop at struc- and prediction of fracture patterns of tempered glass tural glass engineering, so that, the first part of the to the application of computer vision DL methods to present paper is concerned with introducing the basic image classification of the Pummel test and the use theoretical frame of AI and its sub-classes of ML and of semantic segmentation for the detection of cracks DL while the specific needs and requirements for the at the cut edge of glass. In the summary and conclu- application in a structural engineering context are high- sion section, the main findings for the applicability and lighted. Then this paper explores potential applications impact of AI for the presented structural glass research of AI for different subjects within the design, verifi- and industry problems are compiled. It can be seen that in many cases AI, data, software and computing M. A. Kraus ( ) · M. Drass resources are already available today to successfully M&M Network-Ing UG(haftungsbeschränkt), implement AI projects in the glass industry, which Lennebergstr. 40, 55124 Mainz, Germany is demonstrated by the many current examples men- e-mail: kraus@mm-network-ing.com; makraus@stanford.edu; tioned. Future research directories however will need to kraus@ismd.tu-darmstadt.de concentrate on how to introduce further glass-specific M. Drass theoretical and human expert knowledge in the AI train- e-mail: drass@mm-network-ing.com; ing process on the one hand and on the other hand more drass@ismd.tu-darmstadt.de pronunciation has to be laid on the thorough digitiza- M. A. Kraus tion of workflows associated with the structural glass Civil and Environmental Engineering, Stanford University, problem at hand in order to foster the further use of AI Y2E2, 473 Via Ortega, Stanford, CA 94305, USA within this domain in both research and industry. M. Drass · M. A. Kraus Institute of Structural Mechancis and Design, Technische Universität Darmstadt, Franziska-Braun-Str. 3, 64287 Darmstadt, Germany 123 248 M. A. Kraus, M. Drass Keywords Artificial Intelligence · AI4BI · Façades · et al. 2016), Keras (Chollet et al. 2015) or PyTorch Design, Computation and Monitoring · Structural (Paszke et al. 2019) provide many necessary func- Glass Engineering tionalities for no monetary cost, the authors consider it essential for a successful application of AI in the engineering sciences and especially in structural glass 1 Introduction engineering that only a reasonable combination of the methodological knowledge of AI and the expert knowl- Artificial Intelligence, or AI for short, is probably the edge of the engineer result in meaningful and valuable term that leads to the most animated discussions today tools. Hence, the intention of this article is threefold: in companies of the tech sector, universities and start- it serves as a short introduction on the background and ups, but also in other low-tech companies with a small definitions of AI technology, based not on a data sci- degree of digitization. Due to a progressing digitization ence background, but on the background of engineers of all sectors of industry (Barbosa et al. 2017; Lam- as AI users; illustrative examples from a wide range propoulos et al. 2019; Schober 2020) while costs of data of glass engineering topics elaborate capabilities and processing and storage steadily decrease (Kurt Peker impact of AI methods on the field; highlighting future et al. 2020; Kraus and Drass 2020a), AI is currently potentials of AI for glass engineering gives an outlook paving its way from the subject of academic consider- on trends and gains. Therefore this paper is structured as ations into both private and professional everyday life follows: First, the basic concepts and nomenclature of in a wide variety of forms. Most people in academia AI, Machine Learning (ML) and Deep Learning (DL) and industry who are not familiar with the field of AI are introduced and explained. Based on the theoretical imagine the technology to be similar to popular sci- background, the second part of the paper reviews suc- ence fiction movies like “Terminator”, “Blade Runner”, cessful applications of AI in glass and façades related “Matrix” or “A.I. Artificial Intelligence”. Today how- fields of science and engineering. In the third part, ever AI is present in everyday’s life in less spectacular current and potentially promising future trends for the and humanoid forms such as spam filters, recommender implementation and application of AI in the glass and systems or digital language assistants such as “Alexa” façade sector are presented. The last section presents a (Amazon) or “Siri” (Apple) (Kepuska and Bohouta summary and conclusions from the findings, together 2018). An impression of the effects of AI on engineer- with an outlook on the future of AI in construction and ing contexts can be gained by looking at the develop- related industries. ments and findings concerning the self-propelled car (Badue et al. 2019). There a great number of questions 2 Basics on AI, machine learning and deep have to be addressed on several levels, ranging from learning ethical and legal concerns w.r.t. reliability of AI and consequences of failure (Holstein et al. 2018; Green- This section provides a non-comprehensive introduc- blatt 2016) to very technical concerns such as the for- tion on the topics of artificial intelligence, machine mulation of learning problems or the processing of a learning and deep learning, whereas a theoretically growing amount of collected data (Hars 2015; Daily more substantial and elaborated description of AI and et al. 2017). A lot of similar questions arise in case of its sub-classes can be found in (LeCun et al. 2015; applying AI in a civil engineering contexts with differ- Binkhonain and Zhao 2019; Dhall et al. 2020; Goodfel- ent pronunciation. However, this paper will show that low et al. 2016; Frochte 2019; Wolfgang 2017; Rebala AI offers many new potentials and certain advantages et al. 2019; Chowdhary 2020). Furthermore, (Goulet over existing methods for the use in civil and especially 2020) gives in particular a textbook-like introduction glass engineering, development and practice. to AI topics with a focus on civil engineering. Basically, the disciplines of statistics, numerics and optimization play a major role in understanding the data, describing the properties of a data set, finding rela- tionships and patterns in these data and selecting and applying the appropriate AI model. Although nowa- days AI software libraries such as Tensorflow (Abadi 123 Artificial intelligence for structural glass engineering 249 2.1 Artificial intelligence, algorithms, models and Strong AI, on the other hand, is supposed to act in a sim- data ilar way to a human being. It should be noted, however, that while strong AI can act operatively like a human Looking at Fig. 1, one can see that AI is the umbrella being, it is likely to have a completely different cogni- term for all developments of computer science, which tive architecture compared to the human brain and will is mainly concerned with the automation of intelligent have different evolutionary cognitive stages (Bostrom and emergent behavior (Chowdhary 2020). 2017; Frochte 2019). With strong AI, machines can Thus, AI is a cross-disciplinary field of research actually think and perform tasks independently, just for a number of subsequent developments, algorithms, like humans do. In conclusion, strong AI-controlled forms and measures in which artificially intelligent machines have a “mind of their own” in a certain way. action occurs, which was presented initially at a con- Accordingly, they can make independent decisions and ference at Dartmouth University in 1956 (Brownlee process data, while weak AI-based machines can only 2011; McCarthy et al. 1956; Moor 2006). AI is ded- simulate or mimic human behavior. Today, we are still icated to the theory and development of computational in the age of weak AI, where intelligent behavior aims systems, which are capable of performing tasks which to do a specific task particularly well or even better require human intelligence, such as visual perception, than humans would do. However, there are more and speech recognition, language translation and decision more efforts by tech giants from the Silicon Valley in making (Brownlee 2011). Hence, a number of AI sub- California to create AI systems that not only perform a fields have emerged, such as Machine Learning (Turing specific task, but solve a wider range of problems and 1950), which historically has focused on pattern recog- make generalizations about a specific problem. Since nition (Marr 2016). Parallel, the first concept of a neural the field of strong AI is still in its infancy, only weak network was developed by Marvin Minsky (Russell and AI and its components are described in more detail and Norvig 2020), which paved the way for deep learning. used within this article. Further details on the historic Interestingly, artificial intelligence so far has had to development of AI and its definition in weak and strong overcome several lean periods (“AI winters”) over the from or neat and scruffy philosophy for AI are skipped years (Crevier 1993), but at this very present time seems at this stage with reference to (Goodfellow et al. 2016; promising for a broad breakthrough of the AI technol- Brownlee 2011; Chowdhary 2020). ogy in several branches as digital, computational and monetary resources are in place to provide fertile con- 2.1.1 Problem formulation ditions (Goodfellow et al. 2016; Schober 2020; Kraus and Drass 2020a). Models and algorithms are essential building blocks for From a computer science aspect, it is distinguished the application of AI to practical problems, where an between weak (or narrow) and strong (or general/super) algorithm is defined as a set of unambiguous rules given AI (Russell and Norvig 2020; Goodfellow et al. 2016; to an AI program to help it learn on its own. (Mitchell Frochte 2019). In particular weak AI deals with con- 1997; Frochte 2019) defines a computer program to crete application problems and their solution, for which learn “from experience E with respect to some class kind of “intelligence” is required from the basic under- of tasks T and performance measure P, if its perfor- standing. Commonly known digital assistants such as mance at tasks in T , as measured by P, improves with Siri from Apple (www.apple.com/sir) and Alexa from experience E.” This definition allows for a wide vari- Amazon (Ale 2020) can be framed weak AI as their ety of experiences E, tasks T , and performance mea- operations is limited to a predefined range of function- sures P. However, in the remainder of this paper, intu- alities. Basically, pre-trained models search for patterns itive descriptions and examples (Sect. 3) of different in a recognized audio sample and classify the spoken kind of tasks, performance measures, and experiences words accordingly in both cases. However, the men- are introduced to construct machine and deep learning tioned two intelligent agents only react to stimuli which algorithms. At this point, some more details on task T they were trained on and show some pre-defined reac- and performance P as well as the role of data are given. tion. So far, these kind of programs do not understand Before going into detail on T and P within this sec- or deduce any meaning from what has been said in a tion, we elaborate further on the experience E and the wider sense, which marks the difference to strong AI. role of data for AI, ML and DL. E is an entire dataset 123 250 M. A. Kraus, M. Drass Fig. 1 Schematic sketch of: a the hierarchy of artificial intelligence, machine learning and deep learning and b The use of data and theory in different settings for physics-informed/theory-guided AI D, whose elements are called data points (or examples 2013). According to (Frochte 2019; García et al. 2016) (Goodfellow et al. 2016)). A data point or example five quantities can be used to characterize a dataset: consists at least of features x ∈ R , where a feature – volume: amount of data is an individual measurable property or characteristic – velocity: rate information arrives of a phenomenon being observed (Bishop 2006; Kuhn – variety: formats of data (structured, semi-structured, and Johnson 2013). The concept of a feature is closely or unstructured) related to what is known as “explanatory/independent – veracity: necessity for pre-processing procedure variable” in statistical techniques such as linear regres- – value: relevance of data for task T sion. Furthermore the features of E may be split further to separate targets/labels y among the remaining fea- While the first three aspects “volume, velocity, and tures x, where an AI algorithm is used to uncover rela- variety” refer to the generation of data, capturing and tionships between the remaining features and the tar- storage process, “veracity” and “value” aspects mark get of the dataset in case of more specific tasks such as the quality and usefulness of the data to the task T under supervised ML and many DL problems. As an example, consideration and hence are crucial for an extraction when using an AI algorithm for linear regression, the of useful and valuable knowledge from the data. If all n m task T is to find a function f : R −→ R , the model five concepts of the proposed list are given to a certain y = f (x) assigns an input (feature) vector x to the extend, the definition of “big data” is met, which is of target vector y. Finally, AI models or algorithms may partial interest for this publication as will be explained possess hyperparameters, which are tunable entities of in Sect. 3. From a technical point of view, the term an AI algorithm [such as regularization strength (cf. “big data” (which may be auto-associated with AI by Sect. 2.1.3) or depth of a neural net (cf. Sect. 2.3)] and the reader) refers to large and complex amounts of data have to be investigated during the learning or training which require “intelligent methods” to process them. phase using a learning algorithm to train and evaluate At this point more detail on the variety of data is given. the best model (Raschka 2018). Structured data is information, which has a pre-defined data model (Frochte 2019; O’Leary 2013; Rusu et al. 2013), i.e. the location of each part of the data as well 2.1.2 Data as the content is exactly know. Semi-structured data is a form of structured data To be able to process data in a meaningful way, it that does not conform with the formal structure of data must first be collected and, if necessary, refined or pre- models associated with relational databases or other processed (Frochte 2019; Bishop 2006; Goodfellow forms of data tables, but nonetheless contains markers et al. 2016; Mitchell 1997; Brownlee 2016; O’Leary to separate semantic elements and enforce hierarchies 123 Artificial intelligence for structural glass engineering 251 Fig. 2 Examples for a Structured data : Table (with example features); b Unstructured Data: Picture of Fractured Glass of records and fields within the data (Frochte 2019; structured and unstructured data), depending on the Rusu et al. 2013). specific glass-related problem under consideration. In Finally, unstructured data is information that either Sect. 3, it is elaborated that unstructured data in the form does not have a predefined data model or does not fit of photographic data is used for quality inspection and into relational tables, (Frochte 2019; Rusu et al. 2013). production control, whereas structured data in form of Typical examples of structured data are databases or simulation data from numerical mechanical investiga- tables, while videos or pictures are classic examples of tions or experiments is used to infer about patterns or unstructured data and further illustration of data struc- model parameters by an AI algorithm. It is known from tures are given in Fig. 2. literature that the combination of the given data set and While for structured data, the feature definition is structure together with appropriately selected AI algo- mostly straight forward due to the structure (Turner rithms provides meaningful results (Goodfellow et al. et al. 1999; Brownlee 2016), feature generation (i.e. 2016; Frochte 2019; Bishop 2006). defining features) for unstructured data is essential Simulation data mining is of particular interest for and the process is also known as feature engineering the numerical investigations within structural glass (Ozdemir and Susarla 2018). Per se, the volume of data engineering (Brady and Yellig 2005; Burrows et al. typically can be tackled with state of the art AI algo- 2011; Frochte 2019). Simulation of data in the field rithms, whereas a huge number of features may become of structural glass engineering is on the one hand often problematic (where it is often referred to as curse of expensive as simulations quickly become both theo- dimensionality (Frochte 2019; Bishop 2006) and one of retically and numerically evolved (and thus the whole the main tasks here is to elaborate a discrimination into dataset comprehends of just a few observations). On the relevant and irrelevant features (Goodfellow et al. 2016; other hand (e.g. in the case of a Finite Element simula- Frochte 2019; Bishop 2006; Kuhn and Johnson 2013). tion) the number of features and targets per simulation In order to tackle that issue, dimensionality reduction as example may be great in number. This poses hardware well as feature selection techniques can be performed requirements along with the need for a feature selec- in order to reduce and/or select features according to tion or engineering strategy. Experimental data on the their relevance for describing the task. More details on other hand usually consist of a limited (small) number that will be given in the ML section of this paper or can of observations together with a small amount of fea- be found in (Kuhn and Johnson 2013; Brownlee 2016; tures due to monetary reasons and the design of the Bishop 2006). respective experiments (Kraus 2019). At this stage some final notes on data types encoun- As a conclusion, for the practical application of AI to tered in the field of structural glass engineering are problems in the glass industry the final choice on algo- given. In this specific field, information generally rithms has to be made on a case by case basis depending can be expected to be either way (structured, semi- 123 252 M. A. Kraus, M. Drass on the task T and the volume, variety and veracity of multiple loss function choices may be useful for mon- the data. itoring model performance, but there is no guarantee that they will result in the same set of optimal model parameters. 2.1.3 Model and loss function 2.1.4 Data splitting Atask T is the description of how an AI should pro- cess data points. An example of a task T is to classify After possible cleansing and visualization of the data, images of test specimens into “intact” and “failed”. The different AI models are evaluated. The main objective performance measure P evaluates the abilities of a AI is to obtain a robust AI model with a good ability to algorithm and often P is related to specifics of the AI generalize well the extracted knowledge to data, which task T . To continue the previous classification exam- were not used during training the model by the learn- ple, a possible performance measure P is the accu- ing algorithm (Mitchell 1997; Goodfellow et al. 2016; racy of the classification model, where accuracy is the Bishop 2006). proportion of examples for which the model produces This means that at the end of the training process, the the correct output (Goodfellow et al. 2016; Brownlee final model should correctly predict the training data, 2011; Mitchell 1997). The choice of a proper perfor- while at the same time it should also be able to gener- mance measure is not straightforward and objective but alize well to previously unseen data. Poor generaliza- dependent on the problem at hand and is thus a solid part tion can be characterized by overtraining or overfitting of the model building part. As this paper is concerned (cf. Sect. 2.1.5), which describes the situation that the with ML and DL examples only, the task T involves a model just memorizes the training examples and is not mathematical model M. When expressing the perfor- able to give correct results also for patterns that were mance measure P in mathematical terms together with not in the training dataset (Mitchell 1997; Goodfel- the notion of learning, the AI algorithm will update a low et al. 2016; Bishop 2006; Frochte 2019). These mathematical model such that for given experience E two crucial demands (good prediction on training data better performance P is gained. This gain measured as well as good generalization abilities) are conflict- via P is conducted via (numerical) optimization, thus ing and also known as the Bias and Variance dilemma alternative nomenclature in ML or DL contexts may (Bishop 2006; Frochte 2019). In order to judge how call P the “objective, loss or cost function” C.This well a ML or DL model performs on data, there exist paper adopts the notation of (Goodfellow et al. 2016), several types of methods for evaluation (i.e. validation) where from a mathematical point of view, a “function (Raschka 2018): we want to minimize or maximize is called the objective function, or criterion”. Especially if the mathematical – holdout validation model M of an AI algorithm possesses parameters θ – k-fold cross validation and learns from training on the data set (or short: data) – stratified K-fold cross validation D, this is formulated as the minimization of a cost func- – leave-one-out cross validation (LOOCV) tion C: The simplest method for validation is holdout vali- θ = arg min C (θ |D), (1) opt dation, in which the data set is split into training and where typical examples of cost functions C are testing data over a fixed percentage value (Goodfellow et al. 2016; Frochte 2019). Using the holdout method is – quadratic/L2 loss: ||y − f (x|θ)|| (least squares), perfectly acceptable for model evaluation when work- – Lp loss: ||y − f (x)|| , ing with relatively large sample sizes (Raschka 2018). – entropy loss Nevertheless, it was shown that the three-way hold- – accuracy. out validation in particular offers advantages. In the It is explicitly emphasized that the definition of the three-way holdout method, available training data may loss function is part of the model building process be split such that an additional validation dataset is within the AI algorithm and will influence the training formed (Russell and Norvig 2020; Bishop 2006). To results to great extent (Goodfellow et al. 2016; Bishop be more specific, the three data sets are used as fol- 2006; Frochte 2019). In addition, for the same T task, lows: 123 Artificial intelligence for structural glass engineering 253 categorical value. This is called stratified cross valida- tion. However, if a small dataset with a feature count of less than 100 is owned, it has been shown that LOOCV provides good results for the accuracy and robustness of the AI models. This approach omits one data point from the training data, i.e. if there are n data points in the original sample, then n − 1 samples are used to train the model and p points are used as a validation set. This is repeated for all combinations where the original sample can be separated in this way, and then the error is averaged for all trials to obtain the overall effectiveness. The number of possible combinations is Fig. 3 Example of a 5-fold cross validation equal to the number of data points in the original sample or n and hence might be computationally expensive in the case of a large dataset. – training dataset: used to fit the model M (70% of Finally, if the readers are interested in detailed |D|) description on different techniques for data split- – validation dataset: used to provide an unbiased eval- ting, hyperparameter tuning, model selection and final uation of a model M fit on the training dataset while deployment of machine learning models, (Raschka tuning model parameters (20% of |D|) 2018; Bishop 2006; Reitermanova 2010; Frochte 2019; – testing dataset: used to provide an unbiased evalua- ) provide detailed and compre- Goodfellow et al. 2016 tion of a final model fit on the training dataset (10% hensive reading for that essential subject. of |D|) 2.1.5 Over- and underfitting All data in the three sets should have a similar dis- tribution for the entire set to ensure that the data are Two central challenges in learning an AI model by from the same distribution and are representative. Com- learning algorithms have to be introduced: under- and mon choices for the sizes of the amount of data (here overfitting. |D|≡ N is the number of data points within the whole A model is prone to underfitting if it is not able training set) are given in the bullet point list (Frochte to obtain a sufficiently low loss (error) value on the 2019; Bishop 2006). training set, while overfitting occurs when the train- To tackle the problem of so-called over- and under- ing error is significantly different from the test or vali- fitting (i.e. the poor generalization capability of the AI dation error (Frochte 2019; Bishop 2006; Goodfellow model, cf. Sect. 2.1.5) cross validation (CV) may be et al. 2016). The generalization error typically pos- applied for hyperparameter tuning and model selec- sesses an U-shaped curve as a function of model capac- tion. CV is a validation technique for assessing how ity, which is illustrated in Fig. 4. Choosing a simpler the results of a statistical analysis will generalize to model is more likely to generalize well (having a small an independent data set (Raschka 2018). The k-fold gap between training and test error) while at the same cross validation for example has a single parameter k, time still choosing a sufficiently complex hypothesis which refers to the number of groups into which a given to achieve low training error. Training and test error data sample is divided. As such, the procedure is often typically behave differently during training of an AI referred to as k-fold cross validation, where the k is model by a learning algorithm (Frochte 2019; Bishop replaced with the specific choice to form the concrete 2006; Goodfellow et al. 2016). Having a closer look at name (e.g. k = 10 becomes a 10-fold cross-validation Fig. 4, the left end of the graph unveils that training error as depicted in Fig. 3). and generalization error are both high. Thus, this marks In contrast, splitting the data into folds can be con- the underfitting regime. Increasing the model capacity, trolled by criteria such as ensuring that each fold con- it drives the training error to decreases while the gap tains the same proportion of observations with a certain between training and validation error increases. Further 123 254 M. A. Kraus, M. Drass increasing the capacity above the optimal will eventu- 2.2 Machine learning ally lead the size of this gap to outweigh the decrease in training error, which marks the overfitting regime. Machine Learning is a sub-branch of AI, which is con- Increasing model capacity tackles underfitting while cerned with algorithms for automating the solution of overfitting may be handled with regularization tech- complex learning problems that are hard to program niques (Frochte 2019; Bishop 2006; Goodfellow et al. explicitly using conventional methods. ML algorithms 2016; Kuhn and Johnson 2013). Model capacity can be build a mathematical model M to infer between quanti- steered by choosing a hypothesis space, which is the set ties of interest (features; targets) based on data to make of functions that the learning algorithm is allowed to predictions or decisions without being explicitly pro- select as being the solution (Goodfellow et al. 2016). grammedtodoso(Frochte 2019; Rebala et al. 2019; Here, varying the parameters of that function family Chowdhary 2020; Murphy 2012). This section provides is called representational capacity while the effective a brief introduction to the most general principles and capacity takes also into account additional limitations nomenclature, a more thorough introduction and elab- such as optimization problems etc. (Goodfellow et al. oration on the subject is given in (Bishop 2006; Good- 2016). fellow et al. 2016; Mitchell 1997; Rebala et al. 2019; Murphy 2012). A basic premise, however, is that the knowledge gained from the data can be generalized 2.1.6 Trends for AI in the engineering and natural and used for new problem solutions, for the analysis of sciences previously unknown data or for predictions on data not measured (prediction). As elaborated in the previous Recent developments in the field AI related to natural as section on AI, ML also has strong ties to optimiza- well as engineering sciences formed the terms physics- tion as learning problems are typically formulated as informed/theory-guided AI, which is a field, where the minimization of some loss function on a training set authors of this paper are also active in, cf. Fig. 1b. The of examples (Frochte 2019; Bishop 2006; Goodfellow aim here is to achieve two goals: et al. 2016; Murphy 2012). Furthermore ML (as well as DL) are closely related to statistics in terms of methods – Compensate data sparsity. but differ in their goal of drawing population inferences – Utilize available theoretical knowledge in a formal from a sample (statistics) vs. finding generalization pre- way. dictive patterns (Bzdok et al. 2018). Training AI models with few data are at the center Two different main algorithm types can be distin- of knowledge inference in the natural and engineering guished for ML: supervised and unsupervised learn- sciences, in contrast to the typical structure of AI in eco- ing (Mitchell 1997; Bishop 2006; Goodfellow et al. nomics or computer science, where very large amounts 2016; Frochte 2019), which are briefly introduced here of data are available for the problem under consider- and graphically illustrated in Fig. 5 (Deep Learning is ation. The reasons for the sparsity of experimental or treated in the next subsection of this paper and Rein- computational data may result on the one hand from forcement Learning is omitted within this paper at all). the fact that they are expensive or the gathering of a In ML, there is a data set D = (x , t ) with N n n n=1 great amount of those data is prohibitive due to time observations, where x is the feature/influence variable or financial constraints. On the other hand, the formal and t the target/response variable. Both variables can introduction and use of pre-existing and already exist- be continuous or discrete (categorical). While in super- ing theoretical knowledge (both from science and from vised learning a predictive model M based on both experts), e.g. in the form of the loss function (Raissi influence and response variables is to be developed, in 2018), leads to shorter familiarization times through unsupervised learning a model is learned only on the meaningful previous starting points for optimization basis of the features (clustering; dimension reduction). within the AI algorithms or the setting of boundary con- For supervised learning, a distinction is made between ditions to the parameters to be derived. Further informa- classification and regression problems. While in the tion can be found in (Reichstein et al. 2019; Karpatne former case the response variables t can only take dis- et al. 2017; Wagner and Rondinelli 2016; Raissi 2018; crete values, the response variables t are continuous Kraus 2019). for regression problems. 123 Artificial intelligence for structural glass engineering 255 Fig. 4 Typical relationship between capacity and error, marking underfitting zone (left) and overfitting zone (right), from (Goodfellow et al. 2016) Fig. 5 Overview on the ML techniques 123 256 M. A. Kraus, M. Drass The goal of regression is to predict the value of one fore may not perform well on new data (Murphy 2012; or more continuous target variables t given the value Bishop 2006; Goulet 2020). Further details on typical of a vector x of input variables, whereas the goal in algorithms such as “Principal Components Analysis classification is to take an input vector x and to assign (PCA)”, “Manifold Learning” or “Autoencoders” are it to one of K discrete classes C where k = 1,..., K skipped within this article with referencing the reader (Bishop 2006). A more detailed description of super- to (Murphy 2012; Bishop 2006; Goulet 2020; Witten vised ML models such as linear and non-linear regres- et al. 2016; Frochte 2019). sion or generalized linear model regression along with A generally valid scheme of steps involved in a suc- classification is omitted within this paper with refer- cessful ML project is presented in Fig. 6. ence to already mentioned ML textbooks. By using Different aspects of Fig. 6 are discussed at this point, regression or classification models, it is furthermore as the conduction of every step is essential for building possible to catch nonlinear and more complex depen- a sensible AI/ML application. In step 1 and 2 existing dencies between the in- and outputs. For further infor- data are compiled and brought in a form that AI/ML mationitisreferredto(Kraus 2019; Bishop 2006; model can access it and the learning algorithm is able to Goodfellow et al. 2016; Mitchell 1997; Lee et al. 2018; train the model on the present data. This step may take Murphy 2012). minutes to months in dependence of the problem and In Fig. 5 on the right hand side, the main categories the data structure of the respective environment. Espe- of unsupervised learning algorithms are given. These cially when digitizing existing older data from paper. algorithms use input data only to discover structure, It is advisable to consider standardization protocols for patterns and groups of similar examples within the data this step in order to guarantee data consistency within (clustering), or to determine the distribution of data a company. It is however important to note that the pre- within the input space (density estimation), or to project dictive power and accuracy of any data-driven model the data from a high-dimensional space down to lower is based on the accuracy and quality of the input data. dimensions (Goulet 2020; Kraus 2019; Bishop 2006; In the context of this paper, step 3 (feature extrac- Goodfellow et al. 2016; Mitchell 1997; Lee et al. 2018). tion) will be briefly discussed, since this represents the Cluster algorithms use similarity or distance measures interface between AI/ML and engineering on the one between examples in the feature space as loss functions hand, and on the other hand it has a significant influ- to discover dense regions of observations (Hastie et al. ence on the quality of the model’s statements and pre- 2009). Clustering algorithms in contrast to supervised dictions. Different strategies for deriving features exist: learning only use a divide-and conquer strategy to inter- Historically, ML uses statistical features obtained pret the input data and find natural groups or clusters by unsupervised learning methods (e.g. cluster analy- in feature space, where a single cluster is an area of sis, dimensionality reduction, autoencoders, etc.), but density in the feature space where data are closer to the as in the context of glass engineering thermomechani- cluster than other clusters (Witten et al. 2016; Bishop cal as well as chemical models exist, the parameters of 2006; Goulet 2020). Typical clustering algorithms are those equations may also serve as features. The num- “k-means” (Lloyd 1982; Goulet 2020; Bishop 2006) ber of features that can be derived from the data is and the “mixture of Gaussians” (Goulet 2020; Bishop theoretically unlimited, but some techniques are often 2006). Similar to clustering methods, dimensionality used for different types of data. For example, the task reduction aims to exploit inherent (latent (Bishop 2006; of feature selection is to extract certain signal prop- Goodfellow et al. 2016; Lee et al. 2018)) structure in erties from, for example, raw sensor data to generate the data in an unsupervised manner to reduce the num- higher-level information. Feature extraction techniques ber of features to a set of principal variables, where in this context are the detection of peaks, the extraction the approaches can be divided into feature selection of frequency contents by Fourier transform, the iden- and feature extraction (Roweis and Saul 2000; Bishop tification of signal trends by sum statistics (mean and 2006). Fewer input dimensions (i.e. number of features) standard deviation at different experimental times), etc. induce fewer parameters or a simpler structure in the Further details on the individual steps can be found in ML model, referred to as degrees of freedom (Mur- (Bishop 2006; Goodfellow et al. 2016; Chang and Bai phy 2012). A model with too many degrees of free- 2018; Kraus 2019; Tandia et al. 2019; MAT 2016c, dom is likely to overfit the training dataset and there- 2016a,2016b). 123 Artificial intelligence for structural glass engineering 257 Fig. 6 Flowchart for the learning process with AI/ML Fig. 7 Schematic sketch showing the principle architectures of: a Feedforward Neural Network (FNN); b Convolutional Neural Network (CNN) 2.3 Deep learning and Yu 2016; Rudy et al. 2019; Baumeister et al. 2018; Mosavi 2019), pattern recognition of radar systems Deep learning is sub-field of ML (Goodfellow et al. (Chen and Wang 2014), face recognition (Hu et al. 2016), which uses so-called artificial neural networks 2015; Sun et al. 2018; Li and Deng 2020), signal clas- as models to recognize patterns and highly non-linear sification (Kumar et al. 2016; Fawaz et al. 2019), 3D relationships in data. An artificial neural network (NN) reconstruction (Riegler et al. 2017), object recognition is based on a collection of connected nodes (the neu- (Rani et al. 2020; Zhao et al. 2019), sequence recog- ron), which resemble the human brain (cf. Fig. 7). nition for gesture (Elboushaki et al. 2020; Gao et al. Today many of architectures of neural nets are known 2020), speech (Yu and Deng 2016; Nassif et al. 2019), (Van Veen 2016), however in the context of this paper handwriting and text (Zheng et al. 2015; Jaramillo et al. only the specific sub-classes of feedforward neural nets 2018), medical diagnostics (Bejnordi et al. 2017; Ker (FNN) and convolutional neural nets (CNN) are of et al. 2017; Greenspan et al. 2016; Liu et al. 2019) and interest, cf. Fig. 7. Details on the specifics of the var- e-mail spam filtering (Guzella and Caminhas 2009). ious other types of NN can be found for example in The FNN is constructed by connecting layers con- (LeCun et al. 2015; Goodfellow et al. 2016). Due to sisting of several neurons, a schematic sketch is shown th their ability to reproduce and model non-linear pro- in Fig. 7. The first layer (0 ) of the FNN is the input N th cesses, artificial neural networks have found applica- layer of dimension R , the last layer (L )isthe output tions in many areas. These include material modeling layer, and the layers in between are called hidden lay- th K and development (Bhowmik et al. 2019; Goh et al. ers (l ). A neuron is an operator that maps R −→ R 2017; Mauro 2018; Mauro et al. 2016; Elton et al. (with K connections to neurons from the previous layer 2019), system identification and control (De la Rosa l − 1) and described by the equation: 123 258 M. A. Kraus, M. Drass ⎛ ⎞ l−1 on details on convolution operations and several pool- l l l−1 l l ⎝ ⎠ ˆ b = σ w b + b := σ b (2) ing strategies along with training approaches for the k kj k k j =1 different kinds of NN, instead the reader is referred to (Bishop 2006; Frochte 2019; Goodfellow et al. 2016). where σ(·) is a monotone continuous function and com- Further well-known NN are recurrent neural networks monly referred to as activation function. The activation (RNNs) for processing sequential data (Graves 2012; is computed as a linear combination of the neurons in Goodfellow et al. 2016), autoencoder for dimensional- the previous layer l −1 given the corresponding weights ity reduction or feature learning (Skansi 2018; Good- l l w and biases b of layer l,cf. Eq.(2) and Fig. 7.The kj k fellow et al. 2016) and many more, which are not sub- choice of connecting the neurons layer wise is user ject of this paper. DL is a supervised learning strategy dependent, if each neuron is connected to every neuron and may need a great amount of data for meaningful in the two neighbor layers, the FNN is called dense or training, depending on the specifics of the problem at densely connected. In summary, FNN represent a spe- hand (Bishop 2006; Frochte 2019; Goodfellow et al. cific family of parameterized maps (depicted by ◦ for 2016). This situation then may prohibit the use of DL the composition operation), which are surjective if the for some applications in research and practice. In sum- output layer possesses linear activation function and mary, model capacity in case of NN is greater com- can be expressed as: pared to ML models in the sense, that the NN as func- tion space allows for more variety than typical function y = f ◦ ... ◦ f (x) (3) n 1 spaces used in ML models. Thus all points raised in Sects. 2.1.3 to 2.1.5 require special considerations in l l−1 l where f = σ w b + b (tensor notation) repre- the DL setting and hyperparameter tuning along with sents the data transform in one layer l. A neuron is validation issues are essential for generalization NN a non-linear, parameterized function of the input vari- models for successful application in the engineering ables (input neurons; green in Fig. 7). A NN is hence context. a mathematical composition of non-linear functions of two or more neurons via an activation function. This particular non-linear nature of NNs thus is able to iden- 3 AI applied to structural glass and related fields tify and model non-linear behaviors, which may not at all or not properly be captured by other ML meth- So far, an introduction on the basics and background on ods such as regression techniques or PCA etc. Despite AI, ML and DL was given and some concepts for model the biological inspiration of the term neural network building, training and validation were introduced. AI is a NN in ML is a pure mathematical construct which a fast-growing technology that has now entered almost consists of either feed forward or feedback networks every industry worldwide and is expected to revolu- (recurrent). If there are more than three hidden layers, tionize not only industry, but also other social, legal this NN is called a deep NN. The development of the and medical disciplines. Specifically the construction right architecture for an NN or Deep NN is problem industry possesses the lowest rate of digitization (Chui dependent and only few rules of thumb exist for that et al. 2018; Schober 2020). Here new technologies are setup (Bishop 2006; Frochte 2019; Kim 2017; Paluszek introduced hesitantly due to the long lifespan of build- and Thomas 2016). Convolutional (neural) networks ing structures and associated reservations or concerns (LeCun et al. 1995; Goodfellow et al. 2016) (CNN) about the risks and reliability of new methods and prod- mark a specialized kind of NN for processing data with ucts due to the lack of experience. However, consider- grid-like topology. Examples include time-series data ing that about 7 % of all employees worldwide work (1-D grid taking samples at regular time intervals) and in the construction sector, there is a considerable mar- image data (2-D grid of pixels). In contrast to FNN, the ket potential in the development and transfer of new CNN employ the mathematical operation called convo- approaches from AI to this sector (Schober 2020). lution, which is a special kind of general matrix multi- Focusing now on structural glass and façade con- plication in at least one of their layers. In addition to the struction within the whole building industry, this convolution, a pooling operation is applied to the data branch is, in contrast to more established branches such between layers. This paper will not further elaborate as concrete or steel construction or bridge design, rela- 123 Artificial intelligence for structural glass engineering 259 tively progressive, innovative and open to technology. 3.1 AI for engineering user-centered adaptive façades This can be demonstrated by numerous projects in the field of façade constructions, such as the use of adaptive The topic of the building envelope or façade has gained façade elements in the building envelope (Shahin 2019; enormous importance in recent years due to the discus- Romano et al. 2018), switchable glass as sun protection sion on sustainability and energy saving (Aznar et al. (Marchwinski ´ 2014; Vergauwen et al. 2013), numerical 2018), where lately the consideration of the user health, modeling of complex adhesively bonded façade ele- well-being, productivity and interaction with the build- ments (Drass and Kraus 2020a), the consideration of ing/façade was added (Luna-Navarro et al. 2020). The time and temperature dependent material behavior of building envelope on the one hand side determines polymeric interlayers in laminated glasses in the intact design and perception of the building for both users and and post-failure state (Kraus 2019), or the paramet- the public while on the other hand, the building enve- ric design of building envelopes (Wang and Li 2010; lope is a significant structural sub-system for occupant Zhang et al. 2019; Granadeiro et al. 2013; Vergauwen comfort and interaction of the user with the environ- et al. 2013). Within this section, the focus is on struc- ment of the building. Interaction of the user with the tural glass and façade construction within the build- envelope so far was either mainly driven by manual and ing industry. In the remainder of this paper different local personal control (e.g. through opening a window areas of interest for the application of AI are identified, or drawing a curtain) or semi-automated by triggered potentials and possibilities are uncovered and outlooks predefined sequences leading to actions (e.g. switch- to future visions are highlighted. In order to charac- able and smart glazing, dynamic shading) (Day and terize the special flavor of the needs and potentials of O’Brien 2017). This led to the situation of occupants applying AI to this specific field of design, engineering often being dissatisfied even in the scenario with con- and products, the authors created the term “Artificial trol strategies and related interactions with automated Intelligence for the Building Industry (AI4BI)”. systems (Luna-Navarro and Overend 2018; Fabi et al. AI has yet been applied in engineering (Quantrille 2017; Borgstein et al. 2018; Day and O’Brien 2017; and Liu 2012; Patil 2016; Bunker 2018), economy (Var- Bluyssen et al. 2013; Meerbeek et al. 2014). Automa- ian 2018), medicine (Szolovits 2019) and other sectors tion of the building in a combination with AI is a for modeling, identification, optimization, prediction promising solution for low-energy buildings through and control of complex systems and/or components a data-driven yet occupant-informed approach consist- thereof (De la Rosa and Yu 2016; Rudy et al. 2019; ing of actuation systems and ubiquitous sensing devices Baumeister et al. 2018; Mosavi 2019). Some review steered by learning AI algorithms. Concepts so far are articles compile the state of the art of AI in civil engi- concerned with a proper design (structural, service- neering as a whole discipline (Huang et al. 2019; Patil ability, sustainability, user well-being) and adaption of et al. 2017; Lu et al. 2012; Adeli 2001) while a huge façades but dismissed the aspects of health monitoring number of publications deal with specific problems as well as structural design requirements for adaptive from the civil engineering field (which in part were structures over the lifespan of the envelope (cf. Fig. 2 already cited so far in this paper), which are not given in (Aelenei et al. 2016) on the characterization con- explicitly here in order due to reasons of brevity. How- cepts of adaptive façades, where structural aspects are ever, especially for the structural glass engineering con- assumed to be per-fullfilled), which are introduced and text a review paper has not yet been published, which discussed within this section. is partially the intention of this contribution. Table 1 Within this article, three major points for the appli- gives an overview of present examples on applying AI cation of AI in the façade engineering context are high- in structural glass engineering as discussed and firstly lighted: presented in this paper. – multi {physics; user} constrained design by/through In the remaining section, the examples will always AI be elaborated according to the scheme of describing the – data-driven {structural adaptivity; health monitor- problem, explaining the traditional engineering solu- ing; predictive maintenance} tions, elaborating new possibilities and added value – intelligent functional {façade; home; office build- due to using AI and judging challenges and difficul- ing} ties related to this approach. 123 260 M. A. Kraus, M. Drass Table 1 Overview and summary table of the examples of this paper on the application of AI in structural glass engineering and related disciplines Used amount of training data: + small; ++ medium; +++ large AI applicability: 0 not shown yet;  success proved Artificial intelligence for structural glass engineering 261 3.1.1 Driving the multi {physics; user; verification tural verification software, etc.) within the design, plan- code} constraints for AI in the façade ning and verification process in civil engineering and the heterogeneity of associated partners (usually small Basic design principles for civil engineering struc- companies with no formal protocol on a digital work- tures enforce very stringent safety and serviceability flow) in a design and construction project. Furthermore criteria which assume extreme loading and resistance due to keeping competitive advantages many compa- situations, which occur with very low probabilities, nies do not want to share or make publicly accessible hence these structures are over-designed for most of technical solutions in form of a database. their service lives (Akadiri et al. 2012; Senatore et al. 2018). The structural adaption philosophy on the other 3.1.2 Steps towards AI in façade engineering hand reduces material and energy consumption of the building construction through a paradigm of control- A fully digital workflow upon the Building Information ling strength and stiffness in real-time via sensing and Modeling (BIM) (Borrmann et al. 2015; Isikdag 2015) actuation to carry the acting loads (Wada 1989; dos approach for the whole life cycle of a building solves Santos et al. 2015; Wagg et al. 2008; Fischer et al. the digitization problem and allows AI algorithms to be 2009). Over the last couple of years several adaptive applied in several forms. BIM is a digital description façade systems were researched. On the one hand side of every aspect of a construction project and nowadays a “structure focused” branch considered either shap- practiced to some extend in the construction industry. ing façade elements (e.g. thin glass) (Amarante dos The initial idea of BIM is a 3D information model Santos et al. 2020; Silveira et al. 2018; Louter et al. formed from both graphical and non-graphical data, 2018) or enabling rigid façade-components to be adap- which are compiled in a shared digital space called tive (Schleicher et al. 2011; Svetozarevic et al. 2019) (Common Data Environment; CDE). All information while diminishing sustainability and comfort. On the on that specific building is constantly updated as time other hand a “sustainability and user-centered” branch progresses during the life cycle of a building and thus considered strategies for either predefined levels of sus- ensures the model to always be up-to-date (Serrano tainability, energy saving and user comfort by design or 2019). However, BIM today still suffers from techni- allowed for user-control strategies to address occupant- cal challenges across disciplines such as architectural building-interaction in addition to sustainability con- design, structural verification, building physics design, cerns. Taking into account the statements of this and maintenance measurements etc. and a full digital work- the preceding paragraph leads to the conclusion, that flow with AI components from the early first sketches adaptive façades have to be modeled as a multi-criteria to demolition of a building (Ghaffarianhoseini et al. optimization problem with highly nonlinear and impre- 2017; Vass and Gustavsson 2017; Akponeware and cise (in the fuzzy sense; for user/occupant modeling) Adamu 2017) is not yet possible. For the application correlations, which a priori may not be known to a cer- of AI in that multi-criteria optimization and control tain extend (especially the user well-being part of the problem as described earlier in this section, there is equation) or have to be “learned” from data of experi- need of a cyber-physical twin (digital twin, computable ments (e.g. multi-occupant requirements; features from structural model) (Boschert and Rosen 2016; Borrmann multi-sensor measurements). et al. 2015; Raj and Evangeline 2020) within the life For the design stage AI and ML can be used to infer cycle-accompanying BIM paradigm. The digital twin suitable technical solutions to given tasks under consid- is a digital image of a physical system which is heavily eration in an early design stage (AI assisted design and used in industry so far to reduce operational errors and planning) with a potential check for planning errors or optimize product design. The starting point for a digital unlikely verification success of the designed solution. twin is a highly accurate three-dimensional model that The main problem for an immediate introduction and contains all the features and functions of the physical application of AI here is the low digitization rate (Chui system, including material, sensor technology or even et al. 2018; Schober 2020) (especially details for older dynamics of the real structure. The parametric design buildings are highly likely documented on paper rather approach in architecture (Monedero 2000; Wortmann than in a digital ML readable format), the high varia- and Tunçer 2017; Oxman 2017) is a first step in these tion of data formats (CAD formats, formats of struc- directions and seems very suitable for a connection to 123 262 M. A. Kraus, M. Drass Fig. 8 Schematic overview on an intelligent façade with health monitoring capabilities AI as it currently uses optimization algorithms, e.g. with both systems may be prohibitive due to monetary genetic algorithms etc. which are are sub-groups of AI. reasons. However, in buildings to be designed and con- Due to the fact that structural verification is solidly structed from scratch, an integrated approach imple- based on mechanics and theory, the application of AI menting the two functionalities can be considered. In in the verification stage during design of structures is the remainder of this subsection, some background and very likely to be successful as through mechanics it is potential realization outlooks are given. guaranteed to hit a certain solution manifolds of the Both mentioned ideas are rooted in the data-driven problems which itself induces manifolds of feasible approach to identification, control and steering of struc- design solution (which is in contrast to the view of tural systems. There, ML is a rapidly developing field data analysis, where there is a priori no knowledge that is transforming the ability to describe complex sys- of the process by which data is created). First expe- tems from observational data rather than first-principle riences with automating design reviews with AI in a modeling. While for a long time, ML methods were BIM context is delivered by (Sacks et al. 2019), where restricted to the application to static data, more recent building models are checked for conformance to code research concentrates on using ML to characterize clauses of simple form (explicit formulations; implicit dynamical systems, (Brunton and Kutz 2019). Espe- and complex clauses are still beyond the scope of such cially the use of ML to learn a control function, i.e. to applications). determine an effective map from sensor outputs to actu- The combination of health monitoring/predictive ation inputs is most recent. In this context, ML methods maintenance and an intelligent façade/home/building for control include adaptive NN, genetic algorithms and are schematically visualized in Fig. 8 and can be imple- programming and reinforcement learning. mented within one and the same façade project. The The second mentioned issue for an intelligent façade reason for distinguishing these two situations is due to or building is similarly treatable form a mathematical the fact, that the deployment is in dependence of the and AI point of view (Aznar et al. 2018; Luna-Navarro needs of the building owner or user (cf. comments on et al. 2020). Similar ML methods apply in this con- multi-criteria optimization problem earlier in this para- text as well. The overall idea is that given a reasonable graph) and both systems work individually on a partly and suitable loss function, i.e. a function, which is able shared data basis (cf. comments on BIM and the dig- to correctly describe the well-being and comfort of a ital twin earlier in this paragraph). Especially for the user, the façade or building is able to learn the specific situation with existing façade structures the retrofitting domain of comfort for the individual user by training an 123 Artificial intelligence for structural glass engineering 263 AI algorithm for a reference state and continuous user take and when. The structural health monitoring (SHM) feed back about the well-being. Through that approach, thus predicts the future performance of a component it will be possible to provide maximum user comfort by assessing the extent of deviation or degradation of a with minimal invasiveness. system from its expected normal operating conditions If supervised ML or DL algorithms are applied, (NOC) (Brunton and Kutz 2019). The inference of the a loss function (characterizing the control and steer- NOC is based on the analysis of failure modes, detec- ing problem) has to be established in the mentioned tion of early signs of wear and aging and fault condi- contexts. In addition the development of suitable and tions. This is the bottleneck of the AI approach to façade meaningful features, which allow a structurally sen- monitoring, as it is necessary to have initial information sible and unambiguous classification of the condition on the possible failures (including site, mode, cause, of the façade under consideration, is necessary (Aznar and mechanism), which for new façade systems can et al. 2018; Luna-Navarro et al. 2020). For example, the only be learned “on the fly” after installation of the construction-physical principles and interrelationships façade. However, with a data-driven approach, a certain for describing the comfort of the user as a result of exter- initial training phase (e.g. 5 years) can be implemented nal influences and their manipulation, e.g. by control- as a training and identification period for the AI to learn ling the light-directing or heating systems, are already the NOC and to detect derivations of it (Fig. 9). known today in theory, but to date these have not been Concluding this paragraph, AI together with a BIM- taken into account in any approach to the “smart home”. embedded digital twin has the potential to enhance the This is particularly due to the fact that (analogous to built environment with occupant interaction to form the “Internet of Things”) mechanical engineering in sustainable intelligent buildings and façades and hence particular has so far been concerned with the network- deliver satisfying human-centered environments. How- ing of machines and devices without incorporating the ever, more research is needed to build the multi-criteria knowledge specific to civil engineering with regard to loss function for the AI control system via a holistic and the interaction of people and buildings. An AI can be multidisciplinary approach. extended here by building physics criteria and evalu- ate the user data in such a way that a building (living space/work use) learns the preferences of the respective 3.2 AI in glass product development, production and user over time via the diverse data streams and adapts to processing the user. This idea goes far beyond the currently exist- ing approaches of “smart home”, so that a conceptual Glass and façade construction is highly technological in delimitation of the “intelligent home/office” becomes the area of industrial development, production and fur- obvious. ther processing of glass, as glass is a brittle material and Similarly, structural features, such as deflections or inferior quality in processing can lead to fatal events in accelerations, may serve as sensible features together their assembly, construction and/or operation (Schnei- with some signal statistic features to describe well the der et al. 2016; Sedlacek 1999; Siebert and Maniatis structural behavior of a monitored façade under con- 2012). Consequently, high-precision machines are used trol of an AI. In the health monitoring situation, addi- for glass production and processing to enhance the brit- tional information has to be given to the AI algorithm tle material in such a way that it exhibits high qualities. in order to enable it to predict the remaining lifetime Starting from washing the glass, cutting and breaking, or inspection intervals, which is known in mechani- thermal/chemical tempering and lamination to form a cal engineering as predictive/prescriptive maintenance laminated (safety) glass (Schneider et al. 2016; Sed- (Brunton and Kutz 2019). In order to make the nomen- lacek 1999; Siebert and Maniatis 2012). The machine clature clear, predictive maintenance employs sensors technology for glass refinement is constantly being data to precisely collect data describing the conditions improved and optimized to meet customer-specific of an asset and overall operational state. The data are requirements. Currently available established methods then analyzed for prediction of future failure events and either fail or are worse in comparison to AI technolo- their occurrence times. Prescriptive maintenance takes gies, which can be integrated here. The following exam- this analysis to a further state of maturity as it not only ples elaborate in more detail the use of AI for faster and predicts failure events but also recommends actions to more systematic improvements in production and man- 123 264 M. A. Kraus, M. Drass Fig. 9 Online Fault Diagnosis System, from (Niu 2017) ufacturing of glass products. Within this section, four putations for technical products and an estimation of different applications of AI for production and quality the properties by these methods still are prohibitively management of glass are highlighted: expensive and time consuming. From a mathematical point of view, composition of the design of new glasses – Glass Product Development can be seen as a multi-objective optimization problem – Inspection and Control of Laminated Glass with many constraints, which can be easily handled by – Semantic Segmentation of Cut Glass Edges an ML approach. (Hill et al. 2016) – Strength Prediction based on Cutting Process Param- Having at hand significant computing capabilities, eters data-mining algorithms, an efficient data storage infras- tructures and an (publicly) available materials database 3.2.1 AI for data-driven glass product development enables researchers to discover new functional mate- rials by AI within significantly lower temporal and Today, there is increasing demand on highly-functional, monetary efforts than in conventional processes. The manufacturable and inexpensive glasses (Tandia et al. development of new products by data-driven AI meth- 2019), which has led glass researchers to use data- ods relies on the establishing or existence of accessible driven machine learning models to accelerate the devel- databases, which in practice hardly exist for the public opment of glasses and glass products instead of tradi- but do on the level of individual companies or research tional trial-and-error approaches. In this context, data- groups. A very mature compilation of publicly avail- driven materials discovery approaches use statistical able materials databases for model and glass product models as well as ML algorithms, which are trained, development is given in (Tandia et al. 2019). tested, and validated using materials databases. An An example for the data-driven development of a important part of this approach is to develop or access new type of glass is shown in the following, which accurate materials databases at low cost. While it is was presented by (Tandia et al. 2019). In that exam- in principle possible to use first principles approaches ple, the two most important properties for glass design (thermochemical/thermodynamical simulations such are liquidus temperature T and viscosity η, where the as ab initio calculations based on quantum mechanics, glass liquidus temperature is defined as the temper- density functional theory, molecular dynamics, or lat- ature at which the first crystalline phase precipitates tice models etc. (Van Ginhoven et al. 2005; Benoit et al. from the melt of a given glass composition when the 2000)) to compute electronic band structure, formation melt is cooled with very small rate and the viscosity is energy and other thermodynamic parameters, the com- 123 Artificial intelligence for structural glass engineering 265 Fig. 10 a Grey box fitting of temperature-dependent MYEGA perature dependent viscosity with NNs using BO to find the best viscosity with NN using a single layer with eight neurons and architecture to code the MYEGA equation. Both from (Tandia tanh as an activation function on a single layer; b fitting of tem- et al. 2019) relevant for the targeted sheet thickness in the produc- It was found, that the combined NN-MYEGA equa- tion phase. Still today no accurate and generalizable tion approach resulted in a sufficiently accurate pre- physics-based models for glass melt liquidus tempera- diction model with low error in the validation set com- ture or melt viscosity for industrial glasses is known, pared to other models and thus the development of new thus the application of DL is a viable strategy for the glass compositions was possible within significantly development of a predictive model for both liquidus lower time at less money. Further details on this spe- temperature and viscosity. cific example can be found in (Tandia et al. 2019) while Among other ML techniques, a NN is trained for (Mauro et al. 2016; Mauro 2018) provide further appli- the prediction of both parameters, which is presented cation cases of AI for glass material development. in more detail within the context of this paper. The NN as well as the predictive capabilities are shown in 3.2.2 AI for inspection and control of glass products Fig. 10. Due to reasons of brevity, only the AI mod- eling of the viscosity η is described in detail, as this Building products and pre-fabricated building compo- quantity drives the glass thickness within the produc- nents currently have to full fill certain national and tion process. In the approach of (Tandia et al. 2019), the international standards to ensure a minimum level MYEGA model was used in combination with a NN. of reliability and uniformity of these products across The MYEGA model possesses the form: nations (Schneider et al. 2016; Siebert and Maniatis 2012). New production technologies such as additive manufacturing together with new strategies for achiev- B C log η = A + exp , (4) ing the requirements of building regulations demand T T automation of material quality testing with little human intervention to ensure repeatability and objectivity of in which A is negative while B and C are positive con- the testing process. In the status quo of quality control stants. A Bayesian optimization (BO) framework was of glass and glass products, visual inspections are often used for inference of the model parameters (number of carried out by humans to evaluate, for example, the layers, number of neurons in each layer, learning rate, cleanness of the glass, the quality of cut edges (Bukieda activation function etc.). et al. 2020), anisotropy effects caused by thermal tem- 123 266 M. A. Kraus, M. Drass pering of glass (Illguth et al. 2015) or to determine the mer (pummel). The Pummel value is then estimated by degree of adhesion between interlayer and glass (Franz a human inspector based on the surface area of poly- 2015). In these existing testing protocols, the assess- mer interlayer exposed after pummeling (cf. Fig. 11- ment and judgment of a human tester is required to left). Further details on the Pummel test can be found quantify the degree of reaching requirements of build- in (Schneider et al. 2016; Beckmann and Knackstedt ing regulations, hence the human quality quantifica- 1979; Division 2014). tion results are prone to non-negligible statistical uncer- Traditional image-based computer vision methods tainty through the human tester decisions, (Wilber and for evaluating the Pummel test extract image features Writer 2002). Applying AI in the field of production using complex image pre-processing techniques, which control of glass and glass products hence seems promis- in the experience of the authors based on conducted ing for reaching above-human level accuracy in qual- investigations on Pummel test pictures so far marked ity inspection based on objectification, systematization the main difficulties with these approaches. On the one and automation. This approach was already proved suc- hand, the proper choice of a performance metric on the cessful in related scientific fields involving AI and espe- pummel images (e.g. certain quantiles of the cumula- cially DL for computer vision (i.e. how computers can tive distribution function of grey-values or full color gain high-level understanding from digital images or spaces of the images), which is invariant under the videos), where it clearly outperformed humans in sev- widely varying real-world situations for taking such eral areas (Voulodimos et al. 2018; Ferreyra-Ramirez a Pummel image with thin cracks, rough surface, shad- et al. 2019). ows, non-optimal light-conditions in the room of pum- Visual inspections for quality management are typi- mel inspection etc., is demanding and led to no clear cally organized in an inspection process (determined favorable function. On the other hand, the access to in many cases by national or international building just a limited amount of labeled training image data regulations), which probes the whole production pro- formed another obstacle. To address these challenges, cess through several human-based controls of product- this paper proposes an AI-based classification tool (AI- specific quality measures. Since humans in principle Pummel Tool), which uses a deep convolutional neu- are unable to provide an objective result of a quality ral network on grey-value images of pummeled glass control due to their own bias (Nordfjeld 2013), uncer- laminates to completely automate pummel evaluation tainty in objectification and repeatability of the qual- while excluding human bias or complex image pre- ity measures is induced. It is thus preferable to supply processing. a technological solution in the form of combining AI Following the data-driven approach of AI, in Fig. 11 and computer vision to automate the quality inspec- a schematic illustration of the workflow for an AI-based tion while minimizing human intervention. Within the automated pummel classification is given. It relies on scope of this paper an example for the objectifica- the input of grey-value images after pummeling the tion, systematization and automation of a visual prod- laminated glass. These pictures are then processed by uct inspection for laminated glasses by the so-called a pre-trained deep CNN for classification into one of the Pummel test is presented. The Pummel test specifically 11 Pummel value categories. Details on the principal characterizes the degree of adhesion between the poly- architecture of CNNs were already given in Sect. 2.3, meric interlayer and the glass pane of a glass laminate, further details on CNNs especially within the field of where an optical scale ranging from 0 to 10 character- computer vision are not described here in detail with izes the level of adhesion. The resulting Pummel value reference to (LeCun et al. 2015; Voulodimos et al. thus delivers an indicator for the quality and safety 2018). Using this approach, the standard human-based properties of laminated glass, where a value of 0 quan- classification of pummel images into the Pummel cat- tifies no adhesion and 10 very high adhesion (Beck- egories during production control is therefore automa- mann and Knackstedt 1979; Division 2014). The lam- tized and objectified by using the pre-trained CNN for inated glass specimen for the Pummel test consist of prediction of the Pummel class along with a statistically two float glass panes with a maximum thickness of 2 sound quantification of uncertainties of this process. × 4 mm. The specimens are exposed to a climate of Since only a few labeled Pummel image data were −18 C for about 8 h and subsequently get positioned available for training the CNN, the authors used image on an inclined metal block and processed with a ham- data augmentation to expand the training data set. First 123 Artificial intelligence for structural glass engineering 267 Fig. 11 Schematic workflow for the AI supported evaluation of the Pummel test results show a prediction probability of the correct clas- with a quantification of the improvement of the perfor- sification of the pummel value of over 80 %, cf. Fig. 13. mance and robustness of the CNN and further investi- However, a significant performance gain is expected if gations on alternative architectures or even alternative more actual labeled Pummel image data is available in approaches such as clustering (Jain et al. 1999) has to the next step of this project. In order to show the perfor- build upon future studies with an increasing amount of mance of the AI Pummel tool, first validation results are ground-truth Pummel images. illustrated in Fig. 12, where the Pummel image to clas- sify is shown together with the CNN-based prediction 3.2.3 AI prediction of cut-edge of glass via semantic of the Pummel value as well as the Pummel value deter- segmentation mined by manufacturer (ground-truth Pummel value) is also shown. In the production and further processing of annealed Figure 12 together with Fig. 13 proves, that the AI float glass, glass panes are usually brought into the Pummel tool is very well able to generalize, i.e. to cor- required dimensions by a cutting process. In a first step, rectly classify Pummel images which were not used a fissure is generated on the glass surface by using a during the training of the CNN. The accuracy of the cutting wheel. In the second step, the cut is opened classification algorithm within the context of this paper along the fissure by applying a defined bending stress. is measured via the confusion matrix (also known as This cutting process is influenced by many parameters, error matrix ) (cf. Fig. 13). Each row of the matrix rep- where the glass edge strength in particular can be repro- resents the Pummel values predicted by AI, while each ducibly increased by a proper adjustment of the process column represents the actual Pummel value defined by parameters of the cutting machine (Ensslen and Müller- the manufacturer (ground-truth). The name confusion Braun 2017). It could be observed that due to differ- matrix stems from the interpretation of the algorithm, ent cutting process parameters, the resulting damage to here a CNN-classifier, confusing two classes. Interpre- the edge (the crack system) can differ in its extent. In tation of the confusion matrix of the CNN is interesting, addition, this characteristic of the crack system can be as for the pummel value classes 1, 3, 5, 7, 9 the accu- brought into a relationship with the strength (Müller- racy of the prediction is over 92%. The worst prediction Braun et al. 2018). In particular, it has been found that result is obtained for the Pummel value class of 6, where characteristics of the lateral cracks, cf. Fig. 14 viewing an accuracy of 63% was found. On the other hand, the the edge perpendicular to the glass surface, allow best results give rise to questioning the qualitative scale of predictions for the glass edge strength (Müller-Braun 11 classes to be lumped into e.g. 5 or 6 Pummel classes. et al. 2020). However, since the training of the CNN was based on a The challenge here is, however, to detect these lat- small amount of publicly available data, more theoret- eral cracks and the related geometry in an accurate and ical justification for this Pummel class lumping along objective way. Currently, this is conducted by man- 123 268 M. A. Kraus, M. Drass Fig. 12 Example Results of the AI Pummel tool for different Pummel input images (3 successes, 1 misclassification) 123 Artificial intelligence for structural glass engineering 269 In (Drass et al. 2020) AI and especially the prob- lem of semantic segmentation was for the first time applied to identify the cut edges of cut glass to auto- matically generate mask images. The goal is to process an image of a glass cut edge using the DL model U- Net (Ronneberger et al. 2015) in such a way that a mask image is generated by the model without explic- itly programming it to do so. For the problem at hand, the segmentation of the images of cut edges of glass is into two classes “breakage” (black) and undamaged glass (white), i.e. a binary segmentation, is conducted using the U-Net architecture, which is shown in Fig. 15. Accordingly, the mask image should only recognize the cut glass edge from the original image and display it in black in the mask image. More details on the the U-Net Fig. 13 Representation of the confusion matrix for the problem of AI-based prediction of the pummel value architecture, the learning algorithm and hyperparame- ter tuning is given in (Drass et al. 2020). As shown in Fig 16, the trained U-Net is well suited ual tracing due to the fact that the crack contour can to create a mask image from the original image with- sometimes only be recognized roughly by eye. After out the need for further human interaction. It is also manually marking the crack using an image processing obvious that the red-yellow areas, where the NN is program, the contour is then automatically evaluated not sure whether it sees the cut edge or just the pure further. Methods of AI and especially the algorithms glass, are very narrow and hence of minor importance. from the field of AI in computer vision now can be uti- A slight improvement of the mask images created by lized as an alternative to the existing manual approach AI could be achieved by the cut-off condition or binary to automate the step of manual detection of the glass prediction. The presented NN for predicting the cut cut edge. In addition to the enormous time and hence glass edge is therefore very accurate and saves a sig- economic savings, the objectivity and reproducibility nificant amount of time in the prediction and production of detection is an important aspect of improvement. of mask images. In addition, the mask images can be The topic of image classification in the context of com- further processed, for example to make statistical anal- puter vision and DL is well known (Ferreyra-Ramirez yses of the break structure of the cut glass edge. This et al. 2019). As stated in the previous section of this however is not part of the present paper and will not be paper, image classification is concerned with classify- further elaborated hence. ing images based on its visual content. The proposed model as described briefly here and Whilst the recognition of an object is trivial for in detail in (Drass et al. 2020) showed excellent results humans, for computer vision applications robust image for the prediction of the cut glass edge. The validation classification is still a challenge (Russakovsky et al. accuracies of both models exceeded 99 %, which is 2015). An extension of image classification is object sufficient for the generation of the mask image via AI. detection (i.e. enclosing objects by a frame or box within an image). Object detection often just requires a coarse bounding of the object within an image, but in 3.2.4 AI prediction of glass edge strength based on the case at hand it is desirable to extract the contours process parameters of an object as exactly as possible. Semantic segmen- tation in contrast to object detection describes the task This section deals with the prediction of the edge of classifying each individual pixel in an image into a strength of machine-cut glass based on the process specific class (Guo et al. 2018). The task of semantic parameters of the cutting machine using supervised segmentation processes image data in such a way that ML, more specifically an Extra Trees regressor, which an object to be found is segmented or bordered by a is also known as Extremely Randomized Trees (Geurts so-called mask. et al. 2006). 123 270 M. A. Kraus, M. Drass Fig. 14 (1) View on the cut edge of two 4 mm thick glass spec- eral crack to be detected: The crack contour can be difficult to imens, a slight crack system, breaking stress: 78 MPa, b more identify Drass et al. (2020) pronounced crack system, breaking stress: 53 MPa and (2) Lat- Fig. 15 U-Net architecture for the problem of image segmentation of cut glass 123 Artificial intelligence for structural glass engineering 271 Fig. 16 Results of the semantic segmentation using U-Net to predict the cut glass edge on the basis of three test images (axes are in [mm]) 123 272 M. A. Kraus, M. Drass Based on the investigations of (Müller-Braun et al. 2020), architectural glass is cut in two steps. First, a slit is created on the glass surface using a cutting tool and a cutting fluid. An integral part of the cutting tool is the cutting wheel. It is available in various dimensions, although the manufacturers make basic recommenda- tions regarding the cutting wheel angle and cutting wheel diameter for different glass thicknesses and cut- ting tasks. After cutting the glass, it must still be broken by applying some bending to the pane in order to obtain two separate pieces of glass of desired dimension. It is quite known from experience, that the edge strength of cut glass depends significantly on the applied process Fig. 17 Residual plot [MPa] for the AI-Predictor to determine parameters during the cutting process, proved by sim- the edge strength as a function of the process parameters of a ple graphical and statistical evaluation of experimental glass cutting machine for training and test set data in (Müller-Braun et al. 2020), where more details on the background of cutting and breaking glass as well as the experimental investigations of the cutting process on the edge strength of cut glass can be found. However, so far no concrete modeling approach was formulated Other parameters have been included in the test and trained on the data given the complexity of the data series, which are not described in detail here due to correlations. reasons of brevity. A total of 25 features were included In order to deliver a prediction model of the edge for the entire test series. After applying the Boruta fea- strength in dependence of the process parameters of the ture selection algorithm (Kursa et al. 2010), 12 of the glass cutting machine, this paper suggests a ML regres- 25 features could be classified as unimportant, so that sion model, correlating the process parameters of cut- the regression model was trained with a ML algorithm ting to the edge strength target value. With this model, it on a reduced number of 13 features. is possible for the first time to predict the edge strength The model used for this example is an Extra in dependence of the process parameters with high sta- Trees regressor (also known as Extremely Random- tistical certainty without performing destructive tests ized Trees) (Geurts et al. 2006), which is similar to on the cut glass. Providing this AI-based method deliv- a Random Forest regressor. SciKit-Learn (Pedregosa ers remarkable economic and ecological advantages. A et al. 2011; Buitinck et al. 2013) was used together with lot of manpower required for testing the glass is saved the default hyperparameter settings for the Extra Trees along with saving resources by avoiding great amounts regressor without further investigation on the hyper- of glass waste material by non-destructive testing. parameter tuning. The single holdout method has been The main parameters of the cutting process can be applied for splitting the data in training and testing data. summarized as follows: Figure 17 show the residuals (in MPa) between actual and predicted edge strength separately for the training and validation data. Given the R = 0.88 in Fig. 17 – test temperature it is concluded, that the obtained model describes the – relative humidity data well and the scatter is due to the dimension reduc- – glass thickness tion from 25 to 13 features. On the other hand side – glass height from the validation data performance it is concluded, – cutting speed that alternative models calibrated with ML algorithms – cutting force might be more suitable to better represent the data and – type of cutting fluid the presented Extra Trees regress may lack of overfit- – type of cutting wheel ting. A future paper will investigate in more detail an AI – cutting wheel angle based model for predicting the cut glass edge strength – ... (Fig. 18). 123 Artificial intelligence for structural glass engineering 273 for obtaining the parameters of the Helmholtz potential in a Bayesian manner is posed and calibrated. In the context of hyperelasticity, the isochoric or volume-constant Helmholtz free energy function Ψ may be written in form of the Nelder function as I − 3 1,b Ψ b = , (5) x + x I − 3 1 2 ¯ 1,b where I = tr b characterizes the first isochoric, 1,b principal invariant of the left, isochoric Cauchy-Green tensor b. An extension of Eq. 5 by the second invariant of the left Cauchy-Green tensor leads to I − 3 I − 3 ¯ ¯ Fig. 18 Edge strength as a function of the process parameters 1,b 2,b Ψ b = iso,ND of a glass cutting machine predicted by the AI Predictor versus x + x I − 3 x + x I − 3 1 2 ¯ 3 4 ¯ 1,b 2,b the experimentally obtained ground-truth (6) 3.3 AI in structural glass engineering which is the final form of the novel Helmholtz free energy function. As can be concluded from Eq.(6), the proposed This last subsection will highlight and discuss several Helmholtz energy possesses four parameters to be cal- applications of AI in the field of structural glass engi- ibrated θ = {x , x , x , x }. neering. In the context of this paper, two examples on 1 2 3 4 an already successfully conducted application of ML The experimental data of the transparent silicone was presented in (Drass et al. 2018a, b), where the mate- techniques on problems of that field and afterwards two visions for further incorporation of AI are given. Other rial was experimentally characterized in uniaxial ten- sion and compression, shear and biaxial deformation. applications of AI within structural engineering (such as design and verification of buckling for steel hol- The second structural silicone is a carbon black filled silicone adhesive, which was investigated by (Staudt low sections or computation of deflection or bending moment fields of a Kirchhoff plate etc.) were recently et al. 2018) for uniaxial tension and simple shear load- ing. The third material to be investigated is filled elas- published in (Kraus and Drass 2020a). tomer from the tire industry, which has been character- ized under tensile and shear loads (Lahellec et al. 2004). 3.3.1 Example 1: Bayesian calibration of a Helmholtz All material parameters have been determined using potential for hyperelasticity of TSSA silicone Bayesian supervised ML algorithms upon the DREAM MCMC algorithm (Vrugt 2016), cf. Fig. 19. Within the TM Within advanced analysis of polymeric materials in context of this paper only the results for DOWSIL structural glass engineering constitutive models have TSSA are presented for reasons of brevity. to be applied which are able of capturing the non- As can be seen from Fig. 19 a, the presented novel linear stress-strain relationship adequately. In (Drass hyperelastic model Ψ and the extended tube iso,ND 2019; Drass and Kraus 2020b), a novel functional form model are well suited to represent the experimental for the free Helmholtz energy for modeling hyperelas- data of TSSA for four different types of experiments. ticity was introduced and calibrated for various poly- It is interesting that the MCMC simulation on a stan- meric materials, especially for structural silicones such dard laptop lasted about 20 minutes and led to results TM TM as DOWSIL TSSA or DOWSIL 993 as well as (mean values of the parameters) that were very close glass laminate interlayers Poly-Vinyl-Butyral (PVB) to the smallest squares determined with the software and Ethylen-Vinyl-Acetate (EVA) by traditional opti- MATHEMATICA, although MCMC means a signifi- mization techniques. Here, a supervised ML problem cantly higher numerical effort compared to the small- 123 274 M. A. Kraus, M. Drass Fig. 19 Fitting results for TSSA silicone by different approaches the parameters θ of the proposed Helmholtz potential for TM TM and experiments under arbitrary deformations: a DOWSIL DOWSIL TSSA ; with UT = uniaxial tension, UC = uni- TSSA by parameter mean values of the Bayesian optimiza- axial compression, BT = biaxial tension, SPC = shear pancake tion supervised learning b uni- and bivariate distributions of and SPC = shear pancake tests est squares. Finally, it is emphasized, that by applying tern depends on the elastic strain energy density U the Bayesian framework, further deduction of partial and thus the magnitude of the residual stress induced material safety factors is straight forward as the uncer- by a thermal pre-stressing procedure. This is shown tainties in the associated model parameters are natu- in Fig. 20 for thermally tempered glass for differ- rally captured. Hence the application of Bayesian ML ent residual stress levels. It can be clearly observed in this context delivers addition insight on the certainty that the fragment size increases with decreasing resid- about the model parameters as well as model assess- ual stress level. Approaches up to now related mean ment quantities for further use in a reliability analysis quantities such as fracture particle weight, area con- at no extra cost compared to traditional optimization- tent or circumference (Pourmoghaddam and Schneider based material calibration strategies. 2018). To determine the characteristics of fragmentation, an ML algorithm named BREAK was developed in 3.3.2 Example 2: Bayesian reconstruction and (Kraus 2019). The model there combines an energy cri- prediction of glass breakage patterns (BREAK) terion of linear elastic fracture mechanics (LEFM) and the spatial statistical analysis of the fracture pattern of This section deals with the application of ML to cal- tempered glass in order to determine characteristics of ibrate a ML surrogate of the fragmentation pattern the fragmentation pattern (e.g. fragment size, fracture of thermally pre-stressed glasses along with its spa- intensity, etc.) within an observation field. The model- tial characteristics (such as e.g. fragment size, fracture ing approach is based on the idea that the final fracture intensity, etc.) via stochastic tessellations over random pattern is a Voronoi tessellation induced by a stochas- Strauss Point processes as initially suggested by (Kraus tic point process (a Strauss process in the context of 2019). this paper). The parameters of that model are calibrated Several studies on the fragmentation behavior of from statistical analysis of images of several fractured tempered glasses have proven relationships between glass samples. By calibration of that stochastic point the residual stress state, the glass thickness and the process and consecutive tessellation of the region of fragment density (Akeyoshi and Kanai 1965; Lee interest, statistically identically distributed realizations et al. 1965; Sedlacek 1999; Mognato et al. 2017; of fracture patterns can be generated. Further details Pourmoghaddam and Schneider 2018). The fragment on the theoretical background as well as the deriva- density or fracture intensity in an observation field, tion of the model specifics for several point processes the fragment shape and thus the entire fracture pat- 123 Artificial intelligence for structural glass engineering 275 Fig. 20 Fragment size of thermally tempered glass as a function of the residual stress (indication of the biaxial tensile residual stress in the mid-plane) at a plate thickness of t =12mm(Pourmoghaddam et al. 2018) within this semi-supervised ML approach is given in 3.4.1 Designed by AI (Kraus 2019). The schematic connections of the theo- ries and experiments involved for BREAK are given in In this first visionary section two points will be pre- Fig. 21. sented and elaborated: To show explicit results of the BREAK algorithm, a glass plate with thickness of t = 12 mm and a defined – design supported by AI degree of pre-stress of σ = 31.54 MPa (U = 8.754 – structural analysis supported by AI m 0 J /m ) was analysed. After the morphological pro- cessing of the fracture images, the first order statis- There are first publications dealing with the applica- tics of the extracted point pattern were determined in tion of AI in architecture and design, cf. (Mrosla et al. the first step to infer the point process intensity. After 2019; Newton 2019; Baldwin 2019), in which all note the model parameters had been calibrated on the basis that the examples of an AI-generated built environment of the recorded fracture pattern photos, the simula- existing today still need further years of research and tion of statistically equivalent fracture patterns was cooperation between the different fields to achieve the performed using the calibrated Strauss process with announced quality. induced Voronoi tessellation. An exemplary realization For example, (Baldwin 2019) proposed a floor plan based on the mean values of the model parameters is design method by Generative Adversarial Networks shown in Fig. 22. (GAN), cf. Fig. 23. Where a GAN is a special form This application proved, that a combination of AI of NN from the family of NN as presented in Sect. 2.3, algorithms for regression and computer vision enable more details on GANs may be found in (Goodfellow to model more complicated geometrical-numerical et al. 2016; Frochte 2019). dependencies such as glass fracture patterns while car- The GAN floor plan design pipeline uses image rep- rying statistical features of its components, which was resentations of plans as data format for both, GAN- not possible by traditional approaches. models’ inputs and outputs, where Pix2Pix is used as GAN geared towards image-to-image translation. The careful study of the organization learned by each model revealed the existence of a deeper bias, or architectural style. The project aimed to assist the architect in gen- 3.4 Outlook and future vision 1: AI for design and erating a coherent room layout and furnishing and to computation of structures finally reassemble all apartment units into a tentative floor plan The project also included the conversion of Within this section, a visionary outlook on the status floor plans from one style to another. quo and potential capabilities of AI in the fields of A future vision of design by AI based on the works designing and structural verification and computation presented here is the combination of the existing GAN of structures is given. with customer features such as preferences for colors, 123 276 M. A. Kraus, M. Drass Fig. 21 Schematic figure of the BREAK framework, showing the connections of experimental observations to the elements of spatial point patterns and linear fracture mechanics, from (Kraus 2019) Fig. 22 Simulation of a realization of a fracture pattern of a glass pane with thickness of t = 12 mm and level of pre-stress σ = 31.54 MPa with the calibrated SP, from (Kraus 2019) 123 Artificial intelligence for structural glass engineering 277 Fig. 23 Generation pipeline of designed floor plans by GANs, from (Baldwin 2019) shapes etc. By this, a customized design by GANs can ward simulations are conducted to collect the struc- be reached to a high consumer satisfaction level. tural responses given different combinations of design covariates. Here, some aspects have to be considered 3.4.2 Structural verification supported by AI especially: (1) the definition of the prior mean func- tions, (2) covariance and correlation functions and (3) In the context of the structural verification of cer- the formulation required for modeling cases involving tain structural members or in early design stages of a heteroscedastic errors (Goulet 2020). project, AI and its capabilities of establishing surrogate Figure 24b compares the Gaussian process regres- models can be utilized to provide fast conclusions on sion model predictions μ with the true finite element the structural feasibility of a designed structure without model outputs y . In order to obtain a meaningful com- explicit computation. parison between predicted and measured values, it is Surrogate modeling without and with AI methods essential to test the model iteratively using a cross- concern people within the computational mechanics validation procedure whereas at each step, the obser- and optimization communities since several years. An vation corresponding to the prediction location in the comprehensive overview is given by (Forrester et al. validation set is removed from the training set. A further 2008; Adeli 2001; Wortmann et al. 2015). The need for example of providing an AI-based surrogate for fast surrogates in engineering analysis stems from employ- and reliable structural design and verification of steel ing computationally demanding methods such as the hollow sections was recently published by the authors finite element models for analysis, presented in Fig. 24. in (Kraus and Drass 2020a) but it is not further elab- Surrogate modeling in practical terms means that the orated at this stage. As a conclusion, using AI based costly and time-consuming finite element model is surrogates for structural verification provides the com- replaced by a regression model build upon a set of sim- puting structural engineer a fast and reliable method ulated responses. Because observations for the training to check design alternatives or to conduct sensitivity of the surrogate are obtained by the output of a simula- analyses. Furthermore, transferability of the surrogate tion, the observation model does not include any obser- results is reached if a proper formulation of the engi- vation error (except the discretization error is consid- neering problem at hand is done a priori and lets further ered as observation noise). Within this paper, the Gaus- pay of a typically demanding training phase of the sur- sian process regression for the construction of meta- rogates. models for the responses of a structure (cf. Fig. 24a) To summarize this section, AI has the potential to using covariates and a set of simulations is discussed. accelerate design and structural verification processes In the context of surrogate modeling, the engineer to a great demand while customization wishes may has to specify the relevant responses given certain enter more naturally and affordably. The authors are covariates (i.e. design variables), then a number of for- 123 278 M. A. Kraus, M. Drass Fig. 24 Examples of numerically cheap AI based surrogates: a Examples of numerically demanding Finite Element Models; b Training of Gaussian Process as a cheap AI based surrogates. Both from (Goulet 2020) currently at a stage, where first knowledge and experi- thetic polymers in civil engineering is scientifically, ences are gathered with these ideas. Further research of technically and economically highly relevant. Thus, the the authors will consider more building-practical appli- development and safe design of novel structures in vari- cations of the presented ideas. ous fields such as architecture/construction, automotive engineering and aerospace is possible. A data-driven material modeling approach by the 3.5 Outlook and future vision 2: data-driven modeling earlier mentioned physics-informed/theory-guided AI of materials within glass-structures approach is particularly interesting as especially for engineering, in contrast to material sciences (a rather Especially in glass and façade construction, modern big data environment), constitutive models for design materials such as a great variety of polymers are used, have to be created and calibrated mostly on the basis but their constitutive modeling is much more com- of a few experiments (usually a small data environ- plicated than established building materials due to ment). The incorporation of physical laws and theoret- their thermomechanical properties (Kraus 2019; Drass ical knowledge there is of special interest. The develop- 2019). For more than ten years by now, a wide range ment of a reliable, methodologically sound and gener- of experimental and methodical work has been pro- alizable derivation of constitutive laws on the basis of viding the basis for an improved understanding of the techniques of AI and in particular of deep NN from material and load-bearing behavior of these materials, experimental data thus requires a systematic analy- whereby the latest methods place the highest demands sis of the relevant mechanisms, influence parameters on the engineer and the tools available such as finite and modeling strategies regarding the techniques of element software (Drass and Kraus 2020c; Kraus and AI, which can only succeed in a good symbiosis of Drass 2020b; Kraus 2019; Drass 2019). AI-supported the knowledge of the disciplines of material sciences, modeling of the complex constitutive behavior of these civil engineering, numerics and optimization as well as materials is one of the latest developments in AI related computer science. The overall goal of a recent research computational mechanics research as the realistic sim- project of the authors is the development of a validated ulation and design of polymeric components in civil and reliable methodology for the selection and calibra- engineering requires knowledge of the relevant mech- tion of suitable artificial intelligence models for mod- anisms of load transfer, failure and aging (if applica- ular thermodynamically consistent constitutive model- ble) and their effect on the load-bearing behavior. The ing of polymeric materials in civil engineering using methodical handling of the relevant processes and the experimental and simulation-based data. With such a transfer into modern numerical models for the reli- method, complicated material models can be estab- able simulation of the constitutive behavior of syn- 123 Artificial intelligence for structural glass engineering 279 lished on the basis of data from standard experiments of data in an engineering context is limited due to mon- and simulations to capture hyperelastic, viscoelastic etary or confidentiality reasons and thus the establish- and damage effects. As the framework is general, it is ing of publicly accessible databases is hardly possible not only restricted to the mentioned polymer silicone for a greater audience, however on the level of individ- and glass laminated polymers but would apply to any ual companies or research groups, the data stock prob- new material in the field. lem is not severe or even present. Finally a visionary outlook on the role of AI within supporting engineers for an early stage design of structures, the modeling 4 Summary and conclusions of advanced material behavior by physics-informed AI approaches as well as AI-based structural verification Within this paper the reader was introduced to the main surrogates finished the paper. concepts of and a brief background on Artificial Intelli- Within this paper the following conclusions are gence (AI) and its sub-groups Machine Learning (ML) reached: and Deep Learning (DL). The nomenclature along with – AI-supported control of adaptive façades will poten- the meaning of AI core vocabulary on the task T , tially solve the multi-criteria optimization prob- the performance measure P and experience E were lem involving economy, sustainability and user- introduced and illustrated via examples. Furthermore well being a detailed elaboration on the importance of splitting – several examples of a successful application of AI available data into training, validation and test set was in the field of structural glass engineering were pro- given, which then was followed by underlining over- vided and proofed superior compared to existing and underfitting of models during training by AI algo- approaches rithms and strategies to avoid either of that problems. – for the first time ever, AI models made it possible Then two sections on the basic nomenclature and care- to establish numerical predictions for phenomena fully chosen models from ML and DL were presented to such as glass fracture patterns or cut edge strength the reader. In the main part of the paper, a review and – the amount of available data for training AI mod- summary of already successfully conducted applica- els is often limited and hence constrict attainable tions of AI in several disciplines such as medicine, natu- model accuracies and generalizations, e.g. for the ral sciences, system identification and control, mechan- Pummel test as well as the cut edge strength exam- ical as well as civil engineering. A total of six core sec- ples tions then introduced and explained in detail problems – AI-based models can easily be enhanced by uncer- out of structural glass engineering, where AI meth- tainty quantification methods to establish reliability ods enabled training a model at all or were superior statements, e.g. in the case of polymeric materials compare to traditional engineering models. The glass for structural glass engineering or the accuracy of engineering applications range from accelerated glass predicting the Pummel value product development, deep learning based quality con- – future research and industry potentials ly in the trol of glass laminates for the Pummel test, prediction elaboration of AI-empowered design and verifi- of the cutting edge of glass together with prediction cation support systems, which enable to consider of the strength of the cut glass edge to the calibra- user/occupant demands as well as structural relia- tion of a Helmholtz potential and the prediction of the bility and serviceability already in early planning fracture pattern of thermally pre-stressed glass. For all stages examples the amount of necessary data together with the challenges and final solution strategy was reported In principle, we consider the introduction of AI tech- to enable the reader to judge temporal and monetary nologies in glass and façade construction and its neigh- effort for AI methods in comparison to existing engi- boring industries to be possible to a great extend already neering models and approaches (in case these are exist- immediately, since the essential basis of an AI, i.e. the ing). Here several points for taking care or differences existence of data, is already fulfilled in many cases. in an (structural façade and glass) engineering context A first task for research and especially industry will to traditional computer science approaches to AI where now be to structure the existing data in such a way highlighted. It was stated that especially the availability that AI algorithms can apply, train and validate diverse 123 280 M. A. Kraus, M. Drass models on it in order to lead to successful projects in C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Tal- war, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vie- combination with engineering expertise. The theoreti- gas, F.,Vinyals,O., Warden,P., Wattenberg,M., Wicke, cal framework and the respective software are in place M., Yu, Y., Zheng, X.: TensorFlow: Large-Scale Machine but have to be augmented by the knowledge of struc- Learning on Heterogeneous Distributed Systems (2016) tural/civil engineers, who are familiar with the statisti- arXiv:1603.04467 Adeli, H.: Neural networks in civil engineering: 1989–2000. cal and methodical concepts of AI. For this reason, in Comput.-Aided Civil Infrastruct. Eng. 16(2), 126–142 the eyes of the authors of this paper, it is essential to (2001) introduce these methodological knowledge and prac- Aelenei, D., Aelenei, L., Vieira, C.P.: Adaptive façade: con- ticing with AI in the study curricula of students of civil cept, applications, research questions. Energy Procedia 91(Supplement C), 269–275 (2016) engineering and architecture in the near future as well. Akadiri, P.O., Chinyio, E.A., Olomolaiye, P.O.: Design of a sus- tainable building: A conceptual framework for implement- Acknowledgements We would like to express our greatest ing sustainability in the building sector. Buildings 2(2), 126– gratitude to our numerous industry and research partners for 152 (2012) fruitful discussions on the topic and providing samples of mate- Akeyoshi, K., Kanai, E.: Mechanical Properties of Tempered rial or data. Especially the support by Stanford University (Prof. Glass. VII International Glass Congress (paper 80) (1965) Christian Linder, PhD @ CEE) as well as TU Darmstadt (Prof. Akponeware, A.O., Adamu, Z.A.: Clash detection or clash avoid- Dr.-Ing. Jens Schneider @ Fachgebiet Statik) along with their ance? an investigation into coordination problems in 3d bim. great academic guidance and support is highly appreciated. May Buildings 7(3), 75 (2017) this article and our research further impact the civil and structural Amarante dos Santos, F., Bedon, C., Micheletti, A.: Explo- glass engineering community. rative study on adaptive facades with superelastic antago- nistic actuation. Struct. Control Health Monit. 27(4), e2463 Compliance with ethical standards (2020) Aznar, F., Echarri, V., Rizo, C., Rizo, R.: Modelling the thermal behaviour of a building facade using deep learning. PloS Conflicts of interest The authors certify that they have NO affil- one 13(12), e0207616 (2018) iations with or involvement in any organization or entity with any Badue, C., Guidolini, R., Carneiro, R.V., Azevedo, P., Cardoso, financial interest or non-financial interest in the subject matter or V.B., Forechi, A., Jesus, L., Berriel, R., Paixão, T., Mutz, F., materials discussed in this manuscript. et al.: Self-driving cars: A survey (2019). arXiv:1901.04407 Baldwin, E.: Ai creates generative floor plans and styles with Open Access This article is licensed under a Creative Com- machine learning at harvard (2019) URL https://www. mons Attribution 4.0 International License, which permits use, archdaily.com/918471/ai-creates-generative-floor-plans- sharing, adaptation, distribution and reproduction in any medium and-styles-with-machine-learning-at-harvard/ or format, as long as you give appropriate credit to the original Barbosa, F., Woetzel, J., Mischke, J., Ribeirinho, M.J., Sridhar, author(s) and the source, provide a link to the Creative Com- M., Parsons, M., Bertram, N., Brown, S.: Reinventing Con- mons licence, and indicate if changes were made. The images or struction: A Route to Higher Productivity. McKinsey Global other third party material in this article are included in the article’s Institute (2017) Creative Commons licence, unless indicated otherwise in a credit Baumeister, T., Brunton, S.L., Kutz, J.N.: Deep learning and line to the material. If material is not included in the article’s Cre- model predictive control for self-tuning mode-locked lasers. ative Commons licence and your intended use is not permitted by JOSA B 35(3), 617–626 (2018) statutory regulation or exceeds the permitted use, you will need Beckmann, R., Knackstedt, W.: Process for the production of to obtain permission directly from the copyright holder. To view modified, partially acetalized polyvinyl alcohol films U.S. a copy of this licence, visit http://creativecommons.org/licenses/ Patent No. 4,144,376. 13 Mar. (1979) by/4.0/. Bejnordi, B.E., Veta, M., Van Diest, P.J., Van Ginneken, B., Karssemeijer, N., Litjens, G., Van Der Laak, J.A., Hermsen, M., Manson, Q.F., Balkenhol, M., et al.: Diag- nostic assessment of deep learning algorithms for detection References of lymph node metastases in women with breast cancer. Jama 318(22), 2199–2210 (2017) (2016a) Applying Supervised Learning. MathWorks Benoit, M., Ispas, S., Jund, P., Jullien, R.: Model of silica glass (2016b) Applying Unsupervised Learning. MathWorks from combined classical and ab initio molecular-dynamics (2016c) Introducing Machine Learning. MathWorks simulations. Eur. Phys. J. B-Condens. Matter Complex Syst. (2020) URL www.alexa.com 13(4), 631–636 (2000) (2020) URL www.apple.com/siri Bhowmik, A., Castelli, I.E., Garcia-Lastra, J.M., Jørgensen PB, Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, Winther, O., Vegge, T.: A perspective on inverse design C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghe- of battery interphases using multi-scale modelling, experi- mawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., ments and generative deep learning. Energy Storage Mate- Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, rials (2019) J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, 123 Artificial intelligence for structural glass engineering 281 Binkhonain, M., Zhao, L.: A review of machine learning algo- Chollet, F., Others (2015) Keras. https://github.com/fchollet/ rithms for identification and classification of non-functional keras requirements. Expert Syst. Appl. X 1, 100001 (2019). Chowdhary, K.R.: Introducing artificial intelligence. In: https://doi.org/10.1016/j.eswax.2019.100001 Linnhoff-Popien, C., Zaddach, M., Grahl, A. (eds.) Fun- Bishop, C.M.: Pattern Recognition and Machine Learning, Infor- damentals of Artificial Intelligence, Springer, Berlin, pp. mation science and statistics, 1st edn. Springer, Berlin 1–23 (2020) (2006) Chui, M., Manyika, J., Miremadi, M.: What ai can and can’t do Bluyssen, P.M., Oostra, M.A., Meertins, D.: Understanding (yet) for Your Business. McKinsey Quarterly, (2018) the indoor environment: How to assess and improve Crevier, D.: AI: The Tumultuous History of the Search for Arti- indoor environmental quality of people? In: Proceedings of ficial Intelligence. Basic Books Inc, USA (1993) CLIMA 2013: 11th REHVA World Congress & 8th Inter- Daily, M., Medasani, S., Behringer, R., Trivedi, M.: Self-driving national Conference on IAQVEC“ Energy Efficient, Smart cars. Computer 50(12), 18–23 (2017) and Healthy Buildings”, Prag, Czech Republic, 16-19 June Day, J.J.K., O’Brien, W.: Oh behave! survey stories and lessons 2013, Citeseer (2013) learned from building occupants in high-performance build- Borgstein, E., Lamberts, R., Hensen, J.: Mapping failures in ings. Energy Res. Soc. Sci. 31, 11–20 (2017). https://doi. energy and environmental performance of buildings. Energy org/10.1016/j.erss.2017.05.037 Build. 158, 476–485 (2018) De la Rosa, E., Yu, W.: Randomized algorithms for nonlinear Borrmann, A., König, M., Koch, C., Beetz, J.: Building Informa- system identification with deep learning modification. Inf. tion Modeling: Technologische Grundlage und Industrielle Sci. 364, 197–212 (2016) Anwendungen. Springer, Berlin (2015) Dhall, D., Kaur, R., Juneja, M.: Machine learning: A review of the Boschert, S., Rosen, R.: Digital twin—the simulation aspect. In: algorithms and its applications. In: Singh, P.K., Kar, A.K., Mechatronic futures, Springer, Berlin, pp. 59–74 (2016) Singh, Y., Kolekar, M.H., Tanwar, S. (eds.) Proceedings of Bostrom, N.: Superintelligence. Dunod, Paris (2017) ICRIC 2019, pp. 47–63. Springer International Publishing, Brady, T.F., Yellig, E.: Simulation data mining: a new form of Cham (2020) computer simulation output. In: Proceedings of the Winter Division, K.C.L.P.: Pummel Test Standards. URL https://www. Simulation Conference, 2005., IEEE, p 5 ff (2005) trosifol.com/fileadmin/user_upload/TROSIFOL/support/ Brownlee, J.: Clever Algorithms: Nature-Inspired Programming downloads/technical_information/PUMMEL_Book.pdf Recipes. Machine Learning Mastery (2011) (2014) Brownlee, J.: Machine Learning Mastery With Python: Under- dos Santos, F.A., Rodrigues, A., Micheletti, A.: Design and stand Your Data, Create Accurate Models, and Work experimental testing of an adaptive shape-morphing tenseg- Projects End-to-End. Machine Learning Mastery (2016) rity structure, with frequency self-tuning capabilities, using Brunton, S.L., Kutz, J.N.: Data-driven Science and Engineering: shape-memory alloys. Smart Materials Struct. 24(10), Machine Learning, Dynamical Systems, and Control. Cam- 105,008 (2015) bridge University Press, Cambridge (2019) Drass, M., Berthold, H., Kraus, M.A., Müller-Braun, S.: Seman- Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, tic segmentation with deep learning: Detection of cracks at A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., the cut edge of glass (under review). Glass Structures & Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Engineering (2020) Varoquaux, G.: API design for machine learning software: Drass, M., Kraus, M.: Dimensioning of silicone adhesive joints: experiences from the scikit-learn project. In: ECML PKDD Eurocode-compliant, mesh-independent approach using the fem (under review). Glass Structures & Engineering (2020a) Workshop: Languages for Data Mining and Machine Learn- ing, pp. 108–122 (2013) Drass, M., Kraus, M.A.: Rational hyperelastic material model Bukieda, P., Lohr, K., Meiberg, J., Weller, B.: Study on the optical for various engineering application polymers submitted for quality and strength of glass edges after the grinding and publication (2020b) polishing process. Glass Structures & Engineering (2020). Drass, M., Kraus, M.A.: Semi-probabilistic calibration of a par- https://doi.org/10.1007/s40940-020-00121-x tial material safety factor for structural silicone adhesives - Bunker, S.: Artificial Intelligence: Building Smarter Machines part i: Derivation. Int. J. Struct. Glass Adv. Mater. Res. 4, (2018) 56–68 (2020c). https://doi.org/10.3844/sgamrsp.2020.56. Burrows, S., Stein, B., Frochte, J., Wiesner, D., Müller, K.: 68 Simulation data mining for supporting bridge design. Pro- Drass, M.: Constitutive modelling and failure prediction of sili- ceedings of the ninth Australasian data mining conference- cone adhesives in facade design. Doctoral thesis, Technis- volume 121, 163–170 (2011) che Universität Darmstadt (2019) Bzdok, D., Altman, N., Krzywinski, M.: Points of significance: Drass, M., Schwind, G., Schneider, J., Kolling, S.: Adhesive con- statistics versus machine learning (2018). https://doi.org/ nections in glass structures-part i: experiments and analytics 10.1038/nmeth.4642 on thin structural silicone. Glass Struct. Eng. 3(1), 39–54 Chang, N.B., Bai, K.: Multisensor Data Fusion and Machine (2018a). https://doi.org/10.1007/s40940-017-0046-5 Learning for Environmental Remote Sensing. CRC Press, Drass, M., Schwind, G., Schneider, J., Kolling, S.: Adhe- USA (2018) sive connections in glass structures-part ii: material Chen, S., Wang, H.: Sar target recognition based on deep learn- parameter identification on thin structural silicone. Glass ing. In: 2014 International Conference on Data Science and Struct. Eng. 3(1), 55–74 (2018b). https://doi.org/10.1007/ Advanced Analytics (DSAA), IEEE, pp 541–547 (2014) s40940-017-0048-3 123 282 M. A. Kraus, M. Drass Elboushaki, A., Hannane, R., Afdel, K., Koutti, L.: Multid-cnn: Greenblatt, N.A.: Self-driving cars and the law. IEEE Spectr. A multi-dimensional feature learning approach based on 53(2), 46–51 (2016) deep convolutional networks for gesture recognition in rgb- Greenspan, H., Van Ginneken, B., Summers, R.M.: Guest edito- d image sequences. Expert Syst. Appl. 139, 112,829 (2020) rial deep learning in medical imaging: Overview and future Elton, D.C., Boukouvalas, Z., Fuge, M.D., Chung, P.W.: promise of an exciting new technique. IEEE Trans. Med. Deep learning for molecular generation and optimization- Imaging 35(5), 1153–1159 (2016) a review of the state of the art (2019). arXiv preprint Guo, Y., Liu, Y., Georgiou, T., Lew, M.S.: A review of semantic arXiv:1903.04388 segmentation using deep neural networks. Int. J. Multimed. Ensslen, F., Müller-Braun, S.: Kantenfestigkeit von floatglas in Inf. Retr. 7(2), 87–93 (2018) abhängigkeit von wesentlichen schneidprozessparametern. Guzella, T.S., Caminhas, W.M.: A review of machine learn- ce/papers 1(1), 189–202 (2017) ing approaches to spam filtering. Expert Syst. Appl. 36(7), Fabi, V., Spigliantini, G., Corgnati, S.P.: Insights on smart home 10206–10222 (2009) concept and occupants’ interaction with building controls. Hars, A.: Self-driving cars: The digital transformation of mobil- Energy Procedia 111, 759–769 (2017) ity. In: Marktplätze im Umbruch, Springer, Berlin pp 539– Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: 549 (2015) Deep learning for time series classification: a review. Data Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Sta- Mining Knowl. Discov. 33(4), 917–963 (2019) tistical Learning: Data Mining, Inference, and Prediction. Ferreyra-Ramirez, A., Aviles-Cruz, C., Rodriguez-Martinez, E., Springer, Berlin (2009) Villegas-Cortez, J., Zuñiga-Lopez, A.: An improved con- Hill, J., Mulholland, G., Persson, K., Seshadri, R., Wolverton, volutional neural network architecture for image classifi- C., Meredig, B.: Materials science with large-scale data and cation. In: Mexican Conference on Pattern Recognition, informatics: unlocking new opportunities. Mrs Bull. 41(5), Springer, Berlin pp 89–101 (2019) 399–409 (2016) Fischer, M., Wüchner, R., Bletzinger, K.U.: Computa- Holstein, T., Dodig-Crnkovic, G., Pelliccione, P.: Ethical and tional framework for simulation and design of adaptive social aspects of self-driving cars (2018). arXiv:1802.04103 lightweight structures. In: IV ECCOMAS Thematic Confer- Hu, G., Yang, Y., Yi, D., Kittler, J., Christmas, W., Li, S.Z., ence on Smart Structures and Materials 2009, Porto (2009) Hospedales, T.: When face recognition meets with deep Forrester, A., Sobester, A., Keane, A.: Engineering design via learning: an evaluation of convolutional neural networks surrogate modelling: a practical guide. Wiley, New York for face recognition. In: Proceedings of the IEEE interna- (2008) tional conference on computer vision workshops, pp 142– Franz, J.: Untersuchungen zur Resttragfähigkeit von gebroch- 150 (2015) enen Verglasungen: Investigation of the residual load- Huang, Y., Li, J., Fu, J.: Application of artificial intelligence in bearing behaviour of fractured glazing, vol. 45. Springer, civil engineering. Comput. Model. Eng. Sci. 121(3), 845– Berlin (2015) 875 (2019) Frochte, J.: Maschinelles Lernen: Grundlagen und Algorithmen Illguth, M., Schuler, C., Bucak, Ö.: The effect of optical in Python. Carl Hanser Verlag GmbH Co KG, (2019) anisotropies on building glass façades and its measurement Gao, X., Jin, Y., Dou, Q., Heng, P.A.: Automatic gesture recog- methods. Front. Archit. Res. 4(2), 119–126 (2015) nition in robot-assisted surgery with reinforcement learning Isikdag, U.: Enhanced building information models. Springer- and tree search (2020). arXiv:2002.08718 Briefs in Computer Science, pp. 577–590 (2015) García, S., Ramírez-Gallego, S., Luengo, J., Benítez, J.M., Her- Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. rera, F.: Big data preprocessing: methods and prospects. Big ACM Comput. Surv. 31(3), 264–323 (1999). https://doi.org/ Data Anal. 1(1), 9 (2016) 10.1145/331499.331504 Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Jaramillo, J.C.A., Murillo-Fuentes, J.J., Olmos, P.M.: Boost- Mach. Learn. 63(1), 3–42 (2006) ing handwriting text recognition in small databases with Ghaffarianhoseini, A., Tookey, J., Ghaffarianhoseini, A., Nai- transfer learning. In: 2018 16th International Conference smith, N., Azhar, S., Efimova, O., Raahemifar, K.: Building on Frontiers in Handwriting Recognition (ICFHR), IEEE, information modelling (bim) uptake: Clear benefits, under- pp. 429–434 (2018) standing its implementation, risks and challenges. Renew. Karpatne, A., Watkins, W., Read, J., Kumar, V.: Physics-guided Sustain. Energy Rev. 75, 1046–1053 (2017) neural networks (pgnn): An application in lake temperature Goh, G.B., Hodas, N.O., Vishnu, A.: Deep learning for compu- modeling (2017). arXiv:1710.11431 tational chemistry. J. Comput. Chem. 38(16), 1291–1307 Kepuska, V., Bohouta, G.: Next-generation of virtual personal (2017) assistants (microsoft cortana, apple siri, amazon alexa and Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT google home). In: 2018 IEEE 8th Annual Computing Press, USA (2016) and Communication Workshop and Conference (CCWC), Goulet, J.A.: Probabilistic Machine Learning for Civil Engineers. IEEE, pp. 99–103 (2018) MIT Press, USA (2020) Ker, J., Wang, L., Rao, J., Lim, T.: Deep learning applications in Granadeiro, V., Duarte, J.P., Correia, J.R., Leal, V.M.: Building medical image analysis. IEEE Access 6, 9375–9389 (2017) envelope shape design in early stages of the design process: Kim, P.: Matlab deep learning. Mach. Learn. Neural Netw. Artif. Integrating architectural design systems and energy simu- Intell. 130, 21 (2017) lation. Autom. Construct. 32, 196–209 (2013) Kraus, M.A., Drass, M.: Semi-probabilistic calibration of a par- Graves, A.: Sequence transduction with recurrent neural net- tial material safety factor for structural silicone adhesives - works (2012) arXiv:1211.3711 part ii: Verification concept. Int. J. Struct. Glass Adv. Mater. 123 Artificial intelligence for structural glass engineering 283 Res. 4, 10–23 (2020b). https://doi.org/10.3844/sgamrsp. review and classification scheme. Building and Environ- 2020.10.23 ment p 106880 (2020) Kraus, M.A.: Machine learning techniques for the material Luna-Navarro, A., Overend, M.: Towards human-centred intel- parameter identification of laminated glass in the intact and ligent envelopes: A framework for capturing the holistic post-fracture state. PhD thesis, Universität der Bundeswehr effect of smart façades on occupant comfort and satisfac- München (2019) tion (2018) Kraus, M., Drass, M.: Künstliche intelligenz im bauingenieurwe- Marchwinski, ´ J.: Architectural evaluation of switchable glazing sen - hintergründe, status quo und potentiale. Bauingenieur technologies as sun protection measure. Energy Procedia (2020a) 57, 1677–1686 (2014) Kuhn, M., Johnson, K.: Applied Predictive Modeling, vol. 26. Marr, B.: A short history of machine learning–every manager Springer, Berlin (2013) should read (2016). Forbes http://tinyurl.com/gslvr6k Kumar, S., Sharma, A., Mamun, K., Tsunoda, T.: A deep learn- Mauro, J.C.: Decoding the glass genome. Current Opinion in ing approach for motor imagery eeg signal classification. In: Solid State Mater. Sci. 22(2), 58–64 (2018) 2016 3rd Asia-Pacific World Congress on Computer Sci- Mauro, J.C., Tandia, A., Vargheese, K.D., Mauro, Y.Z., Smed- ence and Engineering (APWC on CSE), IEEE, pp. 34–39 skjaer, M.M.: Accelerating the design of functional glasses (2016) through modeling. Chem. Mater. 28(12), 4267–4277 (2016) Kursa, M.B., Rudnicki, W.R., et al.: Feature selection with the McCarthy, J., Minsky, M., Rochester, N.: The dartmouth summer boruta package. J. Stat. Softw. 36(11), 1–13 (2010) research project on artificial intelligence. Artificial intelli- Kurt Peker, Y., Rodriguez, X., Ericsson, J., Lee, S.J., Perez, A.J.: gence: past, present, and future (1956) A cost analysis of internet of things sensor data storage on Meerbeek, B., te Kulve, M., Gritti, T., Aarts, M., van Loenen, blockchain via smart contracts. Electronics 9(2), 244 (2020) E., Aarts, E.: Building automation and perceived control: Lahellec, N., Mazerolle, F., Michel, J.: Second-order esti- a field study on motorized exterior blinds in dutch offices. mate of the macroscopic behavior of periodic hyperelastic Buil. Environ. 79, 66–77 (2014) composites: theory and experimental validation. J. Mech. Mitchell, T.M.: Machine Learning, McGraw-Hill Series in Com- Phys. Solids 52(1), 27–49 (2004). https://doi.org/10.1016/ puter Science, 1st edn. McGraw-Hill (1997) S0022-5096(03)00104-2 Mognato, E., Brocca, S., Barbieri, A.: Thermally processed glass Lampropoulos, G., Siakas, K., Anastasiadis, T.: Internet of things : correlation between surface compression, mechanical and in the context of industry 4.0: an overview. Int. J. Entrep. fragmentation test. Glass Performance Days 2017, 8–11 Knowl. 7(1), 4–19 (2019) (2017) LeCun, Y., Bengio, Y., et al.: Convolutional networks for images, Monedero, J.: Parametric design: a review and some experiences. speech, and time series. Handbook of Brain Theory and Automation in Construction 9(4), 369–377 (2000) Neural Netw. 3361(10), 1995 (1995) Moor, J.: The dartmouth college artificial intelligence confer- LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature ence: The next fifty years. Ai Magazine 27(4), 87–87 (2006) 521(7553), 436–444 (2015) Mosavi, A.: Structural damage diagnosis and prediction using Lee, E.H., Rogers, T.G., Woo, T.C.: Residual stresses in a machine learning and deep learning models: Comprehen- glass plate cooled symmetrically from both surfaces. J. sive review of advances (2019) Am. Ceram. Soc. 48(9), 480–487 (1965). https://doi.org/ Mrosla, L., Koch, V., von Both, P.: Quo vadis ai in architecture?- 10.1111/j.1151-2916.1965.tb14805.x survey of the current possibilities of ai in the architectural Lee, J.H., Shin, J., Realff, M.J.: Machine learning: overview of practice (2019) the recent progresses and implications for the process sys- Müller-Braun, S., Franz, J., Schneider, J., Schneider, F.: Optische tems engineering field. Comput. Chem. Eng. 114, 111–121 merkmale der glaskante nach glaszuschnitt mit schneidräd- (2018) chen. ce/papers 2(1), 99–111 (2018) Li, S., Deng, W.: Deep facial expression recognition: A survey. Müller-Braun, S., Seel, M., König, M., Hof, P., Schneider, J., IEEE Transactions on Affective Computing (2020) Oechsner, M.: Cut edge of annealed float glass: crack system Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, and possibilities to increase the edge strength by adjusting A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., the cutting process. Glass Struct. Eng. 5(1), 3–25 (2020) et al.: A comparison of deep learning performance against Müller-Braun, S., Seel, M., König, M., Hof, P., Schneider, J., health-care professionals in detecting diseases from medical Oechsner, M.: Cut edge of annealed float glass: crack system imaging: a systematic review and meta-analysis. The Lancet and possibilities to increase the edge strength by adjusting Digit. Health 1(6), e271–e297 (2019) the cutting process. Glass Struct. Eng. 5(1), 3–25 (2020). Lloyd, S.: Least squares quantization in pcm. IEEE Trans. Inf. https://doi.org/10.1007/s40940-019-00108-3 Theory 28(2), 129–137 (1982) Murphy, K.P.: Mach. Learning: A Probabilistic Perspective. MIT Louter, C., Akilo, M., Miri, B., Neeskens, T., Ribeiro Silveira, press, (2012) R., Topcu, Ö., van der Weijde, I., Zha, C., Bilow, M., Turrin, Nassif, A.B., Shahin, I., Attili, I., Azzeh, M., Shaalan, K.: M., et al.: Adaptive and composite thin glass concepts for Speech recognition using deep neural networks: A system- architectural applications. Heron 63(1/2), 199–218 (2018) atic review. IEEE Access 7, 19143–19165 (2019) Lu, P., Chen, S., Zheng, Y.: Artificial intelligence in civil engi- Newton, D.: Generative deep learning in architectural design- neering. Math. Prob. Engi. 2012, 145974 (2012). https:// Generative deep learning in architectural design. Tech.| doi.org/10.1155/2012/145974 Archit. + Des. 3(2), 176–189 (2019) Luna-Navarro, A., Loonen, R., Juaristi, M., Monge-Barrio, A., Niu, G.: Data-Driven Technol. Eng. Syst. Health Manag. Attia, S., Overend, M.: Occupant-facade interaction: a Springer, Berlin (2017) 123 284 M. A. Kraus, M. Drass Nordfjeld, A.Á.Z.: The aspect of the human bias in decision Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Den- making within quality management systems & lean theory. zler, J., Carvalhais, N., et al.: Deep learning and process Int. J. Psychol. Behav. Sci. 7(12), 2536–2541 (2013) understanding for data-driven earth system science. Nature O’Leary, D.E.: Artificial intelligence and big data. IEEE Intell. 566(7743), 195 (2019) Syst. 28(2), 96–99 (2013) Reitermanova, Z.: Data splitting. In: WDS 10, 31–36 (2010) Oxman, R.: Thinking difference: Theories and models of para- Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: Learning metric design thinking. Des. Stud. 52, 4–39 (2017) deep 3d representations at high resolutions. In: Proceedings Ozdemir, S., Susarla, D.: Feature Engineering Made Easy: Iden- of the IEEE Conference on Computer Vision and Pattern tify Unique Features from Your Dataset in Order to Build Recognition, pp. 3577–3586 (2017) Powerful Machine Learning Systems. Packt Publishing Ltd, Romano, R., Aelenei, L., Aelenei, D., Mazzuchelli, E.S.: What is (2018) an adaptive façade? analysis of recent terms and definitions Paluszek, M., Thomas, S.: MATLAB Machine Learning. Apress, from an international perspective. J. Facade Des. Eng. 6, (2016) 65–76 (2018) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional net- G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmai- works for biomedical image segmentation. In: International son, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, Conference on Medical image computing and computer- A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, assisted intervention, Springer, Berlin, pp. 234–241 (2015) S.: PyTorch: An imperative style, high-performance deep Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by learning library. In: Wallach, H., Larochelle, H., Beygelz- locally linear embedding. Science 290(5500), 2323–2326 imer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances (2000) in Neural Information Processing Systems 32, pp. 8024– Rudy, S.H., Kutz, J.N., Brunton, S.L.: Deep learning of dynam- 8035. Curran Associates Inc, (2019) ics and signal-noise decomposition with time-stepping con- Patil, P.S.: Review on use artificial intelligence in various appli- straints. J. Comput. Phys. 396, 483–506 (2019) cations of mechanical engineering, medical and healthcare Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, field (2016) S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Patil, A., Patted, L., Tenagi, M., Jahagirdar, V., Patil, M., Gau- Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recogni- tam, R.: Artificial intelligence as a tool in civil engineering-a tion challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 review. In: Proceedings of National conference on advances (2015). https://doi.org/10.1007/s11263-015-0816-y in computational biology, communication, and data ana- Russell, S., Norvig, P.: Artificial intelligence: A modern approach lytics (ACBCDA, : India: IOSR J, p. 2017. Eng, Comput (4th ed.) Hoboken, NJ: Pearson (2020) (2017) Rusu, O., Halcu, I., Grigoriu, O., Neculoiu, G., Sandulescu, V., Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, Marinescu, M., Marinescu V (2013) Converting unstruc- B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., tured and semi-structured data into knowledge. In: 2013 Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, 11th RoEduNet International Conference, IEEE, pp 1–4 D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Sacks, R., Bloch, T., Katz, M., Yosef, R.: Automating design Machine learning in Python. J. Mach. Learn. Res. 12, 2825– review with artificial intelligence and bim: State of the art 2830 (2011) and research framework. In: Computing in Civil Engineer- Pourmoghaddam, N., Kraus, M.A., Schneider, J., Siebert, G.: The ing 2019: Visualization, Information Modeling, and Sim- geometrical properties of random 2d voronoi tesselations ulation, American Society of Civil Engineers Reston, VA, pp. 353–360 (2019) for the prediction of the tempered glass fracture pattern. ce/papers 2.5–6, pp. 325–339 (2018) Schleicher, S., Lienhard, J., Poppinga, S., Masselter, T., Speck, Pourmoghaddam, N., Schneider, J.: Experimental investiga- T., Knippers, J.: Adaptive façade shading systems inspired tion into the fragment size of tempered glass. Glass by natural elastic kinematics (2011) Structures & Engineering (2018). https://doi.org/10.1007/ Schneider, J., Kuntsche, J., Schula, S., Schneider, F., Wörner, s40940-018-0062-0 J.D.: Glasbau: Grundlagen, Berechnung. Konstruktion. Quantrille, T.E., Liu, Y.A.: Artificial intelligence in chemical Springer, Berlin (2016) engineering. Elsevier, Netherlands (2012) Schober, K.S.: Artificial intelligence in the con- Raissi, M.: Deep hidden physics models: Deep learning of non- struction industry (2020). URL https:// linear partial differential equations. J. Mach. Learn. Res. www.rolandberger.com/en/Point-of-View/ 19(1), 932–955 (2018) Artificial-intelligence-in-the-construction-industry.html Raj, P., Evangeline, P.: The Digital Twin Paradigm for Smarter Sedlacek, G.: Glas im konstruktiven Ingenieurbau. Ernst & Sohn, Systems and Environments: The Industry Use Cases. Aca- Berlin (1999) demic Press, USA (2020) Senatore, G., Duffour, P., Winslow, P.: Exploring the application Rani, R., Kumar, R., Singh, A.P.: Deep learning method based domain of adaptive structures. Eng. Struct. 167, 608–628 binary descriptor for object detection. In: Proceedings of (2018) ICETIT 2019, Springer, Berlin, pp. 364–371 (2020) Serrano, W.: Deep reinforcement learning algorithms in intelli- Raschka, S.: Model evaluation, model selection, and algo- gent infrastructure. Infrastructures 4(3), 52 (2019) rithm selection in machine learning (2018). arXiv preprint Shahin, H.S.M.: Adaptive building envelopes of multistory build- arXiv:1811.12808 ings as an example of high performance building skins. Rebala, G., Ravi, A., Churiwala, S.: An Introduction to Machine Alexandria Eng. J. 58(1), 345–352 (2019) Learning. Springer, Berlin (2019) 123 Artificial intelligence for structural glass engineering 285 Siebert, G., Maniatis, I.: Tragende Bauteile aus Glas: Grundla- Vrugt, J.A.: Markov chain monte carlo simulation using the gen, Konstruktion, Bemessung. Beispiele. Wiley, New York dream software package: Theory, concepts, and matlab (2012) implementation. Environ. Modell. Softw. 75, 273–316 Silveira, R.R., Louter, C., Klein, T.: Flexible transparency-a study (2016). https://doi.org/10.1016/j.envsoft.2015.08.013 on adaptive thin glass façade panels. Chall. Glass Conf. Wada, B.: Adaptive structures. In: 30th Structures, Structural Proc. 6, 135–148 (2018) Dynamics and Materials Conference, p. 1160 (1989) Skansi, S.: Autoencoders. In: Introduction to Deep Learning, Wagg, D., Bond, I., Weaver, P., Friswell, M.: Adaptive structures: Springer, Berlin pp. 153–163 (2018) engineering applications. Wiley, New York (2008) Staudt, Y., Odenbreit, C., Schneider, J.: Failure behaviour of sili- Wagner, N., Rondinelli, J.M.: Theory-guided machine learning cone adhesive in bonded connections with simple geometry. in materials science. Front. Mat. 3, 28 (2016) Int. J. Adhes. Adhes. 82, 126–138 (2018). https://doi.org/ Wang, J., Li, J.: Bio-inspired kinetic envelopes for building 10.1016/j.ijadhadh.2017.12.015 energy efficiency based on parametric design of building Sun, X., Wu, P., Hoi, S.C.: Face detection using deep learning: information modeling. In: 2010 Asia-Pacific Power and An improved faster rcnn approach. Neurocomputing 299, Energy Engineering Conference, IEEE, pp. 1–4 (2010) 42–50 (2018) Wilber, J., Writer, R.E.: Q&a with industry experts how are e- Svetozarevic, B., Begle, M., Jayathissa, P., Caranovic, S., Shep- business trends impacting testers and testing teams? (2002) herd, R.F., Nagy, Z., Hischier, I., Hofer, J., Schlueter, Witten, I.H., Frank, E., Hall, M.A., Pal, C.: Data Mining: Practi- A.: Dynamic photovoltaic building envelopes for adaptive cal Machine Learning Tools and Techniques. Morgan Kauf- energy and comfort management. Nature Energy 4(8), 671– mann (2016) 682 (2019) Wolfgang, E.: Introduction to Artificial Intelligence, 2nd edn. Szolovits, P.: Artificial Intelligence in Medicine. Routledge Springer, Berlin (2017) (2019) Wortmann, T., Tunçer, B.: Differentiating parametric design: Tandia, A., Onbasli, M.C., Mauro, J.C.: Machine learning for Digital workflows in contemporary architecture and con- glass modeling. In: Musgraves, J.D., Hu, J., Calvez, L. (eds.) struction. Des. Stud. 52, 173–197 (2017) Springer Handbook of Glass, pp. 1157–1192. Springer, Wortmann, T., Costa, A., Nannicini, G., Schroepfer, T.: Advan- Berlin (2019). https://doi.org/10.1007/978-3-319-93728-1 tages of surrogate models for architectural design optimiza- Turing, A.M.: Computing machinery and intelligence. Mind tion. Artif. Intell. Eng. Des. Anal. Manufact. 29(4), 471–481 LIX(236), 433–460 (1950). https://doi.org/10.1093/mind/ (2015). https://doi.org/10.1017/S0890060415000451 LIX.236.433 Yu, D., Deng, L.: Automatic Speech Recognition. Springer, Turner, C.R., Fuggetta, A., Lavazza, L., Wolf, A.L.: A conceptual Berlin (2016) basis for feature engineering. J. Syst. Softw. 49(1), 3–15 Zhang, J., Liu, N., Wang, S.: A parametric approach for perfor- (1999) mance optimization of residential building design in Bei- Van Ginhoven, R.M., Jónsson, H., Corrales, L.R.: Silica glass jing. Build. Simul. 13, 223–235 (2020). https://doi.org/10. structure generation for ab initio calculations using small 1007/s12273-019-0571-z samples of amorphous silica. Phys. Rev. B 71(2), 024,208 Zhao, Z.Q., Zheng, P., Xu, St, Wu, X.: Object detection with deep (2005) learning: A review. IEEE Trans. Neural Netw. Learn. Syst. Van Veen, F.: The neural network zoo (2016). https://www. 30(11), 3212–3232 (2019) asimovinstitute.org/neural-network-zoo/ Zheng, Y., Cai, Y., Zhong, G., Chherawala, Y., Shi, Y., Dong, J.: Varian, H.: Artificial intelligence, economics, and industrial orga- Stretching deep architectures for text recognition. In: 2015 nization. Technical Report, National Bureau of Economic 13th International Conference on Document Analysis and Research (2018) Recognition (ICDAR), IEEE, pp. 236–240 (2015) Vass, S., Gustavsson, T.K.: Challenges when implementing bim for industry change. Construct. Manag. Econom. 35(10), Publisher’s Note Springer Nature remains neutral with regard 597–610 (2017) to jurisdictional claims in published maps and institutional affil- Vergauwen, A., Alegria Mira, L., Roovers, K., De Temmerman, iations. N.: Parametric design of adaptive shading elements based on curved-line folding. In: Proceedings of the First Conference Transformables 2013 (2013) Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: A Brief Review. Computational intelligence and neuroscience 2018, (2018) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Glass Structures & Engineering Springer Journals

Artificial intelligence for structural glass engineering applications — overview, case studies and future potentials

Glass Structures & Engineering , Volume 5 (3) – Nov 7, 2020

Loading next page...
 
/lp/springer-journals/artificial-intelligence-for-structural-glass-engineering-applications-5vkQwqV1mH
Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2020
ISSN
2363-5142
eISSN
2363-5150
DOI
10.1007/s40940-020-00132-8
Publisher site
See Article on Publisher Site

Abstract

Glass Struct. Eng. (2020) 5:247–285 https://doi.org/10.1007/s40940-020-00132-8 SI: CHALLENGING GLASS Artificial intelligence for structural glass engineering applications — overview, case studies and future potentials M. A. Kraus · M. Drass Received: 31 January 2020 / Accepted: 17 August 2020 / Published online: 7 October 2020 © The Author(s) 2020 Abstract ’Big data’ and the use of ’Artificial Intelli- cation and monitoring of façades and glass structures. gence’ (AI) is currently advancing due to the increas- Finally, the current status of research as well as suc- ing and even cheaper data collection and process- cessfully conducted industry projects by the authors are ing capabilities. Social and economical change is pre- presented. The discussion of specific problems ranges dicted by numerous company leaders, politicians and from supervised ML in case of the material parame- researchers. Machine and Deep Learning (ML/DL) are ter identification of polymeric interlayers used in lami- sub-types of AI, which are gaining high interest within nated glass or the prediction of cut-edge strength based the community of data scientists and engineers world- on the process parameters of a glass cutting machine wide. Obviously, this global trend does not stop at struc- and prediction of fracture patterns of tempered glass tural glass engineering, so that, the first part of the to the application of computer vision DL methods to present paper is concerned with introducing the basic image classification of the Pummel test and the use theoretical frame of AI and its sub-classes of ML and of semantic segmentation for the detection of cracks DL while the specific needs and requirements for the at the cut edge of glass. In the summary and conclu- application in a structural engineering context are high- sion section, the main findings for the applicability and lighted. Then this paper explores potential applications impact of AI for the presented structural glass research of AI for different subjects within the design, verifi- and industry problems are compiled. It can be seen that in many cases AI, data, software and computing M. A. Kraus ( ) · M. Drass resources are already available today to successfully M&M Network-Ing UG(haftungsbeschränkt), implement AI projects in the glass industry, which Lennebergstr. 40, 55124 Mainz, Germany is demonstrated by the many current examples men- e-mail: kraus@mm-network-ing.com; makraus@stanford.edu; tioned. Future research directories however will need to kraus@ismd.tu-darmstadt.de concentrate on how to introduce further glass-specific M. Drass theoretical and human expert knowledge in the AI train- e-mail: drass@mm-network-ing.com; ing process on the one hand and on the other hand more drass@ismd.tu-darmstadt.de pronunciation has to be laid on the thorough digitiza- M. A. Kraus tion of workflows associated with the structural glass Civil and Environmental Engineering, Stanford University, problem at hand in order to foster the further use of AI Y2E2, 473 Via Ortega, Stanford, CA 94305, USA within this domain in both research and industry. M. Drass · M. A. Kraus Institute of Structural Mechancis and Design, Technische Universität Darmstadt, Franziska-Braun-Str. 3, 64287 Darmstadt, Germany 123 248 M. A. Kraus, M. Drass Keywords Artificial Intelligence · AI4BI · Façades · et al. 2016), Keras (Chollet et al. 2015) or PyTorch Design, Computation and Monitoring · Structural (Paszke et al. 2019) provide many necessary func- Glass Engineering tionalities for no monetary cost, the authors consider it essential for a successful application of AI in the engineering sciences and especially in structural glass 1 Introduction engineering that only a reasonable combination of the methodological knowledge of AI and the expert knowl- Artificial Intelligence, or AI for short, is probably the edge of the engineer result in meaningful and valuable term that leads to the most animated discussions today tools. Hence, the intention of this article is threefold: in companies of the tech sector, universities and start- it serves as a short introduction on the background and ups, but also in other low-tech companies with a small definitions of AI technology, based not on a data sci- degree of digitization. Due to a progressing digitization ence background, but on the background of engineers of all sectors of industry (Barbosa et al. 2017; Lam- as AI users; illustrative examples from a wide range propoulos et al. 2019; Schober 2020) while costs of data of glass engineering topics elaborate capabilities and processing and storage steadily decrease (Kurt Peker impact of AI methods on the field; highlighting future et al. 2020; Kraus and Drass 2020a), AI is currently potentials of AI for glass engineering gives an outlook paving its way from the subject of academic consider- on trends and gains. Therefore this paper is structured as ations into both private and professional everyday life follows: First, the basic concepts and nomenclature of in a wide variety of forms. Most people in academia AI, Machine Learning (ML) and Deep Learning (DL) and industry who are not familiar with the field of AI are introduced and explained. Based on the theoretical imagine the technology to be similar to popular sci- background, the second part of the paper reviews suc- ence fiction movies like “Terminator”, “Blade Runner”, cessful applications of AI in glass and façades related “Matrix” or “A.I. Artificial Intelligence”. Today how- fields of science and engineering. In the third part, ever AI is present in everyday’s life in less spectacular current and potentially promising future trends for the and humanoid forms such as spam filters, recommender implementation and application of AI in the glass and systems or digital language assistants such as “Alexa” façade sector are presented. The last section presents a (Amazon) or “Siri” (Apple) (Kepuska and Bohouta summary and conclusions from the findings, together 2018). An impression of the effects of AI on engineer- with an outlook on the future of AI in construction and ing contexts can be gained by looking at the develop- related industries. ments and findings concerning the self-propelled car (Badue et al. 2019). There a great number of questions 2 Basics on AI, machine learning and deep have to be addressed on several levels, ranging from learning ethical and legal concerns w.r.t. reliability of AI and consequences of failure (Holstein et al. 2018; Green- This section provides a non-comprehensive introduc- blatt 2016) to very technical concerns such as the for- tion on the topics of artificial intelligence, machine mulation of learning problems or the processing of a learning and deep learning, whereas a theoretically growing amount of collected data (Hars 2015; Daily more substantial and elaborated description of AI and et al. 2017). A lot of similar questions arise in case of its sub-classes can be found in (LeCun et al. 2015; applying AI in a civil engineering contexts with differ- Binkhonain and Zhao 2019; Dhall et al. 2020; Goodfel- ent pronunciation. However, this paper will show that low et al. 2016; Frochte 2019; Wolfgang 2017; Rebala AI offers many new potentials and certain advantages et al. 2019; Chowdhary 2020). Furthermore, (Goulet over existing methods for the use in civil and especially 2020) gives in particular a textbook-like introduction glass engineering, development and practice. to AI topics with a focus on civil engineering. Basically, the disciplines of statistics, numerics and optimization play a major role in understanding the data, describing the properties of a data set, finding rela- tionships and patterns in these data and selecting and applying the appropriate AI model. Although nowa- days AI software libraries such as Tensorflow (Abadi 123 Artificial intelligence for structural glass engineering 249 2.1 Artificial intelligence, algorithms, models and Strong AI, on the other hand, is supposed to act in a sim- data ilar way to a human being. It should be noted, however, that while strong AI can act operatively like a human Looking at Fig. 1, one can see that AI is the umbrella being, it is likely to have a completely different cogni- term for all developments of computer science, which tive architecture compared to the human brain and will is mainly concerned with the automation of intelligent have different evolutionary cognitive stages (Bostrom and emergent behavior (Chowdhary 2020). 2017; Frochte 2019). With strong AI, machines can Thus, AI is a cross-disciplinary field of research actually think and perform tasks independently, just for a number of subsequent developments, algorithms, like humans do. In conclusion, strong AI-controlled forms and measures in which artificially intelligent machines have a “mind of their own” in a certain way. action occurs, which was presented initially at a con- Accordingly, they can make independent decisions and ference at Dartmouth University in 1956 (Brownlee process data, while weak AI-based machines can only 2011; McCarthy et al. 1956; Moor 2006). AI is ded- simulate or mimic human behavior. Today, we are still icated to the theory and development of computational in the age of weak AI, where intelligent behavior aims systems, which are capable of performing tasks which to do a specific task particularly well or even better require human intelligence, such as visual perception, than humans would do. However, there are more and speech recognition, language translation and decision more efforts by tech giants from the Silicon Valley in making (Brownlee 2011). Hence, a number of AI sub- California to create AI systems that not only perform a fields have emerged, such as Machine Learning (Turing specific task, but solve a wider range of problems and 1950), which historically has focused on pattern recog- make generalizations about a specific problem. Since nition (Marr 2016). Parallel, the first concept of a neural the field of strong AI is still in its infancy, only weak network was developed by Marvin Minsky (Russell and AI and its components are described in more detail and Norvig 2020), which paved the way for deep learning. used within this article. Further details on the historic Interestingly, artificial intelligence so far has had to development of AI and its definition in weak and strong overcome several lean periods (“AI winters”) over the from or neat and scruffy philosophy for AI are skipped years (Crevier 1993), but at this very present time seems at this stage with reference to (Goodfellow et al. 2016; promising for a broad breakthrough of the AI technol- Brownlee 2011; Chowdhary 2020). ogy in several branches as digital, computational and monetary resources are in place to provide fertile con- 2.1.1 Problem formulation ditions (Goodfellow et al. 2016; Schober 2020; Kraus and Drass 2020a). Models and algorithms are essential building blocks for From a computer science aspect, it is distinguished the application of AI to practical problems, where an between weak (or narrow) and strong (or general/super) algorithm is defined as a set of unambiguous rules given AI (Russell and Norvig 2020; Goodfellow et al. 2016; to an AI program to help it learn on its own. (Mitchell Frochte 2019). In particular weak AI deals with con- 1997; Frochte 2019) defines a computer program to crete application problems and their solution, for which learn “from experience E with respect to some class kind of “intelligence” is required from the basic under- of tasks T and performance measure P, if its perfor- standing. Commonly known digital assistants such as mance at tasks in T , as measured by P, improves with Siri from Apple (www.apple.com/sir) and Alexa from experience E.” This definition allows for a wide vari- Amazon (Ale 2020) can be framed weak AI as their ety of experiences E, tasks T , and performance mea- operations is limited to a predefined range of function- sures P. However, in the remainder of this paper, intu- alities. Basically, pre-trained models search for patterns itive descriptions and examples (Sect. 3) of different in a recognized audio sample and classify the spoken kind of tasks, performance measures, and experiences words accordingly in both cases. However, the men- are introduced to construct machine and deep learning tioned two intelligent agents only react to stimuli which algorithms. At this point, some more details on task T they were trained on and show some pre-defined reac- and performance P as well as the role of data are given. tion. So far, these kind of programs do not understand Before going into detail on T and P within this sec- or deduce any meaning from what has been said in a tion, we elaborate further on the experience E and the wider sense, which marks the difference to strong AI. role of data for AI, ML and DL. E is an entire dataset 123 250 M. A. Kraus, M. Drass Fig. 1 Schematic sketch of: a the hierarchy of artificial intelligence, machine learning and deep learning and b The use of data and theory in different settings for physics-informed/theory-guided AI D, whose elements are called data points (or examples 2013). According to (Frochte 2019; García et al. 2016) (Goodfellow et al. 2016)). A data point or example five quantities can be used to characterize a dataset: consists at least of features x ∈ R , where a feature – volume: amount of data is an individual measurable property or characteristic – velocity: rate information arrives of a phenomenon being observed (Bishop 2006; Kuhn – variety: formats of data (structured, semi-structured, and Johnson 2013). The concept of a feature is closely or unstructured) related to what is known as “explanatory/independent – veracity: necessity for pre-processing procedure variable” in statistical techniques such as linear regres- – value: relevance of data for task T sion. Furthermore the features of E may be split further to separate targets/labels y among the remaining fea- While the first three aspects “volume, velocity, and tures x, where an AI algorithm is used to uncover rela- variety” refer to the generation of data, capturing and tionships between the remaining features and the tar- storage process, “veracity” and “value” aspects mark get of the dataset in case of more specific tasks such as the quality and usefulness of the data to the task T under supervised ML and many DL problems. As an example, consideration and hence are crucial for an extraction when using an AI algorithm for linear regression, the of useful and valuable knowledge from the data. If all n m task T is to find a function f : R −→ R , the model five concepts of the proposed list are given to a certain y = f (x) assigns an input (feature) vector x to the extend, the definition of “big data” is met, which is of target vector y. Finally, AI models or algorithms may partial interest for this publication as will be explained possess hyperparameters, which are tunable entities of in Sect. 3. From a technical point of view, the term an AI algorithm [such as regularization strength (cf. “big data” (which may be auto-associated with AI by Sect. 2.1.3) or depth of a neural net (cf. Sect. 2.3)] and the reader) refers to large and complex amounts of data have to be investigated during the learning or training which require “intelligent methods” to process them. phase using a learning algorithm to train and evaluate At this point more detail on the variety of data is given. the best model (Raschka 2018). Structured data is information, which has a pre-defined data model (Frochte 2019; O’Leary 2013; Rusu et al. 2013), i.e. the location of each part of the data as well 2.1.2 Data as the content is exactly know. Semi-structured data is a form of structured data To be able to process data in a meaningful way, it that does not conform with the formal structure of data must first be collected and, if necessary, refined or pre- models associated with relational databases or other processed (Frochte 2019; Bishop 2006; Goodfellow forms of data tables, but nonetheless contains markers et al. 2016; Mitchell 1997; Brownlee 2016; O’Leary to separate semantic elements and enforce hierarchies 123 Artificial intelligence for structural glass engineering 251 Fig. 2 Examples for a Structured data : Table (with example features); b Unstructured Data: Picture of Fractured Glass of records and fields within the data (Frochte 2019; structured and unstructured data), depending on the Rusu et al. 2013). specific glass-related problem under consideration. In Finally, unstructured data is information that either Sect. 3, it is elaborated that unstructured data in the form does not have a predefined data model or does not fit of photographic data is used for quality inspection and into relational tables, (Frochte 2019; Rusu et al. 2013). production control, whereas structured data in form of Typical examples of structured data are databases or simulation data from numerical mechanical investiga- tables, while videos or pictures are classic examples of tions or experiments is used to infer about patterns or unstructured data and further illustration of data struc- model parameters by an AI algorithm. It is known from tures are given in Fig. 2. literature that the combination of the given data set and While for structured data, the feature definition is structure together with appropriately selected AI algo- mostly straight forward due to the structure (Turner rithms provides meaningful results (Goodfellow et al. et al. 1999; Brownlee 2016), feature generation (i.e. 2016; Frochte 2019; Bishop 2006). defining features) for unstructured data is essential Simulation data mining is of particular interest for and the process is also known as feature engineering the numerical investigations within structural glass (Ozdemir and Susarla 2018). Per se, the volume of data engineering (Brady and Yellig 2005; Burrows et al. typically can be tackled with state of the art AI algo- 2011; Frochte 2019). Simulation of data in the field rithms, whereas a huge number of features may become of structural glass engineering is on the one hand often problematic (where it is often referred to as curse of expensive as simulations quickly become both theo- dimensionality (Frochte 2019; Bishop 2006) and one of retically and numerically evolved (and thus the whole the main tasks here is to elaborate a discrimination into dataset comprehends of just a few observations). On the relevant and irrelevant features (Goodfellow et al. 2016; other hand (e.g. in the case of a Finite Element simula- Frochte 2019; Bishop 2006; Kuhn and Johnson 2013). tion) the number of features and targets per simulation In order to tackle that issue, dimensionality reduction as example may be great in number. This poses hardware well as feature selection techniques can be performed requirements along with the need for a feature selec- in order to reduce and/or select features according to tion or engineering strategy. Experimental data on the their relevance for describing the task. More details on other hand usually consist of a limited (small) number that will be given in the ML section of this paper or can of observations together with a small amount of fea- be found in (Kuhn and Johnson 2013; Brownlee 2016; tures due to monetary reasons and the design of the Bishop 2006). respective experiments (Kraus 2019). At this stage some final notes on data types encoun- As a conclusion, for the practical application of AI to tered in the field of structural glass engineering are problems in the glass industry the final choice on algo- given. In this specific field, information generally rithms has to be made on a case by case basis depending can be expected to be either way (structured, semi- 123 252 M. A. Kraus, M. Drass on the task T and the volume, variety and veracity of multiple loss function choices may be useful for mon- the data. itoring model performance, but there is no guarantee that they will result in the same set of optimal model parameters. 2.1.3 Model and loss function 2.1.4 Data splitting Atask T is the description of how an AI should pro- cess data points. An example of a task T is to classify After possible cleansing and visualization of the data, images of test specimens into “intact” and “failed”. The different AI models are evaluated. The main objective performance measure P evaluates the abilities of a AI is to obtain a robust AI model with a good ability to algorithm and often P is related to specifics of the AI generalize well the extracted knowledge to data, which task T . To continue the previous classification exam- were not used during training the model by the learn- ple, a possible performance measure P is the accu- ing algorithm (Mitchell 1997; Goodfellow et al. 2016; racy of the classification model, where accuracy is the Bishop 2006). proportion of examples for which the model produces This means that at the end of the training process, the the correct output (Goodfellow et al. 2016; Brownlee final model should correctly predict the training data, 2011; Mitchell 1997). The choice of a proper perfor- while at the same time it should also be able to gener- mance measure is not straightforward and objective but alize well to previously unseen data. Poor generaliza- dependent on the problem at hand and is thus a solid part tion can be characterized by overtraining or overfitting of the model building part. As this paper is concerned (cf. Sect. 2.1.5), which describes the situation that the with ML and DL examples only, the task T involves a model just memorizes the training examples and is not mathematical model M. When expressing the perfor- able to give correct results also for patterns that were mance measure P in mathematical terms together with not in the training dataset (Mitchell 1997; Goodfel- the notion of learning, the AI algorithm will update a low et al. 2016; Bishop 2006; Frochte 2019). These mathematical model such that for given experience E two crucial demands (good prediction on training data better performance P is gained. This gain measured as well as good generalization abilities) are conflict- via P is conducted via (numerical) optimization, thus ing and also known as the Bias and Variance dilemma alternative nomenclature in ML or DL contexts may (Bishop 2006; Frochte 2019). In order to judge how call P the “objective, loss or cost function” C.This well a ML or DL model performs on data, there exist paper adopts the notation of (Goodfellow et al. 2016), several types of methods for evaluation (i.e. validation) where from a mathematical point of view, a “function (Raschka 2018): we want to minimize or maximize is called the objective function, or criterion”. Especially if the mathematical – holdout validation model M of an AI algorithm possesses parameters θ – k-fold cross validation and learns from training on the data set (or short: data) – stratified K-fold cross validation D, this is formulated as the minimization of a cost func- – leave-one-out cross validation (LOOCV) tion C: The simplest method for validation is holdout vali- θ = arg min C (θ |D), (1) opt dation, in which the data set is split into training and where typical examples of cost functions C are testing data over a fixed percentage value (Goodfellow et al. 2016; Frochte 2019). Using the holdout method is – quadratic/L2 loss: ||y − f (x|θ)|| (least squares), perfectly acceptable for model evaluation when work- – Lp loss: ||y − f (x)|| , ing with relatively large sample sizes (Raschka 2018). – entropy loss Nevertheless, it was shown that the three-way hold- – accuracy. out validation in particular offers advantages. In the It is explicitly emphasized that the definition of the three-way holdout method, available training data may loss function is part of the model building process be split such that an additional validation dataset is within the AI algorithm and will influence the training formed (Russell and Norvig 2020; Bishop 2006). To results to great extent (Goodfellow et al. 2016; Bishop be more specific, the three data sets are used as fol- 2006; Frochte 2019). In addition, for the same T task, lows: 123 Artificial intelligence for structural glass engineering 253 categorical value. This is called stratified cross valida- tion. However, if a small dataset with a feature count of less than 100 is owned, it has been shown that LOOCV provides good results for the accuracy and robustness of the AI models. This approach omits one data point from the training data, i.e. if there are n data points in the original sample, then n − 1 samples are used to train the model and p points are used as a validation set. This is repeated for all combinations where the original sample can be separated in this way, and then the error is averaged for all trials to obtain the overall effectiveness. The number of possible combinations is Fig. 3 Example of a 5-fold cross validation equal to the number of data points in the original sample or n and hence might be computationally expensive in the case of a large dataset. – training dataset: used to fit the model M (70% of Finally, if the readers are interested in detailed |D|) description on different techniques for data split- – validation dataset: used to provide an unbiased eval- ting, hyperparameter tuning, model selection and final uation of a model M fit on the training dataset while deployment of machine learning models, (Raschka tuning model parameters (20% of |D|) 2018; Bishop 2006; Reitermanova 2010; Frochte 2019; – testing dataset: used to provide an unbiased evalua- ) provide detailed and compre- Goodfellow et al. 2016 tion of a final model fit on the training dataset (10% hensive reading for that essential subject. of |D|) 2.1.5 Over- and underfitting All data in the three sets should have a similar dis- tribution for the entire set to ensure that the data are Two central challenges in learning an AI model by from the same distribution and are representative. Com- learning algorithms have to be introduced: under- and mon choices for the sizes of the amount of data (here overfitting. |D|≡ N is the number of data points within the whole A model is prone to underfitting if it is not able training set) are given in the bullet point list (Frochte to obtain a sufficiently low loss (error) value on the 2019; Bishop 2006). training set, while overfitting occurs when the train- To tackle the problem of so-called over- and under- ing error is significantly different from the test or vali- fitting (i.e. the poor generalization capability of the AI dation error (Frochte 2019; Bishop 2006; Goodfellow model, cf. Sect. 2.1.5) cross validation (CV) may be et al. 2016). The generalization error typically pos- applied for hyperparameter tuning and model selec- sesses an U-shaped curve as a function of model capac- tion. CV is a validation technique for assessing how ity, which is illustrated in Fig. 4. Choosing a simpler the results of a statistical analysis will generalize to model is more likely to generalize well (having a small an independent data set (Raschka 2018). The k-fold gap between training and test error) while at the same cross validation for example has a single parameter k, time still choosing a sufficiently complex hypothesis which refers to the number of groups into which a given to achieve low training error. Training and test error data sample is divided. As such, the procedure is often typically behave differently during training of an AI referred to as k-fold cross validation, where the k is model by a learning algorithm (Frochte 2019; Bishop replaced with the specific choice to form the concrete 2006; Goodfellow et al. 2016). Having a closer look at name (e.g. k = 10 becomes a 10-fold cross-validation Fig. 4, the left end of the graph unveils that training error as depicted in Fig. 3). and generalization error are both high. Thus, this marks In contrast, splitting the data into folds can be con- the underfitting regime. Increasing the model capacity, trolled by criteria such as ensuring that each fold con- it drives the training error to decreases while the gap tains the same proportion of observations with a certain between training and validation error increases. Further 123 254 M. A. Kraus, M. Drass increasing the capacity above the optimal will eventu- 2.2 Machine learning ally lead the size of this gap to outweigh the decrease in training error, which marks the overfitting regime. Machine Learning is a sub-branch of AI, which is con- Increasing model capacity tackles underfitting while cerned with algorithms for automating the solution of overfitting may be handled with regularization tech- complex learning problems that are hard to program niques (Frochte 2019; Bishop 2006; Goodfellow et al. explicitly using conventional methods. ML algorithms 2016; Kuhn and Johnson 2013). Model capacity can be build a mathematical model M to infer between quanti- steered by choosing a hypothesis space, which is the set ties of interest (features; targets) based on data to make of functions that the learning algorithm is allowed to predictions or decisions without being explicitly pro- select as being the solution (Goodfellow et al. 2016). grammedtodoso(Frochte 2019; Rebala et al. 2019; Here, varying the parameters of that function family Chowdhary 2020; Murphy 2012). This section provides is called representational capacity while the effective a brief introduction to the most general principles and capacity takes also into account additional limitations nomenclature, a more thorough introduction and elab- such as optimization problems etc. (Goodfellow et al. oration on the subject is given in (Bishop 2006; Good- 2016). fellow et al. 2016; Mitchell 1997; Rebala et al. 2019; Murphy 2012). A basic premise, however, is that the knowledge gained from the data can be generalized 2.1.6 Trends for AI in the engineering and natural and used for new problem solutions, for the analysis of sciences previously unknown data or for predictions on data not measured (prediction). As elaborated in the previous Recent developments in the field AI related to natural as section on AI, ML also has strong ties to optimiza- well as engineering sciences formed the terms physics- tion as learning problems are typically formulated as informed/theory-guided AI, which is a field, where the minimization of some loss function on a training set authors of this paper are also active in, cf. Fig. 1b. The of examples (Frochte 2019; Bishop 2006; Goodfellow aim here is to achieve two goals: et al. 2016; Murphy 2012). Furthermore ML (as well as DL) are closely related to statistics in terms of methods – Compensate data sparsity. but differ in their goal of drawing population inferences – Utilize available theoretical knowledge in a formal from a sample (statistics) vs. finding generalization pre- way. dictive patterns (Bzdok et al. 2018). Training AI models with few data are at the center Two different main algorithm types can be distin- of knowledge inference in the natural and engineering guished for ML: supervised and unsupervised learn- sciences, in contrast to the typical structure of AI in eco- ing (Mitchell 1997; Bishop 2006; Goodfellow et al. nomics or computer science, where very large amounts 2016; Frochte 2019), which are briefly introduced here of data are available for the problem under consider- and graphically illustrated in Fig. 5 (Deep Learning is ation. The reasons for the sparsity of experimental or treated in the next subsection of this paper and Rein- computational data may result on the one hand from forcement Learning is omitted within this paper at all). the fact that they are expensive or the gathering of a In ML, there is a data set D = (x , t ) with N n n n=1 great amount of those data is prohibitive due to time observations, where x is the feature/influence variable or financial constraints. On the other hand, the formal and t the target/response variable. Both variables can introduction and use of pre-existing and already exist- be continuous or discrete (categorical). While in super- ing theoretical knowledge (both from science and from vised learning a predictive model M based on both experts), e.g. in the form of the loss function (Raissi influence and response variables is to be developed, in 2018), leads to shorter familiarization times through unsupervised learning a model is learned only on the meaningful previous starting points for optimization basis of the features (clustering; dimension reduction). within the AI algorithms or the setting of boundary con- For supervised learning, a distinction is made between ditions to the parameters to be derived. Further informa- classification and regression problems. While in the tion can be found in (Reichstein et al. 2019; Karpatne former case the response variables t can only take dis- et al. 2017; Wagner and Rondinelli 2016; Raissi 2018; crete values, the response variables t are continuous Kraus 2019). for regression problems. 123 Artificial intelligence for structural glass engineering 255 Fig. 4 Typical relationship between capacity and error, marking underfitting zone (left) and overfitting zone (right), from (Goodfellow et al. 2016) Fig. 5 Overview on the ML techniques 123 256 M. A. Kraus, M. Drass The goal of regression is to predict the value of one fore may not perform well on new data (Murphy 2012; or more continuous target variables t given the value Bishop 2006; Goulet 2020). Further details on typical of a vector x of input variables, whereas the goal in algorithms such as “Principal Components Analysis classification is to take an input vector x and to assign (PCA)”, “Manifold Learning” or “Autoencoders” are it to one of K discrete classes C where k = 1,..., K skipped within this article with referencing the reader (Bishop 2006). A more detailed description of super- to (Murphy 2012; Bishop 2006; Goulet 2020; Witten vised ML models such as linear and non-linear regres- et al. 2016; Frochte 2019). sion or generalized linear model regression along with A generally valid scheme of steps involved in a suc- classification is omitted within this paper with refer- cessful ML project is presented in Fig. 6. ence to already mentioned ML textbooks. By using Different aspects of Fig. 6 are discussed at this point, regression or classification models, it is furthermore as the conduction of every step is essential for building possible to catch nonlinear and more complex depen- a sensible AI/ML application. In step 1 and 2 existing dencies between the in- and outputs. For further infor- data are compiled and brought in a form that AI/ML mationitisreferredto(Kraus 2019; Bishop 2006; model can access it and the learning algorithm is able to Goodfellow et al. 2016; Mitchell 1997; Lee et al. 2018; train the model on the present data. This step may take Murphy 2012). minutes to months in dependence of the problem and In Fig. 5 on the right hand side, the main categories the data structure of the respective environment. Espe- of unsupervised learning algorithms are given. These cially when digitizing existing older data from paper. algorithms use input data only to discover structure, It is advisable to consider standardization protocols for patterns and groups of similar examples within the data this step in order to guarantee data consistency within (clustering), or to determine the distribution of data a company. It is however important to note that the pre- within the input space (density estimation), or to project dictive power and accuracy of any data-driven model the data from a high-dimensional space down to lower is based on the accuracy and quality of the input data. dimensions (Goulet 2020; Kraus 2019; Bishop 2006; In the context of this paper, step 3 (feature extrac- Goodfellow et al. 2016; Mitchell 1997; Lee et al. 2018). tion) will be briefly discussed, since this represents the Cluster algorithms use similarity or distance measures interface between AI/ML and engineering on the one between examples in the feature space as loss functions hand, and on the other hand it has a significant influ- to discover dense regions of observations (Hastie et al. ence on the quality of the model’s statements and pre- 2009). Clustering algorithms in contrast to supervised dictions. Different strategies for deriving features exist: learning only use a divide-and conquer strategy to inter- Historically, ML uses statistical features obtained pret the input data and find natural groups or clusters by unsupervised learning methods (e.g. cluster analy- in feature space, where a single cluster is an area of sis, dimensionality reduction, autoencoders, etc.), but density in the feature space where data are closer to the as in the context of glass engineering thermomechani- cluster than other clusters (Witten et al. 2016; Bishop cal as well as chemical models exist, the parameters of 2006; Goulet 2020). Typical clustering algorithms are those equations may also serve as features. The num- “k-means” (Lloyd 1982; Goulet 2020; Bishop 2006) ber of features that can be derived from the data is and the “mixture of Gaussians” (Goulet 2020; Bishop theoretically unlimited, but some techniques are often 2006). Similar to clustering methods, dimensionality used for different types of data. For example, the task reduction aims to exploit inherent (latent (Bishop 2006; of feature selection is to extract certain signal prop- Goodfellow et al. 2016; Lee et al. 2018)) structure in erties from, for example, raw sensor data to generate the data in an unsupervised manner to reduce the num- higher-level information. Feature extraction techniques ber of features to a set of principal variables, where in this context are the detection of peaks, the extraction the approaches can be divided into feature selection of frequency contents by Fourier transform, the iden- and feature extraction (Roweis and Saul 2000; Bishop tification of signal trends by sum statistics (mean and 2006). Fewer input dimensions (i.e. number of features) standard deviation at different experimental times), etc. induce fewer parameters or a simpler structure in the Further details on the individual steps can be found in ML model, referred to as degrees of freedom (Mur- (Bishop 2006; Goodfellow et al. 2016; Chang and Bai phy 2012). A model with too many degrees of free- 2018; Kraus 2019; Tandia et al. 2019; MAT 2016c, dom is likely to overfit the training dataset and there- 2016a,2016b). 123 Artificial intelligence for structural glass engineering 257 Fig. 6 Flowchart for the learning process with AI/ML Fig. 7 Schematic sketch showing the principle architectures of: a Feedforward Neural Network (FNN); b Convolutional Neural Network (CNN) 2.3 Deep learning and Yu 2016; Rudy et al. 2019; Baumeister et al. 2018; Mosavi 2019), pattern recognition of radar systems Deep learning is sub-field of ML (Goodfellow et al. (Chen and Wang 2014), face recognition (Hu et al. 2016), which uses so-called artificial neural networks 2015; Sun et al. 2018; Li and Deng 2020), signal clas- as models to recognize patterns and highly non-linear sification (Kumar et al. 2016; Fawaz et al. 2019), 3D relationships in data. An artificial neural network (NN) reconstruction (Riegler et al. 2017), object recognition is based on a collection of connected nodes (the neu- (Rani et al. 2020; Zhao et al. 2019), sequence recog- ron), which resemble the human brain (cf. Fig. 7). nition for gesture (Elboushaki et al. 2020; Gao et al. Today many of architectures of neural nets are known 2020), speech (Yu and Deng 2016; Nassif et al. 2019), (Van Veen 2016), however in the context of this paper handwriting and text (Zheng et al. 2015; Jaramillo et al. only the specific sub-classes of feedforward neural nets 2018), medical diagnostics (Bejnordi et al. 2017; Ker (FNN) and convolutional neural nets (CNN) are of et al. 2017; Greenspan et al. 2016; Liu et al. 2019) and interest, cf. Fig. 7. Details on the specifics of the var- e-mail spam filtering (Guzella and Caminhas 2009). ious other types of NN can be found for example in The FNN is constructed by connecting layers con- (LeCun et al. 2015; Goodfellow et al. 2016). Due to sisting of several neurons, a schematic sketch is shown th their ability to reproduce and model non-linear pro- in Fig. 7. The first layer (0 ) of the FNN is the input N th cesses, artificial neural networks have found applica- layer of dimension R , the last layer (L )isthe output tions in many areas. These include material modeling layer, and the layers in between are called hidden lay- th K and development (Bhowmik et al. 2019; Goh et al. ers (l ). A neuron is an operator that maps R −→ R 2017; Mauro 2018; Mauro et al. 2016; Elton et al. (with K connections to neurons from the previous layer 2019), system identification and control (De la Rosa l − 1) and described by the equation: 123 258 M. A. Kraus, M. Drass ⎛ ⎞ l−1 on details on convolution operations and several pool- l l l−1 l l ⎝ ⎠ ˆ b = σ w b + b := σ b (2) ing strategies along with training approaches for the k kj k k j =1 different kinds of NN, instead the reader is referred to (Bishop 2006; Frochte 2019; Goodfellow et al. 2016). where σ(·) is a monotone continuous function and com- Further well-known NN are recurrent neural networks monly referred to as activation function. The activation (RNNs) for processing sequential data (Graves 2012; is computed as a linear combination of the neurons in Goodfellow et al. 2016), autoencoder for dimensional- the previous layer l −1 given the corresponding weights ity reduction or feature learning (Skansi 2018; Good- l l w and biases b of layer l,cf. Eq.(2) and Fig. 7.The kj k fellow et al. 2016) and many more, which are not sub- choice of connecting the neurons layer wise is user ject of this paper. DL is a supervised learning strategy dependent, if each neuron is connected to every neuron and may need a great amount of data for meaningful in the two neighbor layers, the FNN is called dense or training, depending on the specifics of the problem at densely connected. In summary, FNN represent a spe- hand (Bishop 2006; Frochte 2019; Goodfellow et al. cific family of parameterized maps (depicted by ◦ for 2016). This situation then may prohibit the use of DL the composition operation), which are surjective if the for some applications in research and practice. In sum- output layer possesses linear activation function and mary, model capacity in case of NN is greater com- can be expressed as: pared to ML models in the sense, that the NN as func- tion space allows for more variety than typical function y = f ◦ ... ◦ f (x) (3) n 1 spaces used in ML models. Thus all points raised in Sects. 2.1.3 to 2.1.5 require special considerations in l l−1 l where f = σ w b + b (tensor notation) repre- the DL setting and hyperparameter tuning along with sents the data transform in one layer l. A neuron is validation issues are essential for generalization NN a non-linear, parameterized function of the input vari- models for successful application in the engineering ables (input neurons; green in Fig. 7). A NN is hence context. a mathematical composition of non-linear functions of two or more neurons via an activation function. This particular non-linear nature of NNs thus is able to iden- 3 AI applied to structural glass and related fields tify and model non-linear behaviors, which may not at all or not properly be captured by other ML meth- So far, an introduction on the basics and background on ods such as regression techniques or PCA etc. Despite AI, ML and DL was given and some concepts for model the biological inspiration of the term neural network building, training and validation were introduced. AI is a NN in ML is a pure mathematical construct which a fast-growing technology that has now entered almost consists of either feed forward or feedback networks every industry worldwide and is expected to revolu- (recurrent). If there are more than three hidden layers, tionize not only industry, but also other social, legal this NN is called a deep NN. The development of the and medical disciplines. Specifically the construction right architecture for an NN or Deep NN is problem industry possesses the lowest rate of digitization (Chui dependent and only few rules of thumb exist for that et al. 2018; Schober 2020). Here new technologies are setup (Bishop 2006; Frochte 2019; Kim 2017; Paluszek introduced hesitantly due to the long lifespan of build- and Thomas 2016). Convolutional (neural) networks ing structures and associated reservations or concerns (LeCun et al. 1995; Goodfellow et al. 2016) (CNN) about the risks and reliability of new methods and prod- mark a specialized kind of NN for processing data with ucts due to the lack of experience. However, consider- grid-like topology. Examples include time-series data ing that about 7 % of all employees worldwide work (1-D grid taking samples at regular time intervals) and in the construction sector, there is a considerable mar- image data (2-D grid of pixels). In contrast to FNN, the ket potential in the development and transfer of new CNN employ the mathematical operation called convo- approaches from AI to this sector (Schober 2020). lution, which is a special kind of general matrix multi- Focusing now on structural glass and façade con- plication in at least one of their layers. In addition to the struction within the whole building industry, this convolution, a pooling operation is applied to the data branch is, in contrast to more established branches such between layers. This paper will not further elaborate as concrete or steel construction or bridge design, rela- 123 Artificial intelligence for structural glass engineering 259 tively progressive, innovative and open to technology. 3.1 AI for engineering user-centered adaptive façades This can be demonstrated by numerous projects in the field of façade constructions, such as the use of adaptive The topic of the building envelope or façade has gained façade elements in the building envelope (Shahin 2019; enormous importance in recent years due to the discus- Romano et al. 2018), switchable glass as sun protection sion on sustainability and energy saving (Aznar et al. (Marchwinski ´ 2014; Vergauwen et al. 2013), numerical 2018), where lately the consideration of the user health, modeling of complex adhesively bonded façade ele- well-being, productivity and interaction with the build- ments (Drass and Kraus 2020a), the consideration of ing/façade was added (Luna-Navarro et al. 2020). The time and temperature dependent material behavior of building envelope on the one hand side determines polymeric interlayers in laminated glasses in the intact design and perception of the building for both users and and post-failure state (Kraus 2019), or the paramet- the public while on the other hand, the building enve- ric design of building envelopes (Wang and Li 2010; lope is a significant structural sub-system for occupant Zhang et al. 2019; Granadeiro et al. 2013; Vergauwen comfort and interaction of the user with the environ- et al. 2013). Within this section, the focus is on struc- ment of the building. Interaction of the user with the tural glass and façade construction within the build- envelope so far was either mainly driven by manual and ing industry. In the remainder of this paper different local personal control (e.g. through opening a window areas of interest for the application of AI are identified, or drawing a curtain) or semi-automated by triggered potentials and possibilities are uncovered and outlooks predefined sequences leading to actions (e.g. switch- to future visions are highlighted. In order to charac- able and smart glazing, dynamic shading) (Day and terize the special flavor of the needs and potentials of O’Brien 2017). This led to the situation of occupants applying AI to this specific field of design, engineering often being dissatisfied even in the scenario with con- and products, the authors created the term “Artificial trol strategies and related interactions with automated Intelligence for the Building Industry (AI4BI)”. systems (Luna-Navarro and Overend 2018; Fabi et al. AI has yet been applied in engineering (Quantrille 2017; Borgstein et al. 2018; Day and O’Brien 2017; and Liu 2012; Patil 2016; Bunker 2018), economy (Var- Bluyssen et al. 2013; Meerbeek et al. 2014). Automa- ian 2018), medicine (Szolovits 2019) and other sectors tion of the building in a combination with AI is a for modeling, identification, optimization, prediction promising solution for low-energy buildings through and control of complex systems and/or components a data-driven yet occupant-informed approach consist- thereof (De la Rosa and Yu 2016; Rudy et al. 2019; ing of actuation systems and ubiquitous sensing devices Baumeister et al. 2018; Mosavi 2019). Some review steered by learning AI algorithms. Concepts so far are articles compile the state of the art of AI in civil engi- concerned with a proper design (structural, service- neering as a whole discipline (Huang et al. 2019; Patil ability, sustainability, user well-being) and adaption of et al. 2017; Lu et al. 2012; Adeli 2001) while a huge façades but dismissed the aspects of health monitoring number of publications deal with specific problems as well as structural design requirements for adaptive from the civil engineering field (which in part were structures over the lifespan of the envelope (cf. Fig. 2 already cited so far in this paper), which are not given in (Aelenei et al. 2016) on the characterization con- explicitly here in order due to reasons of brevity. How- cepts of adaptive façades, where structural aspects are ever, especially for the structural glass engineering con- assumed to be per-fullfilled), which are introduced and text a review paper has not yet been published, which discussed within this section. is partially the intention of this contribution. Table 1 Within this article, three major points for the appli- gives an overview of present examples on applying AI cation of AI in the façade engineering context are high- in structural glass engineering as discussed and firstly lighted: presented in this paper. – multi {physics; user} constrained design by/through In the remaining section, the examples will always AI be elaborated according to the scheme of describing the – data-driven {structural adaptivity; health monitor- problem, explaining the traditional engineering solu- ing; predictive maintenance} tions, elaborating new possibilities and added value – intelligent functional {façade; home; office build- due to using AI and judging challenges and difficul- ing} ties related to this approach. 123 260 M. A. Kraus, M. Drass Table 1 Overview and summary table of the examples of this paper on the application of AI in structural glass engineering and related disciplines Used amount of training data: + small; ++ medium; +++ large AI applicability: 0 not shown yet;  success proved Artificial intelligence for structural glass engineering 261 3.1.1 Driving the multi {physics; user; verification tural verification software, etc.) within the design, plan- code} constraints for AI in the façade ning and verification process in civil engineering and the heterogeneity of associated partners (usually small Basic design principles for civil engineering struc- companies with no formal protocol on a digital work- tures enforce very stringent safety and serviceability flow) in a design and construction project. Furthermore criteria which assume extreme loading and resistance due to keeping competitive advantages many compa- situations, which occur with very low probabilities, nies do not want to share or make publicly accessible hence these structures are over-designed for most of technical solutions in form of a database. their service lives (Akadiri et al. 2012; Senatore et al. 2018). The structural adaption philosophy on the other 3.1.2 Steps towards AI in façade engineering hand reduces material and energy consumption of the building construction through a paradigm of control- A fully digital workflow upon the Building Information ling strength and stiffness in real-time via sensing and Modeling (BIM) (Borrmann et al. 2015; Isikdag 2015) actuation to carry the acting loads (Wada 1989; dos approach for the whole life cycle of a building solves Santos et al. 2015; Wagg et al. 2008; Fischer et al. the digitization problem and allows AI algorithms to be 2009). Over the last couple of years several adaptive applied in several forms. BIM is a digital description façade systems were researched. On the one hand side of every aspect of a construction project and nowadays a “structure focused” branch considered either shap- practiced to some extend in the construction industry. ing façade elements (e.g. thin glass) (Amarante dos The initial idea of BIM is a 3D information model Santos et al. 2020; Silveira et al. 2018; Louter et al. formed from both graphical and non-graphical data, 2018) or enabling rigid façade-components to be adap- which are compiled in a shared digital space called tive (Schleicher et al. 2011; Svetozarevic et al. 2019) (Common Data Environment; CDE). All information while diminishing sustainability and comfort. On the on that specific building is constantly updated as time other hand a “sustainability and user-centered” branch progresses during the life cycle of a building and thus considered strategies for either predefined levels of sus- ensures the model to always be up-to-date (Serrano tainability, energy saving and user comfort by design or 2019). However, BIM today still suffers from techni- allowed for user-control strategies to address occupant- cal challenges across disciplines such as architectural building-interaction in addition to sustainability con- design, structural verification, building physics design, cerns. Taking into account the statements of this and maintenance measurements etc. and a full digital work- the preceding paragraph leads to the conclusion, that flow with AI components from the early first sketches adaptive façades have to be modeled as a multi-criteria to demolition of a building (Ghaffarianhoseini et al. optimization problem with highly nonlinear and impre- 2017; Vass and Gustavsson 2017; Akponeware and cise (in the fuzzy sense; for user/occupant modeling) Adamu 2017) is not yet possible. For the application correlations, which a priori may not be known to a cer- of AI in that multi-criteria optimization and control tain extend (especially the user well-being part of the problem as described earlier in this section, there is equation) or have to be “learned” from data of experi- need of a cyber-physical twin (digital twin, computable ments (e.g. multi-occupant requirements; features from structural model) (Boschert and Rosen 2016; Borrmann multi-sensor measurements). et al. 2015; Raj and Evangeline 2020) within the life For the design stage AI and ML can be used to infer cycle-accompanying BIM paradigm. The digital twin suitable technical solutions to given tasks under consid- is a digital image of a physical system which is heavily eration in an early design stage (AI assisted design and used in industry so far to reduce operational errors and planning) with a potential check for planning errors or optimize product design. The starting point for a digital unlikely verification success of the designed solution. twin is a highly accurate three-dimensional model that The main problem for an immediate introduction and contains all the features and functions of the physical application of AI here is the low digitization rate (Chui system, including material, sensor technology or even et al. 2018; Schober 2020) (especially details for older dynamics of the real structure. The parametric design buildings are highly likely documented on paper rather approach in architecture (Monedero 2000; Wortmann than in a digital ML readable format), the high varia- and Tunçer 2017; Oxman 2017) is a first step in these tion of data formats (CAD formats, formats of struc- directions and seems very suitable for a connection to 123 262 M. A. Kraus, M. Drass Fig. 8 Schematic overview on an intelligent façade with health monitoring capabilities AI as it currently uses optimization algorithms, e.g. with both systems may be prohibitive due to monetary genetic algorithms etc. which are are sub-groups of AI. reasons. However, in buildings to be designed and con- Due to the fact that structural verification is solidly structed from scratch, an integrated approach imple- based on mechanics and theory, the application of AI menting the two functionalities can be considered. In in the verification stage during design of structures is the remainder of this subsection, some background and very likely to be successful as through mechanics it is potential realization outlooks are given. guaranteed to hit a certain solution manifolds of the Both mentioned ideas are rooted in the data-driven problems which itself induces manifolds of feasible approach to identification, control and steering of struc- design solution (which is in contrast to the view of tural systems. There, ML is a rapidly developing field data analysis, where there is a priori no knowledge that is transforming the ability to describe complex sys- of the process by which data is created). First expe- tems from observational data rather than first-principle riences with automating design reviews with AI in a modeling. While for a long time, ML methods were BIM context is delivered by (Sacks et al. 2019), where restricted to the application to static data, more recent building models are checked for conformance to code research concentrates on using ML to characterize clauses of simple form (explicit formulations; implicit dynamical systems, (Brunton and Kutz 2019). Espe- and complex clauses are still beyond the scope of such cially the use of ML to learn a control function, i.e. to applications). determine an effective map from sensor outputs to actu- The combination of health monitoring/predictive ation inputs is most recent. In this context, ML methods maintenance and an intelligent façade/home/building for control include adaptive NN, genetic algorithms and are schematically visualized in Fig. 8 and can be imple- programming and reinforcement learning. mented within one and the same façade project. The The second mentioned issue for an intelligent façade reason for distinguishing these two situations is due to or building is similarly treatable form a mathematical the fact, that the deployment is in dependence of the and AI point of view (Aznar et al. 2018; Luna-Navarro needs of the building owner or user (cf. comments on et al. 2020). Similar ML methods apply in this con- multi-criteria optimization problem earlier in this para- text as well. The overall idea is that given a reasonable graph) and both systems work individually on a partly and suitable loss function, i.e. a function, which is able shared data basis (cf. comments on BIM and the dig- to correctly describe the well-being and comfort of a ital twin earlier in this paragraph). Especially for the user, the façade or building is able to learn the specific situation with existing façade structures the retrofitting domain of comfort for the individual user by training an 123 Artificial intelligence for structural glass engineering 263 AI algorithm for a reference state and continuous user take and when. The structural health monitoring (SHM) feed back about the well-being. Through that approach, thus predicts the future performance of a component it will be possible to provide maximum user comfort by assessing the extent of deviation or degradation of a with minimal invasiveness. system from its expected normal operating conditions If supervised ML or DL algorithms are applied, (NOC) (Brunton and Kutz 2019). The inference of the a loss function (characterizing the control and steer- NOC is based on the analysis of failure modes, detec- ing problem) has to be established in the mentioned tion of early signs of wear and aging and fault condi- contexts. In addition the development of suitable and tions. This is the bottleneck of the AI approach to façade meaningful features, which allow a structurally sen- monitoring, as it is necessary to have initial information sible and unambiguous classification of the condition on the possible failures (including site, mode, cause, of the façade under consideration, is necessary (Aznar and mechanism), which for new façade systems can et al. 2018; Luna-Navarro et al. 2020). For example, the only be learned “on the fly” after installation of the construction-physical principles and interrelationships façade. However, with a data-driven approach, a certain for describing the comfort of the user as a result of exter- initial training phase (e.g. 5 years) can be implemented nal influences and their manipulation, e.g. by control- as a training and identification period for the AI to learn ling the light-directing or heating systems, are already the NOC and to detect derivations of it (Fig. 9). known today in theory, but to date these have not been Concluding this paragraph, AI together with a BIM- taken into account in any approach to the “smart home”. embedded digital twin has the potential to enhance the This is particularly due to the fact that (analogous to built environment with occupant interaction to form the “Internet of Things”) mechanical engineering in sustainable intelligent buildings and façades and hence particular has so far been concerned with the network- deliver satisfying human-centered environments. How- ing of machines and devices without incorporating the ever, more research is needed to build the multi-criteria knowledge specific to civil engineering with regard to loss function for the AI control system via a holistic and the interaction of people and buildings. An AI can be multidisciplinary approach. extended here by building physics criteria and evalu- ate the user data in such a way that a building (living space/work use) learns the preferences of the respective 3.2 AI in glass product development, production and user over time via the diverse data streams and adapts to processing the user. This idea goes far beyond the currently exist- ing approaches of “smart home”, so that a conceptual Glass and façade construction is highly technological in delimitation of the “intelligent home/office” becomes the area of industrial development, production and fur- obvious. ther processing of glass, as glass is a brittle material and Similarly, structural features, such as deflections or inferior quality in processing can lead to fatal events in accelerations, may serve as sensible features together their assembly, construction and/or operation (Schnei- with some signal statistic features to describe well the der et al. 2016; Sedlacek 1999; Siebert and Maniatis structural behavior of a monitored façade under con- 2012). Consequently, high-precision machines are used trol of an AI. In the health monitoring situation, addi- for glass production and processing to enhance the brit- tional information has to be given to the AI algorithm tle material in such a way that it exhibits high qualities. in order to enable it to predict the remaining lifetime Starting from washing the glass, cutting and breaking, or inspection intervals, which is known in mechani- thermal/chemical tempering and lamination to form a cal engineering as predictive/prescriptive maintenance laminated (safety) glass (Schneider et al. 2016; Sed- (Brunton and Kutz 2019). In order to make the nomen- lacek 1999; Siebert and Maniatis 2012). The machine clature clear, predictive maintenance employs sensors technology for glass refinement is constantly being data to precisely collect data describing the conditions improved and optimized to meet customer-specific of an asset and overall operational state. The data are requirements. Currently available established methods then analyzed for prediction of future failure events and either fail or are worse in comparison to AI technolo- their occurrence times. Prescriptive maintenance takes gies, which can be integrated here. The following exam- this analysis to a further state of maturity as it not only ples elaborate in more detail the use of AI for faster and predicts failure events but also recommends actions to more systematic improvements in production and man- 123 264 M. A. Kraus, M. Drass Fig. 9 Online Fault Diagnosis System, from (Niu 2017) ufacturing of glass products. Within this section, four putations for technical products and an estimation of different applications of AI for production and quality the properties by these methods still are prohibitively management of glass are highlighted: expensive and time consuming. From a mathematical point of view, composition of the design of new glasses – Glass Product Development can be seen as a multi-objective optimization problem – Inspection and Control of Laminated Glass with many constraints, which can be easily handled by – Semantic Segmentation of Cut Glass Edges an ML approach. (Hill et al. 2016) – Strength Prediction based on Cutting Process Param- Having at hand significant computing capabilities, eters data-mining algorithms, an efficient data storage infras- tructures and an (publicly) available materials database 3.2.1 AI for data-driven glass product development enables researchers to discover new functional mate- rials by AI within significantly lower temporal and Today, there is increasing demand on highly-functional, monetary efforts than in conventional processes. The manufacturable and inexpensive glasses (Tandia et al. development of new products by data-driven AI meth- 2019), which has led glass researchers to use data- ods relies on the establishing or existence of accessible driven machine learning models to accelerate the devel- databases, which in practice hardly exist for the public opment of glasses and glass products instead of tradi- but do on the level of individual companies or research tional trial-and-error approaches. In this context, data- groups. A very mature compilation of publicly avail- driven materials discovery approaches use statistical able materials databases for model and glass product models as well as ML algorithms, which are trained, development is given in (Tandia et al. 2019). tested, and validated using materials databases. An An example for the data-driven development of a important part of this approach is to develop or access new type of glass is shown in the following, which accurate materials databases at low cost. While it is was presented by (Tandia et al. 2019). In that exam- in principle possible to use first principles approaches ple, the two most important properties for glass design (thermochemical/thermodynamical simulations such are liquidus temperature T and viscosity η, where the as ab initio calculations based on quantum mechanics, glass liquidus temperature is defined as the temper- density functional theory, molecular dynamics, or lat- ature at which the first crystalline phase precipitates tice models etc. (Van Ginhoven et al. 2005; Benoit et al. from the melt of a given glass composition when the 2000)) to compute electronic band structure, formation melt is cooled with very small rate and the viscosity is energy and other thermodynamic parameters, the com- 123 Artificial intelligence for structural glass engineering 265 Fig. 10 a Grey box fitting of temperature-dependent MYEGA perature dependent viscosity with NNs using BO to find the best viscosity with NN using a single layer with eight neurons and architecture to code the MYEGA equation. Both from (Tandia tanh as an activation function on a single layer; b fitting of tem- et al. 2019) relevant for the targeted sheet thickness in the produc- It was found, that the combined NN-MYEGA equa- tion phase. Still today no accurate and generalizable tion approach resulted in a sufficiently accurate pre- physics-based models for glass melt liquidus tempera- diction model with low error in the validation set com- ture or melt viscosity for industrial glasses is known, pared to other models and thus the development of new thus the application of DL is a viable strategy for the glass compositions was possible within significantly development of a predictive model for both liquidus lower time at less money. Further details on this spe- temperature and viscosity. cific example can be found in (Tandia et al. 2019) while Among other ML techniques, a NN is trained for (Mauro et al. 2016; Mauro 2018) provide further appli- the prediction of both parameters, which is presented cation cases of AI for glass material development. in more detail within the context of this paper. The NN as well as the predictive capabilities are shown in 3.2.2 AI for inspection and control of glass products Fig. 10. Due to reasons of brevity, only the AI mod- eling of the viscosity η is described in detail, as this Building products and pre-fabricated building compo- quantity drives the glass thickness within the produc- nents currently have to full fill certain national and tion process. In the approach of (Tandia et al. 2019), the international standards to ensure a minimum level MYEGA model was used in combination with a NN. of reliability and uniformity of these products across The MYEGA model possesses the form: nations (Schneider et al. 2016; Siebert and Maniatis 2012). New production technologies such as additive manufacturing together with new strategies for achiev- B C log η = A + exp , (4) ing the requirements of building regulations demand T T automation of material quality testing with little human intervention to ensure repeatability and objectivity of in which A is negative while B and C are positive con- the testing process. In the status quo of quality control stants. A Bayesian optimization (BO) framework was of glass and glass products, visual inspections are often used for inference of the model parameters (number of carried out by humans to evaluate, for example, the layers, number of neurons in each layer, learning rate, cleanness of the glass, the quality of cut edges (Bukieda activation function etc.). et al. 2020), anisotropy effects caused by thermal tem- 123 266 M. A. Kraus, M. Drass pering of glass (Illguth et al. 2015) or to determine the mer (pummel). The Pummel value is then estimated by degree of adhesion between interlayer and glass (Franz a human inspector based on the surface area of poly- 2015). In these existing testing protocols, the assess- mer interlayer exposed after pummeling (cf. Fig. 11- ment and judgment of a human tester is required to left). Further details on the Pummel test can be found quantify the degree of reaching requirements of build- in (Schneider et al. 2016; Beckmann and Knackstedt ing regulations, hence the human quality quantifica- 1979; Division 2014). tion results are prone to non-negligible statistical uncer- Traditional image-based computer vision methods tainty through the human tester decisions, (Wilber and for evaluating the Pummel test extract image features Writer 2002). Applying AI in the field of production using complex image pre-processing techniques, which control of glass and glass products hence seems promis- in the experience of the authors based on conducted ing for reaching above-human level accuracy in qual- investigations on Pummel test pictures so far marked ity inspection based on objectification, systematization the main difficulties with these approaches. On the one and automation. This approach was already proved suc- hand, the proper choice of a performance metric on the cessful in related scientific fields involving AI and espe- pummel images (e.g. certain quantiles of the cumula- cially DL for computer vision (i.e. how computers can tive distribution function of grey-values or full color gain high-level understanding from digital images or spaces of the images), which is invariant under the videos), where it clearly outperformed humans in sev- widely varying real-world situations for taking such eral areas (Voulodimos et al. 2018; Ferreyra-Ramirez a Pummel image with thin cracks, rough surface, shad- et al. 2019). ows, non-optimal light-conditions in the room of pum- Visual inspections for quality management are typi- mel inspection etc., is demanding and led to no clear cally organized in an inspection process (determined favorable function. On the other hand, the access to in many cases by national or international building just a limited amount of labeled training image data regulations), which probes the whole production pro- formed another obstacle. To address these challenges, cess through several human-based controls of product- this paper proposes an AI-based classification tool (AI- specific quality measures. Since humans in principle Pummel Tool), which uses a deep convolutional neu- are unable to provide an objective result of a quality ral network on grey-value images of pummeled glass control due to their own bias (Nordfjeld 2013), uncer- laminates to completely automate pummel evaluation tainty in objectification and repeatability of the qual- while excluding human bias or complex image pre- ity measures is induced. It is thus preferable to supply processing. a technological solution in the form of combining AI Following the data-driven approach of AI, in Fig. 11 and computer vision to automate the quality inspec- a schematic illustration of the workflow for an AI-based tion while minimizing human intervention. Within the automated pummel classification is given. It relies on scope of this paper an example for the objectifica- the input of grey-value images after pummeling the tion, systematization and automation of a visual prod- laminated glass. These pictures are then processed by uct inspection for laminated glasses by the so-called a pre-trained deep CNN for classification into one of the Pummel test is presented. The Pummel test specifically 11 Pummel value categories. Details on the principal characterizes the degree of adhesion between the poly- architecture of CNNs were already given in Sect. 2.3, meric interlayer and the glass pane of a glass laminate, further details on CNNs especially within the field of where an optical scale ranging from 0 to 10 character- computer vision are not described here in detail with izes the level of adhesion. The resulting Pummel value reference to (LeCun et al. 2015; Voulodimos et al. thus delivers an indicator for the quality and safety 2018). Using this approach, the standard human-based properties of laminated glass, where a value of 0 quan- classification of pummel images into the Pummel cat- tifies no adhesion and 10 very high adhesion (Beck- egories during production control is therefore automa- mann and Knackstedt 1979; Division 2014). The lam- tized and objectified by using the pre-trained CNN for inated glass specimen for the Pummel test consist of prediction of the Pummel class along with a statistically two float glass panes with a maximum thickness of 2 sound quantification of uncertainties of this process. × 4 mm. The specimens are exposed to a climate of Since only a few labeled Pummel image data were −18 C for about 8 h and subsequently get positioned available for training the CNN, the authors used image on an inclined metal block and processed with a ham- data augmentation to expand the training data set. First 123 Artificial intelligence for structural glass engineering 267 Fig. 11 Schematic workflow for the AI supported evaluation of the Pummel test results show a prediction probability of the correct clas- with a quantification of the improvement of the perfor- sification of the pummel value of over 80 %, cf. Fig. 13. mance and robustness of the CNN and further investi- However, a significant performance gain is expected if gations on alternative architectures or even alternative more actual labeled Pummel image data is available in approaches such as clustering (Jain et al. 1999) has to the next step of this project. In order to show the perfor- build upon future studies with an increasing amount of mance of the AI Pummel tool, first validation results are ground-truth Pummel images. illustrated in Fig. 12, where the Pummel image to clas- sify is shown together with the CNN-based prediction 3.2.3 AI prediction of cut-edge of glass via semantic of the Pummel value as well as the Pummel value deter- segmentation mined by manufacturer (ground-truth Pummel value) is also shown. In the production and further processing of annealed Figure 12 together with Fig. 13 proves, that the AI float glass, glass panes are usually brought into the Pummel tool is very well able to generalize, i.e. to cor- required dimensions by a cutting process. In a first step, rectly classify Pummel images which were not used a fissure is generated on the glass surface by using a during the training of the CNN. The accuracy of the cutting wheel. In the second step, the cut is opened classification algorithm within the context of this paper along the fissure by applying a defined bending stress. is measured via the confusion matrix (also known as This cutting process is influenced by many parameters, error matrix ) (cf. Fig. 13). Each row of the matrix rep- where the glass edge strength in particular can be repro- resents the Pummel values predicted by AI, while each ducibly increased by a proper adjustment of the process column represents the actual Pummel value defined by parameters of the cutting machine (Ensslen and Müller- the manufacturer (ground-truth). The name confusion Braun 2017). It could be observed that due to differ- matrix stems from the interpretation of the algorithm, ent cutting process parameters, the resulting damage to here a CNN-classifier, confusing two classes. Interpre- the edge (the crack system) can differ in its extent. In tation of the confusion matrix of the CNN is interesting, addition, this characteristic of the crack system can be as for the pummel value classes 1, 3, 5, 7, 9 the accu- brought into a relationship with the strength (Müller- racy of the prediction is over 92%. The worst prediction Braun et al. 2018). In particular, it has been found that result is obtained for the Pummel value class of 6, where characteristics of the lateral cracks, cf. Fig. 14 viewing an accuracy of 63% was found. On the other hand, the the edge perpendicular to the glass surface, allow best results give rise to questioning the qualitative scale of predictions for the glass edge strength (Müller-Braun 11 classes to be lumped into e.g. 5 or 6 Pummel classes. et al. 2020). However, since the training of the CNN was based on a The challenge here is, however, to detect these lat- small amount of publicly available data, more theoret- eral cracks and the related geometry in an accurate and ical justification for this Pummel class lumping along objective way. Currently, this is conducted by man- 123 268 M. A. Kraus, M. Drass Fig. 12 Example Results of the AI Pummel tool for different Pummel input images (3 successes, 1 misclassification) 123 Artificial intelligence for structural glass engineering 269 In (Drass et al. 2020) AI and especially the prob- lem of semantic segmentation was for the first time applied to identify the cut edges of cut glass to auto- matically generate mask images. The goal is to process an image of a glass cut edge using the DL model U- Net (Ronneberger et al. 2015) in such a way that a mask image is generated by the model without explic- itly programming it to do so. For the problem at hand, the segmentation of the images of cut edges of glass is into two classes “breakage” (black) and undamaged glass (white), i.e. a binary segmentation, is conducted using the U-Net architecture, which is shown in Fig. 15. Accordingly, the mask image should only recognize the cut glass edge from the original image and display it in black in the mask image. More details on the the U-Net Fig. 13 Representation of the confusion matrix for the problem of AI-based prediction of the pummel value architecture, the learning algorithm and hyperparame- ter tuning is given in (Drass et al. 2020). As shown in Fig 16, the trained U-Net is well suited ual tracing due to the fact that the crack contour can to create a mask image from the original image with- sometimes only be recognized roughly by eye. After out the need for further human interaction. It is also manually marking the crack using an image processing obvious that the red-yellow areas, where the NN is program, the contour is then automatically evaluated not sure whether it sees the cut edge or just the pure further. Methods of AI and especially the algorithms glass, are very narrow and hence of minor importance. from the field of AI in computer vision now can be uti- A slight improvement of the mask images created by lized as an alternative to the existing manual approach AI could be achieved by the cut-off condition or binary to automate the step of manual detection of the glass prediction. The presented NN for predicting the cut cut edge. In addition to the enormous time and hence glass edge is therefore very accurate and saves a sig- economic savings, the objectivity and reproducibility nificant amount of time in the prediction and production of detection is an important aspect of improvement. of mask images. In addition, the mask images can be The topic of image classification in the context of com- further processed, for example to make statistical anal- puter vision and DL is well known (Ferreyra-Ramirez yses of the break structure of the cut glass edge. This et al. 2019). As stated in the previous section of this however is not part of the present paper and will not be paper, image classification is concerned with classify- further elaborated hence. ing images based on its visual content. The proposed model as described briefly here and Whilst the recognition of an object is trivial for in detail in (Drass et al. 2020) showed excellent results humans, for computer vision applications robust image for the prediction of the cut glass edge. The validation classification is still a challenge (Russakovsky et al. accuracies of both models exceeded 99 %, which is 2015). An extension of image classification is object sufficient for the generation of the mask image via AI. detection (i.e. enclosing objects by a frame or box within an image). Object detection often just requires a coarse bounding of the object within an image, but in 3.2.4 AI prediction of glass edge strength based on the case at hand it is desirable to extract the contours process parameters of an object as exactly as possible. Semantic segmen- tation in contrast to object detection describes the task This section deals with the prediction of the edge of classifying each individual pixel in an image into a strength of machine-cut glass based on the process specific class (Guo et al. 2018). The task of semantic parameters of the cutting machine using supervised segmentation processes image data in such a way that ML, more specifically an Extra Trees regressor, which an object to be found is segmented or bordered by a is also known as Extremely Randomized Trees (Geurts so-called mask. et al. 2006). 123 270 M. A. Kraus, M. Drass Fig. 14 (1) View on the cut edge of two 4 mm thick glass spec- eral crack to be detected: The crack contour can be difficult to imens, a slight crack system, breaking stress: 78 MPa, b more identify Drass et al. (2020) pronounced crack system, breaking stress: 53 MPa and (2) Lat- Fig. 15 U-Net architecture for the problem of image segmentation of cut glass 123 Artificial intelligence for structural glass engineering 271 Fig. 16 Results of the semantic segmentation using U-Net to predict the cut glass edge on the basis of three test images (axes are in [mm]) 123 272 M. A. Kraus, M. Drass Based on the investigations of (Müller-Braun et al. 2020), architectural glass is cut in two steps. First, a slit is created on the glass surface using a cutting tool and a cutting fluid. An integral part of the cutting tool is the cutting wheel. It is available in various dimensions, although the manufacturers make basic recommenda- tions regarding the cutting wheel angle and cutting wheel diameter for different glass thicknesses and cut- ting tasks. After cutting the glass, it must still be broken by applying some bending to the pane in order to obtain two separate pieces of glass of desired dimension. It is quite known from experience, that the edge strength of cut glass depends significantly on the applied process Fig. 17 Residual plot [MPa] for the AI-Predictor to determine parameters during the cutting process, proved by sim- the edge strength as a function of the process parameters of a ple graphical and statistical evaluation of experimental glass cutting machine for training and test set data in (Müller-Braun et al. 2020), where more details on the background of cutting and breaking glass as well as the experimental investigations of the cutting process on the edge strength of cut glass can be found. However, so far no concrete modeling approach was formulated Other parameters have been included in the test and trained on the data given the complexity of the data series, which are not described in detail here due to correlations. reasons of brevity. A total of 25 features were included In order to deliver a prediction model of the edge for the entire test series. After applying the Boruta fea- strength in dependence of the process parameters of the ture selection algorithm (Kursa et al. 2010), 12 of the glass cutting machine, this paper suggests a ML regres- 25 features could be classified as unimportant, so that sion model, correlating the process parameters of cut- the regression model was trained with a ML algorithm ting to the edge strength target value. With this model, it on a reduced number of 13 features. is possible for the first time to predict the edge strength The model used for this example is an Extra in dependence of the process parameters with high sta- Trees regressor (also known as Extremely Random- tistical certainty without performing destructive tests ized Trees) (Geurts et al. 2006), which is similar to on the cut glass. Providing this AI-based method deliv- a Random Forest regressor. SciKit-Learn (Pedregosa ers remarkable economic and ecological advantages. A et al. 2011; Buitinck et al. 2013) was used together with lot of manpower required for testing the glass is saved the default hyperparameter settings for the Extra Trees along with saving resources by avoiding great amounts regressor without further investigation on the hyper- of glass waste material by non-destructive testing. parameter tuning. The single holdout method has been The main parameters of the cutting process can be applied for splitting the data in training and testing data. summarized as follows: Figure 17 show the residuals (in MPa) between actual and predicted edge strength separately for the training and validation data. Given the R = 0.88 in Fig. 17 – test temperature it is concluded, that the obtained model describes the – relative humidity data well and the scatter is due to the dimension reduc- – glass thickness tion from 25 to 13 features. On the other hand side – glass height from the validation data performance it is concluded, – cutting speed that alternative models calibrated with ML algorithms – cutting force might be more suitable to better represent the data and – type of cutting fluid the presented Extra Trees regress may lack of overfit- – type of cutting wheel ting. A future paper will investigate in more detail an AI – cutting wheel angle based model for predicting the cut glass edge strength – ... (Fig. 18). 123 Artificial intelligence for structural glass engineering 273 for obtaining the parameters of the Helmholtz potential in a Bayesian manner is posed and calibrated. In the context of hyperelasticity, the isochoric or volume-constant Helmholtz free energy function Ψ may be written in form of the Nelder function as I − 3 1,b Ψ b = , (5) x + x I − 3 1 2 ¯ 1,b where I = tr b characterizes the first isochoric, 1,b principal invariant of the left, isochoric Cauchy-Green tensor b. An extension of Eq. 5 by the second invariant of the left Cauchy-Green tensor leads to I − 3 I − 3 ¯ ¯ Fig. 18 Edge strength as a function of the process parameters 1,b 2,b Ψ b = iso,ND of a glass cutting machine predicted by the AI Predictor versus x + x I − 3 x + x I − 3 1 2 ¯ 3 4 ¯ 1,b 2,b the experimentally obtained ground-truth (6) 3.3 AI in structural glass engineering which is the final form of the novel Helmholtz free energy function. As can be concluded from Eq.(6), the proposed This last subsection will highlight and discuss several Helmholtz energy possesses four parameters to be cal- applications of AI in the field of structural glass engi- ibrated θ = {x , x , x , x }. neering. In the context of this paper, two examples on 1 2 3 4 an already successfully conducted application of ML The experimental data of the transparent silicone was presented in (Drass et al. 2018a, b), where the mate- techniques on problems of that field and afterwards two visions for further incorporation of AI are given. Other rial was experimentally characterized in uniaxial ten- sion and compression, shear and biaxial deformation. applications of AI within structural engineering (such as design and verification of buckling for steel hol- The second structural silicone is a carbon black filled silicone adhesive, which was investigated by (Staudt low sections or computation of deflection or bending moment fields of a Kirchhoff plate etc.) were recently et al. 2018) for uniaxial tension and simple shear load- ing. The third material to be investigated is filled elas- published in (Kraus and Drass 2020a). tomer from the tire industry, which has been character- ized under tensile and shear loads (Lahellec et al. 2004). 3.3.1 Example 1: Bayesian calibration of a Helmholtz All material parameters have been determined using potential for hyperelasticity of TSSA silicone Bayesian supervised ML algorithms upon the DREAM MCMC algorithm (Vrugt 2016), cf. Fig. 19. Within the TM Within advanced analysis of polymeric materials in context of this paper only the results for DOWSIL structural glass engineering constitutive models have TSSA are presented for reasons of brevity. to be applied which are able of capturing the non- As can be seen from Fig. 19 a, the presented novel linear stress-strain relationship adequately. In (Drass hyperelastic model Ψ and the extended tube iso,ND 2019; Drass and Kraus 2020b), a novel functional form model are well suited to represent the experimental for the free Helmholtz energy for modeling hyperelas- data of TSSA for four different types of experiments. ticity was introduced and calibrated for various poly- It is interesting that the MCMC simulation on a stan- meric materials, especially for structural silicones such dard laptop lasted about 20 minutes and led to results TM TM as DOWSIL TSSA or DOWSIL 993 as well as (mean values of the parameters) that were very close glass laminate interlayers Poly-Vinyl-Butyral (PVB) to the smallest squares determined with the software and Ethylen-Vinyl-Acetate (EVA) by traditional opti- MATHEMATICA, although MCMC means a signifi- mization techniques. Here, a supervised ML problem cantly higher numerical effort compared to the small- 123 274 M. A. Kraus, M. Drass Fig. 19 Fitting results for TSSA silicone by different approaches the parameters θ of the proposed Helmholtz potential for TM TM and experiments under arbitrary deformations: a DOWSIL DOWSIL TSSA ; with UT = uniaxial tension, UC = uni- TSSA by parameter mean values of the Bayesian optimiza- axial compression, BT = biaxial tension, SPC = shear pancake tion supervised learning b uni- and bivariate distributions of and SPC = shear pancake tests est squares. Finally, it is emphasized, that by applying tern depends on the elastic strain energy density U the Bayesian framework, further deduction of partial and thus the magnitude of the residual stress induced material safety factors is straight forward as the uncer- by a thermal pre-stressing procedure. This is shown tainties in the associated model parameters are natu- in Fig. 20 for thermally tempered glass for differ- rally captured. Hence the application of Bayesian ML ent residual stress levels. It can be clearly observed in this context delivers addition insight on the certainty that the fragment size increases with decreasing resid- about the model parameters as well as model assess- ual stress level. Approaches up to now related mean ment quantities for further use in a reliability analysis quantities such as fracture particle weight, area con- at no extra cost compared to traditional optimization- tent or circumference (Pourmoghaddam and Schneider based material calibration strategies. 2018). To determine the characteristics of fragmentation, an ML algorithm named BREAK was developed in 3.3.2 Example 2: Bayesian reconstruction and (Kraus 2019). The model there combines an energy cri- prediction of glass breakage patterns (BREAK) terion of linear elastic fracture mechanics (LEFM) and the spatial statistical analysis of the fracture pattern of This section deals with the application of ML to cal- tempered glass in order to determine characteristics of ibrate a ML surrogate of the fragmentation pattern the fragmentation pattern (e.g. fragment size, fracture of thermally pre-stressed glasses along with its spa- intensity, etc.) within an observation field. The model- tial characteristics (such as e.g. fragment size, fracture ing approach is based on the idea that the final fracture intensity, etc.) via stochastic tessellations over random pattern is a Voronoi tessellation induced by a stochas- Strauss Point processes as initially suggested by (Kraus tic point process (a Strauss process in the context of 2019). this paper). The parameters of that model are calibrated Several studies on the fragmentation behavior of from statistical analysis of images of several fractured tempered glasses have proven relationships between glass samples. By calibration of that stochastic point the residual stress state, the glass thickness and the process and consecutive tessellation of the region of fragment density (Akeyoshi and Kanai 1965; Lee interest, statistically identically distributed realizations et al. 1965; Sedlacek 1999; Mognato et al. 2017; of fracture patterns can be generated. Further details Pourmoghaddam and Schneider 2018). The fragment on the theoretical background as well as the deriva- density or fracture intensity in an observation field, tion of the model specifics for several point processes the fragment shape and thus the entire fracture pat- 123 Artificial intelligence for structural glass engineering 275 Fig. 20 Fragment size of thermally tempered glass as a function of the residual stress (indication of the biaxial tensile residual stress in the mid-plane) at a plate thickness of t =12mm(Pourmoghaddam et al. 2018) within this semi-supervised ML approach is given in 3.4.1 Designed by AI (Kraus 2019). The schematic connections of the theo- ries and experiments involved for BREAK are given in In this first visionary section two points will be pre- Fig. 21. sented and elaborated: To show explicit results of the BREAK algorithm, a glass plate with thickness of t = 12 mm and a defined – design supported by AI degree of pre-stress of σ = 31.54 MPa (U = 8.754 – structural analysis supported by AI m 0 J /m ) was analysed. After the morphological pro- cessing of the fracture images, the first order statis- There are first publications dealing with the applica- tics of the extracted point pattern were determined in tion of AI in architecture and design, cf. (Mrosla et al. the first step to infer the point process intensity. After 2019; Newton 2019; Baldwin 2019), in which all note the model parameters had been calibrated on the basis that the examples of an AI-generated built environment of the recorded fracture pattern photos, the simula- existing today still need further years of research and tion of statistically equivalent fracture patterns was cooperation between the different fields to achieve the performed using the calibrated Strauss process with announced quality. induced Voronoi tessellation. An exemplary realization For example, (Baldwin 2019) proposed a floor plan based on the mean values of the model parameters is design method by Generative Adversarial Networks shown in Fig. 22. (GAN), cf. Fig. 23. Where a GAN is a special form This application proved, that a combination of AI of NN from the family of NN as presented in Sect. 2.3, algorithms for regression and computer vision enable more details on GANs may be found in (Goodfellow to model more complicated geometrical-numerical et al. 2016; Frochte 2019). dependencies such as glass fracture patterns while car- The GAN floor plan design pipeline uses image rep- rying statistical features of its components, which was resentations of plans as data format for both, GAN- not possible by traditional approaches. models’ inputs and outputs, where Pix2Pix is used as GAN geared towards image-to-image translation. The careful study of the organization learned by each model revealed the existence of a deeper bias, or architectural style. The project aimed to assist the architect in gen- 3.4 Outlook and future vision 1: AI for design and erating a coherent room layout and furnishing and to computation of structures finally reassemble all apartment units into a tentative floor plan The project also included the conversion of Within this section, a visionary outlook on the status floor plans from one style to another. quo and potential capabilities of AI in the fields of A future vision of design by AI based on the works designing and structural verification and computation presented here is the combination of the existing GAN of structures is given. with customer features such as preferences for colors, 123 276 M. A. Kraus, M. Drass Fig. 21 Schematic figure of the BREAK framework, showing the connections of experimental observations to the elements of spatial point patterns and linear fracture mechanics, from (Kraus 2019) Fig. 22 Simulation of a realization of a fracture pattern of a glass pane with thickness of t = 12 mm and level of pre-stress σ = 31.54 MPa with the calibrated SP, from (Kraus 2019) 123 Artificial intelligence for structural glass engineering 277 Fig. 23 Generation pipeline of designed floor plans by GANs, from (Baldwin 2019) shapes etc. By this, a customized design by GANs can ward simulations are conducted to collect the struc- be reached to a high consumer satisfaction level. tural responses given different combinations of design covariates. Here, some aspects have to be considered 3.4.2 Structural verification supported by AI especially: (1) the definition of the prior mean func- tions, (2) covariance and correlation functions and (3) In the context of the structural verification of cer- the formulation required for modeling cases involving tain structural members or in early design stages of a heteroscedastic errors (Goulet 2020). project, AI and its capabilities of establishing surrogate Figure 24b compares the Gaussian process regres- models can be utilized to provide fast conclusions on sion model predictions μ with the true finite element the structural feasibility of a designed structure without model outputs y . In order to obtain a meaningful com- explicit computation. parison between predicted and measured values, it is Surrogate modeling without and with AI methods essential to test the model iteratively using a cross- concern people within the computational mechanics validation procedure whereas at each step, the obser- and optimization communities since several years. An vation corresponding to the prediction location in the comprehensive overview is given by (Forrester et al. validation set is removed from the training set. A further 2008; Adeli 2001; Wortmann et al. 2015). The need for example of providing an AI-based surrogate for fast surrogates in engineering analysis stems from employ- and reliable structural design and verification of steel ing computationally demanding methods such as the hollow sections was recently published by the authors finite element models for analysis, presented in Fig. 24. in (Kraus and Drass 2020a) but it is not further elab- Surrogate modeling in practical terms means that the orated at this stage. As a conclusion, using AI based costly and time-consuming finite element model is surrogates for structural verification provides the com- replaced by a regression model build upon a set of sim- puting structural engineer a fast and reliable method ulated responses. Because observations for the training to check design alternatives or to conduct sensitivity of the surrogate are obtained by the output of a simula- analyses. Furthermore, transferability of the surrogate tion, the observation model does not include any obser- results is reached if a proper formulation of the engi- vation error (except the discretization error is consid- neering problem at hand is done a priori and lets further ered as observation noise). Within this paper, the Gaus- pay of a typically demanding training phase of the sur- sian process regression for the construction of meta- rogates. models for the responses of a structure (cf. Fig. 24a) To summarize this section, AI has the potential to using covariates and a set of simulations is discussed. accelerate design and structural verification processes In the context of surrogate modeling, the engineer to a great demand while customization wishes may has to specify the relevant responses given certain enter more naturally and affordably. The authors are covariates (i.e. design variables), then a number of for- 123 278 M. A. Kraus, M. Drass Fig. 24 Examples of numerically cheap AI based surrogates: a Examples of numerically demanding Finite Element Models; b Training of Gaussian Process as a cheap AI based surrogates. Both from (Goulet 2020) currently at a stage, where first knowledge and experi- thetic polymers in civil engineering is scientifically, ences are gathered with these ideas. Further research of technically and economically highly relevant. Thus, the the authors will consider more building-practical appli- development and safe design of novel structures in vari- cations of the presented ideas. ous fields such as architecture/construction, automotive engineering and aerospace is possible. A data-driven material modeling approach by the 3.5 Outlook and future vision 2: data-driven modeling earlier mentioned physics-informed/theory-guided AI of materials within glass-structures approach is particularly interesting as especially for engineering, in contrast to material sciences (a rather Especially in glass and façade construction, modern big data environment), constitutive models for design materials such as a great variety of polymers are used, have to be created and calibrated mostly on the basis but their constitutive modeling is much more com- of a few experiments (usually a small data environ- plicated than established building materials due to ment). The incorporation of physical laws and theoret- their thermomechanical properties (Kraus 2019; Drass ical knowledge there is of special interest. The develop- 2019). For more than ten years by now, a wide range ment of a reliable, methodologically sound and gener- of experimental and methodical work has been pro- alizable derivation of constitutive laws on the basis of viding the basis for an improved understanding of the techniques of AI and in particular of deep NN from material and load-bearing behavior of these materials, experimental data thus requires a systematic analy- whereby the latest methods place the highest demands sis of the relevant mechanisms, influence parameters on the engineer and the tools available such as finite and modeling strategies regarding the techniques of element software (Drass and Kraus 2020c; Kraus and AI, which can only succeed in a good symbiosis of Drass 2020b; Kraus 2019; Drass 2019). AI-supported the knowledge of the disciplines of material sciences, modeling of the complex constitutive behavior of these civil engineering, numerics and optimization as well as materials is one of the latest developments in AI related computer science. The overall goal of a recent research computational mechanics research as the realistic sim- project of the authors is the development of a validated ulation and design of polymeric components in civil and reliable methodology for the selection and calibra- engineering requires knowledge of the relevant mech- tion of suitable artificial intelligence models for mod- anisms of load transfer, failure and aging (if applica- ular thermodynamically consistent constitutive model- ble) and their effect on the load-bearing behavior. The ing of polymeric materials in civil engineering using methodical handling of the relevant processes and the experimental and simulation-based data. With such a transfer into modern numerical models for the reli- method, complicated material models can be estab- able simulation of the constitutive behavior of syn- 123 Artificial intelligence for structural glass engineering 279 lished on the basis of data from standard experiments of data in an engineering context is limited due to mon- and simulations to capture hyperelastic, viscoelastic etary or confidentiality reasons and thus the establish- and damage effects. As the framework is general, it is ing of publicly accessible databases is hardly possible not only restricted to the mentioned polymer silicone for a greater audience, however on the level of individ- and glass laminated polymers but would apply to any ual companies or research groups, the data stock prob- new material in the field. lem is not severe or even present. Finally a visionary outlook on the role of AI within supporting engineers for an early stage design of structures, the modeling 4 Summary and conclusions of advanced material behavior by physics-informed AI approaches as well as AI-based structural verification Within this paper the reader was introduced to the main surrogates finished the paper. concepts of and a brief background on Artificial Intelli- Within this paper the following conclusions are gence (AI) and its sub-groups Machine Learning (ML) reached: and Deep Learning (DL). The nomenclature along with – AI-supported control of adaptive façades will poten- the meaning of AI core vocabulary on the task T , tially solve the multi-criteria optimization prob- the performance measure P and experience E were lem involving economy, sustainability and user- introduced and illustrated via examples. Furthermore well being a detailed elaboration on the importance of splitting – several examples of a successful application of AI available data into training, validation and test set was in the field of structural glass engineering were pro- given, which then was followed by underlining over- vided and proofed superior compared to existing and underfitting of models during training by AI algo- approaches rithms and strategies to avoid either of that problems. – for the first time ever, AI models made it possible Then two sections on the basic nomenclature and care- to establish numerical predictions for phenomena fully chosen models from ML and DL were presented to such as glass fracture patterns or cut edge strength the reader. In the main part of the paper, a review and – the amount of available data for training AI mod- summary of already successfully conducted applica- els is often limited and hence constrict attainable tions of AI in several disciplines such as medicine, natu- model accuracies and generalizations, e.g. for the ral sciences, system identification and control, mechan- Pummel test as well as the cut edge strength exam- ical as well as civil engineering. A total of six core sec- ples tions then introduced and explained in detail problems – AI-based models can easily be enhanced by uncer- out of structural glass engineering, where AI meth- tainty quantification methods to establish reliability ods enabled training a model at all or were superior statements, e.g. in the case of polymeric materials compare to traditional engineering models. The glass for structural glass engineering or the accuracy of engineering applications range from accelerated glass predicting the Pummel value product development, deep learning based quality con- – future research and industry potentials ly in the trol of glass laminates for the Pummel test, prediction elaboration of AI-empowered design and verifi- of the cutting edge of glass together with prediction cation support systems, which enable to consider of the strength of the cut glass edge to the calibra- user/occupant demands as well as structural relia- tion of a Helmholtz potential and the prediction of the bility and serviceability already in early planning fracture pattern of thermally pre-stressed glass. For all stages examples the amount of necessary data together with the challenges and final solution strategy was reported In principle, we consider the introduction of AI tech- to enable the reader to judge temporal and monetary nologies in glass and façade construction and its neigh- effort for AI methods in comparison to existing engi- boring industries to be possible to a great extend already neering models and approaches (in case these are exist- immediately, since the essential basis of an AI, i.e. the ing). Here several points for taking care or differences existence of data, is already fulfilled in many cases. in an (structural façade and glass) engineering context A first task for research and especially industry will to traditional computer science approaches to AI where now be to structure the existing data in such a way highlighted. It was stated that especially the availability that AI algorithms can apply, train and validate diverse 123 280 M. A. Kraus, M. Drass models on it in order to lead to successful projects in C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Tal- war, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vie- combination with engineering expertise. The theoreti- gas, F.,Vinyals,O., Warden,P., Wattenberg,M., Wicke, cal framework and the respective software are in place M., Yu, Y., Zheng, X.: TensorFlow: Large-Scale Machine but have to be augmented by the knowledge of struc- Learning on Heterogeneous Distributed Systems (2016) tural/civil engineers, who are familiar with the statisti- arXiv:1603.04467 Adeli, H.: Neural networks in civil engineering: 1989–2000. cal and methodical concepts of AI. For this reason, in Comput.-Aided Civil Infrastruct. Eng. 16(2), 126–142 the eyes of the authors of this paper, it is essential to (2001) introduce these methodological knowledge and prac- Aelenei, D., Aelenei, L., Vieira, C.P.: Adaptive façade: con- ticing with AI in the study curricula of students of civil cept, applications, research questions. Energy Procedia 91(Supplement C), 269–275 (2016) engineering and architecture in the near future as well. Akadiri, P.O., Chinyio, E.A., Olomolaiye, P.O.: Design of a sus- tainable building: A conceptual framework for implement- Acknowledgements We would like to express our greatest ing sustainability in the building sector. Buildings 2(2), 126– gratitude to our numerous industry and research partners for 152 (2012) fruitful discussions on the topic and providing samples of mate- Akeyoshi, K., Kanai, E.: Mechanical Properties of Tempered rial or data. Especially the support by Stanford University (Prof. Glass. VII International Glass Congress (paper 80) (1965) Christian Linder, PhD @ CEE) as well as TU Darmstadt (Prof. Akponeware, A.O., Adamu, Z.A.: Clash detection or clash avoid- Dr.-Ing. Jens Schneider @ Fachgebiet Statik) along with their ance? an investigation into coordination problems in 3d bim. great academic guidance and support is highly appreciated. May Buildings 7(3), 75 (2017) this article and our research further impact the civil and structural Amarante dos Santos, F., Bedon, C., Micheletti, A.: Explo- glass engineering community. rative study on adaptive facades with superelastic antago- nistic actuation. Struct. Control Health Monit. 27(4), e2463 Compliance with ethical standards (2020) Aznar, F., Echarri, V., Rizo, C., Rizo, R.: Modelling the thermal behaviour of a building facade using deep learning. PloS Conflicts of interest The authors certify that they have NO affil- one 13(12), e0207616 (2018) iations with or involvement in any organization or entity with any Badue, C., Guidolini, R., Carneiro, R.V., Azevedo, P., Cardoso, financial interest or non-financial interest in the subject matter or V.B., Forechi, A., Jesus, L., Berriel, R., Paixão, T., Mutz, F., materials discussed in this manuscript. et al.: Self-driving cars: A survey (2019). arXiv:1901.04407 Baldwin, E.: Ai creates generative floor plans and styles with Open Access This article is licensed under a Creative Com- machine learning at harvard (2019) URL https://www. mons Attribution 4.0 International License, which permits use, archdaily.com/918471/ai-creates-generative-floor-plans- sharing, adaptation, distribution and reproduction in any medium and-styles-with-machine-learning-at-harvard/ or format, as long as you give appropriate credit to the original Barbosa, F., Woetzel, J., Mischke, J., Ribeirinho, M.J., Sridhar, author(s) and the source, provide a link to the Creative Com- M., Parsons, M., Bertram, N., Brown, S.: Reinventing Con- mons licence, and indicate if changes were made. The images or struction: A Route to Higher Productivity. McKinsey Global other third party material in this article are included in the article’s Institute (2017) Creative Commons licence, unless indicated otherwise in a credit Baumeister, T., Brunton, S.L., Kutz, J.N.: Deep learning and line to the material. If material is not included in the article’s Cre- model predictive control for self-tuning mode-locked lasers. ative Commons licence and your intended use is not permitted by JOSA B 35(3), 617–626 (2018) statutory regulation or exceeds the permitted use, you will need Beckmann, R., Knackstedt, W.: Process for the production of to obtain permission directly from the copyright holder. To view modified, partially acetalized polyvinyl alcohol films U.S. a copy of this licence, visit http://creativecommons.org/licenses/ Patent No. 4,144,376. 13 Mar. (1979) by/4.0/. Bejnordi, B.E., Veta, M., Van Diest, P.J., Van Ginneken, B., Karssemeijer, N., Litjens, G., Van Der Laak, J.A., Hermsen, M., Manson, Q.F., Balkenhol, M., et al.: Diag- nostic assessment of deep learning algorithms for detection References of lymph node metastases in women with breast cancer. Jama 318(22), 2199–2210 (2017) (2016a) Applying Supervised Learning. MathWorks Benoit, M., Ispas, S., Jund, P., Jullien, R.: Model of silica glass (2016b) Applying Unsupervised Learning. MathWorks from combined classical and ab initio molecular-dynamics (2016c) Introducing Machine Learning. MathWorks simulations. Eur. Phys. J. B-Condens. Matter Complex Syst. (2020) URL www.alexa.com 13(4), 631–636 (2000) (2020) URL www.apple.com/siri Bhowmik, A., Castelli, I.E., Garcia-Lastra, J.M., Jørgensen PB, Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, Winther, O., Vegge, T.: A perspective on inverse design C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghe- of battery interphases using multi-scale modelling, experi- mawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., ments and generative deep learning. Energy Storage Mate- Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, rials (2019) J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, 123 Artificial intelligence for structural glass engineering 281 Binkhonain, M., Zhao, L.: A review of machine learning algo- Chollet, F., Others (2015) Keras. https://github.com/fchollet/ rithms for identification and classification of non-functional keras requirements. Expert Syst. Appl. X 1, 100001 (2019). Chowdhary, K.R.: Introducing artificial intelligence. In: https://doi.org/10.1016/j.eswax.2019.100001 Linnhoff-Popien, C., Zaddach, M., Grahl, A. (eds.) Fun- Bishop, C.M.: Pattern Recognition and Machine Learning, Infor- damentals of Artificial Intelligence, Springer, Berlin, pp. mation science and statistics, 1st edn. Springer, Berlin 1–23 (2020) (2006) Chui, M., Manyika, J., Miremadi, M.: What ai can and can’t do Bluyssen, P.M., Oostra, M.A., Meertins, D.: Understanding (yet) for Your Business. McKinsey Quarterly, (2018) the indoor environment: How to assess and improve Crevier, D.: AI: The Tumultuous History of the Search for Arti- indoor environmental quality of people? In: Proceedings of ficial Intelligence. Basic Books Inc, USA (1993) CLIMA 2013: 11th REHVA World Congress & 8th Inter- Daily, M., Medasani, S., Behringer, R., Trivedi, M.: Self-driving national Conference on IAQVEC“ Energy Efficient, Smart cars. Computer 50(12), 18–23 (2017) and Healthy Buildings”, Prag, Czech Republic, 16-19 June Day, J.J.K., O’Brien, W.: Oh behave! survey stories and lessons 2013, Citeseer (2013) learned from building occupants in high-performance build- Borgstein, E., Lamberts, R., Hensen, J.: Mapping failures in ings. Energy Res. Soc. Sci. 31, 11–20 (2017). https://doi. energy and environmental performance of buildings. Energy org/10.1016/j.erss.2017.05.037 Build. 158, 476–485 (2018) De la Rosa, E., Yu, W.: Randomized algorithms for nonlinear Borrmann, A., König, M., Koch, C., Beetz, J.: Building Informa- system identification with deep learning modification. Inf. tion Modeling: Technologische Grundlage und Industrielle Sci. 364, 197–212 (2016) Anwendungen. Springer, Berlin (2015) Dhall, D., Kaur, R., Juneja, M.: Machine learning: A review of the Boschert, S., Rosen, R.: Digital twin—the simulation aspect. In: algorithms and its applications. In: Singh, P.K., Kar, A.K., Mechatronic futures, Springer, Berlin, pp. 59–74 (2016) Singh, Y., Kolekar, M.H., Tanwar, S. (eds.) Proceedings of Bostrom, N.: Superintelligence. Dunod, Paris (2017) ICRIC 2019, pp. 47–63. Springer International Publishing, Brady, T.F., Yellig, E.: Simulation data mining: a new form of Cham (2020) computer simulation output. In: Proceedings of the Winter Division, K.C.L.P.: Pummel Test Standards. URL https://www. Simulation Conference, 2005., IEEE, p 5 ff (2005) trosifol.com/fileadmin/user_upload/TROSIFOL/support/ Brownlee, J.: Clever Algorithms: Nature-Inspired Programming downloads/technical_information/PUMMEL_Book.pdf Recipes. Machine Learning Mastery (2011) (2014) Brownlee, J.: Machine Learning Mastery With Python: Under- dos Santos, F.A., Rodrigues, A., Micheletti, A.: Design and stand Your Data, Create Accurate Models, and Work experimental testing of an adaptive shape-morphing tenseg- Projects End-to-End. Machine Learning Mastery (2016) rity structure, with frequency self-tuning capabilities, using Brunton, S.L., Kutz, J.N.: Data-driven Science and Engineering: shape-memory alloys. Smart Materials Struct. 24(10), Machine Learning, Dynamical Systems, and Control. Cam- 105,008 (2015) bridge University Press, Cambridge (2019) Drass, M., Berthold, H., Kraus, M.A., Müller-Braun, S.: Seman- Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, tic segmentation with deep learning: Detection of cracks at A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., the cut edge of glass (under review). Glass Structures & Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Engineering (2020) Varoquaux, G.: API design for machine learning software: Drass, M., Kraus, M.: Dimensioning of silicone adhesive joints: experiences from the scikit-learn project. In: ECML PKDD Eurocode-compliant, mesh-independent approach using the fem (under review). Glass Structures & Engineering (2020a) Workshop: Languages for Data Mining and Machine Learn- ing, pp. 108–122 (2013) Drass, M., Kraus, M.A.: Rational hyperelastic material model Bukieda, P., Lohr, K., Meiberg, J., Weller, B.: Study on the optical for various engineering application polymers submitted for quality and strength of glass edges after the grinding and publication (2020b) polishing process. Glass Structures & Engineering (2020). Drass, M., Kraus, M.A.: Semi-probabilistic calibration of a par- https://doi.org/10.1007/s40940-020-00121-x tial material safety factor for structural silicone adhesives - Bunker, S.: Artificial Intelligence: Building Smarter Machines part i: Derivation. Int. J. Struct. Glass Adv. Mater. Res. 4, (2018) 56–68 (2020c). https://doi.org/10.3844/sgamrsp.2020.56. Burrows, S., Stein, B., Frochte, J., Wiesner, D., Müller, K.: 68 Simulation data mining for supporting bridge design. Pro- Drass, M.: Constitutive modelling and failure prediction of sili- ceedings of the ninth Australasian data mining conference- cone adhesives in facade design. Doctoral thesis, Technis- volume 121, 163–170 (2011) che Universität Darmstadt (2019) Bzdok, D., Altman, N., Krzywinski, M.: Points of significance: Drass, M., Schwind, G., Schneider, J., Kolling, S.: Adhesive con- statistics versus machine learning (2018). https://doi.org/ nections in glass structures-part i: experiments and analytics 10.1038/nmeth.4642 on thin structural silicone. Glass Struct. Eng. 3(1), 39–54 Chang, N.B., Bai, K.: Multisensor Data Fusion and Machine (2018a). https://doi.org/10.1007/s40940-017-0046-5 Learning for Environmental Remote Sensing. CRC Press, Drass, M., Schwind, G., Schneider, J., Kolling, S.: Adhe- USA (2018) sive connections in glass structures-part ii: material Chen, S., Wang, H.: Sar target recognition based on deep learn- parameter identification on thin structural silicone. Glass ing. In: 2014 International Conference on Data Science and Struct. Eng. 3(1), 55–74 (2018b). https://doi.org/10.1007/ Advanced Analytics (DSAA), IEEE, pp 541–547 (2014) s40940-017-0048-3 123 282 M. A. Kraus, M. Drass Elboushaki, A., Hannane, R., Afdel, K., Koutti, L.: Multid-cnn: Greenblatt, N.A.: Self-driving cars and the law. IEEE Spectr. A multi-dimensional feature learning approach based on 53(2), 46–51 (2016) deep convolutional networks for gesture recognition in rgb- Greenspan, H., Van Ginneken, B., Summers, R.M.: Guest edito- d image sequences. Expert Syst. Appl. 139, 112,829 (2020) rial deep learning in medical imaging: Overview and future Elton, D.C., Boukouvalas, Z., Fuge, M.D., Chung, P.W.: promise of an exciting new technique. IEEE Trans. Med. Deep learning for molecular generation and optimization- Imaging 35(5), 1153–1159 (2016) a review of the state of the art (2019). arXiv preprint Guo, Y., Liu, Y., Georgiou, T., Lew, M.S.: A review of semantic arXiv:1903.04388 segmentation using deep neural networks. Int. J. Multimed. Ensslen, F., Müller-Braun, S.: Kantenfestigkeit von floatglas in Inf. Retr. 7(2), 87–93 (2018) abhängigkeit von wesentlichen schneidprozessparametern. Guzella, T.S., Caminhas, W.M.: A review of machine learn- ce/papers 1(1), 189–202 (2017) ing approaches to spam filtering. Expert Syst. Appl. 36(7), Fabi, V., Spigliantini, G., Corgnati, S.P.: Insights on smart home 10206–10222 (2009) concept and occupants’ interaction with building controls. Hars, A.: Self-driving cars: The digital transformation of mobil- Energy Procedia 111, 759–769 (2017) ity. In: Marktplätze im Umbruch, Springer, Berlin pp 539– Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: 549 (2015) Deep learning for time series classification: a review. Data Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Sta- Mining Knowl. Discov. 33(4), 917–963 (2019) tistical Learning: Data Mining, Inference, and Prediction. Ferreyra-Ramirez, A., Aviles-Cruz, C., Rodriguez-Martinez, E., Springer, Berlin (2009) Villegas-Cortez, J., Zuñiga-Lopez, A.: An improved con- Hill, J., Mulholland, G., Persson, K., Seshadri, R., Wolverton, volutional neural network architecture for image classifi- C., Meredig, B.: Materials science with large-scale data and cation. In: Mexican Conference on Pattern Recognition, informatics: unlocking new opportunities. Mrs Bull. 41(5), Springer, Berlin pp 89–101 (2019) 399–409 (2016) Fischer, M., Wüchner, R., Bletzinger, K.U.: Computa- Holstein, T., Dodig-Crnkovic, G., Pelliccione, P.: Ethical and tional framework for simulation and design of adaptive social aspects of self-driving cars (2018). arXiv:1802.04103 lightweight structures. In: IV ECCOMAS Thematic Confer- Hu, G., Yang, Y., Yi, D., Kittler, J., Christmas, W., Li, S.Z., ence on Smart Structures and Materials 2009, Porto (2009) Hospedales, T.: When face recognition meets with deep Forrester, A., Sobester, A., Keane, A.: Engineering design via learning: an evaluation of convolutional neural networks surrogate modelling: a practical guide. Wiley, New York for face recognition. In: Proceedings of the IEEE interna- (2008) tional conference on computer vision workshops, pp 142– Franz, J.: Untersuchungen zur Resttragfähigkeit von gebroch- 150 (2015) enen Verglasungen: Investigation of the residual load- Huang, Y., Li, J., Fu, J.: Application of artificial intelligence in bearing behaviour of fractured glazing, vol. 45. Springer, civil engineering. Comput. Model. Eng. Sci. 121(3), 845– Berlin (2015) 875 (2019) Frochte, J.: Maschinelles Lernen: Grundlagen und Algorithmen Illguth, M., Schuler, C., Bucak, Ö.: The effect of optical in Python. Carl Hanser Verlag GmbH Co KG, (2019) anisotropies on building glass façades and its measurement Gao, X., Jin, Y., Dou, Q., Heng, P.A.: Automatic gesture recog- methods. Front. Archit. Res. 4(2), 119–126 (2015) nition in robot-assisted surgery with reinforcement learning Isikdag, U.: Enhanced building information models. Springer- and tree search (2020). arXiv:2002.08718 Briefs in Computer Science, pp. 577–590 (2015) García, S., Ramírez-Gallego, S., Luengo, J., Benítez, J.M., Her- Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. rera, F.: Big data preprocessing: methods and prospects. Big ACM Comput. Surv. 31(3), 264–323 (1999). https://doi.org/ Data Anal. 1(1), 9 (2016) 10.1145/331499.331504 Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Jaramillo, J.C.A., Murillo-Fuentes, J.J., Olmos, P.M.: Boost- Mach. Learn. 63(1), 3–42 (2006) ing handwriting text recognition in small databases with Ghaffarianhoseini, A., Tookey, J., Ghaffarianhoseini, A., Nai- transfer learning. In: 2018 16th International Conference smith, N., Azhar, S., Efimova, O., Raahemifar, K.: Building on Frontiers in Handwriting Recognition (ICFHR), IEEE, information modelling (bim) uptake: Clear benefits, under- pp. 429–434 (2018) standing its implementation, risks and challenges. Renew. Karpatne, A., Watkins, W., Read, J., Kumar, V.: Physics-guided Sustain. Energy Rev. 75, 1046–1053 (2017) neural networks (pgnn): An application in lake temperature Goh, G.B., Hodas, N.O., Vishnu, A.: Deep learning for compu- modeling (2017). arXiv:1710.11431 tational chemistry. J. Comput. Chem. 38(16), 1291–1307 Kepuska, V., Bohouta, G.: Next-generation of virtual personal (2017) assistants (microsoft cortana, apple siri, amazon alexa and Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT google home). In: 2018 IEEE 8th Annual Computing Press, USA (2016) and Communication Workshop and Conference (CCWC), Goulet, J.A.: Probabilistic Machine Learning for Civil Engineers. IEEE, pp. 99–103 (2018) MIT Press, USA (2020) Ker, J., Wang, L., Rao, J., Lim, T.: Deep learning applications in Granadeiro, V., Duarte, J.P., Correia, J.R., Leal, V.M.: Building medical image analysis. IEEE Access 6, 9375–9389 (2017) envelope shape design in early stages of the design process: Kim, P.: Matlab deep learning. Mach. Learn. Neural Netw. Artif. Integrating architectural design systems and energy simu- Intell. 130, 21 (2017) lation. Autom. Construct. 32, 196–209 (2013) Kraus, M.A., Drass, M.: Semi-probabilistic calibration of a par- Graves, A.: Sequence transduction with recurrent neural net- tial material safety factor for structural silicone adhesives - works (2012) arXiv:1211.3711 part ii: Verification concept. Int. J. Struct. Glass Adv. Mater. 123 Artificial intelligence for structural glass engineering 283 Res. 4, 10–23 (2020b). https://doi.org/10.3844/sgamrsp. review and classification scheme. Building and Environ- 2020.10.23 ment p 106880 (2020) Kraus, M.A.: Machine learning techniques for the material Luna-Navarro, A., Overend, M.: Towards human-centred intel- parameter identification of laminated glass in the intact and ligent envelopes: A framework for capturing the holistic post-fracture state. PhD thesis, Universität der Bundeswehr effect of smart façades on occupant comfort and satisfac- München (2019) tion (2018) Kraus, M., Drass, M.: Künstliche intelligenz im bauingenieurwe- Marchwinski, ´ J.: Architectural evaluation of switchable glazing sen - hintergründe, status quo und potentiale. Bauingenieur technologies as sun protection measure. Energy Procedia (2020a) 57, 1677–1686 (2014) Kuhn, M., Johnson, K.: Applied Predictive Modeling, vol. 26. Marr, B.: A short history of machine learning–every manager Springer, Berlin (2013) should read (2016). Forbes http://tinyurl.com/gslvr6k Kumar, S., Sharma, A., Mamun, K., Tsunoda, T.: A deep learn- Mauro, J.C.: Decoding the glass genome. Current Opinion in ing approach for motor imagery eeg signal classification. In: Solid State Mater. Sci. 22(2), 58–64 (2018) 2016 3rd Asia-Pacific World Congress on Computer Sci- Mauro, J.C., Tandia, A., Vargheese, K.D., Mauro, Y.Z., Smed- ence and Engineering (APWC on CSE), IEEE, pp. 34–39 skjaer, M.M.: Accelerating the design of functional glasses (2016) through modeling. Chem. Mater. 28(12), 4267–4277 (2016) Kursa, M.B., Rudnicki, W.R., et al.: Feature selection with the McCarthy, J., Minsky, M., Rochester, N.: The dartmouth summer boruta package. J. Stat. Softw. 36(11), 1–13 (2010) research project on artificial intelligence. Artificial intelli- Kurt Peker, Y., Rodriguez, X., Ericsson, J., Lee, S.J., Perez, A.J.: gence: past, present, and future (1956) A cost analysis of internet of things sensor data storage on Meerbeek, B., te Kulve, M., Gritti, T., Aarts, M., van Loenen, blockchain via smart contracts. Electronics 9(2), 244 (2020) E., Aarts, E.: Building automation and perceived control: Lahellec, N., Mazerolle, F., Michel, J.: Second-order esti- a field study on motorized exterior blinds in dutch offices. mate of the macroscopic behavior of periodic hyperelastic Buil. Environ. 79, 66–77 (2014) composites: theory and experimental validation. J. Mech. Mitchell, T.M.: Machine Learning, McGraw-Hill Series in Com- Phys. Solids 52(1), 27–49 (2004). https://doi.org/10.1016/ puter Science, 1st edn. McGraw-Hill (1997) S0022-5096(03)00104-2 Mognato, E., Brocca, S., Barbieri, A.: Thermally processed glass Lampropoulos, G., Siakas, K., Anastasiadis, T.: Internet of things : correlation between surface compression, mechanical and in the context of industry 4.0: an overview. Int. J. Entrep. fragmentation test. Glass Performance Days 2017, 8–11 Knowl. 7(1), 4–19 (2019) (2017) LeCun, Y., Bengio, Y., et al.: Convolutional networks for images, Monedero, J.: Parametric design: a review and some experiences. speech, and time series. Handbook of Brain Theory and Automation in Construction 9(4), 369–377 (2000) Neural Netw. 3361(10), 1995 (1995) Moor, J.: The dartmouth college artificial intelligence confer- LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature ence: The next fifty years. Ai Magazine 27(4), 87–87 (2006) 521(7553), 436–444 (2015) Mosavi, A.: Structural damage diagnosis and prediction using Lee, E.H., Rogers, T.G., Woo, T.C.: Residual stresses in a machine learning and deep learning models: Comprehen- glass plate cooled symmetrically from both surfaces. J. sive review of advances (2019) Am. Ceram. Soc. 48(9), 480–487 (1965). https://doi.org/ Mrosla, L., Koch, V., von Both, P.: Quo vadis ai in architecture?- 10.1111/j.1151-2916.1965.tb14805.x survey of the current possibilities of ai in the architectural Lee, J.H., Shin, J., Realff, M.J.: Machine learning: overview of practice (2019) the recent progresses and implications for the process sys- Müller-Braun, S., Franz, J., Schneider, J., Schneider, F.: Optische tems engineering field. Comput. Chem. Eng. 114, 111–121 merkmale der glaskante nach glaszuschnitt mit schneidräd- (2018) chen. ce/papers 2(1), 99–111 (2018) Li, S., Deng, W.: Deep facial expression recognition: A survey. Müller-Braun, S., Seel, M., König, M., Hof, P., Schneider, J., IEEE Transactions on Affective Computing (2020) Oechsner, M.: Cut edge of annealed float glass: crack system Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, and possibilities to increase the edge strength by adjusting A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., the cutting process. Glass Struct. Eng. 5(1), 3–25 (2020) et al.: A comparison of deep learning performance against Müller-Braun, S., Seel, M., König, M., Hof, P., Schneider, J., health-care professionals in detecting diseases from medical Oechsner, M.: Cut edge of annealed float glass: crack system imaging: a systematic review and meta-analysis. The Lancet and possibilities to increase the edge strength by adjusting Digit. Health 1(6), e271–e297 (2019) the cutting process. Glass Struct. Eng. 5(1), 3–25 (2020). Lloyd, S.: Least squares quantization in pcm. IEEE Trans. Inf. https://doi.org/10.1007/s40940-019-00108-3 Theory 28(2), 129–137 (1982) Murphy, K.P.: Mach. Learning: A Probabilistic Perspective. MIT Louter, C., Akilo, M., Miri, B., Neeskens, T., Ribeiro Silveira, press, (2012) R., Topcu, Ö., van der Weijde, I., Zha, C., Bilow, M., Turrin, Nassif, A.B., Shahin, I., Attili, I., Azzeh, M., Shaalan, K.: M., et al.: Adaptive and composite thin glass concepts for Speech recognition using deep neural networks: A system- architectural applications. Heron 63(1/2), 199–218 (2018) atic review. IEEE Access 7, 19143–19165 (2019) Lu, P., Chen, S., Zheng, Y.: Artificial intelligence in civil engi- Newton, D.: Generative deep learning in architectural design- neering. Math. Prob. Engi. 2012, 145974 (2012). https:// Generative deep learning in architectural design. Tech.| doi.org/10.1155/2012/145974 Archit. + Des. 3(2), 176–189 (2019) Luna-Navarro, A., Loonen, R., Juaristi, M., Monge-Barrio, A., Niu, G.: Data-Driven Technol. Eng. Syst. Health Manag. Attia, S., Overend, M.: Occupant-facade interaction: a Springer, Berlin (2017) 123 284 M. A. Kraus, M. Drass Nordfjeld, A.Á.Z.: The aspect of the human bias in decision Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Den- making within quality management systems & lean theory. zler, J., Carvalhais, N., et al.: Deep learning and process Int. J. Psychol. Behav. Sci. 7(12), 2536–2541 (2013) understanding for data-driven earth system science. Nature O’Leary, D.E.: Artificial intelligence and big data. IEEE Intell. 566(7743), 195 (2019) Syst. 28(2), 96–99 (2013) Reitermanova, Z.: Data splitting. In: WDS 10, 31–36 (2010) Oxman, R.: Thinking difference: Theories and models of para- Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: Learning metric design thinking. Des. Stud. 52, 4–39 (2017) deep 3d representations at high resolutions. In: Proceedings Ozdemir, S., Susarla, D.: Feature Engineering Made Easy: Iden- of the IEEE Conference on Computer Vision and Pattern tify Unique Features from Your Dataset in Order to Build Recognition, pp. 3577–3586 (2017) Powerful Machine Learning Systems. Packt Publishing Ltd, Romano, R., Aelenei, L., Aelenei, D., Mazzuchelli, E.S.: What is (2018) an adaptive façade? analysis of recent terms and definitions Paluszek, M., Thomas, S.: MATLAB Machine Learning. Apress, from an international perspective. J. Facade Des. Eng. 6, (2016) 65–76 (2018) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional net- G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmai- works for biomedical image segmentation. In: International son, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, Conference on Medical image computing and computer- A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, assisted intervention, Springer, Berlin, pp. 234–241 (2015) S.: PyTorch: An imperative style, high-performance deep Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by learning library. In: Wallach, H., Larochelle, H., Beygelz- locally linear embedding. Science 290(5500), 2323–2326 imer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances (2000) in Neural Information Processing Systems 32, pp. 8024– Rudy, S.H., Kutz, J.N., Brunton, S.L.: Deep learning of dynam- 8035. Curran Associates Inc, (2019) ics and signal-noise decomposition with time-stepping con- Patil, P.S.: Review on use artificial intelligence in various appli- straints. J. Comput. Phys. 396, 483–506 (2019) cations of mechanical engineering, medical and healthcare Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, field (2016) S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Patil, A., Patted, L., Tenagi, M., Jahagirdar, V., Patil, M., Gau- Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recogni- tam, R.: Artificial intelligence as a tool in civil engineering-a tion challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 review. In: Proceedings of National conference on advances (2015). https://doi.org/10.1007/s11263-015-0816-y in computational biology, communication, and data ana- Russell, S., Norvig, P.: Artificial intelligence: A modern approach lytics (ACBCDA, : India: IOSR J, p. 2017. Eng, Comput (4th ed.) Hoboken, NJ: Pearson (2020) (2017) Rusu, O., Halcu, I., Grigoriu, O., Neculoiu, G., Sandulescu, V., Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, Marinescu, M., Marinescu V (2013) Converting unstruc- B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., tured and semi-structured data into knowledge. In: 2013 Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, 11th RoEduNet International Conference, IEEE, pp 1–4 D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Sacks, R., Bloch, T., Katz, M., Yosef, R.: Automating design Machine learning in Python. J. Mach. Learn. Res. 12, 2825– review with artificial intelligence and bim: State of the art 2830 (2011) and research framework. In: Computing in Civil Engineer- Pourmoghaddam, N., Kraus, M.A., Schneider, J., Siebert, G.: The ing 2019: Visualization, Information Modeling, and Sim- geometrical properties of random 2d voronoi tesselations ulation, American Society of Civil Engineers Reston, VA, pp. 353–360 (2019) for the prediction of the tempered glass fracture pattern. ce/papers 2.5–6, pp. 325–339 (2018) Schleicher, S., Lienhard, J., Poppinga, S., Masselter, T., Speck, Pourmoghaddam, N., Schneider, J.: Experimental investiga- T., Knippers, J.: Adaptive façade shading systems inspired tion into the fragment size of tempered glass. Glass by natural elastic kinematics (2011) Structures & Engineering (2018). https://doi.org/10.1007/ Schneider, J., Kuntsche, J., Schula, S., Schneider, F., Wörner, s40940-018-0062-0 J.D.: Glasbau: Grundlagen, Berechnung. Konstruktion. Quantrille, T.E., Liu, Y.A.: Artificial intelligence in chemical Springer, Berlin (2016) engineering. Elsevier, Netherlands (2012) Schober, K.S.: Artificial intelligence in the con- Raissi, M.: Deep hidden physics models: Deep learning of non- struction industry (2020). URL https:// linear partial differential equations. J. Mach. Learn. Res. www.rolandberger.com/en/Point-of-View/ 19(1), 932–955 (2018) Artificial-intelligence-in-the-construction-industry.html Raj, P., Evangeline, P.: The Digital Twin Paradigm for Smarter Sedlacek, G.: Glas im konstruktiven Ingenieurbau. Ernst & Sohn, Systems and Environments: The Industry Use Cases. Aca- Berlin (1999) demic Press, USA (2020) Senatore, G., Duffour, P., Winslow, P.: Exploring the application Rani, R., Kumar, R., Singh, A.P.: Deep learning method based domain of adaptive structures. Eng. Struct. 167, 608–628 binary descriptor for object detection. In: Proceedings of (2018) ICETIT 2019, Springer, Berlin, pp. 364–371 (2020) Serrano, W.: Deep reinforcement learning algorithms in intelli- Raschka, S.: Model evaluation, model selection, and algo- gent infrastructure. Infrastructures 4(3), 52 (2019) rithm selection in machine learning (2018). arXiv preprint Shahin, H.S.M.: Adaptive building envelopes of multistory build- arXiv:1811.12808 ings as an example of high performance building skins. Rebala, G., Ravi, A., Churiwala, S.: An Introduction to Machine Alexandria Eng. J. 58(1), 345–352 (2019) Learning. Springer, Berlin (2019) 123 Artificial intelligence for structural glass engineering 285 Siebert, G., Maniatis, I.: Tragende Bauteile aus Glas: Grundla- Vrugt, J.A.: Markov chain monte carlo simulation using the gen, Konstruktion, Bemessung. Beispiele. Wiley, New York dream software package: Theory, concepts, and matlab (2012) implementation. Environ. Modell. Softw. 75, 273–316 Silveira, R.R., Louter, C., Klein, T.: Flexible transparency-a study (2016). https://doi.org/10.1016/j.envsoft.2015.08.013 on adaptive thin glass façade panels. Chall. Glass Conf. Wada, B.: Adaptive structures. In: 30th Structures, Structural Proc. 6, 135–148 (2018) Dynamics and Materials Conference, p. 1160 (1989) Skansi, S.: Autoencoders. In: Introduction to Deep Learning, Wagg, D., Bond, I., Weaver, P., Friswell, M.: Adaptive structures: Springer, Berlin pp. 153–163 (2018) engineering applications. Wiley, New York (2008) Staudt, Y., Odenbreit, C., Schneider, J.: Failure behaviour of sili- Wagner, N., Rondinelli, J.M.: Theory-guided machine learning cone adhesive in bonded connections with simple geometry. in materials science. Front. Mat. 3, 28 (2016) Int. J. Adhes. Adhes. 82, 126–138 (2018). https://doi.org/ Wang, J., Li, J.: Bio-inspired kinetic envelopes for building 10.1016/j.ijadhadh.2017.12.015 energy efficiency based on parametric design of building Sun, X., Wu, P., Hoi, S.C.: Face detection using deep learning: information modeling. In: 2010 Asia-Pacific Power and An improved faster rcnn approach. Neurocomputing 299, Energy Engineering Conference, IEEE, pp. 1–4 (2010) 42–50 (2018) Wilber, J., Writer, R.E.: Q&a with industry experts how are e- Svetozarevic, B., Begle, M., Jayathissa, P., Caranovic, S., Shep- business trends impacting testers and testing teams? (2002) herd, R.F., Nagy, Z., Hischier, I., Hofer, J., Schlueter, Witten, I.H., Frank, E., Hall, M.A., Pal, C.: Data Mining: Practi- A.: Dynamic photovoltaic building envelopes for adaptive cal Machine Learning Tools and Techniques. Morgan Kauf- energy and comfort management. Nature Energy 4(8), 671– mann (2016) 682 (2019) Wolfgang, E.: Introduction to Artificial Intelligence, 2nd edn. Szolovits, P.: Artificial Intelligence in Medicine. Routledge Springer, Berlin (2017) (2019) Wortmann, T., Tunçer, B.: Differentiating parametric design: Tandia, A., Onbasli, M.C., Mauro, J.C.: Machine learning for Digital workflows in contemporary architecture and con- glass modeling. In: Musgraves, J.D., Hu, J., Calvez, L. (eds.) struction. Des. Stud. 52, 173–197 (2017) Springer Handbook of Glass, pp. 1157–1192. Springer, Wortmann, T., Costa, A., Nannicini, G., Schroepfer, T.: Advan- Berlin (2019). https://doi.org/10.1007/978-3-319-93728-1 tages of surrogate models for architectural design optimiza- Turing, A.M.: Computing machinery and intelligence. Mind tion. Artif. Intell. Eng. Des. Anal. Manufact. 29(4), 471–481 LIX(236), 433–460 (1950). https://doi.org/10.1093/mind/ (2015). https://doi.org/10.1017/S0890060415000451 LIX.236.433 Yu, D., Deng, L.: Automatic Speech Recognition. Springer, Turner, C.R., Fuggetta, A., Lavazza, L., Wolf, A.L.: A conceptual Berlin (2016) basis for feature engineering. J. Syst. Softw. 49(1), 3–15 Zhang, J., Liu, N., Wang, S.: A parametric approach for perfor- (1999) mance optimization of residential building design in Bei- Van Ginhoven, R.M., Jónsson, H., Corrales, L.R.: Silica glass jing. Build. Simul. 13, 223–235 (2020). https://doi.org/10. structure generation for ab initio calculations using small 1007/s12273-019-0571-z samples of amorphous silica. Phys. Rev. B 71(2), 024,208 Zhao, Z.Q., Zheng, P., Xu, St, Wu, X.: Object detection with deep (2005) learning: A review. IEEE Trans. Neural Netw. Learn. Syst. Van Veen, F.: The neural network zoo (2016). https://www. 30(11), 3212–3232 (2019) asimovinstitute.org/neural-network-zoo/ Zheng, Y., Cai, Y., Zhong, G., Chherawala, Y., Shi, Y., Dong, J.: Varian, H.: Artificial intelligence, economics, and industrial orga- Stretching deep architectures for text recognition. In: 2015 nization. Technical Report, National Bureau of Economic 13th International Conference on Document Analysis and Research (2018) Recognition (ICDAR), IEEE, pp. 236–240 (2015) Vass, S., Gustavsson, T.K.: Challenges when implementing bim for industry change. Construct. Manag. Econom. 35(10), Publisher’s Note Springer Nature remains neutral with regard 597–610 (2017) to jurisdictional claims in published maps and institutional affil- Vergauwen, A., Alegria Mira, L., Roovers, K., De Temmerman, iations. N.: Parametric design of adaptive shading elements based on curved-line folding. In: Proceedings of the First Conference Transformables 2013 (2013) Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: A Brief Review. Computational intelligence and neuroscience 2018, (2018)

Journal

Glass Structures & EngineeringSpringer Journals

Published: Nov 7, 2020

There are no references for this article.