Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

R. Y. Goh; L. S. Lee

doi:10.1155/2019/1974794

Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

Goh, R. Y.;Lee, L. S. 2019-03-13 00:00:00 Hindawi Advances in Operations Research Volume 2019, Article ID 1974794, 30 pages https://doi.org/10.1155/2019/1974794 Review Article Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches 1 1,2 R. Y. Goh andL.S.Lee Laboratory of Computational Statistics and Operations Research, Institute for Mathematical Research, Universiti Putra Malaysia (UPM), Serdang, Selangor, Malaysia Department of Mathematics, Faculty of Science, Universiti Putra Malaysia (UPM), Serdang, Selangor, Malaysia Correspondence should be addressed to L. S. Lee; lls@upm.edu.my Received 1 November 2018; Revised 28 January 2019; Accepted 18 February 2019; Published 13 March 2019 Academic Editor: Eduardo Fernandez Copyright © 2019 R. Y. Goh and L. S. Lee. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Development of credit scoring models is important for financial institutions to identify defaulters and nondefaulters when making credit granting decisions. In recent years, artificial intelligence (AI) techniques have shown successful performance in credit scoring. Support Vector Machines and metaheuristic approaches have constantly received attention from researchers in establishing new credit models. In this paper, two AI techniques are reviewed with detailed discussions on credit scoring models built from both methods since 1997 to 2018. eTh main discussions are based on two main aspects which are model type with issues addressed and assessment procedures. Then, together with the compilation of past experiments results on common datasets, hybrid modelling is the state-of-the-art approach for both methods. Some possible research gaps for future research are identified. 1. Introduction customers database increased tremendously. In 2004, Basel II accord is released. Under credit risk, rather than the Credit scoring is the basis of financial institutions in making previous standardised method, Internal Rating Based (IRB) credit granting decisions. A good credit scoring model will approach could be adopted by bank to compute the minimum be able to effectively group customers into either default or capital requirement. This marked an evolution in the credit nondefault group. The more efficient it is, the more cost can scoring eld fi , where attempts to form sophisticated model be saved for a financial institution. have been actively researched. Together with the rapid growth Credit scoring is used to model available data and of computer technology, formulation of sophisticated models evaluate every instance in the data with a credit score and is made possible. probability of default (PD). Generally, score is a measurement Hand and Henley [1] first published review paper of of the credit-worthiness of customers, while the PD is the credit scoring domain. They reviewed statistical methods and likelihood estimation of a customer fails to meet one’s debt several data mining (DM) methods and concluded that the obligation in a given period of time. Hand and Henly [1] future trend of credit scoring models will be more complex den fi ed credit scoring as “the term used to describe formal methods. Thomas [2] also reviewed on past researches on statistical methods used for classifying applicants for credit credit scoring and pointed out the importance of protfi into ‘good’ and ‘bad’ risk classes”. Since the final decision scoring. The methods discussed ranged from statistical, is binary, credit scoring is thus equivalent to a binary operations research based, and DM approaches. Sadatrasoul classification problem. et al. [3] reviewed DM techniques applied for credit scoring domain in year 2000-2012, showing the tendency of model When credit cards started to be introduced in 1960s, necessity of credit scoring models is triggered. Financial insti- building with DM methods in recent years. tutions started to combine or replace purely judgemental- There are also review papers that specifically focused on based credit granting decisions with statistical models as application scoring [4] and bankruptcy prediction [5, 6]. 2 Advances in Operations Research Martin [4] reviewed application scoring model in a differ- review, particularly on the development of SVM and MA ent perspective, where procedure of scorecard performance only. assessment by past studies are categorized into consistency, All the research articles in this study are obtained from application t, fi and transparency. The author pointed out three online academic databases, Google Scholar, Science the weaknesses of past experiments that only pay attention Direct, and IEEE Xplore/Electronic Library Online. Several to model development and neglected appropriate assess- main keywords applied to select the articles are credit scoring, ment procedures of the models. Sun et al. [6] reviewed credit risk prediction, metaheuristics, data mining, SVM, on bankruptcy prediction credit models by providing clear Genetic Algorithm, Genetic Programming, Evolutionary definitions of bankruptcy prediction as resulted from litera- Computing, machine learning, and artificial intelligence. ture throughout the years and then discussed the techniques From the search results, there are 44 and 43 articles from based on three main approaches, i.e., modelling, sampling, SVM and MA models, respectively, with 12 articles utilizing and featuring. Alaka et al. [5] also focused on bankruptcy both together, thus resulting in a total of 75 research articles prediction models. They identified the popular statistical and being reviewed in this study ranging from year 1997 to artificial intelligence (AI) tools utilized and discovered 13 criteria of the tools usage. Based on the 13 criteria, they The objectives of this study are to review past literatures developed a framework as a guideline for tool selection. of using SVM and MA in developing credit scoring model, Besides, there is one review from Moro and Rita [7] that identify the contributions of both methods, and discuss the adopted a completely different approach than all the other evolving trend that can lead to possible future works in credit review papers. An automated review procedure is conducted scoring domain. with text mining. Important issues in credit scoring domain This paper is organized as follows. Section 2 discusses the and top ranked tools for model development are discovered evolving trend of the credit scoring models from traditional through automated text mining. methods to DM methods. Section 3 briefly describes SVM Baesens et al. [8] are the rfi st to build credit scoring and MA methods. Section 4 summarizes and discusses the model with Support Vector Machines (SVM) and compare results based on model type with issues addressed, assessment its performance with other state-of-the-art methods. They procedures, and results compilation from past experiments. experimented with standard SVM and Least Squares SVM Then, Section 5 suggests several possible future directions (LS-SVM) and reported that LS-SVM yield good perfor- and draws conclusion for this study. mance as compared to other methods. Thereaer ft , SVM has been actively researched in the credit scoring domain, being 2. Trend one of the mostly used DM methods to build credit scoring model. Recent review studies [5, 7] have also identified SVM Before credit scoring model is developed, credit granting as a significant tool to be selected among researchers for credit decision is purely judgemental-based. Statistical models models development. started to be utilized since 1941 where Durand [14] is the first Metaheuristic Approaches (MA), especially Evolutionary to pioneer the usage of discriminant analysis (DA) to classify Algorithm (EA), have also been introduced as the alternative good and bad loans. Altman [15] also used multiple discrim- to form credit scoring model. The surging applications in inant analysis (MDA) to predict company bankruptcy. The credit scoring can be seen in recent years, where most review financial ratios are treated as input variables and modelled papers included discussion of EA as one of the main DM with MDA. MDA model has shown good prediction ability methods (see Alaka et al. [5], Crook et al. [9], Lahsasna et al. and useful for analyst to provide investment recommenda- [10],Lessmann etal. [11],and Louzada et al. [12]). In Louzada tions. Ohlson [16] then introduced a probabilistic approach to et al. [12] timeline discussion, it can be seen that, after year predict credit-worthiness of companies in 1980. The method 2005, SVM and EA have increased research work in credit proposed is Logistic Regression (LOGIT). Few problems of scoring domain. Besides, Marques et al. [13] have reviewed on using MDA are pointed out and LOGIT is believed to be more credit scoring models specifically focusing in EA, indicating robust than MDA. the popular usage of MA. In 1985, Kolesar and Showers [17] introduced mathemat- To date, reviews on credit scoring domain are mainly ical programming (MP) approach to solve credit granting focusing on a wide category of methods to develop score- problem. MP is compared with classical DA and reported card. The active research utilizing SVM prompted a need results showed that MP is more robust and flexible for to particularly review on this method. There was only decision maker. Among the traditional methods, LOGIT one review by Marques et al. [13] that focused on EA turned out to be the standard credit scoring model because in credit modelling. However, EA are just a part of MA, of its ability to fulfil all requirements from the Basel II where there are other MA that have received attention accord. in the credit scoring domain in recent years due to the shift of modelling trend towards AI-based techniques. The Massive improvement in computer technology opens up surging application of both techniques has greatly increased DM approach in model building for credit scoring. There contributed works, leading to a situation where general have been extensive researches done in the past that utilized reviews are insufficient to peek into the development trend DM methods in the credit scoring domain. Most of them of these two methods. Hence, in contrast with past literature compared the adopted methods with the standard LOGIT reviews which are general reviews, this study is a focused model and have shown the DM models are competitive. A Advances in Operations Research 3 few comparative studies and review papers [1, 2, 8–12, 18] (i) Evolutionary Algorithm (EA) reported good performance of DM models, and among the The EA approach has a mechanism to seek solution various methods, SVM and MA (especially EA) have been following the Darwinian principle, which is based widely researched to be the alternative to the credit scoring on the “survival of the tfi test” concept. There are models. four main procedures to search for a solution, i.e., selection, reproduction, mutation, and crossover. The 3. AI Techniques solutions inthe population areimproved inanevo- lutionary manner, guided by the quality of the tness fi .. Support Vector Machines. Support Vector Machines function. The EA applied in credit scoring are Genetic (SVM) were rs fi t introduced by Vapnik [91] in 1998, in the Algorithm (GA) and Genetic Programming (GP). context of statistical learning theory. There are many suc- (ii) Swarm Intelligence (SI) cessful applications of SVM in pattern recognition, indicating SVM to be a competitive classifier. The SI approach is a nature-inspired algorithm that SVM seeks for an optimal hyperplane with maximum conducts solution-seeking based on natural or arti- margin that acts as the decision boundary, to separate the ficial collective behaviour of decentralized and self- two different classes. Given a training set with labelled organized systems. The solutions in the population instance pairs (𝑥 ,𝑦 ),where 𝑖 is the number of instance 𝑖 𝑖 are improved through the interaction of agents with 𝑖 = 1,2,3,...,𝑚 , 𝑥 ∈ R and 𝑦 ∈{−1,+1},the decision 𝑖 𝑖 each other and also with the environment. Similarly, boundary to separate two different classes in SVM is generally the generation of better solution is led by tness fi expressed as function quality. The SI applied in credit scoring are Particle Swarm Optimization (PSO), Ant Colony 𝑤⋅ 𝑥+ 𝑏 = 0. (1) Optimization (ACO), Scatter Search (SS), Artificial Bee Colony Optimization (ACO), Honey Bees Mating Optimal separating hyperplane is the one with maximum Optimization (HBMO), Cuckoo Search (CS), and margin and all training instances are assumed to satisfy the Harmony Search (HS). constraint, (iii) Iterative Based (IB) 𝑦 𝑤⋅ 𝑥 +𝑏 ≥1. ( ) (2) 𝑖 𝑖 The IB approach focus to improve on one single The convex optimization problem is then defined as solution in each iteration around its neighbourhood, where the solution improvement is based on the quality of tness fi function. u Th s, IB is different min 𝜙 (𝑤, 𝑏 ) = ‖𝑤 ‖ +𝐶 ∑ 𝜖 𝑖=1 as compared to EA and SI which are population- (3) based. The IB applied in credit scoring are Simulated s.t.𝑦 (𝑤 ⋅ 𝑥 +𝑏) ≥ 1. 𝑖 𝑖 Annealing (SA) and Tabu Search (TS). The optimal hyperplane is equivalent to the optimization problem of a quadratic function, where Lagrange function is 4. Discussions utilized to find the global maximum. The 𝜖 is the slack vari- .. Model Types with Issues Addressed. SVM and MA credit able introduced to take account for misclassification, with 𝐶 models are categorized according to the type of models as the accompanied penalty cost. For nonlinear classification, formed. There are four main categories of the SVM being kernel trick is used to modify the SVM formulation. Popular utilized in credit scoring: standard SVM and its variants, kernel choices are linear, Radial Basis Function (RBF), and modiﬁed SVM, hybrid SVM, and ensemble models.On the polynomial. other hand, MA approaches applied in credit scoring can (i) Linear: 𝑥 ⋅𝑥 . 𝑖 𝑗 be divided into three categories: standard MA, hybrid MA- MA, and hybrid MA-DM. The models development and main (ii) Polynomial: (𝑥 ⋅𝑥 +𝑐) , 𝑐 >0, 𝑑 ≥2 . 𝑖 𝑗 issues addressed are discussed. (iii) Radial Basis Function (RBF): exp{−𝛾‖𝑥 −𝑥 ‖ }. 𝑖 𝑗 ... SVM .. Metaheuristic Algorithms. Metaheuristic Algorithm (MA) is one of the AI-based data mining approaches () Standard SVM and Its Variants. SVMs applied to build which had gained attention in recent years. MA is an credit scoring models discussed here are the standard SVM automated process to search for a near-optimal solution of and its variants, where these SVMs are being applied directly an optimization problem. The search process is conducted for model building without any modifications. with operators to ensure a balance between exploration and exploitation to efficiently seek for good solutions. Generally, Investigation of Predictive Ability. The predictive ability MA consists of several categories; the description of the of SVMs credit models is examined through two main categories that have been used in credit scoring domain is approaches, i.e., comparative studies [8, 11, 12, 18, 26] and given as follows: application on specific credit domain [19–25, 27]. 4 Advances in Operations Research Various state-of-the-art methods ranging from tradi- outperformances across all the datasets. They concluded BN tional statistical methods to nonparametric DM methods and the Boosting ensemble as generally effective method as have been attempted to form credit scoring models in differ- credit models. These comparative studies have included a wide variety of ent researches. This prompted the necessity of a benchmark study conducted by Baesens et al. [8] in 2003. In their study, classification techniques to formulate credit scoring models SVM and Least Squares SVM (LS-SVM) are first utilized and their predictive ability is assessed across various credit datasets. However, among these credit models, there has not to develop credit scoring model, and their performances are compared with the other state-of-the-art methods across been a clear best technique for the credit scoring domain. eight different datasets. The methods are from the family of Nonetheless, these comparative studies have provided a LOGIT, DA, Linear Programming (LP), Bayesian Network guideline on the update of latest available models in credit (BN), k-nearest neighbourhood (kNN), neural networks scoring, where SVM is initially being tagged as state of the artin[8] and then served asanimportant model inthis (NN), and decision trees (TREE). LS-SVM and NN reported statistically significant better results compared to the other domain. Besides, ensembles are another state of the art as models. Their results also showed that the methods are demonstrated in [11]. Instead of comparative studies with various techniques competitive to each other. Hence, SVM has been actively researched in the credit scoring domain. being benchmarked, a SVM-focused study is conducted by Lessmann et al. [11] updated Baesens et al. [8] research Danenas et al. [26], particularly on bankruptcy prediction dataset. Various types of SVM originating from different by including comparison with more individual classifiers, ensembles, and hybrid methods. They commented the libraries and software, i.e., LIBSVM, LIBLINEAR, WEKA, increasing research on credit scoring domain that urged for and LIBCVM, are included in their experiment. The SVM necessity to update the benchmark study where not only classifiers ranged from linear to nonlinear with a wide variety state-of-the-art classifiers but also advanced techniques like of kernels. Models are investigated on original dataset and reduced dataset already preprocessed with feature selection ensembles should be included as well. u Th s, there are a total of 41 classifiers (from the family of BN, TREE, LOGIT, DA, NN, technique. Comparing the accuracy, different types of SVM kNN, Extreme Learning Machine (ELM), and ensembles) classifiers showed comparable results, with application on reduced dataset having higher accuracy. Besides, in terms of being investigated across six datasets with wide range of size in the benchmark study. The experimental results showed computational effort, linear SVM is the fastest model. that ensemble models took up top 10 ranking among all Another approach to investigate the predictive ability is the 41 classifiers. For individual classifiers, NN showed via the application on specific domains. The specific domains highest ranking. Linear and RBF SVM were investigated; both experimented are multiclass corporate rating [19, 20, 22, 23, SVM models showed similar performance as both scored 25], application scoring [21, 24], and behavioural scoring similar ranking. They also pointed out the importance of [27]. For multiclass corporate rating, credit scoring models developing business-valued scorecard, which should be taken are formed with LS-SVM in [19, 22] and SVM in [20, 23, 25] with the main aim to show the effectiveness of SVM into consideration in proposing new models. Louzada et al. [12] conducted a systematic review on in building corporate rating models. Van et al. [19] and classification methods applied on credit scoring, covering Lai et al. [22] showed that LS-SVM is the best performing credit model compared to other traditional techniques when research papers of year 1992-2015. Their review discussed several aspects in the credit scoring domain: objective of applied on Bankscope and England financial service data, the study, comparison done in the research, dataset used respectively. for model building, data splitting techniques, performance Huang et al. [20] tested SVM on Taiwan and US market measures used, application of missing data imputation, and bond rating data. They also conducted cross market analysis application of feature selection. They carried out experiment with variable contribution analysis from NN models. Lee on 11 main classification techniques (Linear Regression (LR), [23] utilized SVM for Korea corporate credit rating analysis. NN, TREE,DA,LOGIT,FUZZY,BN, SVM,Genetic Pro- Kim and Sohn [25] in 2010 have initiated the building of credit scoring model for small medium enterprises (SME). gramming (GP), hybrid, and ensembles), where inclusion of ensembles started to receive attention as recommended in They focused on Korea SME and the explanatory variables [11]. The problem of imbalance dataset is investigated and included four main aspects: SME characteristics, technology evaluations, financial ratios, and economic indicators. They SVM showed stability in dealing with imbalance dataset. Generally, SVM has better performance and lower compu- believed that SVM model would be suitable to be used tational effort in the experiment. for technology-based SMEs. For all the SVM models on The most recent comparative study is contributed by these four researches on different market data of corporate Boughaci and Alkhawaldeh [18]. They investigated perfor- rating, SVM have outperformed the other methods in every mances of 11 machine learning techniques (from the family of experiment. kNN, BN, NN, TREE, SVM, LOGIT, and ensembles) across For application scoring, Li et al. [21] compared SVM eight different datasets. A main difference of their study with NN on Taiwan bank consumer loan credit scoring. SVM reported higher accuracy than NN and the results is from the previous one is that the datasets utilized involved a mixture of application scoring and bankruptcy prediction statistically significant. Besides, they also experimented the dataset. The experimental results did not suggest a winner effect of different hyperparameter values of SVM on the type I error and type II error, i.e., misclassicfi ation error. They among the techniques investigated as there are no consistent Advances in Operations Research 5 demonstrated the effect on misclassification error across the Dynamic Scoring. Yang[30] modiefi dweightedSVM model hyperparameter values range can serve as a visualization to become a dynamic scoring model. The main idea is to tool for credit scoring model. u Th s, they concluded SVM enable an easy update of credit scoring model without the need to repeat variable selection process when new customers outperformed NN in terms of visualization. Bellotti and Crook [24] tested SVM of different kernels on large credit data became available. Original kernel in weighted SVM card datasets. Comparison with traditional statistical models is modified to become an adaptive kernel. When there is reported only kNN and polynomial SVM have poorer results increment in the data size, adaptive kernel can automatically which may be due to overtfi ting. They suggested using update the optimal solution. Besides, with the trained model, support vectors’ weight as an alternative to select significant Yang [30] suggested an attribute ranking measure to rank the features and compared the selected features with those from kernel attributes. us, Th this became an alternative solution for LOGIT’s Wald statistics. The experiment indicated SVM the black box property of SVM. models are suitable for feature selection for building credit Reject Inference. The aim of reject inference is to include scoring model. rejected instances into model training, then improving clas- For behavioural scoring, South African unsecured lend- sicfi ation performance. Li et al. [31] and Tian et al. [32] pro- ing market is investigated by Mushava and Murray [27]. posed new SVM to solve reject inference problem for online Despite having the aim to show the effectiveness of SVM, this peer-to-peer lending. Li et al. [31] proposed a semisupervised study aimed to examine some extensions of statistical LOGIT L2-SVM (SSVM) to solve reject inference for a peer-to-peer and DA that have been less explored in credit scoring domain, lending data from Lending Club of different years. us Th , with SVM being included as a benchmark comparison in in the SSVM formulation, unlabelled rejected instances are their study. In their fixed window experiment, Generalized added to the optimization constraints of SVM, converting Additive Models have outperformed the others. Although the original quadratic programming problem to a mixed SVM did not show superior performance, the inclusion of integer programming. SSVM reported better performance SVM in this study once again indicated SVM is perceived as than other standard methods. Tian et al. [32] proposed a standard to overcome in credit scoring domain. a kernel-free fuzzy quadratic surface SVM (FQSVM). The () Modiﬁed SVM. Modiefi d SVM involved algorithmic main advantages of the proposed model are the ability to detect outliers, extract information from rejected customers, change in the formulation of SVM. There are a few works that proposed modiefi d SVM for solving dieff rent problems no kernel required for computation, and efficient convex in the credit scoring domain, particularly in application optimization problem. The proposed model is benchmarked against other reject inference methods. FQSVM is reported to scoring. The modifications required changes in the quadratic programming formulation of the original SVM. be superior than SSVM proposed by [31] in terms of several performance measures as well as computational efficiency. Outlier Handling. Wang et al. [28] proposed a bilateral fuzzy Features Selection. There are two new formulation of SVM to SVM (B-FSVM). The method is inspired from the idea that carry out features selection in cost approach [33] and protfi no customer is absolutely good or bad customer as it is approach [34] on Chilean small and microcompanies. The always possible for a classified good customer to default dataset consisted of new and returning customers, indicating and vice versa. us, Th they utilized membership concept in the credit scoring models formed involved both application fuzzy method where each instance in the training set takes and behavioural scoring. Maldonado et al. [33] included positive and negative classes, but with different memberships. variable acquisition costs into formulation of SVM credit This resulted in bilateral weighting of the instances because scoring model to do feature selection. Similar to Li et al. [21], each instance now has to take into account error from they also added additional constraints into the optimization both classes. By including the memberships from fuzzy, problem, converting it to a mixed integer programming prob- SVM algorithm is reformulated to form B-FSVM. They used lem, but the added constraints are the variable acquisition LR, LOGIT, and NN to generate membership function. B- costs. They proposed two models where 1-norm and LP- FSVM are compared with unilateral fuzzy SVM (U-FSVM), norm SVM are both modified with the additional constraints, and other standard methods. Linear, RBF, and polynomial forming two new credit scoring models, namely, L1-mixed kernels are used to form B-FSVM, U-FSVM, and SVM integer SVM (L1-MISVM) and LP-mixed integer SVM (LP- models. MISVM). Due to the ability of the proposed models to take into consideration of variable acquisition costs and good Computational Eﬃciency. Harris [29] introduced clustered performance simultaneously, it is believed that the proposed SVM (CSVM), a method proposed by Gu and Han [92], methods are efficient tool for credit risk as well as business into credit scoring model. The research aimed to reduce analytics. computational time of SVM model. With k-means clustering On the other hand, Maldonado et al. [34] introduced a to form clusters, these clusters will be included into the formulation of SVM optimization problem, changing the prot- fi based framework to do feature selection and classi- original SVM algorithm. Two CSVM models are developed fication with modified SVM as well as protfi performance metrics. Instead of considering acquisition costs for variables, with linear and RBF kernel and compared with LOGIT, SVM, and their hybrids with k-means. Excellent time improvement one by one, they treated the costs as a group to be penalized of CSVM is reported. on whole set of variables. Therefore, the L- ∞ norm is 6 Advances in Operations Research utilized to penalize the group cost. Two models are proposed classification. The proposed method is a two-phase pro- with 1-norm SVM and standard SVM being modified by cedure. In the first phase, they introduced to use three including L-∞ into optimization objective function, forming different algorithms of link relation to extract input features by linking the relations of applicants’ data. So, there are three L1L-∞SVM and L2L-∞SVM. Their proposed models are effective in selecting features with lower acquisition costs, hybrid SVM models being built with respect to the three yet maintaining good performance. The main difference as different algorithms. Recently, Han et al. [50] proposed a compared to the previous research in [33] is that, with this new orthogonal dimension reduction (ODR) method to do newly proposed model, the protfi can be assessed, which feature extraction. They used SVM as the main classifier as posed as a crucial insight for business decision makers. they believed ODR is an eeff ctive preprocessing for SVM and used LOGIT as benchmark classiefi r. There are three () Hybrid SVM. Hybrid SVM credit scoring models have main parts in the experiment. First, they discovered that been developed by collaborating SVM with other techniques variables normalization posed large effect on classification for different purposes. performance of SVM but LOGIT is not strongly affected. Therefore, normalization is applied for all models. Second, Reject Inference. Chen et al. [48] tackled this problem using comparison is done with existing feature reduction method, the credit card data from a local bank of China. They principal component analysis (PCA). Third, they suggested hybridized k-means clustering with SVM, formulating a two- using LOGIT at the start to pick important variables with stage procedure. The rfi st stage is the clustering stage where Wald statistics and then only extract features from the new and accepted customers are grouped homogeneously, reduced variables, which they name as HLG. They concluded isolated customers are deleted, and inconsistent customers ODR is eeff ctive in solving dimension curse for SVM. are relabelled. The clustering procedure of dealing with inconsistent customer is a type of reject inference problem. Features Selection. For feature selection hybrid SVM models, These clustered customers from the rfi st stage are input to there are filter approach [37, 39, 43] and wrapper approach SVM to do classification in the second stage. Instead of [54, 55] used. For lfi ter approach, rough set theory is classifying customers into binary groups, they attempted to employed by Zhou and Bai [37] and Yao et al. [39], where classify into three and four groups. Different cutoff points rough set select input features in the first stage and carried are also set for different groups. They believed the proposed out classification tasks in the second stage with respect to method is able to provide more insight for risk management. the hybridized techniques. There are three main differences in between the hybrid models collaborated with rough sets Rule Extraction. Black box property of SVM has always proposed in these two researches. First, Zhou and Bai [37] been the main weakness, which is also a main concern specified their study on Construction Bank in China whereas for practitioners not using SVM as credit scoring models. Yao et al. [39] study is generally on public datasets. Second, Martens et al. [36] proposed rule extraction technique to be features selection is based on information function in [37] used together with SVM. Three different rule extraction tech- whereas computed variable importance is used to select niques, namely, C4.5, Trepan, and Genetic Rule Extraction features in [39]. Third, the hybrid models developed in [37] (G-REX), are hybridized with SVM. Experiment is conducted are hybridization with NN, SVM, and GA-SVM (GA to tune on different fields that require comprehensibility of model SVM hyperparameters) whereas the hybrid models devel- where credit scoring is one of the efi ld addressed in their oped in [39] are hybridization with SVM, TREE, and NN. research. The proposed models are advantageous in giving Both experiments showed that SVM-based hybrid models clear rules for interpretation. In 2008, Martens et al. [93] obtained best performance. Chen and Li [43] also adapted made an overview on the rule extraction issue, where the a lfi ter approach to do feature selection. They proposed four importance of comprehensibility in credit scoring domain different filter approaches: LDA, TREE, rough set, and F- is addressed again. Zhang et al. [38] proposed a hybrid score. These four approaches are hybridized with SVM. In credit scoring model (HCSM) which hybridized Genetic order for the models to be comparable, the same number of Programming (GP) with SVM. The main advantage of the features is selected based on the four approaches based on proposed technique is the ability to extract rules with GP that variable importance. solved theblack box natureofSVM. Apart from the filter approach of conducting feature selection, Jadhav et al. [54] and Wang et al. [55] incorporated Computational Eﬃciency. Hens and Tiwari [46] integrated filter techniques in developing novel wrapper model for fea- the use of stratified sampling method to form SVM model. ture selection task. The main concept is to guide the wrapper Then, with the smaller sample, F-score approach is used to model to do feature selection with obtained information from do feature selection to compute the rank of features based filter feature ranking techniques. on importance. The proposed model achieved lowest runtime Jadhav et al. [54] proposed Information Gain Directed and comparable performance when compared with other Feature Selection (IGDFS) with wrapper GA model, based methods considered in their experiment. on three main classiefi rs, i.e., SVM, kNN, and NB. Top rank features from information gain are passed on to the Features Extraction. Xu et al. [40] incorporated link anal- wrapper GA models. They compared their three different ysis with SVM, where link analysis is rfi st used to extract IFDFS (based on three different classifiers) with six other models: three standalone classiefi rs with all features included, applicants’ information and then input into SVM to do Advances in Operations Research 7 and three standard GA wrapper models of the classifiers weights can be assigned to solve class imbalance during that conducted feature selections without guided by any filter classification. Different from [42], they recommended the methods. Wang et al. [55] hybridized SVM with multiple use of DOE for hyperparameters tuning due to competitive resultsreported ascompared to DS, GA, and GS, but with population GA to form a wrapper model for feature selection. The method has a two-stage procedure. In the first stage, they lowest computational time. External benchmarked against utilized three lfi ter feature selection approaches to find prior results from [8, 45] are also included. Chen et al. [49] and Hsu et al. [53] integrated ABC information of the features. Feature importance is sorted in descending order, then a wrapper approach is used to find and SVM for hyperparameters tuning in corporate credit optimal subset. With the three feature subsets from the three scoring. Chen et al. [49] applied the proposed ABC-SVM on approaches and probability of a feature to be switched on, Compustat database of America from year 2001-2008. PCA is they formed the initial populations to be passed on to the the data preprocessing method used for extracting important features. Recently, Hsu et al. [53] also researched on ABC- second phase. In the second phase, HMPGA with SVM is run to find the final optimal feature subset; thus HMPGA-SVM is SVM in corporate credit rating with dataset from the same the model with prior information. database as in [49], but including more recent years 2001- 2010. Similarly, they utilized PCA as the preprocessing step. Hyperparameters Tuning. Other than input features, hyper- They conducted a more detailed study on the data, where parameters of SVM models pose great effect on the end using information from PCA, they divided the dataset into model formed. Previous works that proposed models for three categories to study the ability of the credit models to feature selection have applied the conventional Grid Search account for changes in future credit rating trend. (GS) method to find the appropriate hyperparameters. The researches that introduced hybrid SVM for finding hyperpa- Simultaneous Hyperparameters Tuning and Features Selection. rameters are [47, 52] in bankruptcy prediction, [42, 44, 51] in Based on the previous discussed research works, feature application scoring, and [49, 53] in corporate rating. selection and hyperparameter selection for SVM models With the success of linear SVM as experimented in [26], in credit scoring are crucial procedures in model building. Danenas and Garsva [47] examined the use of different linear Therefore, two pieces of research [35, 41] aimed to solve both SVMs available in the LIBLINEAR package on bankruptcy problems simultaneously with wrapper model approach. prediction. All the techniques are hybridized with GA and Huang et al. [35] attempted three strategies for building PSO to do model selection and hyperparameters tuning. The SVM-based credit scoring models: rs fi t, GS to tune SVM hybrid models formed are, namely, GA-linSVM and PSO- hyperparameters with all features included; second, GS to linSVM. Sliding window approach is adapted for building find hyperparameters and F-score to find feature subsets models across different time periods to report model per- for SVM model building; third, the initiation of hybrid formances. GA-linSVM is concluded to be more stable than GA-SVM to search hyperparameters and feature subsets PSO-linSVM by consistently selecting same model across simultaneously. This experimental result indicated GA-SVM dieff rent time periods yet having good performance. In later as a suitable tool for alternative to solve both issues together years, Danenas and Garsva [52] conducted another research but required high computational eor ff t. Zhou et al. [41] also to improve on PSO-linSVM. They modified PSO by using formulated a hybrid model using GA, but using dieff rent integer values for velocity and position, instead of rounding variants of weighted SVM, thus forming GA-WSVM. They up the values as in [47, 51]. mentioned in Huang et al. [35] research that the features PSO-linSVM continued to receive attention by Garsva found did not carry importance of the selected features. and Danenas [51] in application scoring. Similarly, they u Th s, they proposed feature weighting as one of the addition carried out model selection and hyperparameters tuning with procedure in the wrapper approach. The proposed model PSO-linSVM butwitha mixed searchspace for PSO, which aimed to search for hyperparameters of WSVM as well as is a slight modification as compared to previous work [47]. feature subsets with feature weighting. They compared the Comparison is done with SVM and LS-SVM (of different feature weighting method with t-test and entropy based kernels), of which the hyperparameters are tuned with PSO, method. Direct Search (DS), and SA, respectively. To address the data imbalance problem, they investigated the use of True () Ensemble Models. The two main types of ensemble models Positive Rate (TPR) and accuracy as the tness fi function. TPR are homogeneous (combining same classifiers) and hetero- is concluded as appropriate tness fi function for imbalance geneous (combining different classifiers). In credit scoring dataset. domain, [56–58] worked on homogeneous ensembles while Zhou et al. [42] presented Direct Search (DS) to tune Xia et al. [59] worked on heterogeneous ensembles. LS-SVM hyperparameters. They compared DS with GA, GS, and Design of Experiment (DOE). Among the four hybrid ImprovePredictiveAbility. Zhou et al. [56] pointed out induc- tive bias problem of single classifier when using fixed training models, DS-LS-SVM is the recommended approach due to its best performance. Yu et al. [44] conducted a similar samples and parameter settings. Therefore, they introduced ensemble model based on LS-SVM to reduce bias for credit experimentalsetupwith[42]whereDS,GA,GS,andDOEare presented for hyperparameters tuning. The main difference is scoring model. The two main categories of ensemble strate- that they considered class imbalance problem, so the model gies introduced are the reliability-based and weight-based. There are three techniques, respectively, for each category, tuned is weighted LS-SVM (WLS-SVM), where different 8 Advances in Operations Research resulting in a total of six LS-SVM-based ensemble models different GA-derived models are developed which consider being formed. Another research that proposed ensemble seeding (prior information from LOGIT as initial solution) model is by Ghodselahi [57]. They recommended using fuzzy and different encoding scheme (integer or binary). C-means clustering to preprocess the data before fed into Cai et al. [63] and Abdou [65] have included misclassifi- SVM. Then, 10 of the hybrid SVM base models formed the cation costs into their studies. Cai et al. [63] established credit ensemble models, using membership degree method to fuse scoring model with GA. The optimization problem is a linear all the base models as the final ensemble results. Xia et al. [59] scoring problem as in [62]. They computed the appropriate introduced a new technique named as bstacking. The main cutoff as the weighted average of good and bad customers idea is to pool models and fuse the end results in a single critical values. A tness fi function is formed that considered step. Four classifiers are used, i.e., SVM, Random Forest (RF), all components in the confusion matrix together with the XGBoost, and Gaussian Process Classifier (GPC), as the base associated misclassicfi ation costs. Abdou [65] compared per- learners due to their accuracy and efficiency. formance of GP with profit analysis and weight-of-evidence (WOE) model. Two types of GP are examined here: single Data Imbalance. Yu et al. [58] developed Deep Belief Network program GP and team model GP, which is a combination SVM-based (DBN-SVM) ensemble model in a different of single program GP for better results. They conducted approach where the main aim is to solve dataset imbalance sensitivity analysis based on dieff rent misclassicfi ation ratios problem. Their model has a three-stage procedure. In the and emphasized the importance to evaluate scorecard with first stage, data is partitioned into various training subsets misclassification costs. Only Yang et al. [68] addressed the with bagging algorithm, and each subset is resampled to bankruptcy prediction issue with CS. The authors developed rebalance the instances. In the second stage, SVM classifiers aCS model whichused Lev ´ y’s flight to generate new solution. are trained on the rebalanced training subsets. In the third stage, DBN is employed to fuse the final results. Proposed Rules Extraction. There are several researches focused on method is compared with SVM and ensemble SVM with rules extraction with different MA which are GP in [61, 64], majority voting. SA in [66],and ACOin [67,69].Onget al.[61]recom- mended using GP as an alternative to form credit models. GP undergone the same procedures as in GA to search for ... MA solution, but the main difference is that GP generates rules to do classification. The authors concluded several advantages of () Standard MA. Standard MA being attempted to form GP to build credit scorecard: nonparametric that is suitable credit scoring models are GA, GP,SA,ACO,and CS.The for any dataset, automatic discrimination function that do credit scoring problem is formulated with these MA as not require user-defined function as in NN, and better rules an optimization problem to be solved with respect to the obtained compared to TREE and rough set that generated objective functions. lower error. Huang et al. [64] proposed a 2-stage GP (2SGP). In the rfi st stage, IF-THEN rules are derived. They formulated Investigation of Predictive Ability. The predictive ability of MA GP to ensure that only useful rules are extracted. Based on credit models has been tested through application on specific these rules, the dataset will be reduced by removing instances credit domains, i.e., application scoring [60, 62, 63, 65] and that do not satisfy any rules or satisfy more than one rule. bankruptcy prediction [68]. The experiments of Desai et al. Then, the reduced data is passed on to the second stage of [60], Finlay [62], and Abdou [65] are specicfi study on a GP where the discriminant function is used to classify the particular country which are credit unions in Southeastern customers. US, large data of UK credit application, and Egyptian public Dong et al. [66] established SA rule-based extraction sector bank, respectively. In contrast, Cai et al. [63] and Yang algorithm (SAREA) for credit scoring. Similar to the previous et al. [68] conducted a general study based on the public rule extraction with GP in [64], the proposed SAREA is German dataset and simulated database of 20 companies, respectively. also in two-phase, and the extracted rules are the IF-THEN rules. In the rfi st phase, SA is run on initial rules and their Back in 1997, Desai et al. [60] investigated the predictive corresponding best accuracy rules are put into the final rule ability of AI techniques by comparing with traditional credit models LOGIT and DA. The AI techniques studied are three setof rfi stphase.Thebestrule from the rfi st phase isrequired for tness fi function computation in second phase to penalize variants of NN and GA. They classified the credit customers the accuracy. In the second phase, SA is run again on random into three classes (good, poor, and bad) instead of the usual initial rules to find their corresponding best accuracy rules to binary (good and bad). GA is used for discriminant analysis. form the final rule set, but the tness fi computation will be the With the aim to minimize the number of misclassification, penalize accuracy based on the best rule from rfi st phase. an integer problem is formulated with GA to make it acting The importance of comprehensibility for credit scor- equivalently to branch-and-bound method, then, solving the dual problem gives the final separating hyperplane. Finlay ing model had been pointed out [36, 93] as discussed in [62] pointed out the advantage of developing credit scoring Section 4.1.1(2). Martens et al. [67] researched on establishing model with GA due to its ability to form model based on a novel model that has good performance for both accuracy self-decide objective function. They proposed to build a linear and comprehensibility. They introduced ACO algorithm in scoring model with GA that maximizes the GINI coefficient. AntMiner+ as the potential credit scoring model. The rules Large dataset of UK credit application is experimented. Four have high comprehensibility which is crucial for business Advances in Operations Research 9 decisions and they also analysed the extracted rules to tuning of NN [75, 84], and hyperparameters tuning of SVM be integrated with Basel II accord. Recently, ACO-based [37, 42, 44, 47, 49, 51–53]. classification rule induction (CRI) framework is introduced For NN model tuning, Wang et al. [77] hybridized GA with NN to tune the input weight and bias parameters. They by Uthayakumar et al. [69]. They carried out their experiment on both qualitative and quantitative datasets, focusing on employed real-valued encoding for GA, using arithmetic bankruptcy prediction. ACO algorithm is modified based on crossover and nonuniform mutation and concluded that the concept of rule induction. Due to the ability of ACO tuning parameters with GA improved learning ability of NN. to provide better results in CRI, reducing rules complexity On the other hand, Lacerda et al. [75] established GA to tune and effective classification of abundant data, ACO is recom- hyperparameters in NN. They proposed a modified GA based mended in their study. on consideration of redundancy, legality, completeness and casuality. Training samples are clustered and they introduced () Hybrid MA-MA. Hybrid MA-MA involves two dieff rent cluster crossover algorithm for GA. Then, they utilized the MA being integrated together to form a new method. There proposed GA to form a multiobjective GA in seeking for are only two research works that have been proposed thus NN hyperparameters. Correa and Gonzalez [84] presented far. two hybrid models, GA and binary PSO (BPSO) for hyper- parameters tuning in NN. For both MA techniques, cost of Parameters Tuning. Jiang et al. [70] proposed the idea of using the candidate solutions is computed before proceeding to SA to optimize GA, forming hybrid SA-GA. SA is integrated the searching process. They presented a different approach into GA to update population by selecting chromosomes in examining their proposed models, where they conducted using Metropolis sampling concept from SA. Two variants of cost study on three different scoring models, i.e., application NN are the main classifiers used in this experiment. SA-GA scoring, behavioural scoring, and collection scoring. is utilized to search the input weight vector of the combined Several models that utilized GA to tune SVM hyperpa- NN classifiers. rameters [37, 42, 44] have been discussed in Section 4.1.1(3). In Zhou and Bai [37] experiment, the proposed model Rules Extraction. Aliehyaei and Khan [71] presented a hybrid worked best with GA tuned SVM. In Zhou et al. [42] experi- model of ACO and GP with a two-step task. ACO is ment, investigation on SVM hyperparameters tuning is con- responsible for searching for rule sets from the training ducted on DS, GS, DOE, and GA. In Yu et al. [44] experiment, set. The rulesextracted from ACO isthen fed into GP for they investigated LS-SVM hyperparameters tuning also with classification. DS, GS, DOE, and GA. From these three researches, it can be observed that they included GA tuned SVM-based classifiers () Hybrid MA-DM. Hybrid MA-DM methods include the into their experiments for comparison, implying that EA is a usage of a MA technique to assist the classification task of DM good alternative for hyperparameters tuning in SVM. Model classifier, thus improving model performance. selection and hyperparameters tuning on linear SVM from LIBLINEAR using GA and PSO have been investigated in a Rules Extraction. Past researches that aimed to do rules few studies [47, 51, 52], as discussed in Section 4.1.1(3) GA- extraction are presented in [38, 79] which utilized SA and linSVM showed better performance than PSO-linSVM on GP, respectively. Zhang et al. [38] proposed a hybrid credit bankruptcy prediction [47]. Then, Garva and Danenas [51] scoring model (HCSM) which is a 2-stage model incorpo- further experimented on modified PSO to form PSO-linSVM rating GP and SVM. In the first stage, GP is used to extract on application scoring. PSO is further modified by Danenas rules with misclassification of type I error and type II error and Garsva [52], forming a different version of PSO-linSVM taken as the tness fi function. In the second stage, SVM is to build bankruptcy prediction model. There are also two used as the discriminant function to classify the customers. models that tuned SVM with ABC [49, 53] as discussed in Jiang [79] incorporated SA and TREE as a new credit scoring Section 4.1.1(3). Both researches focused on corporate credit model. Rules from TREE are input as initial candidate for rating problem. Chen et al. [49] rfi st formulated ABC-SVM SA, then SA produced new decision rules for classification. then followed by a more detailed study in Hsu et al. [53]. Both They formed three TREE-SA credit models with different studies reached the same conclusion with ABC-SVM being discriminant function that account for type I error and type an effective method to tune hyperparameters and Hsu et al. II error, which is similar to Zhang et al. [38]. [53] indicated that ABC-SVM is also able to capture changing trend of credit rating prediction. Parameters and Hyperparameters Tuning. Generally, param- eters and hyperparameters have significant effect on model Features Extraction. Fogarty and Ireson [72] hybridized the performance. The difference between the two is that, param- GA and BN. GA is utilized to optimize BN by selecting cat- egories and attributes combinations from training data using eters are involved in model training, where the value can be evaluated by the model, whereas hyperparameters are cooccurrence matrix. The attributes combinations generated completely user-defined where its value cannot be evaluated are analogous to extracted features. Liu et al. [76] designed GP by the model. Therefore, tuning appropriate values for param- to extract features by selecting derived characteristics, which eters and hyperparameters are crucial in model building for are attribute combinations but determined with analysis credit scorecard. Metaheuristics is applied for parameters method and human communication. To ensure the derived tuning of NN inputweightand bias [77], hyperparameters characteristics are practical, the characteristics are generated 10 Advances in Operations Research with GP by maximizing information value together with with information gain. On the other hand, for Wang et al. application of human communication. Linear DA model is [55], the idea is to formulate a wrapper-based SVM model then built using these derived characteristics. Zhang et al. with multiple population GA. They incorporated information from different filter techniques to be input as initial solutions [78] formed hybrid model by incorporating GA, k-means clustering, and TREE together. The GA is introduced to do for the multiple population GA. attribute reduction, which is a kind of feature extraction. Marinakis et al. [80] formed wrapper model with ACO Binary encoding is applied and the candidate solutions in GA and PSO. The kNN and its variants (1-NN and weighted kNN) consist of breakpoints. Then k-means clustering is assigned to are being wrapped to do feature selection and classification. remove noise and TREE to do classification. The experiment is on multiclass problem using two datasets from a UK nonfinancial firm where the first dataset is to Features Selection. Hybrid models of MA-DM developed for do credit risk modelling and the second dataset is for audit features selection are [54,55,74,81,83,86,89,90].GA from qualification. Later, Marinaki et al. [82] conducted a research EA category is the most popular method to be hybridized with similar setup as in [80] andusedthe rfi st dataset as with DM classiefi rs [54,55,81,83,86,89] for solving feature in [80]. They proposed a different metaheuristics which is selectionincredit scoring domain. All thehybrid MA-DM HBMO to wrap the kNN and its variants. A music-inspired for feature selection are based on wrapper approach except SI technique, HS, is attempted by Krishnaveni et al. [90] for [87–89] which presented filter approach. recently to form a wrapper model with a kNN variant, i.e., Drezner et al. [74] constructed a new credit scoring 1-NN for feature selection. Besides, parallel computing is also model by incorporating TS with LR for feature selection, attempted with the model and reported a significant time- focusing on bankruptcy prediction. First, the selected feature saving with the paralleled version of the proposed method subset from TS is compared with a known optimal subset, compared to other wrapper models. subset from stepwise regression, and subset from maximum Apart from wrapper approach, filter approach models 𝑅 improvement. The TS-feature subset is reported to be have also been proven useful in the credit scoring domain. very competitive in selecting a near-optimal feature subset. Wang et al. [87] hybridized TS with rough sets (TSRS) Sustersic et al. [81] introduced GA to do feature selection to search for minimal reducts that served as the reduced with Kohonen subsets and random subsets. PCA is the feature subsets to be input into classifiers. Japan dataset is benchmark feature selection method compared with the experimented in the experiment. The feature subset from two GA-generated features. NN and LOGIT models are TSRSis fedinto RBFnetwork, SVM, andLOGIT.Later,Wang developed with the features generated. Their experiment is et al. [88] attempted hybridization of SS with rough sets specified on a Slovenian bank loan data. The authors also (SSRS) for feature selection. Similar experiment setup as in discussed the effect of setting cutoff point on the change in [87] is conducted, but with two differences: one additional typeIerror and typeIIerror.Huang and Wu [83]examined dataset is included and the classifiers included for comparison the effectiveness of GA for feature selection. GA is wrapped are different. Waad et al. [89] proposed another filter-based with kNN, BN, LOGIT, SVM, NN, decision tables, and three feature selection method with GA. The new method is a two- different ensembles to carry out feature selection task. In the stage lfi ter selection model. The rfi st stage formulated an first part, standalone classifiers (without wrapped with GA) optimization problem to be solved with GA. The main idea are compared. In the second part, GA-based feature selection in the rfi st stage is to overcome selection trouble and rank is compared with four filter feature selection techniques, aggregation problem and then sort the features according to i.e., chi-squared statistics, information gain, ReliefF, and their relevance. In the second stage, an algorithm is proposed symmetrical uncertainty. In the third part, every standalone to solve disjoint ranking problem for similar features and classifier is compared with their GA-wrapped counterpart. remove redundant features. Some proposed wrapper models included lfi ter tech- niques to gain useful feature information for improving the Features Discretization. There is only a single research con- standard wrapper approach. This type of wrapper model tributed by Hand and Adams [73] for demonstrating a new has been presented by [54, 55, 86]. Oreski and Oreski [86] model that does feature discretization in credit scoring. They tackled feature selection problem by proposing hybrid GA formed collaborated SA with weight-of-evidence (WOE) with NN. They incorporated four different filter techniques and generalized WOE, forming two wrapper models. The to develop the wrapper GA-NN. With the feature ranking main concept is to effectively discretize continuous attributes from the lt fi er methods, three main procedures are infused into appropriate intervals. The proposed SA discretization into the GA-NN. The three procedures are to reduce search technique is compared with quantile partitioning and equal space using the reduced features from the lfi ter ranking, interval division. refine the search space with GA, and induce diversity in the initial population with incremental stage using GA. There Simultaneous Hyperparameters Tuning and Features Selection. are two other wrapper models for solving feature selection The importance of hyperparameters tuning and feature selec- problem in credit modelling presented by Jadhav et al. tion has urged some researchers to resolve both problems [54] and Wang et al. [55] as discussed in Section 4.1.1(3). simultaneously [35, 41,85].Huang et al.[35]and Zhou et al. [41] developed GA-based wrapper model together with SVM Similarly, both researches formed novel wrapper models with filter information. Jadhav et al. [54] formulated wrapper- and LS-SVM respectively. Both studies have been discussed based SVM, kNN, and NB models with GA, incorporating in Section 4.1.1(3) and their research results have indicated Advances in Operations Research 11 SVM Models 22 2 2 11 1 AB C DE F GH I JK L M N Purpose A: Investigation of Predictive Ability B: Features Selection C: Hyperparameters Tuning D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 1: Summary of SVM models. GA wrapper can solve both problems effectively. Oreski models considered in each experiment for comparison. et al. [85] also solved hyperparameters tuning and feature Table 2 reports the counts of papers categorized by the types of models. Figure 1 illustrates the categorization of all the selection simultaneously but with NN as the main classifier. The research is conducted on a Croatian bank credit applicant SVM models based on the purposes. dataset. The proposed GA-based feature selection wrapped From Table 1, early stage of credit models with SVM are with NN (GA-NN) is benchmarked against other feature basically standalone SVMs for investigation of the predictive selection techniques, i.e., forward selection, information ability. As a result from these investigative experiments that gain, gain ratio, GINI index, correlation, and voting. For validated the effectiveness of SVM, it is soon labelled as hyperparameters tuning, the authors proposed Neural Net- one of the state-of-the-art credit scoring methods. Then, the work Generic Model (NNGM) which employed GA to tune development trend shifted to enhance the original SVM mod- hyperparameters in NN model. The features generated from els, where hybrid models formulation is the most popular the different methods are passed on to NNGM to do classifi- approach that remains active until recent years, with data cation. They also examined the effect of different cutoff points preprocessing and hyperparameters tuning outnumbered the on accuracy and study different misclassification ratios. other research purposes. Ensemble models are the latest research trend in credit scoring due to its ability to improve classification performances. This leads to involvement of ... Summary SVM in two situations; i.e., SVM is one of the benchmark () SVM Models. Developments of SVM models are summa- models against ensembles and SVM is the base classifier used rized in the upcoming tables and gur fi es. Table 1 arranges to form new ensembles. On the other hand, in view of the all the reviewed studies in chronological order to show SVM type in credit modelling, standard SVM has been most the development trend, summarizes the addressed issues, frequently used while SVM variants have apparently lesser research works. In view of the kernel used, linear and RBF and provides additional information on the SVM type with respect to the kernel used as well as details of the other kernel have been widely utilized in this domain. Number of Contributions 12 Advances in Operations Research ff Table 1: Summary of literature for SVM models. Authors SVM type Kernel Other Methods Remarks Standard SVM and Variants Baesens et al. (2003) [8] SVM, LS-SVM linear, RBF - (i) compare with LOGIT, DA, kNN, LP, BN, NN, TREE Van Gestel et al. (2003) [19] LS-SVM RBF - (i) multiclass corporate rating (ii) compare with LR, LOGIT, NN Huang et al. (2004) [20] SVM linear, RBF, polynomial - (i) multiclass corporate rating (ii) compare with NN Li et al. (2006) [21] SVM RBF - (i) compare with NN (ii) study misclassification error Lai et al. (2006) [22] LS-SVM, SVM RBF - (i) multiclass corporate rating (ii) compare with NN, LR, LOGIT Lee et al. (2007) [23] SVM RBF - (i) multiclass corporate rating (ii) compare with NN, DA, CBR Bellotti and Crook (2009) [24] SVM linear, RBF, polynomial - (i) application on large dataset - (ii) compare with LOGIT, LR, DA, kNN - (iii) support vector weights to select significant features Kim and Sohn (2010) [25] SVM RBF - (i) multiclass corporate rating on SME (ii) compare with NN, LOGIT Danenas et al. (2011) [26] SVM linear, RBF, polynomial, - (i) compare between SVMs of dierent libraries (LIBLINEAR, LIBSVM, Laplacian, Pearson, inverse WEKA, LIBCVM) distance, inverse square distance Lessmann et.al (2015) [11] SVM linear, RBF - (i) compare with LOGIT, TREE, ELM, kNN, DA, BN, ensembles (ii) recommendation to use different performance measures Louzada et.al (2016) [12] SVM not mentioned - (i) compare with LR, NN, TREE, DA, LOGIT, FUZZY, BN, SVM, GP, hybrid, ensembles (ii) study class imbalance problem Boughaci and Alkhawaldeh (2018) [18] SVM not mentioned - (i) compare with kNN, BN, NN, TREE, SVM, LOGIT and ensembles Advances in Operations Research 13 fi fi Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Mushava and Murray (2018) [27] SVM RBF - (i) compare with LOGIT, DA, extensions of LOGIT and DA, ensembles Modiﬁed SVM Wang et.al (2005) [28] SVM linear, RBF, polynomial fuzzy membership (i) introduce bilateral weighting error into classification problem (ii) compare with U-FSVM, SVM, LR, LOGIT and NN Harris (2015) [29] SVM linear, RBF k-means cluster (i) reduce computational time (ii) compare with LOGIT, k-means+LOGIT, SVM, k-means+SVM Yang (2017) [30] WSVM RBF, KGPF - (i) dynamic scoring with adaptive kernel (ii) ranking of kernel attributes to solve black box model (iii) compare with LOGIT Li et al. (2017) [31] L2-SVM not mentioned - (i) reject inference (ii) compare with LOGIT, SVM Tian et al. (2018) [32] L2-SVM no kernel - (i) reduce computational time (ii) reject inference and outlier detection (iii) compare with LOGIT, kNN, SVM, SSVM Maldonado et al. (2017) [33] SVM, linear - (i) feature selection 1-norm SVM (ii) acquisition cost into formulation of SVM (iii) application and behavioural scoring (iv) study class imbalance problem (v) compare with SVM (filter and wrapper feature selection) Maldonado et al. (2017) [34] SVM, linear - (i) prot-based feature selection LP-norm SVM (ii) group penalty function included in SVM formulation (iii) compare with LOGIT, SVM (filter, wrapper feature selection) Hybrid SVM Huang et al. (2007) [35] SVM RBF GA (i) hyperparameters tuning, features selection (wrapper approach) (ii)compare with GP, NN, TREE Martens et al. (2007) [36] SVM RBF C4.5, Trepan, (i) rules extraction G-REX (ii) compare with LOGIT, SVM, TREE Zhou and Bai (2008) [37] SVM RBF rough sets (i) features selection (lter approach) (ii) compare with DA, NN, SVM, SVM wrapped by GA Zhang et al. (2008) [38] SVM RBF GP (i) rules extraction (ii) comparewith SVM,GP, LOGIT, NN,TREE Yao (2009) [39] SVM RBF neighbourhood (i) features selection (filter approach) rough set (ii) compare with DA, LOGIT, NN Xu et al. (2009) [40] SVM RBF link analysis (i) features extraction with link relation of applicants (ii) compare with SVM 14 Advances in Operations Research ff Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Zhou et al. (2009) [41] WSVM linear, RBF GA (i) hyperparameters tuning, features selection (wrapper approach) (ii) features weighting (iii) compare with LR, LOGIT, NN, TREE, kNN, Adaboost Zhou et al. (2009) [42] LSSVM RBF DS, GA, GS, DOE (i) hyperparameters tuning (wrapper approach) (ii) compare with LOGIT, kNN, DA, TREE Chen and Li (2010) [43] SVM RBF DA, TREE, (i) features selection (filter approach) rough sets, (ii) compare with SVM F-score Yu et al. (2010) [44] WLS-SVM RBF DS, GA, GS, DOE (i) hyperparameters tuning (wrapper approach) (ii) study class imbalance problem (iii) compare with results from [8, 45] Hens and Tiwari (2011) [46] SVM linear stratified sampling (i) reduce computational time (ii) compare with SVM, NN, GP Danenas and Garsva (2012) [47] SVM from linear PSO, GA (i) model selection LIBLINEAR (ii) hyperparameters tuning (wrapper approach) Chen et al. (2012) [48] SVM RBF k-means cluster (i) reject inference cluster (ii) multiclass problem with different cuto points Chen et al. (2013) [49] SVM RBF ABC (i) hyperparameters tuning (wrapper approach) (ii) compare with SVM tuned with GA and PSO Han et al. (2013) [50] SVM linear orthogonal dimension (i) features extraction with dimension reduction reduction (ii) compare with LOGIT Garsva and Danenas (2014) [51] LS-SVM, linear, RBF, polynomial, PSO, DS, SA (i) model selection SVM from sigmoid (ii) hyperparameters tuning (wrapper approach) LIBLINEAR (iii) study class imbalance problem (iv) compare among all SVM and LS-SVM tuned with PSO, DS, SA Danenas and Garsva (2015) [52] SVM from linear PSO (i) model selection LIBLINEAR (ii) hyperparameters tuning (wrapper approach) (iii) compare with LOGIT, RBF network classifier, SVM tuned with DS Advances in Operations Research 15 fi fi fi Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Hsu et al. (2018) [53] SVM RBF ABC (i) hyperparameters tuning (wrapper approach) (ii) compare with LOGIT, SVM tuned with GS, GA and PSO Jadhav et al. (2018) [54] SVM RBF GA (i) features selection (wrapper approach) (ii) compare with standalone SVM, kNN, NB and their wrappers with GA with standard GA Wang et al. (2018) [55] SVM RBF multiple population GA (i) features selection (wrapper approach) (ii) compare with MPGA-SVM, GA-SVM, SVM Ensemble Model Zhou et al. (2010) [56] LS-SVM RBF fuzzy C-means (i) homogeneous ensemble (ii) compare with ensemble and single classiers Ghodselahi (2011) [57] SVM linear, RBF, polynomial, - (i) homogeneous ensemble sigmoid (ii) compare with ensemble and single classiers Yu et al. (2018) [58] SVM RBF DBN (i) homogeneous ensemble (ii) study class imbalance problem (iii) compare with ensemble and single classifiers Xia et al. (2018) [59] SVM RBF RF, GPC, (i) heterogeneous ensemble XGBoost (ii) compare with ensemble and single classiers 16 Advances in Operations Research Table 2: Type of SVM models. provide details of fitness function and models considered for comparison in each experiment. Then, Table 4 reports the Type Count count of MA models categorized by model type. Figure 2 Standard SVM and its Variants 13 illustrates the research paper counts corresponding to its Modified SVM 7 research purposes. Hybrid SVM 20 Chronological order of the MA models in Table 4 shows Ensembles SVM 4 the modelling trend. Early MA models are standalone MA with investigative purposes. Initiation of MA in credit mod- Total 44 elling is due to the increasing popularity of AI techniques. The development trend in later years is formulation of new hybrid models that persists until recent years. Among the As reported in Table 2, hybrid SVM is the most frequently hybrid models, a majority of the studies are the hybrid MA- adopted approach to construct new SVM credit models. DM where MA techniques act as the assistant of DM to do This is followed by standalone SVM, modified SVM, and the classification task. In view of the MA techniques utilized, ensembles. Louzada et al. [12] review study also revealed the GA is considered as the pioneer as well as the dominant MA same trend where hybrid models are most popular among in credit scoring eld fi since its usage can be observed from the researches. In hybrid models, the method hybridized with earliest study till recent while GP is the second popular MA. SVM acts to assist the classification task without chang- Promising performances with hybrid GA and GP opened up a ing the SVM algorithm. us, Th the construction of hybrid new page for MA in credit modelling where other MA started models is perceived as a direct approach. Standalone SVM to received attentions. comes in second place due to its recognition as the state- Based on the types of models formed with MA as sum- of-the-art technique. Its involvement in recent studies as marized in Table 4, hybrid MA-DM is the obvious dominant, benchmark model further consolidated its recognition in followed by standard MA and hybrid MA-MA. The abundant the credit domain. Modified SVM requires a complicated studies to construct hybrid MA-DM indicates that MA can process to alter the SVM algorithm, thus receiving relatively effectively enhance standalone DM credit models perfor- lesser attention. Ensemble models are new modelling concept mances. Standard MA and hybrid MA-MA have much lesser which have just being researched very lately, leading to the research works. This may be due to the subjectivity of MA least number of contributions. models in formulating the optimization problem to classify Figure 1 provides a quick summary on the research credit customers that pose a difficulty for a general usage. purposes handled in past researches utilizing SVM models. A quick overview of the research purposes with MA According to the counts of papers for each purpose, the top models is illustrated in Figure 2. Features selection is the main research purposes that take up the majority in this review issue dealt that has the most number of contributed works, study are investigation of predictive ability, features selec- followed by rules extraction and hyperparameters tuning. tion, and hyperparameters tuning. Frequent usage of SVM This outcome infers that MA is a useful tool to do data prepro- models in various types of credit datasets and involvement cessing. High comprehensibility is always the crucial criterion in benchmark experiments to investigate models predictive for credit models. Having a number of MA studies that solve ability is an indication of its significance in the domain. this problem with rules extraction is recognition that MA can Data preprocessing with features selection and fine tuning produce transparent credit scorecard. In addition, AI models of SVM hyperparameters in the second place veriefi d the are sensitive to hyperparameters; thus automated tuning with importance of these two to effectively ensure quality clas- MA in place of manual tuning has been under continuous sification of SVM. Therefore, there are another two pieces research. The success of features selection and hyperparame- of works which conduct both tasks simultaneously with the ters tuning in ensuring good performance urged few studies proposed new models. Besides, there are also few researches to conduct both simultaneously. Other than preprocessing which used features extraction to preprocess the dataset data with features selection, there are a few works that use MA instead of features selection. However, these do not solve to do features extraction and discretization. Other minority the main drawbacks of SVM which are black box property research purposes are investigation of predictive ability and and inefficient computation with large instances. Hence, parameter tuning. research on rules extraction and computational efficiency is the remedy corresponding to the two problems. Other credit () Overall Summary. Being two dieff rent AI techniques, both scoring issues confronted using SVM have a minority count methods have been actively researched throughout the years of contributions. They are outlier handling, improvement that unleashed their great potential in the credit scoring of classification performances, reject inferences, dynamic domain. The roles of these two in credit modelling are scoring, and data imbalance. The attempts to solve various illustrated in Figure 3 based on the research purposes. issues with SVM imply its worthiness to be considered as the Features selection is the issue most consistently addressed alternative in credit scoring domain. by both models. For features selection, both SVM and MA () MA Models. MA models development is summarized have almost the same number of works in addressing this in the upcoming tables and figures. Table 3 is the chrono- issue. However, the main difference is that SVM is the tool to logicalsummary of allreviewedstudies with MA to show do classification directly whereas MA indirectly do classifica- the modelling trend, summarize the issues addressed, and tion as it acts as the assistant to the hybridized DM models 𝑇𝑁 𝑇𝑃 Advances in Operations Research 17 fi fi fi ffi ffi 𝑃𝑅 fi Table 3: Summary of literature for MA. Authors MA (category) Other Methods tness function Remarks Standard MA Desai et al. (1997) [60] GA (EA) - no. of misclassication (i) multiclass problem (ii) compare with NN, LOGIT, DA Ong et al. (2005) [61] GP (EA) - mean absolute error (i) rule extraction (ii) compare with LOGIT, NN, TREE, rough sets Finlay (2006) [62] GA (EA) - GINI coecient (i) large credit data application (ii) compare with LOGIT, LR, NN Cai et al. (2009) [63] GA (EA) - error rate (i) include misclassification cost into tness function Huang et al. (2006) [64] 2stage GP (EA) - mean absolute error (i) rules extraction (ii) compare with LOGIT, TREE, kNN, GP Abdou et al. (2009) [65] GP (EA) - (sum of square error) + (i) study effect of misclassification costs (classification error) (ii) compare with prot analysis and weight-of-evidence measure Dong et al. (2009) [66] SA (IB) - accuracy × similarity function × (i) rules extraction ( is a coecient) (ii) compare with DA, kNN, TREE Martens et al. (2010) [67] ACO (SI) - coverage + confidence (i) rules extraction (ii) compare with TREE, SVM, majority vote Yang et al. (2012) [68] CS (SI) - error rates (i) compare with Altman’s Z-score and SVM Uthayakumar et al. (2017) [69] ACO (SI) - + (i) rules extraction (ii) compare with LOGIT, NN, RF, RBF network Hybrid MA-MA Jiang et al. (2011) [70] SA+GA NN 𝛼(1 −) (i) SA optimizes GA (IB+EA) (ii) parameter tuning (iii) compare with standalone NN and GA-optimized NN Aliehyaei and Khan (2014) [71] ACO (SI), GP (EA), - mean absolute error (i) rules extraction by ACO to input to GP ACO+GP (ii) compare with ACO and GP Hybrid MA-DM Fogarty and Ireson (1993) [72] GA (EA) BN accuracy (i) features extraction (ii) large credit data application (iii) compare with default rule, BN, kNN, TREE 𝑇𝑁 𝑇𝑃 18 Advances in Operations Research fi fi 𝐹𝑃 𝑅 𝐹𝑃 𝑇𝑁 𝐹𝑁𝑅 𝛼 𝛼 fi 𝛼 𝜃 𝑁 𝑏 Table 3: Continued. Authors MA (category) Other Methods tness function Remarks Hand and Adams (2000) [73] SA (IB) WOE, weighted likelihood (i) features discretization WOE (ii) compare with LOGIT, DA, two other discretization methods Drezner et al. (2001) [74] TS (IB) LR (i) features selection (wrapper approach) (ii) compare with Altman’s Z-score Lacerda et al. (2005) [75] GA (EA) NN average of individuals with (i) hyperparameters tuning rank of individual (ii) compare with NN, consecutive learning algorithm, SVM from a population Huang et al. (2007) [35] GA (EA) SVM accuracy (i) hyperparameters tuning, feature selection (wrapper approach) (ii) compare with NN, GP, TREE Zhang et al. (2008) [38] GP (EA) SVM accuracy+expected (i) rules extraction misclassification cost (ii) compare with SVM, GP, TREE, LOGIT, NN Liu et al. (2008) [76] GP (EA) DA Information value (i) features extraction (select derived characteristics) (ii) large dataset from nance enterprise (iii) compare with DA Wang et al. (2008) [77] GA (EA) NN 1/MSE (i) parameters tuning (ii) compare with NN Zhang et.al (2008) [78] GA (EA) TREE (1-info entropy)+ (i) features extraction (1 − )(1-info entropy) (ii) compare with TREE, NN, GP, GA-optimized SVM, rough set Jiang (2009) [79] SA (IB) TREE (i) 𝛼⋅ + 𝛽⋅ (i) rules extraction 2 2 (ii) (𝐹− ) +(𝑅− ) (ii) compare with TREE (iii) ⋅ /( + 1) + ⋅ /( + 1) Marinakis et al. (2009) [80] ACO (SI), kNN, 1-NN, not mentioned (i) multiclass problem, features selection (wrapper approach) PSO (SI) weighted kNN (ii) compare with wrapper models of GA and TS with kNN (and variants) Sustersic et al. (2009) [81] GA (EA) NN accuracy>𝜃 and RMSE <𝜃 , (i) feature selection (wrapper approach) , preset threshold (ii) compare with NN, LOGIT (features from PCA) Zhou et al. (2009) [42] GA (EA) SVM error rate (i) hyperparameters tuning (ii) compare with LOGIT, kNN, DA, TREE Zhou et al. (2009) [41] GA (EA) WSVM AUC (i) hyperparameters tuning, feature selection (wrapper approach) (ii) compare with LR, LOGIT, NN, TREE, kNN, Adaboost Marinaki et al. (2010) [82] HBO (SI) kNN, 1-NN, accuracy (i) multiclass problem, features selection (wrapper approach) weighted kNN (ii) compare with wrapper models of GA, PSO, TS, ACO with kNN (and variants) Huang and Wu (2011) [83] GA (EA) kNN, BN, TREE, LR, accuracy (i) feature selection (wrapper approach) SVM, NN, Adaboost, (ii) compare with kNN, BN, TREE, LOGIT, SVM (features from Logitboost, Multiboost lter selection approach) 𝑖𝑗 𝑖𝑗 𝑖𝑗 𝑖𝑗 Advances in Operations Research 19 fi fi fi ff 𝑖𝑠𝑡𝑎𝑛𝑐𝑒 fi 𝑎𝑏𝑠 Table 3: Continued. Authors MA (category) Other Methods tness function Remarks Yu et al. (2011) [44] GA (EA) LS-SVM accuracy (i) hyperparameters tuning (ii) study class imbalance problem (iii) compare with results from [8, 45] Correa and Gonzalez (2011) [84] BPSO (SI), NN AUC (i) hyperparameters tuning GA (EA) (ii) large credit dataset (iii) cost study on application, behavioural and collection scoring (iv) compare with LOGIT, NN, Global Optimum Oreski et al. (2012) [85] GA (EA) NN accuracy (i) hyperparameters tuning, feature selection (wrapper approach) (ii) study eect of misclassication costs (iii) compare their proposed model with features from filter selection approach Danenas and Garsva (2012) [47] GA (EA),PSO (SI) SVM TPR (i) model selection (ii) hyperparameters tuning Chen et al. (2013) [49] ABC (SI) SVM not mentioned (i) multiclass problem (ii) hyperparameters tuning (iii) compare with SVM (tuned with GA and PSO) Garsva and Danenas (2014) [51] PSO (SI) SVM (i) TPR, (ii) accuracy (i) model selection (ii) hyperparameters tuning (iii) study class imbalance problem (iv) compare among all SVM, LS-SVM tuned with PSO, DS, SA Oreski and Oreski (2014) [86] GA (EA) NN accuracy (i) feature selection (wrapper approach) (ii) compare with standard wrapper-based NN with GA Danenas and Garsva (2015) [52] PSO (SI) SVM TPR (i) model selection (ii) hyperparameters tuning (iii) compare with LOGIT, RBF network Wang et al. (2010) [87] TS (IB) NN, SVM, LOGIT entropy (i) features selection (filter approach) (ii) compare with NN, SVM, LOGIT with full features Wang et al. (2012) [88] SS (SI) NN, TREE, LOGIT entropy (i) features selection (filter approach) (ii) compare with NN, TREE, LOGIT with full features Waad et al. (2014) [89] GA (EA) LOGIT, SVM, TREE ∑ 𝑤𝑒𝑔ℎ𝑡 ×𝐷(, ) (i) feature selection (lter approach) (ii) compare with LOGIT, SVM,TREE (features from other lter selection and rank aggregation methods) Hsu et al. (2018) [53] ABC (SI) SVM 1/(1 + (𝑥 )),if (𝑥 )≥ 0 (i) multiclass problem 1+ ( (𝑥 )),if (𝑥 )< 0 (ii) hyperparameters tuning (iii) compare with LOGIT and SVM (tuned with GS, GA, PSO) Jadhav et al. (2018) [54] GA (EA) SVM, kNN, NB accuracy (i) features selection (wrapper approach) (ii) compare with standalone SVM, kNN, NB and their wrappers with GA Wang et al. (2018) [55] multiple population SVM accuracy (i) features selection (wrapper approach) GA (EA) (ii) compare with MPGA-SVM, GA-SVM, SVM Krishnaveni et al. (2018) [90] HS (SI) 1-NN accuracy (i) features selection (wrapper approach) (ii) computational time reduction (iii) compare with standalone SVM, TREE, kNN, NB, NN and their wrappers with GA and PSO 20 Advances in Operations Research MA Models 00 0 0 0 0 AB C D E F G H I J K L M N Purpose A: Investigation of Predictive Ability B: Features Selection C: Hyperparameters Tuning D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 2: Summary of MA models. Table 4: Type of MA models. as the next with seven contributions where all the seven are the collaboration of MA with SVM. u Th s, MA can be viewed Type Count as a recommended tool to tune SVM hyperparameters. Standard MA 10 Simultaneous features selection and hyperparameters tuning Hybrid MA-MA 2 result in a total of three studies, with two of them being hybrid Hybrid MA-DM 31 MA and SVM. The rest of the research purposes have shown dominance Total 43 in either SVM or MA. Feature discretization has only been attempted by MA while reject inferences, improvement of performances, computational ecffi iency, dynamic scoring, which are responsible for the classification. Investigation of imbalance datasets, and outlier handling have only been predictive ability comes as the second top research purposes. addressed using SVM. MA models have taken into account SVM models have much greater number of researches as much lesser issues as compared to SVM since MA have been compared to GA. This indicates that SVM is already a focused more to solve features selection and rules extraction. recognized credit scoring model as it is frequently included in comparative studies and attempted in different specific domains. MA has lesser works under this purpose as it is .. Assessment Procedures. In order to assess credit models performance, they are usually compared with other standard seldom involved in benchmark experiments that may be due to its subjectivity in model building. Then, rules extraction credit models applied on the selected credit datasets and comes in the third largest number of researches, with MA evaluated with appropriate performance measures. us, Th models more than SVM. This shows the great ability of MA to the assessment procedures are categorized into benchmark develop transparent model. Hyperparameters tuning comes experiments, performance measures, and credit datasets. Number of Contributions Advances in Operations Research 21 Overall Categorization AB C D E F G H I J K L M N Purpose SVM Models A: Investigation of Predictive Ability B: Features Selection MA Models C: Hyperparameters Tuning Both D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 3: Overallsummary for bothmodels. ... Benchmark Experiments. Benchmark experiments comparative studies to include sufficient huge number of include comparisons of the proposed models with other methods as benchmark to be able to provide sufficient standard credit models. Tables 1 and 2 provided brief information to serve as a guideline for future research. Since summary on the models considered for comparison in every ensembles are formulated from assembly of a number of experiment. Detailed experiment setup shall be referred to standalone classifiers, authors that proposed new ensembles the original paper. Table 5 presents the categorization of the usually have to compare new ensembles not only with stan- type of benchmark experiment carried out with SVM and dalone classifiers but also with standard available ensembles. MA models. Small scale benchmark is the most common approach which can be further broken down into four main parts, As reported in Table 5, it can be seen that inclusion of model comparison has been a standard approach to make i.e., comparison only with the counterpart techniques of conclusion on the proposed models. Most of the studies the proposed model, comparison only with either statistical adopted internal benchmark approach to make comparison or AI techniques, and comparison with both statistical and with other models based on the same experimental setup AI techniques. For both SVM and MA models, the most for the credit data. Only Yu et al. [44] adopted external preferred small scale benchmark is comparison with both benchmark approach to compare their proposed models with statistical and AI techniques. Besides, LOGIT and NN are other models. Chen et al. [48] and Cai et al. [63] are the the most frequently involved statistical and AI techniques, rare cases in the literature which do not benchmark their respectively. proposed models with others. Large scale benchmark is a rare approach with only several studies presented this in the past. It can be noticed ... Performance Measures. There are four main types of that research work with large scale benchmark are those that performance measures being used to make inference on the conducted comparative studies and formulate new ensemble models performances. Cutoff-dependent measures are those models for performance improvement. It is necessary for directly obtained or computed from the confusion matrix, Number of Contributions 22 Advances in Operations Research Table 5: Benchmark experiments. Benchmark Type SVM Models MA Models Large Scale [8,11,12,18,27,56–59] - SmallScale Counterpart [33,40,43] [70,71,79,80,82,83,85,86,89] [77, 87, 88] [47,49,51,54,55] Statistical [30,50] [65,73,74,76] AI [20, 21, 46] [67, 72, 75, 78] [35] Statistical&AI [22–25,29,31,32,34,36] [60–62,64,66,69,81,84,90] [19, 28, 37, 39] [68] [38,41,42,52,53] External [44] None [48] [63] Table 6: Performance measures. Type SVM Models MA Models Cutoff-dependent [12,19–21,23,25,26,28,30,36,37,39,40,46,48] [60,61,63–72,74–79,81–83,85,86,89,90] [35,38,42,44,47,49,51–53,55] Cutoff-independent [24, 33, 34] [84] [41] Mixture [8,11,18,22,27,29,31,32,43,50,56,57,59] [1,62,87,88] [54] Business-oriented [33, 34] [65, 72, 76, 86] Others [58] - where the cutoff point is often problem-dependent. Cutoff- to provide different perspective of interpretation. In one of independent measures are those computed to determine the latest and largest comparative studies by Lessmann et the discriminating ability of the model. Mixture indicates al. [11], they have recommended the use of more cutoff- the usage of both cutoff-dependent and cutoff-independent independent measures with the aim to explain models in measures while business-oriented measures are those com- different perspective. Therefore, their recommendations have puted with the misclassica fi tion costs. Table 6 summarizes been adopted in recent research [59]. Protfi and loss is oen ft the final aim for financial institu- the performance measures in the literature of SVM and MA models. tions as credit scoring is treated as a risk aversion tool. Only a minority of studies explained their models with business- Cutoff-dependent measures are the most popular indica- oriented measures. They are Expected Misclassfication Costs tor utilized by researchers, especially accuracy or its counter- (EMC) [65, 86], profitability [72], and self-den fi ed profit or part error rate that measures the number of correct classifi- cost measures [33, 34, 76]. Note that all researches utilizing cations in a straightforward manner. Among them, several business-oriented measures also included cutoff-dependent studies ofSVM models [21, 22,28, 30–32, 40,57] and MA or cutoff-independent measures to evaluate their models. models [70, 77, 81, 83] are interested in the misclassifications Only Yu et al. [58] applied weighed accuracy that involved which is believed to pose higher risk for financial institutions; imbalance rate, revenue and cost to compute a new version of thus type I and type II errors are included. Although cutoff- accuracy. dependent measures are direct in presenting performances of classifier, a main drawback has been denoted by [4, 11, 24], where researchers often do not address the cutoff point used. ... Model Evaluation. Researchers make inferences and u Th s, there are a few studies believed cutoff-independent conclusions based on the reported performance measures. performance measures are sucffi ient to serve as guideline for The conclusions are often to show that the proposed models model performances as reported in Table 6, with Area Under are competitive or outperform the other standard credit mod- Receiver Operating Characteristics Curve (AUC) being most els based on the numerical results. In addition, some studies popularly used. conducted statistical tests to show evident improvement of Cutoff-dependent measures from confusion matrix and the proposed models. cutoff-independent measures of model discriminating ability There are several studies of SVM models [29, 36, 46, 59], are both informative for decision makers. Hence, a few stud- MA models [60,69,71, 78,84],and joint MA-SVM [41, ies have included both types of measures in their experiment 47, 51, 52] that do not have numerical outperformance of Advances in Operations Research 23 Table 7: Credit datasets. Type SVM Models MA Models Specific CR: BankScope [19], England [22], CR:UK[80,82] Taiwan/US [20], Korea [23, 25], BP:Simulated data [68] Taiwan [21], credit card [24] AS: Southeastern US [60], UK [62], BP: US [26] Egyptian [65], Compustat [74], AS: German [30], LC P2P [31], Shenzhen [70, 77], Croatian [85], LC/Huijin P2P [32], China [37, 48] UK/simulated [1] BS: Chilean [27, 33, 34] AS/BS/CS:local bank[84] BP:US [47, 52] CR:Compustat [49] General AS:[12,39, 43,57] AS:[61,63,64,66,70,71,75,78,83,87,88,90] AS:[35,38,41,42,44,51,55] Specific & AS:UKCDROM [28],Barbados [29], AS:Croatian[86], General Benelux [36] Tunisian/home equity [89] BP:ANALCAT [69] AS/BP/CR:SME/ BankScope [67] AS:Taiwan [54] CR:Compustat [53] Large scale AS: [8, 11] - comparison AS/BP:[18] their proposed models with the compared models. However, bankruptcy prediction. Behavioural scoring has received very they have contributed to another aspect, i.e., outperformance less attention. For general studies, there are three datasets: in timing [29, 46], proper handling of imbalance [59], German, Australian, and Japanese datasets available in the transparent rules [36, 69], and presentation of competitive UCI repository, and all of them are application scoring data. new methods [41,47,51,52,60,71, 78,84]. Among them, German and Australian have been widely Some comparative studies [8, 11, 12] identified certain involved in research. There are also several specicfi studies that included UCI datasets as evaluation purpose, and sim- methods to have best performance and recommended for future research but they did not penalize the use of other ilarly, German and Australian are still the dominant usage techniques since the reported performance is still very com- among researchers. It can be noticed that almost all of the petitive, whereas comparative studies by [18, 26] did not result studies utilized small numbers of datasets for investigation, in outperformance of any methods among the compared with only comparative studies involving large amount of models. The rest of the research articles reviewed in this study datasets. have reported their proposed models having better results than the others. .. Results from Literature. German (G) and Australian Instead of solely depending on numerical improvement (A) datasets have been frequently included in credit scoring to detect outperformance, a minority of the studies [8, 11, 12, domain using SVM and MA models. This section compiles 19,21,23,29,34–36,43,50, 53, 55,56, 59,60, 67,89] have those contributed works based on the research purposes to utilized statistical tests for a more concrete support of their provide information on which model type with the handled results. Most commonly used statistical tests are paired t-test, issues have shown good performance on both datasets. Note Mc Nemar test, Wilcoxon signed rank test, and Friedman that the compilation is only based on accuracy as it is the test. common reported measures. For researches that reported only error rate, it is converted to accuracy. Researches that ... Credit Datasets. SVM and MA credit models have been have utilized these datasets but do not report accuracy are applied on different types of credit datasets, i.e., application not included here. Then, for usage of more than one standard scoring (AS), behavioural scoring (BS), collection scoring SVM and its variants, as well as studies that formulated more (CS), corporate rating (CR), and bankruptcy prediction (BP). than one new model, only the best performing results are There have been two main types of studies which are specific recorded. The results are compiled in Table 8. studies particularly on a country’s financial institutions and The computed mean in Table 8 is the overall general general studies using publicly available datasets from the UCI performance of the models on both datasets. The high value repository [94]. Table 7 summarizes the credit datasets usage. of mean accuracy is an indication of good performance For specific studies, they have focused on particular from SVM and MA models in the literature. The computed standard deviation in Table 8 is viewed as an indicator to financial institutions, with application scoring being the mostly utilized data type, followed by corporate rating and generalize the range of the accuracy that is believed to take 24 Advances in Operations Research Table 8: Results from literature. Purpose Model Type Authors G A Investigation of predictive standard SVM and Baesens et al. [8] 74.30 ability its variants Lessmann et al. [11] 75.30 86.00 Boughaci and Alkhawaldeh [18] 69.90 80.70 standalone MA Cai et al. [63] . - Computational efficiency modified SVM Harris [29] . - hybrid SVM Hens and Tiwari [46] 75.08 85.98 Improvement of classfication ensembles Zhou et al. [56] . - performance Ghodselahi [57] . - Xia et al. [59] . 86.29 Rules extraction hybrid SVM Martens et al. [36] - 85.10 standalone MA Ong et al. [61] 77.34 . Huang et al. [64] Dong et al. [66] 72.90 - Martens et al. [67] . - Uthayakumar et al. [69] - 86.37 hybrid MA-MA Aliehyaei and Khan [71] 70.70 84.30 hybrid MA-DM Zhang et al. [38] Jiang et al. [79] 73.10 - Features extraction hybrid SVM Xu et al. [40] - Han et al. [50] 75.00 - hybrid MA-DM Zhang et al. [78] 77.76 Features selection hybrid SVM Yao [39] 76.60 87.52 Chen and Li [43] 76.70 86.52 hybrid MA-DM Jadhav et al. [54] . Wang et al. [55] . 86.96 Huang and Wu [83] - 87.54 Oreski and Oreski [86] . Krishnaveni et al. [90] . Wang et al. [88] - 88.90 Hyperparameters tuning hybrid MA-DM Zhou et al. [42] 77.10 86.96 Yu et al. [44] 78.46 90.63 Garsva and Danenas [51] . . Hsu et al. [53] . Lacerda et al. [75] - 86.05 Simultaneous features hybrid MA-DM Huang et al. [35] 77.92 86.90 selection & hyperparameters tuning mean 77.56 87.75 standard deviation 3.35 2.64 into account the variation in different experimental setup for of data partitioning, i.e., k-fold cross validation (k-fold), every model development. holdout validation (holdout), repeated k-fold cross validation The general mean and standard deviation in Table 8 (rep k-fold), and repeated holdout validation (rep holdout). provide an overview on the models performances through Table 9 shows that k-fold and holdout are the most compilation from all studies regardless of the data splitting adapted data splitting strategies while rep k-fold and rep strategies adopted. Data splitting methods are influential holdout are less popular, which may be due to the high computational effort required for both rep k-fold and rep on the experiments end results. The general compilation in Table 8 is further detailed in Table 9 to take into consideration holdout. Instead of a general mean and standard deviation the effect of different data splitting strategies. The detailed as in Table 8, the specific mean and standard deviation for analysis is conducted by categorizing the studies (except Jiang each category are reported in Table 9, which is believed to [79] as no data splitting mentioned) into four main types be less biased and more reliable metrics as being grouped Advances in Operations Research 25 Table 9: Results categorized with data splitting methods. Data Split Authors G A k-fold cv Dong et al. [66] 72.90 - Uthayakumar et al. [69] - 86.37 Zhang et al. [38] Xu et al. [40] - Han et al. [50] 75.00 - Zhang et al. [78] 77.76 Yao [39] 76.60 87.52 Chen and Li [43] 76.70 86.52 Jadhav et al. [54] . Wang et al. [55] . 86.96 Huang and Wu [83] - 87.54 Oreski and Oreski [86] . Wang et al. [88] - . Zhou et al. [42] 77.10 86.96 Yu et al. [44] . Hsu et al. [53] . Huang et al. [35] 77.92 86.90 mean 78.20 88.56 standard deviation 2.93 1.92 holdout Baesens et al. [8] 74.30 Cai et al. [63] . - Boughaci and Alkhawaldeh [18] 69.90 80.70 Harris [29] . - Zhou et al. [56] . - Ghodselahi [57] . - Martens et al. [36] - 85.10 Martens et al. [67] . - Aliehyaei and Khan [71] 70.70 84.30 Garsva and Danenas [51] . . mean 77.07 85.32 standard deviation 4.47 3.20 rep k-fold Lessmann et al. [11] 75.30 86.00 Xia et al. [59] . 86.29 Krishnaveni et al. [90] . mean 78.01 88.60 standard deviation 2.56 4.25 rep holdout Ong et al. [61] 77.34 . Huang et al. [64] Lacerda et al. [75] - 86.05 mean 78.42 87.83 standard deviation 1.52 1.61 homogeneously according to the data splitting methods. For may be due to the usage of SVM without the hyperparameters all the categories, across both datasets, the mean accuracies tuning procedure. Besides, another high standard deviation are relatively high, showing the effectiveness of building new comes from the rep k-fold category only in Australian dataset. credit models with both SVM and MA. This high deviation is from a higher accuracy results from However, for the holdout category, it can be noticed that Krishnaveni et al. [90]. This may be due to the nature of this the standard deviation is much higher than the other cate- dataset that suits well to the proposed method. gories. Observing the holdout category, the high deviation is To provide information on which models are more deduced to be contributed from an unusual lower accuracy effective than the others, models with accuracy higher than from Boughaci and Alkhawaldeh [18]. This lower accuracy the mean is viewed as having greater potential to deal with 26 Advances in Operations Research thecredit scoring problem and would berecommendable terms shall be considered. Danenas et al. [26] provided for future research. Higher potential models are reported in information on the various choice of available kernels for italic (see Table 9), and they are compared with the respective SVM; thus instead of the common linear and RBF, other mean accuracy in every category. All accuracies written in kernels shall be investigated. Modified SVM in the litera- italic in Table 8 correspond to the higher potential credit ture involved modifications on the hyperplane optimization models obtained from Table 9. Generally, rules extraction, problem. Introduction of new kernels can be perceived as hyperparameters tuning and features selection are observed apossiblefutureworkfor modiefi d SVM category, due to as effective measures to be undertaken for well-performed the exib fl ility of SVM itself, where any kernels that follow credit models. This aligned with our previous discussion in the Karush-Kuhn Tucker conditions can be used in SVM. Section 4.1.3. Hybrid approach is the majority for features selection, only For features selection and hyperparameters tuning, both two works [33, 34] adopted cost and protfi view to handle this issue. Future features selection can consider incorporation of SVM and MA appears to be the crucial tools for credit modelling, where SVM is the main classifier and MA is the cost and protfi in model building process as protfi scoring is assistant to be fused with SVM to carry out the targeted suggested as the main future trend in [2, 11]. Besides, other tasks. For rules extraction, MA is the best choice to build research purposes that have few publications are valuable transparent credit scoring models, either in standalone MA future directions to be considered. or hybridized with other black box DM. Among the dieff rent variants of SVM, standard SVM is most commonly adopted ... MA Models. Hybrid MA-DM has been the leading by researchers for new model development. On the other model type throughout the years, and it is believed that hand, among the different MA, GA and GP appear to be the this trend will persist in the future since hybrid models dominant tool, while recent trend has shifted to other types formulation is considered as a direct way to propose new of MA for new model building. models yet able to improve the standalone DM. EA, i.e., Inclusion of German and Australian datasets into large GA and GP, are the most popular MA to be applied in scale benchmark comparative studies of credit models is an credit scoring domain. It can be observed that MA (other indication of the status of these two becoming the standard than EA family) have increasing publications in recent years. credit datasets in this domain. This leads to frequent usage of Since there are various choices for MA in each family, both datasets throughout the years until recent. those that have not yet been investigated in credit scoring domain, such as, Social Cognitive Optimization, Bat Algo- rithm, Local Search, Variable Neighbourhood Search, etc., 5. Conclusions and Future Directions should be considered for new model development. Although This study presented a literature review of credit scoring standard MA is seldom used for model development, the models formulated with SVM and MA. From the two aspects, transparent property, and efl xibility of MA to be tailored for model type with issues addressed and assessment procedures, to solve for specific credit data is a plus point to use MA in together with past results of models applied on UCI credit credit modelling. In addition, business-oriented model is an datasets, hybrid approach is identified as the state-of-the- important prospect for decision makers. Hence, the flexibility art modelling approach for both techniques in the credit of MA shall be employed to incorporate costs and benefits scoring domain. SVM and MA have been the current trend into model formulation. In view of computational efficiency, for credit modelling with SVM being the main classifier and MA is very adaptable where the operators can be carefully MA being the assistant tool for model enhancement with modified or parallelized to achieve high efficiency. hybrid approach. Aligned with the views from [1, 11, 12] that concluded sophisticated models are the future trend, both ... Overall. Features selection, hyperparameters tuning SVM and MA will also have the similar future modelling and rules extraction are concluded as the few popular issues trend. Various issues and assessment procedures in the addressed with SVM and MA models, with higher tendency literature are concluded to point out some future directions to deal with hybrid methods. Results from past experiments as follows. with German and Australian datasets further validate these three as the trend for both models in credit scoring. As compared to SVM, MA is mostly limited to deal with these .. Model Type with Issues Addressed three issues, while SVM dealt with wider varieties of issues. u Th s, instead of limiting MA to build models based on the ... SVM Models. SVM is an ongoing active research in credit scoring, where the future development trend is per- few popular purposes, other issues that have been attempted ceived as building new SVM models based on hybrid and as in SVM models shall be considered since flexibility of MA ensembles method. Several directions are pointed out for to make it tailored to specific problem is always possible. possible future works. There are various SVM variants avail- MA is mostly incorporated with SVM to form hybrid. Other able that are able to account for more efl xible classicfi ation. AI models that have not yet been attempted for the same A benchmark experiment comparing standard SVM and its research purpose are worth being investigated. Besides, MA variants can be conducted to give insight on which SVM is advantageous in rules extraction while SVM is a black box model. These two can be collaborated to formulate type is more adaptable in this domain. When building new models, SVM variants that include different regularization a transparent yet competitive model. Lastly, formation of Advances in Operations Research 27 business-oriented SVM models with hybrid approach is a Acknowledgments more direct task than to modify the algorithm in SVM. This research is funded by Geran Putra-Inisiatif Putra The adaptable property of MA is a good prospect to join it Siswazah (GP-IPS/2018/9646000) supported by Universiti with SVM, where MA is responsible to take account of cost Putra Malaysia (UPM). and protfi in the modelling procedure. Ensembles modelling is also a future research direction as it has just received attention in credit scoring domain lately. Business-oriented References ensemble with MA-tuned SVM model is a recommendable [1] D. J. Hand and W. E. Henley, “Statistical classification methods future research. in consumer credit scoring: a review,” Journal of the Royal Statistical Society: Series A (Statistics in Society),vol.160, no.3, .. Assessment Procedures. The whole assessment proce- pp. 523–541, 1997. dures in each experiment are a subjective decision based on [2] L.C.Thomas,“A survey ofcreditand behavioural scoring: the experimental setup of the researchers. Internal small scale forecasting financial risk of lending to consumers,” International benchmark experiments including both statistical and AI Journal of Forecasting, vol.16,no. 2,pp.149–172,2000. techniques, reporting the numerical improvement of cutoff- [3] S. M. Sadatrasoul, M. R. Gholamian, M. Siami, and Z. Hajimo- dependent measures, is the leading trend in credit scoring hammadi, “Credit scoring in banks and financial institutions via domain. Internal small scale benchmark is the recommended data mining techniques: a literature review,” Journal of AI and experimental setup for future experiments due to two main Data Mining ,vol.1,no.2,pp.119–129, 2013. considerations. First, newly developed model has to be [4] N. Martin, “Assessing scorecard performance: a literature compared with standard credit scoring models from both sta- review and classification,” Expert SystemswithApplications,vol. tistical and AI families as a concrete support of the usefulness 40, no. 16, pp. 6340–6350, 2013. of the proposed models but large scale comparison is not [5] H.A.Alaka,L.O.Oyedele,H.A.Owolabi et al.,“Systematic necessary to avoid loss of focus. Second, external benchmark review of bankruptcy prediction models: Towards a framework for tool selection,” Expert Systems with Applications,vol. 94, pp. comparing with results from other researches can be consid- 164–184, 2018. ered but have to be used very cautiously because different [6] J. Sun,H. Li,Q.-H. Huang,and K.-Y.He, “Predicting financial experiments have different design that may be difficult to distress and corporate failure: a review from the state-of-the- have an apple-to-apple comparison. It is also recommended art definitions, modeling, sampling, and featuring approaches,” to report both cutoff-dependent and independent measures Knowledge-Based Systems,vol.57,pp.41–56,2014. together and disclose the assumed cutoff point when using [7] S. Moro, P. Cortez, and P. Rita, “An automated literature analysis cutoff-dependent measures. There have been varieties of on data mining applications to credit risk assessment,” in performance measures being recommended in Lessmann et Artiﬁcial Intelligence in Financial Markets, pp. 161–177, Palgrave al. [11], butthechoiceshall becarefully madeinstead of using Macmillan, London, 2016. as many as possible of these measures. This is because every [8] B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, J. Suykens, measure evaluates models in different approach; researchers and J. Vanthienen, “Benchmarking state-of-the-art classifica- shall choose based on what is the most important information tion algorithms for credit scoring,” Journal of the Operational needed to be conveyed as it is not always “the more the Research Society,vol.54,no. 6,pp.627–635,2003. better” [4]. Business-oriented measures can be viewed as [9] J.N.Crook,D.B. Edelman, and L. C.Thomas, “Recent a future trend to evaluate models, provided that the credit developments in consumer credit risk assessment,” European dataset should have information for the profit and costs. Journal of Operational Research,vol.183, no.3,pp. 1447–1465, Although numerical improvement is considered a valuable 2007. contribution, it is recommended to include statistical tests [10] A.Lahsasna,R.N.Ainon,andT.Y.Wah,“Creditscoringmodels for further validation on the statistical significance of model using soft computing methods: a survey,” International Arab Journal of Information Technolog,vol.7,no.2,pp. 115–123,2010. performances. The trend of focusing in application scoring [11] S. Lessmann, B. Baesens, H.-V. Seow, and L. C. o Th mas, especially with public UCI datasets may be due to the “Benchmarking state-of-the-art classification algorithms for confidentiality issue of credit scoring domain. Recently, there credit scoring: an update of research,” European Journal of aresomedatasets available from data mining competitions Operational Research, vol.247,no. 1,pp. 124–136,2015. and also Lending Club (LC) online P2P lending platform. [12] F. Louzada, A. Ara, and G. B. Fernandes, “Classification meth- u Th s, other types of dataset shall be involved to check models ods applied to credit scoring: systematic review and overall robustness across different types of data. Besides, LC datasets comparison,” Surveys in Operations Research and Management provided more information that can be considered to build Science,vol.21,no. 2,pp. 117–134, 2016. models for behavioural scoring. For specific research using [13] A. I. Marques, V. Garc´ıa, and J. S. Sanchez ´ , “A literature review private datasets, protfi scoring may be a future research on the application of evolutionary computing to credit scoring,” trend and it is recommended to include UCI datasets as a Journal of the Operational Research Society,vol.64, no.9,pp. benchmark due to their general usage in the literature. 1384–1399, 2013. [14] D. Durand, Risk Elements in Consumer Instalment Financing, NBER Books, 1941. Conflicts of Interest [15] E. I. Altman, “Financial ratios, discriminant analysis and the The authors declare that there are no conflicts of interest prediction of corporate bankruptcy,” e Journal of Finance,vol. regarding the publication of this paper. 23, no. 4, pp. 589–609, 1968. 28 Advances in Operations Research [16] J. A. Ohlson, “Financial ratios and the probabilistic prediction [34] S. Maldonado, C. Bravo, J. Lop ´ ez, and J. Per ´ ez, “Integrated of bankruptcy,” Journal of Accounting Research, vol.18,no.1,pp. framework for profit-based feature selection and SVM classifi- 109–131, 1980. cation in credit scoring,” Decision Support Systems,vol.104, pp. 113–121, 2017. [17] P. Kolesar and J. L. Showers, “A robust credit screening model using categorical data,” Management Science,vol. 31,no. 2,pp. [35] C.L. Huang, M.C. Chen,and C.J.Wang, “Credit scoring with 123–133, 1985. a data mining approach based on support vector machines,” Expert Systems with Applications, vol.33, no.4, pp. 847–856, [18] D. Boughaci and A. A. Alkhawaldeh, “Appropriate machine learning techniques for credit scoring and bankruptcy predic- tion in banking and finance: a comparative study,” Risk and [36] D. Martens, B. Baesens, T. Van Gestel, and J. Vanthienen, Decision Analysis,pp. 1–10,2018. “Comprehensible credit scoring models using rule extraction from support vector machines,” European Journal of Opera- [19] I. T. Van Gestel, B. Baesens, I. J. Garcia, and P. Van Dijcke, “A tional Research, vol.183,no.3,pp.1466–1476, 2007. support vector machine approach to credit scoring,” Bank- en Financiewezen,vol.2,pp.73–82,2003. [37] J. Zhou and T. Bai, “Credit risk assessment using rough set theory and GA-based SVM,” in Proceedings of the rd [20] Z. Huang, H. Chen,C.-J.Hsu, W.-H.Chen, and S. Wu,“Credit International Conference on Grid and Pervasive Computing rating analysis with support vector machines and neural net- Symposia/Workshops (GPC), pp. 320–325, IEEE, 2008. works: a market comparative study,” Decision Support Systems, vol.37,no.4,pp.543–558, 2004. [38] D. Zhang, M. Hifi, Q. Chen,and W. Ye,“A hybrid creditscoring model based on genetic programming and support vector [21] S.-T. Li, W. Shiue, and M.-H. Huang, “eTh evaluation of machines,” in Proceedings of the th International Conference on consumer loans using support vector machines,” Expert Systems Natural Computation (ICNC),vol.7,pp. 8–12,2008. with Applications, vol. 30, no. 4, pp. 772–782, 2006. [39] P. Yao, “Hybrid classifier using neighborhood rough set and [22] K. Lai, L. Yu, L. Zhou, and S. Wang, “Credit risk evaluation with SVM for credit scoring,” in Proceedings of the International least square support vector machine,”RoughSetsandKnowledge Technology, pp. 490–495, 2006. Conference on Business Intelligence and Financial Engineering (BIFE), pp. 138–142, IEEE, 2009. [23] Y.-C. Lee, “Application of support vector machines to corporate [40] X. Xu,C.Zhou, and Z.Wang, “Creditscoring algorithm based credit rating prediction,” Expert Systems with Applications,vol. 33,no. 1, pp.67–74, 2007. on link analysis ranking with support vector machine,” Expert Systems with Applications, vol. 36, no. 2, pp. 2625–2632, 2009. [24] T. Bellotti and J. Crook, “Support vector machines for credit scoring and discovery of significant features,” Expert Systems [41] L.Zhou, K. K.Lai,and J. Yen,“Credit scoring models with AUC with Applications, vol.36,no. 2,pp.3302–3308, 2009. maximization based on weighted SVM,” International Journal of Information Technology & Decision Making, vol.8,no. 4,pp. [25] H. S. Kim and S. Y. Sohn, “Support vector machines for default 677–696, 2009. prediction of SMEs based on technology credit,” European Journal of Operational Research, vol.201,no. 3,pp. 838–846, [42] L.Zhou,K. K.Lai, and L. Yu,“Credit scoring using support 2010. vector machines with direct search for parameters selection,” So Computing-A Fusion of Foundations, Methodologies and [26] P. Danenas, G. Garsva, and S. Gudas, “Credit risk evaluation Applications, vol.13,no. 2,pp.149–155, 2009. model development using support vector based classifiers,” Procedia Computer Science,vol.4,pp. 1699–1707, 2011. [43] F.-L. Chen and F.-C. Li, “Combination of feature selection approaches with SVM in credit scoring,” Expert Systems with [27] J. Mushava and M. Murray, “An experimental comparison of Applications, vol.37,no. 7,pp. 4902–4909, 2010. classification techniques in debt recoveries scoring: evidence from South Africa’s unsecured lending market,” Expert Systems [44] L. Yu, X.Yao,S.Wang, and K.K.Lai,“Credit riskevaluation with Applications, vol. 111, pp. 35–50, 2018. using a weighted least squares SVM classifier with design of experiment for parameter selection,” Expert Systems with [28] Y. Wang, S.Wang,and K.K.Lai, “A new fuzzy support vector Applications,vol. 38, no. 12, pp.15392–15399,2011. machine to evaluate credit risk,” IEEE Transactions on Fuzzy Systems, vol.13,no. 6,pp. 820–831, 2005. [45] D. West, “Neural network credit scoring models,” Computers & Operations Research, vol. 27, no. 11-12, pp. 1131–1152, 2000. [29] T. Harris, “Credit scoring using the clustered support vector machine,” Expert SystemswithApplications,vol.42, no. 2, pp. [46] A. B. Hens and M. K. Tiwari, “Computational time reduction for 741–750, 2015. credit scoring: an integrated approach based on support vector machine and stratified sampling method,” Expert Systems with [30] Y. Yang, “Adaptive credit scoring with kernel learning methods,” Applications, vol.39,no.8,pp.6774–6781,2012. European Journal of Operational Research, vol.183,no. 3, pp. 1521–1536, 2007. [47] P. Danenas and G. Garsva, “Credit risk evaluation modeling [31] Z. Li, Y. Tian, K. Li, F. Zhou, and W. Yang, “Reject inference in using evolutionary linear SVM classifiers and sliding window approach,” Procedia Computer Science, vol.9,pp.1324–1333, credit scoring using semi-supervised support vector machines,” Expert SystemswithApplications, vol.74,pp.105–114,2017. [32] Y. Tian, Z. Yong, and J. Luo, “A new approach for reject [48] W. Chen, G.Xiang,Y.Liu,and K.Wang,“Credit riskevaluation by hybrid data mining technique,” Systems Engineering Procedia, inference in credit scoring using kernel-free fuzzy quadratic surface support vector machines,” Applied So Computing,vol. vol. 3, pp. 194–200, 2012. 73, pp. 96–105, 2018. [49] M.Y.Chen,C.C.Chen,andJ.Y.Liu,“Creditratinganalysiswith [33] S. Maldonado, J. Per ´ ez, and C. Bravo, “Cost-based feature support vector machines and artificial bee colony algorithm,” in selection for support vector machines: an application in credit Proceedings of the International Conference on Industrial, Engi- scoring,” European Journal of Operational Research,vol. 261, no. neering and Other Applications of Applied Intelligent Systems,pp. 2,pp.656–665,2017. 528–534, Springer, Berlin, Heidelberg, Germany, 2013. Advances in Operations Research 29 [50] L. Han, L. Han, and H. Zhao, “Orthogonal support vector [68] X.-S. Yang, S. Deb, M. Karamanoglu, and X. He, “Cuckoo search machine for credit scoring,” Engineering Applications of Artiﬁ- for business optimization applications,” in Proceedings of the cial Intelligence, vol.26, no.2,pp.848–862, 2013. National Conference on Computing and Communication Systems (NCCCS), pp. 1–5, IEEE, 2012. [51] G. Garsva and P. Danenas, “Particle swarm optimization for linear support vector machines based classifier selection,” Non- [69] J. Uthayakumar and T. Vengattaraman, “Swarm intelligence linear Analysis: Modelling and Control, vol.19, no. 1,pp.26–42, based classification rule induction (CRI) framework for qual- itative and quantitative approach: an application of bankruptcy [52] P. Danenas and G. Garsva, “Selection of support vector prediction and credit risk analysis,” Journal of King Saud University - Computer and Information Sciences,2017. machines based classifiers for credit risk domain,” Expert SystemswithApplications, vol.42,no.6,pp.3194–3204,2015. [70] M. Jiang, F. Ji, and R. Li, “eTh applied research of credit scoring [53] F. J. Hsu, M. Y. Chen, and Y. C. Chen, “The human-like combination model based on SA-GA algorithm,” in Proceedings intelligence with bio-inspired computing approach for credit of the Fourth International Conference on Business Intelligence ratings prediction,” Neurocomputing,vol.279,pp.11–18, 2018. and Financial Engineering (BIFE), pp. 491–494, 2011. [54] S. Jadhav, H. He, and K. Jenkins, “Information gain directed [71] R. Aliehyaei and S. Khan, “Ant colony optimization, genetic genetic algorithm wrapper feature selection for credit rating,” programming and a hybrid approach for credit scoring: a Applied So Computing, vol.69, pp.541–553,2018. comparative study,” in Proceedings of the th International [55] D. Wang,Z.Zhang, R.Bai,and Y. Mao,“A hybrid system with Conference on Soware, Knowledge, Information Management filter approach and multiple population genetic algorithm for and Applications (SKIMA), pp. 1–5, IEEE, 2014. feature selection in credit scoring,” Journal of Computational [72] T. C. Fogarty and N. S. Ireson, “Evolving bayesian classifiers andAppliedMathematics, vol.329,pp.307–321, 2018. for credit control—a comparison with other machine-learning [56] L. Zhou, K.K.Lai,and L. Yu, “Least squares supportvector methods,” IMA Journal of Management Mathematics, vol.5,no. machines ensemble models for credit scoring,” Expert Systems 1,pp.63–75,1993. with Applications, vol.37,no.1,pp.127–133,2010. [73] D. J. Hand and N. M. Adams, “Defining attributes for scorecard [57] A. Ghodselahi, “A hybrid support vector machine ensemble construction in credit scoring,” Journal of Applied Statistics,vol. model for credit scoring,” International Journal of Computer 27, no. 5, pp. 527–540, 2000. Applications, vol.17,no.5,pp.1–5,2011. [74] Z. Drezner, G. A. Marcoulides, and M. H. Stohs, “Financial [58] L. Yu, R. Zhou, L. Tang, and R. Chen, “A DBN-based resampling applications of a Tabu search variable selection model,” Journal SVM ensemble learning paradigm for credit classification with of Applied Mathematics and Decision Sciences,vol. 5, no. 4, pp. imbalanced data,” Applied So Computing, vol.69, pp.192–202, 215–234, 2002. [75] E.Lacerda, A.C.Carvalho, A.P. Braga,and T. B.Ludermir, [59] Y. Xia, C. Liu, B. Da, and F. Xie, “A novel heterogeneous “Evolutionary radial basis functions for credit assessment,” ensemble credit scoring model based on bstacking approach,” Applied Intelligence,vol.22,no.3,pp.167–181, 2005. Expert SystemswithApplications, vol.93,pp.182–199, 2018. [76] S.-A.Liu,Q.Wang,and S.Lv,“Application of genetic program- [60] V. S. Desai, D. G. Conway, J. N. Crook, and G. A. Overstreet, ming in credit scoring,” in Proceedings of the Chinese Control “Credit-scoring models in the credit-union environment using and Decision Conference (CCDC), pp. 1106–1110, IEEE, China, neural networks and genetic algorithms,” IMA Journal of Man- agement Mathematics, vol.8,no.4,pp.323–346, 1997. [77] S. Wang, S. Yin, and M. Jiang, “Hybrid neural network based [61] C.-S. Ong, J.-J. Huang, and G.-H. Tzeng, “Building credit on GA-BP for personal credit scoring,” in Proceedings of the th scoring models using genetic programming,” Expert Systems International Conference on Natural Computation (ICNC),vol. with Applications, vol.29,no.1,pp.41–47,2005. 3, pp. 209–214, IEEE, 2008. [62] S. M. Finlay, Using Genetic Algorithms to Develop Scoring Models [78] D. Zhang, S. C. Leung, and Z. Ye, “A decision tree scoring for Alternative Measures of Performance, Springer, Lancaster, model based on genetic algorithm and k-means algorithm,” in England, UK, 2006. Proceedings of the rd International Conference on Convergence [63] W. Cai, X. Lin, X. Huang, and H. Zhong, “A genetic algorithm and Hybrid Information Technology (ICCIT),vol.1, pp.1043– model for personal credit scoring,” in Proceedings of the Inter- 1047, IEEE, 2008. national Conference on Computational Intelligence and Soware [79] J. Yi, “Credit scoring model based on the decision tree and Engineering (CiSE), pp. 1–4, IEEE, 2009. the simulated annealing algorithm,” in Proceedings of the World [64] J. J. Huang, G. H. Tzeng, and C. S. Ong, “Two-stage genetic Congress on Computer Science and Information Engineering programming (2SGP) for the credit scoring model,” Applied (WRI), vol. 4, pp. 18–22, IEEE, 2009. Mathematics and Computation, vol.174,no.2,pp. 1039–1053, [80] Y. Marinakis, M. Marinaki, M. Doumpos, and C. Zopounidis, “Ant colony and particle swarm optimization for financial [65] H. A. Abdou, “Genetic programming for credit scoring: the classification problems,” Expert SystemswithApplications,vol. case of Egyptian public sector banks,” Expert Systems with 36, no. 7, pp. 10604–10611, 2009. Applications,vol. 36, no. 9,pp. 11402–11417, 2009. [81] M. Suˇsterˇsic, ˇ D. Mramor, and J. Zupan, “Consumer credit scor- [66] G. Dong, K. K. Lai, and L. Zhou, “Simulated annealing based ing models with limited data,” Expert Systems with Applications, rule extraction algorithm for credit scoring problem,” in Pro- vol. 36, no. 3, pp. 4736–4744, 2009. ceedings of the International Joint Conference on Computational [82] M. Marinaki, Y. Marinakis, and C. Zopounidis, “Honey bees Sciences and Optimization (CSO), vol. 2, pp. 22–26, IEEE, 2009. mating optimization algorithm for financial classification prob- [67] D. Martens, T. Van Gestel, M. De Backer, R. Haesen, J. lems,” Applied So Computing, vol.10,no. 3,pp.806–812, 2010. Vanthienen, and B. Baesens, “Credit rating prediction using [83] S.-C. Huang and C.-F. Wu, “Customer credit quality assess- ant colony optimization,” Journal of the Operational Research Society, vol.61,no.4,pp.561–573,2010. ments using data mining methods for banking industries,” 30 Advances in Operations Research African Journal of Business Management,vol. 5,no.11, pp. 4438– 4445, 2011. [84] A. B. Correa and A. M. Gonzalez, “Evolutionary algorithms for selecting the architecture of a MLP neural network: a credit scoring case,” in Proceedings of the th IEEE International Conference on Data Mining Workshops (ICDMW), pp. 725–732, IEEE, 2011. [85] S.Oreski,D.Oreski,andG.Oreski,“Hybridsystemwithgenetic algorithm and artificial neural networks and its application to retail credit risk assessment,” Expert Systems with Applications, vol. 39, no. 16, pp. 12605–12617, 2012. [86] S. Oreski and G. Oreski, “Genetic algorithm-based heuristic for feature selection in credit risk assessment,” Expert Systems with Applications, vol.41, no. 4,pp. 2052–2064, 2014. [87] J. Wang,K.Guo,andS. Wang,“Roughsetandtabusearchbased feature selection for credit scoring,” Procedia Computer Science, vol. 1, no. 1, pp. 2425–2432, 2010. [88] J. Wang, A.-R. Hedar, S. Wang, and J. Ma, “Rough set and scatter search metaheuristic based feature selection for credit scoring,” Expert Systems with Applications, vol.39,no. 6,pp.6123–6128, [89] B.Waad,B.M.Ghazi,andL.Mohamed,“A newfeatureselection technique applied to credit scoring data using a rank aggrega- tion approach based on: optimization, genetic algorithm and similarity,” in Knowledge Discovery Process and Methods to Enhance Organisational Performance,K. Osei-Bryson and C. Barclay, Eds., pp. 347–376, CRC Press, Boca Raton, Fla, USA, [90] V. Krishnaveni, K. G. Santhiya, M. P. Ramesh, and M. S. Jaganathan, “A credit scoring prediction model based on har- mony search based 1-NN classifier feature selection approach,” International Journal of Advance Research in Science and Engi- neering,vol. 7,no. 2,pp.468–476, 2018. [91] V. N. Vapnik, Statistical Learning eory, Wiley- Interscience, New York, NY, USA, 1998. [92] Q. Gu and J. Han, “Clustered support vector machines,” Artiﬁ- cial Intelligence and Statistics,pp. 307–315, 2013. [93] D. Martens, J. Huysmans, R. Setiono, J. Vanthienen, and B. Baesens, “Rule extraction from support vector machines: an overview of issues and application in credit scoring,” in Rule Extraction from Support Vector Machines,vol. 80, pp. 33–63, Springer, Berlin, Heidelberg, Germany, 2008. [94] M. Lichman, UCI Machine Learning Repository,School of Information and Computer Science, University of California, Irvine, CA, USA, http://archive.ics.uci.edu/ml/. The Scientific Advances in Advances in Journal of Journal of Operations Research Decision Sciences Applied Mathematics World Journal Probability and Statistics Hindawi Hindawi Hindawi Hindawi Publishing Corporation Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 International Journal of Mathematics and Mathematical Sciences Journal of Optimization Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com International Journal of International Journal of Engineering Mathematics Analysis Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of A Ad dv va an nc ce es i s in n Discrete Dynamics in Mathematical Problems International Journal of Complex Analysis Num Num Num Num Num Num Num Num Num Num Num Nume e e e e e e e e e e er r r r r r r r r r r riiiiiiiiiiiic c c c c c c c c c c cal al al al al al al al al al al al A A A A A A A A A A A Anal nal nal nal nal nal nal nal nal nal nal naly y y y y y y y y y y ys s s s s s s s s s s siiiiiiiiiiiis s s s s s s s s s s s in Engineering Dierential Equations Nature and Society Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com V Volume 2018 olume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 International Journal of Journal of Journal of Abstract and Advances in Stochastic Analysis Mathematics Function Spaces Applied Analysis Mathematical Physics Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Advances in Operations Research Hindawi Publishing Corporation http://www.deepdyve.com/lp/hindawi-publishing-corporation/credit-scoring-a-review-on-support-vector-machines-and-metaheuristic-Zx9lBYa187

Loading next page...

References (97)

J. Uthayakumar and T. Vengattaraman
Swarm intelligence based classification rule induction (CRI) framework for qualitative and quantitative approach: an application of bankruptcy prediction and credit risk analysis,
Journal of King Saud University - Computer and Information Sciences
D. Mramor M. Šušteršič (2009)
Consumer credit scoring models with limited data,
Expert Systems with Applications, 36
J. Kang, Hui Zhao (2012)
Application of Improved Grey Model in Long-term Load Forecasting of Power Engineering
Systems Engineering Procedia, 3
Di Wang, Zuoquan Zhang, Rongquan Bai, Yanan Mao (2018)
A hybrid system with filter approach and multiple population genetic algorithm for feature selection in credit scoring
J. Comput. Appl. Math., 329
D. Hand, W. Henley (1997)
Statistical Classification Methods in Consumer Credit Scoring: a Review
Journal of the Royal Statistical Society: Series A (Statistics in Society), 160
Y. Marinakis, M. Marinaki, M. Doumpos, C. Zopounidis (2009)
Ant colony and particle swarm optimization for financial classification problems
Expert Syst. Appl., 36
E. I. Altman (1968)
Financial ratios, discriminant analysis and the prediction of corporate bankruptcy,
The Journal of Finance, 23
Paulius Danėnas, G. Garsva (2012)
Credit risk evaluation modeling using evolutionary linear SVM classifiers and sliding window approach
S. Maldonado, Juan Pérez, Cristián Bravo (2017)
Cost-based feature selection for Support Vector Machines: An application in credit scoring
Eur. J. Oper. Res., 261
Akhil Hens, M. Tiwari (2012)
Computational time reduction for credit scoring: An integrated approach based on support vector machine and stratified sampling method
Expert Syst. Appl., 39
F. Ji M. Jiang (2011)
The applied research of credit scoring combination model based on SA-GA algorithm,
Proceedings of the Fourth International Conference on Business Intelligence and Financial Engineering (BIFE)
S. C. Leung D. Zhang (2008)
A decision tree scoring model based on genetic algorithm and k-means algorithm,
Proceedings of the 3rd International Conference on Convergence and Hybrid Information Technology (ICCIT), 1
P. Yao (2009)
Hybrid classifier using neighborhood rough set and SVM for credit scoring,
Proceedings of the International Conference on Business Intelligence and Financial Engineering (BIFE)
Xiujuan Xu, Chunguang Zhou, Zhe Wang (2009)
Credit scoring algorithm based on link analysis ranking with support vector machine
Expert Syst. Appl., 36
T. Bellotti, J. Crook (2009)
Support vector machines for credit scoring and discovery of significant features
Expert Syst. Appl., 36
Swati Jadhav, Hongmei He, K. Jenkins (2018)
Information gain directed genetic algorithm wrapper feature selection for credit rating
Appl. Soft Comput., 69
James Ohlson (1980)
FINANCIAL RATIOS AND THE PROBABILISTIC PREDICTION OF BANKRUPTCY
Journal of Accounting Research, 18
Z. Drezner, G. Marcoulides, M. Stohs (2001)
Financial applications of a Tabu search variable selection model
Adv. Decis. Sci., 5
L. Thomas (2000)
A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers
International Journal of Forecasting, 16
Ming-Tao Chou (2013)
Research on the relationship between forward freight agreements and the fuel oil index
African Journal of Business Management, 5
X. Lin W. Cai (2009)
A genetic algorithm model for personal credit scoring,
Proceedings of the International Conference on Computational Intelligence and Software Engineering (CiSE)
Cheng-Lung Huang, Mu-Chen Chen, Chieh-Jen Wang (2007)
Credit scoring with a data mining approach based on support vector machines
Expert Syst. Appl., 33
J. Yi (2009)
Credit scoring model based on the decision tree and the simulated annealing algorithm,
Proceedings of the World Congress on Computer Science and Information Engineering (WRI), 4
Sheng-Tun Li, Weissor Shiue, Meng-Huah Huang (2006)
The evaluation of consumer loans using support vector machines
Expert Syst. Appl., 30
D. Durand (1942)
Risk elements in consumer instalment financing
S. Oreski, G. Oreski (2014)
Genetic algorithm-based heuristic for feature selection in credit risk assessment
Expert Syst. Appl., 41
C. C. Chen M. Y. Chen (2013)
Credit rating analysis with support vector machines and artificial bee colony algorithm,
Proceedings of the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems
B. Baesens S. Lessmann (2015)
Benchmarking state-of-the-art classification algorithms for credit scoring: an update of research,
European Journal of Operational Research, 247
H. Alaka, Lukumon Oyedele, H. Owolabi, Prof Kumar, Saheed Ajayi, Olúgbénga Akinadé, Muhammad Bilal (2018)
Systematic review of bankruptcy prediction models: Towards a framework for tool selection
Expert Syst. Appl., 94
Q. Gu and J. Han (2013)
Clustered support vector machines,
Artificial Intelligence and Statistics
D. Boughaci and A. A. Alkhawaldeh (2018)
Appropriate machine learning techniques for credit scoring and bankruptcy prediction in banking and finance: a comparative study,
Risk and Decision Analysis
(2011)
SpringerReference
P. Cortez S. Moro (2016)
An automated literature analysis on data mining applications to credit risk assessment,
Artificial Intelligence in Financial Markets
M. Hifi D. Zhang (2008)
A hybrid credit scoring model based on genetic programming and support vector machines,
Proceedings of the 4th International Conference on Natural Computation (ICNC), 7
Bin Zhu, Zeshui Xu (2013)
Hesitant fuzzy Bonferroni means for multi-criteria decision making
Journal of the Operational Research Society, 64
J. Brans, P. Vincke (1985)
Note-A Preference Ranking Organisation Method: The PROMETHEE Method for Multiple Criteria Decision-Making
Management Science, 31
R. Aliehyaei and S. Khan (2014)
Ant colony optimization, genetic programming and a hybrid approach for credit scoring: a comparative study,
Proceedings of the 8th International Conference on Software, Knowledge, Information Management and Applications (SKIMA)
Ligang Zhou, K. Lai, Lean Yu (2010)
Least squares support vector machines ensemble models for credit scoring
Expert Syst. Appl., 37
Ligang Zhou, K. Lai, J. Yen (2009)
Credit Scoring Models with AUC Maximization Based on Weighted SVM
Int. J. Inf. Technol. Decis. Mak., 8
Ahmad Ghodselahi (2011)
A Hybrid Support Vector Machine Ensemble Model for Credit Scoring
International Journal of Computer Applications, 17
M. Marinaki, Y. Marinakis, C. Zopounidis (2010)
Honey Bees Mating Optimization algorithm for financial classification problems
Appl. Soft Comput., 10
Paulius Danėnas, G. Garsva, S. Gudas (2011)
Credit Risk Evaluation Model Development Using Support Vector Based Classifiers
N. Martin (2013)
Assessing scorecard performance: A literature review and classification
Expert Syst. Appl., 40
M. Sustersic, Dusan Mramor, J. Zupan (2007)
Consumer Credit Scoring Models with Limited Data
Banking & Financial Institutions eJournal
Jonah Mushava, Michael Murray (2018)
An experimental comparison of classification techniques in debt recoveries scoring: Evidence from South Africa's unsecured lending market
Expert Syst. Appl., 111
Hussein Abdou (2009)
Genetic programming for credit scoring: The case of Egyptian public sector banks
Expert Syst. Appl., 36
G. A. Marcoulides Z. Drezner (2002)
Financial applications of a Tabu search variable selection model,
Journal of Applied Mathematics and Decision Sciences, 5
J. Zhou and T. Bai (2008)
Credit risk assessment using rough set theory and GA-based SVM,
Proceedings of the 3rd International Conference on Grid and Pervasive Computing Symposia/Workshops (GPC)
K. K. Lai L. Zhou (2009)
Credit scoring using support vector machines with direct search for parameters selection,
Soft Computing-A Fusion of Foundations, Methodologies and Applications, 13
Q. Wang S.-A. Liu (2008)
Application of genetic programming in credit scoring,
Proceedings of the Chinese Control and Decision Conference (CCDC)
B. M. Ghazi B. Waad (2014)
A new feature selection technique applied to credit scoring data using a rank aggregation approach based on: optimization, genetic algorithm and similarity,
Knowledge Discovery Process and Methods to Enhance Organisational Performance
Lean Yu, Rongtian Zhou, L. Tang, Rongda Chen (2018)
A DBN-based resampling SVM ensemble learning paradigm for credit classification with imbalanced data
Appl. Soft Comput., 69
10.1007/springerreference
Using Genetic Algorithms to Develop Scoring Models for Alternative Measures of Performance
F. Louzada, Anderson Ara, G. Fernandes (2016)
Classification methods applied to credit scoring: A systematic review and overall comparison
arXiv: Applications
Zan Huang, Hsinchun Chen, Chia-Jung Hsu, Wun-Hwa Chen, Soushan Wu (2004)
Credit rating analysis with support vector machines and neural networks: a market comparative study
Decis. Support Syst., 37
S. Oreski, D. Oreški, G. Oreski (2012)
Hybrid system with genetic algorithm and artificial neural networks and its application to retail credit risk assessment
Expert Syst. Appl., 39
A. B. Correa and A. M. Gonzalez (2011)
Evolutionary algorithms for selecting the architecture of a MLP neural network: a credit scoring case,
Proceedings of the 11th IEEE International Conference on Data Mining Workshops (ICDMW)
Lean Yu, X. Yao, Shouyang Wang, K. Lai (2011)
Credit risk evaluation using a weighted least squares SVM classifier with design of experiment for parameter selection
Expert Syst. Appl., 38
David Martens, Johan Huysmans, R. Setiono, J. Vanthienen, B. Baesens (2008)
Rule Extraction from Support Vector Machines: An Overview of Issues and Application in Credit Scoring
Jie Sun, Hui Li, Qinghua Huang, Kai-Yu He (2014)
Predicting financial distress and corporate failure: A review from the state-of-the-art definitions, modeling, sampling, and featuring approaches
Knowl. Based Syst., 57
David Martens, B. Baesens, T. Gestel, J. Vanthienen (2007)
Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines
New Institutional Economics
Chorng-Shyong Ong, Jih-Jeng Huang, G. Tzeng (2005)
Building credit scoring models using genetic programming
Expert Syst. Appl., 29
Paulius Danėnas, G. Garsva (2015)
Selection of Support Vector Machines based classifiers for credit risk domain
Expert Syst. Appl., 42
Jih-Jeng Huang, G. Tzeng, Chorng-Shyong Ong (2006)
Two-stage genetic programming (2SGP) for the credit scoring model
Appl. Math. Comput., 174
D. West (2000)
Neural network credit scoring models
Comput. Oper. Res., 27
J. Crook, David Edelman, L. Thomas (2007)
Recent developments in consumer credit risk assessment
Eur. J. Oper. Res., 183
G. Garsva, Paulius Danėnas (2014)
Particle swarm optimization for linear support vector machines based classifier selection
Nonlinear Analysis-Modelling and Control, 19
H. Kim, S. Sohn (2010)
Support vector machines for default prediction of SMEs based on technology credit
Eur. J. Oper. Res., 201
L. Simar, P. Wilson (2000)
A general methodology for bootstrapping in non-parametric frontier models
Journal of Applied Statistics, 27
Young-Chan Lee (2007)
Application of support vector machines to corporate credit rating prediction
Expert Syst. Appl., 33
F. Chen, Feng-Chia Li (2010)
Combination of feature selection approaches with SVM in credit scoring
Expert Syst. Appl., 37
T. Fogarty, N. Ireson (1993)
Evolving Bayesian classifiers for credit control—a comparison with other machine-learning methods
Ima Journal of Management Mathematics, 5
S. Yin S. Wang (2008)
Hybrid neural network based on GA-BP for personal credit scoring,
Proceedings of the 4th International Conference on Natural Computation (ICNC), 3
K. G. Santhiya V. Krishnaveni (2018)
A credit scoring prediction model based on harmony search based 1-NN classifier feature selection approach,
International Journal of Advance Research in Science and Engineering, 7
Zhiyong Li, Ye Tian, Keqin Li, Fanyin Zhou, Wei Yang (2016)
Reject Inference in Credit Scoring Using Semi-Supervised Support Vector Machines
S. Deb X.-S. Yang (2012)
Cuckoo search for business optimization applications,
Proceedings of the National Conference on Computing and Communication Systems (NCCCS)
Ye Tian, Ziyang Yong, Jian Luo (2018)
A new approach for reject inference in credit scoring using kernel-free fuzzy quadratic surface support vector machines
Appl. Soft Comput., 73
Terry Harris (2015)
Credit scoring using the clustered support vector machine
Expert Syst. Appl., 42
Lu Han, Liyan Han, Hongwei Zhao (2013)
Orthogonal support vector machine for credit scoring
Eng. Appl. Artif. Intell., 26
Jue Wang, A. Hedar, Shouyang Wang, Jian Ma (2012)
Rough set and scatter search metaheuristic based feature selection for credit scoring
Expert Syst. Appl., 39
Y. Tian Z. Li (2017)
Reject inference in credit scoring using semi-supervised support vector machines,
Expert Systems with Applications, 74
Yongqiao Wang, Shouyang Wang, K. Lai (2005)
A new fuzzy support vector machine to evaluate credit risk
IEEE Transactions on Fuzzy Systems, 13
S. Maldonado, Cristián Bravo, Julio Hernandez, Juan Pérez (2017)
Integrated framework for profit-based feature selection and SVM classification in credit scoring
Decis. Support Syst., 104
C. Helito, M. Demange, M. Bonadio, L. Tirico, R. Gobbi, J. Pécora, G. Camanho (2013)
Anatomy and Histology of the Knee Anterolateral Ligament
Orthopaedic Journal of Sports Medicine, 1
B. Baesens D. Martens (2007)
Comprehensible credit scoring models using rule extraction from support vector machines,
European Journal of Operational Research, 183
Feng-Jui Hsu, Mu-Yen Chen, Yu-Cheng Chen (2017)
The human-like intelligence with bio-inspired computing approach for credit ratings prediction
Neurocomputing, 279
Yingxu Yang (2007)
Adaptive credit scoring with kernel learning methods
Eur. J. Oper. Res., 183
T. Gestel, B. Baesens, J. Garcia, P. Dijcke (2003)
A support vector machine approach to credit scoring
, 2
David Martens, T. Gestel, M. Backer, Raf Haesen, J. Vanthienen, B. Baesens (2010)
Credit rating prediction using Ant Colony Optimization
Journal of the Operational Research Society, 61
Vijay Desai, D. Conway, J. Crook, G. Overstreet (1997)
Credit Scoring Models in the Credit Union Environment Using Neural Networks and Genetic Algorithms
Ima Journal of Management Mathematics, 8
J. Águila, E. Arias, M. Artigao, J. Miralles (2010)
A distributed memory implementation of the False Nearest Neighbors method based on kd-tree applied to electrocardiography
Procedia Computer Science, 1
K. K. Lai G. Dong (2009)
Simulated annealing based rule extraction algorithm for credit scoring problem,
Proceedings of the International Joint Conference on Computational Sciences and Optimization (CSO), 2
L. Yu K. Lai (2006)
Credit risk evaluation with least square support vector machine,
Rough Sets and Knowledge Technology
E. Lacerda, A. Carvalho, A. Braga, Teresa Ludermir (2005)
Evolutionary Radial Basis Functions for Credit Assessment
Applied Intelligence, 22
B. Baesens, T. Gestel, Stijn Viaene, M. Stepanova, J. Suykens, J. Vanthienen (2003)
Benchmarking state-of-the-art classification algorithms for credit scoring
Journal of the Operational Research Society, 54
Yufei Xia, Chuanzhe Liu, Bowen Da, Fangming Xie (2018)
A novel heterogeneous ensemble credit scoring model based on bstacking approach
Expert Syst. Appl., 93
R. N. Ainon A. Lahsasna (2010)
Credit scoring models using soft computing methods: a survey,
International Arab Journal of Information Technolog, 7

Publisher: Hindawi Publishing Corporation
Copyright: Copyright © 2019 R. Y. Goh and L. S. Lee. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN: 1687-9147
eISSN: 1687-9155
DOI: 10.1155/2019/1974794
Publisher site: See Article on Publisher Site

Abstract

Hindawi Advances in Operations Research Volume 2019, Article ID 1974794, 30 pages https://doi.org/10.1155/2019/1974794 Review Article Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches 1 1,2 R. Y. Goh andL.S.Lee Laboratory of Computational Statistics and Operations Research, Institute for Mathematical Research, Universiti Putra Malaysia (UPM), Serdang, Selangor, Malaysia Department of Mathematics, Faculty of Science, Universiti Putra Malaysia (UPM), Serdang, Selangor, Malaysia Correspondence should be addressed to L. S. Lee; lls@upm.edu.my Received 1 November 2018; Revised 28 January 2019; Accepted 18 February 2019; Published 13 March 2019 Academic Editor: Eduardo Fernandez Copyright © 2019 R. Y. Goh and L. S. Lee. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Development of credit scoring models is important for financial institutions to identify defaulters and nondefaulters when making credit granting decisions. In recent years, artificial intelligence (AI) techniques have shown successful performance in credit scoring. Support Vector Machines and metaheuristic approaches have constantly received attention from researchers in establishing new credit models. In this paper, two AI techniques are reviewed with detailed discussions on credit scoring models built from both methods since 1997 to 2018. eTh main discussions are based on two main aspects which are model type with issues addressed and assessment procedures. Then, together with the compilation of past experiments results on common datasets, hybrid modelling is the state-of-the-art approach for both methods. Some possible research gaps for future research are identified. 1. Introduction customers database increased tremendously. In 2004, Basel II accord is released. Under credit risk, rather than the Credit scoring is the basis of financial institutions in making previous standardised method, Internal Rating Based (IRB) credit granting decisions. A good credit scoring model will approach could be adopted by bank to compute the minimum be able to effectively group customers into either default or capital requirement. This marked an evolution in the credit nondefault group. The more efficient it is, the more cost can scoring eld fi , where attempts to form sophisticated model be saved for a financial institution. have been actively researched. Together with the rapid growth Credit scoring is used to model available data and of computer technology, formulation of sophisticated models evaluate every instance in the data with a credit score and is made possible. probability of default (PD). Generally, score is a measurement Hand and Henley [1] first published review paper of of the credit-worthiness of customers, while the PD is the credit scoring domain. They reviewed statistical methods and likelihood estimation of a customer fails to meet one’s debt several data mining (DM) methods and concluded that the obligation in a given period of time. Hand and Henly [1] future trend of credit scoring models will be more complex den fi ed credit scoring as “the term used to describe formal methods. Thomas [2] also reviewed on past researches on statistical methods used for classifying applicants for credit credit scoring and pointed out the importance of protfi into ‘good’ and ‘bad’ risk classes”. Since the final decision scoring. The methods discussed ranged from statistical, is binary, credit scoring is thus equivalent to a binary operations research based, and DM approaches. Sadatrasoul classification problem. et al. [3] reviewed DM techniques applied for credit scoring domain in year 2000-2012, showing the tendency of model When credit cards started to be introduced in 1960s, necessity of credit scoring models is triggered. Financial insti- building with DM methods in recent years. tutions started to combine or replace purely judgemental- There are also review papers that specifically focused on based credit granting decisions with statistical models as application scoring [4] and bankruptcy prediction [5, 6]. 2 Advances in Operations Research Martin [4] reviewed application scoring model in a differ- review, particularly on the development of SVM and MA ent perspective, where procedure of scorecard performance only. assessment by past studies are categorized into consistency, All the research articles in this study are obtained from application t, fi and transparency. The author pointed out three online academic databases, Google Scholar, Science the weaknesses of past experiments that only pay attention Direct, and IEEE Xplore/Electronic Library Online. Several to model development and neglected appropriate assess- main keywords applied to select the articles are credit scoring, ment procedures of the models. Sun et al. [6] reviewed credit risk prediction, metaheuristics, data mining, SVM, on bankruptcy prediction credit models by providing clear Genetic Algorithm, Genetic Programming, Evolutionary definitions of bankruptcy prediction as resulted from litera- Computing, machine learning, and artificial intelligence. ture throughout the years and then discussed the techniques From the search results, there are 44 and 43 articles from based on three main approaches, i.e., modelling, sampling, SVM and MA models, respectively, with 12 articles utilizing and featuring. Alaka et al. [5] also focused on bankruptcy both together, thus resulting in a total of 75 research articles prediction models. They identified the popular statistical and being reviewed in this study ranging from year 1997 to artificial intelligence (AI) tools utilized and discovered 13 criteria of the tools usage. Based on the 13 criteria, they The objectives of this study are to review past literatures developed a framework as a guideline for tool selection. of using SVM and MA in developing credit scoring model, Besides, there is one review from Moro and Rita [7] that identify the contributions of both methods, and discuss the adopted a completely different approach than all the other evolving trend that can lead to possible future works in credit review papers. An automated review procedure is conducted scoring domain. with text mining. Important issues in credit scoring domain This paper is organized as follows. Section 2 discusses the and top ranked tools for model development are discovered evolving trend of the credit scoring models from traditional through automated text mining. methods to DM methods. Section 3 briefly describes SVM Baesens et al. [8] are the rfi st to build credit scoring and MA methods. Section 4 summarizes and discusses the model with Support Vector Machines (SVM) and compare results based on model type with issues addressed, assessment its performance with other state-of-the-art methods. They procedures, and results compilation from past experiments. experimented with standard SVM and Least Squares SVM Then, Section 5 suggests several possible future directions (LS-SVM) and reported that LS-SVM yield good perfor- and draws conclusion for this study. mance as compared to other methods. Thereaer ft , SVM has been actively researched in the credit scoring domain, being 2. Trend one of the mostly used DM methods to build credit scoring model. Recent review studies [5, 7] have also identified SVM Before credit scoring model is developed, credit granting as a significant tool to be selected among researchers for credit decision is purely judgemental-based. Statistical models models development. started to be utilized since 1941 where Durand [14] is the first Metaheuristic Approaches (MA), especially Evolutionary to pioneer the usage of discriminant analysis (DA) to classify Algorithm (EA), have also been introduced as the alternative good and bad loans. Altman [15] also used multiple discrim- to form credit scoring model. The surging applications in inant analysis (MDA) to predict company bankruptcy. The credit scoring can be seen in recent years, where most review financial ratios are treated as input variables and modelled papers included discussion of EA as one of the main DM with MDA. MDA model has shown good prediction ability methods (see Alaka et al. [5], Crook et al. [9], Lahsasna et al. and useful for analyst to provide investment recommenda- [10],Lessmann etal. [11],and Louzada et al. [12]). In Louzada tions. Ohlson [16] then introduced a probabilistic approach to et al. [12] timeline discussion, it can be seen that, after year predict credit-worthiness of companies in 1980. The method 2005, SVM and EA have increased research work in credit proposed is Logistic Regression (LOGIT). Few problems of scoring domain. Besides, Marques et al. [13] have reviewed on using MDA are pointed out and LOGIT is believed to be more credit scoring models specifically focusing in EA, indicating robust than MDA. the popular usage of MA. In 1985, Kolesar and Showers [17] introduced mathemat- To date, reviews on credit scoring domain are mainly ical programming (MP) approach to solve credit granting focusing on a wide category of methods to develop score- problem. MP is compared with classical DA and reported card. The active research utilizing SVM prompted a need results showed that MP is more robust and flexible for to particularly review on this method. There was only decision maker. Among the traditional methods, LOGIT one review by Marques et al. [13] that focused on EA turned out to be the standard credit scoring model because in credit modelling. However, EA are just a part of MA, of its ability to fulfil all requirements from the Basel II where there are other MA that have received attention accord. in the credit scoring domain in recent years due to the shift of modelling trend towards AI-based techniques. The Massive improvement in computer technology opens up surging application of both techniques has greatly increased DM approach in model building for credit scoring. There contributed works, leading to a situation where general have been extensive researches done in the past that utilized reviews are insufficient to peek into the development trend DM methods in the credit scoring domain. Most of them of these two methods. Hence, in contrast with past literature compared the adopted methods with the standard LOGIT reviews which are general reviews, this study is a focused model and have shown the DM models are competitive. A Advances in Operations Research 3 few comparative studies and review papers [1, 2, 8–12, 18] (i) Evolutionary Algorithm (EA) reported good performance of DM models, and among the The EA approach has a mechanism to seek solution various methods, SVM and MA (especially EA) have been following the Darwinian principle, which is based widely researched to be the alternative to the credit scoring on the “survival of the tfi test” concept. There are models. four main procedures to search for a solution, i.e., selection, reproduction, mutation, and crossover. The 3. AI Techniques solutions inthe population areimproved inanevo- lutionary manner, guided by the quality of the tness fi .. Support Vector Machines. Support Vector Machines function. The EA applied in credit scoring are Genetic (SVM) were rs fi t introduced by Vapnik [91] in 1998, in the Algorithm (GA) and Genetic Programming (GP). context of statistical learning theory. There are many suc- (ii) Swarm Intelligence (SI) cessful applications of SVM in pattern recognition, indicating SVM to be a competitive classifier. The SI approach is a nature-inspired algorithm that SVM seeks for an optimal hyperplane with maximum conducts solution-seeking based on natural or arti- margin that acts as the decision boundary, to separate the ficial collective behaviour of decentralized and self- two different classes. Given a training set with labelled organized systems. The solutions in the population instance pairs (𝑥 ,𝑦 ),where 𝑖 is the number of instance 𝑖 𝑖 are improved through the interaction of agents with 𝑖 = 1,2,3,...,𝑚 , 𝑥 ∈ R and 𝑦 ∈{−1,+1},the decision 𝑖 𝑖 each other and also with the environment. Similarly, boundary to separate two different classes in SVM is generally the generation of better solution is led by tness fi expressed as function quality. The SI applied in credit scoring are Particle Swarm Optimization (PSO), Ant Colony 𝑤⋅ 𝑥+ 𝑏 = 0. (1) Optimization (ACO), Scatter Search (SS), Artificial Bee Colony Optimization (ACO), Honey Bees Mating Optimal separating hyperplane is the one with maximum Optimization (HBMO), Cuckoo Search (CS), and margin and all training instances are assumed to satisfy the Harmony Search (HS). constraint, (iii) Iterative Based (IB) 𝑦 𝑤⋅ 𝑥 +𝑏 ≥1. ( ) (2) 𝑖 𝑖 The IB approach focus to improve on one single The convex optimization problem is then defined as solution in each iteration around its neighbourhood, where the solution improvement is based on the quality of tness fi function. u Th s, IB is different min 𝜙 (𝑤, 𝑏 ) = ‖𝑤 ‖ +𝐶 ∑ 𝜖 𝑖=1 as compared to EA and SI which are population- (3) based. The IB applied in credit scoring are Simulated s.t.𝑦 (𝑤 ⋅ 𝑥 +𝑏) ≥ 1. 𝑖 𝑖 Annealing (SA) and Tabu Search (TS). The optimal hyperplane is equivalent to the optimization problem of a quadratic function, where Lagrange function is 4. Discussions utilized to find the global maximum. The 𝜖 is the slack vari- .. Model Types with Issues Addressed. SVM and MA credit able introduced to take account for misclassification, with 𝐶 models are categorized according to the type of models as the accompanied penalty cost. For nonlinear classification, formed. There are four main categories of the SVM being kernel trick is used to modify the SVM formulation. Popular utilized in credit scoring: standard SVM and its variants, kernel choices are linear, Radial Basis Function (RBF), and modiﬁed SVM, hybrid SVM, and ensemble models.On the polynomial. other hand, MA approaches applied in credit scoring can (i) Linear: 𝑥 ⋅𝑥 . 𝑖 𝑗 be divided into three categories: standard MA, hybrid MA- MA, and hybrid MA-DM. The models development and main (ii) Polynomial: (𝑥 ⋅𝑥 +𝑐) , 𝑐 >0, 𝑑 ≥2 . 𝑖 𝑗 issues addressed are discussed. (iii) Radial Basis Function (RBF): exp{−𝛾‖𝑥 −𝑥 ‖ }. 𝑖 𝑗 ... SVM .. Metaheuristic Algorithms. Metaheuristic Algorithm (MA) is one of the AI-based data mining approaches () Standard SVM and Its Variants. SVMs applied to build which had gained attention in recent years. MA is an credit scoring models discussed here are the standard SVM automated process to search for a near-optimal solution of and its variants, where these SVMs are being applied directly an optimization problem. The search process is conducted for model building without any modifications. with operators to ensure a balance between exploration and exploitation to efficiently seek for good solutions. Generally, Investigation of Predictive Ability. The predictive ability MA consists of several categories; the description of the of SVMs credit models is examined through two main categories that have been used in credit scoring domain is approaches, i.e., comparative studies [8, 11, 12, 18, 26] and given as follows: application on specific credit domain [19–25, 27]. 4 Advances in Operations Research Various state-of-the-art methods ranging from tradi- outperformances across all the datasets. They concluded BN tional statistical methods to nonparametric DM methods and the Boosting ensemble as generally effective method as have been attempted to form credit scoring models in differ- credit models. These comparative studies have included a wide variety of ent researches. This prompted the necessity of a benchmark study conducted by Baesens et al. [8] in 2003. In their study, classification techniques to formulate credit scoring models SVM and Least Squares SVM (LS-SVM) are first utilized and their predictive ability is assessed across various credit datasets. However, among these credit models, there has not to develop credit scoring model, and their performances are compared with the other state-of-the-art methods across been a clear best technique for the credit scoring domain. eight different datasets. The methods are from the family of Nonetheless, these comparative studies have provided a LOGIT, DA, Linear Programming (LP), Bayesian Network guideline on the update of latest available models in credit (BN), k-nearest neighbourhood (kNN), neural networks scoring, where SVM is initially being tagged as state of the artin[8] and then served asanimportant model inthis (NN), and decision trees (TREE). LS-SVM and NN reported statistically significant better results compared to the other domain. Besides, ensembles are another state of the art as models. Their results also showed that the methods are demonstrated in [11]. Instead of comparative studies with various techniques competitive to each other. Hence, SVM has been actively researched in the credit scoring domain. being benchmarked, a SVM-focused study is conducted by Lessmann et al. [11] updated Baesens et al. [8] research Danenas et al. [26], particularly on bankruptcy prediction dataset. Various types of SVM originating from different by including comparison with more individual classifiers, ensembles, and hybrid methods. They commented the libraries and software, i.e., LIBSVM, LIBLINEAR, WEKA, increasing research on credit scoring domain that urged for and LIBCVM, are included in their experiment. The SVM necessity to update the benchmark study where not only classifiers ranged from linear to nonlinear with a wide variety state-of-the-art classifiers but also advanced techniques like of kernels. Models are investigated on original dataset and reduced dataset already preprocessed with feature selection ensembles should be included as well. u Th s, there are a total of 41 classifiers (from the family of BN, TREE, LOGIT, DA, NN, technique. Comparing the accuracy, different types of SVM kNN, Extreme Learning Machine (ELM), and ensembles) classifiers showed comparable results, with application on reduced dataset having higher accuracy. Besides, in terms of being investigated across six datasets with wide range of size in the benchmark study. The experimental results showed computational effort, linear SVM is the fastest model. that ensemble models took up top 10 ranking among all Another approach to investigate the predictive ability is the 41 classifiers. For individual classifiers, NN showed via the application on specific domains. The specific domains highest ranking. Linear and RBF SVM were investigated; both experimented are multiclass corporate rating [19, 20, 22, 23, SVM models showed similar performance as both scored 25], application scoring [21, 24], and behavioural scoring similar ranking. They also pointed out the importance of [27]. For multiclass corporate rating, credit scoring models developing business-valued scorecard, which should be taken are formed with LS-SVM in [19, 22] and SVM in [20, 23, 25] with the main aim to show the effectiveness of SVM into consideration in proposing new models. Louzada et al. [12] conducted a systematic review on in building corporate rating models. Van et al. [19] and classification methods applied on credit scoring, covering Lai et al. [22] showed that LS-SVM is the best performing credit model compared to other traditional techniques when research papers of year 1992-2015. Their review discussed several aspects in the credit scoring domain: objective of applied on Bankscope and England financial service data, the study, comparison done in the research, dataset used respectively. for model building, data splitting techniques, performance Huang et al. [20] tested SVM on Taiwan and US market measures used, application of missing data imputation, and bond rating data. They also conducted cross market analysis application of feature selection. They carried out experiment with variable contribution analysis from NN models. Lee on 11 main classification techniques (Linear Regression (LR), [23] utilized SVM for Korea corporate credit rating analysis. NN, TREE,DA,LOGIT,FUZZY,BN, SVM,Genetic Pro- Kim and Sohn [25] in 2010 have initiated the building of credit scoring model for small medium enterprises (SME). gramming (GP), hybrid, and ensembles), where inclusion of ensembles started to receive attention as recommended in They focused on Korea SME and the explanatory variables [11]. The problem of imbalance dataset is investigated and included four main aspects: SME characteristics, technology evaluations, financial ratios, and economic indicators. They SVM showed stability in dealing with imbalance dataset. Generally, SVM has better performance and lower compu- believed that SVM model would be suitable to be used tational effort in the experiment. for technology-based SMEs. For all the SVM models on The most recent comparative study is contributed by these four researches on different market data of corporate Boughaci and Alkhawaldeh [18]. They investigated perfor- rating, SVM have outperformed the other methods in every mances of 11 machine learning techniques (from the family of experiment. kNN, BN, NN, TREE, SVM, LOGIT, and ensembles) across For application scoring, Li et al. [21] compared SVM eight different datasets. A main difference of their study with NN on Taiwan bank consumer loan credit scoring. SVM reported higher accuracy than NN and the results is from the previous one is that the datasets utilized involved a mixture of application scoring and bankruptcy prediction statistically significant. Besides, they also experimented the dataset. The experimental results did not suggest a winner effect of different hyperparameter values of SVM on the type I error and type II error, i.e., misclassicfi ation error. They among the techniques investigated as there are no consistent Advances in Operations Research 5 demonstrated the effect on misclassification error across the Dynamic Scoring. Yang[30] modiefi dweightedSVM model hyperparameter values range can serve as a visualization to become a dynamic scoring model. The main idea is to tool for credit scoring model. u Th s, they concluded SVM enable an easy update of credit scoring model without the need to repeat variable selection process when new customers outperformed NN in terms of visualization. Bellotti and Crook [24] tested SVM of different kernels on large credit data became available. Original kernel in weighted SVM card datasets. Comparison with traditional statistical models is modified to become an adaptive kernel. When there is reported only kNN and polynomial SVM have poorer results increment in the data size, adaptive kernel can automatically which may be due to overtfi ting. They suggested using update the optimal solution. Besides, with the trained model, support vectors’ weight as an alternative to select significant Yang [30] suggested an attribute ranking measure to rank the features and compared the selected features with those from kernel attributes. us, Th this became an alternative solution for LOGIT’s Wald statistics. The experiment indicated SVM the black box property of SVM. models are suitable for feature selection for building credit Reject Inference. The aim of reject inference is to include scoring model. rejected instances into model training, then improving clas- For behavioural scoring, South African unsecured lend- sicfi ation performance. Li et al. [31] and Tian et al. [32] pro- ing market is investigated by Mushava and Murray [27]. posed new SVM to solve reject inference problem for online Despite having the aim to show the effectiveness of SVM, this peer-to-peer lending. Li et al. [31] proposed a semisupervised study aimed to examine some extensions of statistical LOGIT L2-SVM (SSVM) to solve reject inference for a peer-to-peer and DA that have been less explored in credit scoring domain, lending data from Lending Club of different years. us Th , with SVM being included as a benchmark comparison in in the SSVM formulation, unlabelled rejected instances are their study. In their fixed window experiment, Generalized added to the optimization constraints of SVM, converting Additive Models have outperformed the others. Although the original quadratic programming problem to a mixed SVM did not show superior performance, the inclusion of integer programming. SSVM reported better performance SVM in this study once again indicated SVM is perceived as than other standard methods. Tian et al. [32] proposed a standard to overcome in credit scoring domain. a kernel-free fuzzy quadratic surface SVM (FQSVM). The () Modiﬁed SVM. Modiefi d SVM involved algorithmic main advantages of the proposed model are the ability to detect outliers, extract information from rejected customers, change in the formulation of SVM. There are a few works that proposed modiefi d SVM for solving dieff rent problems no kernel required for computation, and efficient convex in the credit scoring domain, particularly in application optimization problem. The proposed model is benchmarked against other reject inference methods. FQSVM is reported to scoring. The modifications required changes in the quadratic programming formulation of the original SVM. be superior than SSVM proposed by [31] in terms of several performance measures as well as computational efficiency. Outlier Handling. Wang et al. [28] proposed a bilateral fuzzy Features Selection. There are two new formulation of SVM to SVM (B-FSVM). The method is inspired from the idea that carry out features selection in cost approach [33] and protfi no customer is absolutely good or bad customer as it is approach [34] on Chilean small and microcompanies. The always possible for a classified good customer to default dataset consisted of new and returning customers, indicating and vice versa. us, Th they utilized membership concept in the credit scoring models formed involved both application fuzzy method where each instance in the training set takes and behavioural scoring. Maldonado et al. [33] included positive and negative classes, but with different memberships. variable acquisition costs into formulation of SVM credit This resulted in bilateral weighting of the instances because scoring model to do feature selection. Similar to Li et al. [21], each instance now has to take into account error from they also added additional constraints into the optimization both classes. By including the memberships from fuzzy, problem, converting it to a mixed integer programming prob- SVM algorithm is reformulated to form B-FSVM. They used lem, but the added constraints are the variable acquisition LR, LOGIT, and NN to generate membership function. B- costs. They proposed two models where 1-norm and LP- FSVM are compared with unilateral fuzzy SVM (U-FSVM), norm SVM are both modified with the additional constraints, and other standard methods. Linear, RBF, and polynomial forming two new credit scoring models, namely, L1-mixed kernels are used to form B-FSVM, U-FSVM, and SVM integer SVM (L1-MISVM) and LP-mixed integer SVM (LP- models. MISVM). Due to the ability of the proposed models to take into consideration of variable acquisition costs and good Computational Eﬃciency. Harris [29] introduced clustered performance simultaneously, it is believed that the proposed SVM (CSVM), a method proposed by Gu and Han [92], methods are efficient tool for credit risk as well as business into credit scoring model. The research aimed to reduce analytics. computational time of SVM model. With k-means clustering On the other hand, Maldonado et al. [34] introduced a to form clusters, these clusters will be included into the formulation of SVM optimization problem, changing the prot- fi based framework to do feature selection and classi- original SVM algorithm. Two CSVM models are developed fication with modified SVM as well as protfi performance metrics. Instead of considering acquisition costs for variables, with linear and RBF kernel and compared with LOGIT, SVM, and their hybrids with k-means. Excellent time improvement one by one, they treated the costs as a group to be penalized of CSVM is reported. on whole set of variables. Therefore, the L- ∞ norm is 6 Advances in Operations Research utilized to penalize the group cost. Two models are proposed classification. The proposed method is a two-phase pro- with 1-norm SVM and standard SVM being modified by cedure. In the first phase, they introduced to use three including L-∞ into optimization objective function, forming different algorithms of link relation to extract input features by linking the relations of applicants’ data. So, there are three L1L-∞SVM and L2L-∞SVM. Their proposed models are effective in selecting features with lower acquisition costs, hybrid SVM models being built with respect to the three yet maintaining good performance. The main difference as different algorithms. Recently, Han et al. [50] proposed a compared to the previous research in [33] is that, with this new orthogonal dimension reduction (ODR) method to do newly proposed model, the protfi can be assessed, which feature extraction. They used SVM as the main classifier as posed as a crucial insight for business decision makers. they believed ODR is an eeff ctive preprocessing for SVM and used LOGIT as benchmark classiefi r. There are three () Hybrid SVM. Hybrid SVM credit scoring models have main parts in the experiment. First, they discovered that been developed by collaborating SVM with other techniques variables normalization posed large effect on classification for different purposes. performance of SVM but LOGIT is not strongly affected. Therefore, normalization is applied for all models. Second, Reject Inference. Chen et al. [48] tackled this problem using comparison is done with existing feature reduction method, the credit card data from a local bank of China. They principal component analysis (PCA). Third, they suggested hybridized k-means clustering with SVM, formulating a two- using LOGIT at the start to pick important variables with stage procedure. The rfi st stage is the clustering stage where Wald statistics and then only extract features from the new and accepted customers are grouped homogeneously, reduced variables, which they name as HLG. They concluded isolated customers are deleted, and inconsistent customers ODR is eeff ctive in solving dimension curse for SVM. are relabelled. The clustering procedure of dealing with inconsistent customer is a type of reject inference problem. Features Selection. For feature selection hybrid SVM models, These clustered customers from the rfi st stage are input to there are filter approach [37, 39, 43] and wrapper approach SVM to do classification in the second stage. Instead of [54, 55] used. For lfi ter approach, rough set theory is classifying customers into binary groups, they attempted to employed by Zhou and Bai [37] and Yao et al. [39], where classify into three and four groups. Different cutoff points rough set select input features in the first stage and carried are also set for different groups. They believed the proposed out classification tasks in the second stage with respect to method is able to provide more insight for risk management. the hybridized techniques. There are three main differences in between the hybrid models collaborated with rough sets Rule Extraction. Black box property of SVM has always proposed in these two researches. First, Zhou and Bai [37] been the main weakness, which is also a main concern specified their study on Construction Bank in China whereas for practitioners not using SVM as credit scoring models. Yao et al. [39] study is generally on public datasets. Second, Martens et al. [36] proposed rule extraction technique to be features selection is based on information function in [37] used together with SVM. Three different rule extraction tech- whereas computed variable importance is used to select niques, namely, C4.5, Trepan, and Genetic Rule Extraction features in [39]. Third, the hybrid models developed in [37] (G-REX), are hybridized with SVM. Experiment is conducted are hybridization with NN, SVM, and GA-SVM (GA to tune on different fields that require comprehensibility of model SVM hyperparameters) whereas the hybrid models devel- where credit scoring is one of the efi ld addressed in their oped in [39] are hybridization with SVM, TREE, and NN. research. The proposed models are advantageous in giving Both experiments showed that SVM-based hybrid models clear rules for interpretation. In 2008, Martens et al. [93] obtained best performance. Chen and Li [43] also adapted made an overview on the rule extraction issue, where the a lfi ter approach to do feature selection. They proposed four importance of comprehensibility in credit scoring domain different filter approaches: LDA, TREE, rough set, and F- is addressed again. Zhang et al. [38] proposed a hybrid score. These four approaches are hybridized with SVM. In credit scoring model (HCSM) which hybridized Genetic order for the models to be comparable, the same number of Programming (GP) with SVM. The main advantage of the features is selected based on the four approaches based on proposed technique is the ability to extract rules with GP that variable importance. solved theblack box natureofSVM. Apart from the filter approach of conducting feature selection, Jadhav et al. [54] and Wang et al. [55] incorporated Computational Eﬃciency. Hens and Tiwari [46] integrated filter techniques in developing novel wrapper model for fea- the use of stratified sampling method to form SVM model. ture selection task. The main concept is to guide the wrapper Then, with the smaller sample, F-score approach is used to model to do feature selection with obtained information from do feature selection to compute the rank of features based filter feature ranking techniques. on importance. The proposed model achieved lowest runtime Jadhav et al. [54] proposed Information Gain Directed and comparable performance when compared with other Feature Selection (IGDFS) with wrapper GA model, based methods considered in their experiment. on three main classiefi rs, i.e., SVM, kNN, and NB. Top rank features from information gain are passed on to the Features Extraction. Xu et al. [40] incorporated link anal- wrapper GA models. They compared their three different ysis with SVM, where link analysis is rfi st used to extract IFDFS (based on three different classifiers) with six other models: three standalone classiefi rs with all features included, applicants’ information and then input into SVM to do Advances in Operations Research 7 and three standard GA wrapper models of the classifiers weights can be assigned to solve class imbalance during that conducted feature selections without guided by any filter classification. Different from [42], they recommended the methods. Wang et al. [55] hybridized SVM with multiple use of DOE for hyperparameters tuning due to competitive resultsreported ascompared to DS, GA, and GS, but with population GA to form a wrapper model for feature selection. The method has a two-stage procedure. In the first stage, they lowest computational time. External benchmarked against utilized three lfi ter feature selection approaches to find prior results from [8, 45] are also included. Chen et al. [49] and Hsu et al. [53] integrated ABC information of the features. Feature importance is sorted in descending order, then a wrapper approach is used to find and SVM for hyperparameters tuning in corporate credit optimal subset. With the three feature subsets from the three scoring. Chen et al. [49] applied the proposed ABC-SVM on approaches and probability of a feature to be switched on, Compustat database of America from year 2001-2008. PCA is they formed the initial populations to be passed on to the the data preprocessing method used for extracting important features. Recently, Hsu et al. [53] also researched on ABC- second phase. In the second phase, HMPGA with SVM is run to find the final optimal feature subset; thus HMPGA-SVM is SVM in corporate credit rating with dataset from the same the model with prior information. database as in [49], but including more recent years 2001- 2010. Similarly, they utilized PCA as the preprocessing step. Hyperparameters Tuning. Other than input features, hyper- They conducted a more detailed study on the data, where parameters of SVM models pose great effect on the end using information from PCA, they divided the dataset into model formed. Previous works that proposed models for three categories to study the ability of the credit models to feature selection have applied the conventional Grid Search account for changes in future credit rating trend. (GS) method to find the appropriate hyperparameters. The researches that introduced hybrid SVM for finding hyperpa- Simultaneous Hyperparameters Tuning and Features Selection. rameters are [47, 52] in bankruptcy prediction, [42, 44, 51] in Based on the previous discussed research works, feature application scoring, and [49, 53] in corporate rating. selection and hyperparameter selection for SVM models With the success of linear SVM as experimented in [26], in credit scoring are crucial procedures in model building. Danenas and Garsva [47] examined the use of different linear Therefore, two pieces of research [35, 41] aimed to solve both SVMs available in the LIBLINEAR package on bankruptcy problems simultaneously with wrapper model approach. prediction. All the techniques are hybridized with GA and Huang et al. [35] attempted three strategies for building PSO to do model selection and hyperparameters tuning. The SVM-based credit scoring models: rs fi t, GS to tune SVM hybrid models formed are, namely, GA-linSVM and PSO- hyperparameters with all features included; second, GS to linSVM. Sliding window approach is adapted for building find hyperparameters and F-score to find feature subsets models across different time periods to report model per- for SVM model building; third, the initiation of hybrid formances. GA-linSVM is concluded to be more stable than GA-SVM to search hyperparameters and feature subsets PSO-linSVM by consistently selecting same model across simultaneously. This experimental result indicated GA-SVM dieff rent time periods yet having good performance. In later as a suitable tool for alternative to solve both issues together years, Danenas and Garsva [52] conducted another research but required high computational eor ff t. Zhou et al. [41] also to improve on PSO-linSVM. They modified PSO by using formulated a hybrid model using GA, but using dieff rent integer values for velocity and position, instead of rounding variants of weighted SVM, thus forming GA-WSVM. They up the values as in [47, 51]. mentioned in Huang et al. [35] research that the features PSO-linSVM continued to receive attention by Garsva found did not carry importance of the selected features. and Danenas [51] in application scoring. Similarly, they u Th s, they proposed feature weighting as one of the addition carried out model selection and hyperparameters tuning with procedure in the wrapper approach. The proposed model PSO-linSVM butwitha mixed searchspace for PSO, which aimed to search for hyperparameters of WSVM as well as is a slight modification as compared to previous work [47]. feature subsets with feature weighting. They compared the Comparison is done with SVM and LS-SVM (of different feature weighting method with t-test and entropy based kernels), of which the hyperparameters are tuned with PSO, method. Direct Search (DS), and SA, respectively. To address the data imbalance problem, they investigated the use of True () Ensemble Models. The two main types of ensemble models Positive Rate (TPR) and accuracy as the tness fi function. TPR are homogeneous (combining same classifiers) and hetero- is concluded as appropriate tness fi function for imbalance geneous (combining different classifiers). In credit scoring dataset. domain, [56–58] worked on homogeneous ensembles while Zhou et al. [42] presented Direct Search (DS) to tune Xia et al. [59] worked on heterogeneous ensembles. LS-SVM hyperparameters. They compared DS with GA, GS, and Design of Experiment (DOE). Among the four hybrid ImprovePredictiveAbility. Zhou et al. [56] pointed out induc- tive bias problem of single classifier when using fixed training models, DS-LS-SVM is the recommended approach due to its best performance. Yu et al. [44] conducted a similar samples and parameter settings. Therefore, they introduced ensemble model based on LS-SVM to reduce bias for credit experimentalsetupwith[42]whereDS,GA,GS,andDOEare presented for hyperparameters tuning. The main difference is scoring model. The two main categories of ensemble strate- that they considered class imbalance problem, so the model gies introduced are the reliability-based and weight-based. There are three techniques, respectively, for each category, tuned is weighted LS-SVM (WLS-SVM), where different 8 Advances in Operations Research resulting in a total of six LS-SVM-based ensemble models different GA-derived models are developed which consider being formed. Another research that proposed ensemble seeding (prior information from LOGIT as initial solution) model is by Ghodselahi [57]. They recommended using fuzzy and different encoding scheme (integer or binary). C-means clustering to preprocess the data before fed into Cai et al. [63] and Abdou [65] have included misclassifi- SVM. Then, 10 of the hybrid SVM base models formed the cation costs into their studies. Cai et al. [63] established credit ensemble models, using membership degree method to fuse scoring model with GA. The optimization problem is a linear all the base models as the final ensemble results. Xia et al. [59] scoring problem as in [62]. They computed the appropriate introduced a new technique named as bstacking. The main cutoff as the weighted average of good and bad customers idea is to pool models and fuse the end results in a single critical values. A tness fi function is formed that considered step. Four classifiers are used, i.e., SVM, Random Forest (RF), all components in the confusion matrix together with the XGBoost, and Gaussian Process Classifier (GPC), as the base associated misclassicfi ation costs. Abdou [65] compared per- learners due to their accuracy and efficiency. formance of GP with profit analysis and weight-of-evidence (WOE) model. Two types of GP are examined here: single Data Imbalance. Yu et al. [58] developed Deep Belief Network program GP and team model GP, which is a combination SVM-based (DBN-SVM) ensemble model in a different of single program GP for better results. They conducted approach where the main aim is to solve dataset imbalance sensitivity analysis based on dieff rent misclassicfi ation ratios problem. Their model has a three-stage procedure. In the and emphasized the importance to evaluate scorecard with first stage, data is partitioned into various training subsets misclassification costs. Only Yang et al. [68] addressed the with bagging algorithm, and each subset is resampled to bankruptcy prediction issue with CS. The authors developed rebalance the instances. In the second stage, SVM classifiers aCS model whichused Lev ´ y’s flight to generate new solution. are trained on the rebalanced training subsets. In the third stage, DBN is employed to fuse the final results. Proposed Rules Extraction. There are several researches focused on method is compared with SVM and ensemble SVM with rules extraction with different MA which are GP in [61, 64], majority voting. SA in [66],and ACOin [67,69].Onget al.[61]recom- mended using GP as an alternative to form credit models. GP undergone the same procedures as in GA to search for ... MA solution, but the main difference is that GP generates rules to do classification. The authors concluded several advantages of () Standard MA. Standard MA being attempted to form GP to build credit scorecard: nonparametric that is suitable credit scoring models are GA, GP,SA,ACO,and CS.The for any dataset, automatic discrimination function that do credit scoring problem is formulated with these MA as not require user-defined function as in NN, and better rules an optimization problem to be solved with respect to the obtained compared to TREE and rough set that generated objective functions. lower error. Huang et al. [64] proposed a 2-stage GP (2SGP). In the rfi st stage, IF-THEN rules are derived. They formulated Investigation of Predictive Ability. The predictive ability of MA GP to ensure that only useful rules are extracted. Based on credit models has been tested through application on specific these rules, the dataset will be reduced by removing instances credit domains, i.e., application scoring [60, 62, 63, 65] and that do not satisfy any rules or satisfy more than one rule. bankruptcy prediction [68]. The experiments of Desai et al. Then, the reduced data is passed on to the second stage of [60], Finlay [62], and Abdou [65] are specicfi study on a GP where the discriminant function is used to classify the particular country which are credit unions in Southeastern customers. US, large data of UK credit application, and Egyptian public Dong et al. [66] established SA rule-based extraction sector bank, respectively. In contrast, Cai et al. [63] and Yang algorithm (SAREA) for credit scoring. Similar to the previous et al. [68] conducted a general study based on the public rule extraction with GP in [64], the proposed SAREA is German dataset and simulated database of 20 companies, respectively. also in two-phase, and the extracted rules are the IF-THEN rules. In the rfi st phase, SA is run on initial rules and their Back in 1997, Desai et al. [60] investigated the predictive corresponding best accuracy rules are put into the final rule ability of AI techniques by comparing with traditional credit models LOGIT and DA. The AI techniques studied are three setof rfi stphase.Thebestrule from the rfi st phase isrequired for tness fi function computation in second phase to penalize variants of NN and GA. They classified the credit customers the accuracy. In the second phase, SA is run again on random into three classes (good, poor, and bad) instead of the usual initial rules to find their corresponding best accuracy rules to binary (good and bad). GA is used for discriminant analysis. form the final rule set, but the tness fi computation will be the With the aim to minimize the number of misclassification, penalize accuracy based on the best rule from rfi st phase. an integer problem is formulated with GA to make it acting The importance of comprehensibility for credit scor- equivalently to branch-and-bound method, then, solving the dual problem gives the final separating hyperplane. Finlay ing model had been pointed out [36, 93] as discussed in [62] pointed out the advantage of developing credit scoring Section 4.1.1(2). Martens et al. [67] researched on establishing model with GA due to its ability to form model based on a novel model that has good performance for both accuracy self-decide objective function. They proposed to build a linear and comprehensibility. They introduced ACO algorithm in scoring model with GA that maximizes the GINI coefficient. AntMiner+ as the potential credit scoring model. The rules Large dataset of UK credit application is experimented. Four have high comprehensibility which is crucial for business Advances in Operations Research 9 decisions and they also analysed the extracted rules to tuning of NN [75, 84], and hyperparameters tuning of SVM be integrated with Basel II accord. Recently, ACO-based [37, 42, 44, 47, 49, 51–53]. classification rule induction (CRI) framework is introduced For NN model tuning, Wang et al. [77] hybridized GA with NN to tune the input weight and bias parameters. They by Uthayakumar et al. [69]. They carried out their experiment on both qualitative and quantitative datasets, focusing on employed real-valued encoding for GA, using arithmetic bankruptcy prediction. ACO algorithm is modified based on crossover and nonuniform mutation and concluded that the concept of rule induction. Due to the ability of ACO tuning parameters with GA improved learning ability of NN. to provide better results in CRI, reducing rules complexity On the other hand, Lacerda et al. [75] established GA to tune and effective classification of abundant data, ACO is recom- hyperparameters in NN. They proposed a modified GA based mended in their study. on consideration of redundancy, legality, completeness and casuality. Training samples are clustered and they introduced () Hybrid MA-MA. Hybrid MA-MA involves two dieff rent cluster crossover algorithm for GA. Then, they utilized the MA being integrated together to form a new method. There proposed GA to form a multiobjective GA in seeking for are only two research works that have been proposed thus NN hyperparameters. Correa and Gonzalez [84] presented far. two hybrid models, GA and binary PSO (BPSO) for hyper- parameters tuning in NN. For both MA techniques, cost of Parameters Tuning. Jiang et al. [70] proposed the idea of using the candidate solutions is computed before proceeding to SA to optimize GA, forming hybrid SA-GA. SA is integrated the searching process. They presented a different approach into GA to update population by selecting chromosomes in examining their proposed models, where they conducted using Metropolis sampling concept from SA. Two variants of cost study on three different scoring models, i.e., application NN are the main classifiers used in this experiment. SA-GA scoring, behavioural scoring, and collection scoring. is utilized to search the input weight vector of the combined Several models that utilized GA to tune SVM hyperpa- NN classifiers. rameters [37, 42, 44] have been discussed in Section 4.1.1(3). In Zhou and Bai [37] experiment, the proposed model Rules Extraction. Aliehyaei and Khan [71] presented a hybrid worked best with GA tuned SVM. In Zhou et al. [42] experi- model of ACO and GP with a two-step task. ACO is ment, investigation on SVM hyperparameters tuning is con- responsible for searching for rule sets from the training ducted on DS, GS, DOE, and GA. In Yu et al. [44] experiment, set. The rulesextracted from ACO isthen fed into GP for they investigated LS-SVM hyperparameters tuning also with classification. DS, GS, DOE, and GA. From these three researches, it can be observed that they included GA tuned SVM-based classifiers () Hybrid MA-DM. Hybrid MA-DM methods include the into their experiments for comparison, implying that EA is a usage of a MA technique to assist the classification task of DM good alternative for hyperparameters tuning in SVM. Model classifier, thus improving model performance. selection and hyperparameters tuning on linear SVM from LIBLINEAR using GA and PSO have been investigated in a Rules Extraction. Past researches that aimed to do rules few studies [47, 51, 52], as discussed in Section 4.1.1(3) GA- extraction are presented in [38, 79] which utilized SA and linSVM showed better performance than PSO-linSVM on GP, respectively. Zhang et al. [38] proposed a hybrid credit bankruptcy prediction [47]. Then, Garva and Danenas [51] scoring model (HCSM) which is a 2-stage model incorpo- further experimented on modified PSO to form PSO-linSVM rating GP and SVM. In the first stage, GP is used to extract on application scoring. PSO is further modified by Danenas rules with misclassification of type I error and type II error and Garsva [52], forming a different version of PSO-linSVM taken as the tness fi function. In the second stage, SVM is to build bankruptcy prediction model. There are also two used as the discriminant function to classify the customers. models that tuned SVM with ABC [49, 53] as discussed in Jiang [79] incorporated SA and TREE as a new credit scoring Section 4.1.1(3). Both researches focused on corporate credit model. Rules from TREE are input as initial candidate for rating problem. Chen et al. [49] rfi st formulated ABC-SVM SA, then SA produced new decision rules for classification. then followed by a more detailed study in Hsu et al. [53]. Both They formed three TREE-SA credit models with different studies reached the same conclusion with ABC-SVM being discriminant function that account for type I error and type an effective method to tune hyperparameters and Hsu et al. II error, which is similar to Zhang et al. [38]. [53] indicated that ABC-SVM is also able to capture changing trend of credit rating prediction. Parameters and Hyperparameters Tuning. Generally, param- eters and hyperparameters have significant effect on model Features Extraction. Fogarty and Ireson [72] hybridized the performance. The difference between the two is that, param- GA and BN. GA is utilized to optimize BN by selecting cat- egories and attributes combinations from training data using eters are involved in model training, where the value can be evaluated by the model, whereas hyperparameters are cooccurrence matrix. The attributes combinations generated completely user-defined where its value cannot be evaluated are analogous to extracted features. Liu et al. [76] designed GP by the model. Therefore, tuning appropriate values for param- to extract features by selecting derived characteristics, which eters and hyperparameters are crucial in model building for are attribute combinations but determined with analysis credit scorecard. Metaheuristics is applied for parameters method and human communication. To ensure the derived tuning of NN inputweightand bias [77], hyperparameters characteristics are practical, the characteristics are generated 10 Advances in Operations Research with GP by maximizing information value together with with information gain. On the other hand, for Wang et al. application of human communication. Linear DA model is [55], the idea is to formulate a wrapper-based SVM model then built using these derived characteristics. Zhang et al. with multiple population GA. They incorporated information from different filter techniques to be input as initial solutions [78] formed hybrid model by incorporating GA, k-means clustering, and TREE together. The GA is introduced to do for the multiple population GA. attribute reduction, which is a kind of feature extraction. Marinakis et al. [80] formed wrapper model with ACO Binary encoding is applied and the candidate solutions in GA and PSO. The kNN and its variants (1-NN and weighted kNN) consist of breakpoints. Then k-means clustering is assigned to are being wrapped to do feature selection and classification. remove noise and TREE to do classification. The experiment is on multiclass problem using two datasets from a UK nonfinancial firm where the first dataset is to Features Selection. Hybrid models of MA-DM developed for do credit risk modelling and the second dataset is for audit features selection are [54,55,74,81,83,86,89,90].GA from qualification. Later, Marinaki et al. [82] conducted a research EA category is the most popular method to be hybridized with similar setup as in [80] andusedthe rfi st dataset as with DM classiefi rs [54,55,81,83,86,89] for solving feature in [80]. They proposed a different metaheuristics which is selectionincredit scoring domain. All thehybrid MA-DM HBMO to wrap the kNN and its variants. A music-inspired for feature selection are based on wrapper approach except SI technique, HS, is attempted by Krishnaveni et al. [90] for [87–89] which presented filter approach. recently to form a wrapper model with a kNN variant, i.e., Drezner et al. [74] constructed a new credit scoring 1-NN for feature selection. Besides, parallel computing is also model by incorporating TS with LR for feature selection, attempted with the model and reported a significant time- focusing on bankruptcy prediction. First, the selected feature saving with the paralleled version of the proposed method subset from TS is compared with a known optimal subset, compared to other wrapper models. subset from stepwise regression, and subset from maximum Apart from wrapper approach, filter approach models 𝑅 improvement. The TS-feature subset is reported to be have also been proven useful in the credit scoring domain. very competitive in selecting a near-optimal feature subset. Wang et al. [87] hybridized TS with rough sets (TSRS) Sustersic et al. [81] introduced GA to do feature selection to search for minimal reducts that served as the reduced with Kohonen subsets and random subsets. PCA is the feature subsets to be input into classifiers. Japan dataset is benchmark feature selection method compared with the experimented in the experiment. The feature subset from two GA-generated features. NN and LOGIT models are TSRSis fedinto RBFnetwork, SVM, andLOGIT.Later,Wang developed with the features generated. Their experiment is et al. [88] attempted hybridization of SS with rough sets specified on a Slovenian bank loan data. The authors also (SSRS) for feature selection. Similar experiment setup as in discussed the effect of setting cutoff point on the change in [87] is conducted, but with two differences: one additional typeIerror and typeIIerror.Huang and Wu [83]examined dataset is included and the classifiers included for comparison the effectiveness of GA for feature selection. GA is wrapped are different. Waad et al. [89] proposed another filter-based with kNN, BN, LOGIT, SVM, NN, decision tables, and three feature selection method with GA. The new method is a two- different ensembles to carry out feature selection task. In the stage lfi ter selection model. The rfi st stage formulated an first part, standalone classifiers (without wrapped with GA) optimization problem to be solved with GA. The main idea are compared. In the second part, GA-based feature selection in the rfi st stage is to overcome selection trouble and rank is compared with four filter feature selection techniques, aggregation problem and then sort the features according to i.e., chi-squared statistics, information gain, ReliefF, and their relevance. In the second stage, an algorithm is proposed symmetrical uncertainty. In the third part, every standalone to solve disjoint ranking problem for similar features and classifier is compared with their GA-wrapped counterpart. remove redundant features. Some proposed wrapper models included lfi ter tech- niques to gain useful feature information for improving the Features Discretization. There is only a single research con- standard wrapper approach. This type of wrapper model tributed by Hand and Adams [73] for demonstrating a new has been presented by [54, 55, 86]. Oreski and Oreski [86] model that does feature discretization in credit scoring. They tackled feature selection problem by proposing hybrid GA formed collaborated SA with weight-of-evidence (WOE) with NN. They incorporated four different filter techniques and generalized WOE, forming two wrapper models. The to develop the wrapper GA-NN. With the feature ranking main concept is to effectively discretize continuous attributes from the lt fi er methods, three main procedures are infused into appropriate intervals. The proposed SA discretization into the GA-NN. The three procedures are to reduce search technique is compared with quantile partitioning and equal space using the reduced features from the lfi ter ranking, interval division. refine the search space with GA, and induce diversity in the initial population with incremental stage using GA. There Simultaneous Hyperparameters Tuning and Features Selection. are two other wrapper models for solving feature selection The importance of hyperparameters tuning and feature selec- problem in credit modelling presented by Jadhav et al. tion has urged some researchers to resolve both problems [54] and Wang et al. [55] as discussed in Section 4.1.1(3). simultaneously [35, 41,85].Huang et al.[35]and Zhou et al. [41] developed GA-based wrapper model together with SVM Similarly, both researches formed novel wrapper models with filter information. Jadhav et al. [54] formulated wrapper- and LS-SVM respectively. Both studies have been discussed based SVM, kNN, and NB models with GA, incorporating in Section 4.1.1(3) and their research results have indicated Advances in Operations Research 11 SVM Models 22 2 2 11 1 AB C DE F GH I JK L M N Purpose A: Investigation of Predictive Ability B: Features Selection C: Hyperparameters Tuning D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 1: Summary of SVM models. GA wrapper can solve both problems effectively. Oreski models considered in each experiment for comparison. et al. [85] also solved hyperparameters tuning and feature Table 2 reports the counts of papers categorized by the types of models. Figure 1 illustrates the categorization of all the selection simultaneously but with NN as the main classifier. The research is conducted on a Croatian bank credit applicant SVM models based on the purposes. dataset. The proposed GA-based feature selection wrapped From Table 1, early stage of credit models with SVM are with NN (GA-NN) is benchmarked against other feature basically standalone SVMs for investigation of the predictive selection techniques, i.e., forward selection, information ability. As a result from these investigative experiments that gain, gain ratio, GINI index, correlation, and voting. For validated the effectiveness of SVM, it is soon labelled as hyperparameters tuning, the authors proposed Neural Net- one of the state-of-the-art credit scoring methods. Then, the work Generic Model (NNGM) which employed GA to tune development trend shifted to enhance the original SVM mod- hyperparameters in NN model. The features generated from els, where hybrid models formulation is the most popular the different methods are passed on to NNGM to do classifi- approach that remains active until recent years, with data cation. They also examined the effect of different cutoff points preprocessing and hyperparameters tuning outnumbered the on accuracy and study different misclassification ratios. other research purposes. Ensemble models are the latest research trend in credit scoring due to its ability to improve classification performances. This leads to involvement of ... Summary SVM in two situations; i.e., SVM is one of the benchmark () SVM Models. Developments of SVM models are summa- models against ensembles and SVM is the base classifier used rized in the upcoming tables and gur fi es. Table 1 arranges to form new ensembles. On the other hand, in view of the all the reviewed studies in chronological order to show SVM type in credit modelling, standard SVM has been most the development trend, summarizes the addressed issues, frequently used while SVM variants have apparently lesser research works. In view of the kernel used, linear and RBF and provides additional information on the SVM type with respect to the kernel used as well as details of the other kernel have been widely utilized in this domain. Number of Contributions 12 Advances in Operations Research ff Table 1: Summary of literature for SVM models. Authors SVM type Kernel Other Methods Remarks Standard SVM and Variants Baesens et al. (2003) [8] SVM, LS-SVM linear, RBF - (i) compare with LOGIT, DA, kNN, LP, BN, NN, TREE Van Gestel et al. (2003) [19] LS-SVM RBF - (i) multiclass corporate rating (ii) compare with LR, LOGIT, NN Huang et al. (2004) [20] SVM linear, RBF, polynomial - (i) multiclass corporate rating (ii) compare with NN Li et al. (2006) [21] SVM RBF - (i) compare with NN (ii) study misclassification error Lai et al. (2006) [22] LS-SVM, SVM RBF - (i) multiclass corporate rating (ii) compare with NN, LR, LOGIT Lee et al. (2007) [23] SVM RBF - (i) multiclass corporate rating (ii) compare with NN, DA, CBR Bellotti and Crook (2009) [24] SVM linear, RBF, polynomial - (i) application on large dataset - (ii) compare with LOGIT, LR, DA, kNN - (iii) support vector weights to select significant features Kim and Sohn (2010) [25] SVM RBF - (i) multiclass corporate rating on SME (ii) compare with NN, LOGIT Danenas et al. (2011) [26] SVM linear, RBF, polynomial, - (i) compare between SVMs of dierent libraries (LIBLINEAR, LIBSVM, Laplacian, Pearson, inverse WEKA, LIBCVM) distance, inverse square distance Lessmann et.al (2015) [11] SVM linear, RBF - (i) compare with LOGIT, TREE, ELM, kNN, DA, BN, ensembles (ii) recommendation to use different performance measures Louzada et.al (2016) [12] SVM not mentioned - (i) compare with LR, NN, TREE, DA, LOGIT, FUZZY, BN, SVM, GP, hybrid, ensembles (ii) study class imbalance problem Boughaci and Alkhawaldeh (2018) [18] SVM not mentioned - (i) compare with kNN, BN, NN, TREE, SVM, LOGIT and ensembles Advances in Operations Research 13 fi fi Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Mushava and Murray (2018) [27] SVM RBF - (i) compare with LOGIT, DA, extensions of LOGIT and DA, ensembles Modiﬁed SVM Wang et.al (2005) [28] SVM linear, RBF, polynomial fuzzy membership (i) introduce bilateral weighting error into classification problem (ii) compare with U-FSVM, SVM, LR, LOGIT and NN Harris (2015) [29] SVM linear, RBF k-means cluster (i) reduce computational time (ii) compare with LOGIT, k-means+LOGIT, SVM, k-means+SVM Yang (2017) [30] WSVM RBF, KGPF - (i) dynamic scoring with adaptive kernel (ii) ranking of kernel attributes to solve black box model (iii) compare with LOGIT Li et al. (2017) [31] L2-SVM not mentioned - (i) reject inference (ii) compare with LOGIT, SVM Tian et al. (2018) [32] L2-SVM no kernel - (i) reduce computational time (ii) reject inference and outlier detection (iii) compare with LOGIT, kNN, SVM, SSVM Maldonado et al. (2017) [33] SVM, linear - (i) feature selection 1-norm SVM (ii) acquisition cost into formulation of SVM (iii) application and behavioural scoring (iv) study class imbalance problem (v) compare with SVM (filter and wrapper feature selection) Maldonado et al. (2017) [34] SVM, linear - (i) prot-based feature selection LP-norm SVM (ii) group penalty function included in SVM formulation (iii) compare with LOGIT, SVM (filter, wrapper feature selection) Hybrid SVM Huang et al. (2007) [35] SVM RBF GA (i) hyperparameters tuning, features selection (wrapper approach) (ii)compare with GP, NN, TREE Martens et al. (2007) [36] SVM RBF C4.5, Trepan, (i) rules extraction G-REX (ii) compare with LOGIT, SVM, TREE Zhou and Bai (2008) [37] SVM RBF rough sets (i) features selection (lter approach) (ii) compare with DA, NN, SVM, SVM wrapped by GA Zhang et al. (2008) [38] SVM RBF GP (i) rules extraction (ii) comparewith SVM,GP, LOGIT, NN,TREE Yao (2009) [39] SVM RBF neighbourhood (i) features selection (filter approach) rough set (ii) compare with DA, LOGIT, NN Xu et al. (2009) [40] SVM RBF link analysis (i) features extraction with link relation of applicants (ii) compare with SVM 14 Advances in Operations Research ff Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Zhou et al. (2009) [41] WSVM linear, RBF GA (i) hyperparameters tuning, features selection (wrapper approach) (ii) features weighting (iii) compare with LR, LOGIT, NN, TREE, kNN, Adaboost Zhou et al. (2009) [42] LSSVM RBF DS, GA, GS, DOE (i) hyperparameters tuning (wrapper approach) (ii) compare with LOGIT, kNN, DA, TREE Chen and Li (2010) [43] SVM RBF DA, TREE, (i) features selection (filter approach) rough sets, (ii) compare with SVM F-score Yu et al. (2010) [44] WLS-SVM RBF DS, GA, GS, DOE (i) hyperparameters tuning (wrapper approach) (ii) study class imbalance problem (iii) compare with results from [8, 45] Hens and Tiwari (2011) [46] SVM linear stratified sampling (i) reduce computational time (ii) compare with SVM, NN, GP Danenas and Garsva (2012) [47] SVM from linear PSO, GA (i) model selection LIBLINEAR (ii) hyperparameters tuning (wrapper approach) Chen et al. (2012) [48] SVM RBF k-means cluster (i) reject inference cluster (ii) multiclass problem with different cuto points Chen et al. (2013) [49] SVM RBF ABC (i) hyperparameters tuning (wrapper approach) (ii) compare with SVM tuned with GA and PSO Han et al. (2013) [50] SVM linear orthogonal dimension (i) features extraction with dimension reduction reduction (ii) compare with LOGIT Garsva and Danenas (2014) [51] LS-SVM, linear, RBF, polynomial, PSO, DS, SA (i) model selection SVM from sigmoid (ii) hyperparameters tuning (wrapper approach) LIBLINEAR (iii) study class imbalance problem (iv) compare among all SVM and LS-SVM tuned with PSO, DS, SA Danenas and Garsva (2015) [52] SVM from linear PSO (i) model selection LIBLINEAR (ii) hyperparameters tuning (wrapper approach) (iii) compare with LOGIT, RBF network classifier, SVM tuned with DS Advances in Operations Research 15 fi fi fi Table 1: Continued. Authors SVM type Kernel Other Methods Remarks Hsu et al. (2018) [53] SVM RBF ABC (i) hyperparameters tuning (wrapper approach) (ii) compare with LOGIT, SVM tuned with GS, GA and PSO Jadhav et al. (2018) [54] SVM RBF GA (i) features selection (wrapper approach) (ii) compare with standalone SVM, kNN, NB and their wrappers with GA with standard GA Wang et al. (2018) [55] SVM RBF multiple population GA (i) features selection (wrapper approach) (ii) compare with MPGA-SVM, GA-SVM, SVM Ensemble Model Zhou et al. (2010) [56] LS-SVM RBF fuzzy C-means (i) homogeneous ensemble (ii) compare with ensemble and single classiers Ghodselahi (2011) [57] SVM linear, RBF, polynomial, - (i) homogeneous ensemble sigmoid (ii) compare with ensemble and single classiers Yu et al. (2018) [58] SVM RBF DBN (i) homogeneous ensemble (ii) study class imbalance problem (iii) compare with ensemble and single classifiers Xia et al. (2018) [59] SVM RBF RF, GPC, (i) heterogeneous ensemble XGBoost (ii) compare with ensemble and single classiers 16 Advances in Operations Research Table 2: Type of SVM models. provide details of fitness function and models considered for comparison in each experiment. Then, Table 4 reports the Type Count count of MA models categorized by model type. Figure 2 Standard SVM and its Variants 13 illustrates the research paper counts corresponding to its Modified SVM 7 research purposes. Hybrid SVM 20 Chronological order of the MA models in Table 4 shows Ensembles SVM 4 the modelling trend. Early MA models are standalone MA with investigative purposes. Initiation of MA in credit mod- Total 44 elling is due to the increasing popularity of AI techniques. The development trend in later years is formulation of new hybrid models that persists until recent years. Among the As reported in Table 2, hybrid SVM is the most frequently hybrid models, a majority of the studies are the hybrid MA- adopted approach to construct new SVM credit models. DM where MA techniques act as the assistant of DM to do This is followed by standalone SVM, modified SVM, and the classification task. In view of the MA techniques utilized, ensembles. Louzada et al. [12] review study also revealed the GA is considered as the pioneer as well as the dominant MA same trend where hybrid models are most popular among in credit scoring eld fi since its usage can be observed from the researches. In hybrid models, the method hybridized with earliest study till recent while GP is the second popular MA. SVM acts to assist the classification task without chang- Promising performances with hybrid GA and GP opened up a ing the SVM algorithm. us, Th the construction of hybrid new page for MA in credit modelling where other MA started models is perceived as a direct approach. Standalone SVM to received attentions. comes in second place due to its recognition as the state- Based on the types of models formed with MA as sum- of-the-art technique. Its involvement in recent studies as marized in Table 4, hybrid MA-DM is the obvious dominant, benchmark model further consolidated its recognition in followed by standard MA and hybrid MA-MA. The abundant the credit domain. Modified SVM requires a complicated studies to construct hybrid MA-DM indicates that MA can process to alter the SVM algorithm, thus receiving relatively effectively enhance standalone DM credit models perfor- lesser attention. Ensemble models are new modelling concept mances. Standard MA and hybrid MA-MA have much lesser which have just being researched very lately, leading to the research works. This may be due to the subjectivity of MA least number of contributions. models in formulating the optimization problem to classify Figure 1 provides a quick summary on the research credit customers that pose a difficulty for a general usage. purposes handled in past researches utilizing SVM models. A quick overview of the research purposes with MA According to the counts of papers for each purpose, the top models is illustrated in Figure 2. Features selection is the main research purposes that take up the majority in this review issue dealt that has the most number of contributed works, study are investigation of predictive ability, features selec- followed by rules extraction and hyperparameters tuning. tion, and hyperparameters tuning. Frequent usage of SVM This outcome infers that MA is a useful tool to do data prepro- models in various types of credit datasets and involvement cessing. High comprehensibility is always the crucial criterion in benchmark experiments to investigate models predictive for credit models. Having a number of MA studies that solve ability is an indication of its significance in the domain. this problem with rules extraction is recognition that MA can Data preprocessing with features selection and fine tuning produce transparent credit scorecard. In addition, AI models of SVM hyperparameters in the second place veriefi d the are sensitive to hyperparameters; thus automated tuning with importance of these two to effectively ensure quality clas- MA in place of manual tuning has been under continuous sification of SVM. Therefore, there are another two pieces research. The success of features selection and hyperparame- of works which conduct both tasks simultaneously with the ters tuning in ensuring good performance urged few studies proposed new models. Besides, there are also few researches to conduct both simultaneously. Other than preprocessing which used features extraction to preprocess the dataset data with features selection, there are a few works that use MA instead of features selection. However, these do not solve to do features extraction and discretization. Other minority the main drawbacks of SVM which are black box property research purposes are investigation of predictive ability and and inefficient computation with large instances. Hence, parameter tuning. research on rules extraction and computational efficiency is the remedy corresponding to the two problems. Other credit () Overall Summary. Being two dieff rent AI techniques, both scoring issues confronted using SVM have a minority count methods have been actively researched throughout the years of contributions. They are outlier handling, improvement that unleashed their great potential in the credit scoring of classification performances, reject inferences, dynamic domain. The roles of these two in credit modelling are scoring, and data imbalance. The attempts to solve various illustrated in Figure 3 based on the research purposes. issues with SVM imply its worthiness to be considered as the Features selection is the issue most consistently addressed alternative in credit scoring domain. by both models. For features selection, both SVM and MA () MA Models. MA models development is summarized have almost the same number of works in addressing this in the upcoming tables and figures. Table 3 is the chrono- issue. However, the main difference is that SVM is the tool to logicalsummary of allreviewedstudies with MA to show do classification directly whereas MA indirectly do classifica- the modelling trend, summarize the issues addressed, and tion as it acts as the assistant to the hybridized DM models 𝑇𝑁 𝑇𝑃 Advances in Operations Research 17 fi fi fi ffi ffi 𝑃𝑅 fi Table 3: Summary of literature for MA. Authors MA (category) Other Methods tness function Remarks Standard MA Desai et al. (1997) [60] GA (EA) - no. of misclassication (i) multiclass problem (ii) compare with NN, LOGIT, DA Ong et al. (2005) [61] GP (EA) - mean absolute error (i) rule extraction (ii) compare with LOGIT, NN, TREE, rough sets Finlay (2006) [62] GA (EA) - GINI coecient (i) large credit data application (ii) compare with LOGIT, LR, NN Cai et al. (2009) [63] GA (EA) - error rate (i) include misclassification cost into tness function Huang et al. (2006) [64] 2stage GP (EA) - mean absolute error (i) rules extraction (ii) compare with LOGIT, TREE, kNN, GP Abdou et al. (2009) [65] GP (EA) - (sum of square error) + (i) study effect of misclassification costs (classification error) (ii) compare with prot analysis and weight-of-evidence measure Dong et al. (2009) [66] SA (IB) - accuracy × similarity function × (i) rules extraction ( is a coecient) (ii) compare with DA, kNN, TREE Martens et al. (2010) [67] ACO (SI) - coverage + confidence (i) rules extraction (ii) compare with TREE, SVM, majority vote Yang et al. (2012) [68] CS (SI) - error rates (i) compare with Altman’s Z-score and SVM Uthayakumar et al. (2017) [69] ACO (SI) - + (i) rules extraction (ii) compare with LOGIT, NN, RF, RBF network Hybrid MA-MA Jiang et al. (2011) [70] SA+GA NN 𝛼(1 −) (i) SA optimizes GA (IB+EA) (ii) parameter tuning (iii) compare with standalone NN and GA-optimized NN Aliehyaei and Khan (2014) [71] ACO (SI), GP (EA), - mean absolute error (i) rules extraction by ACO to input to GP ACO+GP (ii) compare with ACO and GP Hybrid MA-DM Fogarty and Ireson (1993) [72] GA (EA) BN accuracy (i) features extraction (ii) large credit data application (iii) compare with default rule, BN, kNN, TREE 𝑇𝑁 𝑇𝑃 18 Advances in Operations Research fi fi 𝐹𝑃 𝑅 𝐹𝑃 𝑇𝑁 𝐹𝑁𝑅 𝛼 𝛼 fi 𝛼 𝜃 𝑁 𝑏 Table 3: Continued. Authors MA (category) Other Methods tness function Remarks Hand and Adams (2000) [73] SA (IB) WOE, weighted likelihood (i) features discretization WOE (ii) compare with LOGIT, DA, two other discretization methods Drezner et al. (2001) [74] TS (IB) LR (i) features selection (wrapper approach) (ii) compare with Altman’s Z-score Lacerda et al. (2005) [75] GA (EA) NN average of individuals with (i) hyperparameters tuning rank of individual (ii) compare with NN, consecutive learning algorithm, SVM from a population Huang et al. (2007) [35] GA (EA) SVM accuracy (i) hyperparameters tuning, feature selection (wrapper approach) (ii) compare with NN, GP, TREE Zhang et al. (2008) [38] GP (EA) SVM accuracy+expected (i) rules extraction misclassification cost (ii) compare with SVM, GP, TREE, LOGIT, NN Liu et al. (2008) [76] GP (EA) DA Information value (i) features extraction (select derived characteristics) (ii) large dataset from nance enterprise (iii) compare with DA Wang et al. (2008) [77] GA (EA) NN 1/MSE (i) parameters tuning (ii) compare with NN Zhang et.al (2008) [78] GA (EA) TREE (1-info entropy)+ (i) features extraction (1 − )(1-info entropy) (ii) compare with TREE, NN, GP, GA-optimized SVM, rough set Jiang (2009) [79] SA (IB) TREE (i) 𝛼⋅ + 𝛽⋅ (i) rules extraction 2 2 (ii) (𝐹− ) +(𝑅− ) (ii) compare with TREE (iii) ⋅ /( + 1) + ⋅ /( + 1) Marinakis et al. (2009) [80] ACO (SI), kNN, 1-NN, not mentioned (i) multiclass problem, features selection (wrapper approach) PSO (SI) weighted kNN (ii) compare with wrapper models of GA and TS with kNN (and variants) Sustersic et al. (2009) [81] GA (EA) NN accuracy>𝜃 and RMSE <𝜃 , (i) feature selection (wrapper approach) , preset threshold (ii) compare with NN, LOGIT (features from PCA) Zhou et al. (2009) [42] GA (EA) SVM error rate (i) hyperparameters tuning (ii) compare with LOGIT, kNN, DA, TREE Zhou et al. (2009) [41] GA (EA) WSVM AUC (i) hyperparameters tuning, feature selection (wrapper approach) (ii) compare with LR, LOGIT, NN, TREE, kNN, Adaboost Marinaki et al. (2010) [82] HBO (SI) kNN, 1-NN, accuracy (i) multiclass problem, features selection (wrapper approach) weighted kNN (ii) compare with wrapper models of GA, PSO, TS, ACO with kNN (and variants) Huang and Wu (2011) [83] GA (EA) kNN, BN, TREE, LR, accuracy (i) feature selection (wrapper approach) SVM, NN, Adaboost, (ii) compare with kNN, BN, TREE, LOGIT, SVM (features from Logitboost, Multiboost lter selection approach) 𝑖𝑗 𝑖𝑗 𝑖𝑗 𝑖𝑗 Advances in Operations Research 19 fi fi fi ff 𝑖𝑠𝑡𝑎𝑛𝑐𝑒 fi 𝑎𝑏𝑠 Table 3: Continued. Authors MA (category) Other Methods tness function Remarks Yu et al. (2011) [44] GA (EA) LS-SVM accuracy (i) hyperparameters tuning (ii) study class imbalance problem (iii) compare with results from [8, 45] Correa and Gonzalez (2011) [84] BPSO (SI), NN AUC (i) hyperparameters tuning GA (EA) (ii) large credit dataset (iii) cost study on application, behavioural and collection scoring (iv) compare with LOGIT, NN, Global Optimum Oreski et al. (2012) [85] GA (EA) NN accuracy (i) hyperparameters tuning, feature selection (wrapper approach) (ii) study eect of misclassication costs (iii) compare their proposed model with features from filter selection approach Danenas and Garsva (2012) [47] GA (EA),PSO (SI) SVM TPR (i) model selection (ii) hyperparameters tuning Chen et al. (2013) [49] ABC (SI) SVM not mentioned (i) multiclass problem (ii) hyperparameters tuning (iii) compare with SVM (tuned with GA and PSO) Garsva and Danenas (2014) [51] PSO (SI) SVM (i) TPR, (ii) accuracy (i) model selection (ii) hyperparameters tuning (iii) study class imbalance problem (iv) compare among all SVM, LS-SVM tuned with PSO, DS, SA Oreski and Oreski (2014) [86] GA (EA) NN accuracy (i) feature selection (wrapper approach) (ii) compare with standard wrapper-based NN with GA Danenas and Garsva (2015) [52] PSO (SI) SVM TPR (i) model selection (ii) hyperparameters tuning (iii) compare with LOGIT, RBF network Wang et al. (2010) [87] TS (IB) NN, SVM, LOGIT entropy (i) features selection (filter approach) (ii) compare with NN, SVM, LOGIT with full features Wang et al. (2012) [88] SS (SI) NN, TREE, LOGIT entropy (i) features selection (filter approach) (ii) compare with NN, TREE, LOGIT with full features Waad et al. (2014) [89] GA (EA) LOGIT, SVM, TREE ∑ 𝑤𝑒𝑔ℎ𝑡 ×𝐷(, ) (i) feature selection (lter approach) (ii) compare with LOGIT, SVM,TREE (features from other lter selection and rank aggregation methods) Hsu et al. (2018) [53] ABC (SI) SVM 1/(1 + (𝑥 )),if (𝑥 )≥ 0 (i) multiclass problem 1+ ( (𝑥 )),if (𝑥 )< 0 (ii) hyperparameters tuning (iii) compare with LOGIT and SVM (tuned with GS, GA, PSO) Jadhav et al. (2018) [54] GA (EA) SVM, kNN, NB accuracy (i) features selection (wrapper approach) (ii) compare with standalone SVM, kNN, NB and their wrappers with GA Wang et al. (2018) [55] multiple population SVM accuracy (i) features selection (wrapper approach) GA (EA) (ii) compare with MPGA-SVM, GA-SVM, SVM Krishnaveni et al. (2018) [90] HS (SI) 1-NN accuracy (i) features selection (wrapper approach) (ii) computational time reduction (iii) compare with standalone SVM, TREE, kNN, NB, NN and their wrappers with GA and PSO 20 Advances in Operations Research MA Models 00 0 0 0 0 AB C D E F G H I J K L M N Purpose A: Investigation of Predictive Ability B: Features Selection C: Hyperparameters Tuning D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 2: Summary of MA models. Table 4: Type of MA models. as the next with seven contributions where all the seven are the collaboration of MA with SVM. u Th s, MA can be viewed Type Count as a recommended tool to tune SVM hyperparameters. Standard MA 10 Simultaneous features selection and hyperparameters tuning Hybrid MA-MA 2 result in a total of three studies, with two of them being hybrid Hybrid MA-DM 31 MA and SVM. The rest of the research purposes have shown dominance Total 43 in either SVM or MA. Feature discretization has only been attempted by MA while reject inferences, improvement of performances, computational ecffi iency, dynamic scoring, which are responsible for the classification. Investigation of imbalance datasets, and outlier handling have only been predictive ability comes as the second top research purposes. addressed using SVM. MA models have taken into account SVM models have much greater number of researches as much lesser issues as compared to SVM since MA have been compared to GA. This indicates that SVM is already a focused more to solve features selection and rules extraction. recognized credit scoring model as it is frequently included in comparative studies and attempted in different specific domains. MA has lesser works under this purpose as it is .. Assessment Procedures. In order to assess credit models performance, they are usually compared with other standard seldom involved in benchmark experiments that may be due to its subjectivity in model building. Then, rules extraction credit models applied on the selected credit datasets and comes in the third largest number of researches, with MA evaluated with appropriate performance measures. us, Th models more than SVM. This shows the great ability of MA to the assessment procedures are categorized into benchmark develop transparent model. Hyperparameters tuning comes experiments, performance measures, and credit datasets. Number of Contributions Advances in Operations Research 21 Overall Categorization AB C D E F G H I J K L M N Purpose SVM Models A: Investigation of Predictive Ability B: Features Selection MA Models C: Hyperparameters Tuning Both D: Reject Inferences E: Improvement of Performances F: Computational Eﬃciency G: Rules Extraction H: Features Extraction I: Simultaneous J: Outlier Handling K: Dynamic Scoring L: Data Imbalance M: Features Discretization N: Parameters Tuning Figure 3: Overallsummary for bothmodels. ... Benchmark Experiments. Benchmark experiments comparative studies to include sufficient huge number of include comparisons of the proposed models with other methods as benchmark to be able to provide sufficient standard credit models. Tables 1 and 2 provided brief information to serve as a guideline for future research. Since summary on the models considered for comparison in every ensembles are formulated from assembly of a number of experiment. Detailed experiment setup shall be referred to standalone classifiers, authors that proposed new ensembles the original paper. Table 5 presents the categorization of the usually have to compare new ensembles not only with stan- type of benchmark experiment carried out with SVM and dalone classifiers but also with standard available ensembles. MA models. Small scale benchmark is the most common approach which can be further broken down into four main parts, As reported in Table 5, it can be seen that inclusion of model comparison has been a standard approach to make i.e., comparison only with the counterpart techniques of conclusion on the proposed models. Most of the studies the proposed model, comparison only with either statistical adopted internal benchmark approach to make comparison or AI techniques, and comparison with both statistical and with other models based on the same experimental setup AI techniques. For both SVM and MA models, the most for the credit data. Only Yu et al. [44] adopted external preferred small scale benchmark is comparison with both benchmark approach to compare their proposed models with statistical and AI techniques. Besides, LOGIT and NN are other models. Chen et al. [48] and Cai et al. [63] are the the most frequently involved statistical and AI techniques, rare cases in the literature which do not benchmark their respectively. proposed models with others. Large scale benchmark is a rare approach with only several studies presented this in the past. It can be noticed ... Performance Measures. There are four main types of that research work with large scale benchmark are those that performance measures being used to make inference on the conducted comparative studies and formulate new ensemble models performances. Cutoff-dependent measures are those models for performance improvement. It is necessary for directly obtained or computed from the confusion matrix, Number of Contributions 22 Advances in Operations Research Table 5: Benchmark experiments. Benchmark Type SVM Models MA Models Large Scale [8,11,12,18,27,56–59] - SmallScale Counterpart [33,40,43] [70,71,79,80,82,83,85,86,89] [77, 87, 88] [47,49,51,54,55] Statistical [30,50] [65,73,74,76] AI [20, 21, 46] [67, 72, 75, 78] [35] Statistical&AI [22–25,29,31,32,34,36] [60–62,64,66,69,81,84,90] [19, 28, 37, 39] [68] [38,41,42,52,53] External [44] None [48] [63] Table 6: Performance measures. Type SVM Models MA Models Cutoff-dependent [12,19–21,23,25,26,28,30,36,37,39,40,46,48] [60,61,63–72,74–79,81–83,85,86,89,90] [35,38,42,44,47,49,51–53,55] Cutoff-independent [24, 33, 34] [84] [41] Mixture [8,11,18,22,27,29,31,32,43,50,56,57,59] [1,62,87,88] [54] Business-oriented [33, 34] [65, 72, 76, 86] Others [58] - where the cutoff point is often problem-dependent. Cutoff- to provide different perspective of interpretation. In one of independent measures are those computed to determine the latest and largest comparative studies by Lessmann et the discriminating ability of the model. Mixture indicates al. [11], they have recommended the use of more cutoff- the usage of both cutoff-dependent and cutoff-independent independent measures with the aim to explain models in measures while business-oriented measures are those com- different perspective. Therefore, their recommendations have puted with the misclassica fi tion costs. Table 6 summarizes been adopted in recent research [59]. Protfi and loss is oen ft the final aim for financial institu- the performance measures in the literature of SVM and MA models. tions as credit scoring is treated as a risk aversion tool. Only a minority of studies explained their models with business- Cutoff-dependent measures are the most popular indica- oriented measures. They are Expected Misclassfication Costs tor utilized by researchers, especially accuracy or its counter- (EMC) [65, 86], profitability [72], and self-den fi ed profit or part error rate that measures the number of correct classifi- cost measures [33, 34, 76]. Note that all researches utilizing cations in a straightforward manner. Among them, several business-oriented measures also included cutoff-dependent studies ofSVM models [21, 22,28, 30–32, 40,57] and MA or cutoff-independent measures to evaluate their models. models [70, 77, 81, 83] are interested in the misclassifications Only Yu et al. [58] applied weighed accuracy that involved which is believed to pose higher risk for financial institutions; imbalance rate, revenue and cost to compute a new version of thus type I and type II errors are included. Although cutoff- accuracy. dependent measures are direct in presenting performances of classifier, a main drawback has been denoted by [4, 11, 24], where researchers often do not address the cutoff point used. ... Model Evaluation. Researchers make inferences and u Th s, there are a few studies believed cutoff-independent conclusions based on the reported performance measures. performance measures are sucffi ient to serve as guideline for The conclusions are often to show that the proposed models model performances as reported in Table 6, with Area Under are competitive or outperform the other standard credit mod- Receiver Operating Characteristics Curve (AUC) being most els based on the numerical results. In addition, some studies popularly used. conducted statistical tests to show evident improvement of Cutoff-dependent measures from confusion matrix and the proposed models. cutoff-independent measures of model discriminating ability There are several studies of SVM models [29, 36, 46, 59], are both informative for decision makers. Hence, a few stud- MA models [60,69,71, 78,84],and joint MA-SVM [41, ies have included both types of measures in their experiment 47, 51, 52] that do not have numerical outperformance of Advances in Operations Research 23 Table 7: Credit datasets. Type SVM Models MA Models Specific CR: BankScope [19], England [22], CR:UK[80,82] Taiwan/US [20], Korea [23, 25], BP:Simulated data [68] Taiwan [21], credit card [24] AS: Southeastern US [60], UK [62], BP: US [26] Egyptian [65], Compustat [74], AS: German [30], LC P2P [31], Shenzhen [70, 77], Croatian [85], LC/Huijin P2P [32], China [37, 48] UK/simulated [1] BS: Chilean [27, 33, 34] AS/BS/CS:local bank[84] BP:US [47, 52] CR:Compustat [49] General AS:[12,39, 43,57] AS:[61,63,64,66,70,71,75,78,83,87,88,90] AS:[35,38,41,42,44,51,55] Specific & AS:UKCDROM [28],Barbados [29], AS:Croatian[86], General Benelux [36] Tunisian/home equity [89] BP:ANALCAT [69] AS/BP/CR:SME/ BankScope [67] AS:Taiwan [54] CR:Compustat [53] Large scale AS: [8, 11] - comparison AS/BP:[18] their proposed models with the compared models. However, bankruptcy prediction. Behavioural scoring has received very they have contributed to another aspect, i.e., outperformance less attention. For general studies, there are three datasets: in timing [29, 46], proper handling of imbalance [59], German, Australian, and Japanese datasets available in the transparent rules [36, 69], and presentation of competitive UCI repository, and all of them are application scoring data. new methods [41,47,51,52,60,71, 78,84]. Among them, German and Australian have been widely Some comparative studies [8, 11, 12] identified certain involved in research. There are also several specicfi studies that included UCI datasets as evaluation purpose, and sim- methods to have best performance and recommended for future research but they did not penalize the use of other ilarly, German and Australian are still the dominant usage techniques since the reported performance is still very com- among researchers. It can be noticed that almost all of the petitive, whereas comparative studies by [18, 26] did not result studies utilized small numbers of datasets for investigation, in outperformance of any methods among the compared with only comparative studies involving large amount of models. The rest of the research articles reviewed in this study datasets. have reported their proposed models having better results than the others. .. Results from Literature. German (G) and Australian Instead of solely depending on numerical improvement (A) datasets have been frequently included in credit scoring to detect outperformance, a minority of the studies [8, 11, 12, domain using SVM and MA models. This section compiles 19,21,23,29,34–36,43,50, 53, 55,56, 59,60, 67,89] have those contributed works based on the research purposes to utilized statistical tests for a more concrete support of their provide information on which model type with the handled results. Most commonly used statistical tests are paired t-test, issues have shown good performance on both datasets. Note Mc Nemar test, Wilcoxon signed rank test, and Friedman that the compilation is only based on accuracy as it is the test. common reported measures. For researches that reported only error rate, it is converted to accuracy. Researches that ... Credit Datasets. SVM and MA credit models have been have utilized these datasets but do not report accuracy are applied on different types of credit datasets, i.e., application not included here. Then, for usage of more than one standard scoring (AS), behavioural scoring (BS), collection scoring SVM and its variants, as well as studies that formulated more (CS), corporate rating (CR), and bankruptcy prediction (BP). than one new model, only the best performing results are There have been two main types of studies which are specific recorded. The results are compiled in Table 8. studies particularly on a country’s financial institutions and The computed mean in Table 8 is the overall general general studies using publicly available datasets from the UCI performance of the models on both datasets. The high value repository [94]. Table 7 summarizes the credit datasets usage. of mean accuracy is an indication of good performance For specific studies, they have focused on particular from SVM and MA models in the literature. The computed standard deviation in Table 8 is viewed as an indicator to financial institutions, with application scoring being the mostly utilized data type, followed by corporate rating and generalize the range of the accuracy that is believed to take 24 Advances in Operations Research Table 8: Results from literature. Purpose Model Type Authors G A Investigation of predictive standard SVM and Baesens et al. [8] 74.30 ability its variants Lessmann et al. [11] 75.30 86.00 Boughaci and Alkhawaldeh [18] 69.90 80.70 standalone MA Cai et al. [63] . - Computational efficiency modified SVM Harris [29] . - hybrid SVM Hens and Tiwari [46] 75.08 85.98 Improvement of classfication ensembles Zhou et al. [56] . - performance Ghodselahi [57] . - Xia et al. [59] . 86.29 Rules extraction hybrid SVM Martens et al. [36] - 85.10 standalone MA Ong et al. [61] 77.34 . Huang et al. [64] Dong et al. [66] 72.90 - Martens et al. [67] . - Uthayakumar et al. [69] - 86.37 hybrid MA-MA Aliehyaei and Khan [71] 70.70 84.30 hybrid MA-DM Zhang et al. [38] Jiang et al. [79] 73.10 - Features extraction hybrid SVM Xu et al. [40] - Han et al. [50] 75.00 - hybrid MA-DM Zhang et al. [78] 77.76 Features selection hybrid SVM Yao [39] 76.60 87.52 Chen and Li [43] 76.70 86.52 hybrid MA-DM Jadhav et al. [54] . Wang et al. [55] . 86.96 Huang and Wu [83] - 87.54 Oreski and Oreski [86] . Krishnaveni et al. [90] . Wang et al. [88] - 88.90 Hyperparameters tuning hybrid MA-DM Zhou et al. [42] 77.10 86.96 Yu et al. [44] 78.46 90.63 Garsva and Danenas [51] . . Hsu et al. [53] . Lacerda et al. [75] - 86.05 Simultaneous features hybrid MA-DM Huang et al. [35] 77.92 86.90 selection & hyperparameters tuning mean 77.56 87.75 standard deviation 3.35 2.64 into account the variation in different experimental setup for of data partitioning, i.e., k-fold cross validation (k-fold), every model development. holdout validation (holdout), repeated k-fold cross validation The general mean and standard deviation in Table 8 (rep k-fold), and repeated holdout validation (rep holdout). provide an overview on the models performances through Table 9 shows that k-fold and holdout are the most compilation from all studies regardless of the data splitting adapted data splitting strategies while rep k-fold and rep strategies adopted. Data splitting methods are influential holdout are less popular, which may be due to the high computational effort required for both rep k-fold and rep on the experiments end results. The general compilation in Table 8 is further detailed in Table 9 to take into consideration holdout. Instead of a general mean and standard deviation the effect of different data splitting strategies. The detailed as in Table 8, the specific mean and standard deviation for analysis is conducted by categorizing the studies (except Jiang each category are reported in Table 9, which is believed to [79] as no data splitting mentioned) into four main types be less biased and more reliable metrics as being grouped Advances in Operations Research 25 Table 9: Results categorized with data splitting methods. Data Split Authors G A k-fold cv Dong et al. [66] 72.90 - Uthayakumar et al. [69] - 86.37 Zhang et al. [38] Xu et al. [40] - Han et al. [50] 75.00 - Zhang et al. [78] 77.76 Yao [39] 76.60 87.52 Chen and Li [43] 76.70 86.52 Jadhav et al. [54] . Wang et al. [55] . 86.96 Huang and Wu [83] - 87.54 Oreski and Oreski [86] . Wang et al. [88] - . Zhou et al. [42] 77.10 86.96 Yu et al. [44] . Hsu et al. [53] . Huang et al. [35] 77.92 86.90 mean 78.20 88.56 standard deviation 2.93 1.92 holdout Baesens et al. [8] 74.30 Cai et al. [63] . - Boughaci and Alkhawaldeh [18] 69.90 80.70 Harris [29] . - Zhou et al. [56] . - Ghodselahi [57] . - Martens et al. [36] - 85.10 Martens et al. [67] . - Aliehyaei and Khan [71] 70.70 84.30 Garsva and Danenas [51] . . mean 77.07 85.32 standard deviation 4.47 3.20 rep k-fold Lessmann et al. [11] 75.30 86.00 Xia et al. [59] . 86.29 Krishnaveni et al. [90] . mean 78.01 88.60 standard deviation 2.56 4.25 rep holdout Ong et al. [61] 77.34 . Huang et al. [64] Lacerda et al. [75] - 86.05 mean 78.42 87.83 standard deviation 1.52 1.61 homogeneously according to the data splitting methods. For may be due to the usage of SVM without the hyperparameters all the categories, across both datasets, the mean accuracies tuning procedure. Besides, another high standard deviation are relatively high, showing the effectiveness of building new comes from the rep k-fold category only in Australian dataset. credit models with both SVM and MA. This high deviation is from a higher accuracy results from However, for the holdout category, it can be noticed that Krishnaveni et al. [90]. This may be due to the nature of this the standard deviation is much higher than the other cate- dataset that suits well to the proposed method. gories. Observing the holdout category, the high deviation is To provide information on which models are more deduced to be contributed from an unusual lower accuracy effective than the others, models with accuracy higher than from Boughaci and Alkhawaldeh [18]. This lower accuracy the mean is viewed as having greater potential to deal with 26 Advances in Operations Research thecredit scoring problem and would berecommendable terms shall be considered. Danenas et al. [26] provided for future research. Higher potential models are reported in information on the various choice of available kernels for italic (see Table 9), and they are compared with the respective SVM; thus instead of the common linear and RBF, other mean accuracy in every category. All accuracies written in kernels shall be investigated. Modified SVM in the litera- italic in Table 8 correspond to the higher potential credit ture involved modifications on the hyperplane optimization models obtained from Table 9. Generally, rules extraction, problem. Introduction of new kernels can be perceived as hyperparameters tuning and features selection are observed apossiblefutureworkfor modiefi d SVM category, due to as effective measures to be undertaken for well-performed the exib fl ility of SVM itself, where any kernels that follow credit models. This aligned with our previous discussion in the Karush-Kuhn Tucker conditions can be used in SVM. Section 4.1.3. Hybrid approach is the majority for features selection, only For features selection and hyperparameters tuning, both two works [33, 34] adopted cost and protfi view to handle this issue. Future features selection can consider incorporation of SVM and MA appears to be the crucial tools for credit modelling, where SVM is the main classifier and MA is the cost and protfi in model building process as protfi scoring is assistant to be fused with SVM to carry out the targeted suggested as the main future trend in [2, 11]. Besides, other tasks. For rules extraction, MA is the best choice to build research purposes that have few publications are valuable transparent credit scoring models, either in standalone MA future directions to be considered. or hybridized with other black box DM. Among the dieff rent variants of SVM, standard SVM is most commonly adopted ... MA Models. Hybrid MA-DM has been the leading by researchers for new model development. On the other model type throughout the years, and it is believed that hand, among the different MA, GA and GP appear to be the this trend will persist in the future since hybrid models dominant tool, while recent trend has shifted to other types formulation is considered as a direct way to propose new of MA for new model building. models yet able to improve the standalone DM. EA, i.e., Inclusion of German and Australian datasets into large GA and GP, are the most popular MA to be applied in scale benchmark comparative studies of credit models is an credit scoring domain. It can be observed that MA (other indication of the status of these two becoming the standard than EA family) have increasing publications in recent years. credit datasets in this domain. This leads to frequent usage of Since there are various choices for MA in each family, both datasets throughout the years until recent. those that have not yet been investigated in credit scoring domain, such as, Social Cognitive Optimization, Bat Algo- rithm, Local Search, Variable Neighbourhood Search, etc., 5. Conclusions and Future Directions should be considered for new model development. Although This study presented a literature review of credit scoring standard MA is seldom used for model development, the models formulated with SVM and MA. From the two aspects, transparent property, and efl xibility of MA to be tailored for model type with issues addressed and assessment procedures, to solve for specific credit data is a plus point to use MA in together with past results of models applied on UCI credit credit modelling. In addition, business-oriented model is an datasets, hybrid approach is identified as the state-of-the- important prospect for decision makers. Hence, the flexibility art modelling approach for both techniques in the credit of MA shall be employed to incorporate costs and benefits scoring domain. SVM and MA have been the current trend into model formulation. In view of computational efficiency, for credit modelling with SVM being the main classifier and MA is very adaptable where the operators can be carefully MA being the assistant tool for model enhancement with modified or parallelized to achieve high efficiency. hybrid approach. Aligned with the views from [1, 11, 12] that concluded sophisticated models are the future trend, both ... Overall. Features selection, hyperparameters tuning SVM and MA will also have the similar future modelling and rules extraction are concluded as the few popular issues trend. Various issues and assessment procedures in the addressed with SVM and MA models, with higher tendency literature are concluded to point out some future directions to deal with hybrid methods. Results from past experiments as follows. with German and Australian datasets further validate these three as the trend for both models in credit scoring. As compared to SVM, MA is mostly limited to deal with these .. Model Type with Issues Addressed three issues, while SVM dealt with wider varieties of issues. u Th s, instead of limiting MA to build models based on the ... SVM Models. SVM is an ongoing active research in credit scoring, where the future development trend is per- few popular purposes, other issues that have been attempted ceived as building new SVM models based on hybrid and as in SVM models shall be considered since flexibility of MA ensembles method. Several directions are pointed out for to make it tailored to specific problem is always possible. possible future works. There are various SVM variants avail- MA is mostly incorporated with SVM to form hybrid. Other able that are able to account for more efl xible classicfi ation. AI models that have not yet been attempted for the same A benchmark experiment comparing standard SVM and its research purpose are worth being investigated. Besides, MA variants can be conducted to give insight on which SVM is advantageous in rules extraction while SVM is a black box model. These two can be collaborated to formulate type is more adaptable in this domain. When building new models, SVM variants that include different regularization a transparent yet competitive model. Lastly, formation of Advances in Operations Research 27 business-oriented SVM models with hybrid approach is a Acknowledgments more direct task than to modify the algorithm in SVM. This research is funded by Geran Putra-Inisiatif Putra The adaptable property of MA is a good prospect to join it Siswazah (GP-IPS/2018/9646000) supported by Universiti with SVM, where MA is responsible to take account of cost Putra Malaysia (UPM). and protfi in the modelling procedure. Ensembles modelling is also a future research direction as it has just received attention in credit scoring domain lately. Business-oriented References ensemble with MA-tuned SVM model is a recommendable [1] D. J. Hand and W. E. Henley, “Statistical classification methods future research. in consumer credit scoring: a review,” Journal of the Royal Statistical Society: Series A (Statistics in Society),vol.160, no.3, .. Assessment Procedures. The whole assessment proce- pp. 523–541, 1997. dures in each experiment are a subjective decision based on [2] L.C.Thomas,“A survey ofcreditand behavioural scoring: the experimental setup of the researchers. Internal small scale forecasting financial risk of lending to consumers,” International benchmark experiments including both statistical and AI Journal of Forecasting, vol.16,no. 2,pp.149–172,2000. techniques, reporting the numerical improvement of cutoff- [3] S. M. Sadatrasoul, M. R. Gholamian, M. Siami, and Z. Hajimo- dependent measures, is the leading trend in credit scoring hammadi, “Credit scoring in banks and financial institutions via domain. Internal small scale benchmark is the recommended data mining techniques: a literature review,” Journal of AI and experimental setup for future experiments due to two main Data Mining ,vol.1,no.2,pp.119–129, 2013. considerations. First, newly developed model has to be [4] N. Martin, “Assessing scorecard performance: a literature compared with standard credit scoring models from both sta- review and classification,” Expert SystemswithApplications,vol. tistical and AI families as a concrete support of the usefulness 40, no. 16, pp. 6340–6350, 2013. of the proposed models but large scale comparison is not [5] H.A.Alaka,L.O.Oyedele,H.A.Owolabi et al.,“Systematic necessary to avoid loss of focus. Second, external benchmark review of bankruptcy prediction models: Towards a framework for tool selection,” Expert Systems with Applications,vol. 94, pp. comparing with results from other researches can be consid- 164–184, 2018. ered but have to be used very cautiously because different [6] J. Sun,H. Li,Q.-H. Huang,and K.-Y.He, “Predicting financial experiments have different design that may be difficult to distress and corporate failure: a review from the state-of-the- have an apple-to-apple comparison. It is also recommended art definitions, modeling, sampling, and featuring approaches,” to report both cutoff-dependent and independent measures Knowledge-Based Systems,vol.57,pp.41–56,2014. together and disclose the assumed cutoff point when using [7] S. Moro, P. Cortez, and P. Rita, “An automated literature analysis cutoff-dependent measures. There have been varieties of on data mining applications to credit risk assessment,” in performance measures being recommended in Lessmann et Artiﬁcial Intelligence in Financial Markets, pp. 161–177, Palgrave al. [11], butthechoiceshall becarefully madeinstead of using Macmillan, London, 2016. as many as possible of these measures. This is because every [8] B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, J. Suykens, measure evaluates models in different approach; researchers and J. Vanthienen, “Benchmarking state-of-the-art classifica- shall choose based on what is the most important information tion algorithms for credit scoring,” Journal of the Operational needed to be conveyed as it is not always “the more the Research Society,vol.54,no. 6,pp.627–635,2003. better” [4]. Business-oriented measures can be viewed as [9] J.N.Crook,D.B. Edelman, and L. C.Thomas, “Recent a future trend to evaluate models, provided that the credit developments in consumer credit risk assessment,” European dataset should have information for the profit and costs. Journal of Operational Research,vol.183, no.3,pp. 1447–1465, Although numerical improvement is considered a valuable 2007. contribution, it is recommended to include statistical tests [10] A.Lahsasna,R.N.Ainon,andT.Y.Wah,“Creditscoringmodels for further validation on the statistical significance of model using soft computing methods: a survey,” International Arab Journal of Information Technolog,vol.7,no.2,pp. 115–123,2010. performances. The trend of focusing in application scoring [11] S. Lessmann, B. Baesens, H.-V. Seow, and L. C. o Th mas, especially with public UCI datasets may be due to the “Benchmarking state-of-the-art classification algorithms for confidentiality issue of credit scoring domain. Recently, there credit scoring: an update of research,” European Journal of aresomedatasets available from data mining competitions Operational Research, vol.247,no. 1,pp. 124–136,2015. and also Lending Club (LC) online P2P lending platform. [12] F. Louzada, A. Ara, and G. B. Fernandes, “Classification meth- u Th s, other types of dataset shall be involved to check models ods applied to credit scoring: systematic review and overall robustness across different types of data. Besides, LC datasets comparison,” Surveys in Operations Research and Management provided more information that can be considered to build Science,vol.21,no. 2,pp. 117–134, 2016. models for behavioural scoring. For specific research using [13] A. I. Marques, V. Garc´ıa, and J. S. Sanchez ´ , “A literature review private datasets, protfi scoring may be a future research on the application of evolutionary computing to credit scoring,” trend and it is recommended to include UCI datasets as a Journal of the Operational Research Society,vol.64, no.9,pp. benchmark due to their general usage in the literature. 1384–1399, 2013. [14] D. Durand, Risk Elements in Consumer Instalment Financing, NBER Books, 1941. Conflicts of Interest [15] E. I. Altman, “Financial ratios, discriminant analysis and the The authors declare that there are no conflicts of interest prediction of corporate bankruptcy,” e Journal of Finance,vol. regarding the publication of this paper. 23, no. 4, pp. 589–609, 1968. 28 Advances in Operations Research [16] J. A. Ohlson, “Financial ratios and the probabilistic prediction [34] S. Maldonado, C. Bravo, J. Lop ´ ez, and J. Per ´ ez, “Integrated of bankruptcy,” Journal of Accounting Research, vol.18,no.1,pp. framework for profit-based feature selection and SVM classifi- 109–131, 1980. cation in credit scoring,” Decision Support Systems,vol.104, pp. 113–121, 2017. [17] P. Kolesar and J. L. Showers, “A robust credit screening model using categorical data,” Management Science,vol. 31,no. 2,pp. [35] C.L. Huang, M.C. Chen,and C.J.Wang, “Credit scoring with 123–133, 1985. a data mining approach based on support vector machines,” Expert Systems with Applications, vol.33, no.4, pp. 847–856, [18] D. Boughaci and A. A. Alkhawaldeh, “Appropriate machine learning techniques for credit scoring and bankruptcy predic- tion in banking and finance: a comparative study,” Risk and [36] D. Martens, B. Baesens, T. Van Gestel, and J. Vanthienen, Decision Analysis,pp. 1–10,2018. “Comprehensible credit scoring models using rule extraction from support vector machines,” European Journal of Opera- [19] I. T. Van Gestel, B. Baesens, I. J. Garcia, and P. Van Dijcke, “A tional Research, vol.183,no.3,pp.1466–1476, 2007. support vector machine approach to credit scoring,” Bank- en Financiewezen,vol.2,pp.73–82,2003. [37] J. Zhou and T. Bai, “Credit risk assessment using rough set theory and GA-based SVM,” in Proceedings of the rd [20] Z. Huang, H. Chen,C.-J.Hsu, W.-H.Chen, and S. Wu,“Credit International Conference on Grid and Pervasive Computing rating analysis with support vector machines and neural net- Symposia/Workshops (GPC), pp. 320–325, IEEE, 2008. works: a market comparative study,” Decision Support Systems, vol.37,no.4,pp.543–558, 2004. [38] D. Zhang, M. Hifi, Q. Chen,and W. Ye,“A hybrid creditscoring model based on genetic programming and support vector [21] S.-T. Li, W. Shiue, and M.-H. Huang, “eTh evaluation of machines,” in Proceedings of the th International Conference on consumer loans using support vector machines,” Expert Systems Natural Computation (ICNC),vol.7,pp. 8–12,2008. with Applications, vol. 30, no. 4, pp. 772–782, 2006. [39] P. Yao, “Hybrid classifier using neighborhood rough set and [22] K. Lai, L. Yu, L. Zhou, and S. Wang, “Credit risk evaluation with SVM for credit scoring,” in Proceedings of the International least square support vector machine,”RoughSetsandKnowledge Technology, pp. 490–495, 2006. Conference on Business Intelligence and Financial Engineering (BIFE), pp. 138–142, IEEE, 2009. [23] Y.-C. Lee, “Application of support vector machines to corporate [40] X. Xu,C.Zhou, and Z.Wang, “Creditscoring algorithm based credit rating prediction,” Expert Systems with Applications,vol. 33,no. 1, pp.67–74, 2007. on link analysis ranking with support vector machine,” Expert Systems with Applications, vol. 36, no. 2, pp. 2625–2632, 2009. [24] T. Bellotti and J. Crook, “Support vector machines for credit scoring and discovery of significant features,” Expert Systems [41] L.Zhou, K. K.Lai,and J. Yen,“Credit scoring models with AUC with Applications, vol.36,no. 2,pp.3302–3308, 2009. maximization based on weighted SVM,” International Journal of Information Technology & Decision Making, vol.8,no. 4,pp. [25] H. S. Kim and S. Y. Sohn, “Support vector machines for default 677–696, 2009. prediction of SMEs based on technology credit,” European Journal of Operational Research, vol.201,no. 3,pp. 838–846, [42] L.Zhou,K. K.Lai, and L. Yu,“Credit scoring using support 2010. vector machines with direct search for parameters selection,” So Computing-A Fusion of Foundations, Methodologies and [26] P. Danenas, G. Garsva, and S. Gudas, “Credit risk evaluation Applications, vol.13,no. 2,pp.149–155, 2009. model development using support vector based classifiers,” Procedia Computer Science,vol.4,pp. 1699–1707, 2011. [43] F.-L. Chen and F.-C. Li, “Combination of feature selection approaches with SVM in credit scoring,” Expert Systems with [27] J. Mushava and M. Murray, “An experimental comparison of Applications, vol.37,no. 7,pp. 4902–4909, 2010. classification techniques in debt recoveries scoring: evidence from South Africa’s unsecured lending market,” Expert Systems [44] L. Yu, X.Yao,S.Wang, and K.K.Lai,“Credit riskevaluation with Applications, vol. 111, pp. 35–50, 2018. using a weighted least squares SVM classifier with design of experiment for parameter selection,” Expert Systems with [28] Y. Wang, S.Wang,and K.K.Lai, “A new fuzzy support vector Applications,vol. 38, no. 12, pp.15392–15399,2011. machine to evaluate credit risk,” IEEE Transactions on Fuzzy Systems, vol.13,no. 6,pp. 820–831, 2005. [45] D. West, “Neural network credit scoring models,” Computers & Operations Research, vol. 27, no. 11-12, pp. 1131–1152, 2000. [29] T. Harris, “Credit scoring using the clustered support vector machine,” Expert SystemswithApplications,vol.42, no. 2, pp. [46] A. B. Hens and M. K. Tiwari, “Computational time reduction for 741–750, 2015. credit scoring: an integrated approach based on support vector machine and stratified sampling method,” Expert Systems with [30] Y. Yang, “Adaptive credit scoring with kernel learning methods,” Applications, vol.39,no.8,pp.6774–6781,2012. European Journal of Operational Research, vol.183,no. 3, pp. 1521–1536, 2007. [47] P. Danenas and G. Garsva, “Credit risk evaluation modeling [31] Z. Li, Y. Tian, K. Li, F. Zhou, and W. Yang, “Reject inference in using evolutionary linear SVM classifiers and sliding window approach,” Procedia Computer Science, vol.9,pp.1324–1333, credit scoring using semi-supervised support vector machines,” Expert SystemswithApplications, vol.74,pp.105–114,2017. [32] Y. Tian, Z. Yong, and J. Luo, “A new approach for reject [48] W. Chen, G.Xiang,Y.Liu,and K.Wang,“Credit riskevaluation by hybrid data mining technique,” Systems Engineering Procedia, inference in credit scoring using kernel-free fuzzy quadratic surface support vector machines,” Applied So Computing,vol. vol. 3, pp. 194–200, 2012. 73, pp. 96–105, 2018. [49] M.Y.Chen,C.C.Chen,andJ.Y.Liu,“Creditratinganalysiswith [33] S. Maldonado, J. Per ´ ez, and C. Bravo, “Cost-based feature support vector machines and artificial bee colony algorithm,” in selection for support vector machines: an application in credit Proceedings of the International Conference on Industrial, Engi- scoring,” European Journal of Operational Research,vol. 261, no. neering and Other Applications of Applied Intelligent Systems,pp. 2,pp.656–665,2017. 528–534, Springer, Berlin, Heidelberg, Germany, 2013. Advances in Operations Research 29 [50] L. Han, L. Han, and H. Zhao, “Orthogonal support vector [68] X.-S. Yang, S. Deb, M. Karamanoglu, and X. He, “Cuckoo search machine for credit scoring,” Engineering Applications of Artiﬁ- for business optimization applications,” in Proceedings of the cial Intelligence, vol.26, no.2,pp.848–862, 2013. National Conference on Computing and Communication Systems (NCCCS), pp. 1–5, IEEE, 2012. [51] G. Garsva and P. Danenas, “Particle swarm optimization for linear support vector machines based classifier selection,” Non- [69] J. Uthayakumar and T. Vengattaraman, “Swarm intelligence linear Analysis: Modelling and Control, vol.19, no. 1,pp.26–42, based classification rule induction (CRI) framework for qual- itative and quantitative approach: an application of bankruptcy [52] P. Danenas and G. Garsva, “Selection of support vector prediction and credit risk analysis,” Journal of King Saud University - Computer and Information Sciences,2017. machines based classifiers for credit risk domain,” Expert SystemswithApplications, vol.42,no.6,pp.3194–3204,2015. [70] M. Jiang, F. Ji, and R. Li, “eTh applied research of credit scoring [53] F. J. Hsu, M. Y. Chen, and Y. C. Chen, “The human-like combination model based on SA-GA algorithm,” in Proceedings intelligence with bio-inspired computing approach for credit of the Fourth International Conference on Business Intelligence ratings prediction,” Neurocomputing,vol.279,pp.11–18, 2018. and Financial Engineering (BIFE), pp. 491–494, 2011. [54] S. Jadhav, H. He, and K. Jenkins, “Information gain directed [71] R. Aliehyaei and S. Khan, “Ant colony optimization, genetic genetic algorithm wrapper feature selection for credit rating,” programming and a hybrid approach for credit scoring: a Applied So Computing, vol.69, pp.541–553,2018. comparative study,” in Proceedings of the th International [55] D. Wang,Z.Zhang, R.Bai,and Y. Mao,“A hybrid system with Conference on Soware, Knowledge, Information Management filter approach and multiple population genetic algorithm for and Applications (SKIMA), pp. 1–5, IEEE, 2014. feature selection in credit scoring,” Journal of Computational [72] T. C. Fogarty and N. S. Ireson, “Evolving bayesian classifiers andAppliedMathematics, vol.329,pp.307–321, 2018. for credit control—a comparison with other machine-learning [56] L. Zhou, K.K.Lai,and L. Yu, “Least squares supportvector methods,” IMA Journal of Management Mathematics, vol.5,no. machines ensemble models for credit scoring,” Expert Systems 1,pp.63–75,1993. with Applications, vol.37,no.1,pp.127–133,2010. [73] D. J. Hand and N. M. Adams, “Defining attributes for scorecard [57] A. Ghodselahi, “A hybrid support vector machine ensemble construction in credit scoring,” Journal of Applied Statistics,vol. model for credit scoring,” International Journal of Computer 27, no. 5, pp. 527–540, 2000. Applications, vol.17,no.5,pp.1–5,2011. [74] Z. Drezner, G. A. Marcoulides, and M. H. Stohs, “Financial [58] L. Yu, R. Zhou, L. Tang, and R. Chen, “A DBN-based resampling applications of a Tabu search variable selection model,” Journal SVM ensemble learning paradigm for credit classification with of Applied Mathematics and Decision Sciences,vol. 5, no. 4, pp. imbalanced data,” Applied So Computing, vol.69, pp.192–202, 215–234, 2002. [75] E.Lacerda, A.C.Carvalho, A.P. Braga,and T. B.Ludermir, [59] Y. Xia, C. Liu, B. Da, and F. Xie, “A novel heterogeneous “Evolutionary radial basis functions for credit assessment,” ensemble credit scoring model based on bstacking approach,” Applied Intelligence,vol.22,no.3,pp.167–181, 2005. Expert SystemswithApplications, vol.93,pp.182–199, 2018. [76] S.-A.Liu,Q.Wang,and S.Lv,“Application of genetic program- [60] V. S. Desai, D. G. Conway, J. N. Crook, and G. A. Overstreet, ming in credit scoring,” in Proceedings of the Chinese Control “Credit-scoring models in the credit-union environment using and Decision Conference (CCDC), pp. 1106–1110, IEEE, China, neural networks and genetic algorithms,” IMA Journal of Man- agement Mathematics, vol.8,no.4,pp.323–346, 1997. [77] S. Wang, S. Yin, and M. Jiang, “Hybrid neural network based [61] C.-S. Ong, J.-J. Huang, and G.-H. Tzeng, “Building credit on GA-BP for personal credit scoring,” in Proceedings of the th scoring models using genetic programming,” Expert Systems International Conference on Natural Computation (ICNC),vol. with Applications, vol.29,no.1,pp.41–47,2005. 3, pp. 209–214, IEEE, 2008. [62] S. M. Finlay, Using Genetic Algorithms to Develop Scoring Models [78] D. Zhang, S. C. Leung, and Z. Ye, “A decision tree scoring for Alternative Measures of Performance, Springer, Lancaster, model based on genetic algorithm and k-means algorithm,” in England, UK, 2006. Proceedings of the rd International Conference on Convergence [63] W. Cai, X. Lin, X. Huang, and H. Zhong, “A genetic algorithm and Hybrid Information Technology (ICCIT),vol.1, pp.1043– model for personal credit scoring,” in Proceedings of the Inter- 1047, IEEE, 2008. national Conference on Computational Intelligence and Soware [79] J. Yi, “Credit scoring model based on the decision tree and Engineering (CiSE), pp. 1–4, IEEE, 2009. the simulated annealing algorithm,” in Proceedings of the World [64] J. J. Huang, G. H. Tzeng, and C. S. Ong, “Two-stage genetic Congress on Computer Science and Information Engineering programming (2SGP) for the credit scoring model,” Applied (WRI), vol. 4, pp. 18–22, IEEE, 2009. Mathematics and Computation, vol.174,no.2,pp. 1039–1053, [80] Y. Marinakis, M. Marinaki, M. Doumpos, and C. Zopounidis, “Ant colony and particle swarm optimization for financial [65] H. A. Abdou, “Genetic programming for credit scoring: the classification problems,” Expert SystemswithApplications,vol. case of Egyptian public sector banks,” Expert Systems with 36, no. 7, pp. 10604–10611, 2009. Applications,vol. 36, no. 9,pp. 11402–11417, 2009. [81] M. Suˇsterˇsic, ˇ D. Mramor, and J. Zupan, “Consumer credit scor- [66] G. Dong, K. K. Lai, and L. Zhou, “Simulated annealing based ing models with limited data,” Expert Systems with Applications, rule extraction algorithm for credit scoring problem,” in Pro- vol. 36, no. 3, pp. 4736–4744, 2009. ceedings of the International Joint Conference on Computational [82] M. Marinaki, Y. Marinakis, and C. Zopounidis, “Honey bees Sciences and Optimization (CSO), vol. 2, pp. 22–26, IEEE, 2009. mating optimization algorithm for financial classification prob- [67] D. Martens, T. Van Gestel, M. De Backer, R. Haesen, J. lems,” Applied So Computing, vol.10,no. 3,pp.806–812, 2010. Vanthienen, and B. Baesens, “Credit rating prediction using [83] S.-C. Huang and C.-F. Wu, “Customer credit quality assess- ant colony optimization,” Journal of the Operational Research Society, vol.61,no.4,pp.561–573,2010. ments using data mining methods for banking industries,” 30 Advances in Operations Research African Journal of Business Management,vol. 5,no.11, pp. 4438– 4445, 2011. [84] A. B. Correa and A. M. Gonzalez, “Evolutionary algorithms for selecting the architecture of a MLP neural network: a credit scoring case,” in Proceedings of the th IEEE International Conference on Data Mining Workshops (ICDMW), pp. 725–732, IEEE, 2011. [85] S.Oreski,D.Oreski,andG.Oreski,“Hybridsystemwithgenetic algorithm and artificial neural networks and its application to retail credit risk assessment,” Expert Systems with Applications, vol. 39, no. 16, pp. 12605–12617, 2012. [86] S. Oreski and G. Oreski, “Genetic algorithm-based heuristic for feature selection in credit risk assessment,” Expert Systems with Applications, vol.41, no. 4,pp. 2052–2064, 2014. [87] J. Wang,K.Guo,andS. Wang,“Roughsetandtabusearchbased feature selection for credit scoring,” Procedia Computer Science, vol. 1, no. 1, pp. 2425–2432, 2010. [88] J. Wang, A.-R. Hedar, S. Wang, and J. Ma, “Rough set and scatter search metaheuristic based feature selection for credit scoring,” Expert Systems with Applications, vol.39,no. 6,pp.6123–6128, [89] B.Waad,B.M.Ghazi,andL.Mohamed,“A newfeatureselection technique applied to credit scoring data using a rank aggrega- tion approach based on: optimization, genetic algorithm and similarity,” in Knowledge Discovery Process and Methods to Enhance Organisational Performance,K. Osei-Bryson and C. Barclay, Eds., pp. 347–376, CRC Press, Boca Raton, Fla, USA, [90] V. Krishnaveni, K. G. Santhiya, M. P. Ramesh, and M. S. Jaganathan, “A credit scoring prediction model based on har- mony search based 1-NN classifier feature selection approach,” International Journal of Advance Research in Science and Engi- neering,vol. 7,no. 2,pp.468–476, 2018. [91] V. N. Vapnik, Statistical Learning eory, Wiley- Interscience, New York, NY, USA, 1998. [92] Q. Gu and J. Han, “Clustered support vector machines,” Artiﬁ- cial Intelligence and Statistics,pp. 307–315, 2013. [93] D. Martens, J. Huysmans, R. Setiono, J. Vanthienen, and B. Baesens, “Rule extraction from support vector machines: an overview of issues and application in credit scoring,” in Rule Extraction from Support Vector Machines,vol. 80, pp. 33–63, Springer, Berlin, Heidelberg, Germany, 2008. [94] M. Lichman, UCI Machine Learning Repository,School of Information and Computer Science, University of California, Irvine, CA, USA, http://archive.ics.uci.edu/ml/. The Scientific Advances in Advances in Journal of Journal of Operations Research Decision Sciences Applied Mathematics World Journal Probability and Statistics Hindawi Hindawi Hindawi Hindawi Publishing Corporation Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 http://www www.hindawi.com .hindawi.com V Volume 2018 olume 2013 www.hindawi.com Volume 2018 International Journal of Mathematics and Mathematical Sciences Journal of Optimization Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Submit your manuscripts at www.hindawi.com International Journal of International Journal of Engineering Mathematics Analysis Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 Journal of A Ad dv va an nc ce es i s in n Discrete Dynamics in Mathematical Problems International Journal of Complex Analysis Num Num Num Num Num Num Num Num Num Num Num Nume e e e e e e e e e e er r r r r r r r r r r riiiiiiiiiiiic c c c c c c c c c c cal al al al al al al al al al al al A A A A A A A A A A A Anal nal nal nal nal nal nal nal nal nal nal naly y y y y y y y y y y ys s s s s s s s s s s siiiiiiiiiiiis s s s s s s s s s s s in Engineering Dierential Equations Nature and Society Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com V Volume 2018 olume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 International Journal of Journal of Journal of Abstract and Advances in Stochastic Analysis Mathematics Function Spaces Applied Analysis Mathematical Physics Hindawi Hindawi Hindawi Hindawi Hindawi www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018 www.hindawi.com Volume 2018

Journal

Advances in Operations Research – Hindawi Publishing Corporation

Published: Mar 13, 2019

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

Credit Scoring: A Review on Support Vector Machines and Metaheuristic Approaches

References (97)

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies