Access the full text.
Sign up today, get DeepDyve free for 14 days.
S Thrun, TM Mitchell (1995)
Lifelong robot learningRobot Autonom Syst, 15
C Cunningham, IA Nesnas, WL Whittaker (2019)
Improving slip prediction on mars using thermal inertia measurementsAutonom Robots, 43
MA Bekhti, Y Kobayashi (2020)
Regressed terrain traversability cost for autonomous navigation based on image texturesAppl Sci, 10
S Chhaniyara, C Brunskill, B Yeomans, M Matthews, C Saaj, S Ransom, L Richter (2012)
Terrain trafficability analysis and soil mechanical property identification for planetary rovers: a surveyJ Terramech, 49
L Wellhausen, A Dosovitskiy, R Ranftl, K Walas, C Cadena, M Hutter (2019)
Where should i walk? predicting terrain properties from images via self-supervised learningIEEE Robot Autom Lett, 4
RO Chavez-Garcia, J Guzzi, LM Gambardella, A Giusti (2018)
Learning ground traversability from simulationsIEEE Robot Autom Lett, 3
H Liu, J Cai, Y-S Ong (2018)
Remarks on multi-output Gaussian process regressionKnowledge Based Syst, 144
H Inotsume, T Kubota, D Wettergreen (2020)
Robust path planning for slope traversing under uncertainty in slip predictionIEEE Robot Autom Lett, 5
CA Brooks, K Iagnemma (2012)
Self-supervised terrain classification for planetary surface exploration roversJ Field Robot, 29
K Skonieczny, DK Shukla, M Faragalli, M Cole, KD Iagnemma (2019)
Data-driven mobility risk prediction for planetary roversJ Field Robot, 36
Q Yang, Y Zhang, W Dai, SJ Pan (2020)
10.1017/9781139061773Transfer Learning
R Gonzalez, M Fiacchini, K Iagnemma (2018)
Slippage prediction for off-road mobile robots via machine learning regression and proprioceptive sensingRobot Auton Syst, 105
CJ Ostafew, AP Schoellig, TD Barfoot, J Collier (2016)
Learning-based nonlinear model predictive control to improve vision-based mobile robot path trackingJ Field Robot, 33
Z-H Zhou (2012)
10.1201/b12207Ensemble methods: foundations and algorithms
K Ho, T Peynot, S Sukkarieh (2016)
Nonparametric traversability estimation in partially occluded and deformable terrainJ Field Robot, 33
K Otsu, M Ono, TJ Fuchs, I Baldwin, T Kubota (2016)
Autonomous terrain classification with co-and self-training approachIEEE Robot Autom Lett, 1
P Papadakis (2013)
Terrain traversability analysis methods for unmanned ground vehicles: a surveyEng Appl Artif Intell, 26
CE Rasmussen, Z Ghahramani (2002)
Infinite mixtures of gaussian process expertsAdv Neural Inform Process Syst, 2
In this paper, a novel terrain traversability prediction method is proposed for new operation environments. When an off-road vehicle is operated on rough terrains or slopes made up of unconsolidated materials, it is crucial to accurately predict terrain traversability to ensure efficient operations and avoid critical mobility risks. However, the prediction of traversability in new environments is challenging, especially for possibly risky terrains, because the traverse data available for such terrains is either limited or non-existent. To address this limitation, this study proposes an adaptive terrain traversability prediction method based on multi-source transfer Gaussian process regression. The proposed method utilizes the limited data available on low-risk terrains of the target environment to enhance the prediction accuracy on untraversed, possibly higher-risk terrains by leveraging past traverse experiences on multiple types of terrain surface. The effectiveness of the proposed method is demonstrated in scenarios where vehicle slippage and power consumption are predicted using a dataset of various terrain surfaces and geometries. In addition to predict- ing terrain traversability as continuous values, the utility of the proposed method is demonstrated in binary risk level classification of yet to be traversed steep terrains from limited data on safer terrains. Keywords: Off-road vehicle, Terrain traversability prediction, Transfer learning of a vehicle, which increases the time and energy required Introduction for the operation. Furthermore, if the slip becomes sig- The demand for mobile robots and/or autonomous nificant, the vehicle cannot make successful forward pro - ground vehicles in off-road operations has steadily gress, thereby requiring its operators to replan the route increased. Examples of such applications are found in or even the entire mission. Therefore, to ensure success - the construction, forestry, mining, disaster response, and ful operations, it is important to detect risky terrains so planetary exploration industries. To safely and efficiently that effective routes can be properly selected. operate these vehicles, it is necessary to predict and assess However, the accurate prediction of vehicle motion on the traversability of the target terrains. Depending on the natural terrains is difficult owing to complicated vehi - operation, terrain traversability is conventionally meas- cle-terrain interactions. Vehicle behaviors are generally ured by the existence of geometric obstacles, vehicle pos- influenced by several factors including terrain geom - ture alterations, vibrations, required energy, and slippage. etry (slope and roughness), surface type (sand, cohesive Predicting terrain traversability is particularly impor- soil, rocks, bedrock, or mixtures of these), and surface- tant when a vehicle is expected to traverse potentially accumulated conditions (compacted or not, depth, and high-risk terrains, such as slopes made up of loose mate- moisture content) as well as the vehicle size, weight, and rials. One of the dangers of traversing such terrains is the locomotion configurations [1]. slippage of vehicles. Slippage inhibits the smooth traverse Numerous studies have investigated ways to improve the prediction of traversability on natural terrains [2, 3]. *Correspondence: h-inotsume@nec.com Although many of the developed methods are relatively Data Science Research Laboratories, NEC Corporation, 1753 Shimonumabe, Kawasaki, Kanagawa 211-8666, Japan promising, sufficient traverse data on target environments Full list of author information is available at the end of the article © The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 2 of 25 are still required for the accurate prediction of travers- versability prediction challenge when training data is ability. For example, if a vehicle’s target is an ascending severely limited, owing to complicated terrain–vehi- slope, traverse data related to that possibly hazardous ter- cle interactions. rain is primarily required for making an accurate predic- 3. The evaluation of the effectiveness of the proposed tion. However, during operations, vehicles usually avoid method using a dataset collected with a mobile robot risky terrains for safety reasons. Therefore, traverse data on multiple terrain geometries and surface types. on such terrains might be limited or even non-existent. Model parameters solely trained on benign terrains can The basic concept of the proposed method was previ - overfit the limited data, resulting in either overestimat - ously proposed and partially validated by using synthetic ing the traversability or underestimating the risks on more and real datasets in earlier works [5, 6]. The work in this difficult terrains. This makes the traversability prediction paper newly demonstrates the utility of the proposed of risky environments highly challenging. method in 1) slip risk level classification and 2) power This study addresses the above problem, which is inher - consumption prediction on slopes from limited in-situ ent to the prediction of terrain traversability. Specifically, data. This paper also presents thorough additional evalu - this study aims to improve the prediction accuracy of tra- ations of the traversability prediction performance from versability on challenging terrains in new operation envi- the viewpoint of the usability of the method in real oper- ronments on the basis of (1) traverse data on relatively ations, with added data on more terrain surface types and safe areas and (2) past experiences (Fig. 1). Accordingly, conditions compared to the previous work [6]. this study proposes an adaptive terrain traversability pre- The rest of the paper is organized as follows. The next diction method based on multi-source transfer Gauss- section describes related works of this study. After that, ian process regression (MS-TGPR) [4]. The proposed the transfer learning-based traversability prediction method leverages traverse experiences obtained from method is proposed. Then, the experiments for collect - multiple terrain surfaces and improves the prediction ing the traverse data of a mobile robot on multiple types accuracy on possibly risky terrains in the target mission of terrain are described. Following that, the proposed environment, where in-situ traverse data are only avail- method is evaluated using the collected dataset and dis- able from benign terrains beforehand. cusses its usability and extendibility. Finally the last sec- The contributions of this study are as follows. tion summarizes this study and concludes this paper with future work. 1. The development of a method for predicting terrain traversability on untraversed, possibly risky terrains Related works based on multi-source transfer learning. The prediction of terrain traversability for safe autono - 2. The application of the proposed method to vehicle mous vehicle navigation has been studied for the last sev- slip prediction, which is a significantly difficult tra - eral decades [2, 3]. Most of the developed methods are Terrain type 1 Terrain type 2 Terrain type 3 New terrain surface type Slip Slip Slip Slip Slip Slope Data obtained on multiple terrain surfaces Data on benign terrain Model learned beyond training data range Slip Slope Fig. 1 Concept of proposed traversability prediction based on multi-source transfer regression. By leveraging past experiences on multiple terrain surfaces, the proposed method improves the prediction accuracy on possibly risky terrains only from traverse data on safer terrains in new environments [6] Risk I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 3 of 25 sufficiently robust for indoor or partial outdoor situations older data are removed from the training data whenever thanks to their detection and avoidance of dangerous new local traverse data are observed, thereby improving geometric obstacles. However, conventional approaches the adaptation of the model to new operation environ- are not applicable when a vehicle is required to traverse ments. To address the intra-class variability challenge, more difficult terrains characterized by unconsolidated [14] adopted a thermal sensor to examine the correla- materials, where vehicle behaviors are not easy to predict tions between soil properties and the thermal inertia of from pure-terrain geometry or visual information. terrains and showed that a mixture-of-experts approach Learning or model updating using in-situ traverse data utilizing surface thermal information can differentiate is a straightforward way to improve the prediction accu- high-slip sand surfaces from low-slip sandy terrains and racy of the terrain traversability. The work presented in [7 ] improve the slip prediction accuracy on sand-type sur- proposed a learning method that repeatedly improves the faces. However, none of the above-mentioned studies prediction accuracy of terrain traversability or the vehicle explicitly address the challenge of the data limitation in responses during traversals of the same operation area traversability model learning in new environments. multiple times. In [8], models of control disturbances are One possible method for improving the prediction learned and iteratively improved from repeated drives on accuracy on high-risk terrains might be to aggressively the same terrain. The learned models are then used for explore such terrains and acquire the traverse data nec- better path tracking on the same off-road terrain. These essary for learning a better model [15]. However, this methods are effective for vehicles deployed over a long action can trigger critical conditions in the vehicle, duration in confined environments. However, if the vehi - and therefore should be avoided if the risk is uncertain. cles are not supposed to traverse the same locations mul- Another work [16] proposed a method to learn terrain tiple times, this type of approach is not applicable. traversability from physical simulations. As the authors Self-supervised learning approaches have been applied also mentioned as a limitation, since the traversability is for terrain classification and traversability prediction learned solely from terrain geometry for solid surfaces, [9–11], where proprioceptive sensing information is this method is not applicable to deformable, slip-induc- correlated to visual terrain information and utilized to ing terrains that are hard to realistically simulate without supervise a vision-based terrain classifier, and vice versa. rigorous parameter tuning from real data. Another pos- These approaches classify terrains and predict the tra - sible approach is simply to assume that terrains for which versability of distant fields by comparing underfoot ter - no data is available are untraversable. rain responses (such as slip, vibration, or force/torque) to In contrast to the above approaches, this study attempts visual cues at the learning phase. Because these methods to accurately predict the traversability of untraversed ter- only require a small amount of labeled data to train the rains by avoiding an overly conservative prediction for terrain classifiers, they are relatively promising for clas - scenarios where the vehicle is required to traverse such ter- sifying terrains from data on safe regions. However, they rains. The proposed method leverages data obtained from are limited when large variabilities in vehicle behaviors relatively safe terrains in the target environment and from exist among intra-classes (sub-classes). For example, the past experiences of traversing multiple types of terrain sur- behaviors of a vehicle can be significantly different even face, and it improves the prediction accuracy for challenging on similar-looking sandy surfaces depending on the accu- terrains without actually having to traverse such terrains. mulation conditions. In addition, even if the terrain clas- The proposed method is fundamentally a general traversa - sification is successful, it is still challenging to predict the bility learning and prediction method, and as such, it can be traversability on higher-risk terrains of the corresponding integrated with the earlier developed self-supervised terrain class where training data are not available. The predic - classification approaches to develop an end-to-end learning tion model learned on limited available data can possibly framework for operations in new environments. be overfit on a low-risk terrain, thereby resulting in an underestimation of the risk of untraversed terrains. Proposed method Several approaches have been proposed to address the This study assumes that vehicle behaviors on any terrain challenge of intra-class variability. In [12], the prediction can be represented by combinations of behaviors on the accuracy of terrain traversability is improved by globally base terrains that constitute the target terrain. On the and locally modeling traversability. The local variations basis of this hypothesis, this study develops a traversabil- of vehicle motions are captured by learning travers- ity prediction method for new operation environments. ability as a function of positions in addition to the global The proposed method is developed in accordance with model constructed on the basis of terrain appearance and two machine learning paradigms: transfer learning and geometry. In [13], a first-in-first-out data structure was ensemble learning. Transfer learning [17] is a method utilized to update traversability models. In this method, that applies data or learned models in one or more Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 4 of 25 applications/tasks (source domains) to the model for risky terrains, in N surface types (source domains) another application/task (target domain). By transferring S ={S : 1 ≤ i ≤ N} beforehand. The data of n obser- i S the model information from the source domain appropri- vations in each source domain S are denoted as (S ) (S ) (S ) (S ) (S ) i i i i i ately, transfer learning can improve the performance in D ={X , y }= {(x , y )|j = 1, ··· , n } . In j j the target domain, which has only limited training data. contrast, in a specific surface type (target domain) T, only In contrast, ensemble learning [18] is a method that com- traverse data of n observations in relatively safe, benign bines multiple models learned from the same training terrains have been obtained by the vehicle. Similarly to dataset but with different data subsets. This enables the the source domains, the target training data are denoted (T ) (T ) (T ) (T ) (T ) generalization performance of the learned model to be as D = {X , y }= {(x , y )|j = 1, ··· , n } . j j improved; hence, its prediction performance is suitable Each item of source and target domain data has been for unseen data. separately labeled and classified in accordance with the corresponding domain either by human experts or on the Problem definition basis of an existing classification method (e.g., [9, 10]). In this study, the traversability prediction is formalized as Under these assumptions, the objective of this a regression problem in which the relationship between study is to stochastically predict the traversabil- (T ) ∗ various input features x and an output traversability ity f (x ) , which corresponds to the input point index of continuous values f (x) , along with its predictive x in the target domain, using the already described uncertainty, is learned from training data in a supervised available data samples, i.e., to predict the distribu- (T ) ∗ ∗ (S ) (S ) (T ) 1 N learning fashion. As described earlier, the traversability tion p(f (x )|x , D , ··· D , D ) , before actu- index can include, for example, vehicle posture alterna- ally traversing that point. Specifically, this study builds tions, vibration, required energy, or slippage. The input the predictive target traversability model p(f (x)) as a features x represent terrain geometries (terrain pitch, weighted sum of multiple weak traversability models (S ,T ) roll, and/or surface roughness). In general, x can also {p(f (x)) : 1 ≤ i ≤ N } learned from the available data include other features, such as vehicle locations [7, 12], in the target domain T and those in the source domains visual features of the terrain [19], or control inputs to the S, as expressed below: vehicle [8]. This study aims at predicting the probabilis - ∗ (S) (T ) (S ,T ) (S ) (T ) tic distribution of the traversability (i.e., p(f (x) ) rather i i p(f (x)|x, D , D ) = w · p(f (x)|x, D , D ), than performing deterministic predictions, as the vehicle i=1 (2) motions can be significantly varied in difficult off-road environments even when the same vehicle traverses the where w represents the weight of the model learned from same terrain and location multiple times. Operating a the target data and the i-th source domain data, and sat- vehicle based on deterministic traversability prediction isfies w = 1 . Although any algorithm can be adopted (S ,T ) can lead to unexpected hazardous situations in such i to learn the base model f (x) and the weight w of environments. that model, this study applies the MS-TGPR algorithm This study makes two key assumptions. First, the [4], a variant of Gaussian process regression (GPR) [20], vehicle is able to estimate the terrain geometry x ahead for multi-source transfer learning. of it by using on-board exteroceptive sensors (such as a stereo camera or range sensor) before traversing Gaussian process regression the terrain. In addition, the vehicle can estimate the GPR is a data-driven, nonparametric approach to learn- traversability measurement y that corresponds to the ing a regression model that does not require strong predictive target traversability f (x) from the vehicle assumptions in its forms (e.g., linear, polynomial, or response during its traverse on the terrain correspond- exponential), with the shape of the model determined on ing to the input x by using on-board sensory informa- the basis of the training data. In addition, GPs can express tion. The measurement is assumed to have a random the prediction uncertainties as variances, together with noise ε that follows a zero-mean Gaussian distribu- the predictive means. Owing to these features, GPR is 2 2 tion with variance σ , i.e., ε ∼ N (0, σ ) . Then, y can be suitable for modeling complicated vehicle behaviors in expressed as off-road environments. Several studies (e.g., [7, 12, 21]) have demonstrated the effectiveness of GPR for model - y = f (x) + ε. (1) ling terrain traversability when sufficient amounts of data are available. Second, there exist a variety of terrain surface The GPR-based traversability method assumes that classes, or domains. Among these domains, the vehi- the traversability latent function values f (X) of the input cle has collected a set of traverse data, which cov- data X follow a multivariate Gaussian distribution, which ers a range of terrain geometries from safe to I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 5 of 25 Transfer Gaussian process regression is determined by the mean µ( X) and covariance matrix In TGPR [23], the regression for the target domain is mod- K(X,X) . In this study, the mean µ( X) is a zero vector assum- eled from the rich data in the source domain S and the ing that the training traversability data are normalized before small data in the target domain T on the basis of the cor- learning. The (k , l)-th element of the covariance matrix is relation between the two domain. This correlation is cap - computed on the basis of a covariance function (kernel func- tured in addition to the correlation between data points by tion) as K = k(x , x ) . In this study, the exponential covar- k ,l k l using the transfer covariance function k (x, x ) , as iance function defined below is adopted in both the GPR and the transfer GPR (TGPR) for slip-slope modeling: ′ (S ) ′ (T ) k(x, x ) x ∈ X & x ∈ X or (T ) ′ (S ) ′ ′ T ′ k (x, x ) = , x ∈ X & x ∈ X k(x, x ) = a exp −(x − x ) A (x − x ) , (3) 1 2 k(x, x ) otherwise (7) where a is a constant scalar and A is a constant diago- 1 2 where ∈ [−1, 1] denotes the similarity coefficient nal matrix given by A = diag(a ) , where a is a constant i 2 2 2 between the source domain S and the target domain T, vector of the same length as that of the input feature x . i with a higher value indicating a higher similarity. The constants a , a and the measurement noise variance i 1 2 On the basis of the transfer covariance function, the σ are a set of hyper-parameters to be tuned. For mod- covariance matrix for a single TGPR is defined as eling the relationship between the electric power con- sumption of actuators and the terrain slope, the following K K S S i S T i i i linear covariance function is adopted: K = , S T (8) K K i TS TT ′ T ′ k(x, x ) = a + a x x , (4) 3 4 where K and K represent the covariance matrices S T where a and a are another set of hyper-parameters of the individual domains S and T, respectively, while 3 4 i for the power modeling. Note that the linear covariance K = K is the covariance matrix between the two S T i TS function was selected for power consumption because domains. These covariance matrices are computed from strong linear relations can be observed between power Eq. (3) or Eq. (4). (S ) (S ) (S ) i i i consumption and terrain slope, as reported in [22] Given the source domain data D ={X , y } and (T ) (T ) (T ) and also later demonstrated in this paper. While GPR the training data of the target domain D = {X , y } , (S ,T ) is a data-driven approach and does not need a strong the joint distribution of the latent function, f (x) , (S ,T ) (S ) (T ) i i assumption of its model shape, as mentioned earlier, the and the measurements, y = {y , y } , can be model shape can also be restricted by specifying a certain learned by tuning the hyperparameters in the covariance covariance function if prior knowledge on the shape is function and the similarity coefficient . available. The prediction in the target domain is given by Given the training data D = {X, y} of n data points, the (S ,T ) (S ) (T ) (S ,T ) (S ,T ) i i i i p(f (x )|x , D , D ) = N (m (x ), v (x )), ∗ ∗ ∗ ∗ joint distribution of the latent function and measurements, (9) y , can be learned by tuning the hyper-parameters of the with the predictive mean and variance respectively rep- model. In the prediction step, the predictive distribution resented by of the traversability at the point x can be computed by p(f (x )|x , D) = N (m(x ), v(x )) , where the predictive ∗ ∗ ∗ ∗ (S ,T) (Si,T) −1 (S ,T) i i m (x ) = K (x , X )(K + �) y , ∗ ∗ ∗ S T mean m(x ) and variance v(x ) are given by the following ∗ ∗ (10) equations, respectively: (S ,T) (Si,T) v (x ) = k(x , x ) − K (x , X ) ∗ ∗ ∗ ∗ ∗ 2 −1 m(x ) = K(x , X)(K(X, X) + σ I ) y, (5) (11) ∗ ∗ n −1 (Si,T) (K + �) K (X , x ), S T ∗ ∗ 2 −1 v(x ) = k(x , x ) − K(x , X)(K(X, X) + σ I ) K(X, x ), ∗ ∗ ∗ ∗ n ∗ where K (·, ·) denotes the transfer covariance matrix (Si,T ) (S ) (T ) (6) i computed by Eq. (7), X ={X ,X } , and where K(·, ·) denotes the covariance matrix evaluated σ I 0 s S with the pairs of training input points or the prediction i � = , (12) 0 σ I point x based on Eq. (3) or Eq. (4), and I denotes an t ∗ n n × n identity matrix. 2 2 with σ and representing the variances of the meas- S T urement noise in domains and T, respectively. i Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 6 of 25 Multi‑source transfer Gaussian process stacking exp( ) w = (14) In the proposed terrain traversability prediction method, N exp( ) i=1 the weak TGPR models from multiple source domains are stacked to build the target prediction model in an ensemble In addition, a threshold for the similarity coeffi - th learning fashion from Eq. (2). The predictive distribution of cient is introduced in this study. If the domain similar- the traversability can be expressed as ity is lower than the threshold, the weight w of the i i corresponding source domain is set to zero by setting ∗ (S) (T ) (S ,T ) (S ) (T ) i i = −∞ in Eq. (14). This threshold controls the amount p(f (x)|x, D , D ) = w · p(f (x)|x, D , D ) of low-similarity domains included in the final model. i=1 The weight function and threshold can together avoid (S ,T ) (S ,T ) i i = w · N (m (x), v (x)). the negative transfer [26] from irrelevant source domains i=1 that would deteriorate the final learned model. (13) In summary, by learning the similarity coefficient , This can also be viewed as a mixture of Gaussian pro - as well as by tuning the hyper-parameters in the TGPR (S ,T ) cesses [24, 25], where the weight w corresponds to the i i models, {p(f (x)) : 1 ≤ i ≤ N } , from the target train- gating function for the i-th transfer regression model. ing data and the source domain data, the proposed tra- Figure 2 illustrates the general concept of the model versability prediction method develops the prediction learning. model of the target domain p(f (x)) based on Eq. (13). To appropriately combine the multiple TGPR models Here, the set of model (hyper-)parameters, , to be tuned incorporating domain similarities, the weight w of the includes the domain similarity, , for each source–target source domain S is determined on the basis of the simi- combination, the variance of the measurement noise in 2 2 larity coefficient , as in [4]. Specifically, in this study, the the target domain, σ , that of each source domain, σ , T S following softmax function is adopted to assign a higher and the hyper-parameters in the covariance function, weight to a source domain with a higher, positive source- a and a for slip or a and a for power consumption, 1i 2i 3i 4i target similarity. for each source domain S . There are thus (3 + l)N + 1 (hyper-)parameters to learn for a slip model, where l Fig. 2 Outline of traversability model learning based on multi-source transfer Gaussian process regression [6]. The domain similarity between each source S and target T domain, as well as the single TGPR model between them, are learned from the data present in S and T. The prediction i i model for the target domain is then built as an ensemble of all TGPR models. The importance of each model is determined from i I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 7 of 25 denotes the length of the input feature vector x , and categorized by counting the number of data points above 4N + 1 for a power consumption model. In this study, a threshold, this study adopts a predicted traversability these (hyper-)parameters, , are learned such that the distribution for the same assessment. following log-likelihood of the joint distribution of the Specifically, this study considers an estimation of the (T ) target training measurement y is maximized: risk label l of the target terrain x on the basis of the risk ∗ predicted traversability index, as (T) (T) (S) ln p(y |X , D ; �) n N (T) (T) T (S ,T) (S ,T) 2 i i = ln w · N (m (x ), v (x ) + σ ) . High if p (x ) ≥ p j=1 i=1 j j t risk ∗ threshold l (x ) = , ∗ (16) risk Low otherwise (15) As the derivative of Eq. (15) cannot be analytically solved where p (x ) denotes the risk probability that is esti- risk ∗ in a closed form, random/grid search and k-fold cross mated as the probability of the traversability index y(x ) validation are adopted in this study. Specifically, the of being greater than or equal to a threshold y , a s threshold each source domain is determined by random search and p (x ) = P(y(x ) ≥ y ), the σ by grid search with 5-fold cross validation. Other ∗ ∗ (17) risk threshold hyper-parameters, σ , a , a , a , and a for the TGPR S 1i 2i 3i 4i The risk probability threshold p in Eq. (16) con- threshold of the source S , are adopted from those of a pre-tuned trols the extent to which the amount of uncertainty in GPR model for the corresponding source domain to the prediction y(x ) to be incorporated in the classifica - reduce the computation burden and also to increase the tion with a lower threshold makes the classification more learning stability. Although this may limit the accuracy conservative. of the learned model, reasonable results can be obtained, as presented later in the Evaluation and Discussion sec- tion. Alternatively, the Expectation-Maximization (EM) Data collection experiments algorithm [27] or gradient descent-based methods with a To evaluate the effectiveness of the proposed method, mean-squared error loss (as used in [4]) are other pos- a set of experiments was conducted to collect traverse sible approaches for tuning all of the hyper-parameters. data on multiple terrain surfaces and geometries using a In the prediction phase, the geometry of the terrain mobile robot. In this study, vehicle longitudinal slippage ahead of the vehicle, x , is input to the learned model to and electric power consumption of actuators are adopted obtain the predictive distribution of the traversability, as the terrain traversability to be predicted. The traverse p(f (x )) , on the target terrain surface. data required for estimating these were collected in the experiments, along with some other data for future use. Risk level classification In addition to predicting the traversability as a continu- Experimental setting and procedures ous value, the learned regression model can also be used Figure 3 shows the test vehicle and field used in the to predict a discrete risk level of the target terrain, e.g., experiments. “high risk” or “low risk”. In some missions, it may be more In the experiments, a small mobile robot developed at useful to output such discrete labels rather than continu- JAXA/ISAS was used to collect traverse data. The dimen - ous traversability values. sions of the robotic vehicle are 0.85 m × 0.80 m × 0.65 m, One straightforward way to achieve this is to categorize and its mass is approximately 50 kg. The vehicle has four the regression output from the proposed method into wheels with independent driving and steering actuators, one of the risk levels based on pre-defined thresholds. including a differential link for making all wheels come For example, the risk of immobility on deformable ter- into contact with the surface on uneven terrains. The rains can be inferred by checking whether the predicted actuators are controlled by an on-board computer and slip value y is above a threshold or not, similarly to the motor drivers, and their rotation angles are measured approaches in [13, 28, 29]. Although some of the exist- by rotary encoders. The vehicle is also equipped with a ing works classify the risk level in a deterministic manner Stereolabs ZED 2 stereo camera featuring a built-in iner- by assessing mean predictive slip for this risk categoriza- tial measurement unit (IMU). The 6-degrees-of-freedom tion, it would be more appropriate to do so in a stochas- vehicle motion (position and orientation) is estimated tic fashion by taking into account uncertainties in the slip along with the terrain geometry using the stereo visual- prediction for the risk leveling. As the proposed regres- inertial simultaneous localization and mapping (VI- sion method can provide predictive distributions of a tra- SLAM) software provided by Stereolabs. versability index, the method can be easily applied to that An external PC generates the operation commands and direction. While a stochastic risk classification approach sends them to the on-board computer of the vehicle via has also been proposed ([13]), where a risk level is Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 8 of 25 Time profiles Motor voltage & current Power estimation Power Wheel odometry Motor Slip drivers estimation Slip VI odometry Stereo Visual- camera Slope inertial + estimation Point cloud SLAM IMU Slope (a) Test field and mobile robot (b) Sensingsystem and data processingflow Fig. 3 Experiment system. The tiltable test field is covered with one of the surface materials in Fig. 4. The four-wheeled test vehicle is equipped with motor encoders, motor drivers, and a stereo camera with a built-in IMU for traverse data collection. The vehicle slip and terrain inclination are estimated from the wheel odometry and VI-SLAM. The electric power consumption of actuators is estimated from the voltage and current applied to the motors, which are monitored at the motor drivers positioned on the field to improve the accuracy of the VI-SLAM. Before each experiment, the test surface was prepared and the vehicle was positioned in front of the field. The vehicle was then commanded to drive at a speed of Rough sand Loose sand Compactedsand approximately 5.5 cm/s in the longitudinal slope direc- tion without any steering maneuver. During each experiment, the encoder count and the applied current and voltage of the right front and rear wheel motors were obtained from the motor drivers Small rocks Gravel Gravel with sand at 10 Hz. The VI-SLAM computed and recorded the 6-degrees-of-freedom vehicle pose and registered point clouds at 30 Hz and 1 Hz, respectively. In addition, the ground truth of the vehicle pose was recorded at 100 Hz using a set of motion capture cameras placed around the Sandover bedrock Sand-coveredbedrock Bedrock test field. The motion capture data was only used to eval - Fig. 4 Images of terrain surfaces tested in the experiments uate the accuracy of the VI-SLAM. The position errors of the VI-SLAM were at most a couple of centimeters in the 1-meter linear traverse without steering. Raw ste- Wi-Fi. The stereo camera is wire-connected to another reo camera images and IMU data were also recorded for PC and the VI-SLAM is run on the machine owing to the future use; however, they were not utilized in this study. limited computing capability of the on-board computer and the compatibility of the software. Data processing The size of the test field is 1.8 m × 1.8 m, as shown in From each recorded traverse data, terrain inclina- Fig. 3a. The field can be manually tilted by hydraulic jacks tion, vehicle slippage, and electric power consumption and a lifting table, which enables the vehicle to be tested were extracted as data points in the following way. The on a variety of terrain geometries. recorded data were divided into non-overlapping time Nine surface materials and/or conditions (shown in windows of a duration t . Then the terrain inclination, Fig. 4) were set in the test field: rough sand, loose sand, vehicle slippage, and power consumption corresponding compacted sand, small rocks, gravel, gravel with sand, to each time window were estimated. In this study, t sand over bedrock, sand-covered bedrock, and (clean) was set to 1.0 s to capture local vehicle responses. Fig- bedrock. The details of each surface type are summa - ure 3b shows the flow of the data processing. rized in Table 1. In addition, several large rocks were I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 9 of 25 Table 1 Description of surface types used in the experiments Surface domain Description Rough sand Dry silica sand loosened and left in a rough surface condition Loose sand Dry silica sand loosened and then slightly leveled Compacted sand Dry silica sand loosened, then compacted and leveled Small rocks Small volcanic rocks randomly and densely arranged on sand Gravel Small pieces of river gravel Gravel with sand Mixture of gravel and dry sand Sand over bedrock Silica sand accumulated over flagstones with 2 cm depth Sand-covered bedrock Flagstones covered with millimeter-layer of silica sand Bedrock Textured flagstones arranged without large gaps between each Rough sand Compacted sand Gravel Sand over bedrock Bedrock Loosesand Small rocks Gravelwithsand Sand-covered bedrock 1.2 1.0 0.8 0.6 0.4 0.2 0.0 −0.2 0 10 20 30 0 10 20 30 Slope pitch angle (degrees) Slopepitch angle (degrees) Fig. 5 Dataset of slip vs. slope and power consumption vs. slope collected on nine different surfaces. The slip values higher than 1.0 on small rocks and sand-covered bedrock were triggered by the downhill skid as the vehicle attempted to move uphill. The negative slip values mainly resulted from errors in the wheel and visual-inertial odometries, as well as the incomplete time synchronizations of these data. Individual data of each domain is plotted in Fig. 6 for slip and Fig. 7 for power consumption The longitudinal vehicle slippage is measured by the A negative slip indicates that the vehicle travels further slip ratio [1] as follows: than commanded. The total electric power consumption P of the motors 1 − d /d (if d ≤ d ) vi w vi w was estimated from the recorded voltage V and current s = , x (18) d /d − 1 (otherwise) w vi I of the i-th motor ( i ∈[1, 4] ) a s P = V × I . Only i i i the voltage and current of the right wheels were recorded where d represents the longitudinal travel distance in due to device limitations. Therefore, the total power was t , which is estimated from the wheel odometry, and roughly estimated by using the same motor current and d denotes the estimated distance from the VI-SLAM. vi voltage values for the left wheels as for the right, assum- A positive slip ratio indicates that the vehicle moves a ing they are not significantly different when the vehicle shorter distance than commanded, and s = 1.0 indicates travels on longitudinal slope. that the vehicle does not make any forward movement. Slip ratio Power consumption (W) Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 10 of 25 The terrain inclination was estimated using the point sand had more variability and became higher than that clouds obtained from the VI-SLAM. For each time win- on the other sand surfaces. dow, the positions of the vehicle wheels were estimated The largest slip variability was observed on the small from the vehicle’s position and geometry. The point rocks, where the average slip gradually increased with the clouds included in the volume swept by the wheels were terrain inclination. On the slopes of approximately 20 , then extracted, and a best-fit plane to the point clouds the vehicle repeatedly made slight uphill progress and was computed using linear regression. The terrain pitch then slid downhill, which resulted in no successful slope and roll angles were determined from the slope of the ascent. The slip ratio values higher than 1.0 on small plane. The terrain roughness was also measured by the rocks represent the downhill skid motion. residual of the fitted plane from the points. The accuracy On the gravel and gravel with sand, a moderate increase of the slope estimation was approximately 1 . of the slip was observed compared to the sand types. The slip on the gravel was relatively higher than that on the Slip‑slope and power‑slope dataset gravel with sand. The collected slip vs. slope data and power consumption On the bedrock, the average slip was around zero, even vs. slope data are plotted in Fig. 5, with different colors on the slope of approximately 25 , owing to the high fric- representing the surface domains. Individual data in each tional texture of the surfaces. On the 30 slope, the slip surface domain are shown in Figs. 6 and 7 for slip and slightly increased, but the vehicle could make steady power consumption, respectively. uphill progress. Although a similar vehicle behavior was The slip on the rough sand, loose sand, and compacted observed on shallow slopes of the sand-covered bedrock, sand surfaces used in this study exhibited a relatively the slip suddenly increased on more than 15 , and the similar trend, as the slip rapidly increased along with the vehicle could not make continuous forward progress on increase in the terrain slope angle. The slip on the rough the 25 slope. 1.2 1.2 1.2 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0.0 0.0 0.0 −0.2 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (a) rough sand (b) loose sand (c) compacted sand 1.2 1.2 1.2 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0.0 0.0 0.0 −0.2 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (d) small rocks (e) gravel (f) gravel with sand 1.2 1.2 1.2 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0.0 0.0 0.0 −0.2 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (g) sand over bedrock (h) sand-covered bedrock (i) bedrock Fig. 6 Reference GP regression model for the slip vs. slope characteristics of each surface class. Each model was learned from 75% of all the data in the corresponding surface domain. The red curve represents the predictive mean while the blue-shaded area represents the confidence interval with 2-standard deviation around the mean Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 11 of 25 25 25 25 20 20 20 15 15 15 10 10 10 5 5 5 0 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (a) rough sand (b) loose sand (c) compacted sand 25 25 25 20 20 20 15 15 15 10 10 10 5 5 5 0 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (d) small rocks (e) gravel (f) gravel with sand 25 25 25 20 20 20 15 15 15 10 10 10 5 5 5 0 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) Slope (degrees) (g) sand over bedrock (h) sand-covered bedrock (i) bedrock Fig. 7 Reference GP regression model for the power consumption vs. slope characteristics of each surface class. Each model was learned from 75% of all the data in the corresponding surface domain. The red curve represents the predictive mean while the blue-shaded area represents the confidence interval with 2-standard deviation around the mean The slip on the sand over bedrock type differed from that f (x) for the longitudinal slip and power consumption, on the loose and compacted sand despite the similar visual respectively. outlooks. The slip behavior of the sand over bedrock was relatively similar to that on the sand-covered bedrock. Reference GPR models Note that the negative slip values of some data points First, GP regression models for the slip-slope character- were triggered by errors in the wheel and VI odometries, istics and for the power-slope in all surface domains were and also owing to the imperfect time synchronizations of learned for reference. The hyper-parameters of the refer - the signals of these odometries. ence models were also adopted for the TGPR models of The power consumption showed a linear relationship the corresponding source domains in the later evalua- to the terrain slope (as seen in Fig. 7) in every surface tions. In the learning of each model, 75% of the data were domain. Other than that, the trends of the power con- adopted for training and the remaining 25% for testing. sumption between different surface domains were simi - Additional artificial slip data of s = 0.0 was inserted at lar to those of the slippage, as surface types that induced approximately zero degrees, and those of s = 1.0 were higher slippage tended to require higher power for the added over the slope angles, where no uphill progress was vehicle to drive. expected, to stabilize the slip model curves around the two edges, where no test data was obtained. The learned models are presented in Fig. 6 for slip and in Fig. 7 for Results and discussion power consumption. As shown, given sufficient train - This section evaluates the effectiveness of the proposed ing data, GPR could capture the trends of slip and power method for predicting terrain traversability using the consumption in all surface types to a certain extent. dataset described in the previous section. Better models for slip might be obtained by imposing a The terrain slope pitch angle was used as the input fea - ture x to model the separate latent functions f (x) and Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Power consumption (W) Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 12 of 25 constraint of monotonic increase of the slip against the The learned slip-slope models for the rough sand and increase in the slope, as in [12]. sand-covered bedrock domains are plotted in Fig. 8 The root-mean-square errors (RMSEs) and log-like - as examples. The RMSEs and LOGLIKs of the models lihoods (LOGLIKs) of the reference GPR models were learned for all four of the target domains are presented evaluated on the test data, and are presented in Table 2. in Table 3. In addition, this evaluation assesses how the RMSEs indicate the accuracy of the predictive mean learned predictive distribution p(f (x)) can represent (lower is better), while the LOGLIKs relatively indicate the true distribution q(f (x)) based on the Kullback- the feasibility of the predictive distribution (higher is bet- Leibler divergence (KLD) [27]. The KLD measures the ter). The surface type with a larger slip variability resulted difference between two probability distributions, with in a larger RMSE and a lower LOGLIK. a lower value indicating a higher similarity and KLD = 0 indicating that the two distributions are identical. Evaluation 1: Comparison of MST ‑ GPR and GPR Since the true distributions from which the traverse In the first evaluation, the proposed MS-TGPR-based tra - data were generated are unknown, the distributions versability prediction method was compared with GPR- predicted by the reference GPR, presented in Fig. 6, based methods in a slip prediction scenario. Four surface were adopted instead to compute the KLDs. Figure 8a, types, namely, compacted sand, small rocks, gravel, and c, e presents the prediction models for the rough sand bedrock, were adopted as source domains, and four sur- learned using GPR-naive, GPR-conventional, and MS- face types, namely, rough sand, gravel with sand, sand TGPR, respectively. Owing to the very limited data, over bedrock, and sand-covered bedrock, were set as tar- the model learned by GPR-naive predicted just zero get domains. The data presented in each target domain slips on every slope, as shown in (a). This result indi - with slopes shallower than 5 , as well as the entire data cates that the prediction is a challenging extrapolation in the source domains, were used for training each MS- problem. In contrast, by adopting the data augmented TGPR. Each model was then tested with the correspond- from the compacted sand domain, GPR-conventional ing target domain data of the slopes steeper than 5 . This could predict the slip on the test slopes better than simulates the prediction of traversability on slopes in a GPR-naive, as shown in (c). GPR-conventional worked new environment from the data on relatively benign ter- well owing to the relatively similar trends in the slip- rains. In this evaluation, the threshold of the domain sim- slope curves for the rough sand and compacted sand. ilarity, , was set to 0.8. However, some test data were outside of the confi - th In addition to the proposed method, two GPR-based dence interval, indicating that the model sometimes approaches were tested for comparison. The first underestimated the possible vehicle slippage. The pro - approach, GPR-naive, learns a GPR model solely from posed MS-TGPR could also capture the slip trends the limited training data of the target domain. The sec- of the rough sand on the test slopes, as shown in (e), ond approach, GPR-conventional, utilizes the slip data with more test data included in the confidence inter - in source domains with similar visual outlooks. In this val. The source-target similarity and the weight w i i latter approach, the training data comprises the tar- for each source domain are presented in Table 4. High get training data and the data in the source domains. similarity values were learned for the compacted sand, Similar data augmentation methods have been conven- small rocks, and gravel, while the similarity to the bed- tionally adopted in practice when the available train- rock was relatively less significant. As the threshold of ing data is limited [12]. Alternatively, this approach = 0.8 was set, the TGPR models from the former th is also considered a type of online model updating, in three domains were selected to develop the predic- which the pre-trained GPR model for a source domain tion model. As presented in Table 3, although GPR- is updated using the newly obtained in-situ traverse conventional exhibited the lowest RMSE, the proposed data. In this study, the data from the compacted sand method achieved the highest LOGLIK and lowest KLD, were introduced to train the models for the rough sand indicating the best modeling of the predictive distribu- and sand over bedrock, whereas the data on gravel tion and the prediction uncertainty among the three were added to train the model for gravel with sand, methods. and the data obtained from the bedrock were added The slip prediction models for the sand-covered to train the model for the sand-covered bedrock. No bedrock are presented in Fig. 8b, d, f. Again, although pre-processing was performed on the source data for GPR-naive in (b) could capture the trends of the train- the augmentation. The augmented source data was ing data, the model predicted an almost constant slip restricted to the range of the terrain inclination, which on steeper slopes without available training data. At is not included in the target domain, i.e., slopes steeper this point, GPR-conventional could not predict the ◦ ◦ than 5 . abrupt increase in the slip over approximately 20 , a s I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 13 of 25 1.0 1.0 training data training data test data test data 0.8 0.8 predictivemean predictive mean 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope(degrees) Slope(degrees) (a) rough sand, GPR-naive (b) sand-covered bedrock, GPR-naive 1.0 1.0 training data test data 0.8 0.8 augmenteddata predictive mean 0.6 0.6 0.4 0.4 0.2 0.2 training data testdata 0.0 0.0 augmenteddata predictive mean −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope(degrees) Slope(degrees) (c) rough sand, GPR-conventional (d) sand-covered bedrock, GPR-conventional 1.0 1.0 training data training data test data test data 0.8 0.8 predictivemean predictive mean 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope(degrees) Slope(degrees) (e) rough sand, MS-TGPR (proposed) (f) sand-covered bedrock, MS-TGPR(proposed) Fig. 8 Results of Evaluation 1. Slip predictions with and without the data transferred from multiple source domains are plotted. The red curve represents the predictive mean while the blue-shaded area represents 2-standard deviation around the mean. The blue and orange points represent the training and test data, respectively. The green crosses in GPR-conventional ((c) and (d)) depict the augmented data from a source domain As presented in Table 3, similar results were obtained it is biased by the augmented data from the bedrock for the other two target domains. domain, as illustrated in (d). In contrast, MS-TGPR could improve the slip prediction more than the other Evaluation 2: Influence of source domains two approaches, as demonstrated in (f ) and presented In the second evaluation, the effect of the number of in Table 3. Although the predictive mean was slightly source domains on the performance of the learned model lower than the actual slip at slopes higher than 20 , the was analyzed. The same four surface domains as Evalua - test data was covered by the confidence interval with tion 1 were set as the target domains, with data shallower low KLD. For this target domain, high similarity values ◦ ◦ than 5 adopted for training and that steeper than 5 for were learned for the small rocks and bedrock domains, testing. The number of source domains N was set to 1, as presented in Table 4, so the TGPR models of these 2, 3, 5, or 8 (all source domains). For each case except domains were selected to develop the prediction model. Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 14 of 25 Table 2 RMSEs and LOGLIKs of reference GPR models for slip Table 4 Domain similarity and weight w learned for each i i (Fig. 6) and power consumption (Fig. 7) source domain for the slip modeling in Evaluation 1 Surface domain Slip Power Target domain Source domain w i i RMSE LOGLIK RMSE LOGLIK Rough sand Compacted sand 0.974 0.338 Small rocks 0.929 0.324 Rough sand 0.092 84.690 1.804 – 185.559 Gravel 0.973 0.338 Loose sand 0.049 91.877 1.724 –112.760 Bedrock 0.719 0.000 Compacted sand 0.053 102.406 1.394 –117.295 Gravel with sand Compacted sand –0.419 0.000 Small rocks 0.286 –11.315 3.022 –133.898 Small rocks 0.929 0.325 Gravel 0.145 18.488 2.075 –113.108 Gravel 0.973 0.339 Gravel with sand 0.091 61.798 1.728 –126.181 Bedrock 0.962 0.336 Sand over bedrock 0.215 3.857 1.983 –72.022 Sand over bedrock Compacted sand 0.954 0.250 Sand-covered bedrock 0.159 12.642 1.562 –62.648 Small rocks 0.929 0.244 Bedrock 0.110 17.169 2.267 –56.038 Gravel 0.973 0.255 Bedrock 0.962 0.252 Sand-covered bedrock Compacted sand –0.476 0.000 Table 3 RMSEs, LOGLIKs, and KLDs of slip models learned in Small rocks 0.923 0.490 Evaluation 1. Bold numbers indicate best results among the three Gravel –0.476 0.000 methods for each target domain Bedrock 0.962 0.510 Target domain Method RMSE LOGLIK KLD Rough sand GPR-naive 0.749 –4414.641 366.378 domains more similar to the target domain were included GPR-conventional 0.151 –304.836 39.989 in the source domain sets, e.g., loose sand and compacted MS-TGPR (pro- 0.213 144.511 23.088 sand for the rough sand domain. On the other hand, posed) higher RMSE and KLD resulted when the source domain Gravel with sand GPR-naive 0.489 –2525.338 488.280 sets only included non-similar surface types, e.g., sand GPR-conventional 0.142 102.149 11.091 over bedrock, sand-covered bedrock, and/or bedrock for MS-TGPR (pro- 0.104 131.126 15.669 the rough sand. posed) When the number of source domains increased, the Sand over bedrock GPR-naive 0.572 –2586.805 900.773 variability in the RMSE and KLD decreased, whereas GPR-conventional 0.356 –2178.509 350.095 the median value did not significantly differ depending MS-TGPR (pro- 0.314 –87.785 15.430 posed) on the number of source domains. This is because when Sand-covered GPR-naive 0.460 –2002.584 861.105 the number of source domains increases, the possibil- bedrock GPR-conventional 0.388 –1005.425 421.268 ity of similar domains being included in the source sets MS-TGPR (pro- 0.227 –111.624 7.758 increases. The minimum RMSE and KLD rose when the posed) number of source domains increased. The above results indicate that the improvement of the predictive performance does not depend on the available N = 1 and N = 8 , 20 random combinations of source number of source domains; rather, it depends on whether domains were chosen for learning, and the RMSE and similar domains are included in the sources. The model KLD of each learned model were evaluated. performance does not significantly improve even if the Figure 9 shows the statistics of the RMSE and KLD available source domains increase, but they are limited values of the MS-TGPR models learned with the varied to irrelevant ones. For most of the target domains, the number of source domains for each target domain. Note best model with the minimum RMSE and KLD could that, with N = 8 , only one combination of the source be attained when the model was learned with the single domains is possible. Therefore, the RMSE and KLD are most similar source domain. An exception was the large plotted with a cross that represents the single result for KLD of the sand over bedrock domain, for which no sin- each target domain. gle source domain could achieve an accurate predictive When only one or two source domains were used, both distribution when transferred. However, in actual usage, the RMSE and KLD showed large variability due to the the true most similar domain cannot be known until the difference in the available source domains to transfer. traverse data on slopes are obtained. Therefore, it may Basically, lower RMSE and KLD were achieved when I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 15 of 25 0.8 Rough sand 0.7 Gravelwith sand 0.6 Sand over bedrock 0.5 Sand-covered bedrock 0.4 0.3 0.2 0.1 0.0 1 2 3 5 8 Numberofsource domains 1 2 3 5 8 Numberofsource domains Fig. 9 Results of Evaluation 2. Stats of the RMSE and KLD are plotted for the models learned with various numbers of source domains for the four target domains. The result with eight source domains is represented by the cross. The variability in RMSE and KLD decreases along with the increase in the number of source domains be preferable to learn the model with several available represents a model learned without the threshold, mean- source domains instead of learning with a single, possibly ing that all TGPRs were adopted regardless of similari- similar source domain. ties. In contrast, = 0 excludes domains with negative th similarities, and = 0.9 only accepts domains with very th Evaluation 3: Influence of target training data high similarities. This evaluation assessed the influence of the terrain incli - Example MS-TPGR models trained from different tar - nations included in the target training data on the pre- get datasets for the rough sand domain are presented in dictive accuracy of the slippage. The rough sand, gravel Fig. 10. As shown, the model trained on the target data with sand, sand over bedrock, and sand-covered bedrock up to 2.5 could capture the test slip data on steeper were again set as the target domains. When training the slopes in the confidence interval. While the confidence model for each target, all of the eight surface domains interval, or prediction uncertainty, was large in the except for the target domain itself were adopted as source model, the proposed method could rapidly improve the domains for the transfer learning. All data in the target predictive distribution, which made the confidence inter - domain were first divided into training and test data at val tighter when additional target training data on slightly a ratio of 3:1. Multiple training datasets were then cre- steeper terrains were available. ated from the training data with each dataset contain- Figure 11 shows the RMSEs and KLDs of the models ing the data with upper boundaries of the slope angle learned based on GPR-naive, GPR-augmented, and MS- ◦ ◦ varied from 2.5 to 30 . Prediction models were learned TGPR for the varied maximum slope angles included from each dataset and then evaluated with the test data. in the training dataset. Those of the reference GPR are For MS-TGPR, five different domain similarity threshold also plotted. As shown, the proposed MS-TGPR-based values (−∞ , 0.0, 0.5, 0.8, 0.9) were tested. = −∞ th th KL divergence RMSE Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 16 of 25 training data test data predictivemean ◦ ◦ ◦ Max. slopein train data=2.5 Max. slopeintrain data=5 Max. slopeintrain data=7.5 RMSE=0.312, KLD=26.942 RMSE=0.315, KLD=26.906 RMSE=0.096, KLD=6.905 1.0 1.0 1.0 0.5 0.5 0.5 0.0 0.0 0.0 0 10 20 30 0 10 20 30 0 10 20 30 Slope (degrees) Slope (degrees) Slope (degrees) ◦ ◦ ◦ Max. slopein train data=10 Max. slopeintrain data=15 Max. slopeintrain data=25 RMSE=0.107, KLD=15.375 RMSE=0.100, KLD=7.278 RMSE=0.091, KLD=3.901 1.0 1.0 1.0 0.5 0.5 0.5 0.0 0.0 0.0 0 10 20 30 0 10 20 30 0 10 20 30 Slope (degrees) Slope (degrees) Slope (degrees) Fig. 10 Models trained on different target training data in Evaluation 3. Slip-slope models learned based on the proposed method are shown for a series of increased target training datasets with increased slope steepness. By using additional target training data, the proposed method could rapidly improve the model distribution with a tighter confidence interval. Results for the rough sand surface class are presented. Note that the shape of the model trained on the data up to 5 (upper center) is slightly different from that shown in Fig. 8e because the source domains used for training are not the same method could learn the prediction model with relatively the irrelevant source domains to be transferred, and smaller RMSE and KLD even when the target terrain thus avoided negative transfer. However, when more data was limited to up to 2.5 . The model was improved data were available on steeper terrains, the character- with the additional traverse data on steeper terrains, and istics of the target domains started becoming distinct depending on the target domain, its performance reached from those of the source domains. Therefore, domain almost the same level as the reference GPR model from similarities between the target and sources decreased. the data up to 7.5–15 , which indicates that a signifi - In this regard, the threshold = 0.9 might have been th cantly shallower terrain is required to train MS-TGPR too strict to transfer source models of moderate simi- models than the reference GPR models. larities to construct a good model. This resulted in the Figure 12 shows the RMSEs and KLDs of the models large fluctuation in the performances of the model with learned based on MS-TGPR with various domain simi- the high threshold value. Overall, = 0.5 showed a th larity thresholds . The model learned with = 0.9 relatively better stability in the performance over varied th th achieved a lower RMSE or KLD than the other settings training datasets and various target domains. in the rough sand and sand-covered bedrock domains, especially when the training data were very limited on Evaluation 4: Slip risk classification shallow slopes. This can be attributed to the fact that In this evaluation, the proposed risk classification the high threshold in the domain similarity excluded method was assessed in a scenario where the immobility (See figure on next page.) Fig. 11 Influence of target training data on learned models in Evaluation 3. a RMSE of the slip prediction and b KLD of the distribution are plotted for the models learned from the training data with different upper boundaries of terrain inclination. The results for the four target domains are shown. RMSE and KLD decrease along with the increase of the maximum terrain inclination included in the training data. The proposed MS-TGPR-based method (green) learned the model with relatively smaller RMSE and KLD even when the target data was limited to approximately ◦ ◦ 3 . Its performance was almost as good as that of the reference GPR model with data up to approximately 7.5–15 , depending on the target domain Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio Slip ratio I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 17 of 25 Rough sand Gravel with sand 0.7 0.7 GPR-naive GPR-conventional 0.6 0.6 MS-TGPR (λ =0.5) th 0.5 GPR reference 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slopein training data (degrees) Maximum slopeintraining data(degrees) Sand over bedrock Sand-coveredbedrock 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slopein training data (degrees) Maximum slopeintraining data(degrees) (a) RMSE Rough sand Gravel with sand GPR-naive 800 800 GPR-conventional MS-TGPR (λ =0.5) th 600 600 GPR reference 400 400 200 200 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slopein training data (degrees) Maximum slope in training data(degrees) Sand over bedrock Sand-coveredbedrock 800 800 600 600 400 400 200 200 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slopein training data (degrees) Maximum slope in training data(degrees) (b) KLD Fig. 11 (See legend on previous page.) KL divergence KL divergence RMSE RMSE KL divergence KL divergence RMSE RMSE Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 18 of 25 Rough sand Gravelwithsand 0.40 0.40 MS-TGPR (w/o λ ) th 0.35 0.35 MS-TGPR (λ =0.0) th MS-TGPR (λ =0.5) th 0.30 0.30 MS-TGPR (λ =0.8) th 0.25 0.25 MS-TGPR (λ =0.9) th GPR reference 0.20 0.20 0.15 0.15 0.10 0.10 0.05 0.05 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slope in trainingdata (degrees) Maximum slope in training data (degrees) Sand over bedrock Sand-covered bedrock 0.40 0.40 0.35 0.35 0.30 0.30 0.25 0.25 0.20 0.20 0.15 0.15 0.10 0.10 0.05 0.05 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slope in trainingdata (degrees) Maximum slope in training data (degrees) (a) RMSE Rough sand Gravelwithsand 120 120 MS-TGPR (w/o λ ) th MS-TGPR (λ =0.0) th 100 100 MS-TGPR (λ =0.5) th 80 80 MS-TGPR (λ =0.8) th MS-TGPR (λ =0.9) th 60 60 GPR reference 40 40 20 20 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slope in training data(degrees) Maximum slope in training data (degrees) Sand over bedrock Sand-covered bedrock 120 120 100 100 80 80 60 60 40 40 20 20 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Maximum slope in training data(degrees) Maximum slope in training data (degrees) (b) KLD Fig. 12 Influence of target training data on models with different in Evaluation 3. a MSE and b KLD of the predictive slip models are presented th for various domain similarity threshold values . The results for the four target domains are shown. = 0.5 (green) shows a relatively stable th th performance for varied training datasets KL divergence KL divergence RMSE RMSE KL divergence KL divergence RMSE RMSE I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 19 of 25 1.2 1.2 high risk data high risk data low risk data lowrisk data 1.0 1.0 threshold threshold 0.8 0.8 predictivemean predictive mean 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) (a) p =0.01 (b) p =0.05 threshold threshold 1.2 1.2 high risk data high risk data 1.0 low risk data 1.0 lowrisk data threshold threshold 0.8 0.8 predictivemean predictive mean 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 −0.2 −0.2 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) (c) p =0.10 (d) p =0.50 threshold threshold Fig. 13 Classification results of risk levels in Evaluation 4. The red and green areas represent the classified “ high-risk” and “ low-risk” slopes, respectively. The blue curve represents the predictive slip mean, while the blue-shaded area represents the confidence interval corresponding to the risk probability threshold p . The dashed line represents the slip threshold value y = 0.6 with which the risk level was classified. threshold threshold The red and green points represent the data of high and low risk, respectively. The slip regression model was trained from the target data on slopes shallower than 5 . Lower p makes the risk level boundary shift to a shallower slope and provides more conservative risk prediction. Results threshold for the rough sand surface class are presented risk of a vehicle was predicted. The slip prediction mod - figure, the lower p moved the boundary of the two threshold els trained for the target domains in Evaluation 1 were risk levels toward the shallower slope, indicating a more adopted to classify the risk. Here, as the slip threshold conservative risk assessment. in Eq. (16), y = 0.6 was adopted for classifying Figure 14 presents the confusion matrices correspond- threshold the risk level l into high or low. This threshold value ing to the classification results shown in Fig. 13. With the risk is also utilized in NASA’s Mars exploration rover mis- lower p , more true high-risk data were correctly threshold sions as one of the slip threshold values [30]; detecting classified as high risk, while miss-classification of true a single slip event over this value will immediately stop low-risk samples to high risk slightly increased. the vehicle from driving. Another possible slip threshold The classification performance was quantitatively eval - may include 0.2 or 0.8, where the former provides rigor- uated based on the following recall and precision scores ous risk assessment and the latter may be used when the for the high-risk class. vehicle is required to travel over high-slip terrains. No. of correctly classified high-risk samples In this evaluation, the risk probability threshold Recall = No. of true high-risk samples p in Eq. (16) was varied from 0.01 to 0.50, and the threshold (19) classification performance of the different p vari- threshold ations was compared. The lower the threshold, the more No. of correctly classified high-risk samples Precision = . predictive uncertainty is taken into consideration, with No. of samples classified as high-risk p = 0.50 corresponding to classifying the risk (20) threshold only from the mean slip prediction. Higher recall indicates that a smaller number of high- Example classification results for the rough sand risk samples are erroneously underestimated as low risk. domain are presented in Fig. 13 for p = 0.01, 0.05, threshold On the other hand, higher precision indicates a smaller 0.10, and 0.50. The slopes classified as high and low risk number of low-risk samples are incorrectly classified as are colored red and green, respectively. As shown in the high risk. Generally, recall and precision are in a trade-off Slip ratio Slip ratio Slip ratio Slip ratio Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 20 of 25 1.0 1.0 0.8 0.8 0.990 0.010 0.923 0.077 0.6 0.6 0.4 0.4 0.118 0.882 0.118 0.882 0.2 0.2 0.0 0.0 high risk low risk high risk lowrisk Prediction Prediction (a) p =0.01 (b) p =0.05 threshold threshold 1.0 1.0 0.8 0.8 0.829 0.171 0.678 0.322 0.6 0.6 0.4 0.4 0.059 0.941 0.000 1.000 0.2 0.2 0.0 0.0 high risk low risk high risk lowrisk Prediction Prediction (c) p =0.10 (d) p =0.50 threshold threshold Fig. 14 Confusion matrices in Evaluation 4. Example confusion matrices of the risk level classification for different risk probability threshold p threshold values. The risk level classifiers were tested on the target data of slopes steeper than 5 , which were not used for training the model. With lower p , more true high-risk data were correctly classified as high risk while miss-classification of true low-risk samples to high risk increased threshold slightly. Results for the rough sand surface class are presented relationship, as trying to increase recall can result in a for the above reason, this study adopts recall and preci- conservative classifier in which most of the samples are sion to assess both scores individually rather than using classified as high risk, resulting in low precision. In tra - the integrated F1-score. versal risk assessment, miss-classifications of high-risk Figure 15 presents the recall and precision scores terrains as low risk and that of low-risk terrains as high resulting from the varied risk probability threshold risk cannot be treated equally. While the latter may result p in the four tested target domains. As qualita- threshold in an inefficient operation (e.g., taking a longer route to tively observed in Fig. 13, increasing the risk probability avoid terrains that are actually not hazardous), the for- threshold p results in a lower recall and increased threshold mer can lead to a catastrophic event, such as wheels precision, meaning a higher possibility of miss-classifying embedded in the sand or vehicle turnover. Therefore, it high-risk terrain as low risk, which is undesirable. is preferable for a risk level classifier to obtain high recall This result implies that it may be better to select and moderate-to-high precision scores so that it can p between 0.05 and 0.3 (depending on the mis- threshold probably detect true risky terrains while not over-con- sion) to keep the recall score high while not sacrific- servatively assessing the risk. Note that F-1 score, a har- ing the precision, and to correctly detect high-risk monic average of the recall and precision, is often used terrains with a moderate possibility of incorrectly pre- for evaluating the classification performance. However, dicting true low-risk terrain as high risk. How much True True lowrisk high risk lowrisk high risk True True lowrisk high risk lowrisk high risk I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 21 of 25 Evaluation 5: Learning and prediction of power Rough sand Gravelwith sand Sandoverbedrock Sand-covered bedrock Mean consumption 1.00 The final evaluation assesses how well the proposed 0.95 method can learn and predict a traversability metric 0.90 other than vehicle slippage, specifically, power con - 0.85 sumption. The conditions of the source domains and 0.80 target domains adopted are the same as those used in 0.75 Evaluation 1. The resulting models for power consumption vs. 0.70 slope are presented in Fig. 16. The RMSE and KLD of 0.65 0.0 0.1 0.2 0.3 0.4 0.5 the learned models are listed in Table 5. Overall, the Risk probability threshold models learned on the basis of the proposed method (a) recall Rough sand Gravelwith sand Sandoverbedrock Sand-covered bedrock Mean could addequately predict the power consumption on slopes that were not used for training, as the confidence 1.0 intervals covered most of the test data. 0.8 The reason for the relatively large error on the sand over bedrock domain was the miss-learning of the 0.6 domain similarities and the assignation of a high simi- larity only to the small rocks domain, resulting in a 0.4 model learned only from the TGPR of that domain. The learned domain similarity and weight w of each 0.2 source domain are presented in Table 6. 0.0 0.1 0.2 0.3 0.4 0.5 Risk probability threshold When comparing the learned domain similarities for (b) precision the power consumption model (Table 6) and for the Fig. 15 Classification scores in Evaluation 4. a Recall and b precision slip model (Table 4), it is clear that different similari - scores of varied risk probability thresholds, p , are plotted for threshold ties are learned for these two models. These differences the four target domains. The mean of the four domains is also plotted occurred because the power and slip models were inde- pendently learned. It might be possible to improve the predictive accuracy of both models by learning them in a coupled fashion, where the same similarity coefficient uncertainty should be incorporated also depends on set is assigned to models of different traversability met - the accuracy of the regression model, and on the vari- rics. Adopting a multi-output GP method [32] could be ability of the traversability. one way of accomplishing that. The relatively low precision in the risk classification Overall, the results indicate the possible capability of for the sand-covered bedrock domain is attributed to the proposed method to predict traversability indices the relatively wide confidence interval for the domain. other than vehicle slippage. Note that the slip prediction model was learned from the target training data on slopes shallower than 5 and tested on the data of slopes steeper than 5 . A s Discussion the proposed MS-TGPR-based traversability learn- The above evaluations demonstrate that the proposed ing method can improve the prediction model with method can improve traversability prediction accuracy increased data from slightly steeper slopes, a higher and its predictive distribution when available training risk classification score should be possible with the data are limited to only those obtained on benign terrains improved model. in the target environments. The results also indicate that The above results demonstrate the effectiveness of the required terrains to traverse for obtaining a mature the proposed traversability prediction method for clas- learned model are significantly shallower than those for sifying the risk level from limited target data with a the models trained without transfer learning. simple risk probability assessment. However, classify- If sufficient target domain data for learning the pre - ing the risk level on the basis of a more sophisticated diction model are available, the performance improve- measure, such as conditional value at risk (CVaR), ment by the proposed learning will be very limited. In would provide a better risk assessment [31]. such cases, it is preferable to adopt the basic GPR alone owing to its lower computational cost and better learn- ing stability compared to MS-TGPR. Precision Recall Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 22 of 25 trainingdata trainingdata 25 25 test data test data predictive mean predictive mean 20 20 15 15 10 10 5 5 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) (a) rough sand (b) gravel withsand trainingdata trainingdata 25 25 test data test data predictive mean predictive mean 20 20 15 15 10 10 5 5 0 0 0 5 10 15 20 25 30 0 5 10 15 20 25 30 Slope (degrees) Slope (degrees) (c) sand over bedrock (d) sand-coveredbedrock Fig. 16 Results of Evaluation 5. Prediction of power consumption for the four target domains with MS-TGPR-based approach. The red curve represents the predictive mean, while the blue-shaded area represents 2-standard deviation around the mean. The blue and orange points represent the training and test data, respectively Table 5 RMSEs and KLDs of power consumption models Table 6 Domain similarity and weight w learned for each i i learned based on MS-TGPR in Evaluation 5 source domain for the power consumption modeling in Evaluation 5 Target domain RMSE KLD Target domain Source domain w i i Rough sand 1.974 14.117 Gravel with sand 1.982 8.038 Rough sand Compacted sand 0.999 1.000 Sand over bedrock 5.135 35.221 Small rocks –0.105 0.000 Sand-covered bedrock 2.526 10.421 Gravel –0.466 0.000 Bedrock –0.446 0.000 Gravel with sand Compacted sand 0.999 0.351 One of the limitations of the current approach is that Small rocks 0.740 0.000 the method determines the source-target similarity Gravel 0.937 0.329 from limited target data and assumes that the simi- Bedrock 0.907 0.320 larity does not significantly differ over different ter - Sand over bedrock Compacted sand 0.576 0.000 rain geometries. However, this is not always the case. Small rocks 0.945 1.000 Sometimes high similarity can be assigned to irrelevant Gravel 0.356 0.000 domains, resulting in a low prediction accuracy, and Bedrock 0.719 0.000 also in a wide predictive distribution, or uncertainty. Sand-covered bedrock Compacted sand 0.790 0.000 The large uncertainties can be triggered because the Small rocks 0.945 0.509 predictive distribution is estimated as a mixture of the Gravel 0.628 0.000 distributions of the multiple TGPRs included in the Bedrock 0.907 0.491 model. To improve the traversability prediction, it is important to better estimate the similarity coefficient from the limited data. One possible approach to achiev- ing this would be to use additional feature inputs, such Powerconsumption(W) Powerconsumption(W) Powerconsumption(W) Powerconsumption(W) I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 23 of 25 as exteroceptive sensor information (e.g., visual fea- terrain classes with a high confidence, a traversability tures) or other proprioceptive sensor information (e.g., regression model already learned for that class is adopted vehicle vibration). Adopting a multi-output regression to predict the traversability of the same terrain in a dis- approach might also improve the predictive perfor- tant field. If not, the data is assumed to belong to a novel mance by learning models for multiple traversability terrain class. In this case, a new traversability prediction indices in a coupled manner, thus making the best use model is learned for the novel terrain class by using the of the underlying relationships between these indices. method proposed in this study. A vision-based classifier, As clarified in the evaluations, the predictive capability if available, can be also updated with the newly defined of the proposed method differs depending on the target class at this stage in a self-supervised manner for detect- domain, as well as on the available source domains to be ing the same surface class in distant fields. Once a suf - transferred. The key concept of the proposed method is ficient amount of data samples are collected, the new the extrapolation of limited traverse data on the target terrain class is added as a new source domain for future terrain by interpolating the data from source domains. use. The evaluation results presented in the previous sec - Accordingly, if the target terrain geometry is located out- tion indicate that the model performance and robustness side the data range in the source domains, the proposed can be improved when more source domains are availa- method may fail in its prediction. Similarly, if the terrain ble for learning. This suggests that the proposed method, traversability of the target domain is significantly outside combined with self-supervised terrain classification, can the traversability characteristic envelope covered by the lead to a lifelong learning paradigm [33], as the vehicle source domain data, the improvement of the prediction will become more adaptive to newly encountered terrains accuracy by the proposed method would also be lim- by utilizing more traverse experience on multiple types of ited, as interpolating the data from the source domain terrain in its lifetime. data is no longer possible in such situations. For exam- Furthermore, the proposed prediction method can be ple, because the bedrock class is located at the edges of combined with path planning. In addition to the pre- all domains, the proposed method cannot perfectly pre- dictive mean accuracy, the proposed method showed a dict the traversability for this class. Developing a method capability in improving the predictive distribution, which that can utilize available source domain data when the is crucial for generating a safe path if a significant vari - traversability characteristic of the target domain sig- ability and/or uncertainties in the vehicle motion exist. nificantly differs from that of the source domains is an When combined with a stochastic planning approach important future work. (e.g., [34]), the proposed method can enable vehicles to As discussed earlier, the proposed method learns the be more safely and efficiently operated in novel and/or traversability model in the target domain by learning the difficult off-road environments. similarities between the small amount of target domain data and the data in source domains. Therefore, it is Conclusion fundamentally not applicable if the vehicle has not been This study proposed a transfer regression method to driven on the target environment yet, or when in-situ improve the traversability prediction accuracy when in- data are completely unavailable even on safe terrains. In situ traverse data are only available on gentle terrains. such cases, the operators are required to implement some Combined with the limited in-situ data, the proposed existing terrain classification and/or traversability predic - method leverages past traverse experiences on multiple tion methods using information from on-board extero- types of surface to improve the prediction accuracy of ceptive sensors or orbital/aerial observations. However, the traversability on untraversed slopes. The effectiveness due to the lack of in-situ traverse data, the prediction of the proposed method was demonstrated in evalua- result may not always be reliable enough. Once the vehi- tions using a traverse dataset collected from experiments. cle obtains even a few traverse data on the target envi- The results showed that the proposed method can learn ronment, the proposed method can contribute to better a more accurate prediction model than conventional predicting the traversability on the new environment. methods, both in terms of the predictive mean and dis- There are several directions in which the proposed tribution, from just data on almost flat surfaces of the method can be extended. For example, it could be imple- target domain. In addition, the proposed method can mented as part of an end-to-end learning framework by rapidly improve the model performance with additional combining it with a self-supervised terrain classifica - data obtained from significantly less steep terrains com - tion method (e.g., [9, 10]), in the following fashion. First, pared to methods without transfer learning. These find - traverse data newly obtained in a target environment are ings demonstrate the high adaptability of the proposed classified using a pre-trained proprioceptive terrain clas - method to new environments. sifier. If the data are classified to one of the pre-defined Inotsume and Kubota ROBOMECH Journal (2022) 9:6 Page 24 of 25 Author details The proposed prediction method was also applied to Data Science Research Laboratories, NEC Corporation, 1753 Shimonumabe, traversal risk classification in which the slip distribu - Kawasaki, Kanagawa 211-8666, Japan. Institute of Space and Astronautical tions predicted from the learned regression model were Science, Japan Aerospace Exploration Agency, 3-1-1 Yoshinodai, Sagamihara, Kanagawa 252-5210, Japan. used for binary classification of immobilization risk. The learned regression models showed a reasonable risk Received: 2 April 2021 Accepted: 28 December 2021 detection capability on yet to be traversed slopes. In addi- tion, the proposed method was adopted to learn and pre- dict the power consumption of the vehicle actuators on various terrains. The results demonstrate the applicability References 1. Wong JY (2008) Theory of ground vehicles. Wiley, Hoboken of the proposed method to various traversability indices. 2. Chhaniyara S, Brunskill C, Yeomans B, Matthews M, Saaj C, Ransom S, Although this study evaluated the proposed method for Richter L (2012) Terrain trafficability analysis and soil mechanical property the prediction of the vehicle’s slippage in its longitudinal identification for planetary rovers: a survey. J Terramech 49(2):115–128 3. Papadakis P (2013) Terrain traversability analysis methods for unmanned direction, it can also be used in more general cases where ground vehicles: a survey. Eng Appl Artif Intell 26(4):1373–1385 the vehicle traverses 2.5-dimensional terrain surfaces 4. Wei P, Sagarna R, Ke Y, Ong Y-S, Goh C-K(2017) Source-target similar- with steering actions. In such scenarios, the proposed ity modelings for multi-source transfer gaussian process regression. In: International Conference on Machine Learning, pp. 3722– 3731 method can be adopted to learn and predict the vehicle’s 5. Inotsume H, Kubota T ( 2020) Slip prediction for exploration rover based lateral and/or yaw slippage in addition to the longitudi- on transfer learning. In: 2020 International Symposium on Artificial Intel- nal one with additional input features, e.g., the slope roll ligence, Robotics and Automation in Space (i-SAIRAS) 6. Inotsume H, Kubota T ( 2021) Adaptive terrain traversability prediction angle and roughness as well as the control input to the based on multi-source transfer gaussian processes. In: 2021 IEEE/RSJ vehicle. Moreover, it can be fundamentally applied to any International Conference on Intelligent Robots and Systems (IROS) type of traversability metric, such as vibration or attitude 7. Martin S, Corke P ( 2014) Long-term exploration & tours for energy con- strained robots with online proprioceptive traversability estimation. In: change on rough/deformable terrains. Also, while the 2014 IEEE International Conference on Robotics and Automation (ICRA), evaluation results reported in this paper were for specific pp. 5778– 5785. IEEE vehicles and terrain surfaces, the authors feel that thanks 8. Ostafew CJ, Schoellig AP, Barfoot TD, Collier J (2016) Learning-based nonlinear model predictive control to improve vision-based mobile robot to the general form of the proposed method, the quali- path tracking. J Field Robot 33(1):133–152 tative trends observed in this study will not significantly 9. Brooks CA, Iagnemma K (2012) Self-supervised terrain classification for vary with regard to vehicle size, weight, locomotion con- planetary surface exploration rovers. J Field Robot 29(3):445–468 10. Otsu K, Ono M, Fuchs TJ, Baldwin I, Kubota T (2016) Autonomous terrain figuration, or terrain type. classification with co-and self-training approach. IEEE Robot Autom Lett One possible direction of future efforts would be to 1(2):814–819 investigate ways of improving the proposed method to 11. Wellhausen L, Dosovitskiy A, Ranftl R, Walas K, Cadena C, Hutter M (2019) Where should i walk? predicting terrain properties from images via self- enhance the selection and utilization of source domains supervised learning. IEEE Robot Autom Lett 4(2):1509–1516 for transfer, especially in cases where the target data are 12. Cunningham C, Ono M, Nesnas I, Yen J, Whittaker WL( 2017) Locally- severely limited. Currently, the proposed approach solely adaptive slip prediction for planetary rovers using gaussian processes. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), utilizes proprioceptive data for learning the weight of pp. 5487– 5494 . IEEE each source domain. Adding other sensory information, 13. Skonieczny K, Shukla DK, Faragalli M, Cole M, Iagnemma KD (2019) such as features from IMU data and/or camera images, Data-driven mobility risk prediction for planetary rovers. J Field Robot 36(2):475–491 may be effective in improving the predictive perfor - 14. Cunningham C, Nesnas IA, Whittaker WL (2019) Improving slip predic- mance. Validating the proposed method in fields with a tion on mars using thermal inertia measurements. Autonom Robots longer traverse and in an online learning and prediction 43(2):503–521 15. Oliveira R, Ott L, Ramos F (2016) Active perception for modelling energy fashion is another important direction of future research. consumption in off-road navigation. In: 2016 Australasian Conference on Robotics and Automation (ACRA) Acknowledgements 16. Chavez-Garcia RO, Guzzi J, Gambardella LM, Giusti A (2018) Learn- The authors are grateful to Dr. Masatsugu Otsuki of JAXA/ISAS and Mr. Masa- ing ground traversability from simulations. IEEE Robot Autom Lett toshi Motohashi of the University of Tokyo for their help with setting up the 3(3):1695–1702 mobile test robot adopted for the data collection. 17. Yang Q, Zhang Y, Dai W, Pan SJ (2020) Transfer Learning. Cambridge University Press, Cambridge Authors’ contributions 18. Zhou Z-H (2012) Ensemble methods: foundations and algorithms. CRC HI carried out the main part of this study and wrote the manuscript. TK super- Press, Boca Raton vised and provided advice based on knowledge of the research domain. Both 19. Bekhti MA, Kobayashi Y (2020) Regressed terrain traversability cost for authors read and approved the final manuscript. autonomous navigation based on image textures. Appl Sci 10(4):1195 20. Williams CK, Rasmussen CE (2006) Gaussian processes for machine learn- Declarations ing. MIT press, Cambridge 21. Ho K, Peynot T, Sukkarieh S (2016) Nonparametric traversability Competing interests estimation in partially occluded and deformable terrain. J Field Robot The authors declare that they have no competing interests. 33(8):1131–1158 22. Otsu K, Kubota T ( 2016) Energy-aware terrain analysis for mobile robot exploration. In: Field and Service Robotics, pp. 373– 388 . Springer I notsume and Kubota ROBOMECH Journal (2022) 9:6 Page 25 of 25 23. Cao B, Pan SJ, Zhang Y, Yeung D-Y, Yang Q (2010) Adaptive transfer learn- ing. In: AAAI 2:7 24. Tresp V (2001) Mixtures of gaussian processes. Advances in neural infor- mation processing systems, 654–660 25. Rasmussen CE, Ghahramani Z (2002) Infinite mixtures of gaussian process experts. Adv Neural Inform Process Syst 2:881–888 26. Rosenstein MT, Marx Z, Kaelbling LP, Dietterich TG ( 2005) To transfer or not to transfer. In: NIPS 2005 Workshop on Transfer Learning, vol. 898, pp. 1– 4 27. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York 28. Gonzalez R, Fiacchini M, Iagnemma K (2018) Slippage prediction for off- road mobile robots via machine learning regression and proprioceptive sensing. Robot Auton Syst 105:85–93 29. Endo M, Endo S, Nagaoka K, Yoshida K (2021) Terrain-dependent slip risk prediction for planetary exploration rovers. Robotica, 1–14 30. Rankin A, Maimone M, Biesiadecki J, Patel N, Levine D, Toupet O (2021) Mars curiosity rover mobility trends during the first 7 years. J Field Robot 31. Majumdar A, Pavone M (2020) How should a robot assess risk? towards an axiomatic theory of risk in robotics. In: Robotics Research, pp. 75– 84. Springer, Cham 32. Liu H, Cai J, Ong Y-S (2018) Remarks on multi-output Gaussian process regression. Knowledge Based Syst 144:102–121 33. Thrun S, Mitchell TM (1995) Lifelong robot learning. Robot Autonom Syst 15(1–2):25–46 34. Inotsume H, Kubota T, Wettergreen D (2020) Robust path planning for slope traversing under uncertainty in slip prediction. IEEE Robot Autom Lett 5(2):3390–3397 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub- lished maps and institutional affiliations.
ROBOMECH Journal – Springer Journals
Published: Feb 22, 2022
Keywords: Off-road vehicle; Terrain traversability prediction; Transfer learning
You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.