A comparative analysis of weight-based machine learning methods for landslide susceptibility mapping in Ha Giang area

ABSTRACT Landslide susceptibility maps (LSMs) are very crucial for planning policies in hazardous areas. However, the accuracy and reliability of LSMs depend on available data and the selection of suitable methods. This study is conducted to produce LSMs by combinations of machine learning methods and weighting techniques for Ha Giang province, Vietnam, where has limited data. In study area, we gather 11 landslide conditioning factors and establish a landslide inventory map. Computing the weights of classes (or factors) is very important to prepare data for machine learning methods to generate LSMs. We first use frequency ratio (FR) and analytic hierarchy process (AHP) techniques to generate the weights. Then, random forest (RF), support vector machine (SVM), logistic regression (LR), and AHP methods are combined with FR and AHP weights to yield accurate and reliable LSMs. Finally, the performance of these methods is evaluated by five statistical metrics, ROC and R-index. The empirical results have shown that RF is the best method in terms of R-index and the five metrics, i.e. TP rate (0.9661), FP rate (0.0), ACC (0.9835), MAE (0.0046), and RMSE (0.0350) for this study area. This study opens the perspective of weight-based machine learning methods for landslide susceptibility mapping.


Introduction
Landslides are considered as one of the most devastating geological hazards in mountainous regions across the world (Achour et al., 2017;Pham et al., 2021;Tien Bui, Pradhan, Lofman, Revhaug, & Dick, 2012;Yalcin, Reis, Aydinoglu, & Yomralioglu, 2011;Zhou et al., 2020).The majority of the mainland in Vietnam are hilly regions where landslides affect people through damage of property and loss of life (Hung, Batelaan, San, & Van, 2005;Long & De Smedt, 2012;Trinh et al., 2016).Ha Giang province is one of the most mountainous areas that occur landslides every year (Hung et al., 2016;Khien, Hung, & Long, 2012;Le et al., 2018).Hence, landslide susceptibility mapping is very crucial for the development planning of this province (Ha et al., 2020;Hung et al., 2005Hung et al., , 2017)).Landslide susceptibility can be defined as the probability of a given terrain to yield slope failures (Yalcin, 2008).In other words, landslide susceptibility is expressed as the probability of the landslide occurrence that is caused by a combination of different conditioning parameters in a specific zone (Chalkias, Ferentinou, & Polykretis, 2014;Hong, Pradhan, Xu, & Tien Bui, 2015).
Several landslide susceptibility mapping methods have been proposed (He, Hu, Sun, Zhu, & Liu, 2019;Jaafari, 2018;Mahalingam, Olsen, & O'Banion, 2016;Nguyen & Liu, 2019).Those methods can be classified into two types of methods, i.e. qualitative and quantitative methods (van Westen, Rengers, Terlien, & Soeters, 1997).Analytic hierarchy process (AHP) (Saaty, 1980), a famous example of qualitative methods, is used and developed by many researchers (Gudiyangada Nachappa, Kienberger, Meena, H¨olbling, & Blaschke, 2020;He et al., 2019;Kayastha, Dhital, & De Smedt, 2013;Nguyen & Liu, 2019;Vojtekov ´a & Vojtek, 2020).AHP method works based on field experiences and the physical process with factors relevant to landslide occurrences.Experts assign a suitable weight to each corresponding class (or factor) based on their views about the degree of impact of the class (factor).Quantitative methods are taken to build the relationship between observed landslides and relevant factors (Jaafari, 2018).Frequency ratio (FR) method (Lee & Pradhan, 2006;Rasyid, Bhandary, & Yatabe, 2016;Yalcin et al., 2011) is well-known for the quantitative methods.The likelihood of a given area to landslide is estimated based on the fraction of the number of landslides within the given area over the total landslides in the whole study area, and the ratio between the given area and the whole area.AHP and FR methods are named as weighting methods (He et al., 2019).However, the performance of these weighting methods depends on the quality and availability of data to yield accurate and reliable results.The limitation of recorded landslides and the shortage of relevant conditioning factors are unavoidable in any study area.
In this paper, we carry out a study with the objective to utilize the strengths of weighting methods and statistical machine learning methods to generate the best landslide susceptibility maps (LSMs) for Ha Giang province.Three wellknown methods, i.e. random forest (RF), support vector machine (SVM), and logistic regression (LR), are selected to accomplish LSMs with highly accurate and reliable results of the study area, where landslides occur frequently.
Landslide locations are verified in fieldwork, and the landslide inventory map is established.Eleven relevant factors are selected and provided by Vietnam institute of geosciences and mineral resources (VIGMR). 1 We first select AHP and FR methods to compute the weights of classes.Then, we combine RF, SVM, and LR methods with AHP and FR weights, respectively, to produce LSMs.
We call the combined methods as weight-based machine learning ones.And, AHP method is also used to generate LSMs.Finally, the performance of AHP and weight-based methods is first evaluated by five statistical metrics as well as the ROC curve.R-index is then used to evaluate the correlation between landslide occurrences and landslide susceptibility levels in different LSMs.The empirical results have shown that the FR weights and AHP weights decide the performance of methods, in which RF is the best one.And, other methods with AHP weights produce the worst results, i.e.LR and AHP.Hence, weight-based machine learning methods are very effective for landslide susceptibility mapping and efficient for landslide risk reduction.
The remainder of this paper is organized as follows.Section 2 briefly gives an overview of the study area and conditioning factors.The methodology of this work is presented in Section 3. The results and analysis are shown in Section 4. Discussions of this work are described in Section 5. Finally, Section 6 gives conclusions.

Study area
Ha Giang province is located in the northern region of Vietnam, which covers the mountainous area of approximately 7,900 km 2 .This study area is geographically bounded between the latitudes 22°10 ′ 00 " N and 23°25 ′ 00 " N, and between longitudes 104°20 ′ 00-" E and 105°35 ′ 00 " E, and placed at an altitude of about 37 m to 2427 m above sea level.This province has annual rainfall ranging from 2300 mm to 2400 m, and the humidity of around 85%.The annual temperature of this province fluctuates from 18° to 23°C.Ha Giang province has three main rivers and highly dense streams.Lo river runs from the northwest to the southeast and supplies water to the central of the study area.Chay river starts from Tay Con Linh mountain and provides water for the west of the study area.The east part of this study area is supplied by Gam river.Sedimentary rocks with the principal sedimentary-carbonate components distribute in the southeast edges and the north part of the study area.This part is famous for Dong Van Karst Plateau Geopark, including 90% carbonate areas and sharp reefs.Intrusive rocks distribute in the west and southwest, and metamorphic rocks with rich aluminosilicate appear in the central and southeast of this study area.
Due to the climatic and geo-topographical characteristics, there are two main types of natural hazards in this study area, including flash floods and landslides.According to the report of the disaster management office, dozens of landslide occurrences with different volumes have been recorded every year.This work is carried out to select the best method to yield high accurate and reliable LSMs that can be used for landslide mitigation and prevention, as well as development planning of Ha Giang province.

Data used
This section describes the preparation of the landslide inventory map and a list of landslide conditioning factors.

Landslide inventory map
The preparation of the landslide inventory map is the fundamental step of the process of establishing LSMs in any study area.
We first used Google Earth digitization to detect landslide locations in the study area, then a total of 324 landslide polygons were verified by fieldwork.These polygon sizes change from 90 m 2 to 218,651 m 2 .We also collected 894 recorded landslide points in fieldwork, which was conducted by VIGMR.The majority of these landslide points happened by human modification of slopes (or cut-slope), which occurred along roads.The field survey indicated that most landslides were of shallow, fall, and complex types.The landslide inventory map was generated from a combination of recorded landslide points and verified landslide polygons.This map is illustrated in Figure 1.
Figure 1a presents the landslide inventory and the study area.Figure 1a only contains landslide locations (points and polygons).For generating non-landslide locations, we randomly select locations in the study area that have slope angle with less than 5° (degree).The locations are very difficult to slide.We consider the locations as non-landslides.
Figure 1b and 1c illustrates the detection of one landslide polygon through Google Earth, and this landslide is verified by fieldwork.And, we can see that several houses were damaged by this landslide.Figure 1d describes the location of one recorded landslide occurred along roads.
The recorded landslide locations can provide valuable information to obtain the distribution of landslide occurrences in a specific area, and the locations evaluate a degree of impact from different conditioning factors.Hence, the landslide locations are used to establish a correlation between landslide occurrences and conditioning factors.In this study, we randomly divided occurred landslide locations into two parts: 70% for training (625 points and 226 polygons) and 30% (269 points and 98 polygons) for validation.We first created a number of non-landslides, which were equivalent to the total number of occurred landslide locations.And then, 70% of non-landslides were used for the training part, and the rest were used for the validation part.

Landslide conditioning factors
According to many previous works (Gudiyangada Nachappa et al., 2020;He et al., 2019;Park & Kim, 2019), selecting the conditioning factors depends on data availability and experts' knowledge.Hence, we selected 11 factors for the landslide susceptibility mapping in the study area, i.e. elevation, slope, aspect, curvature, geology, weathering crust, land-cover, distance to road, road density, distance to fault, and fault density factors.These 11 maps were provided by VIGMR.
The slope, aspect, and curvature factors were derived from a digital elevation model (DEM) with 20x20m cell size .The geology and fault factors were with the scale of 1:200,000.The weathering crust and land-cover factors were with the scale of 1:50,000.These factors and the road factors were converted into raster maps with a 20 m resolution.These 11 factors are shown in Figure 2.

Slope.
Slope angle (slope) is expressed as one of the most crucial features of slope stability analysis.In other words, slope is engaged with the occurrence of landslides (Yalcin, 2008).Slope is adopted as an essential factor to creating landslide susceptibility prediction models.The slope factor of this study area was first derived from DEM, and then the natural breaks method was used to classify this factor into five classes: 0 − 11. 87, 11.87-22.75, 22.75-32.31, 32.31-43.20, and 43.20-84.09degrees (Figure 2b).

Aspect.
Aspect factor is described as another important parameter of slope features that influences the occurrence of landslides.The factor is directly associated with the process of evapotranspiration and moisture in hilly areas (Meten, PrakashBhandary, & Yatabe, 2015).Thus, aspect factor is also treated as an important conditioning one.We classified this factor into nine classes as follows: Flat (−1°  2c).

Curvature.
Curvature factor is also included in slope analysis and often used in many studies of landslide susceptibility.Streams and rains erode the curvature by time; hence, this factor relates to the divergence and convergence of landslide materials as well as displacement (Carson & Kirkby, 1972;Kaur et al., 2019).We classified the curvature factor into three classes: concave (<−0.5),flat (−0.5 − 0.5) and convex (>0.5) (Figure 2d).

Land-cover.
Land-cover is a different conditioning factor for landslide occurrences.The absence or presence of vegetation layers affects the stability of slopes (Barlow, Martin, & Franklin, 2003).This factor was also selected in existing works (Nguyen & Liu, 2019;Wang et al., 2020).The land-cover factor was classified into seven classes, which were rivers and lakes, barren lands, populated lands, natural forests, planted forests, cultivated lands, bushes, and shrubs (shown in Figure 2e).

Road density and Distance to road.
The majority of the landslide inventory points in this work that were recorded occurred along roads.The majority of this study area is mountainous terrains and hilly reefs; hence, the road factors are treated as conditioning factors.Distance to road factor is considered as a causative factor in several works (Wang et al., 2020;Yalcin, 2008;Yalcin et al., 2011).Hence, distance to road factor is taken to be a contributive factor.This factor was categorized into five classes, i.e. 0 − 20, 20 − 40, 40 − 60, 60 − 80, and >80 m, and illustrated in Figure 2h.Road density factor was used to establish the correlation between landslide points and roads.This factor was demonstrated in Figure 2i with five classes: very low, low, moderate, high, and very high.

Fault density and distance to fault.
Faults are also considered as a factor that causes landslide occurrences (Hung et al., 2016;Nampak, Pradhan, & Manap, 2014).In this study, we obtained two factors, Distance to fault and Fault density, from the faults map.Distance to fault factor was generated with five classes as follows: 0-100, 100-200, 200-300, 300-400, and >400 m from fault lines.And, fault density factor was established with five classes: very low, low, moderate, high, and very high.These two factors are shown in Figures 2j  and 2k.

Correlation between factors
We use Pearson coefficient to compute correlations between conditioning factors.

Methodology
In this section, we describe the methodology in our study.Figure 4 illustrates the flow chart of our methodology; it consists of five steps as follows.
Step 1: Data preparation.We prepare the landslide inventory map and select available conditioning factors.This step is described in Section 2.
Step 2: Weighting.Frequency ratio (FR) and analytic hierarchy process (AHP) methods are used to compute weights of classes, respectively.Step 2 is presented in more details in the following Section 3.1.
Step 3: Machine learning methods.Three well-known machine learning and one weighting method are selected to generate landslide susceptibility maps (LSMs) with high accuracy and more reliability.These methods are random forest (RF), support vector machine (SVM), logistic regression (LR), and AHP.The four methods are combined with AHP and FR weights (in Step 2) to produce LSMs.This step is detailed in Section 3.2.
Step 4: Validation.In this step, both training and validation parts are taken into account to evaluate the performance of comparison methods.Section 3.3 describes the details of elevation metrics.
Step 5: Landslide susceptibility maps.This step is discussed in Section 4.

Frequency Ratio (FR)
FR method is a simple weighting one that is generally applied to landslide susceptibility analysis (Mahalingam et al., 2016;Yalcin et al., 2011).This method is used to compute the fraction of the probability of landslide occurrences and that of landslide non-occurrences for each conditioning factor (Wang & Li, 2017).For each conditioning factor j, we calculate the weight for class i in the factor j with respect to the occurrence of landslides, denoted by FR j i , in the following Equation 1.
where n i j is the number of landslide pixels of class i in factor j, N is the whole number of landslide pixels,S i j is the whole pixels of class i and S is the total of pixels in the study area.If the value of FR j i is larger than 1, that means the class i is more susceptible to landslide occurrences.Otherwise, the lower value indicates that this class is not relevant to landslide occurrences.
In this study, FR is used to obtain the weights of classes of factors from the training part.And, FR weights are shown in Table 1, Table 2, Table 3.

Analytic hierarchy process (AHP)
AHP method was developed and improved by Saaty (Saaty, 1980;Saaty & Vargas, 1984).This method is described as a decision-making process to address complicated issues (Saaty, 1980).In other words, AHP method is also considered as a semi-quantitative one, which is based on the knowledge of experts who are involved to solve complicated problems.AHP method has been used in many studies about landslide susceptibility mapping (Kayastha et al., 2013;Nguyen & Liu, 2019;Trinh et al., 2016).To yield LSMs, AHP process is divided into five following steps: Step 1: Conditioning factors are included in the discussion process, which is conducted by experts.
Step 2: Constructing a hierarchical model.
Step 3: Establishing a pairwise matrix between factors (classes) bases on the comparison of factors.The pairwise comparisons are described in Table 1.
Step 4: The eigenvector of the pairwise matrix is first computed, then weights of factors are obtained.Step 5: In AHP process, the consistency ratio (CR) is used to validate the adjustment of the pairwise matrix.First, the consistency index (CI) is computed as Equation 2, then CR is calculated as Equation 3. If the value of CR > 0.1, the process will return to Step 2 to reconstruct the model and readjust the pairwise ranking between factors.Otherwise, the weights in Step 4 are used to assign to corresponding factors.
where λ max is the largest eigenvalue, and d is the order of the comparison matrix in Step 3.
where RI is the corresponding value of the random index to the order of the matrix, and obtained from Table 2.
In this study, we use AHP method to compute both the weights of classes and the weights of factors, and these AHP weights are shown in Table 3.

Machine learnings methods
FR weights and AHP weights of classes are used in three machine learning methods, which are described in the following sections.

Logistic regression
Logistic regression (LR) method is used to establish a multivariate regression relationship between a dependent variable and a set of independent variables (Atkinson & Massari, 1998).This method is very effective to predict the presence or absence of an object based on a set of values of independent variables (Bai et al., 2010).Hence, this method is applicable to the dependent variable that is binary or dichotomous.Logistic regression method has been widely used in landslide susceptibility mapping (Bai et al., 2010;Rasyid et al., 2016;Vojtekov´a & Vojtek, 2020).The dependent variable (y) is the absence (0) or the presence (1) of a landslide occurrence.Given a set of independent variables x, the conditional probability of the occurrence of a landslide occurs is denoted by P(y = 1|x).The logit of the logistic regression is expressed as Equation 4: where b 0 is the intercept of the equation, and b 1 , . ..,b n are the coefficients of independent variables x 1 , x 2 , . ..,x n .The probability P(y = 1|x) is computed in the LR method as follows: where e is the exponential value, and is 2.718.

Random forest
Random forest (RF) method proposed by Breiman (Breiman, 2001) is an ensemble learning method of supervised learning, which is used to address problems of classification, regression, and high dimensional data (Trinh et al., 2016).RF has been used to predict the occurrence of landslides in many studies (Kaur et al., 2019;Park & Kim, 2019).RF method can be briefly explained as follows: Given a training data set T = {(x i ,y i )} M i=1 , M is number of samples in the training set T, and x i is the set of n features (factors), y i ∈ Y = {0,1} which is the absence (0) or the presence (1) of a landslide occurrence.RF model is generated as following steps: Step 1: The bagging method (Breiman, 1996) is used to produce K subset bootstraps B.
Step 2: For each B, a corresponding decision tree is formed.At each node of the decision tree, this method randomly samples mtry features to partition B and selects the best split based on Gini measure to generate children nodes.This process continues until all leaf nodes are obtained, and the decision tree is constructed.mtry is given, and often set to (k) Fault density .The whole K trees form the random forest model.
Step 3: A new object comes to the RF model; the value of this object is computed by the average of all values of K individual trees.
In the empirical analysis, we set K = 100, and mtry = 4.

Support vector machine
Support vector machine (SVM) was proposed and developed by Vapnik (Cortes & Vapnik, 1995;Vapnik, 1995).SVM is a well-known supervised learning method that works based on the identification of an optimal separating hyperplane.SVM method has been applied to produce landslide susceptibility maps in may works (Hong et al., 2015;Nhu et al., 2020).
Given data set T = {(x i ,y i )} M i=1 , SVM method splits this data into a high-dimensional feature space x, then the optimal hyperplane will be determined to classify y as landslides or nonlandslides.This hyperplane is formed by a set of support vectors.In this paper, we use the radial basis function kernel (RBF) in the SVM model, RBF can be described as the following equation.
where γ is the gamma parameter.

Comparison methods
We first use two different weighting methods, FR and AHP, to compute the weights of classes of each factor.Then, four methods (AHP, RF, SVM, LR) are combined with FR and AHP weights, respectively.Hence, we obtain eight different methods which are used to generate eight different landslide susceptibility maps.The eight methods are listed below: The four methods use FR weights as follows: RF-FR: Random forest method with FR weights LR-FR: Logistic regression method with FR weights SVM-FR: Support vector machine method with FR weights AHP-FR: AHP method with FR weights The other four methods use AHP weights as follows: RF-AHP: Random forest method with AHP weights LR-AHP: Logistic regression method with AHP weights SVM-AHP: Support vector machine method with AHP weights AHP-AHP: AHP method with AHP weights Platform.ArcGIS 10.3 and RStudio softwares were used in experiments.All methods were implemented in R and executed on a Window 10 machine with 3.4 GHz dual-core CPU and 16 GB memory.

Evaluation metrics
Statistical metrics.Five statistical metrics are used to make the comparison of performance of the eight different methods.Those five metrics are true positive rate (TP rate), false positive rate (FP rate), accuracy (ACC), mean absolute error (MAE), and root mean squared error (RMSE).TP rate, FP rate, and ACC metrics are computed from the confusion matrix (shown in Table 4).These three metrics are defined as the following equations.
where TP, TN, FP, and FN are obtained in Table 4. P and N is the number of landslides and the number of non-landslides, respectively.TP is the number of occurred landslides, which are predicted correctly.FN is the number of occurred landslides, which are predicted incorrectly.FP is the number of non-landslides, which are predicted incorrectly.TN is the number of non-landslides, which are predicted correctly.
These other two metrics, MAE and RMSE, are widely used to measure accuracy for regression models.Therefore, MAE and RMSE are adopted to evaluate the differences between the real values and predicted values.These two metrics are defined in Equation 10 and Equation 11, respectively.where y i and y predicated i are the actual values and the predicted values.M is the number of samples.
Receiver operating characteristics and AUC.The receiver operating characteristics (ROC) is another measure that is widely used to validate the performance of machine learning models.ROC curve demonstrates the percentage of true positive rate and the percentage of false negative rate.The area under the ROC curve (AUC) is considered as a measure to compare the performance of classifiers (Bradley, 1997).
Relative landslide density (R-index).To verify the reliability of susceptibility levels with landslide occurrences, we use the index of relative landslide density, denoted by R-index.R-index is defined by the ratio between the density of landslide occurrences of a given susceptibility level and the area of this given level (Baeza & Corominas, 2001).This index is defined as Equation 12.
where l i is the percentage of occurred landslides within a given susceptibility level i and L i is the percentage of area occupied by the level i.Higher value of R-index indicates that the level i is more susceptible to landslide occurrences.

Results and comparative analysis
Experiments were conducted on both the training and the validation parts.

Comparison of five statistical metrics
The performance of models was evaluated based on the training and validation parts.The five statistical metrics are shown in Table 5.The performance of models is good, if the values of TP rate, ACC are close to 1, and the values of FP rate, MAE and RMSE are towards 0.
In the training part, the results of FP rate metric produced by eight methods are very precise and close to 0. RF-FR and RF-AHP methods produce the best values of TP rate, ACC, MAE, and RMSE metrics, and these four statistical results of the two methods are not much different.RF-FR takes the best values of TP rate with 0.9661 and ACC with 0.9831, and the In the validation part, all eight methods still yield the best values of FP rate metric.RF-FR and RF-AHP are still the best methods in terms of TP rate, ACC, MAE, and RMSE statistical metrics.However, the results of these metrics produced by RF-AHP and RF-FR are quite different.RF-AHP takes the first place of these four statistical metrics, followed by RF-FR.LR-FR and SVMAHP methods occupy the next places.Similar to training part, LR-AHP and AHP-AHP are still the worst methods in terms of the four metrics for validation part.

Comparison of ROC and AUC, t-test
The ROC and AUC of eight methods are illustrated in Figure 5.The performance of these methods is assessed by the accuracy of AUC.The AUC values change from 0.5 to 1.0.The values closer to 1 indicate a more accurate method.
We can observe that RF-AHP and RF-FR methods are the best methods for training and validation parts.In contrast, LR-AHP and AHP-AHP result in the worst performance.
We use t-test with the 0.05 significance level to evaluate the statistical significances between methods.Table 6 shows the p-value of t-test for each pairwise comparison.Through this table, we can see that all p-value between each pairwise comparison is less than the critical value of 0.05, except the pair RF-AHP and RF-FR.Through the comparison of t-test, we find out that there are statistical differences between all methods except the pair RF-AHP and RF-FR.

Comparison of R-index
R-index is used to evaluate the relative landslide density between landslide susceptibility maps (LSMs) and the number of observed landslides.Higher values of R-index indicate that LSMs are more accurate and reliable.The natural breaks classification method in ArcGIS is selected to classify landslide susceptibility maps into five susceptibility levels, which are very low, low, moderate, high, and very high.The LSMs with these five levels are demonstrated in Figure 6. Figure 7 demonstrates the percentage of areas that are occupied by the five susceptibility levels.Figure 8 illustrates the percentage of occurred landslide locations within the five different levels for both training part and validation part.The results of R-index are shown in Figure 9.
We can observe that the RF-FR and RF-AHP methods produce the highest R-index value of the very high susceptibility level compared to the other methods for both training and validation parts.The R-index value indicates that the very high level of LR-AHP is relevant to landslide occurrences.However, SVM-FR yields the lowest value of R-index for the very high level.It can be explained as follows: the percentage of area occupied by the very high level is the highest in SVM-FR (illustrated in Figure 7); and the percentage of landslide locations in the very high level is not much higher than that of the high level (shown in Figure 8).
We can conclude that higher levels of LSMs generated by RF method are very relevant to the occurred landslide locations.The R-index results of RF method are more accurate and reliable than those of the other methods for both training and validation parts.

Discussion
Identifying susceptible areas with landslide occurrences is one of the most critical issues in land management and plan makers for civil protection.The objectives of many works are to model landslide susceptibility, and evaluate the performance of models.Moreover, a method that is selected need to base on specific scientific objectives of the study (Elith & Leathwick, 2009), such as the accuracy and the reliability of LSMs.Many single and hybrid methods have been developed to model landslide susceptibility (Catani et al., 2013;Chen et al., 2018;Hong et al., 2015;Nhu et al., 2020;Pham et al., 2019;Tien Bui et al., 2019); however, there are room for improvement by utilize the strengths of weighting methods, particularly expert's knowledge, and effectiveness of machine learning techniques.
This paper aims to make a comparative study of four methods that are combined with weighting methods for landslide susceptibility mapping in Ha Giang province, Vietnam.The type of landslide occurrences in the study area are mostly shallow landslides.Furthermore, the landslide locations are recorded and verified along and not far to roads.Thus, weathering crust, geology, slope factors have the highest values of AHP weights due to experts' opinion.The comparisons of different methods in terms of TP rate, FP rate, ACC, MAE, RMSE, and AUC have shown that RF, SVM methods are very efficient and effective for landslide susceptibility mapping for this study area.RF method produces the best performance for landslide susceptibility mapping.RF is an ensemble method and computes the value of each object (landslide or nonl-andslide) by averaging all results from many trees in the forest.Therefore, this model can enhance the accuracy of predicted values.And this conclusion is also in agreement with the finding of the works (Catani et al., 2013;Chen et al., 2018;Kaur et al., 2019;Park & Kim, 2019).
The performance of SVM depends on the different values of features (or classes in each factor).If the values are much different, SVM works very effectively.The values of AHP weights range in 0 and 1, and the values of FR weights change from 0 to 5 (seen in Table 3).That is why the results of SVM-AHP are higher than those of SVM-FR.And the results of SVM model are reasonable in modeling landslide susceptibility (Goetz et al., 2015;Tien Bui, Tuan, Klempe, Pradhan, & Revhaug, 2016).
The performance of LR method also depends on features.However, LR try to predict the value of target classes (0 (non-landslide) and 1 (landslide)) by an equation, i.e.Equation 5. Thus, if the values of features are much similar, LR predicts more accurately.Hence, LR-FR performs better than LR-AHP.These things are also found in the work (Trigila et al., 2015).
AHP method results in worse outcomes.Because AHP method works based on the opinions of experts; therefore, the results of this method are in favour of factors that are assigned to higher weights.In contrast, RF, LR and SVM overcome this issue, except LR with AHP weights.
In our study, the strengths of expert's knowledge are utilized to compute the weights of classes of each conditioning factor.Because the distribution of collected landslides is often not equal and balanced in all classes of factors.
Moreover, the size of study area and the scale of factor maps are also common problems.Thus, these facts lead to a bias in results of models.Hence, the valuable knowledge is very critical to adjust the important degree of classes of each factor.In our work, we can solve these problems by a combination of the knowledge of experts and effectiveness of machine learnings methods (Trinh et al., 2020(Trinh et al., , 2016) ) in addressing noise and data over-fitting.Therefore, applying machine learning methods (for example, RF, SVM and LR) is necessary to get better results and to reduce the bias in favor of experts' opinions for factors.Moreover, the combinations of machine learning methods and weighting methods (FR and AHP) should be used to gain the best results.In addition, the landslide inventory map is the vital key for landslide susceptibility mapping.First, landslide locations that were collected and verified in the field trip are correct and relevant to highly susceptible level areas, which are illustrated in Figure 9. Second, investigating and collecting the information of landslide locations in fieldwork are the most critical steps for landslide susceptibility mapping.
This study has some limitations.First, the landslide inventory map contains landslide locations that are verified in accessible areas.Hence, these landslides can not represent all characteristics of all classes in each conditioning factor.Second, the total of verified landslide areas (or cells) is too small compared to the whole study area with 7,900 km 2 .
Moreover, the available datasets and the choice of landslide conditioning factors are limited and based on experts' knowledge.Although the results of R-index have shown that higher susceptibility level areas are more relevant to landslide occurrences, we still need to collect more historic landslides with different shapes across this study area to confirm the accuracy and reliability of LSMs.

Conclusion
In this paper, we presented a comparative study of machine learning methods with weighting techniques for landslide susceptibility mapping for the entire of Ha Giang province.We used landslide locations collected and verified from the fieldwork as the landslide inventory map, in which 70% of them were used for training part and 30% were treated as validation part.The 11 conditioning factors were selected for this study.AHP and FR methods were first used to compute weights of classes of each factor.Then, RF, SVM, LR, and AHP methods were integrated with AHP and FR weights to generate landslide susceptibility maps.The performance of these methods was evaluated by several metrics: TP rate, FP rate, ACC, MAE, RMSE, AUC, as well as R-index.The results of training and validation parts from these methods have shown that RF is very effective for landslide susceptibility mapping for this study area.AHP-AHP and LR-AHP produce the worst performance.As a result, we recommend RF method to generate LSMs to reduce and prevent the impact of landslides in this study area and other areas with similar contexts.
Thi Hai Van Nguyen received the MSc degree in ecological marine management from Free University of Brussels, Belgium.She is currently a principal researcher with the Center for Remote Sensing and Geohazards, Vietnam Institute of Geosciences and Mineral Resources.She has been a leader of many projects, including institute levels and ministry levels.

Figure 1 .
Figure 1.Location of the study area and the landslide inventory map.
Figure 3 illustrates Pearson correlations between these factors.If the value of Pearson is 1, that means two factors can represent to each other.We can observe that the maximum of Pearson values from Figure 3 is approximately 0.70.Hence, all factors they are independent to each other, and they can not represent to each other.

Figure 4 .
Figure 4.The flow chart of the methodology for landslide susceptibility mapping.

Figure 5 .
Figure 5. statistics for the methods used for landslide susceptibility mapping.(a) Training.(b) Validation.

Figure 8 .
Figure 8. Distribution of landslides percentage in five susceptibility levels in eight methods.

Figure 9 .
Figure 9. R-index in five susceptibility levels for training and validation parts.

Figure 7 .
Figure 7.The percentages of areas occupied by five susceptibility levels.
She published several papers on her interests.Her research includes climate change, natural disasters management, geohazards, and landslides.Khanh Quoc Nguyen received the PhD degree in remote sensing and geomatics from University of Greifswald, Germany.He is currently a Director of Center for Remote Sensing and Geohazards, Vietnam Institute of Geosciences and Mineral Resources.His research includes remote sensing, GIS, climate change, natural disasters management, and geohazards.Lien Thi Nguyen received the MSc degree in disaster management from Thuyloi University, Vietnam.She is currently a researcher with the Center for Remote Sensing and Geohazards, Vietnam Institute of Geosciences and Mineral Resources.Her research includes forecasting disasters and climate change. /orcid.org/0000-0002-6973-9749

Table 3 .
FR weights and AHP weights.

Table 5 .
Performance of comparison methods for training and validation parts.values of MAE (0.0045) and RMSE (0.0347) are occupied by RF-AHP.The other methods take the next places, in which LR-AHP and AHP-AHP produce the worst performance, especially, the values of TP rate are 0.3535 and 0.3839 for LR-AHP and AHP-AHP, respectively. best

Table 6 .
Pairwise comparison of methods in terms of t− test.