Comparative analysis of hybrid models of firefly optimization algorithm with support vector machines and multilayer perceptron for predicting soil temperature at different depths

Comparative analysis of hybrid models of firefly optimization algorithm with support vector machines and multilayer perceptron for predicting soil temperature at different ABSTRACT This research aims to model soil temperature (ST) using machine learning models of multilayer perceptron (MLP) algorithm and support vector machine (SVM) in hybrid form with the Firefly opti-mizationalgorithm,i.e.MLP-FFAandSVM-FFA.Inthecurrentstudy,measuredSTandmeteorologicalparametersofTabrizandAharweatherstationsinaperiodof2013–2015areusedfortrainingandtestingofthestudiedmodelswithoneandtwodaysasadelay.Toascertainconclusiveresultsforvalidationoftheproposedhybridmodels,theerrormetricsarebenchmarkedinanindependenttestingperiod.Moreover,Taylordiagramsutilizedforthatpurpose.Obtainedresultsshowedthat,inacaseofonedaydelay,exceptinpredictingSTat5cmbelowthesoilsurface(ST5 cm ) at Tabriz station, MLP-FFA produced superior results compared with MLP, SVM, and SVM-FFA models. How-ever, for two days delay, MLP-FFA indicated increased accuracy in predicting ST 5cm and ST 20cm of Tabriz station and ST 10cm of Ahar station in comparison with SVM-FFA. Additionally, for all of the pre-scribed models, the performance of the MLP-FFA and SVM-FFA hybrid models in the testing phase was found to be meaningfully superior to the classical MLP and SVM models.


Introduction
Soil temperature (ST) and its spatial and temporal changes directly or indirectly affect the extent and direction of many processes occurring in soil (Mehdizadeh et al., 2018) such as seed germination, root elongation, evaporation, storage and movement of water and microbial activities, nutrient cycle, and many other dynamic processes of the soil (Beltrami, 2001;Citakoglu, 2017;Qian et al., 2011). Soil temperature is affected by several factors, including topography, solar radiation, air temperature, precipitation, soil moisture content and soil thermal conductivity, and heat transfer coefficients (Bilgili, 2010(Bilgili, , 2011Hillel, 1998). ST at various depths may be either directly measured or estimated by air temperature modeling. Therefore, methods that can provide acceptable results may be a suitable solution for estimating ST at the needed locations as well as predicting for subsequent months. The use of artificial neural network models has been widely considered in recent years (Chau & Muttil, 2007;Cheng et al., 2005;Fotovatikhah et al., 2018;Kazemi et al., 2018;Kim & Singh, 2014;Kisi et al., 2015;Mazou et al., 2013;Moazenzadeh et al., 2018;Qasem et al., 2019;Samadianfard et al., 2019;Samadianfard, Ghorbani, et al., 2018;Wu et al., 2013;Yaseen et al., 2019). So far, many studies have been carried out on soil temperature estimation. Using the numerical method, Hanks et al. (1971) estimated ST as a function of time and depth. In this study, the calculated temperature difference was 1.5°C with the actual temperature. Zheng et al. (1993) using air temperature and applying linear regression, estimated ST at a depth of 10 cm under six climate types in the United States. For ST estimation, Plauborg (2002) offered experimental and straightforward relationships. The outcomes revealed that the experimental model with a correlation coefficient of 0.98 could predict ST precisely. Gill et al. (2006) applied SVM for ST estimation and compared its results with the estimations of ANN models. They stated that SVM models achieved better ST estimations than ANN models. Sabziparvar, Tabari, et al. (2010) used multivariate regressions for ST estimation at eight selected meteorological stations of Iran using climatic variables. The results indicated that RMSE varied from 2.06 to 2.75 (°C). Moreover, Sabziparvar, Zare Abyaneh, et al. (2010) conducted a study on ST soil temperature estimation at Zahedan and Ramsar stations using multivariate regression. In the best and worst cases, the correlation coefficient was 0.94 and 0.64, respectively. Bilgili (2010) compared regression and artificial neural network models in predicting ST. A neural network with a precision of 0.998 (°C) has more exceptional performance than regression methods. Using the artificial neural network and monthly meteorological data, Bilgili (2011) predicted ST in the next month. His results exhibited that the neural network is a trustworthy model for predicting soil temperature. Tabari et al. (2011) investigated the performances of ANN for estimating ST values and stated that the estimates by ANN were in better agreement with observations than those by regression methods. Tabari et al. (2015) used the neural network (ANN) and multivariate regression (MLR) using a large number of meteorological variables and estimated soil temperature of up to 100 cm depths at five stations located at a dry desert area of Iran. The results of this study revealed that among the atmospheric variables, average temperature and relative humidity had the most significant effects on soil temperature. Samadianfard, Asadi, et al. (2018) investigated the effectiveness of wavelet neural networks (WNN) for estimating ST at various depths. Obtained results exposed that the WNN produced more precise predictions. Additionally, they concluded that with increasing soil depth, the estimation accuracy for all methods reduced drastically. Sanikhani et al. (2018) implemented monthly ST estimation utilizing extreme learning machine (ELM), ANN and M5 Model Tree. Obtained results revealed that the ELM model was produced as the most precise estimations of ST compared to the other considered models. In another study, Sihag et al. (2019) examined the precision of MLP, Gaussian Process (GP), Random Forest (RF) and the M5P models in estimating ST 5cm at Tabriz and Ahar stations. They only evaluated the accurateness of the considered models in estimating ST 5cm and stated that MLP had more accurate predictions than others.
Although ST is recognized as an effective variable in environmental and agricultural practices, only a few research works have developed data-driven forecasting models for its prediction. By this basic recognition and a need to perform such research in semi-arid conditions, the contemporary explorations in this area have intended to study the association among ST and other meteorological parameters using novel and innovative hybrid learning models such as SVM-FFA model. The utility of such research is expected to have significant implications in maintaining the ecological sustainability of the environment and empowering agricultural productivity, including a pivotal role in formulating more strategic soil management policies by decision-makers in the agricultural sector.
Considering those mentioned above, the purpose of this study is to: (1) construct and evaluate MLP-FFA and SVM-FFA models in predicting the ST at different depths, (2) apply the mentioned models trained with data from Tabriz and Ahar synoptic stations with meteorological parameters, (3) evaluate different modeling scenarios that represent the different combinations of meteorological parameters established to study their contributory roles in modeling ST at different depths in the study region.

Multilayer perceptron neural network
The artificial neural network was first presented in 1943 by Mcculloch and Pitts (McClelland & Rumelhart, 1988). Generally, the layered perceptron neural network structure consists of input, output, and hidden layers, each with a different number of neurons. The number of the mentioned layers is related to the nature of the considered problem, and the number of hidden layers is selected by trial and error procedure for minimizing the prediction error (Firat & Gungor, 2009;Ustaoglu et al., 2008). Additionally, the effect of each variable is determined by giving a weight to every neuron of the input layer (Ghorbani et al., 2016) and hyperbolic-tangent sigmoid function, Logarithmic sigmoid function, and Linear function were utilized as the activation function for input, hidden and output layers, as presented in equations 1-3, respectively.

Support vector machines
Support vector machine (SVM), as one of the learning methods, is resulted from statistical learning theory (Vapnik, 1995). The mentioned method utilizes a decision surface for splitting target classes by maximizing their margins (Burges, 1998). This decision surface is typically called an optimum hyperplane and the nearest data point to this plane are named support vectors. Notably, these support vectors are vital elements of the training process. The form of the SVM model is described as following: where w and b represent the coefficients of the weight vector.

Firefly algorithm
Yang (2008) presented an optimization technique based on the movement of fireflies and named it as 'firefly algorithm' (Ghorbani et al., 2017;Yang, 2008). Fireflies live in nature as in colonies and always a firefly with less light travels toward the light and brilliant firefly (Gazi & Passino, 2004). The brightness is proportional to the distance from the source and attractiveness changes with absorption (Fister et al., 2013).
In the above relation r is the distance, I s is the intensity of the light source, for an environment with a constant light gain coefficient γ the intensity of light I change with r (Fister et al., 2013).
That I 0 is the intensity of the original light.
Because the attraction of a firefly is relative to the observed light intensity by the nearby fireflies, we can define the charm of β as follows (Fister et al., 2013): That β 0 is the charm in r = 0. In a real implementation, the charm function β(r) can be any descending uniform function, as in the following general form (Fister et al., 2013): It should be noted that for improving the performance of ST estimations, FFA was utilized for selecting optimized values of the SVM parameters. So, Figure 1 shows the FFA algorithm based on the ANN model.

Study area
This study was performed using data from Tabriz and Ahar stations (Figure 2), located in East Azerbaijan province, Iran. The mentioned province is positioned in the range of 45 • 07 to 48 • 20 east longitude and 36 • 45 to 39 • 26 north latitude and is ranked first in the northwest of Iran regarding agriculture production. The meteorological variables studied in this research are listed in Table 1. Moreover, typical earth thermometers were utilized for measuring ST values at different depths. Table 2 depicts the statistical parameters of all data. It comprehended from this table that T, ST and RH show normal distributions with low skewness values. Also, the   correlation coefficients between meteorological parameters and ST values presented in Table 3. The air temperature and relative humidity have the most significant direct and inverse correlations with ST values, respectively. Figure 3 demonstrates the ST variations at all depths.

Evaluation criteria
For evaluating the prediction accuracy of the studied models, root mean squared error (RMSE), mean absolute error (MAE), and correlation coefficient (R) were utilized as presented in equations 9-11. Additionally, the Taylor diagram, which is a graphical evaluation design, was used  to confirm the accurateness of considered models. In the Taylor diagram, several aspects can be reviewed instantly (Gleckler et al., 2008;Taylor, 2001).
Where p i is the ith predicted value by models, o i is the ith observed value and n is the total number of sample data.

Results and discussion
In the current research, five input arrangements based on meteorological parameters were assembled and evaluated  to determine whether the proposed MLP-FFA and SVM-FFA hybrid models were capable of data-driven tools for modeling the soil temperature at different depths with one and two days delay (Table 4). It should be noted that in the case of one day as delay, the current values and in the case of two days as delay, the current and previous correspondent values were implemented. Also, note that the input combination (1) has employed the most highly correlated variable but as a single input only, whereas input combination (2) has incorporated the two most highly direct and inverse correlated variables and input combination (3) has utilized the three most relevant variables. Likewise, input combination (4) has used all variables to model the ST data. The ideal combination of these model inputs aimed to produce accurate models that represented the most substantial variables, as well as a carefully selected combination of these variables to check their influence on the testing performances. In all models, we followed the notion that there is no rule of thumb for that the universal way the training and  testing data partitioned. For example, the study of Kurup and Dudani (2002) used a total of their 63% of data for model development while Pal (2006) applied 69%, Samadianfard et al. (2013) and Samadianfard et al. (2014) used 67% of total data to develop their models. So, the current utilized data parted into training (70%) and testing (30%). In other words, meteorological data of 2013 and 2014 (total of 730) implemented for training the considered models, while the correspondent data of 2015 (total of 365) utilized for testing them. It is imperative to note that trial and error procedures implemented for finding the optimal structures of the models. This followed the fact that there is not any definite verified process for indicating the ideal number of hidden neurons and parameters, and these are typically selected in an iterative manner (Deo & Sahin, 2015, 2017. Additionally, after examining the different training algorithms and transfer functions, the Levenberg-Marquardt algorithm and the sigmoid transfer function were selected for the current research. Various kernel functions have been analyzed and the optimum kernel for each input combination with optimized hyperparameters was selected based on the error functions. The obtained error meters are presented in Tables 5 and 6. As can be seen from Tables 5 and 6, a comprehensive evaluation has been carried out for revealing the capabilities of the mentioned hybrid models using several statistical meters such as R 2 , MAE and RMSE indices. So, in the case of predicting ST 5cm at Tabriz station, SVM-FFA1 and MLP-FFA4 with RMSE values of 2.15°C and 1.71°C produced better results in the case of one and two days delay, respectively. Also, they improved the accuracies of standalone SVM and MLP models with 31.75% and 47.55%, respectively. However, somehow the reverse trend has been seen in predicting ST 5cm at Ahar station. In this case, MLP-FFA3 and SVM-FFA4 with RMSE values of 4.64°C and 4.60°C, respectively, predicted ST values more precisely than standalone MLP and SVM models with one and two days delay. Also, the improvement percentage of MLP and SVM models were 23.31% Figure 5. Scatterplots of the estimated-observed ST values by best models with a delay of two days at different depths. and 30.20%, respectively. Thus, it can be comprehended that MLP-FFA4 and SVM-FFA4 with the same meteorological parameters of T, RH, SUN and W provided superior predictions of ST 5cm at Tabriz and Ahar stations. Comparing these obtained results with findings of Sihag et al. (2019) shows that the RMSE values of MLP-FFA4 and SVM-FFA4 (1.71°C and 4.60°C) are lower than the best accurate models of Sihag et al. (2019) (MLP with RMSE of 3.26°C and 6.33°C). So, MLP, GP, RF and the M5P models in estimating ST 5cm are not recommended. Additionally, in the case of predicting ST 10cm , it can be revealed from Tables 5 and 6 that MLP-FFA1, SVM-FFA4 (Tabriz station) and MLP-FFA4, MLP-FFA1 (Ahar station) in the case of one and two days delay and with RMSE values of 1.58°C, 1.63°C, 4.51°C, and 4.47°C predicted ST values more accurately than standalone MLP and SVM models with different input combinations. In other words, they improved the prediction accuracy of standalone MLP and SVM models by 35.51%, 38.26%, 17.85%, and 20.60%, respectively. Conclusively, MLP-FFA1 which only uses the input parameter of T has been selected as the precise model in predicting ST 10cm at both studied stations. Finally, in predicting ST 20cm at Tabriz station, it can be found from Tables 5 and 6 that MLP-FFA5 and MLP-FFA3 with one and two days delay and having the RMSE values of 1.17°C and 1.15°C, respectively, predicted ST values with lower error values than standalone MLP and SVM and other hybrid models with different input combinations. The mentioned MLP-FFA5 and MLP-FFA3 in the prediction of ST 20cm at Tabriz station reduced the RMSE values of correspondent standalone MLP models by 41.79% and 45.75%, respectively.
Moreover, at Ahar station, MLP-FFA3 and SVM-FFA4 with one and two days delay, respectively, predicted ST 20cm values more precisely than standalone MLP and SVM models. In other words, the improvement rates of standalone MLP and SVM models, in this case, were 9.40% and 28.76%, respectively. MLP-FFA3 with input parameters of T, RH, SUN and SVM-FFA4 with input parameters of T, RH, SUN, and W in the case of two days delay presented more accurate predictions of ST 20cm values than other hybrid models of MLP and SVM with different input combinations. As a remarkable point, it is clear that forth input combination with parameters of T,  RH, SUN, and W is in the majority for selecting as the best input combination for ST predicting at both studied stations. The importance of these factors in estimating soil temperature as the inputs of the model considered.
The following graphs (Figures 4 and 5) show the dispersion scattering points at a depth of 5, 10 and 20 cm than modeling with best MLP, SVM and MLP-FFA, SVM-FFA models.
Moreover, the comprehensive comparison between the results of all studied models with one and two days as delay revealed that the accuracy of the best models in the case of two days as delay were higher than the correspondent models with one day as delay except for ST 10cm at Tabriz station. This may be related to the fact that using previous values of meteorological parameters can be useful in increasing the accuracies of the studied models.
Comparing the obtained results with findings of Samadianfard, Asadi, et al. (2018) showed that MLP-FFA1 with one day delay and MLP-FFA3 with tow days delay had better performances in comparison with the best model WNN models in estimating ST 10cm and ST 20cm , respectively. But in the case of ST 5cm , the accuracy of WNN in the study of Samadianfard, Asadi, et al. (2018) was higher than the best FFA integrated model in the current study.
For additional assessing the accuracy of the established models, Figures 4 and 5 present a scatter plot of observed and predicted ST values. Obviously, the R 2value for all panels exposes the higher accuracy of the hybrid models, which agrees with previous results in Tables 5 and 6. Also, Figures 6 and 7 display threedimensional bar graphs of the RMSE produced by studied hybrid models analyzed using different combinations of meteorological parameters. Agreeing with earlier results interpretations, the predicted ST values at different depths discovered a superior effectiveness of the hybrid models. Additionally, errors produced by hybrid models were expressively lower than those of the classical models (by 9.40% to almost 45.75%).
Besides, Taylor diagrams were implemented for exploring the accurateness of hybrid models (Figures 8  and 9). In Taylor's diagrams, the distance from the reference green point is an amount of the centered RMSE (Taylor, 2001). Therefore, an accurate model is selected based on the distance of the correspondent point to the green one. It is clear from Taylor's diagrams that MLP-FFA and SVM-FFA models provided the most precise predictions than standalone MLP and SVM models.
As it is clear by an overabundance of statistical parameters and graphical evaluation, it is convincing that the accuracy of the MLP-FFA and SVM-FFA models far surpasses the non-optimized models. Therefore, FFA certifies that the hybrid model can avoid any premature convergence of process, which is likely to maximize the ability of the hybrid MLP-FFA and SVM-FFA models. According to statistical and graphical evaluations, MLP-FFA and SVM-FFA were found to be a sufficient tool for predicting soil temperature values using meteorological parameters as the inputs and can be used as practical models with a high degree of applicability for ST estimation.

Sensitivity analysis
For investigating the influence of input parameters on the ST prediction, the RMSE and R 2 evaluation criteria utilized for different groupings of input variables. For this purpose, the SVM model in predicting ST 10cm with a delay of one and two days selected for sensitivity analysis (Tables 7 and 8). Each model confirmed the extents to which the eliminated variable would affect the model accuracy. As it is clear from Tables 7 and 8, the precision of the SVM model decreased if each of T, RH, SUN, and W input parameters removed in the modeling. Furthermore, it comprehended that T has the most significant effect in increasing the prediction accuracy. In other words, eliminating T caused a sharp increase in RMSE value in all studied conditions. Additionally, as the limitations of the current study, it should be noted that the used dataset was related to two Tabriz and Ahar stations, Iran. Therefore, the mentioned stations have approximately similar climates. Thus, it would have been better if extra stations with different environments were tested for examining the accuracy of implemented methods.

Conclusions
Soil temperature as one of the main characteristics of the soil affects many aspects of life, especially the distribution of plants, animals, biological activities, and water movement in the soil. Furthermore, soil temperatures are useful in cases such as the length of growth, the spread of plant diseases, the readability of the soil water, the growth and development of roots. The determination of models that can accurately estimate the temperature of different soil levels is essential for agricultural research.
In the present research, we tried to estimate ST values at various depths of 5, 10, and 20 cm using the MLP, SVM, and their hybrid version with a firefly optimization algorithm (MLP-FFA and SVM-FFA). The meteorological parameters of Tabriz station have been gathered, and five diverse groups were prepared with one and two days delay. The results attained exposed that in the case of using one day delay for the depths of 5, 10 and 20 cm, the most precise models were SVVM-FFA1, MLP-FFA1, MLP-FFA5 (with RMSE values of 2.15˚C, 1.58˚C, and 1.17˚C) at Tabriz station and MLP-FFA3, MLP-FFA4, MLP-FFA3 (with RMSE values of 4.64˚C, 4.51˚C and 3.95˚C) at Ahar station, respectively. Additionally, in the case of using two days delay for the depths of 5, 10 and 20 cm, the most precise models were MLP-FFA4, SVM-FFA4, MLP-FFA3 (with RMSE values of 1.71˚C, 1.63˚C and 1.15˚C) at Tabriz station and SVM-FFA4, MLP-FFA1, SVM-FFA4 (with RMSE values of 4.60˚C, 4.47˚C, and 3.69˚C) at Ahar station, respectively. Moreover, it can be comprehended from obtained results that the hybrid models of FFA (i.e. MLP-FFA and SVM-FFA) produced a substantial decrease in error metrics compared to the standalone MLP and SVM models. In conclusion, the acquired results endorsed the adequacy of the hybrid MLP-FFA and SVM-FFA models and pointed out the efficiency of the FFA algorithm for soil temperature estimations. Future work can be done for examining the effects of different optimization algorithms on the accuracy of MLP and SVM models for ST estimation at different climates.