Spatial evaluation of L-band satellite-based soil moisture products in the upper Huai River basin of China

ABSTRACT Using dense soil moisture (SM) measurements in the upper Huai River basin of China, this study evaluated the spatial patterns of L-band satellite-based SM products, including Soil Moisture Active Passive (SMAP) L3, Soil Moisture and Ocean Salinity (SMOS) L3 and the European Space Agency’s Climate Change Initiative (ESA CCI) SM products. The mean difference (MD), root mean squared error (RMSE), unbiased root mean square error (ubRMSE) and Pearson correlation coefficient (R), were used in the evaluation. The evaluation results presented that SMAP and ESA CCI products can well capture the temporal variation of SM at single points quite well, with average R values of 0.51 and 0.46, respectively. And SMAP had the highest overall accuracy among the three satellite-based products in study area. We also analyzed the correlations between the four accuracy indexes and six environmental factors including the proportions of five land use/land cover types (i.e. water bodies, paddy fields, construction land, dryland and forest) and the average NDVI (Normalized Difference Vegetation Index) in 2016 in each grid. Analysis showed that the proportions of paddy fields and water bodies in each grid had significant positive correlations with MD, RMSE and ubRMSE, while NDVI, and the proportions of dryland and construction land had significant negative correlations with these three indexes. The significant correlations between the accuracy of SMAP, SMOS and ESA CCI SM products and environmental factors indicate that there exist systematic biases in these products, which can provide valuable insights into algorithm improvements.


Introduction
Soil moisture (SM) has essential impacts on the partition of energy and water over land surface, and it also exerts a critical control on land-atmosphere interaction, hydrological and biogeochemical cycles (Brocca et al., 2016;Paloscia, Pettinato, & Santi, 2012). Therefore, the knowledge of surface SM is crucial for many studies, such as climate simulation, flood forecasting as well as land surface modeling (Brocca, Ciabatta, Massari, Camici, & Tarpanelli, 2017;Vittucci et al., 2013).
Passive microwave remote sensing is very sensitive to surface SM content (Jackson, Vine, Hsu, & Oldak, 1999) and has become a promising approach to mapping regional or global SM. During the past few decades, passive microwaves of different frequency (e.g. X, C and L bands) have been widely used to estimate SM (Kolassa, Gentine, Prigent, & Aires, 2016;Schmugge, Gloersen, Wilheit, & Geiger, 1974;Vittucci et al., 2013). The L-band observation can penetrate a deeper depth than X-and C-band, which makes it more optimal for SM retrieval. At present, Soil Moisture Active Passive (SMAP, 1.41 GHz) and Soil Moisture and Ocean Salinity (SMOS, 1.4 GHz) are currently available satellites carrying L-band microwave radiometers. Global surface SM products retrieved from the observations of SMAP and SMOS have been released to the public, which have great value for related researches and applications (Anam, Chishtie, Ghuffar, Qazi, & Shahid, 2017;Shellito et al., 2016).
Satellite-based SM products need to be validated before their application with certainty. In this regard, several researches have been conducted to evaluate the SMAP and SMOS passive SM products (Dong, Crow, & Bindlish, 2018;Dorigo et al., 2015;Pierdicca, Pulvirenti, Fascetti, Crapolicchio, & Talone, 2013). For example, for the evaluation of SMAP passive SM product, Colliander et al. (2017) validated SMAP SM products with in situ SM measurements from 34 in situ stations around the world. Results showed that the SMAP SM product can meet an accuracy of 0.04 m 3 /m 3 (unbiased root mean square error, ubRMSE). Pan, Cai, Chaney, Entekhabi, and Wood (2016) acquired similar results by comparing the SMAP L3 passive SM product with in situ measurements from 258 in situ stations over the continental United States. For the evaluation of SMOS SM product, Yee et al. (2017) evaluated two different versions of SMOS L3 products with SM measurements from 37 in situ stations in southeast Australia. They found that the SMOS SM retrieved from morning overpasses was more accurate compared with those retrieved from afternoon overpasses. In addition, by comparing SMOS SM products with the in situ measurements (10 stations in the Genhe area of China), Cui et al. (2017) pointed out that the temporal variation of SMOS L3 SM data is noisy and unstable.
The validation studies based on in situ measurements place more emphasis on the overall accuracy of satellite-based SM products rather than the spatial patterns. As a representative of spatial pattern evaluation, Li et al. (2018) evaluated the SMAP Enhanced SM products by comparing them with high-resolution model simulations. The result shows that the spatial patterns of SM for the SMAP Enhanced SM products and the model simulation agree well. However, few studies have been done to evaluate the spatial patterns of satellite-based SM products with in situ SM measurements due to the limited density or spatial coverage of in situ stations. Several research questions remain to be answered, including (1) whether the L-band satellitebased passive SM products can capture the real-world spatial patterns at the regional scale? (2) How is the spatial distribution of the accuracy of satellite-based SM products and which environmental factors affect the accuracy? This study aims to answer these questions. The answer to these questions can provide valuable clues to further improve the retrieval algorithm of remote sensing SM.
In this study, two currently available L-band SMAP and SMOS SM products was evaluated using the in situ measurements from a dense network in the upper Huai River basin, China. The European Space Agency's Climate Change Initiative (ESA CCI) passive SM product (version v04.2), which combined L-band SMOS measurements, was also evaluated as a reference. The specific objectives of this study were (1) to detect the spatial patterns of the L-band satellite-based passive SM products; (2) to analyze the relationships between the accuracy of satellite-based SM products and the environmental factors.

Study area
In this study, the upper Huai River basin was selected as the study area ( Figure 1). In this area, the terrain is mainly plains, and only a few mountains are distributed in the southwest. The main land cover is farmland in the plain areas, including dryland and paddy fields, which accounts for 75.6% of the total area. Forest and grass are mainly located in the mountainous area. The rainfall decreases progressively from the south to the north with rainfall amount ranging from 400 to 1730 mm in 2016, and about 80% of the rainfall is concentrated between June and August.

Data sets
In situ SM data set There are 131 in situ SM stations deployed in this area (shown in Figure 1). The in situ SM data were collected at 8 am (Beijing time) every day from 1 January 2016 to 1 January 2017, which allows for evaluation of SMAP, SMOS and ESA CCI SM products for an entire year 2016. The volumetric SM was measured every 10 cm within a depth of 0-1 m below land surface, and the Time Domain Reflectometry sensor was used to measure the SM.
The L-band satellite SM products can only provide the near-surface SM observation (5 cm depth or so). But there is no SM observation at 5 cm depth in the study area. Here we made an assumption that the in situ SM at 10 cm is close to that at 5 cm, and the in situ SM measurements at the depth of 10 cm were chosen as the reference to evaluate the satellite-based products. The intention of this assumption is to take advantage of the dense SM stations in this study area although it is admitted that it will bring uncertainty to the evaluation. In addition, the same assumption was also made by some previous studies Dorigo et al., 2015;Peng, Niesel, Loew, Zhang, & Wang, 2015;Qiu, Gao, Wang, & Su, 2016).

SMAP L3 passive SM product
The SMAP mission was developed by the National Aeronautics and Space Administration (NASA). The satellite was launched on 31 January 2015, and was placed into a polar sun synchronous orbit (descending orbit at 6 am local solar time and ascending orbit at 6 pm local solar time) (Colliander et al., 2017). The satellite carries both an L-band radar and an L-band radiometer. Unfortunately, the radar instrument stopped working a few months later after launching due to an irrecoverable hardware failure. The left radiometer onboard still can provide 36 kmresolution passive observations every 2-3 days (O'Neill, Chan, Njoku, & Jackson, 2015). The SMAP L3 passive SM product with single channel algorithm at vertically polarization (V polarization) was used in this study. In the operational retrieval algorithm, the mixed emissivity is firstly calculated from the physical temperature. Then soil emissivity is extracted from the mixed emissivity after correcting for the vegetation and surface roughness. Finally, the dielectric constant is calculated from the Fresnel equation and a dielectric mixing model is adopted to retrieve SM data (Wigneron et al., 2017). In this process, the physical temperature is estimated independently and provided by the NASA Goddard Earth Observing System Model, and the Normalized Difference Vegetation Index (NDVI) derived from MODIS datasets were used to correct the mixed emissivity.
In this study, the SMAP L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture was used (Version 4, https://nsidc.org/data/smap/smap-data. html). The daily SMAP SM data retrievals from morning overpasses were chosen, and the data were filtrated according to the retrieval quality flag (Colliander et al., 2017).

SMOS L3 SM product
The SMOS mission was the second Earth Explorer Opportunity mission of ESA. The satellite was launched on 2 November 2009, carrying a twodimensional interferometric microwave radiometer that can provide both multiangular and dual-polarization measurements. It was the first L-band microwave satellite that was dedicated for global near-surface SM observation (5 cm depth or so). This satellite can revisit one area within 2-3 days at 6 am local solar time (ascending orbit) and 6 pm local solar time (descending orbit). The L-band Microwave Emission of the Biosphere (L-MEB) model was used for SMOS SM retrieval (Wigneron et al., 2007). In the L-MEB model algorithm, the parameters of surface SM and vegetation optical depth are retrieved simultaneously taking advantage of the SMOS multiangular data. More details about the L-MEB model can be found in Wigneron et al. (2007) and Kerr et al. (2012).
The SMOS SM product evaluated in this study was the L3 product from the Centre National d'Etudes Spatiales and the Centre Aval de Traitement des Données SMOS (https://www.catds.fr/Products/ Available-products-from-CPDC) (Jacquette et al., 2010). These data are organized by EASE-Grid 2.0 at a spatial resolution of 25 km (Chen et al., 2017). Daily SMOS SM retrievals from the morning overpasses were validated, and the data were also filtered if the retrieval quality fields (Soil_Moisture_Dqx and Rfi_Prob fields) indicate poor quality .

ESA CCI passive SM product
The ESA CCI SM product was developed as part of the ESA Program on the Global Monitoring of Essential Climate Variables (Liu et al., 2012). Several versions of the ESA CCI SM products have been published since 2010, including passive product, active product and combined product. For the passive SM product, the Land Parameter Retrieval Model (LPRM) was used to convert the observed brightness temperatures to SM (Owe, de Jeu, & Holmes, 2008). In general, the LPRM model links surface SM to brightness temperature, and the SM, vegetation density as well as surface temperature are retrieved simultaneously.
The latest version of the ESA CCI SM product (i.e. CCI SM v04.2) at a spatial resolution of 0.25°was released in 18 January 2018 by the Vienna University of Technology, covering the period from 1978 to 2016 . The brightness temperature measurements of the SMOS satellite were also used to generate this product (Gruber, Dorigo, Crow, & Wagner, 2017). The daily Passive Soil Moisture Product of CCI SM v04.2 was also evaluated as a reference in present study (http://www.esa-soilmoisture-cci.org/node/145).

Auxiliary data sets
Land use and NDVI data sets were used to analyze the relationships between environmental factors and the accuracy of satellite-based SM products. The land use data set was derived from 1km Â 1 km resolution land use map of China, which was released by the Institute of Geographical Sciences and Natural Resource Research, Chinese Academy of Sciences (http://www.resdc.cn/ Default.aspx). The yearly averaged NDVI was calculated with the surface reflectance of MODIS MOD09A1.005 products, and these MODIS products were downloaded from Google's Earth Engine. The land use and NDVI data sets were upscaled to the same resolution of satellite-based SM products before the analysis. For the land use data, the proportion for each type of land use in every selected satellite data grid was calculated, and for the NDVI data, the average value in every selected satellite data grid was calculated. The precipitation and elevation data sets were only used for visualization and were not used to analyze the spatial pattern of SM data, so no resolution transformation was needed. The precipitation data set is derived from a 0:1 Â 0:1 resolution gridded precipitation product, which can be downloaded from China Meteorological Administration (http://data.cma.cn/site/index.html). The 1km Â 1km resolution elevation data can be downloaded from the Institute of Geographical Sciences and Natural Resource Research, Chinese Academy of Sciences (http://www.resdc.cn/Default.aspx).

Matching in situ measurements with satellite-based SM data
There exists a scale mismatch between satellite-based SM data at the grid scale and the in situ measurements at the point scale. To conduct the validation, point-scale measurements were converted into the grid scale. The ordinary kriging method was used to interpolate the daily in situ SM measurements into a spatially continuous field at a finer spatial resolution of 1 km, which was then aggregated to the grid scales of each satellite-based SM data (36-km resolution for SMAP, 25-km resolution for SMOS and 0.25°for CCI) through averaging.
For interpolation, the error is minimal at the location of interpolation point. In order to minimize the error of in situ measurements, only the grid with at least one in situ site within it was used in further evaluation, and the grids without any in situ site in them were left out. In addition, multiple in situ sites may locate in one grid, and the spatial resolution of the satellite-based SM products is different. Those two factors result in different number of grids for the three satellite-based SM products were chosen for evaluation. In the end, the numbers of such grids were 91, 120 and 116 for SMAP, SMOS and ESA CCI, respectively (Figure 1(b), (c) and (d)).

Statistic analysis methods
Four common accuracy indexes, including mean difference (MD), root mean squared error (RMSE), ubRMSE as well as Pearson correlation coefficient (R), were chosen to validate the satellite-based SM products (Crow et al., 2012;Kawanishi et al., 2003;Wang, Mo, Liu, Lin, & Hu, 2016). The four statistic indexes are calculated using the following formulas: Here, S_SM represents the SM value of satellite-based product, while M_SM represents the in situ SM measurement (m 3 /m 3 ). In formula (4), μ s represents average satellite-based SM, μ m represents average in situ SM (m 3 /m 3 ), δ s and δ m are the standard deviations of the two types of SM data (m 3 /m 3 ), respectively.

Spatial autocorrelation analysis
Moran's I was used to quantify the spatial autocorrelation of the four accuracy indexes in this study. The positive value of Moran's I means the variable is aggregated in space, the negative value means the variable is discrete, and the zero value means the variable is random. The Z-score was used to indicate the confidence level. For reference, the Z-score for a 95% confidence level is larger than 1.96 or smaller than −1.96, while the Z-score for a 99% confidence level is larger than 2.58 or smaller than −2.58.

Results
Spatial patterns of satellite-based SM Figure 2 shows the average daily SM of SMAP L3, SMOS L3, ESA CCI and the interpolation of in situ SM measurements for the year 2016. All SM products displayed progressive decreasing trend from south to north, which showed a high degree of similarity to the spatial pattern of rainfall in this area (Figure 1(d)). In addition, the ranges of satellite-based SM products were larger than the interpolated in situ SM; they were 0.13-0. The wettest area in the interpolated SM map was located in the southwest of the study area, which was not well captured by all the three SM products. Compared with SMAP and ESA CCI, the spatial distribution of SMOS SM was more fragmented in space, which may be because that SMOS suffered more significantly from radio frequency interference (Colliander et al., 2017). There existed large differences among these satellite-based SM products in the south part of the study area, where the SM was very high in SMAP (ranging from 0.4 m 3 /m 3 to 0.6 m 3 /m 3 ) but very low in SMOS (ranging from 0.1 m 3 /m 3 to 0.2 m 3 /m 3 ), and was null in ESA CCI due to poor retrieval quality. The interpolated in situ SM in the south part was in the medium level ranging from 0.15 m 3 /m 3 to 0.3 m 3 /m 3 . These results suggest that large uncertainty existed in the satellite-based SM products in the south part of the study area.

Overall accuracy of satellite-based SM products
The evaluation indexes of the SMAP, SMOS and ESA CCI SM products are shown in Table 1. SMAP had the overall best performance with the lowest RMSE and moderate correlation coefficient. ESA CCI had the highest correlation coefficient, which means that it could represent the temporal and spatial variation of SM better than SMAP and SMOS in this study area, but it was very poor in terms of higher MD and RMSE.
As Figure 3 shows, the accuracy of the satellitebased SM products varied in different months. For example, in terms of MD, RMSE and ubRMSE, the ESA CCI SM product had the best performance in July, while SMAP and SMOS had the worst performance in June. The correlation coefficients were generally higher in the first half (January-June) than in the second half of the year for all the three products. The smallest correlation coefficient occurred in August for all the three products, being close to 0, which means that none of these products can capture the spatial-temporal variation of SM in August. Figure 4 shows the spatial distribution of the accuracy index for SMAP (the left column), SMOS (the middle column) and ESA CCI (the right column). As mentioned above, only the grid with at least one in situ site within it was used in further evaluation, and the grids without any in situ site in them were left out. The blank areas in Figure 4 are the grids without any in situ. For the MD index (the first row in Figure 4), the spatial patterns of SMAP, SMOS and ESA CCI were similar with larger values in the southeast and smaller values in the northwest. Both SMAP and SMOS overestimated SM in the southeast and underestimated SM in the northeast, while ESA CCI overestimated SM over the whole area. The RMSE of ESA CCI SM products (Figure 4(f)) showed an increasing trend from the northwest to southeast. There was no obvious spatial trend for the RMSE of SMAP and SMOS (Figure 4(d-e)). For ubRMSE (the third row in Figure 4), it is interesting that the spatial distribution of SMAP showed an obvious increasing trend from northwest to southeast, while the spatial trends of SMOS and ESA CCI were not obvious. The average RMSE of all the selected grids was 0.09 for SMAP, 0.13 for SMOS and 0.18 for ESA CCI. The average ubRMSE was 0.05 for SMAP, 0.12 for SMOS and 0.09 for ESA CCI.

Spatial distribution of the accuracy of satellite-based SM products
The average RMSE and ubRMSE calculated here were smaller than those in Table 1. This difference is because that to calculate the values in Table 1, the mean square error of all the SM data of all the selected grids were calculated first and then its square root was taken, while here RMSE of each grid was calculated first and then averaged.
The fourth row in Figure 4 shows the correlation coefficients at each selected grid. The average correlation coefficients were 0.51, 0.20 and 0.46 for SMAP, SMOS and ESA CCI, respectively. The result means that SMAP and ESA CCI SM products can well capture the temporal variation of SM compared with SMOS. The correlation coefficients of SMAP and ESA CCI also had stronger spatial aggregation than those of SMOS (see the Moran-I values in Table 2). Table 2 summarizes the results of Moran's I and the corresponding Z-scores. All the Z-scores are larger than 2.58, which means that the four indexes had significant spatial aggregation. MD had the strongest aggregation among the four indexes. For MD and ubRMSE, SMAP showed the strongest spatial  aggregation, while for RMSE and R, ESA CCI showed the strongest spatial aggregation.

Discussion
The capability of the satellite-based products in capturing spatial patterns of SM In order to quantify the capability of satellite-based products in capturing the spatial patterns of SM, the scatterplots of average satellite-based SM against average in situ SM in 2016 are shown in Figure 5. The in situ SM measurements were concentrated in a narrow range (from 0.2 to 0.3 m 3 /m 3 ), while the satellite-based SM cover much wider ranges (about  from 0.15 to 0.4 m 3 /m 3 for SMAP and SMOS, from 0.24 to 0.6 m 3 /m 3 for ESA CCI). The determination coefficients (R 2 ) are very low (<0.10), indicating that all the three satellite-based SM products performs poorly in capturing the spatial patterns of real SM in study area. However, as shown in Figure 4, satellite-based SM products, especially SMAP and ESA CCI, can capture the temporal variation of SM in each grid fairly well. These results suggest that the current satellite-based soil products are more suitable for temporal analysis rather than spatial analysis.

Relationship between the accuracy of satellite-based SM and environmental factors
In order to detect relationship between the accuracy of satellite-based SM products and environmental factors, Tables 3-5 show the correlation coefficients between the four accuracy indexes and six environmental factors for SMAP, SMOS and ESA CCI products, respectively. The six environmental factors included the proportions of five land use/land cover types (i.e. water bodies, paddy fields, construction land, dryland and forest) and the average NDVI in each grid in 2016. The correlations between MD, RMSE, ubRMSE and the six environmental factors show a good consistency for all the three satellite-based SM products. The proportions of water bodies and paddy fields showed positive correlations with MD, RMSE and ubRMSE, while NDVI, the proportions of dryland and construction land showed negative correlations. Most of these correlations are significant at 0.01 level. There were significant correlation between R and the proportion of paddy fields and dryland for all the three products. For SMAP and SMOS, the correlation was positive between R and the proportion of paddy fields, and was negative between R and the proportion of dryland, while the opposite was true for ESA CCI.
The scatterplots of MD against the proportions of five land use/land cover types and the average NDVI are shown in Figure 6 for SMAP, Figure 7 for SMOS and Figure 8 for ESA CCI. The spatial resolutions of current satellite-based SM products were coarse, so there usually exist multiple land use/land cover types within one grid. If there are water bodies in a grid, the microwave signals received by the satellites contain not only the information of soil but also that of land-surface water. Because water has higher dielectric constant than soil, the overall dielectric constant of the grid will be elevated. Higher dielectric constant will lead to higher estimated SM (Chan et al., 2018;Gruber et al., 2017). This is the reason why the proportions of water bodies and paddy fields in each grid show positive correlations with MD for all the three    satellite-based SM products. The significant positive correlations between MD and the proportions of dryland may be caused by the negative correlations between the proportions of dryland and those of paddy rice. It is worth noting that when the proportions of water bodies/paddy fields were zero, MD was negative for SMAP and SMOS SM products and positive for ESA CCI product. This means that in normal conditions without the influence of water bodies, SMAP and SMOS generally underestimated SM in this region, while ESA CCI overestimated SM.  By contrast, the construction land had lower dielectric constant than soil, so it reduced the dielectric constant of the whole grid and led to underestimated SM. The proportions of forest were concentrated in small values, so the obtained relationship may not be stable. NDVI had significant negative correlations with MD for all the three products in this region. Because different SM products adopted different algorithms to deal with vegetative biomass, the reason of the significant negative correlations between MD and NDVI needs further research.
Previous studies have pointed out that SM retrieved from passive microwave remote sensing would be influenced by the complexity and variability of vegetation canopy and land surface roughness (Chan et al., 2018;Cui et al., 2017;Zhan, Crow, Jackson, & O'Neill, 2008). It was also reported that the accuracy of satellite-based SM products varies with land cover types (Zhang, Zhang, Huang, Hong, & Meng, 2017). Our results further verify these existing conclusions with dense SM measurements in a new study area, which can provide valuable guidance for algorithm improvements in the future.

Conclusions
This study evaluated the spatial patterns of SMAP, SMOS and ESA CCI passive SM products and explored the spatial distribution of their accuracy with dense SM measurements from 131 in situ stations in the upper Huai River basin of China. According to the obtained results, following conclusions can be drawn: (1) SMAP had the overall best performance with the lowest RMSE and moderate correlation coefficient, while ESA CCI had the highest correlation coefficient. (2) The four accuracy indexes (i.e. MD, RMSE, ubRMSE and R) all had significant spatial aggregation. (3) SMAP and ESA CCI SM products could well capture the temporal variation of SM in each grid, but all the three satellite-based SM products can hardly capture the spatial patterns of measured SM. (4) For all the three satellite-based SM products, the proportions of water bodies and paddy fields showed positive correlations with MD, RMSE and ubRMSE, while NDVI, the proportions of dryland and construction land showed negative correlations.
Compared to other works on the evaluation of SM on the plot scale, this study used a more dense in situ observation network and most grid cells in the study area have SM stations in them. Thus, this study could analyze the spatial distribution of the accuracy of the three satellite SM products and detect the relationships between the accuracy of satellite-based SM products and the environmental factors. We hope these results could provide valuable information for the spatial distribution of errors of satellite-based moisture products and complement existing researches based on high-resolution model simulations (Li et al., 2018).