Brightness gradient-corrected hyperspectral image mosaics for fractional vegetation cover mapping in northern California

ABSTRACT We evaluated the effectiveness of different approaches to compensate for across-track brightness gradients within a hyperspectral image mosaic comprised of multiple flight lines in the San Francisco Bay Area. We calculated the spectral consistency of adjacent flight lines and conducted regression-based unmixing of woody- and non-woody vegetation fractions to assess the comparative benefits of the methods. Results showed that a class-wise empirical approach produced the most spectrally consistent, nearly seamless image mosaics and led to accurate vegetation fraction maps (mean absolute error = 12.6%). Overall, a class-wise empirical approach is recommended as a simple, flexible and transferable technique to compensate for brightness gradients over a global empirical approach, brightness normalization or continuum removal.


Introduction
Imaging spectroscopy, or hyperspectral remote sensing, is becoming increasingly viable for quantitative vegetation assessments. However, large area ecosystem analysis with hyperspectral data relies on mosaics from multiple airborne acquisitions acquired over the course of one or multiple days, due to a lack of functional spaceborne hyperspectral sensors. Changing sun-sensor geometries during this period often lead to distinct across-track brightness gradients within and between flight lines. These variations hinder pixel spectral comparisons within and among images, affect spectral ratios, complicate image mosaicking, and hamper the integration of lab-or field-based spectral libraries into image analysis (Beisl and Woodhouse 2004;Ben-Dor, Levin, and Saaroni 2001).
Brightness gradients result from a surface's anisotropic reflectance and depend on the view angle of a sensor, the solar illumination conditions during the acquisition, the respective wavelength, and the structural and optical properties of surface materials (Lucht, Schaaf, and Strahler 2000). Surface reflectance anisotropy can be comprehensively described by the bidirectional reflectance distribution function (BRDF). We refer to Nicodemus et al. (1977) for a comprehensive overview of the theoretical underpinnings of BRDF. Different empirical and semi-empirical modelling approaches have been proposed to correct brightness gradients. Empirical models describe brightness gradients as a mathematical function of the view angle that is used to normalize each pixel spectrum to nadir view (Collings et al. 2010;Kennedy, Cohen, and Takao 1997). Due to surface type dependent variations in anisotropic reflectance, several authors suggest the use of class-wise empirical models based on a pre-classification of the image (Kennedy, Cohen, and Takao 1997;Schiefer, Hostert, and Damm 2006). The most common semi-empirical approaches are so-called kernel-based models, which describe brightness gradients with multivariate models and can account for other angular and surface properties, e.g., volumetric and geometric scattering (Jensen et al. 2018;Schiefer, Hostert, and Damm 2006). Despite much research on semi-empirical approaches, no 'state of the art' solution is available and used, yet empirical models are used in almost all applications. We here refer to Schiefer, Hostert, and Damm (2006) for an in-depth overview of brightness gradient correction.
As an alternative to the aforementioned modelling approaches, normalized reflectance spectra have been used to account for brightness gradients in hyperspectral images (Berman et al. 2004;Tane et al. 2018). Normalized spectra emphasize the spectral shape, i.e., absorption features, while differences due to illumination and shading effects are minimized. Normalization approaches divide each spectrum by its sum or mean, known as 'brightness normalization' (Berman et al. 2004), or by its convex hull, known as 'continuum removal' (Tane et al. 2018;Youngentob et al. 2011).
The NASA's Hyperspectral InfraRed Imager (HyspIRI) Airborne Campaign (Lee et al. 2015) provides a unique source of Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) data covering different ecosystems in California, USA. The datasets, acquired over large areas in multiple flight lines on a seasonal and yearly basis, intend to support pre-launch activities of upcoming hyperspectral satellite missions, e.g., the Surface Biology and Geology mission (SGB, National Academy of Sciences 2018), formerly known as HyspIRI, or the Environmental Mapping and Analysis Program (EnMAP, Guanter et al. 2015). However, due to the wide field-of-view of AVIRIS and the changing sun-sensor geometries during data collection over extended time periods, the datasets are affected by severe brightness gradients that may negatively impact applications. Such brightness effects are expected to impact future sensors, but to a lower extent, given their smaller fields of view. However, these effects will be significantly increased when off-nadir images are acquired (e.g. utilizing EnMAP's tilting capability) to create image mosaics for larger areas.
Against this background, the overarching goal of this study was to evaluate the effectiveness of different brightness gradient correction methods for the HyspIRI Airborne Campaign data and their influence on land cover mapping approaches. We explicitly focused on method simplicity and, therefore, considered normalization approaches (brightness normalization and continuum removal) and empirical modelling strategies (global and class-wise). To assess the comparative benefits of these methods, we calculated the spectral consistency between adjacent flight lines and conducted a vegetation fraction mapping with regression-based unmixing. Our specific objectives were 1) to evaluate the use of the four brightness gradient correction approaches to enhance the quality of the image mosaic, and 2) to demonstrate the impact of these methods on the accuracy of the fraction maps.

Study site and image data
We chose the HyspIRI Airborne Campaign site San Francisco Bay Area, California, USA, as our study region (Lee et al. 2015). The area is characterized by a Mediterranean climate with a strong marine influence from the Pacific Ocean. Temperate coniferous forests dominate the northwest coastal mountains, Mediterranean forests, woodlands and shrublands are prevalent in the south and interior mountains, and annual grassland and agriculture characterize the east (Clark 2017).
We used 11 AVIRIS flight lines, which were acquired on 7 June 2013 between 10:37 and 15:56 local time. All images were provided as geo-corrected reflectance products (Thompson et al. 2015) with a spatial resolution between 16.2 and 16.9 m, however, with remaining georeferencing errors and characteristic brightness gradients ( Figure 1) due to changing sun-sensor geometries during the acquisition ( Figure 2).  As part of the pre-launch activities of the EnMAP mission (Guanter et al. 2015), we simulated each AVIRIS flight line to an EnMAP-like image with 30 m resolution and 242 spectral bands using the EnMAP end-to-end simulator (Segl et al. 2012). The brightness gradients were neither corrected prior to nor modified during the EnMAP simulation. We performed a secondary geometric correction of each EnMAP-like image according to Scheffler et al. (2017). Based on visual assessment, we removed 42 bands that were strongly affected by atmospheric water absorption (1,311-1,465 nm and 1,783-2,044 nm) and 5 bands with poor reflectance retrieval (934-952 nm). Due to the focus on vegetation mapping, we masked out all water bodies.

Brightness gradient correction
An overview of different methods utilized for brightness gradient correction is provided in Table 1. The reflectance normalization approaches were brightness normalization (BNORM) and continuum removal (CONREM). For BNORM, each image spectrum was divided by its sum, projecting the spectra onto a plane thereby removing brightness differences while emphasizing the spectrum's shape (Berman et al. 2004;Collings et al. 2010). For CONREM, each image spectrum was divided by its convex hull after spectral smoothing using a Savitzky-Golay filter (Savitzky and Golay 1964). Similar to BNORM, this removes brightness differences and enhances the absorption features (Clark and Roush 1984).
Both empirical modelling approaches were based on the strategy proposed by Kennedy, Cohen, and Takao (1997). First, average reflectance values were calculated for each acrosstrack view angle. Second, a polynomial function was fitted through the average reflectance values. Finally, the function was used to normalize reflectance values of all pixels to nadir view. We used second order polynomials following the practices by Schiefer, Hostert, and Damm (2006) and Kennedy, Cohen, and Takao (1997). Empirical brightness correction was conducted prior to orthorectification and view angles were approximated by column numbers (Figure 3). For the class-wise empirical correction (EMPCLASS), we first pre-classified the images into green vegetation (GV) and non-photosynthetically active vegetation (NPV) based on manually set thresholds using the Normalized Difference Vegetation Index (NDVI) and the Plant Senescence Reflectance Index (Merzlyak et al. 1999). Due to the image acquisition date in the dry summer period, the GV class mainly comprised trees and shrubs whereas the NPV class mainly comprised senesced annual grasslands. Depending on their condition, agricultural crops were present in both classes. Then, two second-order polynomial functions, i.e., one for GV and one for NPV, were used for brightness gradient correction. For the global empirical correction (EMPGLOB), we fitted a single second-order polynomial to all vegetation pixels, i.e., GV and NPV combined. Note that for both empirical approaches, all non-vegetation landcover types were not considered for the fitting but were later included in the correction. Spectral consistency refers to any measure of similarity between spectral measurements of the same surface (Collings et al. 2010;Jensen et al. 2018). In the context of this study, the spectral consistency between adjacent flight lines was assessed using spectra from the regions where images overlapped. This way, we evaluated how well two measurements of the same surface observed under different sun-sensor geometries match before and after brightness gradient correction. For the 11 flight lines, 10 overlap regions exist at extreme positive and negative view angles. For each overlap region, we identified 20 patches of 4 × 4 to 9 × 9 EnMAP pixel blocks representing homogeneous surface types. We extracted the mean spectrum per patch, resulting in 20 spectral pairs per overlap region. The spectral consistency (SC) between spectral pairs was calculated by using the equation where N b is the number of bands and min b is the lower and max b the higher reflectance value of each band b. Then, we computed the average of all 20 ratios as a measure of spectral consistency between two adjacent flight lines. Spectral ratios of 1 indicate a perfect agreement, while ratios lower than 1 indicate disagreement. We used a ratio as a dimensionless measure to allow for a direct comparison of reflectance and normalized spectra.

Regression-based unmixing of vegetation fractions
We used regression-based unmixing with synthetically mixed training data from a spectral library for vegetation fraction mapping (Okujeni et al. 2017(Okujeni et al. , 2013. Endmembers for the spectral library were identified with help of high resolution Google Earth imagery and extracted from the simulated EnMAP imagery. Endmembers were collected for the two target classes woody vegetation (e.g., trees, shrubs, perennial crops) and non-woody vegetation (e.g., herbaceous, annual crops), as well as for a background class (e.g., bare soil, impervious). Endmembers were exclusively extracted from the nadir regions of the flight lines. Then, the library was used to create synthetically mixed training data, i.e., pairs of synthetically mixed spectra and associated mixing fractions for the two target categories. We created 1,200 random synthetic samples, which included mixtures of 2, 3 and 4 endmembers randomly drawn from the library and mixed with random mixing fractions. They were used to train Random Forest Regression models for each target class and the models were applied to the images to derive woody and non-woody vegetation fraction maps. We iterated the procedure 10 times and averaged intermediate predictions. This way, a multitude of different synthetic mixtures were included into the unmixing process while the training sample size was kept low (Okujeni et al. 2017).
For fraction map validation, we calculated the mean absolute error (MAE) between estimated fraction (Frac' i ) and available polygon-based reference fractions (Frac i ) from visual interpretation of high resolution Google-Earth imagery (Cooper et al. 2019) with the equation In total, we used 208 polygons with the extent of 3 × 3 EnMAP pixel blocks for the validation.

Spectral consistency of adjacent flight lines
The uncorrected images (NOCORR) show the lowest spectral consistency (Figure 4), particularly for the western and central flight lines (49-55). For the eastern flight lines (57-59) the ratio is close to 1.0. All four brightness gradient correction approaches led to improved spectral consistencies, though the general increase in consistency from west to east remains, regardless of the applied method. CONREM shows the greatest increase in spectral consistency, followed by BNORM, EMPCLASS, and EMPGLOB. Both empirical approaches show ratios below NOCORR for overlap regions 57-58 and 58-59.

Vegetation fraction mapping
Results show that the brightness gradients within uncorrected scenes adversely affect the fractional accuracy (Table 2). CONREM and BNORM do not significantly improve the accuracies. In contrast, both empirical approaches result in considerably improved accuracies. Differences in average MAEs compared to NOCORR are −3.1 for EMPGLOB and −4.6 for EMPCLASS, respectively. Average MAEs of fraction maps based on reference data grouped into five different view angle categories are illustrated in Table 3. Uncorrected images with high positive view angles tend to have lower fractional accuracies. CONREM and BNORM lead to improved estimates at these high positive view angles, yet tend to reduce accuracies in the other angle categories. EMPGLOB and EMPCLASS considerably improve accuracies at high positive view angles, while accuracies for all other angle categories remain stable or are slightly improved.
EnMAP mosaics (Figures 1 & 5) and fraction maps ( Figure 5) visually highlight the enhancement in image and map quality after the brightness gradient correction with EMPCLASS. Only the EMPCLASS mosaic is shown because it had good spectral consistency and the best fractional accuracies compared to the other methods. The corrected EnMAP mosaic is almost seamless between adjacent flight lines. Fraction maps derived from the uncorrected images show a strong over-and underestimation of non-woody vegetation and woody vegetation   along the seams, leading to interrupted gradients of the cover types between flight lines. These effects are compensated for in the fraction maps obtained from the corrected images, where the landscape is more realistically represented by continuous transitions inside and between vegetation types.

Discussion
Our first objective was to evaluate the use of four brightness gradient correction approaches to enhance the quality of hyperspectral image mosaics from multiple flight lines. Each of the correction techniques led to greater spectral consistencies between adjacent flight lines as compared to uncorrected imagery. Both reflectance normalization approaches (BNORM, CONREM) resulted in slightly better spectral consistencies than the empirical approaches (EMPGLOB, EMPCLASS). Comparison between both empirical approaches shows the benefit of using surface type dependent models. However, results for flight lines which are not severely affected by brightness gradients indicate that empirical correction may also slightly decrease the spectral consistency. We therefore recommend a careful evaluation of the sunsensor constellation during the image acquisition (compare Figure 2) and the spectral consistency between overlapping flight lines (compare Section 2.3 and 3.1) to determine the need for brightness gradient correction prior to image mosaicking and land cover mapping. Our second objective was to demonstrate the impact of the four correction methods on regression-based unmixing. Class-wise and average MAEs across the entire study site and stratified by view angle show that both normalization approaches (BNORM, CONREM) did not improve results. Normalized reflectance data emphasize the shape of the spectra, minimize illumination and shading effects that relate to vegetation structure at the stand level, and remove characteristic brightness differences between darker and brighter vegetation types (e.g., broadleaf trees vs. green grass). However, map accuracies suggest that this information is also critical for quantitative differentiation of the vegetation types. In contrast, both empirical correction methods (EMPGLOB, EMPCLASS) considerably improve the accuracies, and fraction maps more realistically represent continuous transitions between vegetation types along the seams of adjacent flight lines. This indicates that both approaches preserve the spectral information critical for fractional estimation of woody and non-woody vegetation cover. The class-wise empirical correction performed better than the global approach, indicating that heterogeneous ecosystems with different vegetation types require more than one polynomial model to effectively account for brightness gradients caused by surface dependent differences in reflectance anisotropy. We showed that a pre-classification into GV and NPV effectively supports brightness gradient correction and improves subsequent mapping. More refined vegetation classes might further improve the correction, as more detailed classspecific anisotropic reflectance characteristics will be accounted for. However, such a refined pre-classification will be impeded by spectral confusion caused by the brightness gradient itself and by the impact of spectral mixing. Further, more detailed classes may not occur across the range of view angles to adequately fit an empirical model. Given this, our approach favoured the simple solution to avoid propagating errors through the correction procedure.

Conclusions
This study underpins the importance of brightness gradient correction on hyperspectral image mosaics from multiple flight lines. A class-wise empirical correction strategy best enhanced the quality of the image mosaic, while preserving the spectral information needed for mapping woody and non-woody vegetation fractions. Overall, we showed that good brightness gradient correction is needed to ensure high quality mapping of ecosystems by means hyperspectral image mosaics. By automating the pre-classification, the presented class-wise empirical approach is recommended as a simple and reproducible technique that may be integrated into pre-processing chains to compensate for brightness gradients during reflectance retrieval.