Influence of input reflectance values on climate-based daylight metrics using sensitivity analysis

The insertion of climate-based daylight metrics as a requirement in several design guidelines calls for a better understanding of their effectiveness. This paper draws attention to the sensitivity of annual daylight metrics to changes in input reflectance values. The uncertainties related to the choice of guidelines and of simulation techniques were also considered. Total Annual Illumination (TAI) showed the most consistent correlation and the highest sensitivity to variations in reflectance (up to ±60% from the benchmark), independently of the geometrical characteristics of the space. Other annual metrics were less sensitive, or showed a poorer correlation. The deviations among different simulation techniques varied with the chosen metric too ( for TAI), but all techniques were equally affected by variations in reflectance. The results highlighted the importance of selecting appropriate metrics for annual climate-based daylight evaluations.


Introduction
In daylighting practice the use of appropriate surface finishes and geometry to bring daylight deeper into the spaces is a well known and effective strategy to increase diffuse light penetration and brighten up deep plan spaces. The daylight standards currently employed have long since recognised the importance of inter-reflected light. One of the commonly used 'rule of thumb' methods is the equation to predict the average daylight factor in a space (Reinhart and LoVerso 2010). First proposed by Lynes (1979), the equation was revised by Crisp and Littlefair (1984) following validation tests using scale models . The revised equation is (1) Here T is the effective transmittance of the window(s); W is the net area of side window(s); θ is the angle in degrees subtended in vertical plane by sky visible from the centre of a window; M is the maintenance factor; A is the total area of bounding surfaces of an interior: Floor + ceiling + walls, including window(s); R is the areaweighted mean reflectance of interior bounding surfaces. Thus, for any given space geometry, there is strong dependance of the estimated average daylight factor value on the reflectance of the building surfaces. The average daylight factor equation was, of course, formulated and then calibrated to be applicable for standard overcast sky conditions, and a notional external reflectance value since there *Corresponding author. Email: E.Brembilla@lboro.ac.uk is no option to vary this quantity. Illuminance gradients occurring in and around a space under realistic conditions with sun and non-overcast sky luminance patterns will invariably be significantly greater than those experienced under overcast skies (CIE standard or otherwise). Correct specification of realistic optical properties for opaque and transparent building surfaces in daylighting simulation has always been an important, if largely academic, consideration. However, the increasing role placed on achieving compliance targets at the design stage has cast a greater emphasis on both the correct assignment of optical properties, and on the understanding of the key sensitivities in the outputs to variations in these properties. Since many compliance criteria are in terms of single threshold values, a small change in, say, wall reflectance, could result in the difference between pass and fail.
Obtaining the correct properties of clear glazing systems is usually a straightforward matter as the data are readily available from manufacturers. All the other surfaces in an ordinary room are likely to be opaque, either diffuse or specular. For realistic renderings and impressions of the building design, as well as for glare analyses, the correct assignment of all optical properties (i.e. colour, reflectance, specularity and roughness when using Radiance) can make a significant difference in the appearance and in the evaluation of the simulated scenes. For this kind of purposes, it is usually suggested to take exact measurements of the desired materials when possible (Brembilla et al. 2016), or to use databases that reports the measured properties of a number of different surface types, organised in libraries (Jakubiec 2016).
For quantitative evaluations all the materials can be considered as perfect diffusers, i.e. Lambertian, and the specular component therefore ignored (SLL and NPL 2001;CIBSE 2015), with the exception of purely mirroring surfaces of considerable size. The problem is reduced to the assignment of reflectance values for all the elements of the design project, but even so the physical uncertainty related to this practice remains high for a number of reasons; e.g. the finishes might be unknown, they might change later on, the furniture or the fixtures are not part of the modelled geometry, the cleanliness or maintenance factor cannot be told a priori. Several guidelines on the calculation of metrics for daylighting, traditional and annual, report indications of this kind: Use actual surface reflectance values for walls, floors, ceilings, and furniture. If actual reflectance values are not known, use the following default reflectance values. (Illuminating Engineering Society 2012) Some of the suggested values are reported in Table 1, with the indication of the guideline they were sourced from: the Illuminating Engineering Society (IES) LM-83-12 (Illuminating Engineering Society 2012), used as reference for LEED v4 (US Green Building Council USGBC); CIBSE Application Manual 11 on Building Performance Modelling (CIBSE 2015); two CIBSE Lighting Guides, LG5 Lighting for Education (CIBSE/SLL 2011) and LG7 Offices (CIBSE/SLL 2005); and the requirements for the Priority Schools Building Programme (PSBP) promoted by the UK EFA (Education Funding Agency 2014). In other codes, such as the Illuminating Engineering Society of North America Handbook (IESNA 2000) and the British Standard 8206 Part 2 (BSI 2008), a list of reflectance values relative to several construction materials is given instead of standard values.
In 2013 the UK Education Funding Agency (EFA) made climate-based daylight modelling (CBDM) a mandatory requirement for the evaluation of designs submitted for the PSBP. School designs submitted to the PSBP must achieve certain 'target' criteria for the useful daylight illuminance metric. This is believed to be the first major upgrade to mandatory daylight requirements since the introduction of the daylight factor more than half a century ago. In the US, a climate-based daylight metric approved by the IESNA has appeared in the latest version of LEED. These moves have placed the practical application of CBDM in the spotlight. In particular, issues related to quality assurance and the assumptions often employed to set key input parameters such as reflectance since, crucially, pass/fail outcomes can depend on the values chosen.
In this paper, a thorough analysis of the sensitivity of CBDM evaluations to reflectance values assignment is taken as illustrative case to analyse in detail the use of annual metrics for uncertainty and parametric studies. To account for the variability in some input factors, that is, inevitably present when performing building performance simulations, it is now common to perform uncertainty and sensitivity analyses through widely applied statistical methods (Hopfe 2009). In daylighting, the literature offers less examples in which the same concept has been applied; a previous work by Tregenza (2016) touched on the subject, but keeping the error propagation analysis restricted to analytical models to calculate Daylight Coefficients, rather than using computer simulation for a full climate-based daylight modelling (CBDM) evaluation. On the other hand, parametric and optimisation analyses that use annual daylight metrics are becoming widespread. They usually employ metrics required by current guidelines, without considering the actual sensitivity of those metrics to the analysed parameters.

Methodology
To tackle the problem of the sensitivity of CBDM to surface reflectance values, several types of uncertainty in the input factors were considered, and each of them was analysed with an appropriate methodology. The main factors taken into account were: • The uncertainty deriving from different guideline requirements. A brute-force approach was applied in this case, comparing the values suggested in the literature. • The uncertainty related to a specific scenario. Four case study classrooms with different designs were used throughout the study, to understand which considerations can be generalised to a broader suite of spaces and which cannot. • The physical uncertainty of the reflectance measured or assumed for the single elements in the rooms, i.e. walls, floor, ceiling, external ground and window frames. The influence of each of these surfaces and their relationship with the final results was investigated thanks to a Sensitivity Analysis (SA) run with the Method of Morris. • The modelling uncertainty related to the choice of simulation method. Five Radiance-based methods were used to evaluate all four classrooms and an inter-model comparison was carried out.
These four types of uncertainty -i.e. those related to guidelines, scenario, physical and modelling choices -are likely to be encountered by a designer at the initial stages of a luminous performance evaluation. For this reason, they were all considered in the analysis, to give a comprehensive evaluation of how CBDM might be affected by the assignment of optical properties. CBDM typically generates a time-series of illuminance values at every 'sensor' point. The most straightforward way to evaluate provision over the year is to determine at each sensor point the number of hours for which an illuminance level, say 300 lux, is achieved. This is known as 'daylight autonomy' (DA) and the evaluation period is typically the working day. The DA gives an indication of the time of the year for which, in principle, artificial lighting can be avoided because the design-level illuminance (e.g. 300 lux) is being provided by daylight. Another climate-based metric similar to daylight autonomy is 'useful daylight illuminance' or UDI. The key difference is that UDI predicts the occurrence of illuminance within ranges, and the occurrence of illuminances outside those ranges. Another metric is the cumulative daylight illuminance received at a point (or across an area) for the entire year. This measure is often used in conservation studies to describe the exposure of, say, artworks to potentially damaging daylight exposure. A single number for entire space under evaluation can be derived by computing, say, the space averaged value for a particular metric. Usually, this is the space average taken across the sensor grid at desk height. For compliance purposes, a single number 'target' is often given since that simplifies the specification (and subsequent checking) to a straightforward pass/fail criterion. To have the greatest relevance to existing guidelines, a range of metricsdescribed below -were employed for the sensitivity study.
All the results deriving from the analyses were expressed with the following annual daylight metrics: ing plane that complies with DA requirements for more than 50% of the occupied time (Reinhart et al. 2014). The concept is similar to that of Spatial Daylight Autonomy (sDA), but without considering any model for the operation of dynamic shadings, e.g. as the model inserted in IES LM-83-12.
These metrics were compared with each other, revealing their sensitivity to the factors previously mentioned. Daylight Factor (DF) values were also reported for comparison against traditional guidelines. Annual Sunlight Exposure (ASE) was not used, although is currently employed in IES LM-83-12 and LEED v4 recommendations, because it is a metric based on direct sunlight illuminance alone, and therefore not affected by changes in reflectance values. The Daylit Area metric was preferred to the sDA (also recommended IES LM-83-12) as the intention of this study was to understand the intrinsic performance of the space.
A daylight evaluation of a space without (simulated) occupant(s) deploying blinds/shades, etc., discloses what may be termed the intrinsic or asset daylighting performance of the space. It is arguably the case that a prediction of daylight performance which includes, say, simulated deployment of blinds should be closer to that of the actual building when it is in normal use. However, the uncertainties in occupant behaviour are significant for individual side-lit office spaces, and they can become overwhelming for larger open-plan spaces where the permutations for shade deployment -and consequent impact on daylight provision -become enormous. The most commonplace shading fixtures tend to be some form of venetian blinds. The optical properties, i.e. Bidirectional Scattering Distribution Function (BSDF), of a venetian blind are highly dependent on the slat-angle and slat-separation -both of which are at the whim of users who will lower and adjust shades to varying degrees according instantaneous conditions and personal preferences. This is in addition to the reflective properties of the slats which tend to be a combination of diffuse and part-specular. Thus, the venetian blind and its impact on the daylight entering a space can be enormously confounding to attempt to model accurately. Cutting straight to an evaluation for the occupied building may result in the designer missing out on opportunities to improve the intrinsic daylighting potential of the building since this might be masked by uncertainties present in both the probabilistic models of occupant behaviour and the optical properties of the blinds.
Hereafter, the four classrooms that were used as case studies and their main geometrical characteristics are presented, followed by a description of the five simulation methods and by an overview of the Method of Morris used for the SA.

Description of the case studies
As noted in the Introduction, the PSBP mandatory requirement for CBDM is the first of its kind anywhere in the world. Accordingly, the authors decided that the scenarios for the evaluation of sensitivity should focus on classroom spaces since (UK) school buildings are, arguably, those under the greatest scrutiny for daylight modellingapproval to proceed with a school building proposal costing several tens of millions of pounds may depend on the outcome of a CBDM evaluation. The case studies selected for the present work were four existing classrooms in the UK, characterised by different period of construction, size, orientation and window size. The rooms are part of another research project that is relating the subjective impression of daylight performance to objective measures of the luminous environment through long-term monitoring with High Dynamic Range (HDR) images (Drosou et al. 2015). These were chosen in preference to, say, idealised 'shoe-box models', to include a range of classroom types where the basic forms were founded on real-world examples. Nevertheless, for this study, the classrooms serve as exemplars of a variety of realistic types, rather than particular examples in unique settings/context. The window to wall area ratios for the four classrooms ranged from 7% to 69%. And the predicted average and median daylight factors across the horizontal working plane for the four classrooms ranged from 1.5% to 6.8% (average) and 0.8% to 4.6% (median). Table 2 reports all characteristic values in more detail.
The code names assigned to the four classrooms are L3, L7, M1 and M5. Figure 1 shows, for each classroom, an interior rendering created with Radiance; an exterior view of the 3D model created in SketchUp; and the floor plan with an indication about the North direction. L3 is a sidelit space with a glazed curtain-wall facing approximately North-West direction; L7 is a multi-aspect room with the major windows oriented towards North-East and others towards South-East; M1 is a deep plan space with the aperture on the smaller side that faces South; M5 is characterised by a sloped ceiling and has apertures on opposite sides, with the main window towards North and an additional clerestory window on the South side, where the ceiling is higher. The glass properties and the shading systems were not included in this analysis, and a transmittance of T vis = 0.80 was assumed for all the windows, in all rooms. For the choice of these spaces, one of the requisites was that 'traditional' taught classes were the main activity held in them; the choice of an horizontal plane (h = 0.8 m) for the simulated illuminance records was therefore deemed appropriate for this type of tasks. The climate data used for the simulation was the EPW for London Gatwick with an hourly time step and the occupancy schedule was considered to be from 8 am to 4 pm.

Simulation tools
The number of programs that can perform CBDM is constantly growing and it is getting easier to find annual daylighting simulation capabilities within whole-building performance software. They can either be based on radiosity or ray-tracing engines, although the latter is still more common. Especially for research purposes, Radiance (Ward Larson et al. 1998) is the preferred ray-tracer and provided the back-bone for CBDM development since the late 1990s. Several Radiance-based methods to perform annual daylight evaluations appeared since then, characterised by different techniques to describe the sky vault and the contribution of the sun.
For these reasons, in the present work it was chosen to limit the analysis to Radiance-based methods, and to software where the simulations could be performed via command-line scripts or other parametric tools. As a matter of fact, the implementation of SA techniques can be very challenging to apply on a Graphical User Interface (GUI) that does not offer the possibility to automate the process.  The methods considered for the current analysis were: (i) Four-Component method: It uses the Daylight Coefficients (DC) method with a Tregenza subdivision (i.e. 145 patches) and blended CIE luminance models for the stochastic calculation of sky light, while sunlight is calculated deterministically from 2056 light point sources evenly distributed over the hemisphere. An rtrace run is performed for each of the patches. (ii) DAYSIM: One of the most widespread back-end tools to perform CBDM. It implements a modified version of rtrace for the light redistribution simulation. The publicly available version use the Tregenza patches scheme for skylight and up to 65 points over the sun path as sunlight sources, with the sun luminance interpolated between the closest four points to the actual sun position. The luminance distribution is derived from weather files data using the Perez All-Weather model (Perez et al. 1993). (iii) Two-Phase method: Instead of the classic rtrace command to simulate light behaviour, a new rcontrib (initially called rtcontrib) command was specifically introduced for annual simulations. The sun luminance is assigned to the three sky patches closest to the actual sun position and the sky subdivision can have variable resolution. The sun and sky contributions can therefore be accounted for in a single run and the computational cost can noticeably diminish. However, in order for this method to work, the ambient interpolation has to be switched off (i.e. -aa 0, -as 0, -ar 0), giving rise to noisier images and requiring a higher number of ambient divisions (-ad). (iv) Three-Phase method: In order to simulate the behaviour of Complex Fenestration System (CFS), this method was introduced on top of the Two-Phase method, using the same rcontrib command, but splitting the ray-tracing process in two, one run for the exterior scene and one for the interior.
The results matrix can be then multiplied to the matrix that describes the window BSDF material. This kind of function is generally built on a Klems basis hemisphere and is used to spatially relate the luminous flux coming from the exterior to the one transmitted by the window system itself towards the interior. (v) Five-Phase method: To increase the accuracy of the Three-Phase method when evaluating the performance of CFSs, the direct sunlight contribution is re-simulated using 5185 point-like sources evenly distributed over the hemisphere and applying a variable resolution Tensor-Tree BSDF material instead of the Klems one; in this way, peaks of light can be traced more reliably from the sun position and then accurately accounted for at the window transmission step.
The sky vault subdivision used in this paper for the Two-, Three-and Five-Phase methods followed a Reinhart Note: The values reported in brackets were used for class M1, which has a deeper plan.
MF:2 scheme, equivalent to 577 sky patches and one patch for the ground. The ambient calculation settings for each method are reported in Table 3. The same settings were used for all rooms but for M1, which has a deeper plan that requires an higher number of bounces (or equivalent change in parameters). What is probably still considered the definitive validation study for any daylight prediction method (physical model, analytical or simulation) was carried out in the mid-1990s using data collected by the BRE as part of the International Daylight Measurement Programme -the data are sometimes referred to as the BRE-IDMP validation dataset (Mardaljevic 1995(Mardaljevic , 2001. That study showed that illuminances predicted using the Radiance system could be within ± 10% of measured values, i.e. within the accuracy limits of the measuring instruments themselves. This, quite remarkable, degree of precision needs to be judged alongside the high level of inaccuracies (often in excess of 100%) that were determined to be fairly typical for physical modelling (Cannon-Brookes 1997). Using the same BRE-IDMP dataset, the Four-Component method was shown to have comparable high accuracy to the standard Radiance calculation (Mardaljevic 2000). Accordingly, the authors consider the Four-Component method to be the benchmark CBDM formulation for this study.

SA: the method of Morris
The enhanced Morris method can be used for an initial screening on the inputs when performing an SA on a model. It has been previously applied to Building Performance Simulation (BPS) models in a number of works (Hopfe and Hensen 2011;McLeod et al. 2013;Østergård et al. 2015) and it is considered a reliable method for non-monotonic models with interacting input factors, even with a low number of samples (Campolongo et al. 2007;Tian 2013). Due to the long computational load, that is, required by some of the considered daylight simulation techniques, the Morris method was adopted here for the low number of runs necessary to the analysis. All the steps to run the SA, i.e. the sampling process to obtain the input values and the actual analysis on the simulation results, were performed using the SALib v0.7.1 package in Python (Usher et al. 2016).
In a related work that preceded this paper (Brembilla et al. 2015), the method of Morris was applied to a wide range of reflectance values, that spanned from 0.01 to 0.99 to include all possible values and to give the same importance to all the elements considered in the analysis.
In the present work however, relevance is given to the variability of the results when realistic reflectance values are assigned to the model. For the majority of daylit real-world architectural spaces, the area-weighted average reflectance is typically in the range 40-60 % , whilst values approaching 70% are extremely rare, e.g. a entirely bright white room (including the floor) with a small window. Real daylit spaces with an area-weighted average reflectance exceeding 70% are highly implausible since the effective reflectance of normal glazing is ∼ 10%. Note, using Radiance to predict illumination levels for improbably high reflectance spaces will result in considerable under-prediction. This was noted (but not explained) in a 2005 paper on datasets for validation of lighting programs which considered test scenes with area-weighted average reflectance values ranging from 0% to 95% (Maamari et al. 2005). The reported huge under-prediction from Radiance for very high reflectance scenes is caused by one of the many optimisations employed in the software to make the modelling of real-world spaces as efficient as possible. Specifically, the Radiance AVGREFL macro which is preset at compilation to 0.50, i.e. it assumes an area-weighted average reflectance of 50% for the scene. This value determines how many rays will be traced in deeper levels, and the preset value works effectively for the overwhelming majority of real-world architectural spaces. The preset value for the AVGREFL macro can be overridden by recompiling Radiance, a relatively routine task. It was deliberately not made one of the parameters which can be adjusted by the user at execution since, for all plausible real-world architectural spaces, the potential negative impact on computational efficiency could be enormous (Greg Ward -private communication with John Mardaljevic, 22/23 February 2005). Consequently, only plausible, real-world surface reflectance values were selected for the SA reported in this paper. Table 4 reports the range limits that were set for each of the considered element; these ranges were deemed to represent variable, and at the same time realistic, conditions that can be found in common spaces such as classrooms or offices.
As more methods and classrooms have been added to the analysis, the number of samples here was lowered to reduce the computational load. Rather, here the Morris method is applied with the use of optimal trajectories (8 trajectories, 8 levels and 4 grid-jumps) so that the final results from the SA can still be considered reliable (Campolongo et al. 2007;Confalonieri et al. 2010). By using 8 trajectories the total number of samples is 48, as per Equation (2) where D is the number of input parameters (5), k the number of trajectories and n the final samples.
Each input space is therefore divided into eight parts and from each of these parts one value can be picked for the random sampling process. Between two consecutive simulation runs, only one of the input parameters is changed, as the method of Morris applies a One-At-a-Time procedure. The differences in results due to the input variations are called elementary effects, as sometimes the Morris method itself is called. The final analysis looks at the normal distribution of these elementary effects for each of the input parameters, and gives an indication of the parameter's importance rank, as well as its relationship with the overall results obtained from the simulation model, which can be linear, monotonic or non-monotonic.

Results and discussion
This section presents the results obtained for each of the analyses conducted on reflectance values in CBDM. The first set of results (Section 3.1) derives from the brute-force comparison made between existing guideline recommendations. The second part (Section 3.2) focuses on the output variation when evaluating rooms with different geometries, while altering reflectances. Preliminary considerations on the use of different annual daylight metrics were drawn in Section 3.3. The influence of the single elements reflectance in specific room configurations were brought out by the Morris analysis. Last, Section 3.4 shows the outcomes from the inter-model comparison and the SA results obtained with different modelling methods.

Comparison between different guideline recommendations
One of the common approach to reflectance assignment for daylight simulation purposes is to consult existing guidelines and adopt the suggested values. Using the 'standard' reflectances specified in Table 1, the evaluation on the four case study classrooms was performed with the Four-Component method and the results in terms of TAI and DA are presented in Figure 2. Where acceptable reflectance ranges were suggested instead of a single value, both the minimum and the maximum limits were compared. Each room is obviously characterised by different results, as the four geometries are noticeably different in features that affect access and redistribution of light; therefore, the selected metrics (in this case TAI and DA) should  Table 1. Where minimum and maximum allowed values were found, a different simulation was run for each of them. The dark grey bars represent the results for the minimum reflectances and the light grey bars for the maximum. correctly display a variation in daylighting performance. However, something more can be noticed in the graph of Figure 2. Even though the five assigned reflectance datasets (i.e. one for each of the considered guidelines) were the same, some rooms showed wider variations in results when changing reflectance values than others. For each room, Table 5 reports the mean and standard deviation of all results obtained using different guideline recommendations for reflectance assignment, together with their coefficient of variation. The coefficient of variation (C V ) is the ratio between the standard deviation and the mean of a distribution; it gives a non-dimensional characterisation of the dispersion of the values, i.e. their variability in relation to their average. Room L7 results in the highest mean TAI value of 6750 ± 1152 klx hrs; its DA mean value shows however the smallest dispersion, C V = 4%. On the opposite hand, DA results for room M1 experience a variation of C V = 19%. To generalise the results and to understand in greater detail what are the causes of the variability in annual CBDM results when changes are made to the initial reflectance values, an SA was carried out.

SA results: geometrical variations
In the preliminary analysis, either only the minimum or the maximum values were applied, following guidelines suggestions for default reflectances. What if the combination of low and high values on different interior surfaces is applied? What if the designer decides to apply realistic optical properties found from material suppliers or databases? The aim of the SA was to understand how the results change given a random combination of values within plausible limits, and to distinguish the surfaces that strongly affect the overall results trend from the ones that do not. Additionally, considerations on how these changes affect rooms with different characteristics were drawn, as well as considerations on how some of the most commonly used annual metrics describe these variations overall.
The sampling process carried out with the Method of Morris identified eight values within each of the ranges defined in relation to the five input parameters. At the top of Figure 3, the input reflectance values are represented in the order given from the random seeding process of the Morris method. Each surface was assigned one out of eight fixed reflectance values within its specified limits, and each simulation run differed from the previous for one of these values only. From the left to the right of the image, the inputs and outputs of the 48 simulation runs for all the four rooms are displayed. Even though the outputs can be considered single instances, they are represented with lines to highlight the trend resulting from variable reflectance values. The first graph below the input reflectances shows the DF results, while the successive graphs represents four annual metrics: TAI; DA (300 lx threshold); DA 300 [50%] (300 lx threshold for 50% of the time); UDI-c (illuminance within 100 and 3000 lx); and UDI-x (illuminance over 3000 lx). For each room, five solid lines illustrate CBDM results obtained from the five methods analysed: Two-, Three-, Five-Phase, Four-Component and DAYSIM. The base case scenario result for each room is also reported with a dash-dot line; that is considered here the benchmark value, obtained with the Four-Component method and with the PSBP suggested reflectance values; 0.2 for floor and external ground, 0.5 for interior walls and 0.7 for ceiling.
For both DF and TAI, the four rooms show a similar behaviour among each other when varying the reflectance across the 48 simulation runs. There are also similarities between DF and TAI themselves, suggesting that the relationship between reflectance values and final results is mostly maintained when passing from static to annual analyses. DA values behave differently, with highly lit rooms showing a small variation when changing reflectance, and darker rooms strongly responding to the varying reflectance inputs. This could be due to the fact that dark rooms are more dependent on inter-reflected light, but it can be attributed to the choice of DA threshold too, equal to 300 lx. Bright spaces are characterised by a benchmark DA of about 80%, and any increase in illuminance values due to higher reflectances that is actually accounted for by this metric would be for instances that passed from being lower than 300 lx to higher than that threshold. This is bound to happen more frequently for rooms whose performance starts from lower DA values, such as M1 and M5, which consequently show a higher variability in results. It is interesting to notice that for these two rooms the high variability is not present only in result ranges, but between different simulation methods employed too. This aspect will be investigated further on, in Section 3.4. DA 300 [50%] is affected by the same problem; for rooms with a benchmark DA 300 [50%] value of 100%, this metric does not respond to any additional increment in incoming light, as the whole workplane is already identified as daylit area. When the starting levels are very high, as for room L7, lowering the overall reflectance does not influence the final result.
Rooms with lower benchmark values, as M1 and M5, are subjected to wider output variations than the variation recorded in DA values. The difference among simulation methods is also more prominent, likely because this metric takes into account the spatial distribution as well. UDI results are not behaving in a monotonic fashion as other annual CBDM metrics. Having both a lower and a higher threshold, UDI-c can potentially decrease when the overall reflectance increases, or the opposite. This is visible in the UDI-c results graph for room L7; starting from a benchmark value of 80%, already lowered by the presence of many illuminance instances higher than 3000 lx (UDI-x = 17%), any increase in reflectance values results in a higher UDI-x and a lower UDI-c.
The analysis on the correlation between reflectance assignment and geometrical features of the design went a step further with the use of the Method of Morris. Once it was understood that TAI is the annual metric that better correlates with reflectance values, this metric was used as an indicator for the analysis on the single input parameters, i.e. the main surfaces in the model.
The Morris plots for TAI values are presented in Figure 4, for rooms L3 and M1, simulated with the Four-Component method. The results for rooms L7 and M5 are very similar to those found for rooms L3 and M1, respectively. Morris analyses can give a ranking of input parameters, i.e. the classrooms interior surfaces here, ordered by their influence on the overall results, as displayed on the left of the figures. They can also give an indication of the parameters' relationship with the results, based on the ratio σ/μ * , where σ is the standard deviation of the elementary effects (i.e. differences in results due to input variations) distribution, and μ * is the mean absolute value of the distribution. Those parameters that sits in the graph below the line σ/μ * = 0.1 can be considered to have an almost linear relationship with the results; if they appear below the lines σ/μ * = 0.5 and σ/μ * = 1 than they have respectively a monotonic and an almost-monotonic behaviour; above the line σ/μ * = 1, the parameters show a highly nonlinear relationship with the final results, indicating that there might be an interaction with other input factors.
For all the rooms, floor and frame exhibit a small influence on the overall results, independently of which metric is used. Ceiling, walls and external ground play a more important role, but their ranking changes slightly for various geometries. The results for room L3, for example, are mostly affected by variations in the outdoor ground plane, while for room M1 the walls are the most influential factor; the explanation of this can possibly be found in the difference of Window-to-Wall Ratio (WWR) between the two spaces. Room L3 has a WWR of 69%, while for M1 the WWR is equal to 25%. It can be easily inferred that the reflectance values of the exterior environment play a bigger role for rooms with larger apertures, while for rooms with smaller windows, the role of each element is more balanced.

SA results: CBDM metrics
To understand better how the analysed metrics are differently affected by changes in surface reflectance, their values were normalised against the mean of the series of 48 results. Figure 5 shows the normalisation for the six metrics considered previously (DF, TAI, DA, sDA, UDI-c and UDI-x), for the results obtained with the Four-Component method. First of all, both DF and TAI show a consistent behaviour overall, independently of the room geometry and features; only TAI values for room L3 are slightly divergent from the other rooms at the highest and lowest peaks. There is a striking similarity between the amplitude of the variations expressed with DF and with TAI; both metrics reach values up to 60% higher and 30% lower than the mean value. On the other hand, the variations registered by DA are strongly affected by the initial amount received in the benchmark room design. The more lit the space is, the smaller variations in DA are recorded ( ± 7%), as for room L7. For darker spaces, as M1, the variation is dramatically increased (within ± 36%). This can be recognised also by Figure 5. Results obtained from all simulation runs using the Four-Component method, expressed with different metrics: DF, TAI, DA 300 , sDA 300,50% , UDI-c 100-3000 lx, and UDI-x ( > 3000 lx). All values were normalised against the mean of each series, to highlight the difference in relative variations among the considered metrics. Available in colour online. looking at the statistical distribution of TAI and DA results reported in Table 6. The coefficients of variation (C V ) for TAI values range from 13% to 23%, and do not seem to be related to the rooms' luminous levels. Instead, the C V for DA are notably different for rooms L3 and L7, compared to rooms M1 and M5. While the latter can be compared with the dispersion found for TAI values, the former display a much lower variability. The higher the mean DA value is, the lower the variation becomes.
DA 300 [50%] shows a peculiar behaviour, different from both DA and TAI. The extent of the variation reaches values as wide as for TAI (up to + 61%), but only for the rooms with a low baseline illumination (M1 and M5). For the other rooms, some plateaus are noticeable in the graph, corresponding to the lack of any variation whenever the DA 300 [50%] = 100% cap is reached. UDI-c values are all contained in a ± 20% range, but the rooms that receive more light see an inversion in the relationship between input parameters and output metric. Room L7 records variations practically specular to the ones obtained for rooms M1 and M5. Room L3, which is the best performing one in terms of UDI-c (91%), results in the smallest variation range of all four classrooms (within ± 7%). This would suggest that the performance initially assessed with this metric on a well designed baseline case is not strongly affected by realistic changes in surface reflectance. UDIx seems to be the most sensitive metrics among them all, although for most rooms the mean of the series are too small to provide a reliable normalisation. Only in room L7 the benchmark UDI-x value (14%) can be considered significant; for this room, the increase in reflectances is almost doubling the mean UDI-x. DA and the useful daylight achieved range can be considered to be measures of daylight sufficiency. However, as is the case with daylight measures based on a threshold value, the metrics are prone to saturation to varying degrees depending of the particular threshold values used. UDI is, on the whole, less prone to saturation than DA because high illuminances will reduce the occurrence of UDI achieved provided that they are greater than the UDI exceeded threshold value (i.e. 3000 lx). But, for cases where illuminances are rarely greater than 3000 lx, then both DA 300 and UDI supplementary will give largely similar results. For this reason, total annual illumination -a continuous, cumulative measure -proves to be the daylight metric that is the most responsive to changes in reflectance values. And therefore, arguably, a measure that should be included as an additional factor in daylight optimisation studies using CBDM.

SA results: CBDM techniques
The relations between changes in reflectance and geometrical features were assessed in the previous analyses. The study then proceeded with the comparison between different CBDM techniques and how each of them behaves when reflectance values vary. The evaluation on internal reflectances can be considered a proxy to assess how each of the simulation methods deal with inter-reflections.
To look at the differences due to the chosen simulation method, rather than the model geometry, the results from the Four-Component method are now presented with those from DAYSIM and the Two-, Three-and Five-Phase methods (Figure 6), considering only one room. Classroom M1 was chosen as the five techniques showed the largest differences when assessing it. Being a deep plan space, the inter-reflections played a bigger role than in the other rooms.
The results in Figure 6 were sorted in ascending order, based on the area-weighted mean reflectance. This value follows the same concept as the term R in the analytical DF formula, but the reflectance of the external ground was assigned to the window area instead.
It is possible to identify the group of 'phased' methods, i.e. Two-, Three-and Five-Phase methods, as behaving very similarly between each others. This was expected for the Three-and Five-Phase methods, as they differ only for the direct sunlight calculation, which is not affected by changes in reflectance, while they both rely on the Three-Phase method to calculate the diffuse and inter-reflected parts of daylight. Less foreseeable was the strong agreement that the Two-Phase method shows with them too, which holds true for almost all cases and all rooms, except for very few instances where there is a very high overall reflectance.
The Four-Component method and DAYSIM tend to record lower illuminance levels than the 'phased' methods, starting from the base case results (signed with dash-dot lines). This happens prevalently for rooms M1 and M5, Figure 6. Inputs and outputs for all 48 simulation runs of room M1, ordered by increasing area-weighted mean reflectance. At the top the input area-weighted mean reflectance values are shown, with a dashed line signalling the 0.5 reflectance. Below, the corresponding results expressed as TAI, DA, UDI-x (over 3000 lx), UDI-c (100-3000 lx) and UDI-n (0-100 lx) are reported, for each of the five CBDM methods analysed. The base case results, obtained with default reflectance values, are reported for all methods with dash-dot lines. Available in colour online. while in the rooms with higher levels of daylight all the methods reach an almost perfect agreement. For room M1, TAI values obtained with the Three-Phase method are about 14% higher than those obtained with the Four-Component method, while for DA values the relative difference is 12%. From the plotted UDI results in Figure 6, it can be seen how the main differences are recorded for illuminance values lower than 3000 lx; both DAYSIM and the Four-Component method shows a higher number of instances that fall into the 0-100 lx range, while the 'phased' methods have a higher ratio of 100-3000 lx instances. Table 7 reports the Mean Bias Deviation (MBD) and Root Mean Square Deviation (RMSD) for all simulation techniques, compared against the benchmark 4-component method. If expressed in terms of TAI and DA, all techniques are characterised by deviations lower than 15%, which can be considered within the limits of the typical uncertainty for daylight evaluations (Reinhart and Andersen 2006). DA 300 [50%] results exhibits higher deviations, that might be related to the different spatial distribution of illuminance values obtained by the different simulation methods. For these three metrics, the larger difference with the 4-component method is found for the 3-phase methods, and the smallest for DAYSIM. UDI-c shows very small deviations for all methods, probably because of its wide range of accepted illuminances (100-3000 lx). UDIx reports the biggest deviations of all metrics, but the absolute values for room M1 were all lower than 7.7%, and the significance of the deviation is therefore reduced. All techniques show a consistent behaviour across the 48 simulation runs and the corresponding reflectance variations. This is confirmed by the similarities between MBD and RMSD, which indicates a 'systematic' bias throughout the simulations series. This overall agreement is proved even further when looking at the results from the SA. The Morris plots displayed in Figure 7 help examining the similarities of three CBDM techniques among the five under analysis. The Five-Phase method is not represented here as it held exactly the same results as the Three-Phase method, while the Four-Component method was already pictured in Figure 4. The influence that each element within the room has on the final TAI values is extremely similar among the different techniques. The type of relation between input and output is also the same for all the elements, mainly characterised by monotonic behaviour. These similarities are present also when comparing these three techniques with the benchmark method, i.e. the Four-Component method.

Conclusions
This paper presented a thorough analysis of the influence that input reflectance values in CBDM have on annual daylight metrics. The simulations were run on four case study classrooms with different geometrical features and using five different CBDM techniques: the Four-Component method (used as benchmark); DAYSIM; the Two-Phase; Three-Phase; and Five-Phase methods.
The analysis focused on four main sources of uncertainty related to the assignment of reflectance values: (i) the use of different guidelines; (ii) the influence of different geometrical and physical characteristics; (iii) the expression of results with different annual CBDM metrics; and (iv) the employment of different simulation techniques.

Sensitivity to chosen guideline
The existence of various guidelines with differing suggestions about the standard reflectance values to apply at design stage is likely to lead to the assignment of different reflectances to the main surfaces in the model. The authors believe that the most successful daylight designs depend on the building fixed form, and little can be done a posteriori to amend bad designs through adjustments in materials and finishes. At concept stage, especially for design competitions, it would be fairer to apply common standard reflectances to all proposed designs, so that the performance comparison focusses on the shape itself. Furthermore, guidelines would need to be complete; too often the values and detail to assign to the external environment is not included. Results from the SA carried out in this paper showed that even the reflectance assigned to a simple plane representing the external ground can significantly affect the final annual results. The effect that an accurate exterior environment has on the simulation results is a rather complex matter, that was not investigated in detail here, but has been attempted in other studies (Sadeghi Nahrkhalaji 2017).

Sensitivity to physical and scenario uncertainties
Assuming that annual daylight metrics correctly describe the long-term luminous performance of a space, an indication of how this performance might change, in case that the actual reflectance characteristics differ from the intended ones, is given to designers. The results are particularly relevant for educational environments, where alterations of the surface optical properties can be frequent, due to redecoration or to the use of the walls for teaching purposes. From the SA conducted with the Method of Morris, it was found that walls reflectance can account to up to 25% of the variation in TAI in a classroom.

Sensitivity of the metric
The difference in sensitivity found among the analysed annual daylight metrics is deemed of particular importance for further studies on CBDM. TAI was found to correlate better than all the other metrics with changes in surface reflectance. Further work is needed to understand whether this is valid for other input parameters, e.g. the reference climate conditions, or if each metric is sensitive to a specific subset of simulation inputs. Nevertheless, it is believed that any parametric-or optimisation-based design should be preceded by an SA to assess the significance of the investigated parameters in relation to the metrics used to express the results. For example, the use of ASE or DA 300 [50%] to investigate the optimum surface finishes might lead to erroneous conclusions, even though some of the current guidelines require those two metrics only.

Sensitivity to chosen simulation technique
The simulation techniques analysed in this work were found largely in agreement, with MBD and RMSD for TAI, UDI-c and DA smaller than 15% for all methods and rooms. Given that the surface reflectance governs the behaviour of inter-reflected light, and all analysed methods are based on similar, if not identical, ray-tracing techniques, this agreement is perfectly justifiable. It would be interesting to compare the behaviour of simulation software based on different simulation engines, such as radiosity or photon-mapping, to understand whether the different treatment of inter-reflected light might affect annual results in a dissimilar manner.
However, DA 300 [50%] (calculated without the implementation of automated dynamic shading) led to larger deviations (RMSD up to 32%), which might be due to differences in spatial distribution of the illuminance levels obtained from different techniques. The largest discrepancies were found between the 4-component and the 3phase methods. Disagreements between techniques were accentuated when analysing the two rooms that receive less direct light, because of a deep plan geometry (M1) or because of North-oriented apertures (M5). The Four-Component method and DAYSIM generally resulted in lower illuminance records, compared with the 'phased' methods. Without a comparison with real data, it is however impossible to state whether some of the methods are under-predicting the luminous level in those spaces, or the other methods are over-predicting them. Correlations with monitoring data in the case study rooms are planned for future work, and will likely help determining the overall accuracy of the investigated CBDM techniques.