On the sensitivity to different aspects of occupant behaviour for selecting the appropriate modelling complexity in building performance predictions

The reliability of building performance simulation (BPS) predictions is impaired by a number of uncertainties, among which occupant behaviour (OB) plays a major role. Methods to relevantly model OB are essential to achieve energy efficient and comfortable buildings. This study contributes to the ongoing discussion concerning how to model OB in BPS. Specifically, a sensitivity analysis to various aspects of OB is used to assess the impact of using different levels of modelling complexity in the conceptual design phase. A method based on the statistical Mann–Whitney test is proposed to identify those aspects of OB that are influential for a performance indicator, and which might require a higher modelling complexity. Sixteen variants of an individual office constitute the case study. The results show how generalizations concerning robustness of a building typology to OB are not possible. Increasing modelling complexity does not necessarily lead to more accurate, or even to different results.


Introduction
Building performance simulation (BPS) tools are used during building design and operation to help achieve energy efficient and comfortable buildings. However, the reliability of these tools is hindered by a number of uncertainties, such as weather and occupant behaviour (OB). As such, the role of uncertainties in building performance predictions is still under active investigation (Hopfe and Hensen 2011;Rezaee et al. 2015).When it comes to uncertainty related to OB, most research efforts are directed towards: quantifying its impact (e.g. Branco et al. 2004;Guerra Santin, Itard, and Visscher 2009;Lin and Hong 2013), data-mining to derive OB patterns (e.g. Duarte, Van Den Wymelenberg, and Rieger 2013;Ren, Yan, and Hong 2015;Zhao et al. 2014), and developing models to be integrated into BPS tools (e.g. Haldi and Robinson 2010;Reinhart 2004).
Accurate building performance predictions are an essential prerequisite to enable the realization of concepts such as performance contracting, net-zero-energy buildings and demand side management. Thus, increasing the reliability of predictions is crucial. The complexity of existing approaches to OB modelling can be classified according to their underlying principle as: schedules, deterministic models, non-probabilistic models (or datadriven models), stochastic or probabilistic models and agent-based stochastic models. Schedules are at the lowest *Corresponding author. Email: i.i.gaetani@tue.nl end of modelling complexity, while agent-based stochastic models are at the highest. Little work has been done to provide guidelines for users of BPS tools about the most appropriate OB modelling approach for different cases.
Currently, the most common approach to represent occupants and their behaviour is to use fixed schedules, or hourly fractions (0-1) that multiply the maximum internal gains due to people's presence, lighting loads, equipment loads, etc.
This approach is unable to reflect the unpredictability and diversity of OB, and it can lead to buildings that are optimized for a standardized scenario, rather than for actual operation. In turn, it could lead to over-or underestimations of the energy and comfort performance.
Literature shows that there is a trade-off between approximation error and uncertainty due to estimation when changing model complexity (Zeigler, Kim, and Praehofer 2000). In other words, whereas complex models may offer a better approximation of reality, the trustworthiness of their predictions could be undermined by the higher number of parameters that need to be input, and which might not always be known or certain.
Moreover, some building typologies are more affected by OB than others; for example, occupants will have a much more important effect on the energy and comfort performance of a cellular office with individual climate control rather than in an open plan office with central climate control. Hence, the choice of the most suitable modelling complexity should be dependent on the considered case and purpose of the simulation (Gaetani, Hoes, and Hensen 2016;Mahdavi and Tahmasebi 2016).
The literature also shows general agreement with the conclusion that thermally well-insulated buildings are more sensitive to OB (Hoes et al. 2009), while only a small number of authors reach the opposite conclusion (Buso et al. 2015). While this deduction is intuitive for the influence of OB on internal gains, which play a bigger role in the indoor air heat balance of well-insulated buildings, the impact of other actions such as regulating the thermostat or operating windows and blinds is not obvious.
Most available studies tend to focus on one aspect of OB only. Noteworthy studies that try to combine multiple aspects to derive a simulation framework are rare (Chapman, Siebers, and Robinson 2014;Rysanek and Choudhary 2015;Tanimoto, Hagishima, and Sagara 2008). However, those that do exist tend to adopt one complexity level for all aspects of OB, without considering their relative importance. In this respect, a possible improvement could be made by determining the modelling complexity of various aspects depending on their relevance for the results.
The goal of this paper is to contribute to a better understanding of the appropriate use of OB models. The longer-term aim is to develop a fit-for-purpose occupant behaviour modelling (FFP-OBm) strategy that can aid simulation users to select the right complexity level in relation to the considered case and the objective of the simulation. One of the hypotheses underlying the FFP-OBm approach is that different aspects of OB have a dissimilar influence on the performance indicators (PIs) of different buildings (see Figure 1): non-influential aspects should not be modelled with the same modelling complexity as influential ones. There is hence a need for a method that can separate influential and non-influential aspects.
While this concept was introduced earlier (Gaetani, Hoes, and Hensen 2016), the value of the present study is that it offers a practical method to separate influential and non-influential aspects, and it demonstrates the validity of the hypothesis.
Section 2 outlines the method used to distinguish between influential and non-influential aspects of OB. Section 3 describes the case study. Section 4 presents the results of the application of diversity patterns on the selected PIs. Section 5 demonstrates how results are more or less sensitive to different aspects of OB depending on building, climate, PI and building use scenario. In Section 6, two building variants that differ in sensitivity to light, window and blind use are selected to investigate the effect of increasing modelling complexity for both influential and non-influential aspects of OB. The results are discussed in Section 7.

Methodology
The current study proposes a method to quantify the sensitivity of results to different aspects of OB and to distinguish influential from non-influential aspects. The overall hypothesis is that a higher modelling complexity might be needed for the influential aspects of OB. As the influential aspects supposedly depend on the building, climate, purpose of simulation (PI) and use scenario, this hypothesis highlights how the appropriate modelling complexity might be derived from the object -and objective -of the simulation.
We propose to use the statistical Mann-Whitney U test to determine whether an aspect of OB is relevant for the results. The Mann-Whitney U test is a nonparametric test which is used to assess whether two independent groups are significantly different from each other. Its strength in comparison to sensitivity analysis methods that are traditionally used in BPS (Tian 2013), is that this test is able to process correlated and non-correlated inputs, whose variation is not a uniform or normal distribution. In practice, the test helps to quantify the influence on the results of an aspect of OB. The proposed method is tested using a case study.
First, the case study and the purpose of the simulation in terms of PIs are defined (Section 3). The case study consists of 16 building variants for which the uncertain aspects of OB are modelled by means of diversity patterns.
Secondly, the performance of the building variants is assessed using the diversity patterns (Section 4).
Then, if the range in the PI shows a visible effect of OB due to the patterns, a sensitivity analysis takes place to identify the influential aspects of OB (Section 5). This step allows relevance to be ascribed only to those aspects that truly affect the results for a specific case, and is achieved by means of the statistical Mann-Whitney U test. Here, the two groups are characterized by all combinations with the same pattern for an uncertain aspect of OB. In particular, all results characterized by a certain type of OB, referred to as pattern A, are compared with those with the same type of behaviour referred to as pattern B (see Section 3.2). There is a statistically significant influence of the behaviour on the results if the statistical p value < .05. The sensitivity analysis allows the identification of those aspects of OB that are responsible for the spread in the results, so that more attention can be directed to such aspects.
Finally, a higher modelling complexity is applied to the influential and non-influential aspects for two building variants, to test the effect of changing modelling complexity for aspects of OB which showed a different sensitivity (Section 6).
The applicability of the proposed method, based on the statistical Mann-Whitney U test, is discussed in Section 7.

Case study description
Different buildings, climates, purposes of simulation and use scenarios are defined to verify the hypothesis that various aspects of OB are influential in different cases, and hence the appropriate OB modelling complexity depends on the object -and objective -of the simulation. As for the phases in the building lifecycle, this study investigates the conceptual design phase only, when no data about the actual building performance is available. The sole purpose of formulating the building variants is to create a spectrum of different cases to be investigated by means of the methodology presented here. The buildings' characteristics, as well as the choice of climates, will necessarily have an impact on the sensitivity of the results to certain aspects of OB. For example, the relatively small openable window fraction may result in a lower-than-expected sensitivity to window use. However, this paper aims at proposing a methodology rather than at drawing conclusions about the comparative importance of various aspects of OB.

Characteristics of the investigated building variants
A testbed made of 16 different variants of a cubicle office is defined using EnergyPlus v8.3. The office dimensions are 5 × 5 × 3 m 3 , and the south-facing wall faces outside, while all other walls, ceiling and floor are assumed to be adiabatic, as if the office was surrounded by other cubicles in thermal equilibrium with it. An operable window equipped with an external shading device is placed on the external wall. Two climates are considered: Amsterdam, the Netherlands, and Rome, Italy. Two variations of window-to-wall ratio (WWR) are defined, namely 40% and 80%. The fraction of window that can actually open is set as 10% of the total window area, which corresponds to 0.6 m 2 for WWR = 40% and 1.2 m 2 for WWR = 80%. The other variations concern the power density of lights and equipment, and the construction of wall and window, for a total of 16 building variants (see Table 1). Heating and cooling are provided by means of an ideal system, whose size has been capped based on the results of preliminary simulation runs.

Definition of diversity patterns
A number of aspects related to OB were assumed to be uncertain, namely: occupants' presence, Heating, Ventilation and Air-Conditioning (HVAC) use, equipment and  light use, heating and cooling setpoint, blind and window use. For these aspects, diversity patterns were defined as in Table 2. Most of the variations are as in Hong and Lin (2012). Clearly, the assumptions made in formulating the diversity patterns will have a great impact on the results of the sensitivity analysis, as further clarified in Section 7. Where possible, the user of this methodology should corroborate his/her assumptions with data, so that the diversity patterns are representative for possible variations of OB. In this case, the scope of implementing diversity patterns is to investigate the sensitivity of different building variants to standardized variations in OB. The diversity patterns are not implemented one-at-atime in the simulation model, but rather in all possible combinations of patterns A and patterns B. Doing so leads to a total number of 2 8 = 256 scenarios -64 scenarios where T sp,heating > T sp,cooling = 192 scenarios. It should be noted that occupants' presence is not only considered in terms of internal gains due to occupancy, but also acts as a trigger for most other adaptive actions (see Table 2).

Definition of PIs
Heating and cooling energy and weighted overheating hours (WOH) are the selected PIs.
WOH ([h]) are calculated with the simplified formula where h is the number of occupied hours, T op is the operative temperature [°C] and T max is the maximum allowed temperature. T max is here assumed to be 28°C, corresponding to Class D temperature summer limits in actively cooled buildings (Boerstra, van Hoof, and van Weele 2015). For comfort-related PIs, a maximum acceptable value is typically defined. A threshold of 500 h is taken for illustrative purposes. The order of magnitude of this figure is based on the daily limits for weighted exceedance. According to CIBSE (2013), the number of WOH shall be less than or equal to 6 in any one day in the cooling season, or equal to 109 (weekdays May 1st-September 30th) × 6 = 654 h. It is supposed that when no-adaptive T max are considered, as in the case of active cooling, the limit should be more stringent.

Case study results: impact of diversity patterns on
PIs The impact of OB diversity patterns is evaluated for the aforementioned PIs and building variants. This intermediate step is taken to establish whether OB as a whole has an effect on the PIs. In cases where the effect of OB is negligible, it is assumed that there is no point in modelling it with further detail. Instead, in cases where there is a visible effect of OB, sensitivity analysis is undertaken. Figure  2 shows the range of cooling energy use due to OB. A significant variation in all building variants can be seen. Ultimately, it will depend on the purpose of the simulation whether a given variation is considered acceptable or not. Figure 3 represents the impact of OB on heating energy use. For all building variants located in Rome (9-16), the heating energy demand is lower than 10 kWh/m 2 y regardless of OB. Depending on the purpose of the simulation, the relative variation may be considered important or not. In this example we consider all building variants to be sensitive to OB.
An analysis of the impact of OB on WOH (Figure 4) reveals that all buildings located in Rome, and two buildings located in Amsterdam (building 7 and 8, characterized by WWR = 80% and low thermal insulation) exceed the threshold of 500 h.
In summary, we assume that the effect of OB on results needs to be further examined for: cooling energy in all buildings, heating energy in all buildings, and WOH for buildings 7, 8 and 9-16. For these cases, a sensitivity analysis with the Mann-Whitney U test is applied to determine which aspects of OB are statistically significant for the results.  Figure 2. Variation in cooling energy use due to diversity patterns for uncertain aspects of OB in building variants 1-16 (see Table 1).

Sensitivity analysis of cooling energy use to OB
As expected, the results of the Mann-Whitney U test show that cooling energy use depends on the HVAC use for all building variants, and does not depend on the heating setpoint temperature in any of the building variants. The results for all types of behaviour are shown in Figure 5. Only buildings 4, 8 and 16 (variants with low thermal insulation and low power density for both Amsterdam and Rome WWR = 80%) are not influenced by occupants' presence. This result can be explained as the cooling demand of such variants is highly dependent on solar heat gains and thermal exchange through the building envelope. Internal heat gains -which depend on presence -are relatively less important.
In contrast, equipment use is only relevant for the results in building variants 1 and 9, characterized by low WWR, high thermal insulation and high PD.
The results are relatively more sensitive to light use than equipment use due to the higher power density of lights, with only four variants (4, 8, 14 and 16) not being affected by this aspect of OB.
According to expectations, the cooling temperature setpoint is a decisive factor when it comes to cooling energy use in almost all buildings. The only building variant which is not sensitive is variant 15, located in Rome, with high WWR, low insulation and high power density.
As expected, higher WWR are more affected by window opening due to the larger opening area. The only building variant with WWR 40% which shows sensitivity to window use is variant 11, characterized by low thermal isolation and high PD in Rome. As for the sensitivity to blind use, the only building variants which are not affected are 1 and 9, both characterized by low WWR, high thermal insulation and high power density. In these variants the windows are characterized by a very Figure 5. Sensitivity of cooling energy use to various OB aspects for all building variants; building variants with 1 − p > .95 are considered sensitive (black-filled bars). low solar heat gain coefficient (SHGC), which weakens the effect of blinds. The use of blinds is shown to have a greater effect on building variants characterized by higher SHGC.

Sensitivity analysis of heating energy to OB
The sensitivity analysis of heating energy use to OB for building variants 1-16 shows that all variants are sensitive to heating setpoint and HVAC use. Building variants 1 and 8 are sensitive to occupant's presence. Building variant 1 (low WWR, high thermal insulation, high PD) is sensitive to equipment use. Variants 1-3, 5, 7 and 9-15 are sensitive to light use, confirming the hypothesis that the energy consumption of variants with low thermal insulation and low PD is less affected by internal gains. Building variants 13-15, characterized by high WWR in Rome, are affected by cooling setpoint. Window use is non-influential for heating energy in all buildings but variant 13, located in Rome with high WWR, well-insulated envelope and high PD. Building variants 4, 7 and 8 (all characterized by high SHGC) are sensitive to blind use ( Figure 6).

Sensitivity analysis of WOH to OB
Only building variants 7, 8 and 9-16 were within the scope of the sensitivity analysis, as for all other variants it was assumed that the influence of OB did not cause reaching the assumed threshold. For WOH, no building variant is sensitive to heating setpoint temperature or equipment use. Instead, all building variants are sensitive to HVAC use. Building 9, 10, 12, 14 and 16 are sensitive to occupants' presence. Building variants 11 and 13 are sensitive to light use. The cooling temperature setpoint significantly affects the WOH only for building variants characterized by high thermal insulation in Rome, WWR = 80%, while blind use is relevant for all variants with low thermal insulation. Building variants 7, 11, 12, 13, 15 and 16 are sensitive to window use when it comes to overheating hours (Figure 7).

Case study results: increasing model complexity to
stochastic models Only the cooling energy of two building variants and three adaptive behaviours (light, blind and window operation) have been selected to test the effect on the results of applying a higher modelling complexity for different aspects of OB. Building variants 1 and 16 are chosen as they show a different sensitivity of cooling energy use to light use, window use and blind use (see Figure 5). The hypothesis is that the non-influential aspects of OB can be modelled with the lowest complexity, while adding complexity to the influential aspects will give further insights into the PI, if compared with the simplistic diversity patterns.
Instead of applying a higher modelling complexity only to the aspects of OB that are influential for the given PI/building variant combination, the effect of performing such an operation on the distribution of the cooling energy is investigated for both considered building variants. There are two main reasons for applying higher complexity models to both variants rather than only to the sensitive one: (i) to test whether the Mann-Whitney U test leads to reliable results, that is to verify that changing modelling complexity of a non-influential aspect does not have an impact on the performance indicator; (ii) to inspect existing interrelations among aspects of OB (i.e. to understand whether a non-influential aspect of OB can be ignored when adding complexity).
A higher modelling complexity is firstly implemented for each aspect of OB one-at-the-time, and then simultaneously to investigate possible interrelations between the various aspects. It has to be noted that the initial number of scenarios due to the combinations of diversity patterns for all uncertain aspects of OB changes when performing this operation. In fact, if one aspect is modelled stochastically, the number of scenarios reduces from 192 to 96. If two aspects are modelled by means of a stochastic model, the resulting number of scenarios is 48, while if all three considered aspects are modelled in this way, there will be only 24 scenarios.
The implemented OB models are well-established stochastic models taken from literature: Reinhart's Lightswitch-2002model (Reinhart 2004, Haldi and Robinson's window operation model (Haldi and Robinson 2009), and Haldi and Robinson's blind operation model (Haldi and Robinson 2010). These models were developed for cellular offices in climates different than those considered in the case study, and there is no evidence that their combined use leads to representative results. However, they have been widely used in conjunction and for a number of buildings and climates (Gilani et al. 2016;Gunay, O'Brien, and Beausoleil-Morrison 2016) and represent the current state-of-the-art in OB modelling research. The OB models are here implemented in the building model by means of the EMS feature of EnergyPlus, as in Gunay, O'Brien, and Beausoleil-Morrison (2015). The models have been run an appropriate number of times to take their stochasticity into account. A detailed description of the method used to determine the minimum number of runs is out of the scope of the research presented here.

Implementation of stochastic models to building variant 1
The cooling energy of building variant 1 is shown to be sensitive to light use, while it is not sensitive to window use or blind use (see Figure 5). Modelling light use by means of Reinhart's Lightswitch-2002 model causes the distribution in the results to change radically. As expected, adding modelling complexity to the other considered aspects of OB leads to negligible differences in the results. Combinations of aspects have been considered to investigate the interactions among behaviour; while some effect is noticeable, for the case under investigation, modelling the lights' operation alone causes the greatest variation ( Figure 8).

Implementation of stochastic models to building variant 16
The Mann-Whitney U test identified the cooling energy of building variant 16 as dependent on blind and window operation, while the results are not sensitive to light operation (see Figure 5). Figure 9 shows how applying a stochastic model to windows and blinds leads to a great variation of the performance indicator. Adding further modelling complexity to the light use has a marginal influence on the results. Figure 9 also shows how adding modelling complexity to different influential aspects of OB has a diverse impact on the variation of the performance indicator. In this case, the blind operation model has a much stronger impact than the window operation model. This result could be due to the fact that the patterns already gave a similar representation of window operation to the one obtained by means of the stochastic model. Another plausible explanation is that, in spite of both aspects being classified as "influential" in the Mann-Whitney U test, blind operation has a stronger influence on the cooling energy.

Discussion
The results in Section 4 confirm that different buildings and PIs show a dissimilar sensitivity to OB. The Mann-Whitney U test (Section 5) proved to be a suitable method to determine the aspects of OB that are influential for the results. In the considered case, all PIs were sensitive to HVAC use. The sensitivity to all other aspects of OB changed according to building variant and performance indicator. Generally speaking, blind use appeared to be more relevant in buildings with high SHGC for all PIs. Light and equipment use had a greater effect for buildings with a use scenario characterized by higher power density. Building variants with bigger window areas are more sensitive to window use. Although macro-trends are visible, it would have been impossible to establish a priori which aspects of OB are influential for the results. The current methodology is proposed to separate influential and non-influential aspects of OB for any case study at hand. An analysis of different climates and building variants is expected to lead to a diverse sensitivity of the PIs to various OB aspects. In Section 6 the effect of applying a higher modelling complexity on the range of cooling energy use was investigated both for influential and non-influential aspects. This paper makes a number of simplifications that ought to be pointed out.
Firstly, the results of the sensitivity analysis strictly depend on the definition of diversity patterns, which should represent a plausible spectrum of the uncertainty of the considered aspect of OB. For example, all results were sensitive to HVAC use as there was a fundamental difference between Pattern A, in which the system is switched off when the building is unoccupied, and Pattern B, where the system is always on and a setback temperature is used. Moreover, the modelled triggering conditions for window opening in this case is Open if occupied, T in > T sp,cooling and T in−out > 2°C (Table 2). This assumption may be valid for office buildings, where window opening behaviour is mainly influenced by thermal discomfort (Haldi and Robinson 2009). However, it precludes de facto the sensitivity of energy-related PIs to window operation in the heating season ( Figure 6). The user of this method should be aware of the realistic spectrum of uncertainty in his/her case, to ensure that the modelling assumptions do not inhibit the significance of the results.
Secondly, as the diversity patterns are implemented in the form of schedules or deterministic built-in software functions, it was stated that implementing stochastic models is equivalent to increasing modelling complexity. While this is certainly true, in reality, modelling complexity is continuous rather than discrete, as a category (e.g. stochastic models) can be characterized by different complexities according to the model's size and resolution (Gaetani, Hoes, and Hensen 2016).
Thirdly, while the scientific community agrees that fixed schedules may not be representative of actual behaviours, it has not reached agreement concerning higher modelling complexity. Hence, while it is reasonable to state that higher complexity models offer a better approximation of reality, it is not yet proven that their predictions indeed lead to more realistic results (e.g. Mahdavi and Tahmasebi 2015). The reliability of such models in this context is subject to the evaluation and validation process undertaken by the single models.
Finally, selecting the fit-for-purpose model is not only about modelling complexity. In fact, different models have been developed for different building typologies, climates, PIs, etc. If there is no evident match between the investigated case and the available models, all suitable models of a given complexity should be implemented.
As pointed out in the introduction, this study represents an important step towards achieving a FFP-OBm strategy. The strategy aims to support the simulation user in the selection of the appropriate modelling approach for OB. Further research is being devoted to quantifying the trade-off between estimation uncertainty and approximation error. Moreover, while this study is performed on a single zone, in the future the whole building will be taken into account. Other phases in the building lifecycle such as detailed design or operation also ought to be considered to verify that the FFP-OBm indeed leads to efficient decision making and improved modelling predictive ability.

Conclusion
A practical approach to identify the most influential aspects of OB was introduced and tested in the conceptual design phase for eight building variants of a cellular office in Amsterdam and Rome using EnergyPlus v8.3. The Mann-Whitney U test proved to be a suitable statistical method to perform a sensitivity analysis in this context. The results highlighted how different buildings and PIs are influenced by the various aspects of OB in a dissimilar way. A deeper analysis of two building variants confirmed the findings of the Mann-Whitney U test. In fact, increasing the modelling complexity of aspects of OB that appear to be non-influential for the results has a marginal influence on the distribution of the performance indicator. It can be concluded that, for the investigated case, adding modelling complexity to those aspects of OB that the Mann-Whitney U test identified as non-influential might be an unnecessary time expenditure, depending on the purpose of the simulation. In cases where higher accuracy of the results is required, it might be necessary to model all aspects of OB that could be interrelated. Indeed, a small effect of such interrelation is visible, but is negligible if compared to the effects obtained by changing the modelling complexity of the influential aspects.