Post Occupancy Evaluation in Architectural Education and Practice

Post Occupancy Evaluation (POE) is a research method that examines how buildings function; when the functions include social life, social science methods must be employed. This paper advocates using POE social research both in architectural practice and in architectural education to promote evidence‐based design. Based on four decades of experience teaching POE to undergraduates at the University of California Berkeley, we show how POE can be conducted and taught: gather the research questions, set up teams to collect data using different data collection techniques, and analyze the results by comparing and contrasting the findings of each team. We discuss the importance of POE research to architectural practice, education, and accumulated institutional knowledge.


Introduction
Post Occupancy Evaluation (POE) is a research method that examines how buildings function and contributes both to improvements in the building being studied and to general knowledge about how to improve buildings. POEs evaluate social performance of a building, preferably once it is occupied for at least six months. In order to establish POE as part of professional practice we teach architecture undergraduates how to conduct POEs. We share that method and pedagogy here. Through programming and evaluation research, architects can respond creatively to both manifest and latent behavioral and cultural patterns and learn which of several design alternatives best serve end users.

Post Occupancy Evaluation Process
Programming often constitutes the initial step of the design process. During programming, the client presents a set of spatial needs and desired relationships to the architect. The general product of a programming process is a set of desired functions, spatial use diagrams, and square footage breakdowns. Many times, architectural programs of previous buildings serve as a precedent, and programming may not involve analyzing site-specific user behavior and cultural norms. In contrast, POE examines how an occupied building functions from its users' point of view. Where programming represents idealized relationships, the POE examines real outcomes. Optimally, POE serves programming by informing the next building of that type, or by informing renovation and expansion projects about what worked best and what failed, thus acting as a pre-design tool.
Post Occupancy Evaluation (POE) is a research method that examines how buildings function; when the functions include social life, social science methods must be employed. This paper advocates using POE social research both in architectural practice and in architectural education to promote evidence-based design. Based on four decades of experience teaching POE to undergraduates at the University of California Berkeley, we show how POE can be conducted and taught: gather the research questions, set up teams to collect data using different data collection techniques, and analyze the results by comparing and contrasting the findings of each team. We discuss the importance of POE research to architectural practice, education, and accumulated institutional knowledge.
POEs can use both qualitative and quantitative data. As Becker argues, both are valuable: "...rigor and discipline is achieved by the way in which the organization processes the information it collects, and not in declaring certain types of information off-limits." 1 The scientific process in conducting POE requires hypotheses and concepts to be operationalized (made measurable), so that researchers can determine if the spatial arrangements produced more or less of a desired outcome. We assess the actual performance and user experience of a building by asking research questions relevant to the building's intended function and design goals. The next step is to employ the correct data collection techniques in relation to the research questions. Analysis uses common techniques such as frequencies and simple two-way cross-tabulations. Importantly, similarities and differences in the results gathered from different data collection techniques help create a multi-faced view of a building's social performance.

Establishing the Questions
The first step in the process is to work with clients to establish the questions they want answered. Once we understand the issues at stake for the clients, we turn to the literature to place the project in the larger field of research and help focus the research question and establish appropriate methods. For example, in one study, the clients were the managers of a multi-tenant LEED-Platinum office building. They were curious about how visitors responded to the green features of the overall architecture. To frame our research design and questions, we reviewed the literature on how buildings communicate, and the literature concerning perceptions of sustainable buildings .2 Note that POEs do not only serve single clients but, if the work is published, they also contribute to larger fields, such as person-environment studies and the social use of space.
In the case of the new East Asian Library in Berkeley, our clients were the librarians who had eight basic questions about visitor usage, starting with "What do people use the building for?" and including "What are the user and staff perceptions of aesthetics, vertigo, and acoustics?" We discovered that the library serves a wider public than that for which it was originally intended. Because of its aesthetics, most visitors used the library as a study space, rather than for its collections ( Figure 1). 3 Once the research questions unique to each case study are established and connected to larger questions in the literature, we collect background information about the project. Depending on the questions and the site, the nature of the documents from which the background is sourced will differ, but might include architectural drawings, photographs of the site, funding documents, meeting notes with the architect, or memos about decisions that were made. This background is important for understanding the intentions of the design team. The operators of the building generally identify functional problems or user issues in the building that the original design did not anticipate. In fact, the recognition of these problems is often what spurs the clients' interest in obtaining a POE.

Data Collection
Once the primary and secondary research questions are identified, data collection is the next step. Collecting reliable and valid data is important to ensure the POE answers the research questions. Data collection techniques vary, depending on the kinds of questions. Observations, interviews, surveys, and archives including newspapers are the fundamental sources. Participant observation and ethnography are also relevant techniques for in-depth studies. Whenever data on human activity is collected for publishable research, one must acquire approval from Institutional Review Boards (IRBs) on the ethics of the study, to ensure that participants-whether witting or unwitting-are protected, including their right to privacy.
Observations are one of the best methods for collecting behavioral data. They can be time-consuming to collect and analyze because of the quantity of discrete data points, but they provide objective information about how people use the space and give students hands-on experience with managing data. All researchers should use the same protocol and data collection sheets: for example, plans of a room with furniture noted to make it easy to mark seat usage, or a chart for timing how long people spend in a room. In one study of a teen library seating area, each researcher used a floor plan of the space including furniture and design features to record the number of users, the postures they assumed, and the paths of each trip though the space; multiple plans served to show change over time. 4 In a study in a nursing home in Israel using behavioral maps, students noted characteristics of different groups and activities in the common area. The analysis led to greater understanding of how the common space served as a living room with multiple types of activities, while the original plan assumed that the primary activity would be twelve residents eating at the same time at a single large table. 5 Indirect methods of observation can provide rich data with less time commitment. Video observations take extensive time to review and analyze, yet can capture far more time than can a typical research team and can lead to accessible presentations, like Whyte's popular film about New York City plaza usage. 6 Electronic data, such as key card swipes into controlled-access buildings, can show how many people use a building and when. Using locational data tracked through smart phones to record where people are in buildings and how long they spend there is a burgeoning field; museums are beginning to make use of this type of data to uncover which exhibitions hold visitors' attention.
Subjective, self-reported data highlight how users respond experientially to design features. In interviews, respondents can express their opinions in depth and with nuance. Semantic ethnography offers a way to use interviews to understand the participants' worldviews through an understanding of how they use words about space, people, and activities. 7 Sometimes photographs accompany interviews to draw attention to details and prompt engaged responses. 8 In photovoice interviews, the interviewees themselves take the pictures and explain their photographs to the researcher, putting control of the images in the hands of those being asked about the space. 9 In a POE of new library spaces in Philadelphia, a researcher asked young people to take pictures of what they liked best and least about the renovated libraries; the participants took pictures of things like TAD 5 : 1 Post Occupancy Evaluation in Architectural Education and Practice snack machines, trash left by other users, and plaster ceilings-all items that the researchers had not identified as vital parts of the library experience, highlighting the importance of open-ended research questions. Surveys with close-ended questions offer a comparatively quick way to capture a large number of opinions; these can be particularly useful when designers are called upon to create spaces without time or resources to devote to extensive study. For example, in an unpublished study conducted for a museum on how an entry sequence affected visitors' experience of the art, visitors were asked to rate their experience. Using a one-question survey increased the response rate.

Data Synthesis and Triangulation
In classroom settings when we have over 100 students working on the same POE, we divide the class into four to six sections and each section, in turn divided into five teams of four, researches only one question. Each team uses one of five different data collection techniques: (1) direct and indirect observation, (2) interviews, (3) questionnaires, (4) photo elicitation, or (5) archival and precedent analysis (Table 1). This results in a matrix of questions and answers; each question is answered by five different data collection techniques, providing triangulation and a comprehensive analysis with minimal effort from each person. 10 If what people do and what they say concur or vary, that in itself provides another source of insight about the environment and about the value of using different data collection methods. The research project gives students hands-on experience, allows comparison and contrast for each data collection technique, and gives a multi-faceted picture regarding how users respond to various building features. Students also get to do public service for a local building. Conducting POEs by collating the research of multiple teams is a practice that is applicable to professional offices as well as academic settings.

Design Implications
Pedagogically POEs are a valuable way to teach research methods and to comprehend the ways in which environment and behavior theories play out in real-life situations. An equally important benefit of conducting POE is the implication on redesign. We ask students to develop design guidelines and a redesign proposal using the evidence they collected. Having thought deeply about the design implications of the user-centered data they uncovered, students are positioned to make informed suggestions. In one case, the David Brower Center POE, a research team of students uncovered why the building's courtyard was underused. The facility followed through on students' recommendation and added umbrellas and tables to attract employees to a formerly underused outdoor space. 11

Integrating POE into Architectural Practice
The benefits of using POE are varied. POEs reveal unintended consequences and can be used as a means of continuous social learning and refinement for facility managers. For example, POEs of the 1995 San Francisco Public Library have been useful to management over twenty years to prioritize renovations (Figure 2). 12 Spatial programs that list square footages are insufficient to explain complex relationships and building conditions. Architectural programs seldom refer to workflows, organizational values, and cultural and behavioral needs unique to each space. Each POE study contributes to further developing the environment and behavior theories and concepts unique to specific building types. For example, a recent analysis in healthcare facilities shows that fundamental theoretical concepts, such as proxemics and territoriality, continue to be useful years after their first conceptualization. 13 Studying visitors' waiting behavior in an internal medicine ward showed that visitors and caregivers prefer to keep an auditory and visual connection with the patients. Lack of proper waiting spaces, which would respond to this proxemic and territorial need, explains why the ward corridor gets overly crowded. 14 The benefits of evidence-based design go beyond increasing users' satisfaction. Companies and institutions benefit from evidence-driven, tested design ideas by saving money or increasing productivity. For instance, in one green office building remodeling, changes led to a 39 percent reduction in sick days. 15 Another study suggests that savings from productivity and reduced sick days can equal $37 to $55 per square foot. 16 A third study calculated that it can take as little as three years to gain back a $29 million construction investment in evidence-based design of healthcare facilities, thanks to reduced operating costs. 17 Despite these advantages, architectural services rarely include funds for conducting research about the effectiveness of design decisions. In an example from one of the authors' POE consulting practices (Core Space Planning), a senior architect from a global architecture firm stated they design the same courtyards and gallery spaces everywhere because they apply the same approach to these spaces assuming they will work in predictable ways to direct people to move in certain directions or to increase informal communication. Yet they also acknowledged nobody actually knows if what they claim is, in fact, what happens in real life. These rules of thumb rely on general assumptions and professional expertise rather than an examination of end users' needs, behavior, or a review of relevant research.
Important barriers to POE research include the possibility that POEs may unearth negative results about the architectural design, resulting in a potential increase in insurance fees, liability concerns, and fear of losing a client. 18 The American Institute of Architects (AIA) could play an active role in encouraging the use of POE by improving the terms of specialized service and consultancy contracts to respond to such concerns.

Integrating POE into Architectural Education
Institutionalizing POE into routine professional practice requires early socialization to the practice as part of architectural education. Considering the multiple benefits of POE, we advocate that conducting POEs as part of the architectural design process should be actively encouraged by Association of Collegiate Schools of Architecture (ACSA) and National Architectural Accrediting Board (NAAB). Current NAAB 2020 criteria for accreditation make no provision for educating architects to follow up after a design is completed and a project built. Rather, NAAB criteria emphasize teaching professional practice, broadly defined, with an emphasis on supporting diversity and inclusion, which in our view should include longer-term evaluation of building performance.
POE and user-centered design courses should be securely integrated into architectural curricula. Ideally, each faculty would have at least one person who focuses on teaching undergraduate and graduate students about user-centered design and POE methods while focusing their research on these subjects. This would require individual colleges of architecture and architectural faculty to emphasize the significance of POE methods education, education on social and cultural theories, and user-centered design methods. More research funds should be allocated to conduct not only basic historical and theoretical research but also applied research, since POE and user-centered design approaches have direct impact on design and architectural studios. When architecture students and professionals in continuing education learn how POEs can be conducted, they experience for themselves how the benefits of POE outweigh the barriers.

Conclusions
Overall, assessing a building's social performance in architectural education and practice receives limited attention. Yet our examples indicate that each POE contributes to improving shared knowledge about how to design buildings that meet design intentions and respond to users' both manifest and latent needs. POE should be an integral part of the design process and training, so that architectural professionals can avoid repeating mistakes.
Making POE a routine part of practice will help reduce the amount of failing building stock. Having to remodel soon after being built (as in the case of the SF Public Library) wastes valuable resources and frustrates dissatisfied occupants. Incorporating POE methods into architectural design curricula would acculturate future design professionals to make use of published POEs and to be comfortable conducting their own user-focused POEs.