Visual content analysis of visitors’ engagement with an instagrammable exhibition

ABSTRACT This study aims to show how a museum exhibition designed to encourage visitors to take pictures can affect visitors’ behavior. We analyzed visitors’ engagement with the Yumi's Cell Special Exhibition (hereafter Yumi) held in South Korea, employing computer vision for the analysis of visitors’ Instagram pictures. Our research questions are: What types of pictures do visitors post on Instagram during or after their visit? And can Yumi’s instagrammable features make visitors interact more with the exhibition? We also formulated two corresponding hypotheses: Visitors are primarily interested in taking selfies in the instagrammable environment; and visitors struck more active poses when taking pictures in an instagrammable exhibition than in a traditional art exhibition. Through the image analysis of Instagram posts of the exhibition, we found many pictures of people, but the proportion of selfies was relatively limited. This suggests that visitors were more interested in interacting with the exhibition rather than taking selfies. This has also been confirmed by the pose analysis, which showed that the participatory feature of the exhibition encouraged visitors to take photos in active poses, interacting, mimicking, and performing. The framework presented and the findings offer insights about how to design exhibitions to increase visitors’ participation.

and curation of art (Sokolowsky 2017). Many artists also have recognized the value of creating Instagram-worthy art: It can contribute to an artist's popularity and success in the art industry (Moss 2018), dramatically increase the attendance of visitors, and create new meanings for art consumption (Raphelson 2017;Kang, Chen, and Kang 2019). Through the increased use of social media, J. H. Falk's proposition 'the museum is not an island onto itself, but a dynamic interaction between the public and society' (Falk 2016, 241) seems to become more relevant (Arias 2018).
While many museums are embracing the age of the influencer (i.e., a person with the ability to influence potential buyers of a product or service by promoting or recommending the items on social media) as part of their marketing efforts, there are also some spaces and events specifically created to attract the Instagram generation (Straughan 2019). A new form of art exhibition emerged in the past years, the so-called 'Instagrammable exhibition.' This new form of exhibition aims at fulfilling younger generations' desire for experiences that are worth sharing on Instagram (Stomberg 2018;Charr 2020). The explosion of 'made-for-Instagram' exhibitions has created a new art market (Raphelson 2017;Kang, Chen, and Kang 2019). Research on Instagrammable exhibitions suggests a positive correlation between 'Instagrammable' art and the affluence of museum visitors, raising questions as to the type of art the museums of tomorrow will exhibit (Phan and Melissa 2018). Some exhibitions and museums are now being designed specifically to invite photo sharing, offering interactive exhibits, colorful settings, and flattering lighting so that visitors can get that perfect snap. While Instagrammable exhibitions have drawn the attention of many visitors and the art industry, so far scholars have shown little interest in their values and visitors' perception of them (Kwandras 2019). This article focuses on examining visitors' engagement with South Korean Yumi's Cell Special Exhibition, employing computer vision for the analysis of visitors' Instagram pictures. Since art institutions have started to respond to the public's desire to document and share art experiences online, this study provides museum professionals with an insight into how visitors engage with Instagrammable exhibitions and talk about them online.

Theoretical foundation
2.1. The rise of instagrammable exhibitions 'Instagrammable' describes anything that is 'visually appealing in a way that is suitable for being photographed for posting on social media' (Anastasi 2020). According to Wikipedia, an 'Instagram museum' is a type of art gallery or installation designed to provide a setting for visitors to pose in photographs to be posted on social media sites such as Instagram (Wikipedia). Although the term 'instagrammable exhibition' is much covered in the media, it has not been academically defined and analyzed (Quilty 2018). The concept of Instagrammable exhibition originated from that of 'pop-up museum (Stockham 2019),' a temporary art venue situated outside its usual institutional location to enhance public engagement (Giordano 2013, 462). With the rapid spread of Instagram's popularity, the word 'Instagrammable exhibition' has gradually taken the place of pop-up museums, and in the last couple of years there has been an unprecedented surge in immersive, highly photogenic Instagrammable exhibitions across the globe.
Instagrammable exhibitions can be classified into two categories: 'Institution-based' exhibition, where the temporary or special exhibition has a strong anchorage with promoting and marketing effect in museums, and 'enterprise-based' exhibition created by commercial organizations (Giordano 2013). The former is an art exhibition held in a traditional museum. Recently, major art exhibitions are getting enormous publicity and promotion from Instagram posts, whether or not they intend to exploit social media communication. For instance, Yayoi Kusama: Infinity Mirrors (2017) at the Hirshhorn Museum and Sculpture Garden, combining highly photographable visuals, such as bright colors decorated with Polka dot motifs, immersive spaces, and perfect selfie backdrops, was seemingly tailor-made for the Instagram age (Sokolowsky 2017;Yulianti and Yuni Deviyanti 2019;Albright 2019) and the #InfiniteKusama hashtag reached 90 million Twitter and Instagram accounts, with 330 million impressions (Smithsonian 2017 (Nagata 2017;Kwandras 2019). They have colorful backdrops, interactive or oversize props, and optical illusions; and key factors when curating such exhibitions are the novelty and innovation of the presentation (Lauson 2019;Charr 2020). In the case of the Museum of Ice Cream, the #museumoficecrean hashtag has been used over 181,000 times on Instagram (BBC 2018). Since the Museum of Ice Cream has gained huge popularity, similar exhibitions spread in the global exhibition industry. In the Korean art market, there have been many successful instagrammable exhibitions, such as the Museum of Fruit, the Colorful Museum, Sugar Planet, and the Beauty Inside Exhibition. Additionally, some exhibitions experimentally combined Instagrammable features and story-driven curatorial strategies: Alice: into the rabbit hall (2017), My Name is Red-Haired Anne (2019), Yumi's Cell Special Exhibition (2020-2021), and Moomin Original (2020-2021). These exhibitions are attracting more diverse kinds of visitors than traditional art museums, bringing into museums people who would be unlikely to visit them otherwise.

Previous research on instagram images in relation to museum experiences
Instagram as a platform for engagement provides an online archive to document and to share what visitors encounter (Weilenmann, Hillman, and Jungselius 2013;Budge 2018). They use Instagram to combine their museum experiences with their own narratives with various purposes: self-image construction, remembrance, memory, aesthetic inspiration, interpretation, creation, self-curation, play, place-making, and social presence (Weilen-  Rhee et al. 2021). In short, Instagram can be a vehicle of popular expression which also enables us to see a museum through the visitors' eyes (Rose 2012).
Previous studies exploring the content of museum visitors' Instagram posts (Suess 2014;Budge 2017;Budge and Burness 2018;Zingone 2019;Rhee et al. 2021) have highlighted how the exhibition material or objects have been of central focus to the posts shared on Instagram. Qualitative research conducted on Gerhard Richter's exhibition at the Queensland Gallery of Modern Art showed that the use of Instagram at the gallery engaged visitors in a manner that transcended the physical space and extended their aesthetic experience (Suess 2014;Suess 2018). Another study of Instagram posts of the exhibition Recollect: Shoes, held at the Museum of Applied Arts and Sciences in Sydney, revealed that Instagram images primarily focused on the objects on display and selfies were a minority of posts (Budge 2017). And for the Museum of Contemporary Art in Sydney, Instagram images were categorized into object (47.0%), people and objects (28.0%), people (17.0%), and others (8.0%) (Budge and Burness 2018). In an analysis of the official profiles of the Louvre Museum and Metropolitan Museums of Art, 30 Instagram posts for the Louvre Museum and 34 posts for the Metropolitan Museums of Art have been categorized into spaces, objects, and people as macro categories (Zingone 2019). All findings of existing research show that visitors are primarily interested in portraying objects and spaces rather than including themselves or their partners in pictures taken at museums.
Although they are not among the most common type of pictures taken at museums, a strand of research focused on the phenomenon of selfies and pictures of people in museums. In the study conducted at the Art Institute of Chicago, researchers found three variations of behaviors during selfie-taking in terms of the co-performativity of a visitor's body with the displayed objects (Hunter 2018). In another study, the data analysis and observation revealed multiple types of museum selfies: art interactions, blending into art, mirror selfies, silly/clever selfies, contemplative selfies, and iconic selfies (Kozinets, Gretzel, and Dinhopl 2017). Observing photos of visitors to the fine arts museum posted on Instagram, researchers found several creative photos, like imitating the style of characters in paintings, matching the faces of models in paintings, pretending to lift objects in paintings, making objects as body parts, objects being heads, increasing the number of faces in paintings, and completing body parts of objects (Siswowihardjo 2020). These studies provide an understanding not only of how visitors take pictures in the museum but also of how they interact with the exhibits. However, none of the previous research focused specifically on Instagrammable exhibitions and the kind of pictures that visitors take and share online. Here we aim at filling this gap by presenting a case study that analyzes a South Korean Instagrammable exhibition.

Case study: Yumi's cell special exhibition
Yumi's Cells Special Exhibition (July 15, 2020 -March 14, 2021), created by Media N Art, was held at the Ground Seesaw in Seoul, South Korea. It remediated the webtoon Yumi's Cells by Lee Dong-Geun, one of the most successful webtoon series in Korea in 2015. The webtoon tells the story of an ordinary woman in her 30s, Kim Yumi, and the 200 different cells living inside her brain, which control her behavior and emotions. The exhibition offers visitors an opportunity to reflect on themselves and it fosters empathy with the character for female visitors in their 20s and 30s. The exhibition consists of three main zones, each having sub-zones (  (Koleva et al. 2009;Bannon et al. 2005), taking visitors on a rich cultural journey (Stogner 2011): large-scale art installations, illustrations, photographs, multi-screen video, 2D animation, sound and music, participatory zones, and interactive media. Visitors are encouraged to engage with interactive media and participatory activities: For example, they can take a short survey to find out their own Prime Cell, or they can take an exam to test their knowledge about the webtoon. Moreover, readers, who disliked Yumi's ex-boyfriend, can enjoy punching a sandbag portraying him. In the projection-mapping room, a famous Korean pop song (Jaurim's 'Something Good') is used as background music while visitors can watch a five-minute video showing the memories of Yumi and Woong. To better understand the specificity of the Instagrammable exhibition, we compared the automated image analysis of Yumi's Instagram posts with that of posts related to a museum with more traditional art exhibitions, the Museum of Modern and Contemporary Art (hereafter MMCA) in Seoul.

Analysis of instagram images
As of January 10th, 2021, there were 8,352 posts on Instagram generated by users regarding Yumi and we randomly collected a bit more than 50.0% of them (4,267 posts, 24,312 images). We then categorized the images using the Google Vision API, a computer vision technique able to recognize objects, returning at most ten labels for each image (e.g., 'Toy' or 'Person', cf. Figure 2). The labels correspond to categories that the algorithm learned from previous training done by Google and have a confidence score and a topicality score, indicating the reliability of the object recognition and the object's contextual importance within the image.
As the next step, we classified the images using an existing list of labels derived from museum-related Instagram posts, which were clustered based on their semantic similarity, i.e., how frequently they appear together in an image (Rhee et al. 2021). We combined the existing clusters with top-down art categories relevant for our research and type of exhibition, creating the following final list of image types: artwork (e.g., cartoon, toy, illustration, and graphics), people (e.g., non-face body parts and people), selfies (e.g., faces), goods & ticket, exhibition space (e.g., interior design), food, landscape (e.g., outdoor spaces and nature), and architecture (e.g., buildings, or parts of them, and indoor spaces).
The second kind of image analysis that we performed focused on the pose of visitors to estimate a quantitative score for the kind of interaction that visitors have with the exhibition. Indeed, while people are often simply standing when contemplating art objects in the traditional art museums, we noticed a variety of interactive poses in Yumi's pictures. To analyze visitors' poses in pictures, we used a framework to cluster the images according to the pose of the anthropomorphic figures in them and an interaction score associated with them. The process, summarized in Figure 3, involves the identification of the  pose, the assignment of an interaction score to the pose, the clustering of images according to the presence of the same pose, and the cluster-wise interaction analysis. In brief, we measured the arms and leg's joint coordinates of the figures in an image (Cao et al. 2019), then calculated the cosine of the angles between the joints' coordinates (Huang, Hu, and Chang 2011;Zheng, Chen, and Lu 2020;Xu et al. 2020), and finally clustered the images according to their cosine value (thus standardizing values between −1 and +1), which represent the visitors' pose. Then, we determined an interaction score using a representative image of each cluster. In this process, some images were eliminated because their poses can be considered outliers with respect to the majority of other poses.

Image analysis
Most of the images received the maximum number of labels (ten) by the computer vision algorithm, with two categories appearing much more frequently than others: artwork (53.8%) and people (31.9%) ( Table 1). Differently from previous research (Rhee et al. 2021), the 'people' category was very frequent, but the proportion of selfies was limited only to 1.4%, which is lower than what we found for traditional museums. Browsing a sample of the images, we noted that some subjects were more common than others: e.g., the human-size blue Cells (Figure 4), categorized as 'artwork', were present in each zone and appear in many pictures. There are not many close-ups of objects' details, differently from what was reported by previous research (Villaespesa and Wowkowych 2020;Rhee et al. 2021). In this exhibition, the webtoon characters (e.g., Kim Yumi and the Cells) have been turned into various types of merchandise, including stuffed toys, postcards, books, stationery, etc. The goods images have been uploaded twice as much as those of food, although Koreans usually take many pictures of food (Kim 2018). In previous research (Rhee et al. 2021), 'architecture/landscape' was the second most popular category pair, suggesting that visitors take pictures of museums from the outside to show their presence in a place devoted to art. However, 'architecture' was the least frequent category in the case of Yumi, even though the exhibition space was in an architectural setting (i.e., the Brickwell building) which has Instagram-worthy features. Most landscape images consisted of photos of surroundings taken from the terrace which was part of the exhibition space.
Since the visitors were encouraged to engage in participatory activities, a large portion of images included artworks and people together (8,492, 34.9% of total). The skeleton analysis of anthropomorphic figures (humans and sculptures) showed that Yumi prompted visitors to take much more pictures in active poses than in more traditional art museums, like the MMCA. The difference is huge: 61.2% and 1.5%, respectively (Table 2). Active poses consist of specific interactions with the artworks, such as mimicking them or performing beside them. They can be categorized into three types of poses: Interaction pose, e.g., crouching to peep a book hold by a Cell; Mimic pose e.g., sitting crosslegged like a Cell; Performance pose, e.g., holding a trophy and express joy as a winner of the award ceremony which is part of the story ( Figure 5). In Instagram pictures about Yumi, interaction (29.1%) and mimic (24.7%) are the two most frequent types of active poses, appearing as frequently as passive standing poses (27.8%), which is the most popular type of pose in traditional art museums.

Discussion
Although Yumi provided visitors with perfect selfie backdrops, selfies turned out not to be the dominant kind of Instagram posts, contrary to our hypothesis (H1). Our computer vision-based image analysis identified that more than a half of the images covered artworks. People-related pictures were more than one-third of total images, but the proportion of selfies was relatively limited among them (RQ1). This suggests that visitors are more interested in showing themselves in the context of the exhibition rather than taking selfies. Indeed, most of Yumi's visitors belong to Generation Z (sample surveyed at the exit), who values individual expression and pursue more memorable experience for social media feeds (Rhee et al. 2021). The aforementioned results have also been confirmed by the pose analysis: the great number of pictures with active poses (H2) revealed that engagement and interaction with the installations is an important aspect for visitors of Instagrammable exhibitions (RQ2). Visitors' 'performative gestures' (Burness 2016, 97) and 'performative memories' (Arias 2018) are preserved in Instagram posts and ultimately can contribute to selfexpression and identity construction. We discovered several behaviors in visitors' pictures in terms of the co-performativity of visitor's body with the displayed objects: some visitors position their bodies to appear as part of the object on display; some use their bodies to recreate the bodies depicted in artwork on display (Hunter 2018); some stand alongside with the objects, against the backdrop of the wall installations; some take a pose for selfexpression; and some imitate the style of facial expression or body gesture of the artwork (Siswowihardjo 2020). Previous reflections on visitors' pictures focused often on selfies: To quote JiaJia Fei, director of the digital at the Jewish Museum of New York, the message of museum selfies is evident: I was there. I came, I saw, and I 'selfied' (Fei 2016). Selfies taken in the art space enrich the visitor's experience as networked material-discursive entanglements (Warfield 2016). The act of taking pictures of oneself within an art space can be regarded as an act of art engagement and meaning-making (Zhao and Zappavigna 2018;Seo 2019). However, we provided evidence that young Korean visitors prefer to show their whole bodies within the exhibition space, an activity requiring the coordination with a companion who takes the picture. We also found that people tend to take pictures while engaging in activities with friends, since group pictures provide an opportunity for individuals to present themselves as part of a social group and to highlight desirable social relationships through visual memories (Kim and Chock 2017;Serafinelli 2020).
For visitors of younger generations, an exhibition is an experience to be shared with others, not just after leaving the museum but starting from social interactions within the exhibition space. In this context, the space itself needs to be curated, highly photogenic and immersive, not just the displayed installations. In the case of Yumi, only 3.9% of pictures was related to the exhibition space, but two reasons can explain this apparent contradiction: first, Yumi's space was filled with sculpture-like art installations, which have been identified as 'artwork' by the computer vision algorithm; second, when there are people in the foreground, the exhibition space is not technically recognized as a central element of the picture.
Artworks placed in the background of Instagram posts are used as wallpaper or prop to create both an online memory and evidence that visitors have been there. Therefore, the creation of artwork images represents a new way of looking within the exhibition environment (Burness 2016). In other words, visitors can create complex layered forms of visual communication during and after the visit. The images posted on Instagram provide the opportunity for visitors to communicate their experience through the choice of the photo subject, and for museums they are a way to promote and recommend the exhibition, potentially reaching and attracting more visitors (Suess 2014;Amanatidis et al. 2020). The pictures taken by visitors are devices for recording and documenting relationships and experiences in a material cultural space (Carah 2014). They give an insight into how visitors perceived objects at that moment (Budge and Burness 2018). In addition, visitors are re-curating the exhibits that they share online, by re-organizing the exhibition in their own ways through pictures and social media posts (Weilenmann, Hillman, and Jungselius 2013).

Conclusion
While museums use different sets of metrics to evaluate their social media activities and better understand their online audiences, very few examples exist of how to analyze a large quantity of Instagram data regarding visitors' experiences. This study analyzed visitors' engagement with the Korean Yumi's Cell Special Exhibition, employing computer vision for the analyses of both visitors' Instagram photos and the poses found in them.
Yumi had a strong storytelling-entertainment feature associated with the original webtoon, which positively influenced visitors' engagement. One of the challenges for museums wanting to curate Instagrammable exhibitions related to art is to add storytelling elements and social moments able to foreground the aesthetic value of artworks. Although Instagram-worthy exhibitions are sometimes devalued as commercial exhibitions for profit-making, this study suggests that storytelling features and curation of the exhibition space can be leveraged to enhance visitors' engagement with art. To sum up, Instagrammable exhibitions offer new possibilities to curators who would like to attract young visitors, but there are also challenges related to the embedding of aesthetic and cultural values within social and entertaining experiences.
The intentionally performative pictures found in Instagram posts are evidence of how visitors take pictures in the exhibition and how they engage with the exhibits, illustrating the dialogue and exchange occurring between objects and people (Dudley 2010). In this view, the museum space becomes primarily a performance space (Greenberg 2005, 228) where the focus is on the visitor's stories and narratives with art (Steyn 2014, 230), rather than on the contemplation of artworks, like in traditional museum. In conclusion, the image analysis of Instagram posts generates insights into visitors' behavior, how they interact with objects and what is valued for the visitors.
The interdisciplinary framework presented and the findings in this research can inform museum professional practices. Every methodology has its pros and cons: Traditionally observation is a valid and reliable analytical method for measuring timing, tracking the choice of exhibition zone, the attention-span, and visitor behavior; however, observation is used to collect data concerning non-verbal behavior, and therefore does not provide a direct contribution in the problem areas of understanding and learning by the public who visit an exhibition (Bollo and Dal Pozzolo 2005). In addition, manual tracking and coding of behavior can be laborious and somewhat dependent on the coder, and questionnaires are subjective by nature and cannot provide accurate behavioral data (Lanir et al. 2017, 2). Social media image analysis using computer vision can be a valuable complementary technique, useful to show recurring patterns and how exhibitions and visitors' behavior are communicated outside the museum's space.
The techniques presented in this study can be used to efficiently process and analyze a large quantity of data. It can provide museum personnel with detailed and accurate information that relies on data of hundreds and thousands of visitors rather than on a few observations. The analysis of this kind of data can then be used by museum curators and personnel for assessment and for getting insights about visitor behavior at their museum. We are currently participating in a project of the Seoul Museum of Art in Korea to develop a platform for predicting visitors' behavior. In this project, technologies such as big data, artificial intelligence, topic modeling approach, and visitors' position, movement and interaction in the exhibition environment are being studied. We hope this project will present a technology-oriented research framework for museum visitor studies in the near future.