Designing Interactive Music Systems with and for People with Dementia

: In this paper, we outline the process applied in designing and developing an interactive and networked music system for people with dementia. We first describe the strategy we used to include both people with dementia and caretakers. As the symptoms of dementia vary greatly, we decided to address this diversity of needs. We describe the use-scenarios, which illustrate how people with dementia can gain access to curated selections of biographically meaningful music and take part in music therapy even if restricted mobility prevents their physical presence at sessions. Secondly, we focus on the structural and aesthetic requirements for the interaction design of tangible and screen-based user-interfaces. We conclude with issues of usability and levels of playfulness based on interactive prototypes that differ in complexity and form, and we share proposals for designing with and for dementia patients.


Introduction
In Western societies a growing number of the elderly population lives with symptoms of dementia. As a neurodegenerative disease it affects cognitive abilities. People with dementia sufferdepending on the level of illness -from memory loss, deficits in attention, loss of orientation and often depression. It has an adverse effect on everyday life and challenges patients as well as caretakers, particularly when people with dementia are being taken care of in their homes. The speed of demential progression differs individually in both long and short term: The decrease of cognitive abilities over months or years proceeds at different rates as well as a patient's mood and orientation depends on daily conditions. The positive effect of music on people with dementia has been known for quite some time and is described in several studies (Särkämö et al., 2014, Morrissey & McCarthy, 2015, Svansdottir & Snaedal, 2006. People with dementia often act and behave in an agitated manner. In advanced cases they often suffer from apathy, lose their day-night-rhythm and feel disoriented. To some degree, personal biographies and the ability of expression can get lost. Listening to music and participating in musical group activities can help to delay symptoms and even reactivate long-term memory (Särkämö et al., 2014). But it has been demonstrated that its effects go beyond the reduction of behavioral and psychological symptoms. As McDermott et al. argue »music is closely linked to personal identity and life history of an individual" and thus it does affect the human foundations of emotional stability and self-concept. The question » [...] how people at all stages of dementia can access music [...]« (McDermott et al. 2014, p. 715) provides the basis for our project goals. Furthermore, their conclusion emblematizes our motivation for a design-driven approach by means of participatory methods, iterative prototyping and testing of interfaces: »Sustaining musical and interpersonal connectedness particularly when the progress of dementia becomes more prominent would help value who the person is and maintain the person's quality of life.« (ibid.)

The Divergence between Users' Needs and Accessibility
To ensure an early and active involvement of the users we design with and for, we have worked with a set of workshop methods adapted to the specific requirements of the elderly and people with dementia (Hendriks et al., 2013, p. 655 et seqq.). We invited people with dementia to a workshop in order to understand their relation to music in various respects: How listening to music used to be an integral part of their everyday lives, e.g. through listening to the radio regularly or through the usage of sound recording mediums. What could be quantitative or qualitative parameters for mapping a musical biography? Furthermore we surveyed how the participants could handle different sound storage mediums and whether speculations about their technological know-how are founded in personal-epochal or pathological occasions. Using participatory methods (Sanders, 2000), we engaged the participants in discussions (Figure 2), gained insights into their specific everyday routines ( Figure 1) and identified patterns of interactivity with a set of haptic materials. As Hendriks et al. suggest (2013, p. 656), we prepared visual elements and physical artifacts in order to stimulate personal memories and narratives: images of ballrooms and domestic singing as well as different instruments were shown; haptic playback devices, different sound recording media and musical toys embodied representatives of past musical events.
Even though they had only so-called mild or moderate symptoms of dementia -like loss of orientation and loss of short-term memory -all of our participants were accompanied by caretakers or relatives. Hence, we ensured their physical and mental well-being and unintentionally received comprehensive comments in the discussions and on observations. Symptoms and behavioral change caused by the disease occur in different types and vary from person to person. It is essential not to jump to hasty generalizations. But the emotional reactions to the music we played in the beginning of the workshops were universally positive: smiling, singing along to the songs and tears of joy.  Apparently, there is a divergence between potential therapeutical intervention and psychological well-being through music on the one hand and a significant absence of music in the everyday lives of people with dementia on the other hand, in spite of a profound desire for musical experiences.

Project Goal, Use Cases and Setup
The overarching goal of the project is to overcome this gap between the obvious benefit of music in the everyday lives of people with dementia and the missing structures and accessibility for it. Even though we think a relative, friend or caregiver providing access to music and musical therapy is the ideal scenario, we propose a technological system for the many cases where restricted mobility, deficits in personal day care or other circumstances lead to a lack of musical experiences. Thus, we aim to design the interfaces of a musical system for people with dementia and to explore possible interactive scenarios. First, the technical setup ought to provide easy access to a digital streaming platform enabling people with dementia to enjoy the individual experience of listening to music they like. Second, the system should allow patients to participate in music therapy sessions through telepresence in case of restricted mobility that prevents their physical presence at sessions.

DESIGNING ACCESS TO MUSIC AND INTERACTIVE MUSIC-BASED INTERVENTIONS
Taking a designer's perspective, we framed the following questions arising from the aforementioned discrepancy between the positive effect of listening to music and participating in musical activities and the dramatic deprivation of access to music caused by the onset of dementia. These challenges are also rooted in the observations we made during the participatory workshops with people with dementia: • How to design access to digital music platforms?
• How to take personal biographies and preferences into account if people with dementia can hardly remember titles, albums and artists? • How to structure an interactive system that adapts to the cognitive and physical abilities of people with dementia and allows for a decremental shift of complexity in the user interface and its use?
The project's challenges and tasks were taken on by a research cooperation with participants from the fields of software engineering, hardware engineering, health professionals, retirement home institutions, data privacy and design. After we had identified the users' needs, understood patterns of the users' technical know-how and observed everyday routines of people with dementia, we transferred these insights to our project partners in order to map the technical setup for the interactive music system.
Mutual approval of scenarios, use cases and hardware was achieved through the usage of personas, paper prototypes (cf. Snyder, 2003) and mockups. Interactive click dummies were used to clarify operative levels within the menu, restricted access areas for people with dementia and caretakers as well as data privacy issues.

Designing Interactive User Interfaces and Experimental Prototypes
From our observations during the workshops and interviews we conceptualized a framework for interfaces that aim at compensating the loss of technical know-how and the ability to interact with music systems. The following categorical approaches are not to be seen as a generalization with respect to the symptoms of cognitive deficiencies. Rather, they present an experimental starting point for further research on the relation between memory loss concerning the use of technical devices, the design of tangible interfaces, and the affordance to interact with a (therapeutic) music system as described by Dankbar et al. (2016, p. 166 et seqq.).

MEMORIES OF HISTORICAL CONVENTIONAL USE CASES -THE RADIO
Before recorded songs were accessible as digital streaming or download offers, the majority of radio and stereo systems for domestic use were equipped with a haptic interface, basically push buttons, rotary knobs and slide controls. A definite mapping of control and response led to restricted but reliable interaction scenarios: According to our observations with the people with dementia, starting and stopping the recordings, controlling the volume etc. were well-known procedures recalled from their long-term memories. Adopting these types of interaction scenarios to a streaming platform delineates our first category: Referring to historical and commonly known use cases and conjoining them with a digital structure that is invisible for users.

LINKING PERSONAL IMAGES WITH SONG -THE INTERACTIVE ALBUM
For the second category we augmented familiar objects with an additional layer of music. A userspecific photo album is linked to a personalized playlist. As music has a strong relation to the personal lives (McDermott et al., 2014) and can help to reactivate long-term memory (Särkämö et al., 2014) the interactive album links events to meaningful songs. The photo album is meant to be filled with individual photographs and songs by relatives or caretakes. Songs are played back by touching a sensor area on each page ( Figure 6). Working with conductive ink and paper, the haptic and visual characteristics of an traditional photo book remain. With this approach we propose a remix of analog media with known electronic interfaces. Thus, we provide easy-to-handle technology, that is recognized by people with dementia.

UNTROUBLED PLAYFULNESS -PUZZLING OUT COMBINATIONS WITH SOFT MATERIALS
For handling the third type of interfaces, prior knowledge of technical systems is not mandatory as the control elements are not based on known patterns. References to historic devices, symbolic markings and forms are avoided. We emphasized a new correlation of tangible representations of songs and playlists together with soft textiles attached to a basic element. The models call upon a need for haptic investigation by the users. Curiosity for haptic and physical qualities as well as the need for understanding the interplay of flexible and modular elements provoke users to interact with them. This approach and type of playful interfaces results from workshop observations: textile and soft materials were given to the participants with dementia. Intended for the use in mapping methods, planned tasks and research methods were not completed. In lieu thereof, severeral workshop participants with dementia started avidly playing with the given samples. The actions of bouncing, squeezing, sorting and stroking disclosed a strong interest in the material quality and structure of soft materials. Giving this playfulness a high priority in the design of interfaces for people with dementia is a promising approach. Puzzling out combinations with soft materials seems to increase the motivation to play.

CONCLUSION
In this paper we described our approaches associated with participatory design with and for people with dementia. We described methods we applied during the process and suggest categories for designing interfaces that vary in their reference to existing technologies. With the presented approaches we are aiming at a debate on referential interfaces for people with dementia and levels of playfulness in order to provide access to digital contents, especially with a focus on music. How these types of interfaces are being deployed in everyday use needs to be tested. The lessons we learned help us to derive design guidelines for future projects and bear witness that people with dementia can be actively involved throughout research and development process.