The Cyborg Hand: Gesture, Technology, Disability and Interdisciplinarity in Whatever Weighs You Down

ABSTRACT This article examines the collaborative genesis of a new work for solo performer co-composed by Neil Luck and the author, Whatever Weighs You Down (2022), with a focus on the use of MiMU sensor gloves, and the development of a work-specific gestural approach to this technology. This gestural approach was strongly influenced by a collaborator on the project, Deaf performance artist Chisato Minamimura. Minamimura’s expertise in BSL, dance, ‘sign mime’, and other approaches to movement allowed the creative team to collaboratively develop an approach to the gloves that explored the communicative dimensions of gesture, rather than simply maximising their functions as instruments. This collaboration demonstrates how new and innovative approaches to digital instruments can emerge through collaborations that include disabled artists.

This article explores the intersection of compositional approaches to digital instruments and the collaborative development of musical works with disabled artists, providing new perspectives on the role of disability in the collaborative creation of new works integrating new interactive technologies.It examines the collaborative genesis of a new work for solo performer co-composed by Neil Luck and myself, Whatever Weighs You Down (2022), with a focus on the use of MiMU sensor gloves, and the development of a work-specific gestural approach to this technology.The work was commissioned as part of my UKRI Future Leaders Fellowship project, Cyborg Soloists, a research project that draws together composers, performers, researchers and industry partners to explore new ways of using recent technologies (including digital instruments) in live performance. 1 The project included a third collaborator: Deaf performance artist, Chisato Minamimura, whose performance appears on screen as part of the film parts of the work.Her collaborative workshops with Luck and myself, and the influence this had on Luck's approach to gesture in the work, and my programming of the gloves, raise important questions about the relationships between instruments and gesture, the importance of gesture to musical works in the expanded field, and the role of technology in facilitating collaborations with disabled artists.

MiMU Gloves-A Wearable Digital Instrument
Gesture and motion sensors attached to the performer's body have been used by experimental musicians since the 1980s, including by improviser and violinist Jon Rose using early SensorLab technology at STEIM (Overholt 2012, 85).They have also been used extensively for the study of musicians' movements (including in Caruso et al. (2016)) and the use of gestural instruments in performance has become its own field of research, including major contributions by Claude Cadoz (1988), Rolf Gødoy and Mark Leman (2010), Tanaka (2009), andEduardo Miranda andMarc Wanderley (2006).In the last 15 years, the use of these types of digital devices has been gradually democratised (both financially and in the technological learning curve required to use them).From my own experience working with different generations of this technology over the last 10 years (Kanga 2017a(Kanga , 2017b)), the technology has developed from wired, large, cumbersome and ill-designed for use on a performer's body, with few options for gesture-tosound mapping, to wireless, sophisticated digital instruments with a large range of mapping options, placed into easily wearable rings and gloves and integrated with common Digital Audio Workstations such as Ableton Live and Logic.As Atau Tanaka has discussed, these devices are not fixed instruments, but interfaces with wide musical affordances that allow for a variety of idiosyncratic 'instruments' to be created: The autonomous, standalone quality-the embodied nature-of a mobile instrument allows specific, possibly idiosyncratic, mappings to be implemented.In this, the hardware itself may remain a general purpose device, and the underlying software framework may be generic, but the instantiation of these resources in the form of an instrument are specific for any given musical situation.(Tanaka 2010, 92) Thus, these technologies are not just a single instrument, but a variety of spaces of possibilities and modalities, open for the musician to define and refine through the choice of mapping and sounds.
The role of the performer when working with these tools is complex.Except in cases where the composer has a high level of expertise in the specific device, the performer functions as a type of instrument-builder, designing the patches required to turn the device from a mere interface into an instrument that is calibrated to their own gestures.As with many digital instruments, there are additional barriers to building a digital instrument that is well-integrated with the performing musician, as Ben Spatz (2015, 13) points out: 'the "cyborg" nature of technologically focussed practice does not (yet) entitle us to dissolve the conceptual differences between biology, ecology, and technology'.Spatz points to wealth as a major barrier to this dissolution.But as sensors have become increasingly affordable, I have found that time (and the expertise it facilitates) is the most important currency required, and it is time that was needed at all stages of the case study discussed below to discover a new approach to 'cyborg' performance.
The development of wearable sensor instruments is guided (in part) by the desire for greater degrees of control when gesturing with the hands alone-allowing the hands to function as an instrument.Claude Cadoz and Marc Wanderley (2000, 78) have defined the three types of hand gestures that can be used in performance: ergotic (where there is an energy exchange between a hand and an object), epistemic (which typically involves the performer's capacity for touch and muscular/articulatory perception) and semiotic (which involves meaning or communicative intent).The desire to utilise all of these within a single gestural instrument is a possible way of understanding the impetus for developing a technology like the MiMU gloves.And yet when using an instrument with all these capabilities, it does not automatically follow that all three of these gestural types will be explored in the work.
I have used a number of different types of wired and wireless motion sensors (with built in accelerometers) for around 10 years, and in my experience, the MiMU gloves are a substantial advancement over previous gesture and motion-based instruments.MiMU, a UK-based company (and industry partner of Cyborg Soloists), developed the gloves in collaboration with singer, songwriter and multi-instrumentalist Imogen Heap. 2 The gloves combine a 3D accelerometer/gravimeter that measures movement and orientation, but also has flexor sensors for each finger, allowing it to trigger sounds or other events using hand gestures.MiMU uses a bespoke software interface, Glover, which uses machine learning to recognise new gestures, which can then be mapped onto sounds or effects within a Digital Audio Workstation (DAW).A Glover patch is shown in Figure 1.This shows how movements in three dimensions (pitch, yaw and roll in each hand), saved gestures (such as 'Open Hand' or 'Two fingers'), the movement of individual joints (such as the proximal joint of the index finger) and combinations of all these can be mapped to MIDI outputs.This is just a small sample of the mapping options-many other modes of control and types of outputs are possible.
I worked with the MiMU gloves on many other projects, both self-composed and composed by collaborating composers, before the final development of Whatever Weighs You Down.These allowed me to understand the idiosyncracies of the gloves and the Glover software, including how well it can differentiate between subtly different gestures.My own work, Steel on Bone (2021), uses the gloves to shape the sounds created using steel knitting needles on the strings of the piano, with the motions linked to delay and granulation effects, and with finger gestures triggering recorded samples to create seamless transitions between the live and virtual piano sounds.In collaborations with other composers, the mapping process is often left to me, although the sounds and gestures are often provided by the composer.This was the case with Laura Bowler on her work, SHOW(ti) ME (2022), where the gloves were used to both trigger samples (that exaggerated the theatre of the work) as well as shaping the live sounds of the piano.In Whatever Weighs You Down, I again took on the role of mapping the gloves and creating the work-specific instrument.But it was fundamentally different to previous collaborations in that this task would be shaped by our collaboration with Minamimura, who was unfamiliar with the gloves but still influenced our use of them in profound ways.

Technology, Music and Disability
When discussing disabled artists, it is important to note that 'disability' is a loaded, and often stigmatising term.Disability is defined in terms of societal norms that are contingent on both an individual and the culture they live in, and only some artists who might be considered to have disabilities publicly identify as disabled artists.I use the term 'disabled musician' as used by Samuels and Schroeder (2019, 477) in their article on technology and accessibility: using the 'social model of disability'.This model is also used within Disability Studies and by the Disability Rights Movement.As Howe, Jensen-Moulton, Lerner and Straus explain in The Oxford Handbook of Music and Disability Studies (2016): 'under this model, disability is not a fixed, medical condition; rather, it emerges from a society that chooses to accommodate some bodies and exclude others' (2).
There are many examples of technology being used to improve the accessibility of contemporary music to disabled people.Samuels and Schroeder (2019, 484) have discussed their own experimental VR instrument, created as part of the 'Performance without Barriers' Research Project at the Sonic Arts Research Centre (Queens University, Belfast), to allow musicians with movement disabilities to improvise using electronic sound.Samuels (2015, 26-28) has also written about his fieldwork on Drake Music Northern Island, and their work using MIDI controllers with control features (such as joysticks or modified keyboards) that facilitate greater accessibility.Vahakn Matossian and Rolf Gehlhaar (2015, 206) have examined two different mouth-controlled MIDI-devices, to function as keyboard-like controllers.And Emma Frid (2019, 1) has surveyed the range of digital instruments created to enable greater accessibility to disabled people, identifying 83 different instruments, ranging across touchless controllers, Brain-Computer Music Interfaces (BCMIs), adapted instruments, wearable controllers or prosthetic devices, mouth-operated controllers, audio controllers, gaze controllers, and touchscreen controllers.
There are also ensembles that utilise digital technology to facilitate performances that include disabled musicians, including the UK-based Paraorchestra, which features 40 musicians who identify as disabled, D/deaf, or neurodivergent. 3Their collaboration with Hannah Peel on The Unfolding (2022), featuring 14 acoustic and electronic instruments alongside onscreen animated visuals, illustrates how digital technologies can facilitate complex multimedia performances by ensembles featuring both disabled and nondisabled performers. 4Similarly, the Sweden-based Elefantöra ensemble combines nondisabled and disabled musicians in works that explore the integration of digital technologies, and Henry Boothby (2023, 27-29) has discussed their reappropriation of Zoom and the overlaying of pre-recorded sound to create his complex online work, Listening with Elephant Ears (2020).
Although technology can play an important accessibility role for disabled artists, there are many common ways in which all artists (with and without disabilities) extend the capabilities of their body with technologies.Jennifer Iverson points to the French architect Le Corbusier's view that all humans need technological supplementation: 'We all need means of supplementing our natural capabilities, since nature is indifferent, inhuman (extra-human) and inclement; we are born naked and with insufficient armour' (Le Corbosier 1972in Iverson 2016, 160).In discussing Björk's electro-acoustic extensions of her voice and body, Iverson argues that this type of cyborg music breaks down the ideal of technology as a prosthesis to assist disabled people: the 'wholeness' or 'naturalness' of bodies is a fiction.The prosthesis, the supplement, the cyborg, is in all of us.We cannot be outside of this technological supplement, because it is us.There is no productive distinction to be made between self and other, between human and cyborg, between nature and technology or between dis/abled.(Iverson 2016, 171) Laurie Stras finds similarities between enhancement of musicians through technologies such as auto-tune, and the ethical debates around performance-enhancing technologies in sport such as prosthetics used by paralympians (Stras 2016, 181).She points to a future technological and aesthetic convergence between musicians who require technology for access, and those who use similar technologies to enhance their performing bodies (188).
I have observed this ambiguity between non-disabled and disabled artists in my own collaborations.Amble Skuse identifies as a disabled artist but has developed a sophisticated practice around the MiMU gloves that both suits her body, and surpasses the complexity of practice of many other composers who have worked with the same technology.She describes her practice as reflecting 'a disability-inclusive model that embraces diverse individuality and celebrates what is created in the moment, and doesn't seek to impose one definition of "normal" or any pre-conceived notion of what ought to be achieved' (Skuse, correspondence with the author, 2023).
Disabled artists aren't just 'supplemented' through the use of technology, but bring insights and expertise to these devices and instruments that exceed the abilities of many non-disabled artists, foregrounding the collaborative party's expertise, rather than varying bodily abilities.The affordances of technologies like the MiMU gloves facilitate these inclusive forms of musical artistry, allowing disabled musicians to bring their expertise and lived experiences to provide new perspectives on both the technical and musical capabilities of those devices.Although the collaboration below discusses an artist who selfidentifies as a Deaf artist, this case study is not about disability, but about the unique expertise that each artist brought to the project to create a collaborative work.

Whatever Weighs You Down (2022)
Whatever Weighs You Down is a 40-minute work for piano, MiMU sensor gloves, live and fixed electronics and two videos, composed by Neil Luck in collaboration with me. 5 The work draws together the preceding topics: the use of new technologies, and a collaboration with an artist with a disability.
Neil Luck 6 is a UK-based composer, whose work often uses many different art forms in combination-live and electronic music, theatrical elements, videos and both spoken and recorded text-in contrapuntal layers, with a density of content that all points towards, without spelling out, a central concept.Whatever Weighs You Down builds on two previous collaborations between Luck andme: 2018 (2016), which explored a sci-fi world of genetically engineered pianists, and Modern Times and Forms of Love (2021), which explored our relationship to technology through parodies of, and tributes to, keyboard advertisements and keyboard-based ballads.Both of these works utilised multiple layers of narrative and communication, with a high density of information conveyed through text, visuals and music, as well as non-standard hand gestures.All these aspects were extended and developed in Whatever Weighs You Down, but for the first time his approach to gesture and theatre could be sonified.The themes of the work are not overt, but focus around different forms of falling and descent, contrasting with different images and gestures of strength and recovery.One can interpret sections of the work as exploring states of depression, and the recovery from this as linked to new forms of communication, but Luck has not articulated the work's themes so literally.
When composing Whatever Weighs You Down, Luck was keen to use the MiMU gloves, and to work with an actor or performance artist who was versed in British Sign Language (BSL).This followed from his interest in the practice and techniques of Object-Based Media.This is an approach to media (such as television content) that facilitates the production of different versions catering to people's individual needs and disabilities to increase accessibility, including different sound mixes, audio description or BSL interpreters. 7Luck previously explored Object-Based Media in Any's Responses (2015), Deepy Kaye (2018) and his 'Screen Test' series (2017)(2018).After our initial meetings, we both agreed that we needed to work with a Deaf performer to have the most respectful and genuine treatment of BSL in the piece, given it is not only a language but part of Deaf culture.
Chisato Minamimura 8 was an ideal collaborator for this role and brought an artistic skillset and experience into the collaboration that complemented both our skills and artistic ambitions.Minamimura is a Deaf artistic director, performer and BSL art guide based in London.Her artistic practice involves the creation of new art forms through the meshing of sensory perception, 'sign mime', BSL art guiding, digital technology and performance.She has a particular interest in the visualisation of sound and music from her Deaf perspective, and has developed what she terms a 'visual score', which uses dance movement to convey musical sound. 9Minamimura's experience and range of movement and gesture-based skills were well-suited to both Luck's compositional interests, and my interest in exploring new approaches to the sensor gloves.We discussed her role as an onscreen performer, and proceeded with a series of workshop days at Royal Holloway in late 2021 and early 2022 culminating in a filming day.

Workshops: Developing a Unique Approach to Gesture and Technology
The workshop sessions had elements of composer-performer workshops familiar to both Luck and myself, but also many unfamiliar practices.We required two BSL interpreters to translate between Minamimura and the two of us, so the workshops had one additional filter to the communication between Minamimura and Luck and me.Despite this, the workshops were free-flowing and playful, exploring multiple approaches to the role of movement (both mine and Minamimura's) in the work.
Luck arrived with a number of pre-composed score fragments, as well as larger concepts he wanted to explore around falling and descent, strength and lifting, and multiple forms of communication (and miscommunication) and language across media.We began by focusing on how Luck's score fragments could be experienced and re-interpreted by Minamimura.This involved experiencing the sounds of me playing these fragments by placing her hands and feet against the body of the piano (Figure 2); imitating and re-interpreting my performing gestures to become a movement-based version of sections of the score (Figure 3); interpreting the text of a 'song' Neil had written through gesture (Figure 4); and engaging in a type of BSL-related communication with me where we both attempted to explain a single story (prompted by Luck) through gesture alone (Figure 5).These different approaches became different sections of the piece.
Of the many different types of gesture and collaboration we undertook, three in particular relate to the use of the MiMU gloves, which we called 'power gestures', 'the conversation' and 'the song'.The 'power gestures' are poses intended to express power or strength (in contrast to the themes of being weighed down, falling and descent that dominate the first half of the piece).Luck had collected images of these poses, including superhero comics and photos of strongmen, as cues for improvisation but was keen for original, rather than clichéd, gestures.As can be observed in this screenshot (Figure 6), we played several games with improvising gestures, throwing and mirroring gestures with slight variations until we found a set of sequences that would be both expressive, and would work with the functional limits of the gloves.In developing the 'conversation' Luck's cue for this improvisation was that we both attempted a conversation about a dream where we are falling into a void (again relating to the central concepts of the piece) and workshopped these with different restrictions-Minamimura would use BSL from increasingly restricted positions, and I would attempt to convey my ideas through larger, non-BSL gestures.Experimenting with increasingly restricted positions, Minamimura finally performed her part from a curled-up position under the piano, abstracting it from the original BSL movements.Meanwhile, versions of my larger miming movements were used for glove-controlling gestures.The 'song' featured the interpretation of text for a song through gesture-Minamimura developed her movement-based interpretations of these texts with us, with the end goal for me to perform the 'song' on the piano (both melody and accompaniment transcribed for one hand) with the other hand interacting gesturally using a glove, while her interpretation of the text would appear onscreen. 10 These playful, exploratory sessions focused on movement and gesture in relation to the score and my performance of it.Minamimura was particularly positive about the free approach to workshops, 'I really felt a strong sense of freedom in the processes we went through together … I especially appreciated the opportunity to delve so deeply into the world of sound' (Minamimura, correspondence with the author, 2023).In turn, were afforded the opportunity to delve deeply into the world of BSL and other forms of communicative gesture.
At this stage of the collaboration, the mapping of the gloves, and the sounds they would control, had not yet been determined, so this specific instantiation of this 'instrument' could be designed in response to Minamimura's explorations and interpretations.So, although Minamimura never uses the gloves in this work, she had a profound influence on the way we devised their use and function as instruments.She had created gestures, sequences of abstract movements, and re-interpretations of both the text and the piano part, generating a gestural approach to the use of the gloves that opened up new approaches to this technology.

Composition as Building an Instrument
Creating the Glover and Ableton patches required us to build a work-specific 'instrument'-a particular instantiation of the many possible mappings and sounds for the gloves.This has similarities to Helmut Lachenmann's (2004, 3) conception of composition as 'building an instrument ' which, in Samuel Wilson's (2013, 57) interpretation, is about how Lachenmann conceives the relationship between the body and instrument in performance, 'as the body is always involved in playing instruments, building instruments encompasses wider concerns about what we understand the body to be, how we  use our bodies and how bodies are performed'.This metaphor is particularly relevant when applied to the use of digital instruments (Kanga 2017b, 117).
Translating the glove gestures and movements (and their associated sounds) into a score, and mapping these scored gestures to sound, was a challenge.Luck used photographs and drawings to show the specific gestures he required within the score, as well as textual descriptions of the sounds and the way movement would affect them, to convey the full functionality of the instrument (as realised in the work-specific mapping in Glover).The sounds included speech, synthesiser sounds, field recordings, and other sound effects, chosen to cohere into sets that could work in sequence, despite their differences.The gestures were drawn from what was generated in our workshop with Minamimura, alongside some contrasting gestures.My role as performer and technological consultant was to translate this score into a particular version of the gloves, as defined by this mapping: in effect, to turn them from an interface, into a work-specific gestural instrument.Figure 7 shows the information Luck provided for the 'power gestures'.The instructions describe how the designated sounds and their manipulations (including filters, pitch shifting, granulation, delays and EQ filters) should be mapped onto a set of gestures and associated hand movements.
For the 'conversation' section, Luck created a 'gestural passacaglia' using a repeating and gradually varying cycle of gestures from which I could improvise and vary (the sequence shown in Figure 8, which is repeated with variations).He provided a large collection of gestures, each one to be matched to a sound sample, with instructions for mapping these movements onto audio processes (such as filters and pitch-shifters), shown on the left side of Figure 8.
And for a final section that includes the 'song', Minamimura's interpretation of the lyrics were shown onscreen, alongside fragments of the original text.I performed the score of the 'song' with my left hand, while creating sounds through gesture with my right hand, using a subset of the 'passacaglia' gestures and some altered sounds in response to Minamimura's onscreen movement and dance-based interpretation of the 'song'.
The result of all these requirements of the MiMU gloves is shown in Figure 9: a Glover patch that was so large, I had to add a second set of virtual glove instruments to allow us to define enough different gestures, as seen below (with two pairs of gloves icons visible in the top left).
I also created an accompanying Ableton Live project which allows the use of individual samples to be controlled and altered independently by mapping all the gestures to particular sounds, and all the movements to particular effects, with multiple changes of state throughout the piece.
This process of instrument creation was my responsibility by necessity, because I was the only collaborator familiar enough with the gloves and Glover to understand their strengths and limitations when mapping gestures and movements to sounds and effects.This transference of technological responsibility is also a creative transference -the technology catalyses a greater distribution of creative input, so that the gestures were derived by Minamimura, the sounds and effects functionality are designed by Luck, but the instrument is implemented and calibrated by me.If we see creating an instrument as a fundamental aspect of composition, then this collaboratively created instrument is the core of the work, a combination of technology, gesture, composition/choreography and programming (gestural mapping) to create a workspecific instantiation of a digital instrument.

The Gesture-Instrument in Performance
The completed work was performed at Gaudeamus Festival, Utrecht, on 11 September 2022 and at Café OTO in London 13 October 2022. 11In performance, the use of performative gesture with the gloves was a central focus for the work, controlling the sound, and mirroring or communicating with Minamimura on screen in a complex play of sound, text and both on-stage and on-screen movement.Figure 10 shows the 'power gestures' as it was performed at Café OTO, with my onstage gestures with the gloves mirroring Minamimura's onscreen performance.These interactions between screen, gesture and sound might seem complex, but the connections were clear to viewers, as evidenced by a review of the Café OTO performance by Ilia Rogatchevski for The Wire, who described their impressions of the interaction: Black and white projections of Deaf performance artist Chisato Minamimura appear, showing her hands, mid-sign, glitching like electronic doves, while Kanga's MiMU gloves echo their movements.Composed for left hand, the third movement is reflective, Minamimura appears upside-down, like a half-forgotten fragment of a falling dream.(Rogatchevski 2023, 95) The work was subsequently featured in The New York Times, in an article that featured collaborations featuring new technologies to facilitate new types of creative interactions between artists with and without disabilities.Hugh Morris described the stage-screen gestural interactions: As the performance of 'Whatever Weighs You Down' drew to a close, it reached a striking semi-synthesis.Onscreen, Minamimura's gestures mirrored Kanga's onstage hand movements.Both performers provided a kind of accompaniment for each other, experienced in entirely different ways by audience members, depending on their relationship to sound.(Morris 2022) As both these descriptions demonstrate, there is a complex play of authorship and hierarchies played out on stage during the performance (Morris 2022; Rogatchevski 2023, 95).To the audience, the relationship of Minamimura's performance to mine is in constant flux: sometimes in counterpoint, sometimes directing and controlling my movements, and at other times in close dialogue, even though it is a fixed media film.The result is a densely contrapuntal work, with a complex interplay of musical content, gestures, media, modes of communication, and creative agencies-that this play of hierarchies and authorship is effectively communicated to audiences is a particular strength of the work.

Discussion
In thinking about Minamimura's contributions to the work, Luck said the following: For me the choice of the gloves provoked the choice to work with a BSL interpreter.So, the fact we were working with Chisato at all was inspired by the use of the Gloves, and foregrounding of the hand as a presence.(Luck, correspondence with the author, 2023) As Luck points out, the choice of technology preceded the focus on hand gestures, but once this was the focus, Minamimura's collaboration became an essential aspect of the creative process around this technology.He continued to discuss our collaborative development of the gestures: The kinds of gestures that were effective with the gloves very much fed into how we worked with Chisato.I think we were always looking for connections and resonances so pushed the movement in every area into this kind of shared gestural pool or zone.I remember we often tried to describe to Chisato what the gloves were doing, or could do sonically.I guess that inevitably influenced her movement and how we worked together.(Luck, correspondence with the author, 2023) This 'gestural pool' was collaboratively created, but heavily influenced by Minamimura's approach to the material as a Deaf artist.In many of our discussions, the gestures were negotiated, balancing out Luck's aesthetic aims, Minamimura's interpretation of the score, and my descriptions about the technical limits of the gloves.But Luck was adamant about one aspect of the gestures: Specifically, I also remember we tried hard, however, for the gloves gestures NOT to look like BSL.But rather to work with these more concrete, or absolute/functional gestures.We tried hard to avoid simplifying the act of signing, or to break it as a language (without knowing the language to begin with).(Luck, correspondence with the author, 2023) This final aspect was important in finding a respectful use of gesture on stage.Although we found gestures that were influenced or abstracted from BSL, it was important that literal BSL was not used with the gloves, so as to be respectful of the language and its purpose.
After seeing my performances (or online video clips) featuring the MiMU gloves, audience members have asked about what seems like an obvious use of the MiMU gloves: translating BSL into spoken audio.But as Luck explains, BSL is already a very sophisticated communication technology, without a need for extension or improvement, and the process was much more about us learning from this long-established 'technology' to build on our own practice with a nascent piece of technology.
Well language is a technology, writing is a technology.Translation is a technology.At least as far as Marshall McLuhan was concerned. 12All these things mediate understanding or meaning via some manmade system.But BSL to me seems like a highly developed and complex technology.Some people asked if MiMU could possibly help with BSL translation using audio samples etc, but to me that's a null question as this amazing technology of sign language and interpretation exists.So much of interpretation is not about the hands at all, but the facial gestures, body language, the context of a comment, the personality of the signer and the interpreter.
I think for that reason I tried to avoid interfering with the language and its meaning too much.Rather I think the approach we took was more … a sea of signs of different forms and medias and languages that could freely resonate with one another.(Luck, Personal correspondence with the author, 2023) Luck and I couldn't have derived this sea of gestural signs on our own.In Whatever Weighs You Down, as with many of my collaborations around new digital instruments, Luck and I originally aspired to push the limits of the affordances of our chosen digital instruments, and to explore how I as a performer can embody the knowledge of the instrument's abilities and limits.But Minamimura allowed us to see these gloves entirely differently-as a medium through which gestural signs and meaning can transform collaborative performance.The functionality of the gloves and their technical and musical affordances are subservient to a gestural grammar that could only have come from Minamimura, even though BSL is absent from the final glove movements.
My role in developing this piece was a parallel form of translation: from gesture into sound, through the MiMU gloves.Indeed, McLuhan's (1964, 9) definition of translation as its own type of technology (and language as a type of music) finds a different relevance here when related to the mapping of gesture to sound in a digital instrument, chiming with Lachemann's concept of 'building an instrument' (as explored in Kanga 2017).This mapping of collaboratively-generated gesture to sound, and the live performance of this translation is a fundamentally different task for an instrumentalist in comparison to performing on a conventional instrument (such as the piano part in this work).As Thor Magnusson (2019, 68) has pointed out, all instruments have agency, but new digital instruments have a greater variety of interactive options-their agency is multiplied, with many different types of opportunities and resistances, in comparison to a conventional acoustic instrument.The role of mapping to create a work-specific instrument is the crucial step of translation that differentiates the performer of digital instruments, from a performer working only with conventional instruments.
There was also an act of translation within the collaborative process in making the sounds and functionality of the gloves clear to Minamimura, who could then adjust and refine the gestures she was composing.The audience became part of this collaboration during the performance, with Minamimura watching their reactions to assist with her understanding of how the gloves were controlling sound, as she explained, 'when watching the gloves section, my main feelings are around reflecting on the hearing audience's responses to them' (Minamimura, correspondence with the author, 2023).
When working with new digital instruments, the performer's role in both creating and calibrating this new, work-specific instrument is a crucial bridge between the vast options and potential posed by the technology and the specific vision of the composer.It is both a technical and creative task-a link between the engineers who created the device, and the artists who desire a particular realisation of it as an instrument.But the building of the 'instrument' was also collaboratively distributed and integrated.The iterative translations of media, practices and creative agencies is entangled, in a network combining all three artists, our expertise and range of practice, and the technology.
Finally, Luck's role in the work was not only that of a composer but also that of a filmmaker, dramaturg and re-translator.These layers of translation (sound to movement to sound) were overlapped and stacked with the other musical, gestural and media content to point to, but never literally explain, the central ideas of the piece.Luck's management of these layers allows the piece to be not just about translation, or about the capabilities of the technology, but about the work's broader themes-a progression from descent to lifting, from weakness to strength, and fundamentally, from miscommunication to communication.
Whatever Weighs You Down demonstrates how the distribution of creative practice among diverse artists leads not just to new innovative art, but to new ways of using digital instruments.The development of a gestural approach to the gloves that truly explored not just the ergotic and epistemic, but also semiotic gestures that Cadoz described, was possible through Minamimura's expertise in using gesture to communicate and re-interpret music, through Luck's craft in structuring these gestures into an extended piece, layered with the piano part, two videos, electronics and theatrical elements, to create a rich counterpoint of communication, and my technical expertise in realising this instrument through the mapping process, to turn an interface into a work-specific instrument.The case demonstrates the potential contributions disabled artists might make to the development and innovative uses of digital instruments, given they can bring specific expertise and a highly developed practice that can be applied to the instrument's use.Diverse collaborations may require additional support, but they also widen the knowledge base for music-technology interaction, opening up new approaches to the creative interplay between the body, gesture and digital instruments.
Notes Or the strongest man in the world Or 2000 quid in beef chuck Or the down-force of your sleep paralysis ghoul.(Luck 2022)

Figure 1 .
Figure 1.Excerpt from Glover patch for Whatever Weighs You Down.

Figure 2 .
Figure 2. Workshop with Chisato Minamimura and Neil Luck, 8 January 2022, 'listening' to the piano with hands and feet.

Figure 3 .
Figure 3. Workshop with Chisato Minamimura and Neil Luck, 15 December 2021, creating parallel gestures between the dance/movement gestures and the piano gestures.

Figure 4 .
Figure 4. Workshop with Chisato Minamimura and Neil Luck, 8 January 2022, developing the gestural interpretation of the 'song'.

Figure 7 .
Figure 7. Score excerpt from Whatever Weighs You Down by Neil Luck and Zubin Kanga, showing hand positions and corresponding effects.

Figure 8 .
Figure 8. Score excerpt from Whatever Weighs You Down by Neil Luck and Zubin Kanga, showing mapping of movements, and sequence of movements for the 'conversation' passacaglia.

Figure 9 .
Figure 9. Glover patch for Whatever Weighs You Down by Neil Luck and Zubin Kanga.

Figure 10 .
Figure 10.Performance photo of Whatever Weighs You Down at Café OTO, 13 October 2022.Photo by Sisi Burn.