Expression, communication and meaning in music

Renee Timmers

Professor Renee Timmers

Renee Timmers is Professor in Psychology of Music and director of the Music, Mind, Machine research centre. She has degrees in musicology (MA) and psychology (PhD) and pursues interdisciplinary research applying psychological and computational research methods to investigate aspects of the perception, performance and experience of music. She is former co-editor of Empirical Musicology Review, associate editor of Psychomusicology: Music, Mind and Brain, and has served on the editorial board of several interdisciplinary journals including Frontiers in Performance Science, and Psychology of Music. She is Past-President of the European Society of the Cognitive Sciences of Music, in which capacity she organised the Cross-European Winter School on Musical Ability (3-5 Feb 2021) and the ICMPC16/ESCOM11 conference (28-31 July 2021).

Renee has published over 50 journal articles, she has (co-)edited three volumes (Together in Music: Coordination, expression, participation; Expressiveness in music performance: empirical approaches across styles and cultures; The Routledge companion to music cognition), three special journal issues, and written multiple chapter contributions to edited volumes, including to leading handbooks such as Foundations of music psychology. Her research interests concern the multimodal experience of music, relationships between emotion and cognition in musical experiences, embodied music cognition, music for health and wellbeing, and music perception with hearing impairment.


Examples of funded projects are listed below (since 2014). She is Co-I on the projects COVID-19 and Sheffield's cultural sector: planning for recovery and Designing music technology for older adults' wellbeing

Mapping music for health and wellbeing in Sheffield


Sheffield, EOI-QR funding


What is the status and range of current practices that use music to support mental, social and physical health in Sheffield? Awareness and evidence are growing of the value of music-making and listening for health and wellbeing in diverse contexts and across ages. As a research unit for music psychology and its applications in everyday life, we are aware of some of the initiatives and activities in Sheffield that use music to support health and wellbeing. However, this knowledge is patchy and a comprehensive map of existing activity is lacking. Moreover, little is known about working practices, support needs, resource reliance and the role policy and public funding plays in promoting or challenging activities. To address these gaps, a comprehensive survey is conducted of organisations and activities in Sheffield, interviews are held with key stakeholders, and a conference is organised that brings together practitioners, academics, and policymakers from Sheffield and surroundings to showcase and discuss the use of music for health and wellbeing. From the survey, a map is created that locates activities including an index of working practices. The map is accompanied by a discussion of approaches, challenges and implications for policy and governance. Promoting visibility of what is already happening and discussing what needs to be done and improved, the conference will offer networking opportunities, dialogue between diverse stakeholders, learning from examples, and critical reflection.

Link: Mapping music for wellbeing website

https://sites.google.com/sheffield.ac.uk/mappingmusicforsheffield/home


Neural multimodal integration underlying synchronization with a co-performer in music: influences of motor expertise and visual information

Collaboration with researchers in Sydney Australia, funded by Leverhulme Trust International Academic Fellowship.

Sensorimotor synchronization is a general skill that musicians have developed to the highest levels of performance, including synchronization in timing and articulation. This study investigated neurocognitive processes that enable such high levels of performance, specifically testing the relevance of 1) motor resonance and sharing high levels of motor expertise with the co-performer, and 2) the role of visual information in addition to auditory information. Musicians with varying levels of piano expertise (including non-pianists) performed on a single piano key with their right hand along with recordings of a pianist who performed simple melodies with the left hand, synchronizing timing and articulation. The prerecorded performances were presented as audio-only, audio-video, or audio-animation stimuli. Double pulse Transcranial Magnetic Stimulation (dTMS) was applied to test the contribution of the right dorsal premotor cortex (dPMC), an area implicated in motor resonance with observed (left-hand) actions, and the contribution of the right intraparietal sulcus (IPS), an area known for multisensory binding. Results showed effects of dTMS in the conditions that included visual information. IPS stimulation improved synchronization ability, although this effect was found to reverse for the video condition with higher levels of relevant motor expertise. dPMC stimulation improved or worsened synchronization ability. Level of relevant motor expertise was found to influence this direction in the video condition. These results indicate that high levels of relevant motor expertise are required to beneficially employ visual and motor information of a co-performer for sensorimotor synchronization, which may qualify the effects of dPMC and IPS involvement.

Link: http://eprints.whiterose.ac.uk/156399/

Expression and communication in ensemble performance

WRoCAH funded PhD network with the Universities of Leeds and York.

Research into music performance has always relied on and benefited from technical innovations. A major boost was given by the rise of MIDI technology allowing detailed recording of timing, dynamics and articulation in MIDI (Musical Instrument Digital Interface) instruments (generally keyboard). The increasing availability of audio analysis tools has opened possibilities to investigate performances from audio recordings paving the way for historical and contemporary databases of recordings to be analyzed with the constraint that different audio sources (e.g. different voices and instruments) cannot be separated. This reliance on technical innovations has however pigeon-holed performance analysis within two categories of academia: a computer-driven approach tackling large data sets using machine-learning techniques and a qualitative and small scale data analysis approach driven by performers and musicologists. This Network bridges the gap across these disciplines, using digital transformation to fully integrate the approaches from the arts and sciences to provide meaningful research outcomes for theory and practice. Moreover, building on recent innovations, it will push forward the investigation of expressive communication in ensemble performance countering the existing focus on either solo performance or the ensemble outcome as a whole.

Expressive communication refers to the coordination and synchronisation of gestures in movement and sound between musicians performing different parts in a unified manner. We are interested in the subconscious communicative processes between performers that go beyond what is written in the score, and in audience responses to these. Overarching questions will be: What is communicated through performance? What processes facilitate synchronization and coordination? How do experts differ from novices? How is ensemble coordination perceived or experienced? How may this differ between audience and performers? How can our findings benefit education and music technology? What applications may this have in other domains like dance, amateur music making, vocal rehabilitation? These are large questions that cannot be addressed through individual projects. However, a PhD network has allowed doctoral students to investigate educationally, psychologically and musically relevant questions related to ensemble performance without the danger that projects become technically intractable.

Links to WRoCAH network website: http://wrocahensemble.sites.sheffield.ac.uk/home

Links to edited volume Together in Music: Coordination, expression, participation

Families and Music: An interdisciplinary perspective on fostering family bonding

Mexico City, GCRF funded conference and workshop

The Families & Music conference was organised in order to bring together academic and non-academic audiences with an interest in the theme of families & music, in particular how music may be used to foster family bonding and reduce stress within family contexts. The conference was the first of its kind held at UNAM, Mexico City, and attracted a mixed audience consisting of academics and professionals working in and outside of music. A shared theme among the spoken presentations was the power of music to support individuals and groups. This ranged from support of individuals in difficult situations, strengthening of affective and social bonds between people, including among children and between parents and infants, to contributions to positive and healthy development and aging, including neurocognitive rehabilitation. A second set of research presentations concerned family structures and predicting factors affecting risk behaviour in young people. The first highlighted the great value placed on family across 16 investigated countries, as well as increasing vulnerabilities associated with changes in family structures. The second reported on factors that can predict displays or absence of risk behaviour in young people, highlighting implications for support that can be offered to young people in terms of resilience to peer pressure and engagement with positive activities. The conference highlighted the relevance of involving the family in music developments and interventions and of investigating music cognition from a family perspective.

Links: Mexico Conference Report

Mexico Conference Report.pdf

Cross-modal perception of music

British Academy funded International Mobility Network between Israel and UK

People involved: Renee Timmers, Zohar Eitan, Roni Granot and Nicola Dibben

Listening to music is far from a solely auditory activity. Rather, sonic characteristics are habitually understood in terms of phenomena in diverse sense modalities and conceptual domains. Motor, emotion, and visual areas of the brain are activated in response to music, and evidence is growing that these associations are involved in music perception, whether in perception of dance music, perception of motion and emotion in performance, or in subconscious mapping between auditory and visual modalities.

The network between researcher in Israel and the UK facilitated the exchange of expertise, the exploration of new research trajectories, and the training of postgraduate students. An international conference was organised on the multimodal experience of music (ICMEM), which led to the publication of an online Proceedings and a dedicated journal issue on the topic in Psychomusicology: Music, Mind and Brain. Work was further published in the edited volume ‘Music as shape’. This work is continued in research by Caroline Curwen and Shen Li, among others.

Links:

Special issue on Music as a multimodal experience

Proceedings of the International conference on the multimodal experience of music

Chapter in Music as shape

Attendees in the Families & Music Workshop, Mexico City, Mexico