A Knowledge-based, Data-driven Method for Action-sound Mapping
dc.contributor.author | Visi, F | |
dc.contributor.author | Caramiaux, B | |
dc.contributor.author | Miranda, Eduardo | |
dc.contributor.author | Mcloughlin, M | |
dc.date.accessioned | 2018-03-05T14:09:15Z | |
dc.date.available | 2018-03-05T14:09:15Z | |
dc.date.issued | 2017-05-15 | |
dc.identifier.uri | http://hdl.handle.net/10026.1/10946 | |
dc.description.abstract |
This paper presents a knowledge-based, data-driven method for using data describing action-sound couplings collected from a group of people to generate multiple complex mappings between the performance movements of a musician and sound synthesis. This is done by using a database of multimodal motion data collected from multiple subjects coupled with sound synthesis parameters. A series of sound stimuli is synthesised using the sound engine that will be used in performance. Multimodal motion data is collected by asking each participant to listen to each sound stimulus and move as if they were producing the sound using a musical instrument they are given. Multimodal data is recorded during each performance, and paired with the synthesis parameters used for generating the sound stimulus. The dataset created using this method is then used to build a topological representation of the performance movements of the subjects. This representation is then used to interactively generate training data for machine learning algorithms, and define mappings for real-time performance. To better illustrate each step of the procedure, we describe an implementation involving clarinet, motion capture, wearable sensor armbands, and waveguide synthesis. | |
dc.language.iso | en | |
dc.title | A Knowledge-based, Data-driven Method for Action-sound Mapping | |
dc.type | conference | |
plymouth.date-start | 2017-05-15 | |
plymouth.date-finish | 2017-05-19 | |
plymouth.conference-name | NIME 2017 : New Interfaces for Musical Expression | |
plymouth.journal | Proceedings of NIME 2017 : New Interfaces for Musical Expression | |
plymouth.organisational-group | /Plymouth | |
plymouth.organisational-group | /Plymouth/Faculty of Arts, Humanities and Business | |
plymouth.organisational-group | /Plymouth/REF 2021 Researchers by UoA | |
plymouth.organisational-group | /Plymouth/REF 2021 Researchers by UoA/UoA33 Music, Drama, Dance, Performing Arts, Film and Screen Studies | |
plymouth.organisational-group | /Plymouth/Users by role | |
plymouth.organisational-group | /Plymouth/Users by role/Academics | |
plymouth.organisational-group | /Plymouth/Users by role/Researchers in ResearchFish submission | |
dc.publisher.place | Copenhagen | |
dcterms.dateAccepted | 2017-03-28 | |
dc.rights.embargoperiod | Not known | |
rioxxterms.version | Accepted Manuscript | |
rioxxterms.licenseref.uri | http://www.rioxx.net/licenses/all-rights-reserved | |
rioxxterms.licenseref.startdate | 2017-05-15 | |
rioxxterms.type | Conference Paper/Proceeding/Abstract |