EXPLORING AND UNDERSTANDING PATTERNS OF EMOTIONS IN THE OVERALL SOUND ENERGY OF THE HUMAN VOICE IN A NATURAL ENVIRONMENT WITH AND THROUGH AN INTERACTIVE INSTALLATION AND VR IMMERSIVE VIDEOS

Abstract

In today’s era we can say that human reality is often reliant on or intersected with computers. The need of improved communication with devices in a natural way has arisen and led to development of interactive systems. The new channel for detecting emotions presented in this thesis is not so much important for humans as it is for computers that still don’t have human capabilities of recognising emotions. The vision that drives this research project is of an interactive kind of art through which it is possible to implement and enhance our knowledge of emotions in relation to sound, assisted by new technologies. My research concerns the relation between sound and emotions. This research was carried out within two art installations. Research goals • Discover and outline the patterns in the overall sound energy of the human voice, set in a natural environment, that reflect one or more basic emotion (I.e. anger, happiness, sadness); • Use artistic instruments and environment (interactive installation) to carry out this research; and • Represent the collected data and results of the research using virtual reality (VR ).

My original contribution to knowledge lies, on the one hand, within the function that both Sounding Out and Seams have on a person’s ability to gain a better understanding of how emotions reflect on sound energy and how sound energy analysis makes the computer able to recognise what we are feeling without understanding the contents we are expressing. On the other hand, this thesis forms a valuable tool of analysis, as well as the installations that I created: Sounding Out is an effective setting for the live detection of emotions, it allows the use of normal persons instead of trained ones. The innovative contribution to the research in the field is making the process of analysis faster and the recognition of emotions in a real-life setting is possible. The installation Seams is innovative in the way it is used to present academic research data. Seams is original in its intentions as it aims to make data more accessible and intuitive and particularly disseminate the study not only to researchers but to a broader audience.