Spatial sound and emotions: A literature survey on the relationship between spatially rendered audio and listeners’ affective responses


  • Antonina Stefanowska Faculty of Computer Science, Białystok University of Technology
  • Sławomir K. Zieliński Faculty of Computer Science, Białystok University of Technology


With the development of the entertainment industry, the need for immersive and emotionally impactful sound design has emerged. Utilization of spatial sound is potentially the next step to improve the audio experiences for listeners in terms of their emotional engagement. Hence, the relationship between spatial audio characteristics and emotional responses of the listeners has been the main focus of several recent studies. This paper provides a systematic overview of the above reports, including the analysis of commonly utilized methodology and technology. The survey was undertaken using four literature repositories, namely, Google Scholar, Scopus, IEEE Xplore, and AES E-Library. The overviewed papers were selected according to the empirical validity and quality of the reported studies. According to the survey outcomes, there is growing evidence of a positive influence of the selected spatial audio characteristics on the listeners’ affective responses. However, more data is required to build reliable, universal, and useful models explaining the above relationship. Furthermore, the two research trends on this topic were identified. Namely, the studies undertaken so far can be classified as either technology-oriented or technology-agnostic, depending on the research questions or experimental factors examined. Prospective future research directions regarding this topic are identified and discussed. They include better utilization of scene-based paradigms, affective computing techniques, and exploring the emotional effects of dynamic changes in spatial audio scenes.


B. Wu, A. Horner, and C. Lee “The Correspondence of Music Emotion

and Timbre in Sustained Musical Instrument Sounds,” Journal of the

Audio Engineering Society, vol. 62, no. 10, pp. 663-675, 2014.

C. Chau, B. Wu, and A. Horner, “The Emotional Characteristics and

Timbre of Nonsustaining Instrument Sounds,” Journal of the Audio

Engineering Society, vol. 63, no. 4, pp. 228-244, 2015.

J. Guo, J. Liu, Z. Li, J. Zhu, and W. Jiang, „A Study on the Relationship

Between Timbre Perception Features and Emotion in Musical Sounds,” in

International Conference on Culture-oriented Science & Technology

(ICCST), Beijing, pp. 22-27, 2020.

C. Chau, R. Mo, and A. Horner, “The Emotional Characteristics of Piano

Sounds with Different Pitch and Dynamics,” Journal of Audio

Engineering Society, vol. 64, no. 11, pp. 918-932, 2016.

C. Chau, S. J. M. Gilburt, R. Mo, and A. Horner, “The Emotional

Characteristics of Bowed String Instruments with Different Pitch and

Dynamics,” Journal of Audio Engineering Society, vol. 65, no. 7/8, pp.

-588, 2017.

R. Mo, B. Wu, and A. Horner, “The Effects of Reverberation on the

Emotional Characteristics of Musical Instruments,” Journal of Audio

Engineering Society, vol. 63, no. 12, pp. 966-979, 2015.

R. Mo, R. H. Y. So, and A. Horner, “An Investigation into How

Reverberation Effects the Space of Instrument Emotional Characteristics,”

Journal of Audio Engineering Society, vol. 64, no. 12, pp. 988-1002, 2016.

R. Mo, G. L. Choi, C. Lee, and A. Horner, “The Effects of MP3

Compression on Perceived Emotional Characteristics in Musical

Instruments,” Journal of Audio Engineering Society, vol. 64, no. 11, pp.

-867, 2016.

Y. Hong, C. Chau, and A. Horner, “An Analysis of Low-Arousal Piano

Music Ratings to Uncover What Makes Calm and Sad Music So Difficult

to Distinguish in Music Emotion Recognition,” Journal of Audio

Engineering Society, vol. 65, no. 4, pp. 304-320, 2017.

W. L. Sin, X. Ma, and A. Horner, “The emotional characteristics of rain

sound effects,” in 2018 International Computer Music Conference

(ICMC), Daegu, pp. 344-349, 2018.

W. L. Sin, B. Y. Chang, X. Ma, and A. Horner, “The Acoustic Features

and Their Relationship to the Emotional Characteristics of Rain Sound

Effects,” in 45th International Computer Music Conference (ICMC) and

International Computer Music Conference New York City Electroacoustic

Music Festival (NYCEMF), New York, pp. 84-89, 2019.

C. Gafni and R. Tsur, “Some experimental evidence for sound-emotion

interaction,” Scientific Study of Literature, vol. 9, no. 1, pp. 53-71, 2019.

T. A. Alam and N. Dibben, “A Comparison of Presence and Emotion

Between Immersive Virtual Reality and Desktop Displays for Musical

Multimedia,” in Future Directions of Music Cognition, Virtual, pp. 97-

, 2021.

A. Algargoosh, B. Soleimani, S. O’Modhrain, and M. Navvab, “The

impact of the acoustic environment on human emotion and experience: A

case study of worship spaces,” Building Acoustics, vol. 29, no. 1, pp. 85-

, 2021.

F. Cuadrado, I. Lopez-Cobo, T. Mateos-Blanco, and A. Tajadura-Jiménez,

“Arousing the Sound: A Field Study on the Emotional Impact on Children

of Arousing Sound Design and 3D Audio Spatialization in an Audio

Story,” Frontiers in Psychology, vol. 11, pp. 737, 2020.

K. Drossos, A. Floros, A. Giannakoulopoulos, and N. Kanellopoulos,

“Investigating the Impact of Sound Angular Position on the Listener

Affective State,” IEEE Transactions on Affective Computing, vol. 6, no.

, pp. 27-42, 2015.

I. Ekman, and R. Kajastila, “Localization Cues Affect Emotional

Judgments – Results from a User Study on Scary Sound,” in 35th Audio

Engineering Society International Conference: Audio for Games, London,

pp. 166-171, 2009.

E. D. Filippi, T. Schmele, A. Nandi, A. G. Torres, and A. Pereda-Baños,

“Emotional Impact of Source Localization in Music Using Machine

Learning and EEG: a proof-of-concept study,” TechRxiv, 2022.

J. Gong, Y. Shi, J. Wang, D. Shi, and Y. Xu, “Escape from the Dark

Jungle: A 3D Audio Game for Emotion Regulation,” in Virtual,

Augmented and Mixed Reality: Applications in Health, Cultural Heritage,

and Industry (VAMR 2018), Las Vegas, pp. 57-76, 2018,

E. Hahn, “Musical Emotions Evoked by 3D Audio,” in Audio Engineering

Society International Conference on Spatial Reproduction – Aesthetics

and Science, Tokyo, 2018.


Y. Hyodo, C. Sugai, J. Suzuki, M. Takahashi, M. Koizumi, A. Tomura, Y.

Mitsufuji, and Y. Komoriya, “Psychophysiological Effect of Immersive

Spatial Audio Experience Enhanced Using Sound Field Synthesis,” in 9th

International Conference on Affective Computing and Intelligent

Interaction (ACII), Nara, 2021.

G. Kailas and N. Tiwari, “An Empirical Measurement Tool for Overall

Listening Experience of Immersive Audio,” in IEEE International

Conference on Consumer Electronics (ICCE), Las Vegas, 2021.

P. Larsson and D. Västfjäll, “Emotional and behavioural responses to

auditory interfaces in commercial vehicles,” International Journal of

Vehicle Noise and Vibration, vol. 9, no. 1-2, pp. 75-95, 2013.

P. Larsson, A. Opperud, K. Fredriksson, and D. Västfjäll, “Emotional and

Behavioral Response to Auditory Icons and Earcons in Driver-Vehicle

Interfaces,” in 21st International Technical Conference on the Enhanced

Safety of Vehicles (ESV), Stuttgart, 2009.

S. Lepa, S. Weinzierl, H.-J. Maempel, and E. Ungeheuer, “Emotional

Impact of Different Forms of Spatialization in Everyday Mediatized Music

Listening: Placebo or Technology Effects?” in 136th Audio Engineering

Society Convention, Berlin, 2014.


J. Li, L. Maffei, A. Pascale, and M. Masullo, “Effects of the Spatialisation

of Water-Sounds Sequences on the Perception of Traffic Noise,”

Vibrations in Physical Systems, vol. 33, no. 1, 2022.

G. Moiragias and J. Mourjopoulos, “A listener preference model for

spatial sound reproduction, incorporating affective response,” PLoS One,

vol. 18, no. 6, 2023. https:/

S. Oode and A. Ando, “Estimation of Kandoh Degree with Emphasis on

Spatial Sound Impressions,” in 13th ACIS International Conference on

Software Engineering, Artificial Intelligence, Networking and

Parallel/Distributed Computing, Kyoto, 2012.

Y. Ooishi, M. Kobayashi, M. Kashino, and K. Ueno, “Presence of Three-

Dimensional Sound Field Facilitates Listeners’ Mood, Felt Emotion, and

Respiration Rate When Listening to Music,” Frontiers in Psychology, vol.

, pp. 650777, 2021.

J. Pätynen, and T. Lokki, “Concert halls with strong and lateral sound

increase the emotional impact of orchestra music,” The Journal of the

Acoustical Society of America, vol. 139, no. 3, pp. 1214-24, 2016.

A. P. Pinheiro, D. Lima, P. B. Albuquerque, A. Anikin, and C. F. Lima,

“Spatial location and emotion modulate voice perception,” Cognition and

Emotion, vol. 33, no. 8, pp. 1577-1586, 2019.

J. Ramalho and T. Chambel, “Immersive 360º Mobile Video with an

Emotional Perspective,” in ACM International workshop on Immersive

media experiences (ImmersiveMe ’13), Barcelona, pp. 35-40, 2013.

M. Shin, S. W. Song, S. J. Kim, and F. Biocca, “The effects of 3D sound

in a 360-degree live concert video on social presence, parasocial

interaction, enjoyment, and intent of financial supportive action,”

International Journal of Human-Computer Studies, vol. 126, pp. 81-93,

A. Tajadura-Jiménez, P. Larsson, A. Väljamäe, D. Västfjäll, and M.

Kleiner, “When room size matters: acoustic influences on emotional

responses to sounds,” Emotion, vol. 10, no. 3, pp. 416-422, 2010.

D. Västfjäll, “The Subjective Sense of Presence, Emotion Recognition,

and Experienced Emotions in Auditory Virtual Environments,”

CyberPsychology & Behavior, vol. 6, no. 2, pp. 181-188, 2003.

R. Warp, M. Zhu, I. Kiprijanovska, J. Wiesler, S. Stafford, and I.

Mavridou, “Moved By Sound: How head-tracked spatial audio affects

autonomic emotional state and immersion-driven auditory orienting

response in VR Environments,” in 152nd Audio Engineering Society

Convention, The Hague, 2022.


R. Warp, M. Zhu, I. Kiprijanovska, J. Wiesler, S. Stafford, and I.

Mavridou, “Validating the effects of immersion and spatial audio using

novel continuous biometric sensor measures for Virtual Reality,” in IEEE

International Symposium on Mixed and Augmented Reality Adjunct

(ISMAR-Adjunct), Singapore, 2022.

Y. Wycisk, K. Sander, R. Kopiez, F. Platz, S. Preihs, and J. Peissig,

“Development of the Immersive Music Experience Inventory (IMEI),”

Frontiers in Psychology, vol. 13, pp. 951161, 2022.

D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, The PRISMA Group,

“Preferred Reporting Items for Systematic Reviews and Meta-Analyses:

The PRISMA Statement,” PLoS Medicine, vol. 6, no. 7, pp. e1000097,

ITU-R. BS. 775-4 Recommendation. “Multichannel stereophonic sound

system with and without accompanying picture,” International

Telecommunication Union, Geneva, 2022.

F. Rumsey, “Spatial Audio,” 1st Edition, Routledge, London, 2001.

J. Blauert ed., “The Technology of Binaural Listening,” Springer, Berlin,

J. Paterson and H. Lee ed., “3D Audio,” Routledge, New York, 2021.

J. A. Russell, “Affective space is bipolar,” Journal of Personality and

Social Psychology, vol. 37, no. 3, pp. 345-356, 1979.

A. Mehrabian, “Relations among personality scales of aggression,

violence, and empathy: Validational evidence bearing on the Risk of

Eruptive Violence Scale,” Aggressive Behavior, vol. 23, no. 6, pp. 433-

, 1997.


P. Ekman, “An argument for basic emotions,” Cognition and Emotion, vol.

, no. 3-4, pp. 169-200, (1992). doi:10.1080/02699939208411068

R. Plutchik, “A psychoevolutionary theory of emotions,” Social Science

Information, vol. 21, no. 4-5, pp. 529-553, (1982).


F. Rumsey, “Spatial Quality Evaluation for Reproduced Sound:

Terminology, Meaning, and a Scene-Based Paradigm,” Journal of Audio

Engineering Society, vol. 50, no. 9, pp. 651-666, 2002.

S. K. Zieliński, F. Rumsey, and S. Bech, “Effects of Down-Mix

Algorithms on Quality of Surround Sound,” Journal of Audio Engineering

Society, vol. 51, no. 9, pp. 780-798, 2003.


S. K. Zieliński, F. Rumsey, and S. Bech, “Effects of Bandwidth Limitation

on Audio Quality in Consumer Multichannel Audiovisual Delivery

Systems,” Journal of Audio Engineering Society, vol. 51, no. 6, pp. 475

, 2003.

M. M. Bradley, and P. J. Lang, “Measuring emotion: The self-assessment

manikin and the semantic differential,” Journal of Behavior Therapy and

Experimental Psychiatry, vol. 25, no. 1, pp. 49-59, 1994.

L. F. Barrett and J. A. Russell, “Independence and Bipolarity in the

Structure of Current Affect,” Journal of Personality and Social

Psychology, vol. 74, no. 4, pp. 967-984, 1998.

A. Betella and P. F. M. J. Verschure, “The Affective Slider: A Digital Self-

Assessment Scale for the Measurement of Human Emotions,” PLoS One,

vol 11, no. 2, pp. e0148037, 2016.

Y. Wang, W. Song, W. Tao, A. Liotta, D. Yang, X. Li, S. Gao, Y. Sun, W.

Ge, W. Zhang, and W. Zhang, “A systemic review on affective computing:

emotion models, databases, and recent advances”, Information Fusion,

vol. 83-84, pp. 19-52, 2022.

F. L. Wightman & D. J. Kistler, “Resolution of front-back ambiguity in

spatial hearing by listener and source movement”, The Journal of the

Acoustical Society of America, vol. 105, no. 5, pp. 2841-2853, 1999.

A. J. Berkhout, D. de Vries, and P. Vogel, “Acoustic control by wave field

synthesis,” The Journal of the Acoustical Society of America, vol. 93, no.

, pp. 2764-2778, 1993.

Y. Mitsufuji, A. Tomura, and K. Ohkuri, “Creating a highly-realistic

“Acoustic Vessel Odyssey” using Sound Field Synthesis with 576

Loudspeakers,” in Audio Engineering Society International Conference on

Spatial Reproduction – Aesthetics and Science, Tokyo, 2018.

Additional Files