Perceptual Characterization of Motion Evoked by Sounds for Synthesis Control Purposes


Authors: Merer A., Aramaki M., Ystad S., Kronland-Martinet R.
Publication Date: February 2013
Journal: ACM Transactions on Applied Perception (vol. 10(1), pp. 1-24, 2013)

Tags: , ,

Abstract

This article addresses the question of synthesis and control of sound attributes from a perceptual point of view. We focused on an attribute related to the general concept of motion evoked by sounds. To investigate this concept, we tested 40 monophonic abstract sounds on listeners via a questionnaire and drawings, using a parametrized custom interface. This original procedure, which was defined with synthesis and control perspectives in mind, provides an alternative means of determining intuitive control parameters for synthesizing sounds evoking motion. Results showed that three main shape categories (linear, with regular oscillations, and with circular oscillations) and three types of direction (rising, descending, and horizontal) were distinguished by the listeners. In addition, the subjects were able to perceive the low-frequency oscillations (below 8 Hz) quite accurately. Three size categories (small, medium, and large) and three levels of randomness (none, low amplitude irregularities, and high amplitude irregularities) and speed (constant speed and speeds showing medium and large variations) were also observed in our analyses of the participants’ drawings. We further performed a perceptual test to confirm the relevance of the contribution of some variables with synthesized sounds combined with visual trajectories. Based on these results, a general typology of evoked motion was drawn up and an intuitive control strategy was designed, based on a symbolic representation of continuous trajectories (provided by devices such as motion capture systems, pen tablets, etc.). These generic tools could be used in a wide range of applications such as sound design, virtual reality, sonification, and music.


Graphical user interface used in the test:

Examples of the drawing possibilities of the interface:

Download Standalone Win / Mac

The original version in French is also available: Mac (Windows version not available)

Sounds and time frequency representation

Download all sounds

Download all Gabor Transform

Synthesis tool

InterfaceSynth

Download Standalone  Mac
Watch video examples made with the synthesizer:

Graphical User Interface used for the validation test

Validation

Download Max/MSP patch: english / français

(need Max/Msp Runtime version 5.1.4 or higher available here)

Watch video examples of sound/video associations: