Sonifying Drawings: characterization of perceptual attributes of sounds produced by human gestures
Authors: Thoret E., Aramaki M., Kronland-Martinet R., Velay J-L., Ystad S.
Publication Date: April 2012
Journal: Proceedings of Acoustics 2012 Conference (pp. 1089-1094, Nantes, France, 23-27 April 2012)
Tags: Sonification, Sound and Motion
Friction sounds produced by the pencil of a person who is drawing on a paper are audible and could bring information about his/her gestures. This study firstly focuses on the perceptual significance of the morphology of such sounds, and to what extent gestures could be retrieved by sounds. Sounds recorded during drawing sessions were used in association tests where subjects had to univocally associate friction sounds with different shapes. Results showed that subjects were able to associate sounds with correct shapes and that the auditory characterization of the shape depended on the velocity profile.\\ A sonification strategy was then proposed for human drawings using a friction sound synthesis model. Inspired by the work of Viviani et al. (1982) who exhibited a 2/3-power law relation between kinetics and shape curvature for visual perception, experiments were carried out with the synthesis model, where subjects were asked to adjust the power law exponent so that the most realistic sound was obtained. Results revealed an exponent close to 2/3 as previously found in vision and thus highlighted a similar power law in the auditory modality, providing an ecological way to determine velocity profiles from static shapes and to generate sounds coherent with human gestures.
Examples of synthesized friction sounds
Shapes and stimuli of Experiment 2
Stimuli of Experiment 3
Cercles and Ellipses are the same as in the Experiment 2.
In addition to Cercle and Ellipse, the two following shapes and associated sounds are considered:
Application to the Sonification of Shapes
Example of Sonification of the previous shape :