Controlling the perceived material in an impact sound synthesizer
Authors: Aramaki M., Besson M., Kronland-Martinet R., Ystad S.
Publication Date: September 2011
Journal: IEEE Transactions on Audio, Speech, and Language Processing (vol. 19(2), pp. 301–314, 2011)
Tags: Brain Imaging, Solid Sounds, Sound Design, Timbre
Abstract
In this study, we focused on the identification of the perceptual properties of impacted materials to provide an intuitive control of an impact sound synthesizer. To investigate such properties, impact sounds from everyday life objects, made of different materials (Wood, Metal and Glass), were recorded and analyzed. These sounds were synthesized using an analysis-synthesis technique and tuned to the same chroma. Sound continua were created to simulate progressive transitions between materials. Sounds from these continua were then used in a categorization experiment to determine sound categories representative of each material (called typical sounds). We also examined changes in electrical brain activity (using Event Related Potentials -ERPs- method) associated with the categorization of these typical sounds. Moreover, acoustic analysis was conducted to investigate the relevance of acoustic descriptors known to be relevant for both timbre perception and material identification. Both acoustic and electrophysiological data confirmed the importance of damping and highlighted the relevance of spectral content for material perception. Based on these findings, controls for damping and spectral shaping were tested in synthesis applications. A global control strategy, with a three-layer architecture, was proposed for the synthesizer allowing the user to intuitively navigate in a “Material space” and defining impact sounds directly from the material label.
Stimuli
Analysis-synthesis and tuning of natural sounds:
Impact sounds from everyday life objects made of Wood, Metal and Glass materials were recorded, analyzed, synthesized and tuned to the same chroma.
WOOD:
METAL:
GLASS:
Sound continua:
Then, we created sound continua that simulate progressive transitions between two sounds of different materials by mixing the spectra and by morphing the damping.
An example of such a continuum between a wood and a metal sound :
The whole set of sounds used for the categorization experiment is available below.
The transition should be read from left to right and from top to bottom.
1- Transition between wood and metal
2- Transition between wood and glass
3- Transition between glass and metal
Importance of the spectral centroid (CSG ; related to the brighness) and the spectral bandwidth (SB) in sound categorization
Both CSG and SB have shown to be of little influence on the material categorization. This can be illustrated by the following examples:
Sounds with identical CSG obtained by impacting objets of different materials:
Sounds with identical SB obtained by impacting objets of different materials:
Influence of damping and spectral complexity on material perception
The perception of materials mainly correlates with the damping of the spectral components (sound examples can be heard here). Nevertheless, damping cannot explain by itself the material categorization.
We constructed 9 stimuli by independently combining the spectra of typical sounds (of Wood, Metal or Glass) with the damping characteristics of these sounds.
These stimuli correspond to the 9 possible combinations between the 3 spectra and the 3 damping law characteristics of each material category.
| Wood damping | Metal damping | Glass damping | |
|---|---|---|---|
| Wood spectrum |
|
|
|
| Metal spectrum |
|
|
|
| Glass spectrum |
|
|
|
Participants were asked to categorize these 9 sounds as Wood, Metal or Glass.
Results confirmed that damping is not sufficient to characterize the material categories and that both damping and the spectral content of sounds are relevant cues for sound categorization.
Control of the perceived material in a sound synthesizer: “The Material Space”
Control of sound dynamics (damping)
For impact sounds, sound dynamics is exclusively governed by damping.
A control of damping is proposed through the control of 2 parameters characterizing the frequency-dependent damping law.
The first parameter reflects global damping. The second one reflects the frequency-relative damping, i.e., difference between high frequency component damping and low frequency component damping.
The manipulation of these 2 parameters gives a refined control of the sound dynamics, allowing the synthesis of a wide variety of realistic sounds.
Control of spectral shaping
Two strategies are proposed to control the spectral shaping:
– spectral dilation. This control is provided by using 2 parameters. The first parameter reflects global shaping. The second one reflects the relative shaping.
– amplitude and frequency modulations. This control is provided by acting on modulation index I and the value of the modulating frequency.
The spectral shaping control can also be applied locally on a specific frequency range of the original spectrum by using a window function W.
Control strategy based on damping and spectral dilation:
Sounds corresponding to the numbered position illustrate the navigation in the Material space:
1- Transition between Wood and Metal
along the circle : W 20 19 18 17 16 15 14 13 12 11 M
straight : W 40 39 38 37 36 M
through the center : W 51 50 C 48 49 M
2- Transition between Wood and Glass
along the circle : W 21 22 23 24 25 26 27 28 29 30 G
straight : W 41 42 43 44 45 G
through the center : W 51 50 C 46 47 G
3- Transition between Glass and Metal
along the circle : G 1 2 3 4 5 6 7 8 9 10 M
straight : G 31 32 33 34 35 M
through the center : G 47 46 C 48 49 M
Control strategy based on damping and amplitude modulation:
Transition between Glass and Metal:
Sound effects: