Dear all,
Following a mail exchange with Giorgio and Eric, I come to you for your
comments or advises or ideas on the following point. We are developing a
sound FX generator based on MPEG4-SA. This works is partially carried out in
an European project called SoNG (for portalS of Next Generation). The leader
of the project is France Telecom and more precisely Olivier Avaro who is
also the chairman of MPEG4/7/21 System groups.
("we" means myself and Guillaume Fayemendy
(guillaume.fayemendy@cnet.francetelecom.fr
<mailto:guillaume.fayemendy@cnet.francetelecom.fr>
<mailto:guillaume.fayemendy@cnet.francetelecom.fr
<mailto:guillaume.fayemendy@cnet.francetelecom.fr> > ) who is working as a
PHD student on this subject).
The idea behind this work is to offer to the designers of MM applications
(web pages, 3D scenes, virtual world, etc..) the possibility to enrich their
applications with different kind of sounds, and this at low cost in terms of
storage or transmission since this will be based on SA concepts.
Some examples :
* audio metaphors for navigation guidance (alerts in case of erroneous
behavior, sounds of opened and/or closed doors, footsteps, etc.)
* musical signatures in waiting phases for instance.
* ambient sounds, e.g. rustling leaves, crowd noise, the sea, the wind
We will mainly focus on the SA (.mp4) generation side, i.e. we plan to use
existing SA decoders for the demos, not to develop any of them. Today we use
SFRONT which is almost real-time and we are very much looking in direction
of SAINT for "real" realtime.
Concerning the generation part, the following points of
investigation have
been identified so far
* short (briefs) and monophonic sounds
* high semantic value (i.e. bells, doors, claps, footsteps, ...).
Identification of "good sounds" for MM applications is an important
part of this work.
* few synthesis methods : granular, FM, ....(the list is not yet fixed
but should cover 90% of our needs)
* presets parameters to generate obvious sounds
* high levels sound descriptors (similar to the perceptual Vs physical
room effects in MPEG4). Such as : tapped, whistled, plucked, brilliance,
grain, noisiness, "harmonicity", ...To be used to make the GUI more
intuitive.
* objective analysis of natural sounds to derive low level descriptors
* mapping of low level descriptors to high level descriptors
* coupling of low parameters with the synthesis methods to get
synthetic sounds as perceptually close as possible to given natural sounds
* coupling of high parameters with the synthesis methods to get
synthetic sounds as perceptually close as possible to desired sounds.
Eric thought that
" From the list of things you want, I think there is quite a
bit of
information in the synthesis world that you will be able to use
easily, and it will be interesting to move such work into MPEG-4 SA and have
it available that way.
I would suggest to post to saol-users and see if other people are
interested in helping you assemble such things.
Of course, some of these things are still research topics, for
example you mention perceptual matching of a desired sound to a set of
possible algorithms. This is a difficult problem and many people are
interested in it. There are quite some papers in the last 3-5 years on this
topic.
"
Could you react on that ? In particular about :
* relevant synthesis methods and availability (papers, codes, ....)
* relevant analysis methods and availability (papers, codes, ....)
* perceptual parameters (papers)
and anything you think is relevant to our work
Many thanks
Best Regards,
Jean-Bernard RAULT
France Telecom - CNET
Tel : +33 2 99 12 46 78
Fax : +33 2 99 12 40 98
jeanbernard.rault@cnet.francetelecom.fr
<mailto:jeanbernard.rault@cnet.francetelecom.fr>
<mailto:Jeanbernard.rault@cent.francetelecom.fr
<mailto:Jeanbernard.rault@cent.francetelecom.fr> >
This archive was generated by hypermail 2b29 : Mon Jan 28 2002 - 11:46:37 EST