INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO
ISO/IEC JTC1/SC29/WG11
MPEG96/N1420
Nov 1996/Maceio
Source: Audio Subgroup
Title: Overview of the Report on the Formal Subjective Listening Tests of
MPEG-2 NBC multichannel audio coding
Authors: David Kirby (BBC), Kaoru Watanabe (NHK)
| This web page contains only excerpts of the document. The complete document is available as PDF file. |
This document presents an overview of the preparations for, and the results of, the subjective listening tests on the MPEG-2 Non-Backwards Compatible (NBC) multichannel audio coding algorithm carried out by the BBC and NHK between 16 September and 11 October 1996. It is derived from the detailed report of the tests [4]. The tests evaluated the following multichannel codecs:
The test procedure and environments complied with ITU-R Recommendation BS-1116. Listener reliability and test procedure checks were included and a detailed statistical analysis of the results was performed.
The results showed good performance for all of the codecs. The MPEG-2 NBC codec at 320 kbit/s generally performed better than the other codecs and, although not quite transparent for a few test excerpts under these rigorous conditions, it passed the EBU criterion for “indistinguishable quality”. Overall, the performance of MPEG-2 NBC low complexity version at 320 kbit/s was, by a small margin, not quite as good as that of MPEG-2 NBC at 320 kbit/s.
It is worth emphasising that these tests were conducted according to the most rigorous of test methods. Comparisons to other test results using less rigorous methodologies should not be made.
In March 1994, Deutsche Telekom and the BBC reported the results of formal listening tests on the MPEG-2 Backwards Compatible multichannel coding algorithms [1]. Eight codecs were evaluated at that time: six MPEG-2 Backwards Compatible (BC) implementations and two Non-Backwards Compatible (NBC) codecs. The results indicated that none of the codecs tested was acceptable for high quality applications at the tested bitrates. It was also observed that the BC codecs did not perform as well as the NBC codecs at the same bitrate.
As a result of those findings, MPEG decided on two courses of action: firstly, to include, in the proposed MPEG-2 audio standard, additional features which would deliver better audio quality and, secondly, to initiate the development of a Non-Backwards Compatible coding technique. The first of these has led, in stages, to the improved performance of the MPEG-2 BC codecs, reported by earlier subjective tests [2].
The development of the MPEG-2 NBC coding technique has proceeded over the last two years and has reached the stage where formal testing of the multichannel implementation is appropriate. Accordingly, at the July 1996 MPEG meeting, the BBC and NHK were jointly charged to conduct formal subjective tests aimed at quantifying the performance of MPEG-2 Non-Backwards Compatible audio codecs operating in a multichannel mode [3].
During September and October 1996, subjective testing was therefore carried out at the BBC Research and Development Department at Kingswood Warren, UK and at NHK Science and Technical Research Labs, Tokyo, Japan.
This paper provides a summary of the full test report [4] to which the reader should refer for more detailed information and references to the MPEG-2 NBC development work.
Heiko Purnhagen 16-Feb-1998