INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO

ISO/IEC JTC1/SC29/WG11
MPEG96/N1419
Nov 1996/Maceio

Source: Audio Subgroup
Title: Report on the Formal Subjective Listening Tests of MPEG-2 NBC multichannel audio coding
Authors: David Kirby (BBC), Kaoru Watanabe (NHK)

Report on the Formal Subjective Listening Tests of MPEG-2 NBC multichannel audio coding

Preface

This document reports on the preparations for, and presents the results of, the subjective listening tests on the MPEG-2 (NBC) coding algorithm carried out in Kingswood Warren, UK, and Tokyo, Japan, between 16 September and 11 October 1996. The following additional contributions are also included:

• Selection Panel Report
• Instructions to listeners
• Test conditions and equipment
• Statistical procedures

Introduction -- Background

In March 1994, Deutsche Telekom and the BBC reported the results of formal listening tests on the MPEG-2 Backwards Compatible multichannel coding algorithms [1]. Eight codecs were evaluated at that time: six MPEG-2 Backwards Compatible (BC) implementations and two Non-Backwards Compatible (NBC) codecs. The results indicated that all of the codecs tested were not acceptable for high quality applications at the tested bitrates. It was also observed that the BC codecs did not perform as well as the NBC codecs at the same bitrate.

As a result of those findings, MPEG decided on two courses of action: firstly, to include, in the proposed MPEG-2 audio standard, additional features which would deliver better audio quality and, secondly, to initiate the development of a Non- Backwards Compatible coding technique. The first of these, together with general improvements to the codec implementations, has led to the improved performance of the MPEG-2 BC codecs, reported by a series of subjective tests [2,3,4].

The development of the MPEG-2 NBC coding technique has proceeded over the last two years and during that time has been subject to various stages of optimisation [5 to 11] based on a combination of technical developments and proving core experiments [12 to 15]. In the early stages, the developments were based on monophonic embodiments, whilst more recently both two-channel and multichannel implementations have been produced.

This programme of work has reached the stage where formal testing of the multichannel implementation is appropriate. Accordingly, at the July 1996 meeting of MPEG in Tampere, the BBC and NHK were jointly charged to conduct formal subjective tests aimed at quantifying the performance of MPEG-2 Non-Backwards Compatible audio codecs operating in a multichannel mode [16].

During September and October 1996, subjective testing was therefore carried out at the BBC Research and Development Department at Kingswood Warren, UK and at NHK Science and Technical Research Labs, Tokyo, Japan. This report describes, in detail, the various stages of these formal tests and presents the results obtained.