by Leonardo Chiariglione, Convenor, ISO/IEC JTC1/SC29, WG11, MPEG
When the Bell Labs engineer Harry Nyquist was building his “telephotography” machine in 1918 and performing his seminal work on sampling analogue signals in 1928, he probably could not have anticipated where his early investigation would lead.
First and foremost this was because, to go from the “Nyquist theorem” to the ubiquitous digital audio and video of today, there was a need for the electronic computer to be invented, specifically in today’s form as a personal computer, with a host of researchers devising smart ways to reduce the number of binary digits (bits) required to represent high-quality audio and video.
Secondly, it needed transistors and integrated circuits (IC) to be invented. And thirdly, yes, it needed the Moving Picture Experts Group (MPEG), because when it comes to deploying new forms of communication – audio and video in this case – standards are required.
Why did ISO and IEC engage in setting communication standards when there was already an international body, the ITU, in charge of them? Because MPEG does not deal with communication systems per se, it deals “only” with the abstract form in which audio and video information is efficiently represented with bits that can be utilized in different communication systems.
One of the most successful MPEG standards – though possibly least known to the general public – is MPEG-2, which is used throughout the world to send digital television programmes on VHF/UHF, telephone lines, cables, satellites, DVD and the Internet, for example. The information has a single form and it is always the same set of bits, but they are differently “wrapped” for carriage by different transmission systems.
MPEG was initially established in 1988 as an “Experts Group on Moving Pictures” within ISO/IEC JTC1/SC2, WG8, but three years later it became SC29/WG11. It was a timely decision because for more than 20 years there had been worldwide investment in finding algorithms capable of reducing (“compressing”) the huge volumes of bits per second (bit/s) generated by the bulk digitization of audio and video to much smaller values, but without significant quality impairment.
As an indication, say 1,41 Mbit/s (million bits per second) are required for stereo audio on a compact disc and 216 Mbit/s for digital television in the studio.
Until the late 1980s the conundrum was that the more efficient the compression algorithm – and therefore the more promising the exploitation potential – the more difficult its implementation in an IC. This, however, was the time when compression algorithm complexity and IC capability were converging. What was missing was a standard “Coded Representation of Audio and Video Information”, the original title of MPEG.
The creation of this standard is what MPEG set out to do. The first standard – MPEG-1 – is used in hundreds of millions of Video CD players. Billions of videos are stored on CDs around the world.
MP3, a special case of MPEG-1 Audio and now a household name, has changed the music experience and is being used in hundreds of millions of PCs and MP3 players. The number of MP3 files can probably be counted in the hundreds of billions.
The second standard – MPEG-2 – has already been mentioned above and the number of devices and items of content can again be measured in the hundreds of millions and billions, respectively with thousands of hours of content continuously generated every day. As a result an industry that did not exist before – the digital audio and video industry – was created.
Today ICs continue to play an important role, but the audio and video experience of many people comes more and more from the PC. MPEG did not miss the importance of implementing audio and video compression standards in software as opposed to IC hardware. MPEG-4 – in fact the third MPEG standard, despite its name – extended the scope of interest of the series exactly to that type of application domain.
One of the results of this effort is Advanced Audio Coding (AAC), an audio compression standard that outperforms MP3 by a factor of two. AAC is used by two of the best known Internet music services. Another result is MPEG-4 Visual that is widely used on the Web, is loaded in almost all digital cameras and supported by new mobile handsets. A third result is the MPEG File Format, a broadly used format for carrying audio and video files.
In more recent years MPEG has been engaged in a new type of standard – MPEG-7. Unlike its predecessors, MPEG-7 is not about compression but about “description” of what is in a video or audio signal.
In parallel, MPEG also has developed a suite of standards under the title MPEG-21 Multimedia Framework. Worthy of mention are Digital Item Declaration (DID, part 2) addressing the problem of representing multiple digital objects; Digital Item Identification (DII, part 3) addressing the problem of how to assign unique identifiers to the composite objects defined in part 2; Intellectual Property Management and Protection Components (part 4) addressing the issue of interoperability of Digital Rights Management (DRM) systems; Rights Expression Language (REL, part 5) specifying a computer-processable language capable of expressing rights to digital objects; and Rights Data Dictionary (RDD, part 6) defining the semantic of the terms used by part 5 and many more.
The latest entry in the MPEG family of standards is MPEG-A (A for Application). Unlike other MPEG standards conceived for use in a stand-alone fashion, the MPEG-A project aims at creating standards for systems that incorporate a suite of standards drawn from the MPEG “toolkit”. The first element of MPEG-A is Music Player Application Format, built with standard elements taken from MPEG-1, MPEG-4, MPEG-7 and MPEG-21.
In the audio and video compression field MPEG has recently released Advanced Video Coding (AVC), with better than twice the compression efficiency of MPEG-2 and which is planned to be deployed in a broad range of application domains. The new High Efficiency AAC provides a compression factor double that of AAC.
At the same time MPEG is already investigating or developing standards in such fields as:
scalable video coding, achieving the same compression as AVC but with scalability;
spatial audio coding, recreating full 5.1 multichannel audio with a handful of bits added to the stereo version;
scalable audio and speech coding, efficiently compressing both music and speech signals;
3D AV, representing a set of video signals from a number of video cameras shooting the same scene; and
multimedia middleware, which is a set of APIs to execute middleware functions.
Today MPEG continues to produce standards to satisfy the needs of its broad constituency as well as exploring several new opportunities.