Streaming media formats (Designing Web Audio)

5.2. Streaming media formats

There are currently more than a dozen formats for streaming audio over the Web, from widely used formats, such as RealNetworks' RealAudio, streaming MP3, Macromedia's Flash and Director Shockwave, Microsoft's Windows Media, and Apple's QuickTime, to more recent entries that synchronize sounds with events on a web page, such as RealMedia G2 with SMIL and Beatnik's Rich Music Format (RMF). Also included are a host of downloadable formats, including Liquid Audio, MP3, MIDI, WAV, and AU.

While the high quality of MP3 has sent shockwaves through the recording industry, streaming formats like RealAudio remain the dominant audio technology on the Web right now. Indeed MP3 is being folded into multimedia streaming formats like QuickTime and Windows Media.

Throughout this book, we take an in-depth look at many of the more prevalent streaming formats. However, in this chapter, we will review all the streaming formats on the Web, including Windows Media and QuickTime, which are not featured in later chapters.

5.2.1. RealMedia and RealAudio

RealMedia is the most widely adopted streaming media format on the Web. Its popularity is due in large part to the fact that it was the first streaming technology on the market. But it's popular also because of RealNetworks' laser focus on ease of use, deployment of a wide palette of developer tools, continuous support for the latest multimedia technologies, and support for both Windows and Unix platforms. RealMedia is the format of choice for professionals who want advanced controls for serving, tracking, and managing large numbers of audio streams. RealNetworks has been a trailblazer in making advanced server features, which were once accessible only to those with advanced programming skills, available to the public.

And RealMedia is likely to attract more fans as web developers begin to use the RealSystem G2 and SMIL to stream synchronized multimedia presentations over the Web. G2's major advance is the ability to simultaneously stream multiple media types as separate files instead of as one RealMedia-encoded file. This makes updating multimedia content easier, since you can simply upload one element of a presentation instead of re-encoding the whole media file.

Perhaps the most powerful feature of RealSystem G2 is RealNetworks' server architecture. Broadcasting audio with a dedicated RealServer provides the following advantages over HTTP pseudo-streaming from a standard web server:

Bandwidth negotiation: Ensures that all users receive the appropriate encoded content for the best audio quality at their available bandwidth, from slower analog modems to faster cable or xDSL connections. RealSystem G2's new SureStream technology is even more efficient than bandwidth negotiation. SureStream can dynamically change data rates midstream to accommodate fluctuating bandwidth.
Robust RTSP transmission

While RealMedia's powerful server-side architecture supports and manages robust streaming to large audiences, this core strength results in limited interactivity. Like Windows Media and other server-side streaming technologies, RealMedia waits for a request from a listener's browser before it begins to stream media files. This helps RealMedia negotiate bandwidth congestion on the fly by sending an appropriate size stream that matches the listener's real bandwidth. But it also produces a significant time gap of a few seconds between the listener's request and the response from the server. This small time gap is inconsequential with long-playing video and audio files, but it prohibits the use of interactive sound effects such as button rollovers, sound transitions from one page to another, and loops that must respond instantaneously to a mouse click.

Thus, RealMedia is inappropriate for high impact presentations with interactive sound effects and loops. Despite significant advancements in RealSystem G2, RealAudio still trails Flash and Director Shockwave when it comes to smooth playback of high-impact interactive multimedia. High-powered interactive media requires a client-side solution such as Flash, Shockwave, or Beatnik.

5.2.2. Windows Media Technologies (Netshow)

Microsoft's Windows Media Technologies for NT/Windows 2000 includes a comprehensive suite of authoring tools and streaming services for delivering audio, video, animation, and other multimedia over the Internet. Windows Media comes with a complete set of tools for encoding and authoring streaming content including Windows Media T.A.G.Author, a utility for arranging media elements along a timeline. Windows Media presentations are played back with the Windows Media Player, which plays most local and streamed media file types including Advanced Streaming Format (Windows' native file format), MPEG, WAV, AVI, QuickTime, and RealAudio/RealVideo. Since Media Player is distributed with Windows, it has widespread distribution.

If you need a Windows NT 4.0-based solution, Windows Media Services offers several advantages:

The Windows Media Server comes free with unlimited streams with Windows NT Server 4.0 and later.
It allows for better playback over machines running Windows. To enable smooth multimedia playback over the Web and avoid the problematic issue of cumbersome plug-in downloads altogether, Microsoft is moving towards integrating Windows Media Player, along with Internet Explorer, directly into the Windows operating system.
Windows Media Server integrates with Microsoft Site Server to enable pay-per-view and pay-per-minute billing capabilities, usage analysis reporting, and personalized ad insertion.
Tools for tracking behavior are tightly integrated with the Windows NT Event Viewer and Performance Monitor, making it easy for seasoned NT administrators to manage the Windows Media Server.
For multimedia content developers, Microsoft provides helpful authoring tools. Creating a slide show of images with synchronized audio can be accomplished by using the Windows Media T.A.G. Author.

Compared to RealMedia, however, Windows Media has some serious drawbacks:

It runs only on Windows NT/2000. Many developers have reported problems with the stability of Windows NT for mission-critical applications such as 24-hour live broadcasts. This can be a show-stopper for those who demand the stability of Unix or Linux servers. In contrast, RealNetworks supports NT as well as Linux, FreeBSD, Solaris, and IRIX.
It does not support Macromedia Flash or the Synchronized Multimedia Integration Language (SMIL) standard, both of which are supported by RealNetworks.

There are also some key differences in the way Windows Media and RealMedia encode and deliver multimedia content. With RealMedia, you can create multimedia presentations by using the SMIL markup language to connect various media elements together. These media elements are encoded as separate files: RealAudio, RealVideo, RealPix, RealText, QuickTime, MPEG, and so on. The RealServer, much the same way a standard web page is served up and delivered, then streams the presentation as separate media files held together by SMIL.

"Since G2 developers are creating multimedia presentations rather than simply encoding audio or video streams, the format has a new level of complexity," says Leah Goldberg, G2 media producer for CMPnet. "However, web developers have long been familiar with the flexibility and convenience of this approach to media delivery. The challenge with G2," Goldberg claims, "is working out the timing in the component RealPix, RealText, and RealFlash files. Since the idea is to synchronize all the different media elements together, working out the sub-timing issues within each of the component files can be quite complex."

In contrast, Windows Media wraps all media elements into one Active Streaming File (ASF), Microsoft's proprietary streaming media format. According to Microsoft, with ASF any object can be placed into an ASF data stream, including audio and video, scripts, ActiveX controls, and HTML documents with T.A.G. Author. This approach, similar to Flash and Shockwave movies, provides less flexibility in terms of updating and serving content, but it offers more stable client-side playback of various media elements and tighter authoring controls. For more information about creating ASF content, visit the Microsoft web site. Microsoft provides free code to members of its Developer Network.

5.2.3. QuickTime

Apple Computer's QuickTime enables the delivery and playback of video, audio, animation, 3-D, and panoramic images for Macintosh and Windows. QuickTime is also the leading video production platform for both Windows and Macintosh. Most multimedia on computers begins with or involves QuickTime. Accordingly, the QuickTime technology is a natural for high-quality audio and video playback over the Web. Similar to Windows Media, QuickTime does not charge licensing fees for the number of simultaneous streams served. QuickTime can be streamed from the Mac OS X Server, the Darwin Streaming Media Server, and RealNetworks' RealServer 8.0.

The latest version, QuickTime 4, features many enhancements including:

Smaller "component" codec architecture so that the initial download is as low as 1.7 MB. Additional codecs are transparently downloaded in the background when required for a specific media element on a page.

Support for an increased number of formats, including MP3, Flash, MIDI, and almost every audio, video, animation, 3-D, and virtual reality format available.
Improved codecs.
True RTSP streaming when used in conjunction with the Mac OS X Server.

One of the keys to the success of the QuickTime technology and plug-in is that it can handle all types of media elements. For those of you trying to design for the greatest number of users and the least number of plug-ins, this can be a significant benefit.

In addition to playing MP3 content, QuickTime supports Timecode tracks as well as MIDI standards, including the Roland Sound Canvas and GS format extensions. QuickTime also supports key standards for web streaming, including HTTP, RTP, and RTSP. Plus, QuickTime supports every major file format for images, including JPEG, BMP, PICT, PNG, and GIF. QuickTime also features built-in support for digital video, including MiniDV, DVCPro, and DVCam camcorder formats, as well as support for AVI, AVR, MPEG-1, and OpenDML.

Finally, the newly designed interface is attractive and user friendly. In addition to the traditional controls you'd expect to find on a television -- like volume controls and pause and play buttons -- the QuickTime Player gives you enhanced controls for online movie playback. The QuickTime Player's LCD section includes a time display, a time slider that shows you the length of the file being played, and a chapter marker. You can switch chapters on the fly even at the beginning of a video stream.

Unlike Liquid Audio, MP3 is not a proprietary end-to-end music delivery system. This distinction is important since companies concerned about copyright protection and secure delivery may decide to use the Liquid Music System for music distribution instead of merely posting MP3 files on a web page. On the other hand, the fact that MP3 is an accessible standard means it has the advantage of widespread industry support and compatibility with many applications and media players, including RealPlayer G2, Beatnik, Shockwave, QuickTime 4, and Windows Media.

5.2.7. Liquid Audio

Liquid Audio provides a complete end-to-end solution for secure music delivery over the Internet. Unlike Flash or RealAudio, Liquid Audio is less of a sound design format for adding audio to your web site than it is a professional utility for music sales and distribution. Accordingly, if you want to sell digitized music files over the web, Liquid Audio is the clear choice. You can purchase a starter package for less than $1,000. If you just want to broadcast audio so listeners can preview your music, you may wish to use a less expensive option such as RealAudio or MP3.

The Liquid Music System consists of four core products: Liquifier Pro, Liquid Server, Liquid Player, and Liquid Express. Every component of the Liquid Music System has been designed specifically for electronic music distribution. Here is what each component lets you do:

Liquifier Pro

Liquid Audio now also supports secure MP3 delivery through its watermarking and file security technology. This helps ensure that appropriate copyright and security information will adhere to MP3 files distributed over the Internet, providing the first step to some form of copyright standardization and unified structure to MP3 delivery.

5.2.8. MIDI

Although MIDI is not a streaming format, it downloads so quickly and is so widely used that we decided to include it in this list. If you are looking for an easy, low-cost solution for adding a little theme music or a button rollover sound to your web site, but you don't want the long waits associated with downloading digitized audio clips, MIDI may be a great option.

MIDI (Musical Instrument Digital Interface) is a super-compact musical language that transmits instructions such as pitch, volume, and note duration to MIDI-compatible sound cards and synthesizers. Since MIDI is a text-based musical scoring language, it downloads super-fast and is ideally suited for HTTP delivery.

The downside is that MIDI is not sound itself; rather, it is the coded representation or score of how the sound should be reproduced by the user's MIDI sound engine. Many browsers and computer systems feature different MIDI sound engines that greatly vary in quality and instrument playback style. This variation makes it difficult for developers to predict what an end user is going to hear.

5.2. Streaming media formats

5.2.1. RealMedia and RealAudio

5.2.2. Windows Media Technologies (Netshow)

5.2.3. QuickTime

5.2.4. Flash and Director Shockwave

5.2.5. Beatnik's Rich Music Format (RMF)

5.2.6. MP3

5.2.7. Liquid Audio

5.2.8. MIDI