From Disney's 'Bumblebee' to Pixar's 'Brave', how has improvement in surround sound technology altered the audio experience in film and television?

Introduction

This report shall investigate the professional implementation and practices of surround sound technology*. It will critically analyse and discuss the reasons for using such technology whilst looking at the various formats that have been developed. The final sections shall debate the issues that localisation brings up as it is fundamental to the subjective experience of the listener and the technicalities of how surround sound is incorporated in broadcast audio.

[* Surround sound will also be referred to as multichannel audio or as its numerical format as will be defined later]

Why use surround sound?

 - Historical Background and Context

Throughout a large majority of the 20^th century and up to the current day, audio technologies have been continuously developed from a mono channel through to the multichannel surround sound. Even as far back as the renaissance, “spatial separation” (Holman, 2008, p2) was being experimented upon in Venice by musical composers Adrian Willaert and Giovanni Gabrieli (Arnold & Carver, n.d). Technologically speaking, it wasn’t until almost halfway through the 20^th century that stereophonic sound became commercially available (Davis, 2003), developed thanks in part to experiments by Alan Blumlein in the 1930’s (BBC, 2008). However, in film it was Walt Disney’s ‘Fantasound’ in 1940 (IEEE, n.d.) that really set the ball rolling and “invented an industry” when engineers experimented with five speakers: three at the front and two at the rear (Holman, 2008, p4). However, as significant a development as it was, there was little interest in ‘Fantasound’ after its introduction and relatively brief use (Davis, 2003).

Understandably the second world war halted progress in the development although, despite the hardship of the times, resulted in new technologies being created (or discovered) that would shape the future of audio in general: e.g. magnetic recording tape salvaged from Germany (Southall et al, 2009, p32-33) The next notable development came after a downturn in cinema

viewing figures due to the increase in popularity of television; it arrived under the name ‘Cinerama’ (nwvdberg, 2011). It was a wide-screen film experience, accompanied by a multichannel sound system, developed by Hazard Reeves (Cinerama, 2000), that incorporated 7 speakers that could also alternate between different surround arrays; anticipating future developments by half a century (Holman, 2008, p5).

This improvement in technology and understanding is all well and good however it doesn’t really suggest why it was required. However, sound designers, Directors and many others involved in the audio of films are always trying to creatively enhance the experience of the viewer, to make it more realistic. Gary Rydstrom states “since we hear all around us, while seeing only to the front, sounds have long been used to alert us to danger…sound becomes a key storyteller” (Holman, 2008, p195).  So it has become a tool for the sound designer that has progressed as the art of filmmaking has changed. 

Formats that are in use

“In the late 1980’s an agreement was made in the USA that the minimum number of channels for digital film sound was to be five – called 5 point 1 channel sound” (Amyes, 1998, p136) However 5.1 is not the be-all and end-all of surround sound: “Surround sound systems range from four to seven channels” (Amyes, 1998, p136).

At the time of writing there are the variety of combinations of speaker that exist from 1 – 64 discrete channels including:

-       1.0

Everything is played back via one channel and is uniform through any speaker present.

-       3.0

Left and right channels were added to a centralised speaker to produce stereophonic sound; the dialogue traditionally being assigned to the centre channel.   

-       4.0

There is the inclusion of surround speakers, however they are mono and playback only one channel.

-       5.1

This combination of speakers allows for localisation from left to right in the surround channels in addition to the stereo channels. A subwoofer, which is noted as the ‘0.1’, carries frequencies below 120Hz (Holman, 2008, p181) to help enhance the speaker performance of the stereo and surrounds.

-       7.1

This arrangement allows for even more specific localisation with the addition of ‘back-left’ and ‘back-right’ channels and is the standard cinema-mixing format.

-       9.1, 11.1 and 13.1

These arrangements are obvious progressions from 7.1 but the formats weren’t deemed good enough by content creators (Bowling, n.d.)

-       62.2 (Atmos)

Dolby released ‘Atmos’ as its answer to the issues pertaining to the issues with previous formats and as the next development. It allows for 64 channels  (two of which belonging to low-frequency-effects channels) and, for the first time, incorporates the use of ‘pan-through array’. This allows for more specific localisation of a sound throughout the sonic environment as individual sounds can be panned through individual speakers as opposed to groups of speakers that make up the surround channels on previous formats (Bowling, n.d.).

As can be seen from the image below, Atmos requires overhead speakers for added realism and has drawn remarks from listeners such as:

-        “The new sound system is incredibly realistic. During one demonstration, audio clips of a thunderstorm were so realistic — with the sound of the rain pummeling (sic.) down from above — I thought I might need a real umbrella.” (Bilton, 2012)

Digital cinema projections are a relatively recent development compared with the traditional 35mm optical release prints however both are in use across numerous theatres. For example ‘Vue Entertainment’, one of the UK’s leading cinema developers and operators, state that of its 657 screens, 121 are digital (Vue, n.d.)

This image is of the edge of a piece of 35mm release print. From left to right can be seen the Sony Dynamic Digital Sound (SDDS) which is stored on both outer edges, Dolby Digital (between the perforations), the optical analogue soundtrack and the white dotted line of the DTS time code, which is accompanied by low bit-rate coded audio on a CD-ROM (Amyes, 1998, p137). These innovations have been in place since around 1991 – Dolby Digital (DolbyDigital, n.d.), 1993 – DTS (DTS, n.d.) and 1994 – SDDS (SDDS, n.d.). Image: (Holman, 2008, p22).

Even with developments and improvements such as Dolby Surround Ex (6.1) and Surround 7.1, the encoding/decoding process means that backwards compatibility to older formats or smaller channel systems doesn’t compromise the functionality (DolbyDigitalEX, n.d.). This leads directly onto home theatre systems and their compatibility, which is mentioned below.  

The aforementioned surround sound developers would have ensured that the soundtrack was mixed in an environment suitable to their standards, but since there is no strict requirement for this in digital cinema (Welsh, 2007) there are aspects that should be followed for quality purposes:

There should be one ‘broadcast wave file’ per channel (per reel - reels being separated into about 10 – 20 minute sections) with a 24bit sampling bit depth and a sampling rate of 48kHz. If transferring audio between workstations a “2-pop” or a 1kHz tone that is 1 frame in length should be placed 2 seconds before the first frame of the visual to ensure synchronisation. If the film is at a rate of 24 frames per second (and the sampling frequency rate is 48kHz) then 2000 samples should be perfectly aligned within one frame (Whittlesey, 2007). The delivery format should be either a DVD-ROM or on via a USB/Firewire external hard-drive (Whittlesey, 2007).

Home Theatre

These recommended layouts were designed in specially built acoustic locations that optimise the sonic field in order to recreate as close an exact environment as was originally planned by the audio mixer. Cinemas are often built to a specification in order to achieve a ‘standard’. Surround sound developers such as Dolby and THX have their own respective ‘certifications’ but exact details as to what makes these ‘standards’ are limited unless working towards becoming certified with a non-disclosure agreement in place (Jerry Zernicke, email correspondence, 25^th October 2012).

In reality, the general public doesn’t have houses built to accommodate the acoustic specifications suggested and thus cannot implement surround sound in a manner that will achieve the highest quality. Developers do however release technology tailored for the home.

Dolby has their home theatre systems, which include the Pro Logic (II, IIx and IIz) series that has evolved to incorporate height channels in a 9.1 format (DolbyProLogic, n.d.). Likewise THX have their systems that relate to their certification system such that the consumer gets the appropriate setup: from ‘THX Certified Multimedia Products’ for desktop applications to the ‘THX Ultra 2’ which is designed for viewing spaces of 12 feet upwards (THX, n.d.). The developers have taken into account that not all rooms are the same size and as is demonstrated by THX, they go as far as working with individual consumers to get the best system depending on the resources.

Left:

An example of a 2.1 system as recommended by Dolby (Dolby¹, 2012).

Left:

Dolby’s recommended layout for the 5.1 format (Dolby², 2012).

Left:

The 7.1 format recommendations (Dolby³, 2012).

Note that, in each of the previous diagrams, the subwoofer has no specific placement requirements in relation to the listener except perhaps to avoid the middle of the room.

When positioning the subwoofer it is often suggested that it is placed near a corner as wall and floor reflections help to ‘load’ the wave (Holman, 2008, p38). This can lead to problems such as standing wave due to the natural resonant frequencies of the room being enhanced.

Localisation

Localisation is by far the biggest issue when it comes to surround sound effectiveness. The problem is perhaps predictable as there are issues that arise when listening closer to one speaker in a 2-channel stereo layout. There is usually a phantom centre produced directly between the speakers and an auditory sweet spot dependent on the placement, but taking into account the law of the first wavefront or ‘precedence effect’ (Holman, 2008, p184) and it is possible to see where issue will arise.

When localising sound sources, three key aspects are vital:

-       Level difference between the ears

-       Time difference between the ears

-       Head-related transfer functions

(Holman 2008, p177)

The range of human hearing spans 10 octaves (Holman, 2008, p178) or from about 20Hz – 20,000Hz (DK) and with the speed of sound in air being around 340ms^-1 (Kramer, 1996, p494), depending on the conditions, this means that the wavelengths can vary from 17m (for a 20Hz tone) – 0.017m (for a 20kHz tone) [See Appendix 1 for scientific equations]. Due to such large differences, means of localisation differ depending on the specific wavelength: for bass frequencies – the Interaural Time Difference (ITD) is necessary, whereas for the higher frequencies – the Interaural Level Difference (ILD) applies (Holman, 2008, p179). For sources that are directly above, in front or behind, where the ITD and ILD are equal or have no difference, the unique shape of the pinna comes into play as it creates differing resonances and frequency responses depending upon the sound location – the brain becomes familiar with these changes to the point where it is instinctive (Holman, 2008, p179-180).

The purpose of surround sound is to try and accurately recreate a sonic environment thus placing the listener inside - envelopment. Two-channel stereophonic imaging can only give the impression of depth – spaciousness, whilst only multichannel stereophonic imaging can produce the sensation of being a part of the environment (Holman, 2008, p187). Human perception allows a sound to be identified as being the same wherever it is in an acoustic (Holman, 2008, p182) and this is a direct result of the three sonic influences:

-       Direct sound (from source)

-       Reflected sound (from surfaces)

-       Reverberation (acoustic)

Studies have shown that the minimum number of channels required to produce a true feeling of envelopment is 5. This already creates a problem with the three aforementioned sonic influences as this now means that there will be 5 direct sound sources, 5 different reflections and 5 different reverberant patterns produced whereas with a natural sound source there is only one set – a false or phantom sonic environment is created (Holman, 2008, p183). It is all related to the issue of crosstalk: a single sound will arrive at each ear once (direct sound), whereas with two-channel stereo there are two arrivals at each ear and so it continues. Crosstalk is also noted as one of the downfalls of matrix decoders in home theatre systems that reproduce a surround environment from the phasing and amplitude differences in the stereo signal, ultimately localisation is reduced (P&E, 2004, p1-4).

It is perhaps strange too that for the experience of envelopment, the minimum 5-channel speaker layout wasn’t as is used and suggested in the image previously (Dolby², 2012) and placement of the speakers was deemed better at ±36°, ±108° and 180° (0° being the forward viewing position). In essence the main L+R and LS+RS stay the same but the centre channel becomes a back-centre channel (Holman, 2008, p188).

It noted that people state a larger difference in perception between two-channel and 5.1-channel environments, which is understandable however these are perceivable differences between 5.1 and 10.2. From here it is suggested that the “limit of perception has not been reached” (Holman, 2008, p191) and if the consideration that the incorporation of wide-front channels allows for a smoother panning of sound – which in 5.1 is less effective due to spectral distortion relating to HRTF’s (Holman, 2008, p182) – then perhaps the solution is to increase the number of channels. Michael Gerzon suggested that to achieve total perfection one would need “many thousands of channels and a million or so speakers” (Gerzon, 1975); this suggests that the Dolby Atmos 64 channel, pan-through array system is the correct manner in which to proceed.

Broadcast Audio

Part of the progression of surround sound technology is that it has become a part of the home environment, not just on DVD’s, but also as part of the broadcast television. With the switch to digital in recent years, it has allowed the medium of surround sound to enter the home in an easier manner as the audio signals can be broadcast and decoded appropriately at the receiver into the format that is in use.

Television

Corporations such as the BBC state in their ‘technical delivery standards’ that programmes mixed in surround must carry a stereo mixdown whether it is a separate stereo mix or a downmix using an encoder (BBC², 2012, p13). All surround sound is transmitted as 5.1 and like digital cinema audio; the files are required to be PCM at a sampling rate of 48kHz and 24bit sample depth (BBC², 2012, p17).

5.0 is a common broadcast format with the five channels carrying normal levels of bass, this is due to most television productions not requiring the increased bass that films do, thus rendering the LFE channel unnecessary (Robjohns, n.d., p24). There is an internationally agreed 8-track layout: L, R, C, LFE, Ls, Rs, Lt, Rt – the Lt and Rt being the stereo mix which in itself could include the 5.1 downmix that could be decoded by a Pro Logic box in the home (Robjohns, n.d., p27-28).

Sports broadcast

With sporting broadcasts, the aim is to make the viewer feel like they are at the event but not by recreating the arena (Lehrman, 2006). Unlike TV productions that have a larger time scale, mixing has to be done in real-time on location with up to 60 audio sources that sometimes include: in-game programme jingles/effects, referee audio, commentators, and stadium microphones (Lehrman, 2006). It is important to avoid cutting between the different audio sources associated with the changing cameras as this can create confusion (Lehrman, 2006) in the way that dialogue is centred in films (Holman, 2008, p118) despite character positions changing. Crowd sounds usually fill up the surround channels but mixers will try to include some sense of directionality: they will place the more unique sounds, such as a band or particular set of loud fans, and place them in the surround channels.

Multichannel radio

With the use of Digital Audio Broadcasting – also available via a digital television receiver – it has been proposed that multichannel audio may enter radio broadcasting (Holman, 2008, 144). It would work on the same principle as current matrix decoded 5.1 by broadcasting as a 2-channel Lt/Rt audio signal and being ‘upmixed’ at the receiver (Prosch et al, 2008). This would be aimed more at a musical audience, as traditional film and TV soundtracks would have little interest without visual interaction.

Conclusion

Since the days of Disney’s ‘Fantasound’, surround sound has developed through the medium of cinema with the likes of Dolby developing matrix surround systems before the technology began to cross over into the home environment through an improvement in home theatre development and expanding capabilities of DVD’s (Robjohns n.d., p3).

The advances have taken audio systems from mono up to 64 discrete channels and the potential for home theatre systems that can be tailored to the needs of the consumer. More speakers seem to be the way forward if Holman, Gerzon and the Dolby Atmos reviews are anything to go by and with the digital age perhaps all media needs will soon be accessible through television receivers and almost exclusively be in surround sound.

References

Amyes, T. (1998) Audio Post-production in Video and Film. 2nd Ed. Oxford: Focal Press.

Arnold, D., Carver, A.F. (n.d.) Cori spezzati. [Online] Grove Music Online, Oxford Music Online. Available from: <http://www.oxfordmusiconline.com/subscriber/article/grove/music/06486> [Accessed 23rd October 2012].

BBC (2008) Celebrating a stereo pioneer: Alan Blumlein. [Online] British Broadcasting Corporation. Available from: <http://news.bbc.co.uk/1/hi/technology/7538152.stm> [Accessed 21st October 2012].

BBC2 (2012) TECHNICAL STANDARDS FOR DELIVERY OF TELEVISION PROGRAMMES TO BBC. [Online] Available from: <http://www.bbc.co.uk/guidelines/dq/pdf/tv/TechnicalDeliveryStandardsBBCv3.pdf> [Accessed 15th December 2012].

Bilton, N. (2012) New Dolby Technology to Make Horror Movies Scarier. [Online] Available from: <http://bits.blogs.nytimes.com/2012/04/24/new-dolby-technology-to-make-horror-movies-scarier/> [Accessed 21st December 2012].

Bowling, S. (n.d.) Dolby Atmos Video. [Online Video] n.d. Available from: <http://www.dolby.com/gb/en/professional/technology/cinema/dolby-atmos-video.html> [Accessed 15thDecember 2012].

Breithaupt, J. (2008) AQA Physics A: AS. Cheltenham: Nelson Thornes.

Cinerama (2000) Hazard Reeves. [Online] C.A. Productions. Available from: <http://www.cineramaadventure.com/reeves.htm> [Accessed 23rd October 2012].

Davis, M.F. (2003) History Of Spatial Coding. Journal of the Audio Engineering Society [Online], 51 (6) June, pp. 554-569. Available from: <http://www.aes.org/> [Accessed 25th October 2012].

Dolby1 (2012) Home Theater Speaker Guide – 2.1. [Online] Available from: <http://www.dolby.com/uploadedFiles/Assets/US/Doc/Consumer/Dolby-Home-Theatre-Speaker-Guide-2.1-6-8.pdf> [Accessed 31st December 2012].

Dolby2 (2012) Home Theater Speaker Guide – 5.1. [Online] Available from: <http://www.dolby.com/uploadedFiles/Assets/US/Doc/Consumer/Dolby-Home-Theatre-Speaker-Guide-5.1-8-12.pdf> [Accessed 31st December 2012].

Dolby3 (2012) Home Theater Speaker Guide – 7.1. [Online] Available from:<http://www.dolby.com/uploadedFiles/Assets/US/Doc/Consumer/Dolby-Home-Theatre-Speaker-Guide-7.1-6-8.pdf> [Accessed 31st December 2012].

DolbyDigital (n.d.) DOLBY HISTORY. [Online] Available from: <http://www.dolby.com/gb/en/about-us/who-we-are/dolby-history/index.html> [Accessed 5th January 2013].

DolbyDigitalEX (n.d.) Dolby: Digital Surround EX. [Online] Available from: <http://www.dolby.com/gb/en/professional/technology/cinema/dolby-digital-surround-ex.html> [Accessed 19th December 2012].

DolbyProLogic (n.d.) Dolby Pro Logic IIz: Details. [Online] Available from: <http://www.dolby.com/gb/en/consumer/technology/home-theater/dolby-pro-logic-iiz-details.html> [Accessed 20th December 2012].

DTS (n.d.) History. [Online] Available from: <http://www.dts.com/corporate/dts-history.aspx> [Accessed 5th January 2013].

Gerzon, M. (1975) Ambisonics. Part one: General system description. Studio Sound. 17 (40) August, pp. 20-22.

Holman, T. (2008) Surround Sound – Up and Running. 2nd ed. Oxford: Focal Press.

IEEE (n.d.) Fantasound. [Online] Available from: <http://www.ieeeghn.org/wiki/index.php/Fantasound> [Accessed 21st October 2012].

Kramer, A. (1996) Sound, in: The Dorling Kindersley Children’s Illustrated Encylopedia. 4th ed. London: Dorling Kindersley.

Lehrman, P. D. (2006) The Surround Game. [Online] Available from: <http://mixonline.com/sound4picture/film_tv/audio_surround_game/> [Accessed 17th December 2012].

nwvdberg (2011) Cinerama Adventure Trailer. [Online Video], 8 February. Available from: <http://www.youtube.com/watch?v=1uwNYHGhnaQ> [Accessed 23rd October 2012].

P&E (2004) Recommendations For Surround Sound Production: Discrete vs. Matrix. [Online] Available from: <http://www2.grammy.com/PDFs/Recording_Academy/Producers_And_Engineers/5_1_Rec.pdf> [Accessed 6th January 2013].

Prosch, M. Hellmuth, O. Trommler, M. (2008) Surround sound for digital radio. [Online] Available from: <http://www.worlddab.org/public_document/file/3/Final_Surround_Sound_for_digital_Broadcasting_engl.pdf> [Accessed 8th January 2013].

Robjohns, H. (n.d.) Surround Sound in HDTV. [Online] Available from: <http://downloads.bbc.co.uk/commissioning/site/51_seminar.pdf> [Accessed 14th December 2012].

SDDS (n.d.) Sony: 1994 – Sony Dynamic Digital Sound. [Online] Available from: <http://www.sdds.com/> [Accessed 5th January 2013].

Southall, B. Vince, P. Rouse, A. (2009) Abbey Road. Revised edition. London: Omnibus Press.

THX (n.d.) THX Certification Performance Categories. [Online] Available from: <http://www.thx.com/consumer/home-entertainment/home-theater/thx-certification-performance-categories/> [Accessed 6th January 2013].

Vue (n.d.) About Us. [Online] Available from: <http://www.myvue.com/about-us> [Accessed 6th January 2013].

Welsh, R. (2007) The EDCF Guide to Digital Cinema MASTERING: Audio Processing. [Online] Available from: <http://www.cinematechnologymagazine.com/pdf/EDCF%20Mastering29pageswithcubelogo.pdf> [Accessed 5th January 2013].

Whittlesey, J. (2007) The EDCF Guide to Digital Cinema MASTERING: Audio File Requirements for Deluxe Digital Cinema, 10-20-06. [Online] Available from: <http://www.cinematechnologymagazine.com/pdf/EDCF%20Mastering29pageswithcubelogo.pdf> [Accessed 5th January 2013].

Appendix 1 – Wavelength Equations

Equations: (Breithaupt, 2008)