Interview: Understanding Dolby Atmos with Producer Paul Novotny

On a recent visit to Paul Novotny’s home studio, it became apparent I was in for an awakening. Understanding the ramifications of every forward step into a swiftly developing technological future is imperative. With that, let's go on a deep-dive into the Dolby Atmos immersive experience.

Our conversation takes us through the nuts and bolts of the process. What’s absent is the actual listening experience. Novotny played a surreal Atmos mix of Elton John’s “Rocket Man” which, to my ears, was extraordinary. We listened to Peter Gabriel’s “i/o’ recorded with Atmos mixing and mastering in mind. Before we get to that conversation, here's Novotny's backstory.

Paul Novotny is a multifaceted force, skillfully integrating his prowess as a bassist, composer, educator and technological innovator. A distinguished figure in the Canadian jazz scene, Novotny's accolades include multiple Juno nominations and a remarkable collaborative repertoire with international jazz icons such as Cedar Walton, Geoff Keezer, Junior Mance and Kenny Wheeler.

Beyond his virtuoso performances, Novotny's imprint on the music industry extends to his role as a producer, contributing to the success of notable recordings like Joe Sealy's Juno-winning Africville Suite and Carol Welsman's Juno-nominated Lucky to be Me, both released under his label, Triplet Records.

Venturing into the realm of screen media composition, Novotny has left an indelible mark with orchestral compositions for CBC's The National and dynamic themes for programs like George Stroumboulopoulos'The Hour and CBC'sNews Now. His musical work extends to the realm of video games, having played bass on the critically acclaimed and commercially successful Cuphead soundtrack, which boasted over 6 million sales by 2020.

In the academic arena, Novotny's commitment to advancing the understanding of music production and technology is evident through his recent completion of an MA in music composition. His pioneering research, supported by the Canada Council, delves into the intricacies of 3D immersive Dolby Atmos, challenging and reshaping the accepted specifications for the design of a Dolby Atmos music control room.

As a PhD musicology candidate at York University, Novotny continues to push the boundaries of innovation. His focus on creating a cutting-edge 7.1.4 Dolby Atmos Hyper Near-Field Tiny Studio, outlined in a forthcoming chapter from the 2022 InMusic Innovation in Music Conference, showcases his dedication to providing highly effective, economical, large-scope solutions for independent music creators, educational institutions, students and professional recording studios.

Novotny's journey epitomizes the intersection of artistry and technology. Here, he tells us about the future of sound innovation.

In a nutshell, tell me what Dolby Atmos is.

Like traditional stereo music mastering, Atmos music mastering will prepare a Dolby Atmos music mix for the best possible audience reception on a variety of playback devices and systems. The traditionally separate stages of mix and mastering still exist in the production workflow of Atmos music.

It's called immersive listening?

Yes correct, Atmos is immersive, it’s a three-dimensional audio format. The listening experience is highly stimulating. Atmos was originally designed for cinematic theatrical entertainment. It started in 2012 and was first used in the feature film Brave. In 2017 the first piece of music mixed in Dolby Atmos was Elton John’s "Rocket Man," mixed by producer/engineer Greg Penny.

Does panning achieve it? You hear sound above your head and around you.

Yes! Let's think of it this way, monaural recordings are one-dimensional, stereo recordings are two-dimensional, and now with Dolby Atmos we have a three-dimensional expression of recorded sound. An Atmos speaker system creates a 360-degree playback sphere. Atmos music speaker systems involve a standard mid-level plane of seven speakers that surround us at ear level. There is also a subwoofer placed on the floor, which gives you low-frequency effects and deep omnidirectional bass.

In my Atmos Hyper Near-Field Tiny Studio there are four height speakers named as the left/right top fronts and left/right top rear speakers. The immersive experience is best heard when sitting in the sweet spot of this equidistant speaker system. The sound comes from all around you. and yes, panning is how we achieve it. Essentially, all instruments become audio objects, and you can place them anywhere in the 3D sound box.

When you say seven, what do you mean by that?

The number seven is just one number of an accepted three-number system used to describe the number of speakers and planes involved in an Atmos speaker playback system, the period between the numbers separates the speaker planes. The standard Atmos system involves a total of 12 speakers and is called a 7.1.4 speaker sphere. The number seven specifies the seven surrounding speakers placed at ear level. The center dialogue speaker is placed between traditional stereo left and right speakers (L, C, R). Additionally left- and right-side speakers (LS, RS) are placed to the sides — just a bit behind the listener. And then left-rear and right-rear, surround speakers (LSS, and RSS) are placed further behind, but to the rear sides of the listener.

With this playback system sound can rotate around you, which is impossible in 2D stereo. When you gather the single LFE subwoofer, the seven-speaker ear-level plane, and the four top-height speakers, the numerical descriptor of that playback system is called a 7.1.4 Atmos playback system. When seated in the sweet spot, you will hear the music like you are in the middle of it and it’s a delightful sensation.

How would you mix this?

We have many new creator tools. Atmos mixing practice builds on stereo doctrines, but when you leave the 2D sound stage you can dislodge instruments from the front wall and bring them forward into the 3D sound box. This removes masking which is inherent in a 2D stereo image. In 3D audio, you can liberate all the individual audio objects or instruments from that flat 2D image to emphasize their independence and unique spatial nature. This means that we now don't need to use audio compression as much. In 2D we must stuff all the sound into just two speakers, but in Atmos there is six times as much space. 12 or more speakers can express an instrument's unique character. The Atmos format introduces dimensional separation and brings with it a significant increase in natural dynamic range that befits all genres of music.

Now, what's the difference between this and surround sound?

Surround sound refers to a 5.1 speaker setup, involving speakers named, center, left, right, left side, right side, and low frequency effect or enhancement (C, L, R, LS, RS, LFE). No top-height speakers are involved in surround sound. The 5.1 format was named by [sound designer] Walter Murch and [his work on] the movie Apocalypse Now is a great example of how effective it can be, specifically experienced when the helicopters enter the soundstage from the rear of the auditorium. The earliest iteration of cinematic surround sound goes back to the movie Fantasia in 1941, when Walt Disney and Leopold Stokowski envisioned a performance of “The Flight of the Bumblebee” to be fixed in the front speakers, but also make it fly around the auditorium. That scene never made it into the film, but this film history involving sonic spatial advancement is now driving the music business via Atmos.

Recently, a decision has been made to make 7.1.4 the standard archival format for 3D immersive audio going forward. For record labels, broadcasters, film, and TV content makers, 7.1.4 is how sound will be stored and larger formats will down-mix into the 7.1.4 archival format.

And the labels are going to this now? There was a period when the industry was suspicious.

There's a lot of industry, economic and cultural influence at play as we transition from 2D stereo to 3D Atmos. Dolby, Apple, Tidal, Universal Music, Warner, Netflix, Amazon Music Unlimited, Amazon Prime etc. — they recognize that music must now be in 3D to future-proof it. Curiously, Spotify is not in the game, but that stance can’t last. Gaming and automotive are the predominant growth sectors ahead and the global immersive audio economy is already estimated to be worth about 3.6 billion dollars.

However, the immersive audio industry is woefully lacking in female practitioners. Our new Audio Engineering Society (AES) president Leslie Gaston-Bird is a female immersive audio expert. This is very good! Many of my students are excited by Atmos and all record labels (including mine) are reaching into their back catalogue, pulling out the multi-tracks, adapting and re-releasing music in Atmos. The 5.1 format still serves as a technical backbone in the Atmos format.

What separates Atmos from previous attempts at surround sound is that we are now able to hear 3D Atmos music binaurally. It is encoded for playback on any set of stereo headphones. Binaural Atmos is constantly improving, and Virtuoso is the best binaural renderer available, but mixers must still work on speakers because favourable room acoustics involve acoustic cross-feed.

That cross-feed is essential for reliable transference to a variety of playback spaces that range between Far-Field cinematic theatres, Atmos Hyper Near-Field Tiny Studios, car cabins, smart speakers, home theatre sound bars, headphones and VR headsets.

I saw this state-of-the-art cutting-edge 3D motion sculpting technology used in the film Lasting Impressions in 3D, which featured160 masterpieces from 16 masters like Degas, Cézanne, Gauguin, Monet. It was mind-blowing seeing those paintings in motion. Folks on the streets of Paris are moving about. The action floats above and around the head. There is much to think about in the future.

Atmos will be significant in the realms of Virtual and Mixed Reality. One of the most exhilarating things about the Atmos format is how naturally it touches our biological hearing system. Our ears perceive sound in a 360-degree radius — they have developed to be our primary mechanism for safety. Mixing music in Atmos benefits from an understanding of how the human auditory system works. For example, the highest tip of our outer ears (the pinnae) is extra sensitive to high frequencies, by about 2-3 decibels (dB). This is how we perceive height, like birds above us. When I work with a set of drums, the low impact of the kick drum will be expressed in the LFE. I’ll also place it in the center speaker, along with the snare — like [producer[ Bob Clearmountain often does. With knowledge of how we perceive height, I’ll spread the drum overheads into the height plane, specifically the cymbals shoulder into the front top height speakers. It results in a very natural image of the drums. It’s like what a drummer hears when seated behind the kit.

2D stereo reverbs have always had the illusion of receding away from us as they fade way behind the front speakers. This expresses depth, width, distance and perspective change. But in Atmos, reverberation can come forward and pass right over you, and then situate in the rear speakers. It can also move upward which is truly novel and unnatural — but insanely compelling. 3D reverb feels deeply enveloping and can convincingly re-create a genuine concert hall experience.

Many people have auditioned my Atmos Hyper Near-Field Tiny Studio and the most common response has been “This is the most natural example of recorded music I've ever heard.” My studio is meticulously calibrated and has class-leading linear response. It was measured and approved by Ed Segeren who has designed and constructed large Dolby approved Atmos and 5.1 rooms in Los Angeles, New York and Toronto.

I've assembled an Apple Music playlist with over 24 hours of compelling examples of Atmos music — however, not all Atmos music is compelling. I've been building this playlist for six months as part of my PhD research which studies the Musicology and Technology of the format.

I argue that Atmos is here to stay. Many people think that because we only have two ears and two speakers, that's all that is needed. That is not a durable argument. Since we came out of the womb, we've heard sound in natural 3D. Compositionally, 3D music began in about 1550 with the Flemish composer Adrian Willaert. He experimented with left and right antiphonal choral composition, which was then expanded upon by Giovani Gabrielli in about 1585.

As a trained composer, orchestrator, arranger, musician, music producer and audio engineer, I see spatial music as being about 500 years old, but now, the ability for speakers and headphones to express the immersive spatial nature of music is possible and as you said, I’m passionate about it.

An accessible and fast-growing distribution system for 3D immersive music already exists, and Apple is even rewarding 3D music creators with a higher rate of remuneration for their streamed plays. New methods and a fast-evolving culture to support 3D music pedagogy, creation, consumption and economics take time to crystalize, but it’s all advancing daily, and that makes today an exciting time to be a music creator.

Innovation In Music, Cultures and Contexts: (available for pre-order on March 6^th, 2024).