Last month, we covered the 80-year evolution of the many types of multi-channel arrays and surround sound formats — now becoming immersive sound.
Now let’s look closer into the new world of immersive live-sound systems. To recap, we covered how object-based immersive software allows the designers and users more options to scale the numbers of loudspeakers, based on the budget, target SPL, the size of the venue, the amount of spatial resolution desired and amount of envelopment. Some systems use dedicated DSP; others have their immersive software program running on a high-power Mac or PC with generic routers; some include mixer control features. Fig. 1 compares the changes in coverage from a left/right system and a system with several speakers across the stage.
While immersive systems do require considerably more speaker locations than before, the arrays don’t have to be as large as a system with less numbers of arrays. Fig. 2, courtesy of d&b audiotechnik, shows improvements that can be made to sightlines by using several smaller speakers; some allow for gradual system enlargement over time. Note that these systems can provide from less than 180° (across stage) up to 360° of sound immersion. Much more on these immersive system topics can be seen, in the expert replies, in the Q & A section on page 55.
This past year, NARAS/The Recording Academy changed the “Surround Sound” category in the annual Grammy Awards to an “Immersive Audio” category. This update might seem trivial to an outsider, but that is not the case; the changes have been more technical than that, so it is no longer the same category. Sound production has gone from being surround, meaning the sound comes from horizontal locations around the listener, to being immersive or “3D.” This means that the sound can now also come from above or below, creating a new, multidimensional audio field. An additional difference is that surround sound was a fixed channel-based design, such as 5.1, whereas immersive systems are object-based, that provide users more options to scale the channels/systems — even multiple formats of the same mix. The term “immersive” is now being used by many manufacturers and pro-sound users to better describe this expanded and scalable 3D sound-field.
Dan Daley, pro-sound engineer and FRONT of HOUSE columnist explained that “a bit of an arms race is already in the making around the new immersive live-sound systems, with channel counts in the low- to mid-double digits. Formats like Dolby’s Atmos (in cinemas) and Fraunhofer Institute’s MPEG-H have made deep strides into broadcasting, and music-touring artists are taking live-sound versions of immersive systems on the road. L-Acoustics’ L-ISA (Immersive Sound Art) — the live-sound iteration is known as L-ISA Live — is considered the market leader at the moment. Aside from an opportunity to sell more speakers (which is fine), I view immersive audio as the next step in a progression that seeks to simulate ever more precisely the natural environment. The ability to hear sound and process it for localization and threat assessment has been critical to the survival of our (and just about every other) species. Atmos finally acknowledges the cougar sitting in the tree waiting for dinner — us — to pass by below. So, in addition to being an incremental step from stereo, LCR, 5.1, etc., it’s also representative of evolution itself.”
Turnkey vs. DIY Immersive Systems
This month we will focus on a typical turnkey immersive sound system, providing everything following the sources and mixing console. d&b audiotechnik provided an immersive speaker layout and block diagram (Figs. 3-4) showing how d&b typically bundles a turnkey solution for an immersive sound system, shown in ways that a pro-sound user who’s new to immersive sound (like myself) can understand.
Turnkey immersive solutions, as provided by d&b, EAW, L-Acoustics, Martin, Meyer, and others to come, provide a simplified path for system designers and contractors toward an immersive sound system design and supply. DIY immersive sound systems, using standalone DSP and/or immersive software, are now available from several companies such as: Astro Spatial Audio, Dolby Atmos, ImmerGo, IRCAM/Flux, Ina GRM and OutBoard; such roll-your-own immersive sound systems are inherently speaker-type-agnostic, while more complex and flexible to design (multiple sources and DIY troubleshooting). A few replies from immersive DSP makers are included below.
Today, the immersive live sound “universe” is ever-expanding at a rapid pace. Next month, we’ll take a closer look at different ways to integrate complete immersive systems. Stay tuned!
David K. Kennedy operates David Kennedy Associates, consulting on the design of architectural acoustics and live-sound systems, along with contract applications engineering and market research for speaker manufacturers. He has designed hundreds of auditorium sound systems for churches, schools, performing arts centers and AV contractors. Visit him at immersive-pa.com.
Immersive Sound Developments: Experts Speak Out
Due to the overwhelming response to last month’s dialog (FOH, Oct. 2019, page 44), we are continuing that discussion with a few leading sound consultants and some key manufacturers involved in immersive live-sound systems.
Q: What is the difference between legacy surround sound approaches and the new immersive systems?
Bjorn Van Munster, Astro Spatial Audio (ASA): Technically, the ASA SARA II Premium Rendering Engine is capable of dealing with flexible speaker layouts — not channel-based like, 5.1 — by means of true object-based audio and advanced algorithms. Additionally, there are design guidelines that need to be followed to optimize the experience in the venue. This includes positioning of audio drivers, but also dispersion, frequency response, peak levels and interconnectivity between the equipment.
Marc Lopez, d&b: Soundscape has the benefit of delivering a consistent and natural audio experience to each listener in the audience no matter where they are seated. Our heritage stems from the live production perspective, so our focus for these features is targeted at making the workflow for system design, deployment and control (real-time or programmed) absolutely simple and predictable to use in a live situation, whether permanently installed or temporarily set up. There is also a shift in design approach when considering a Soundscape system. While more loudspeaker positions are required than a traditional L-R system, fewer and smaller loudspeakers in each position are typically required to achieve the same perceived loudness as a traditional system, as many more loudspeakers are producing sound at the same time. There is a misconception that Soundscape systems will dramatically increase the cost of the sound system, whereas in many cases a Soundscape system will not cost more than a traditional design.
James King, Martin Audio: Not all immersive audio systems are created equal. Users should beware of phrases such as “360-degree sound” or systems that purport to be immersive but in fact rely on old channel-based technologies. This is the exactly where Sound Adventures comes into its own. Our use of object-based audio means we are not limited by speaker position — we can place audio where we want the audience to experience it. Of course, the more the speakers deployed, the higher the granularity and more precise and object can be depicted. The SARA II Premium Rendering Engine from ASA (Astro Spatial Audio), which is at the heart of Sound Adventures, utilizes true object-based audio providing full control over position, level, acoustic characteristics and distance, thereby delivering scalability — a production can be transferred between entirely different venues with completely different loudspeaker configurations and yet retain the same performance simply by entering the X, Y, Z coordinates of the speaker layout. It’s also possible to achieve 3D, immersive sound with far fewer speakers than in the past, with ASA, we have achieved great results with a simple 5.1 system.
Robert Scovill, Avid: I think immersive stands to completely reshape — for the better — the listening experience, not only for the listeners but the people mixing the events. With immersive, the massive improvement in overall intelligibility for the entire listening geometry, coupled with the power of the localization of “watching” someone play, and having it sound as if the audio is coming right from where you are looking, can’t be underestimated. Over the years, we’ve become so acclimated to the shortcomings of stereo in a large space that we’ve lost sight of how good it could actually be in this type of format. And once you mix in an immersive setting during a live performance, and then go back to stereo, it just feels so inadequate for the job at hand. Immersive is here to stay, even though it’s going to take considerable time for it to be fully adopted for pop and rock music concerts. But for music and entertainment formats like performing arts centers and houses of worship, I view it as a “silver bullet.” The improvement for the entire audience is far from subtle.
Q: How do theatrical surround systems, electronic acoustics systems and live-sound immersive applications compare/differ and can they be combined into one large, flexible system?
Marc Lopez, d&b: Soundscape is designed to cover a number of immersive applications for live theater and live sound applications that include spatial positioning (through our En-Scene module) and spatial reverberation (through our En-Space module). We receive a fair amount of questions about the capabilities of Soundscape, including how it might address cinema needs where a venue might be showing movies in addition to live programs or providing reverberation enhancement for orchestral or choral performances. While Soundscape cannot perform cinema audio decoding, the DS100 engine certainly can provide the matrix and processing to properly assign decoded Surround Sound or Atmos channels to the correct loudspeakers. In the case of reverberation enhancement, the En-Space module was designed to produce a natural simulation of other spaces that we have captured, all controlled directly by the sound engineer. En-Space can be applied with suspended microphones or applied directly to inputs from live or playback sources.
Steve Ellison, Meyer Sound: Theatrical sound designers for years have helped encourage the development of tools to create engaging sonic experiences. We cut our teeth in live theatre, as early as 1993 with Cirque’s Mystère in Las Vegas. In this show, like most theater, the spatial cues are determined in production ahead of time, programmed on a computer, and triggered in a cue list with dozens and — not uncommonly — more than 100 loudspeakers to be addressed. Over the last few years, interest from live sound engineers as well as artists and producers has led to greater awareness of the potential for multi-channel audio to create more engaging live music experiences. We are creating a new system that will provide spatial mixing with a multi-touch interface, built around our Galaxy platform. Our approach with “live sound immersive” is to put the tools for multichannel spatial mixing at the operator’s fingertips. Electronic acoustics systems can provide truly immersive acoustics at the press of a button. These systems are typically fixed installations. So, while very quick to operate, they probably take the longest amount of time to install, calibrate and voice; whereas live-sound system configuration typically needs to be much faster than installs.
Q: If LCR is difficult to implement (special mix, video conflict, more rigging) in large venues, how can immersive systems become popular?
Scott Sugden, L-Acoustics: More complex show designs have been happening in every trade since the inception of live production. The real question to ask with any new technology is, is it better? If the technology is indeed substantially better, then the challenge of a more complex show will be solved. Our industry has made it a habit of managing increased complexity at live shows over the past 30 years. The production managers, sound designers, rental providers and manufacturers are a creative set of people who have and will develop new methods of installing more complex sound systems. Early adopters will lead the way and become the industry experts of tomorrow.
Marc Lopez, d&b: The content and vision of the venue really drives the need for an immersive system. Many venues are seeking to distinguish themselves and improve the experience and engagement that they provide. An immersive system like d&b’s Soundscape can offer this value. Sound engineers have the benefit of more control and less work to achieve their mixes. With a Soundscape system, a sound engineer uses less EQ, less dynamic processing, has more gain available before feedback, and more headroom because the sound system utilizes more outputs than the traditional 2- or 3-output system. The audience receives the obvious benefit of an improved listening experience and a significantly encompassing sweet spot — every seat becomes the best seat in the house,
Bjorn Van Munster, Astro Spatial Audio: The new generation is growing up in a world with AR, VR, gaming and cinema experiences as part of their everyday lives. It is not too much to suggest that in a few years they will expect immersive audio for live events, as it will be such a normal part of their entertainment. I believe that the requirement for immersive systems will be as a result of organic growth in audience demand.
Steve Ellison, Meyer Sound: Truly immersive systems surround the audience with loudspeakers that provide a highly coherent and phase-accurate experience. Touring with surround positions has an additional cost but people have high expectations these days. A system that purports to provide “frontal immersion” is only able to do so through psychoacoustic tricks that negatively impact coherence and phase accuracy. If it was possible to truly immerse the audience from the stage, cinemas would have done this ages ago, because they are very cost-sensitive. Obviously, that’s not possible and as a result, we are seeing wide adoption of new immersive cinema formats that use many loudspeakers surrounding the audience. There are really two barriers to widespread adoption in touring sound. First is the cost of rigging, infrastructure, setup time and transportation. This is a hard cost, which can be optimized but is unavoidable. The second barrier is the ease of use of the system providing the spatial mixing. Adapting on the fly to changing setups and having a creative feel that focuses on the mix — and not on the technology — are key.
Q: How is venue coverage modeling done/predicted for immersive systems?
Scott Sugden, L-Acoustics: With any sound design, it is important to understand how the sound system will cover the venue in amplitude, tonal and temporal metrics. With L-ISA, all design work is accomplished in Soundvision’s 3D modeling environment to predict the quality of the immersive design for the audience. With established goals for the design, multiple designs can be compared. The primary objective considerations are that of SPL Max, SPL Distribution and Time along with subjective considerations, such as Spatial Resolution, Horizontal Localization, Vertical Localization, Panorama and Envelopment.
Steve Ellison, Meyer Sound: The success of the system is dependent on both the system and the room acoustics. Understanding the acoustics of the room — not only reverberation time, but background noise level — can help designers better determine the loudspeaker density required. Coverage is important, but good modeling of room acoustics has a just as large of an impact over the quality of the result.
Marc Lopez, d&b: Planning a d&b Soundscape system starts with a similar process as designing a traditional system, but it quickly deviates onto its own path. ArrayCalc, the d&b prediction software, is utilized for precise design and coverage modeling. As Soundscape is capable of achieving many types of immersive applications, understanding the requirements of the space and how will it be used are important to establish upfront. Soundscape systems can be deployed as a 180 system across the proscenium or as a 360 system to encompass the audience area, or anything in between. Some venues have taken a step-by-step approach into Soundscape, where they utilize their existing LCR format for the first year, and plan for additional loudspeaker positions the following year. Even Soundscape applied to a traditional LCR setup can greatly improve the mix experience and improve the localization to a wider set of seats in the house. Learning a different design approach will take some time and experience, not so different than learning how to design a distributed point-source system.
James King, Martin Audio: Sound Adventures uses object-based audio, and represents a collaboration between Martin Audio and Astro Spatial Audio, whose SARA II premium rendering engine uses algorithms based on Wave Field Synthesis to transform audio channels into audio objects. Dedicated design guidelines are formulated for a proper system design. Among other SPL requirements, this includes frequency response, dispersion, etc., but also, installation requirements, such as flush mounting and architectural integration. The user interface has been designed to be extremely intuitive so new users can step up and start using it within minutes. Then as your tour moves on the next location, you simply repeat the process for whichever new room you find yourself in.
Q: Do immersive systems matrix signals across multiple loudspeakers, to get effective coverage, and would that cause delay problems for listeners in some locations?
Bjorn Van Munster, Astro Spatial Audio: We are able to matrix systems between different loudspeakers. How this is done depends on the situation and the purpose of the speaker at its location. Timing is always a critical issue in the eventual quality of the system. Time alignment is something we pay a lot of attention to in order to get the maximum performance out of our system.
Steve Ellison, Meyer Sound: Typically they do, but this is often under designer/artist control and is ultimately an artistic decision. Some sounds (such as effects returns, keyboard pads and backing vocals) lend themselves well to being spread over multiple loudspeakers. The combination of an impactful frontal system with a diffuse surround/overhead system provides ultimate flexibility for direct/diffuse control of mix elements.
Marc Lopez, d&b: By design, d&b Soundscape utilizes both level and delay to reproduce sound in the way that humans naturally hear. That means multiple loudspeakers are producing signal for a given object each with a calculated level and delay emulating the way sound propagates and reflects to be localized by the listener. The DS100 processor is aware of the loudspeaker positions and orientations that are set in ArrayCalc software, which allows it to calculate the precise level and delay times for each individual loudspeaker across multiple objects (inputs), so there is a coherent wavefront for the entire listening area.
James King, Martin Audio: Timing is a critical issue in the experienced quality of a system. The use of object-based audio means there are zero phase-related or delay issues, regardless of the location. We demonstrate this during our open day sessions at our UK headquarters. While our visitors are listening to the system, we turn off the algorithm that has been eliminating the Doppler effect! The reaction is always the same — absolute shock at the sudden difference! Then we turn the algorithm back on and suddenly the transition from speaker to speaker is absolutely seamless again.
Robert Scovill, Avid: This is typically being done with WFS (Wave Field Synthesis) or Amplitude Panning — or even some hybrid of the two — not matrixing ala Vance Brashears’ [of Idibri] work etc. And a lot of it is based on the work and research at the IRCAM labs. But everybody seems to be kind of building in their own special sauce into their DSP now. It’s going to be really fun to watch and hear the results.
Rocky Giannetta, GIA Design Group: My biggest concern with the immersive systems is how it affects specific metrics. It seems obvious that the more image sources (speakers), the higher the potential for localization, for a greater number of listeners within a defined area. C7 scores should be high, but what about C50 or STI, as we introduce more sources into an acoustical environment? I don’t know if we currently have the tools to quantify this, as you would need to have .gll files for a specific immersive loudspeaker system, for a given image placement. The immersive systems depend on loudspeakers with relatively wide coverage angles in the horizontal plane, and multiples of them to provide coverage across that plane. My guess is that system Q goes down. My other guess is that there is a benefit in having a more diffuse room decay with more reflection paths for a given seat location. So, what do you want to compromise to get a specific result?
Q: Do some immersive systems have critical/special features that competitors don’t have?
Steve Ellison, Meyer Sound: Of course! Every system is different — informed by the experience of the developers and company behind the system. Some systems are format-driven; others are more designer-driven.
Scott Sugden, L-Acoustics: L-ISA is a full system approach — design, processing and control — in one cohesive package. Having all three stages integrated together makes it possible to have a consistent/dynamic multi-channel sound system at every show. The starting point of any Immersive Hyperreal L-ISA system is the design. Multiple options exist depending on the needs of the performance. The patent-pending L-ISA Focus design is specifically arranged for shows that demand a lot of LF resources like Pop/Rock/EDM. This approach tackles the problems with LF distribution in large venues without the need for many multiples of full range and sub enclosures. The room engine within the L-ISA Processor is designed to ensure the direct sound of an object always precedes the reverberant field generated — no matter the size of the venue or the placement of object in the mix. The room engine is open for any sound designer or mixer to create the space they would like to hear and is capable of individually rendering for each of the 96 input objects and up to 64 outputs.
Bjorn Van Munster, Astro Spatial: Our Rendering Engine works with true object-based audio, based on Wave Field Synthesis. This implies that each object is calculated in real-time for the speaker layout. Our high-powered engine allows us to move sources very quickly without any audible phasing or Doppler effects. Furthermore, we are able to scale productions for different speaker layouts. Hence transportability of productions between venues (e.g., studio to theater) is very easy, as the production is no longer dependent on the speaker channels and position to obtain the experience you had during production.
Marc Lopez, d&b: d&b Soundscape’s proprietary algorithms utilize both level and delay for positioning, which produces an incredibly wide “sweet spot” that covers every seat in the listening area. Although delay is utilized for positioning, movement of the audio object can freely be moved in real-time. The Function Groups set by the user in ArrayCalc lets the processor understand the desired behavior of each loudspeaker. This allows each speaker output to automatically calculate the required level and delay as well as alignments between function groups. The vast majority of programming happens automatically when a user designs their system in ArrayCalc, like with traditional d&b systems.
Dave Haydon, OutBoard: Unique to TiMax is the ability to get involved under the hood to an infinitely variable degree with the individual level/delay parameters which form the Image Definition spatial objects. This creates a comfort zone with leading and experienced sound designers for the more challenging and complex immersive projects for which the latest slew of “black box” algorithm-based devices can be less flexible and revealing about what’s going on. TiMax is inherently speaker-type-agnostic and also less confined to rigid speaker system geometries and topologies than spatial algorithm devices that may be based on WFS, Ambisonic or IRCAM-based principles or derivatives. TiMax is much more of a spatial immersive Swiss Army knife and, as some designers have commented, perhaps only TiMax can handle the most challenging immersive demands.
Q: Are immersive systems worth the extra cost and time to install in a venue? Can average concert/meeting/worship goers hear the sound improvement?
Steve Ellison, Meyer Sound: Without a doubt; but it’s critical there are no weak links in the chain. If care is taken all the way from room evaluation in the design process, installation, understanding of the operator’s capabilities and a willingness of the artists to engage with the technology, the result can be truly stunning.
Scott Sugden, L-Acoustics: Yes. Multi-channel L-ISA Hyperreal or immersive deployments are worth the work. L-ISA has already been used in more than 10,000 live performances with over 10,000,000 people hearing and reacting to the sound design improvements.
Bjorn Van Munster, Astro Spatial Audio: It’s all about added value. Until now, every customer that’s used our system has been truly convinced about the added value we establish with the SARA II Premium Rendering Engine system. Critically, we are brand-agnostic, so we are on the side of the customer. We create the optimum performance for any space and have no interest in adding more equipment than is needed. We also work with different brands on single projects, in order to optimize audio performance, audience experiences and budgets.
James King, Martin Audio: Immersive audio has captured the imagination of artists, engineers and sound designers around the globe. The combination of additional loudspeakers and electronics means there is a higher cost, but this can be offset elsewhere and critically noticeable consumer experience is leading to opportunities opening up across the board. Put simply, ticket buyers are also more aware of factors like sound quality than in the past. For those in the business of live production, the question is not whether they should invest in 3D sound, but how they can do so in a way that is cost-effective and future-proof.
Robert Scovill, Avid: As for the costs associated with it, if you view it strictly through the lens of what we now pay for stereo, it’s going to feel slightly more expensive in terms of hardware and labor etc. But the result bears out those costs IMO and, as audio engineers, it’s a matter of making an adequate value proposition to the people that write the checks. Let’s rewind 25 years and ask an artist or a production manager what they would be willing to budget for video production in their shows 25 years into the future. No one saw the astronomical video production budgets being spent today coming, but that money goes out the door with little resistance these days.
