If there is one thing that currently separates us from our spotted history in game audio, it is the publicly available middleware toolsets that allow us to sound design "in the box" interactivity in a close approximation of how it will sound once it finds it's way into the game. With the powerful combination of several game and audio engines, this trend towards enabling the Sound Designer to be more involved and in control of game related sound decisions is in stark contrast to the old model of handing sounds off to a programmer for implementation. In an attempt to expose the history of game audio toolsets I've rounded up a selection of interfaces I was able to find at large either on the web or embedded in various presentations over the years and link to said documents where applicable. I'm always on the lookout for public screen shots of proprietary toolsets for education purposes in order to illustrate a greater understanding of what has become a critical component to interactive audio.
Audio Toolsets
-Industry Snapshot
Microsoft Direct Music Producer
Based on the Microsoft Direct Music/ Direct Sound API for PC's, Direct Music Producer enabled functionality within the toolset to create and audition interactive functionality using the features of the API. This included the ability to loop files, specify random or sequential playback behavior, create sample banks (DLS) and specify parameters for interactive playback. More info on the specific functionality can be found here:, in addition to a terrific write up by Dave Javelosa over at the IASIG website: DirectMusic Producer for the Masses
Microsoft Direct Music Wiki
Microsoft Direct Music Producer Download
An Overview of DirectMusic Producer
Direct Music Producer
Creative Labs
In order to access features of it's sound cards and the OpenAL audio library, Creative Labs released several tools to assist audio developers.
EAGLE: Environmental Audio Graphical Editor
The Eagle toolset, announced in 2000, provided access to environmental modeling in order to simulate environmental, obstruction, and source model parameters for use in conjunction with EAX (Environmental Audio Extensions). The toolset provided Sound Designers with the ability to "easily create a variety of different audio data sets, or models. These models include: unique environments which simulate reflection and reverberation properties of a room; source property sets which provide initialization parameters for sounds such as distance attenuation and sound cones; and obstacle behavior models that simulate the effects of sound moving through, and around, doors, boxes, windows or other virtual objects. In addition, EAGLE aids the sound designer in creating these models by providing interactive 3D graphical representations as well as real-time auditioning of multiple data sets."
EAGLE: Cached Announcement
Creative Labs EAX® ADVANCED HD
Creative Labs EAGLE: Environment
Creative Labs EAGLE: Obstruction
Creative Labs EAGLE: Source Model
EAX2
The EAX2 toolset allows for the auditioning of environmental Reverb parameters to be used in the programming of presets for EAX used in OpenAL and various middleware solutions such as the FMOD and Miles Sound Systems.
Creative Labs EAX
EAX2 Panel
Creative Labs ISACT
One of the tools that most resembles modern middleware toolsets is Creative Labs ISACT or Interactive Spatial Audio Composition Technology. When it was released in 2005, ISACT provided one of the first visualy focused toolset which allowed an audio designer to make decisions about, and audition, the way their designed sounds should be used in conjunction with a 3d audio engine beyond simple sound file playback. In a lot of way it set the standard as an all-in-one environment to author and abstract sound with the immediate feedback provided by the tool.
Creative Labs Developer Central
Audio Middleware: GameCoda & ISACT
Creative Labs Introducting GameCoda & ISACT
Soundblaster 2003 Milestones
OpenAL
ISACT Production Studio
Microsoft XACT & XAudio2
Microsoft has taken great strides to provide audio tools to specifically leverage the features of it's audio libraries. Beginning with Direct Music Producer, and extending to the XBOX and XBOX 360 consoles in the form of XACT. While many of it's features and functionality have been forward thinking and directly in the service of advanced audio professionals, there's been a recent backlash in the XNA community due to the difficulty involved with simply “playing a sound file”. Some of the standout additions to the engine and tool include the ability to author sound propagation curves, global variables including Doppler and speed of sound, realtime parameter control, and Reverb settings.
MSDN XAXT Overview
The Microsoft Cross-platform Audio Creation Tool
XACT Rant1
XACT Rant2
XACT Audio Field Guide
Hands-on Workshop - XACT Audio
DirectX SDK
GameAudioTools - Scott Selfon
USC Game Audio Course Slides
Also see:
Microsoft Gamefest 2006 Audio Presentations
Microsoft Gamefest 2007 Audio Presentations
Microsoft Gamefest 2008 Audio Presentations
Microsoft XACT
FMOD
Firelight Technologies introduced FMOD in 2002 as a crossplatform audio runtime library for playing back sound for video games. By Since then it has brached into a low-level audio engine, an abstracted Event system, and Designer tool that allows the Sound Designer to access the features of the runtime audio engine. Within the FMOD Designer toolset a sound designer can define the basic 3d/2d paramters for a sound or event, in addition to the ability to effectivly mock up complex parametric relationships between different sounds using intuitive crossfading and the ability to draw in DSP curves to effect the audio.
Firelight Technologies - FMOD
FMOD Designer
Audiokinetic: Wwise
Introduced in 2006, the Wwise audio engine and toolset provids comprehensive access to the features of their engine. In addition to an abstracted Event system, they further enable the ability to make choices regarding additional fuctionality, such as volume changes or switch/ state changes, outside of just playing sounds. The ability to mock up every aspect of the engines ability brings the Wwise toolset further into a full prototype simulation outside of the game engine.
Audiokinetic Wwise
Wwise Toolset
The article is meant more as a brief in the history of Audio Middleware, and not as a comprehensive investigation of features and functionality. If you're interested in a deeper look into the differences between toolsets, there is a link to a great series of articles from Mix Magazine that covers in depth all of the available audio middleware solutions from 2007.
Audio Middleware: The Essential Link From Studio to Game Design
In addition to this I would recommend where applicable, each manufacturers website for further information on the current state of development. It's a great time to be a part of the continuous innovation happening in game audio!
Proprietary Developer Toolsets
Developer created proprietary toolsets continue to be a vital form of Sound Design in the industry, especially in cases where the gameplay specific features need to be exposed and interacted with. In the days before robust audio middleware toolsets this may have been the only way to get past the basic "Play Sound" fuctionality of an audio engine. Because of the nature of game development and custom technology, there is not a lot of information made publicly available regarding these tools. What little there is out there however is of value to those who might be involved with tool creation and especially those who are attempting to understand some of the additional esoteric aspects of audio.
Sony SCREAM
Veiled in secrecy, the SONY SCREAM tool provides direct access to Sony specific audio libraries. Public information has been scarce, unless you are a developer for the PS2/PS3 and the only screens around are found embedded in Presentations. One interesting thing to note is Sony's adoption of the iXMF interactive standard established by the IASIG in 2009 for their future toolset iterations including AWESOME and SULPHA.
Sony's Page: 'Next Gen Audio - Is That It?'
SCEE to implement iXMF
Next-Gen Audio Square-Off: PlayStation 3 vs. Xbox 360 IASIG - iXMF
Sony SCREAM
Bungie: Guerilla Toolset
The Guerrilla Toolset used by Bungie for the Halo games has been discussed in several articles, but aside from some general information and a tiny picture in the wild there arent many supporting details.
Halo 3 Audio: Locked and Loaded
GDC 2005 Report: Audio Production for Halo 2
Bungie Guerrila: Dialog
Swordfish Studios: 50 Cent Blood on the Sand
50 Cent Audio Postmortem: Dialog Database
50 Cent Devlop Interview
Dialog Database
Bethesda Software: Oblivion
Some of the audio functionality of the Elder Scrolls Construction Set is overviewed in this insightful development journal, along with what are becoming some of the best practices for Ambient sound integration. It's great to see a visual representation of the ambient being used across an area, along with the parameters for playback.
Elder Scrolls Construction Set
Elder Scrolls Region Editor
Day 1 Studios: Ducking
In a technical article the overviews the theory of Ducking, Zach Quarles exposes the tool he used at Day 1 Studio's to accomplish the interactive effect.
Article: Game Audio Theory Ducking
Internal Ruminations of an Audio Monkey
Ducking Tool
0.a.d. Sound Tool
The 0.a.d. Sound Tool was created in order to add an abstracted level of playback for audio files used by the OpenAL audio engine.
Radical Games: Prototype
In a series of articles regarding the ambient sound design of Prototype, details of their use of Max/MSP style procedural, runtime reverb, and the proprietary toolset "Audio Builder"
Article: Dynamic Game Audio Ambience
Article: The Sounds of Prototype
Prototype Ambient Tool: Swarm Editor
Audio Builder
Battlefield Bad Company
In a presentation given at the Game Developes Conferance in 2009, DICE Audio Programmer Anders Clerwall gave an overview of their "High Dynamic Range" audio solution.
In addition, here are some screens from the Audio Panel within the Frostbite Editor.
Frostbite Editor
EA: The Simpsons
From a presentation given by the audio team at EA Redwood shores for The Simpsons Game several screens for proprietary tools are exposed including a view of the EA proprietary procedural toolset, and several custom tools used for modifying parameters related to their physics implementation.
Presentation Slides: Sound Design for The Simpsons Game
Simpsons Procedural Tool
Simpsons Impact Tuning
Simpsons Roll Tuning
Simpsons Slide Tuning
Despite the rise of audio middleware in today's market, it is clear that is not a one size fits all mentality when it comes to creating a toolset and workflow to support the strengths of a particular games design. Hopefully by seeing, and understanding the choices studio's and game audio professionals are employing to help them creatively add sound to games, we can continue to drive the innovation of game audio through the current console generation and beyond.
Based on the Microsoft Direct Music/ Direct Sound API for PC's, Direct Music Producer enabled functionality within the toolset to create and audition interactive functionality using the features of the API. This included the ability to loop files, specify random or sequential playback behavior, create sample banks (DLS) and specify parameters for interactive playback. More info on the specific functionality can be found here:, in addition to a terrific write up by Dave Javelosa over at the IASIG website: DirectMusic Producer for the Masses
Microsoft Direct Music Wiki
Microsoft Direct Music Producer Download
An Overview of DirectMusic Producer
Creative Labs
In order to access features of it's sound cards and the OpenAL audio library, Creative Labs released several tools to assist audio developers.
EAGLE: Environmental Audio Graphical Editor
The Eagle toolset, announced in 2000, provided access to environmental modeling in order to simulate environmental, obstruction, and source model parameters for use in conjunction with EAX (Environmental Audio Extensions). The toolset provided Sound Designers with the ability to "easily create a variety of different audio data sets, or models. These models include: unique environments which simulate reflection and reverberation properties of a room; source property sets which provide initialization parameters for sounds such as distance attenuation and sound cones; and obstacle behavior models that simulate the effects of sound moving through, and around, doors, boxes, windows or other virtual objects. In addition, EAGLE aids the sound designer in creating these models by providing interactive 3D graphical representations as well as real-time auditioning of multiple data sets."
EAGLE: Cached Announcement
Creative Labs EAX® ADVANCED HD
EAX2
The EAX2 toolset allows for the auditioning of environmental Reverb parameters to be used in the programming of presets for EAX used in OpenAL and various middleware solutions such as the FMOD and Miles Sound Systems.
Creative Labs EAX
Creative Labs ISACT
One of the tools that most resembles modern middleware toolsets is Creative Labs ISACT or Interactive Spatial Audio Composition Technology. When it was released in 2005, ISACT provided one of the first visualy focused toolset which allowed an audio designer to make decisions about, and audition, the way their designed sounds should be used in conjunction with a 3d audio engine beyond simple sound file playback. In a lot of way it set the standard as an all-in-one environment to author and abstract sound with the immediate feedback provided by the tool.
Creative Labs Developer Central
Audio Middleware: GameCoda & ISACT
Creative Labs Introducting GameCoda & ISACT
Soundblaster 2003 Milestones
OpenAL
Microsoft XACT & XAudio2
Microsoft has taken great strides to provide audio tools to specifically leverage the features of it's audio libraries. Beginning with Direct Music Producer, and extending to the XBOX and XBOX 360 consoles in the form of XACT. While many of it's features and functionality have been forward thinking and directly in the service of advanced audio professionals, there's been a recent backlash in the XNA community due to the difficulty involved with simply “playing a sound file”. Some of the standout additions to the engine and tool include the ability to author sound propagation curves, global variables including Doppler and speed of sound, realtime parameter control, and Reverb settings.
MSDN XAXT Overview
The Microsoft Cross-platform Audio Creation Tool
XACT Rant1
XACT Rant2
XACT Audio Field Guide
Hands-on Workshop - XACT Audio
DirectX SDK
GameAudioTools - Scott Selfon
USC Game Audio Course Slides
Also see:
Microsoft Gamefest 2006 Audio Presentations
Microsoft Gamefest 2007 Audio Presentations
Microsoft Gamefest 2008 Audio Presentations
FMOD
Firelight Technologies introduced FMOD in 2002 as a crossplatform audio runtime library for playing back sound for video games. By Since then it has brached into a low-level audio engine, an abstracted Event system, and Designer tool that allows the Sound Designer to access the features of the runtime audio engine. Within the FMOD Designer toolset a sound designer can define the basic 3d/2d paramters for a sound or event, in addition to the ability to effectivly mock up complex parametric relationships between different sounds using intuitive crossfading and the ability to draw in DSP curves to effect the audio.
Firelight Technologies - FMOD
Audiokinetic: Wwise
Introduced in 2006, the Wwise audio engine and toolset provids comprehensive access to the features of their engine. In addition to an abstracted Event system, they further enable the ability to make choices regarding additional fuctionality, such as volume changes or switch/ state changes, outside of just playing sounds. The ability to mock up every aspect of the engines ability brings the Wwise toolset further into a full prototype simulation outside of the game engine.
Audiokinetic Wwise
The article is meant more as a brief in the history of Audio Middleware, and not as a comprehensive investigation of features and functionality. If you're interested in a deeper look into the differences between toolsets, there is a link to a great series of articles from Mix Magazine that covers in depth all of the available audio middleware solutions from 2007.
Audio Middleware: The Essential Link From Studio to Game Design
In addition to this I would recommend where applicable, each manufacturers website for further information on the current state of development. It's a great time to be a part of the continuous innovation happening in game audio!
Proprietary Developer Toolsets
Developer created proprietary toolsets continue to be a vital form of Sound Design in the industry, especially in cases where the gameplay specific features need to be exposed and interacted with. In the days before robust audio middleware toolsets this may have been the only way to get past the basic "Play Sound" fuctionality of an audio engine. Because of the nature of game development and custom technology, there is not a lot of information made publicly available regarding these tools. What little there is out there however is of value to those who might be involved with tool creation and especially those who are attempting to understand some of the additional esoteric aspects of audio.
Sony SCREAM
Veiled in secrecy, the SONY SCREAM tool provides direct access to Sony specific audio libraries. Public information has been scarce, unless you are a developer for the PS2/PS3 and the only screens around are found embedded in Presentations. One interesting thing to note is Sony's adoption of the iXMF interactive standard established by the IASIG in 2009 for their future toolset iterations including AWESOME and SULPHA.
Sony's Page: 'Next Gen Audio - Is That It?'
SCEE to implement iXMF
Next-Gen Audio Square-Off: PlayStation 3 vs. Xbox 360 IASIG - iXMF
Bungie: Guerilla Toolset
The Guerrilla Toolset used by Bungie for the Halo games has been discussed in several articles, but aside from some general information and a tiny picture in the wild there arent many supporting details.
Halo 3 Audio: Locked and Loaded
GDC 2005 Report: Audio Production for Halo 2
Swordfish Studios: 50 Cent Blood on the Sand
50 Cent Audio Postmortem: Dialog Database
50 Cent Devlop Interview
Bethesda Software: Oblivion
Some of the audio functionality of the Elder Scrolls Construction Set is overviewed in this insightful development journal, along with what are becoming some of the best practices for Ambient sound integration. It's great to see a visual representation of the ambient being used across an area, along with the parameters for playback.
Elder Scrolls Construction Set
Day 1 Studios: Ducking
In a technical article the overviews the theory of Ducking, Zach Quarles exposes the tool he used at Day 1 Studio's to accomplish the interactive effect.
Article: Game Audio Theory Ducking
Internal Ruminations of an Audio Monkey
0.a.d. Sound Tool
The 0.a.d. Sound Tool was created in order to add an abstracted level of playback for audio files used by the OpenAL audio engine.
Radical Games: Prototype
In a series of articles regarding the ambient sound design of Prototype, details of their use of Max/MSP style procedural, runtime reverb, and the proprietary toolset "Audio Builder"
Article: Dynamic Game Audio Ambience
Article: The Sounds of Prototype
Battlefield Bad Company
In a presentation given at the Game Developes Conferance in 2009, DICE Audio Programmer Anders Clerwall gave an overview of their "High Dynamic Range" audio solution.
In addition, here are some screens from the Audio Panel within the Frostbite Editor.
EA: The Simpsons
From a presentation given by the audio team at EA Redwood shores for The Simpsons Game several screens for proprietary tools are exposed including a view of the EA proprietary procedural toolset, and several custom tools used for modifying parameters related to their physics implementation.
Presentation Slides: Sound Design for The Simpsons Game
Despite the rise of audio middleware in today's market, it is clear that is not a one size fits all mentality when it comes to creating a toolset and workflow to support the strengths of a particular games design. Hopefully by seeing, and understanding the choices studio's and game audio professionals are employing to help them creatively add sound to games, we can continue to drive the innovation of game audio through the current console generation and beyond.