Adaptive music in games
When it comes to crafting the most immersive game experiences, music plays a role of vital importance. Successful games often employ adaptive music systems of varying levels of complexity.
These systems help enhance the player's immersion by modifying compositions and audio parameters in real-time during gameplay. This enables the background music to emotionally or aesthetically highlight whatever's happening in the game world at any given time.
To build adaptive music systems though, it is often necessary to employ adaptive music composition and arrangement techniques. Additionally, adaptive music tools will need to be built to apply them.
These tools can be built along with the rest of the game inside the game's engine or framework, however they can make for a significant investment of resources and expertise.
Another easier option is to use specialized software for the task. That's what I'll be covering in this article. This software comes in the form of audio middleware solutions and game engine plugins. They allow developers to make use of adaptive music technology in a faster and easier way.
Audio middleware solutions
An audio middleware solution is a software tool that allows for the creation of adaptive audio systems within a game. When compared with the built-in tools of popular game engines, audio middleware solutions generally offer functionality that enables the creation of complex behaviours in an easier way.
They also often present visual environments that make it easier to create, manage, preview and test audio events. Audio middleware solutions generally provide an extended feature set and force the design principle of separation of concerns. This facilitates collaboration between programmers and audio professionals.
Implementing adaptive music compositions is also commonly a much easier task. For a deeper dive into the differences between audio middleware solutions and the built-in tools game engines provide, make sure to check out this article: How to choose the right audio engine for your game.
Let's now have a look at some of the best available options for game developers:
Audiokinetic Wwise
Wwise is an audio middleware solution designed for game development and interactive media. Widely used in the game industry Wwise serves as a powerful audio engine and integration tool, enabling developers to create immersive and interactive audio experiences in their games.
Wwise is the middleware solution behind thousands of games. Some of them include Elden Ring, Far Cry 6 and Doom Eternal.
This middleware offers dynamic mixing techniques, spatial audio, built-in synthesis and features for building adaptive/dynamic music systems.
It features an open API and integrations for popular game engines such as Unity, Unreal, CryEngine and O3DE. Community maintained integrations for other engines are also available.
The desktop authoring app runs on Windows and macOS.
Wwise can be used for free if a game or project has a production budget of less than $250k*
For more information visit the Audiokinetic Wwise product page.
Firelight FMOD
Developed by Firelight Technologies, FMOD was designed for game development, interactive media, and virtual reality (VR) applications. Similarly to Wwise, FMOD is widely used in the game industry to handle various aspects of game audio.
FMOD is the middleware solution behind popular game series such as Deus Ex, Forza Motorsport and Just Cause.
The authoring app (FMOD Studio) features an interface similar to a DAW (Digital Audio Workstation) where tracks are laid out on a timeline. This can make it instantly familiar for anyone who has worked or experimented with this kind of software before.
The FMOD workflow makes it easy to implement behaviours, mix, and improvise in real-time during gameplay.
Some features of FMOD include randomization and modulation tools, a full suite of effects, mixer with a prioritized snapshot system and a built-in profiler. FMOD is also a great option for video game music as it provides a feature set that makes it easy to design and implement adaptive music compositions.
This middleware solution has ready to use integrations for Unity, Unreal and CryEngine. The desktop authoring app is available for Windows, macOS and Linux.
FMOD can be used for free if a game or project makes less than $200k revenue per year, on a small (under $500k) development budget.*
For more information visit the FMOD website.
Criware CRI ADX 2
CRI ADX2 is an audio middleware solution that enables developers to create immersive experiences with precise control over assets, music transitions, and interactive audio systems.
CRI ADX2 is the middleware behind games such as Persona 5, One Piece World Seeker and Street Fighter V.
Similarly to FMOD, it features an authoring tool with a multi-track DAW like interface with added functionality to create interactive sounds.
CRI ADX2 includes features for the creation of complex audio behaviors and adaptive/interactive music. Some of these features include spatial audio, auto-ducking, dialogue localization and an advanced profiler.
This middleware can easily be integrated with projects using game engines such as Unity, Unreal and Cocos Creator.
The authoring tool named CRI AtomCraft is available on Windows only. However, the Criware documentation mentions that a macOS version is currently in development.
CRI ADX2 can be used for free if a game or project generates less than $25k monthly in revenue.*
For more information visit the Criware CRI ADX 2 product page.
Elias by House of Elias
In contrast to the options presented before, this middleware solution developed by House of Elias (formerly Elias Software) was purposely built as an adaptive music engine.
The newest version of the Elias music software (Elias 4) is currently in development, and it's expected to be unveiled to the public during 2023. According to the Elias team, with the release of version 4, this middleware will also be expanded to include a full-fledged audio engine.
The current version (Elias 3) is still available and supported. Nonetheless, it's no longer being actively developed.
Just like the other middleware solutions mentioned before this software features an authoring tool named "The studio". You can use Elias Studio to build adaptive arrangements in an easy-to-use arrangement view.
Other highlight features include support for making precise loops of music tracks with overlapping tails, transition presets and the possibility to use midi and samplers. All of these features are built to perfectly tailor the music for video games.
Currently the only way to gain access to Elias 3 or an early access version of Elias 4 is by directly getting in touch with House of Elias. Also, get in touch for up-to-date pricing and licensing details. There's a contact form you can use on the main page of the Elias Software website.
Pure Data (Pd)
Pure Data isn't really an audio middleware solution like the ones presented before but a free and open-source visual programming language. It totally deserves a mention in the audio middleware category, though.
Pure Data is primarily used for multimedia, interactive music and audio applications, but It can also be used in game development. Pure Data has been used in the popular game Spore released in 2008 among other titles, for example.
Pure Data provides a flexible and powerful environment for synthesis, signal processing, and real-time audio manipulation. It can be used in games for the creation of custom sound effects, procedural sound generation, interactive music systems, adaptive music and other audio and music related functionalities.
However, this isn't a simple plug-and-play solution with integrations available for game engines. Integrating Pure Data with a game engine or framework usually involves custom coding and implementation.
The exact implementation process may depend on the game engine, programming language, and the specific requirements of the project. This can require an advanced programming knowledge.
Pure Data is a great tool for game audio development, but it is often used in conjunction with other middleware solutions or audio engines for a complete audio experience. These can complement Pure Data with other comprehensive features specifically tailored for games, including spatial audio, dynamic mixing and asset management.
This software can be downloaded as source code or as an OS specific package. There are versions available for Windows, IRIX, GNU/Linux, BSD, and macOS.
For more information visit the Pure Data website.
Game engine plugins
Besides audio middleware solutions, there are also plugins that extend the sound capabilities of game engines. Let's explore two of them:
Tazman-Audio Fabric
Fabric is a Unity engine plugin that provides an extensive set of high-level audio components. These components are the building blocks that allow for the creation of sophisticated audio designs by combining them in interesting ways.
Fabric also offers a range of features and tools that facilitate the implementation of adaptive music, including the ability to transition between components synced to music tempo.
Fabric is used by a number of big publishers and developers and is the solution behind games such as Angry Birds 2, Dreamfall Chapters and Aragami.
Being an engine plugin, it doesn't offer an authoring tool, instead it leverages the Unity's object based hierarchical structure for the process of creating complex audio behaviours.
Fabric is entirely written with the Unity's scripting language and doesn't require any external native plug-ins. It can be deployed on any platform that Unity supports.
Fabric is available for free under the Indie or Small license agreements. The "Indie License" is available for projects with a budget below $100k.*
For more information visit the Tazman-Audio Fabric product page.
Sonic Bloom Koreographer
Koreographer is also a Unity engine plugin albeit with a slightly different purpose than Fabric. Although this plugin isn't designed for creating adaptive/dynamic music systems it can be very useful for interactive music experiences. That's why it deserves a mention in this article.
As a refresher, interactive music implies an interaction between the player and the music as is the case in games like Guitar Hero, for example.
In Guitar Hero players must press the corresponding buttons and strum the guitar controller in time with the currently playing pieces of music. You can read more about different game music industry terms in our article Adaptive music in video games: What it is and how it works.
Koreographer simplifies the process of synchronizing background music and gameplay. Its simple editing interface allows you to use rhythms, beats, notes, volume and other dynamics of the musical composition to drive events in the game.
This plugin can be used in various ways such as creating rhythm games, precisely synchronizing game events to music, lip syncing or adding subtitles to audio tracks. It has been used in multiple games including titles such as Inscryption, Floor Kids and Audio Trip.
Koreographer is divided into two versions: Koreographer and Koreographer Professional Edition. The Professional Edition offers integration with middleware solutions like Wwise and FMOD and integration with the Fabric plugin mentioned above. It also gives you access to the source code among other features.
Koreographer is available through the Unity Asset Store as a one time purchase. Get the links and learn more by visiting the Sonic Bloom Koreographer product page.
Conclusion
Adaptive music helps to enhance player's immersion and elevates the overall gameplay experience. Fortunately these days there are no reasons to not take advantage of it.
As explored, there are multiple tools available that can make the implementation process easy. Professional adaptive video game music is also available for game developers of all budgets.
In this article I detailed some of the top choices for designing adaptive and interactive soundtracks for your games. Most of the presented choices can even be used for free under certain conditions.
Each presented software solution has its own unique set of features and strengths. The best software for building adaptive music systems depends on the project's specific needs and the creative vision behind it. You should also consider your individual or the team's expertise when selecting one of these tools.
Evaluating factors such as integration support, platform compatibility, licensing, documentation and community resources can also be important to determine the most appropriate choice. Also, don't forget to factor in the project's budget.
Note: Blips has a complete FMOD adaptive music tutorial you may find useful. A complete adaptive soundtrack is also provided for free.
*Visit each audio middleware solution website for the most complete and up-to-date licensing details.
