Sound: Enhancing the Illyriad experience with audio

In Illyriad we will be using audio in three ways:

1. Motif background music

We will be mixing these in and out of the game environment dependent upon the map focus on biome. Some examples could be Mediterranean music, jungle music, desert music or polar music. We also plan to have some generic musical motifs and themes that enhance the game experience, such as triumphal themes that linger after you win a battle or trepidation themes that build anticipation before a diplomatic event such as sabotage or assassination. We are generally writing specific music for all of these themes as opposed to licencing them. We are also exploring the Web Audio API for this purpose – almost treating it as a sequencer – so that we can slave a standardized or fractional BPM to the game server clock and use all the clever tricks. For example, we may increase tempo of the music with the player’s proximity to the event outcome.

Weird, I know, but during the midlife crisis that impelled me to start Illyriad, I also released a reasonably successful album of ambient and dance music.  So I’m pretty familiar with creating soundscapes and – whilst I haven’t yet used it in anger – the toolset I see in the Web Audio API looks pretty comprehensive. I would bet my bottom dollar that someone will build (if they haven’t already) a full Web Audio API sequencer, and I can also see (the equivalent of) VST plugins and softsynths being sold on the Chrome webstore.

2. Notification/event audio

These are the standardized sounds that occur during gameplay based on triggers such as event queuing or occurrences. For example, starting building construction in game with sound of carpenters sawing or the messenger noise you get when you receive an email. We’re looking at making some of these specific to the game races, so the new mail sound for receiving an in game message is different for Elves and Orcs. Some of these are being worked on currently by James Bell, our audio engineer. He is out in the field recording the sounds at historic battle re-enactment events around the UK so that we can really get it right!

We’re also looking to mix these on the fly in the browser. A battle notification audio event would be made up of many different audio snippets in the mix. For example, a player who attacks a pack of lions, using his cavalry, in a savannah environment that wins the battle will get a very different sound effect notification from a player who attacks a horde of skeleton warriors, using his bowmen, in the arctic north that loses the battle.

3. Environmental audio

Different from the motif music, these are specifically audio snippets (some looped, some one-off or randomly triggered) that are geo-specific, especially within the WebGL 3D environment. If you go up close to the grinding cog-wheels of a working flourmill or approach the river bank, the audio source is located (in the stereo and depth pan) in reference to your viewing position and that of the sound source with volume controlled by proximity.

Some of these effects are environment proximity-related and others are time-slaved (such as the dawn chorus of birds or the frogs and crickets of dusk) or environment-slaved (for example, wind, rain or thunder) as the game world progresses and changes.

WebGL Experiments: Illyriad’s 3d Town

What can WebGL do? Can it do what we want? We were wondering.. and so decided to put it to the test…

To test the upper bounds of WebGL we put together a rough and ready demo [caution – it’s bandwidth hungry].  It’s very rough, not optimized and currently only runs on Chrome [working in all browsers that support WebGL is our priority]; but that’s kind of the point – its a technology tester to ensure we weren’t making a mistake.

The results speak of themselves – it definitely proves itself!  Sure it needs a bit more polish, but we are now confident that the actual in-game libraries we are building have a lot of head room to use. Below are a couple screenshots of the town during the day:


And another at night:

Of course there were many trials along the way and things that didn’t quite work as we’d planned as can seen below:

   

We learnt the importance of GPU compressed textures and had to write a pixel shader decompressor of our own, as WebGL doesn’t currently support them natively – but with a cost.  The loss of mip-maping this causes it can clearly be seen; and we will have to work around this if they are not supported soon.

Overall we are very pleased with the result, which you can check out here.  Remember to press space to unlock your mouse to look around – if you aren’t fond of reading on-screen instructions 😉

Naturally this is just a taster of what we have waiting in the wings. We’ll look to provide some follow-up blog posts about the techniques and tools being used in this early experiment including:

  • Web Audio API
  • Pixel shader texture decompression
  • Deferred shading

3rd Party Libraries in use

Sound: Recording battle sound for Illyriad

In the field location recording of Medieval Battle.

A few weeks ago saw my first foray into recording sound for Illyriad. Much travelling was involved – well, London to Leominster day return by National Rail which I assure you, is more gruelling than one might expect. This took me to a medieval re-enactment of The Battle for Mortimer’s Cross. As far as re-enactments go, this was a smaller one but that’s no bad thing since at others (yes, I’ve been to a few) there are commentator, crowd and even tank (not so medieval) noise to cope with. Unless you have the budget to set up purpose made scenarios for your location recording in the middle of nowhere with no noisy interruptions, this sort of thing is the bane of a sound recordist’s life.

The idea of this little jaunt was to mostly record battle noise along with anything else that seemed appropriate. Medieval music as well as a blacksmith were also captured (whereby I mean recorded as opposed to taken prisoner!).

Readying for battle

The other bane of a sound recordist’s life is that what you record doesn’t necessarily sound like what you recorded. Let me give an example. In film work, much of the audio is re-recorded after a scene was shot. This is done so that extraneous, unwanted sound is removed, and replaced with, cleanly recorded sound from a studio. By doing this, the film sound can be tailored to exactly the way the sound designer wishes it to be. It’s not uncommon for a scene in a film to be left with absolutely no original sound whatsoever, dialogue included.

Anyway, back to the point. For this reason, when replacing sounds, it is quite usual to use something other than what is seen on screen to make the sound for an object. A good example is newspaper. Instead of a broadsheet, a piece of cardboard actually makes a more convincing newspaper sound (imagine gentleman sitting on train, shaking his paper out to better see each sheet). Another classic is horse hooves – these are very difficult to record. I mean how would you get the horse you were recording to run in time with the one on screen?! Even with the horse recording edited to fit, it would have been recorded outside and again, have a lot of extraneous noise. Instead, it’s done in a studio with, you guessed it, coconut shells.

My point being that a recorded medieval battle does not necessarily sound like one. In fact, I’d wager that most people’s first guess would be that I had just gone and rattled around a few pots and pans in the kitchen drawer. And don’t get me wrong, these people are very particular about their authenticity. The cause of unrecognisable sound won’t be through lack of having the right equipment, armour not made from the correct materials or in the correct way or substandard weaponry. I still have a lot of usable sound, it’s just it won’t be quite enough on its own.

This train of thought brings me to another closely related point though. If Hollywood (and yes, I’m sure I generalise here) wanted to make the sound of a medieval battle, they would craft it from other elements. Surely, you should here the ringing of swords as they clash against solid armour? The chings and chinks of the melee should be ear splitting, no? It seems this is an illusion fabricated by film to convince the public all the more of its authenticity. It is however wrong. So, I could go down a similar route (and I’m sure I will to a certain extent) but do we want Illyriad to sound like Hollywood?

The victors stand proud

Instead, this made me realise, that though I don’t want a battle to sonically resemble a kitchen being turned upside down, it should still sound real. Gritty. It won’t always be sunny in Illyriad, and not everyone here is a hero, so perhaps, sometimes, you’ll hear the rain, the mud, the lowly cry of a feeble army cowering before a mighty foe. Sometimes that might be you.

Though I currently have little plan as to how Illyriad will be shaped sonically, this at least gives me an ethos to begin moving forward with. It’s something I had already known instinctively but before now, just hadn’t quite found the words for.

Sound: Illyriad Sound – The Beginning

Let me introduce myself.

My name is James Bell and over the past few months I’ve been speaking with the powers that be about how sound might best fit into the world of Illyriad. There is certainly room for a few helpful notifications (message received, battle won/lost etc.) but the game certainly has scope to go beyond this – construction sounds when building, a smithy hammering away at the new batch of swords currently in production, trees being felled. It’s a pretty busy world after all and sound could really help convey this.

Now of course what you don’t want is stuff that’s just going to get in the way. Some irritating little tune that plays on constant loop will have people reaching straight for the mute switch (and yes, each player will have sound options for their account). A lot of people spend a fair amount of time in front of this and we don’t want to be pushing anyone to the brink of insanity! Instead it should encourage emersion into, and convey a subtle expansion of the world.

Much of the implementation is still to be decided and the techie bods are poring over HTML5 audio to see how to best achieve things. However, I have already started to collect original sound to create the library that will then be used for the game audio. Keeping true to the Illyriad “better than it needs to be” ethos, mass creation of effects from sample CDs will be avoided, and instead, Illyriad’s sonic world will have as much character as the rest of the game along with the people that play it.

This is the beginning of a long journey and I can honestly say that at the moment, I just don’t know when you’ll see (or rather hear) Illyriad’s first sound. Hopefully not too long for some of the simple, smaller parts, though the rest will no doubt prove to be quite a task! Keep an eye on this blog though and you’ll see how things develop 🙂

James