Audio tutorial 1: making sound

Audio synthesis in LuaAV is designed to be efficient yet flexible, and tightly linked into the timing system of LuaAV scripts. However, compared to events and function calls in Lua, or rendering a frame of graphics to a window, producing sound is a relatively continuous process. While there might be typically around thirty updates to a window per second, there will be more than forty thousand consecutive frames of sample data output to the audio device drivers each second. It just isn’t possible to produce this data quickly enough, and reliably enough, directly in Lua. Instead, we create objects running in a background audio engine to produce audio signals as more or less complex functions of time. These objects are defined by combining simple generators into more complex compound expressions. They can be used in the Lua script in a form very much like standard Lua tables, but in fact the expressions they represent are implemented directly in machine code for efficiency.

Using the audio.Def module follows a workflow of three or four stages:

  1. create a compound expression object from built-in expression generators
  2. use the Def function to turn this expression object into a synth constructor (JIT compiled)
  3. call this synth constructor to create active voices (generating sounds)
  4. optional: call methods on the voice objects to change their parameters or stop them

Building expressions

Before we can build audio expressions in Lua, we need to load them into our script. The main module is audio.Def. We’re also going to make use of a few of the expression generator functions in this module, so it is good practice to make local references to these at the top of the script:

local Def = require "audio.Def"

local SinOsc = Def.SinOsc
local Env = Def.Env
local P = Def.P

Now we can start to define the sample-generating process of a synthesizer, starting from basic expression generators. Most audio expression generators take a single table of arguments, so that arguments are identified by name. The Env function creates an expression object representing a decaying ramp (from amplitude 1 to amplitude 0) over a duration specified by the dur argument. The SinOsc function creates an expression object representing a sinewave as function of time, at a frequency given by the freq argument.

local env = Env{ dur = 1 }
local osc = SinOsc{ freq = 440 }

print(osc) --> Expr(SinOsc)

Note that these expression objects are simply data-structures for building specifications, and do not themselves make any sound!

We can use the standard mathematical operators on these expression objects to return compound expression objects:

local expr = osc * env
print(expr) --> Expr(Mul)

We can also build more interesting compound expression objects by using other expression objects as arguments. As a simple example, here’s a frequency sweep using the same Env input:

local env = Env{ dur = 1 }
local osc = SinOsc{ freq = 440 * env }
local expr = osc * env

Creating Synth constructors using the Def call

To turn these expressions into something that can actually generate audio, we call the Def module as if it were a function.

local mysound = Def { expr }

print(mysound) --> function

Def takes the expression (and other parameters as we shall see later) and does some parsing, inferring, code-generating and compiling in the background, creating an efficient machine-code representation of the expression for the purposes of synthesizing audio sample streams. In a way, it makes a particular compound expression become ‘concrete’. Note that there is no limit to how many different Def-compiled synthesis definitions you can use in your script.

The call to Def returns a function that can be called to create new instances (‘voices’) of this concretized synthesis definition. Now we are ready to make sound.

Creating voices from a synthesis definition

The synth constructors returned from calls to Def create active voices in the audio engine. Each voice is a different object that can be started, stopped and modified independently and at different times:

for i = 1, 4 do
  -- launch a voice:
  mysound()
  -- wait some time before launching the next one:
  wait(1/i)
end

Adding parameters to the definition

We can vary the sound between voices by using parameters in the synthesis definition. The parameters are declared in the hash portion of the table of arguments in the Def call, along with default values. These parameters can then be referred to in the compound expression using the P function. In effect, P”foo” function creates an expression representing ‘the value of parameter foo’.

local mysound2 = Def{
  -- the parameter defaults:
  amp = 0.25, dur = 1, freq = 440,
  -- the compound expression object:
  SinOsc{ freq = P"freq" } * Env{ dur = P"dur" } * P"amp"
}

Now these parameter names can be used when creating voices:

for i = 1, 4 do
  -- launch a voice:
  mysound2{ freq = i*330, dur = 1/i }
  -- wait some time before launching the next one:
  wait(0.5)
end

They can also be used to change the value of a voice parameter while it is playing. Though we haven’t used it yet, the call to the synthesis constructor returns a voice object in Lua (which acts as a proxy to the sounding process in the audio engine). Every parameter name is assignable on this voice object.

local voice = mysound2{}
print(voice) --> Synth

for i = 1, 10 do
  -- parameters of a voice can be set:
  voice.freq = math.random(10) * 110
  wait(0.1)
end

-- parameters can also be read:
print(voice.freq) --> 660

Note that it is perfectly valid to define an expression object using P parameters before the Def call is made:

local expr = SinOsc{ freq = P"freq" } * Env{ dur = P"dur" } * P"amp"
local mysound2 = Def{
  amp = 0.25, dur = 1, freq = 440,
  expr
}

It is also perfectly valid to use a parameter name that is not specified in the Def arguments. In that case, the parameter still exists, but defaults to a value of zero.

Stopping a synth

If you have been experimenting with building different compound expressions, you may have noticed that when no Env is used in the expression, the voices never stop playing (even if the voice object is fit for garbage collection). Every voice has a :stop() method, which can be used to stop the sound from playing. After calling :stop() on a voice, it will no longer respond to any other changes, and cannot be restarted.

local mysound3 = Def{
  amp = 0.25, freq = 440,
  -- the compound expression object (Note: has no Env):
  SinOsc{ freq = P"freq" } * P"amp"
}

local voice = mysound3{ freq = 1210 }
wait(1)

-- make it stop!
voice:stop()

Calling :stop() was not necessary for the definitions that embedded Env expression objects, because Env effectively triggers the same effect as :stop() automatically when it reaches the envelope end (the specified duration).
Note: this behavior of Env can be disabled or ‘paused’ using the done and hold arguments (see the Reference documentation for more details).

This entry was posted in Tutorials. Bookmark the permalink.

One Response to Audio tutorial 1: making sound

  1. Pingback: Stephen

Leave a Reply