LuaAV Audio Tutorial

In brief: Audio synthesis instances (Nodes) are created in the audio engine from user-defined synthesis definitions (Defs). Nodes encapsulate a relatively fixed synthesis structure for efficiency, but can be modulated through input signals and parameters. A Node is created from a Def using the play() function.

Definitions

A simple example:


local def = require "audio.def"
local Def = def.Def
local Mix = def.Mix
local Env = def.Env
local SinOsc = def.SinOsc


local simpledef = Def{
	Mix{ "out", Env{ 1 } * SinOsc{ 440 } }
}


simpledef:play()

The first line, require "audio.def", loads the audio def module into the script's environment. The following lines create local references to functions in the audio.def module. Local references are good: they are safe, and more efficient. However, there are so many functions in this module, it can be useful to make them all globals when focusing on audio. The statement audio.def.globalize() makes all functions in the module global, and will be used in this tutorial to make all the examples shorter:

require "audio.def"
audio.def.globalize()

local simpledef = Def{
	Mix{ "out", 0.5 * Env{ 1 } * SinOsc{ 440 } }
}

simpledef:play()

The next line creates a new synthesis node definition, using the Def{} constructor. Def does not make any sound by itself. Def's job is to parse a configuration table into a synthesis graph and its associated state, and JIT (just-in-time compile) this into machine code instructions for use in play(). In this case, the configuration table contains one expression, constructed from audio.Def expression generators.

The Mix{} generator mixes (adds, overdubs) the sub-expression 0.5 * Env{ 1 } * SinOsc{ 440 } into the "out" signal bus. (Every script has an "out" bus predefined, which is connected to the main audio driver outputs, and probably thus to your headphones or loudspeakers.) The SinOsc{ 440 } generator describes a simple sinewave oscillator at 440 Hz, and the Env{ 1 } generator a simple decaying envelope of 1 second duration; multiplying them together creates a decaying sinewave, which is then used by Mix{}.

Note that no actual synthesis or calculation has been performed; this is a purely declarative (rather than imperative) statement, creating a data structure to describe audio computations.

The last line uses this definition and its JIT-compiled machine code to create a Node (an instance of a synthesis defintion running in the audio engine) by calling its :play()method.

Many nodes can be created from a single definition, but there's only so much you can do with such a simple sound:

simpledef:play()
wait(1)
simpledef:play()
wait(2)
simpledef:play()

Of course definitions don't have to be so static! We can add parameters to the Def configuration table, and use them within the :play() method:

local simpledef = Def{
	freq = 440,
	dur = 1,
	Mix{ "out", 0.5 * Env{ "dur" } * SinOsc{ "freq" } }
}

Any named fields in the configuration table become state variables in the synths described; these variables can be used within the synthesis expression that follows. More importantly, we can now pass instance configuration tables to :play() in order to create more varied sounds:

simpledef:play({ freq = 800, dur = 0.5 })
wait(0.5)
simpledef:play({ freq = 700, dur = 1 })
wait(0.5)
simpledef:play({ freq = 500, dur = 2.5 })

We can also add an amplitude scalar (for loudness), however we can't just multiply{ "dur" } * SinOsc{ "freq" } }with a string, because Lua won't allow operator overloading on strings. Instead, we use the special V constructor to convert "amp" into an audio.def expression.

local simpledef = Def{
	freq = 440,
	dur = 1,
	amp = 0.5,
	Mix{ "out", V"amp" * Env{ "dur" } * SinOsc{ "freq" } }
}

Note that in Lua, a function call with a single string argument can be shortened from f("string") to f"string", and a function call with a single table argument can be shortened from f({table}) to f{table}. Here's a primitive form of additive synthesis:

simpledef:play{ freq = 800, dur = 0.5, amp = 0.5 }
simpledef:play{ freq = 700, dur = 1.5, amp = 0.3 }
simpledef:play{ freq = 500, dur = 2.5, amp = 0.2 }

Parameters can also be modified during playback. The :play() constructor returns a handle to the Node instance it creates, which we use to manipulate it during playback. In particular, we can assign new values to any of its parameter names as if they were fields:

local s = simpledef:play{ dur = 2.5, amp = 0.5 }
for i = 1, 10 do
	s.freq = i * 100
	wait(0.1)
end

Multi-channel expansion

If any element in the expression graph is a list, it will cause that expression to expand into a multi-channel expression. The number of channels will cascade up to each parent expression. Thus, in the following example, the SinOsc frequency is a list of two sub-expressions, causing the overall Def to be stereo:

local simpledef = Def{
	freq = 440,
	dur = 1,
	amp = 0.5,
	Mix{ "out", V"amp" * Env{ "dur" } * SinOsc{ {V"freq"*1.5, V"freq"*2} } }
}

local s = simpledef:play{ dur = 2.5, amp = 0.5 }
for i = 1, 10 do
	s.freq = i * 100
	wait(0.1)
end

Parameters can also be defined as multi-channel, by assigning a list of values rather than a single number. Here's our example modified such that the frequency parameter becomes stereo. Note how the assignment to the parameter is also a list of numbers:

local simpledef = Def{
	freq = { 400, 404 },
	dur = 1,
	amp = 0.5,
	Mix{ "out", V"amp" * Env{ "dur" } * SinOsc{ "freq" } }
}


local s = simpledef{ dur = 2.5, amp = 0.5 }
for i = 1, 10 do
	s.freq = { i * 100, i * 101 }
	wait(0.1)
end

Assigning nil is ignored, so we can modify parameter channels independently:

local simpledef = Def{
	freq = { 400, 404 },
	dur = 1,
	amp = 0.5,
	Mix{ "out", V"amp" * Env{ "dur" } * SinOsc{ "freq" } }
}


local s = simpledef{ dur = 5, amp = 0.5, freq = { 100, 100 } }
for i = 1, 10 do
	s.freq = { math.random(10) * 100, nil }
	wait(0.2)
	s.freq = { nil, math.random(10) * 100 }
	wait(0.2)
end

More to come soon...

 

audio.Def{ configuration }

Def() takes a configuration table, and returns a new synthdef constructor derived from the configuration parameters (or throws an error for bad configurations). The synthdef constructor can be used by play() to create synth instances.

Def can be an expensive call: it parses the configuration table to produce an abstract synthesis graph structure, and then generates machine code for an optimized implementation of the synthesis graph (JIT compiled).

The configuration table includes a hash portion of parameter definitions, and an array portion of expression definitions.

Variable expressions (strings) in the expression definitions are checked first against the parameter list of the Def configuration. If not found, the variable is assumed to refer to a Bus in the containing Context. (If the name is not found when the synth is instantiated, an error is thrown.)

Parameter definitions:

Parameters are modifiable, stateful variables in the generated synth. Parameter definitions follow a standard format:

name = { [init values...], [channels = number] }

Init values (numbers) can be used to initialize these parameters. If not stated, the init value defaults to 0.

Channel number, if given, should be an integer greater than zero. If not stated, the default is 1, or the number of default values given (whichever is greater). The actual channel count used may increase, if more are needed for the expression definitions that follow.

Expression expressions

Expression definitions are used to build up the synthesis graph. Expressions follow a standard format:

ExprGen { [input args...], [rate=ratesymbol], [channels=number] }

ExprGen can be one of a library of expression generators; see the audio.def module in the docs.

Expressions also support Lua's standard mathematical operators (+, -, *, /, %, ^), generating compound expressions accordingly. A string argument can be manually promoted to an Expression by calling V(str).

Expression definitions can be nested deeply, and an expression can be used multiple times within the same configuration. Recursion is not currently supported (i.e. an expression cannot have itself as one of its inputs (or inputs' inputs etc.); however feedback (single-sample delay) is supported through the use of named variables.

Input args can be numbers, strings or other Expression definitions. If the input arg is not an Expr, it will be promoted to an expression using the V function.

Rate symbol is an optional hint; the actual rate used may be modified according to the rates of input args. Valid rate symbols are:

Channels number is another optional hint; the actual channels count used may be modified according to the channel counts of input args.

audio.V(arg)

Promotes arg to a full Expr:

Example

require "audio.def"
audio.def.globalize()

local simple = Def{
	freq = 400, dur = 1, amp = 0.5,
	Mix{ "out", 
		Pan2{ 
			V"amp" * Env{ "dur" } * SinOsc{ "freq" }, 
			pan=SinOsc{ freq=V"freq"*0.01 } 
		}, 
	},
}

for i = 1, 10 do
	simple:play{ 
		amp = 0.05, 
		freq = math.random() + i * 200, 
		dur = 4+i*.1, 
	}
	wait(0.2)
end

synth = synthdef:play(params, context)

Takes a set of parameters and an optional context (defaults to the root context) and creates, schedules and returns a new synth instance.

The parameters are mapped onto the parameter names defined in the Def.

The context is the space in which the synth plays; when the context is stopped, the synth will stop too. Any unbound signal variables in the synth are looked for in the context.

Parameters can also be updated after a synth has been created, using the synthdef[name] = value setter form.

Example:

require "audio.def"
audio.def.globalize()

local simple = Def{
	freq = 400, dur = 1, amp = 0.5,
	Mix{ "out", V"amp" * Env{ "dur" } * SinOsc{ "freq" } },
}

simple:play{ freq = 1000, dur = 0.5 }
simple:play{ freq = 800, dur = 1 }

synth[key] = value

Sets a parameter in a synth.

Example:

require "audio.def"
audio.def.globalize()

local example = Def{
	freq = 400, dur = 1, amp = 0.5,
	Mix{ "out", 
		Pan2{ 
			V"amp" * Env{ "dur" } * SinOsc{ Lag{ "freq", 0.7 } }, 
			pan=SinOsc{ freq=V"freq"*0.01 } 
		}, 
	},
}

local synth = example()

while now() < 1 do
	synth.freq = math.random(10) * 100
	wait(0.1)
end

synth:stop(), synthdef:close(),

Stops playing a synth.

Example:

require "audio.def"
audio.def.globalize()

local simple = Def{
	freq = 400, amp = 0.5,
	Mix{ "out", V"amp" * SinOsc{ "freq" } },
}

local synth = simple:play{ freq = 1000, dur = 0.5 }
wait(1)
synth:stop()