Move to github + New Binary

In the interest of transparency and collaboration, we’ve moved LuaAV to github.  We’ll be working out of that repository from now on and retiring the SVN repository on the MAT servers eventually.  We’ve also posted a new OSX binary with the latest bug fixes and updates to the LuaAV github downloads page.

Posted in Development | 1 Comment

Glue: Binding C++ objects to Lua via Type Traits

Binding plain old functions written in C to the Lua language is not difficult, thanks to Lua’s well designed C API. Binding C++ objects to Lua however can quickly turn into a complex nightmare, as issues of inheritance, memory management, callbacks from C++, and so on multiply levels of required intricacy. Several solutions to ease the writing of bindings exist with various strengths and weaknesses, such as Luabind, CppLua, Luna, SWIG, etc. (see this page on the Lua wiki). For LuaAV however, we ended writing our own C++ binding for several reasons, outlined below. Our binding is called Glue.

First, we were not concerned to auto-generate bindings from header files; often the portions you want to expose to the scripting language are not the same as what you would expose to the C++ developer, and sometimes you may want to expose a more elegant interface based upon the flexible capabilities of the scripting language, rather than a more brittle direct translation.

Secondly, we did not want to rely on a secondary annotation language and separate build step to generate bindings (the approach taken in SWIG, for example). Coding the bindings directly in C++ makes debugging easier, lessens the cognitive load, and allows us to insert ad hoc code into the bindings as needed.

Thirdly, we wanted the binding to be as lightweight as possible – in terms of size, dependencies and run-time performance. In the interests of size and dependencies we avoided the Boost-based LuaBind. In the interest of run-time performance we leveraged C++ templates to move as much computation as possible from run-time to compilation-time.

Finally, we wanted the binding to be as unobtrusive as possible. Several lightweight Lua/C++ binding libraries make use of inheritance, either deriving from a generic Userdata base class, or deriving a Userdata type from the C++ class to be bound. We avoided these approaches via template-based Type Traits. No special derived or base classes are needed at all. Instead, a new type of Glue<T> (where T is the type to bind) is defined and specialized to specify the necessary properties and methods to bind a type to Lua. Note that the Glue<T> type is never instantiated, it has no member data and its methods are all static functions (class methods). By using C++ templates, the entire binding fits into a single C++ header of a few hundred lines of code, whose only dependencies are the Lua headers themselves.

Usage

Probably at this point, an example would be more helpful. Consider the following definition of a class that we wish to bind to Lua:

class Foo  {
public:
Foo(int x) : Base(), x(x) {}
~Foo() {}
int x;
};

The first thing we need to define (the only mandatory item) for a binding to Lua is the name of the class. This will be used in the default implementation of the __tostring metamethod, for example. The type trait of Glue<T> is annotated with the name “Foo” by means of a specialization of the Glue<T>::usr_name() function:

template&lt;&gt; const char * Glue::usr_name() { return "Foo"; }

To install this new binding in a Lua state, Glue<Foo>::define(L) must be called. Typically this would occur in the luaopen_xxx method of a Lua module, or in the construction of the lua_State embedded in an application. Glue<>::define creates the metatable associated with the Foo type (additional methods can be added to the metatable by specializing the Glue<>::usr_mt method).

To push objects of type Foo into Lua, we can now use Glue<Foo>::push(L, foo). This will create a new userdata (a boxed pointer to the object), wrapped in the the proper metatable. To retrieve objects of type Foo from the Lua stack, we can use Glue<Foo>::checkto(L), Glue<Foo>::is(L), and so on.

Caveats

Glue does not make any attempt to ensure a one-to-one binding between userdata and the objects pointed to; it is possible to push a C++ object into Lua several times and end up with several distinct userdata. Glue does, by default, install an __eq metamethod that checks the equality of the boxed pointer (rather than the userdata themselves), whose implementation can be overridden by specializing Glue<>::usr_eq. Glue also offers two methods (usr_set_reference_map and usr_set_reference_map) to implement a one-to-one correspondence between userdata and boxed pointers.

If the reference map is not used, the possibility of multiple pushes may lead to memory errors.

Listing

The full listing can be found in the LuaAV source (lua_glue.h), but here’s a short version:

template&lt;&gt;
class Glue {
public:
/*
required hook to define metatable name
*/

static const char * usr_name();

/*
optional hook to define metatable superclass name
*/

static const char * usr_supername();

/*
optional hook to define a create function (default returns NULL)
arguments at stack indices 1+
*/

static T * usr_new(lua_State * L);

/*
optional hook to specify __gc method (default is no action)
NB: multiple calls to push() the same object will result in an equal
number of calls to gc() the same object (i.e. some form of reference
counting may be appropriate)
*/

static void usr_gc(lua_State * L, T * u);

/*
optional hook to apply additional behavior when pushing a T into Lua
(e.g. reference count increment, create userdata environment, etc.)
userdata is at stack index -1
*/

static void usr_push(lua_State * L, T * u);

/*
optional hook to signal the use of usr_index.  If defined,
the __index metamethod will us usr_index instead of directly
using the metatable
*/

static bool usr_has_index();

/*
optional hook to specify __index method (default is to use the metatable)
key will be at stack index 2
(userdata itself is at index 1)

NOTE: usr_has_index() must return true if this is used
*/

static void usr_index(lua_State * L, T * u);

/*
optional hook to specify __newindex method (default is to signal an error)
key will be at stack index 2, value will be at stack index 3
(userdata itself is at index 1)
*/

static void usr_newindex(lua_State * L, T * u);

/*
optional hook to specify __eq method
(default tests equality of pointers a and b)
NB: Will only apply for objects of the same T (not superclasses)
The Lua manual states that the __eq metamethod "only is selected
when both objects being compared have the same type and the same
metamethod for the selected operation."
*/

static bool usr_eq(lua_State * L, T * a, T * b);

/*
optional hook to convert non-userdata value at stack index idx to a T
e.g. convert a number into a T object...
*/

static T * usr_reinterpret(lua_State * L, int idx);

/*
optional extra hook when retrieving a T from Lua (invoked by Glue::to(), Glue::checkto() etc.)
this is an opportunity to insert extra conditions on verifying the userdata as of valid type
(e.g. checking for a magic number within u)
return u if the condition is met; return NULL if not.
(Lua userdata is at stack index idx)
*/

static T * usr_to(lua_State * L, T * u, int idx);

/*
optional hook to override the default __tostring method
*/

static int usr_tostring(lua_State * L, T * u);

/*
optional hook to add additional fields to metatable
metatable is at stack index -1
*/

static void usr_mt(lua_State * L);

/*  create the metatable
if superclass != NULL, metatable will inherit from the superclass metatable
(which must already be published)
call this e.g. in luaopen_xxx
*/

static void define(lua_State * L);

/*  register either
the constructor function (create; usr_new must be defined)
or the metatable itself
to the table at stack index -1, under the name usr_name
call this e.g. in luaopen_xxx
*/

static void register_ctor(lua_State * L);
static void register_table(lua_State * L);

/*
make the metatable callable with __call
*/

static void register_class(lua_State * L);

/*  Install additional methods to metatable via a luaL_Reg array */
static void usr_lib(lua_State * L, const luaL_Reg * lib);

/*
push a T pointer to the Lua space (also calls usr_push if defined)
NB: pushing the same object will create a new userdatum for each push
*/

static int push(lua_State * L, T * u);
/*  if index idx is a T (checks metatable key), returns it, else return NULL */
static T * to(lua_State * L, int idx = 1);
/*  as above but throws error if not found */
static T * checkto(lua_State * L, int idx = 1);
/*  Lua bound constructor (usr_new must be defined) */
static int create(lua_State * L);
/*  zero the pointer in the userdata */
static void erase(lua_State * L, int idx);

/*
Intended for use in a custom usr_mt function for registering userdata
functions that operate like table fields as opposed to function calls.
Additionally adds functionality for per-userdata custom fields with __newindex.
*/

static void usr_attr_mt(
lua_State *L,
luaL_reg *methods,
luaL_reg *getters = NULL,
luaL_reg *setters = NULL
);

/*
Intended for use in a custom usr_push.  It adds an environment table to the
userdata where per-userdata custom fields are stored.  An environment table is
required if usr_attr_index and usr_attr_newindex are also used.
*/

static void usr_attr_push(lua_State * L);

/*
Intended for use in usr_index.  First checks the enviroment table and then the
metatable.
*/

static int usr_attr_index(lua_State *L, T *u);

/*
Intended for use in usr_newindex.  First checks the metatable and then uses the
environment table.
*/

static int usr_attr_newindex(lua_State *L, T *u);

/*
For userdata with attributes, prototype constructors (table constructors) can
be used to automatically set settable attributes.  This function checks to see
if the argument passed to usr_new is a prototype argument.
*/

static bool usr_attr_is_prototype(lua_State *L);

/*
Uses a prototype constructor to set a userdata's attributes.  Intended to be used
in usr_attr_push where index 1 is the prototype table and index -1 is the userdata.
usr_attr_prototype will call usr_attr_is_prototype to check if it can be used.
*/

static int usr_attr_prototype(lua_State *L);

/*
Utility functions for mapping pointers to userdata values.  Used most often in callback
situations when you need to go from a pointer to the actual userdata value in Lua
and don't want to create a totally new userdata.
*/

static int usr_set_reference_map(lua_State *L, T *u);
static int usr_get_reference_map(lua_State *L, T *u);

// internal methods:
static const char * mt_name(lua_State * L);
static int gc(lua_State * L);
static int tostring(lua_State * L);
static int newindex(lua_State * L);
static int index(lua_State * L);
static int eq(lua_State * L);
private:
// internal methods:
static int class_create(lua_State *L);
// object wrapper:
struct Box { T * ptr; };
};
Posted in Development | 2 Comments

Live coding

Live coding (wikipedia) can be an intense and rewarding experience. Here are some notes on how it works in LuaAV.

script.live = true

LuaAV is responsive to user editing by reloading the script whenever it is saved. However, reloading will destroy and re-create any resources that were running, such as synthesis routines and OpenGL windows. The reloading behavior can be changed in any script by setting the global property script.live = true.

Now when a script is saved in the external editor, LuaAV does not close the script, but instead re-runs it using the existing Lua state. That means that every line of your script is run again, which may or may not modify state and resources that had been created previously.

This feature is possible because Lua is a fully re-entrant language. However there are several things to be careful about when using script.live=true:

locals and globals

A local variable is only visible in the block in which it is declared. When a script is run, variables declared local in the script will be available only for that run, and for any functions defined in that run. (You can imagine that running a script is like putting “function()” … “end” around the entire script, and calling that function) Re-running the script will not be able to see the locals created in a previous run.

Therefore, if you want a value to be visible in a future run, it should be not declared local.

However, if your script changes were minor, it is likely that many global variables will be overwritten by reloading the script. If you want to avoid this behavior, you should test and set the global only if it doesn’t already exist; a handy idiom looks like this:

myglobal = myglobal or 100.

On the first run of the script, the variable myglobal does not exist, so it is created and initialized with a value of 100. On a subsequent run, myglobal already exists and is not modified.

This is especially important for creating persistent resources such as OpenGL windows and audio synths. Failing to use the win = win or Window() idiom will lead to opening new windows every time the script is re-run.

coroutines and go()

Similarly, a coroutines launched using go() in a script, or by while true … wait() … end at global scope, will spawn another coroutine each time the script is re-run, potentially leading to many parallel tasks. If you want these coroutines only to be launched on the first load of a script, one strategy could be to use an initialization variable:

if not initialized then
  go(myfunc)
  initialized = true  -- initialized must not be a local variable
end

synths with no envelopes

Synths may have a similar problem: if they do not have an envelope to stop themselves, you need to make sure you can still reach a reference to them to call their :stop() method, otherwise you can end up with playing synths that cannot be canceled.

Posted in Tutorials | Leave a comment

New release (January 2011)

We’re pleased to announce a new release of LuaAV – grab it from here!

API changes:

There have been some important changes in how some of the modules are used. For users of the previous release, here’s a quick summary:

  • audio.def module is renamed audio.Def, and usage has changed (see this post for a tutorial) as summarized in this code snippet:
    -- was: require "audio.def"
    local Def = require "audio.Def"
    -- was: audio.def.globalize()
    Def.globalize()

    local mydef = Def{
        freq = 440,
        -- was: Mix{ "out", SinOsc{ V"freq" } }
        SinOsc{ P"freq" }
    }

    Other usage remains the same.

  • Lattice is now called Array (see this post for more). Usage looks like this:
    local Array = require("Array")

    local arr = Array{
        components = 4,         -- The number of components (samples per-cell)
        type = array.Float32,   -- The data type
        dim = {512, 512},      -- The dimensions
        align = 4,            -- Optional byte-alignment
    }

    print("components: "..arr.components)
    print("type:       "..types[arr.type])
    print("dim:        "..table.concat(arr.dim, " "))

    -- set data in cells:
    arr:setcell(128, 128, { math.random(), math.random() })

    -- integer lookup
    print(unpack(arr:getcell(128, 128) ))

    -- interpolated floating point lookup
    print(unpack(arr:read(128.5, 128.5) ))

    All existing objects that used matrix or lattice now use array, e.g. Image:array().

  • All module names that are actually object constructors have now been capitalized.  This includes both Lua modules and C modules in the following list:
    • image -> Image
    • all of the gui sub-modules
    • array -> Array
    • audio.def -> audio.Def

Changes:

  • Entirely rewritten audio.Def module.
    • Many changes have been made to improve the JIT compilation time (with no impact on the performance of the generated machine code). Around a 10x speedup was noted during testing.
    • New generators: Buzz, Saw & Square oscillators, biquad (low-pass, high-pass, band-pass, band-reject, all-pass), noise (white, pink, brown) and other utility generators.
    • Better handling of multi-channel streams in the generators (including new Channels, Mono & Stereo generators)
  • Stability improvements in the audio engine, and some optimizations to reduce construction overhead.
  • Lua interpreter updated to LuaJIT 2.0 beta5 (OSX).
  • Clang module now supports C++ and embedded in libluaav
  • Image module on OSX uses CoreGraphics native image I/O framework
  • opengl module has been completely re-written and now includes sub-modules for textures, shaders, slabs, and utility drawing functions
  • font module now uses both FreeType and native Cocoa NSFont class for font rendering
  • New opencl module for parallel data processing on both the CPU and GPU
  • The LuaAV console now highlights errors for easy scanning of console output
  • Lots of other docfixes and bugfixes (thanks to the students of UCSB MAT 594P especially for pointing many out!)
Posted in Development | 1 Comment

Multi-channel audio

Multi-channel expressions

LuaAV’s audio.Def expressions can handle multi-channel streams quite easily. To expand an expression into multiple channels is a matter of replacing a singular argument with a list. Taking this simple sine tone:

local mono = Def{
  SinOsc{ freq=440 } * Env{ dur=1 }
}

We can replace the freq argument of SinOsc with a list, to produce a stereo version:

local stereo = Def{
  SinOsc{ freq={440, 550} } * Env{ dur=1 }
}

In this case, the multiply expression implicitly becomes stereo also, since the first input (the sinewave) is stereo.

For most expression objects, including mathematical operators, the number of output channels will match the greatest number of input channels for any of its inputs, in order to prevent loss of information. Some expression operators calculate their number of output channels differently – for example spatialization expressions such as Pan2.

Where an input stream doesn’t have enough channels for the output stream, input stream channels cycle back again. So, for example, if an expression outputs a four-channel stream, but an input is only two-channel, these inputs will map to outputs as 1, 2, 1, 2. If you’ve used SuperCollider before, this should all seem familiar.

To select individual channels or re-map channels, the Channels expression generator can be used. Very simply, it takes a multi-channel expression as first argument, and one or more channel number arguments to determine which channels of the input stream will be output. For example, to select only the right channel:

local osc = SinOsc{ freq={440, 550} } -- stereo
local right = Channels{ osc, 2 }    -- mono

And to swap left and right channels:

local swapped = Channels{ osc, 2, 1 }

A more common need is to mix a multi-channel stream down to mono or stereo; the Mono and Stereo expressions make this easy:

-- a four-channel stream:
local expr = SinOsc{ freq={440, 550, 660, 770} } * Env{ dur=1 } * 0.25

-- no mix; on a stereo system, you will only hear the first 2 channels:
local mono = Def{ expr }

-- mix to mono (all channels summed):
local mono = Def{ Mono{ expr } }

-- mix to stereo (left gets channels 1 & 3, right gets channels 2 & 4):
local stereo = Def{ Stereo{ expr } }

Multi-channel parameters

Parameters can also be expanded in a similar way. In the following example, the freq parameter is implicitly set to be stereo by assigning a list of two default values. Consequently any use of P”freq” in the expression is interpreted as a stereo stream:

local pstereo = Def{
  dur = 1,
  freq = { 440, 550 },
  SinOsc{ freq=P"freq" } * Env{ dur=P"dur" }
}

Since the parameter is multi-channel, we can assign multi-channel values to it; again, by assigning a list of values.

local voice = pstereo()
for i = 1, 10 do
  voice.freq = { i * 110, (10-i) * 110 } -- set left & right channels to different frequencies
  wait(0.1)
end

If the list is too long, extra values will be ignored (it is not possible to change the number of channels of a parameter). If the list contains nils, these channels will not be updated (the previous parameter value for that channel will still hold). In the following example, left and right frequency parameters are updated alternately:

local voice = pstereo{ dur=2 }
for i = 1, 10 do
  voice.freq = { nil, (10-i) * 110 } -- set right channel only
  wait(0.1)
  voice.freq = { i * 110, nil } -- set left channel only
  wait(0.1)
end

Setting a multi-channel parameter with a single number however will apply that value to all channels:

local voice = pstereo()
voice.freq = 550 -- sets all channels to 550Hz

Panning

Distributing channels can be more rich though the use of panning laws and spatialization techniques. More will be added to this tutorial soon, but for now, here’s the simplest example of a panning oscillator:

local panning = Def{
  Pan2{
    SinOsc{  440 } * Env{ dur=1 },
    pan = SinOsc{ 4 }
  }
}
Posted in Tutorials | 1 Comment

Audio tutorial 1: making sound

Audio synthesis in LuaAV is designed to be efficient yet flexible, and tightly linked into the timing system of LuaAV scripts. However, compared to events and function calls in Lua, or rendering a frame of graphics to a window, producing sound is a relatively continuous process. While there might be typically around thirty updates to a window per second, there will be more than forty thousand consecutive frames of sample data output to the audio device drivers each second. It just isn’t possible to produce this data quickly enough, and reliably enough, directly in Lua. Instead, we create objects running in a background audio engine to produce audio signals as more or less complex functions of time. These objects are defined by combining simple generators into more complex compound expressions. They can be used in the Lua script in a form very much like standard Lua tables, but in fact the expressions they represent are implemented directly in machine code for efficiency.

Using the audio.Def module follows a workflow of three or four stages:

  1. create a compound expression object from built-in expression generators
  2. use the Def function to turn this expression object into a synth constructor (JIT compiled)
  3. call this synth constructor to create active voices (generating sounds)
  4. optional: call methods on the voice objects to change their parameters or stop them

Building expressions

Before we can build audio expressions in Lua, we need to load them into our script. The main module is audio.Def. We’re also going to make use of a few of the expression generator functions in this module, so it is good practice to make local references to these at the top of the script:

local Def = require "audio.Def"

local SinOsc = Def.SinOsc
local Env = Def.Env
local P = Def.P

Now we can start to define the sample-generating process of a synthesizer, starting from basic expression generators. Most audio expression generators take a single table of arguments, so that arguments are identified by name. The Env function creates an expression object representing a decaying ramp (from amplitude 1 to amplitude 0) over a duration specified by the dur argument. The SinOsc function creates an expression object representing a sinewave as function of time, at a frequency given by the freq argument.

local env = Env{ dur = 1 }
local osc = SinOsc{ freq = 440 }

print(osc) --> Expr(SinOsc)

Note that these expression objects are simply data-structures for building specifications, and do not themselves make any sound!

We can use the standard mathematical operators on these expression objects to return compound expression objects:

local expr = osc * env
print(expr) --> Expr(Mul)

We can also build more interesting compound expression objects by using other expression objects as arguments. As a simple example, here’s a frequency sweep using the same Env input:

local env = Env{ dur = 1 }
local osc = SinOsc{ freq = 440 * env }
local expr = osc * env

Creating Synth constructors using the Def call

To turn these expressions into something that can actually generate audio, we call the Def module as if it were a function.

local mysound = Def { expr }

print(mysound) --> function

Def takes the expression (and other parameters as we shall see later) and does some parsing, inferring, code-generating and compiling in the background, creating an efficient machine-code representation of the expression for the purposes of synthesizing audio sample streams. In a way, it makes a particular compound expression become ‘concrete’. Note that there is no limit to how many different Def-compiled synthesis definitions you can use in your script.

The call to Def returns a function that can be called to create new instances (‘voices’) of this concretized synthesis definition. Now we are ready to make sound.

Creating voices from a synthesis definition

The synth constructors returned from calls to Def create active voices in the audio engine. Each voice is a different object that can be started, stopped and modified independently and at different times:

for i = 1, 4 do
  -- launch a voice:
  mysound()
  -- wait some time before launching the next one:
  wait(1/i)
end

Adding parameters to the definition

We can vary the sound between voices by using parameters in the synthesis definition. The parameters are declared in the hash portion of the table of arguments in the Def call, along with default values. These parameters can then be referred to in the compound expression using the P function. In effect, P”foo” function creates an expression representing ‘the value of parameter foo’.

local mysound2 = Def{
  -- the parameter defaults:
  amp = 0.25, dur = 1, freq = 440,
  -- the compound expression object:
  SinOsc{ freq = P"freq" } * Env{ dur = P"dur" } * P"amp"
}

Now these parameter names can be used when creating voices:

for i = 1, 4 do
  -- launch a voice:
  mysound2{ freq = i*330, dur = 1/i }
  -- wait some time before launching the next one:
  wait(0.5)
end

They can also be used to change the value of a voice parameter while it is playing. Though we haven’t used it yet, the call to the synthesis constructor returns a voice object in Lua (which acts as a proxy to the sounding process in the audio engine). Every parameter name is assignable on this voice object.

local voice = mysound2{}
print(voice) --> Synth

for i = 1, 10 do
  -- parameters of a voice can be set:
  voice.freq = math.random(10) * 110
  wait(0.1)
end

-- parameters can also be read:
print(voice.freq) --> 660

Note that it is perfectly valid to define an expression object using P parameters before the Def call is made:

local expr = SinOsc{ freq = P"freq" } * Env{ dur = P"dur" } * P"amp"
local mysound2 = Def{
  amp = 0.25, dur = 1, freq = 440,
  expr
}

It is also perfectly valid to use a parameter name that is not specified in the Def arguments. In that case, the parameter still exists, but defaults to a value of zero.

Stopping a synth

If you have been experimenting with building different compound expressions, you may have noticed that when no Env is used in the expression, the voices never stop playing (even if the voice object is fit for garbage collection). Every voice has a :stop() method, which can be used to stop the sound from playing. After calling :stop() on a voice, it will no longer respond to any other changes, and cannot be restarted.

local mysound3 = Def{
  amp = 0.25, freq = 440,
  -- the compound expression object (Note: has no Env):
  SinOsc{ freq = P"freq" } * P"amp"
}

local voice = mysound3{ freq = 1210 }
wait(1)

-- make it stop!
voice:stop()

Calling :stop() was not necessary for the definitions that embedded Env expression objects, because Env effectively triggers the same effect as :stop() automatically when it reaches the envelope end (the specified duration).
Note: this behavior of Env can be disabled or ‘paused’ using the done and hold arguments (see the Reference documentation for more details).

Posted in Tutorials | 1 Comment

polar^m [mirrored]

This gallery contains 2 photos.

The polar^m [mirrored] landscape explores the noise intelligence present in ephemeral and apparently random radiation phenomena through micro and macro transitions Continue reading

More Galleries | 2 Comments

Array Is the New Lattice

For a long time now, we’ve had an ongoing internal debate over what to call the data structure we use to pass large chunks of memory around between audio and graphics processing code. We’ve variously used names like Matrix, Buffer, Lattice, and Array, but never really settled on one particular name. Depending on if you’re writing audio, 3D graphics, image processing, or mathematical software, the names that makes sense will change.

Our data structure can be any basic type (char, int, float, double and all their variations), have a number of different components (XYZ, RGBA, …) and also be multi-dimensional.  Originally, we used the term Matrix in the graphics code, but then moved to using Lattice because it was already being used in another code base that we were merging in to LuaAV.  Lattice wasn’t quite the right word though since it implies something more akin to crystal structures and is not used in computer science at all for what we were trying to describe.

In the end, we took a vote and Array came out on top, so everything in our code that used to be Matrix or Lattice is now Array.  For example, loading a texture with an image now looks like:

local img = Image("banana.jpg")
local tex = Texture(ctx)
tex:fromarray(img:array())

Array ended up being the most flexible and closely matched term. A Matrix, for example, is defined as 2D, and a Lattice usually refers to sets or groups. It’s interesting to look at the origins of these terms and compare them to current usage.

Matrix

late 14c., from O.Fr. matrice, from L. matrix (gen. matricis) “pregnant animal,” in L.L. “womb,” also “source, origin,” from mater (gen. matris) “mother.” Sense of “place or medium where something is developed” is first recorded 1550s; sense of “embedding or enclosing mass” first recorded 1640s. Logical sense of “array of possible combinations of truth-values” is attested from 1914.

In mathematics, a matrix is a rectangular array of numbers. Matrices consisting of only one column or row define the components of vectors, while higher-dimensional (e.g., three-dimensional) arrays of numbers define the components of a generalization of a vector called a tensor.

Lattice

c.1300, from O.Fr. latta “lath,” from Frank. (cf. O.H.G. latta “lath”). See lath.lathO.E. *laððe, variant of lætt “lath,” apparently from P.Gmc. *laþþo (cf. O.N. latta, M.Du., Ger. latte “lath,” M.H.G. lade”plank,” which is source of Ger. Laden “counter,” hence, “shop”).but also note the related term cancel:late 14c., “cross out with lines,” from Anglo-Fr. canceler, from L. cancellare “to make resemble a lattice,” which in L.L. took on a sense “cross out something written” by marking it with crossed lines, from cancelli, pl. of cancellus”lattice, grating,” dim. of cancer “crossed bars, lattice,” a var. of carcer “prison.”

In mathematics, a lattice is a partially ordered set (also called a poset) in which any two elements have a unique supremum (the elements’ least upper bound; called their join) and an infimum (greatest lower bound; called their meet).

In physics, a lattice (group) model is a physical model that is defined on a lattice, as opposed to the continuum of space or spacetime.

In geometry and crystallography, a Bravais lattice is an infinite set of points generated by a set of discrete translation operations (not necessarily mutually orthogonal).

In mathematics, especially in geometry and group theory, a lattice in Rn is a discrete subgroup of Rn which spans the real vector space Rn. Every lattice in Rn can be generated from a basis for the vector space by forming all linear combinations with integer coefficients. A lattice may be viewed as a regular tiling of a space by a primitive cell.

Buffer

1835, from obsolete verb buff “make a dull sound when struck” (mid-16c.), from O.Fr. buffe “a blow;” hence “something that absorbs a blow.”

In computer science, a buffer is a region of memory used to temporarily hold data while it is being moved from one place to another.

Array

c.1300, from O.Fr. areer “to put in order,” from V.L. *ar-redare (cf. It. arredare), from L. ad- “to” (see ad-) + Frank.*ræd- “ready” (cognate with Goth. garadis, O.E. geræde “ready;” see ready).

In computer science, an array data structure or simply array is a data structure consisting of a collection of elements (values or variables), each identified by one or more integer indices, stored so that the address of each element can be computed from its index tuple by a simple mathematical formula.

Posted in Development | Tagged | Leave a comment

Timing & Scheduling

LuaAV has a powerful timing system, using on an internal scheduler that preserves deterministic ordering and logical timestamps to nanosecond accuracy. The deterministic ordering and accuracy is preserved in many messages to the audio system, such as adding/removing synths.

The main scheduler follows the cpu clock as closely as possible, usually within and accuracy of around 10 milliseconds; however when slow functions are called (such as loading files and creating complex resources such as windows), the scheduler may experience a temporary drop-out, from which it will attempt to recover as soon as possible.

The Lua language itself does not have means to control time, however this has been added in LuaAV via the LuaAV.time module. Several of the functions in this module are so useful that they are pre-loaded as globals in every script:

  1. now()
  2. wait()
  3. go()
  4. event()

Printing out now() in a new script will return the logical time (in seconds) since the script was loaded. Until we start scheduling with time, all script actions occur immediately, so now() will return 0.

The wait() function allows us to pause the execution of the script for a number of seconds, after which it will continue. The following script will print out the current logical time every 0.1 seconds (100 ms):

local period = 0.1
while true do
  print(now())
  wait(period)
end

This is ok, but there’s nothing we can do in between each wait, so this kind of control over time is still quite minor.

Fortunately, Lua provides a way to create parallelism within a script, using a coroutines. One way of thinking about a coroutines is that it is like a parallel function or script state; another way to think about it is as a function that can be paused in mid execution, while Lua goes off to execute some other code, and to later returned to (resume) at the point at which it paused (yielded).

LuaAV adds more power to coroutines by connecting them with the scheduler. A convenience function go() will take a function and arguments, create a coroutine based on that function and arguments, and schedule this coroutine within the scheduler. Now we can create many parallel copies of the same function that can be scheduled alongside each other, each with potentially distinct timing, but without losing deterministic accuracy:

function clockprinter(name, period)
  while true do
    print(now(), name)
    wait(period)
  end
end

go(clockprinter, "T O C K!", 4)
go(clockprinter, "...tick...", 1)

The go() function can also take an optional first argument (delay in seconds), which allows us to schedule it to occur at some point in the future:

 go(2, clockprinter, "...tick...", 1) -- will start 2 seconds later

Note that the even if the delay is 0, or is not given, the coroutine will not run immediately; go() simply adds the coroutine to the internal scheduler. (Lua is single-threaded by design, which means that only one actual function is executing at any time.) The scheduled function will not begin running until the context in which it was launched using go() yields with a wait() command, or reaches the end of the script.

Scheduling with events

Sometimes we want to schedule activity to occur not at a given time, but when a given situation occurs. To support this, the go() and wait() functions can also take a string argument in place of a duration. The string represents a unique event.

The event() function can then be used to resume ALL coroutines that were scheduled against or waiting upon a particular event. A classic use-case of this is to make sure that OpenGL rendering commands only execute during a window’s draw() method. The following example shows how a function drawstuff() is scheduled to execute only when the draw event occurs, and once it does, to wait until subsequent drawevents to continue rendering:

local gl = require("opengl")
local GL = gl

win = Window("test")

function drawstuff()
  while true do
    gl.Begin(gl.LINES)
      gl.Vertex(-math.cos(now()), math.sin(now()), 0)
      gl.Vertex(0, 0, 0)
    gl.End()
    wait("draw")
  end
end

go("draw", drawstuff)

function win:draw()
  -- resume any functions waiting on the draw event
  event("draw")

end

The event() function can also take additional arguments; these arguments are returned by any corresponding wait(). Using this feature quite powerful event-based programming systems are possible.

Script-controlled schedulers

Users can create their own schedulers, which are independent of CPU clock, using the LuaAV.time.scheduler() function. A scheduler is an object that provides its own scheduler.now(), scheduler.wait() and scheduler.go() functions. The logical time of this user-controlled scheduler is not tied to the CPU clock, but instead is advanced within the script using the scheduler.advance() or scheduler.update() functions. For more information, see the LuaAV.time module in the reference documentation.

Timing and the OpenGL Window

Callbacks to draw the OpenGL window might not be scheduled with this tight accuracy, since they are dependent on timing factors outside their control (such as graphics drivers and GPU load). However, they will occur as closely as possible to the rate specified by the win.fps attribute.

Posted in Tutorials | 1 Comment

Lua Tutorial

What is Lua?

From wikipedia:

Lua ( /ˈluː.ə/ LOO-ə) is a lightweight, reflective, imperative and functional programming language, designed as a scripting language with extensible semantics as a primary goal. The name comes from the Portuguese word lua meaning “moon”.

What’s the difference between a static language (like C or C++) and a dynamic language like Lua?

Dynamic programming language is a term used broadly in computer science to describe a class of high-level programming languages that execute at runtime many common behaviors that other languages might perform during compilation, if at all. (wikipedia)

For example, where a variable in C has a type bound to it which cannot change (static typing), a variable in Lua is just a name under which any type of value can be stored. (In a way, all variables in Lua behave like pointers.) More interestingly, any string of valid Lua code can be executed from within another script; opening the possibilty of generating new functions at run-time. Data structures can change size and layout during execution, and functions can be passed around just like any other object, all according to the vagaries of user input. (Some of these features are possible in C/C++ with a lot of clever coding, but the cleverer that code gets, the more it becomes like the virual machine of an interpreted langauge anyway…)

What’s exciting about Lua?

Fast (one of the fastest dynamic languages). “Several benchmarks show Lua as the fastest language in the realm of interpreted scripting languages.” For this reason in particular, it is perhaps most widely used for game scripting. In addition, the LuaJIT project uses a trace compiler to convert hot code paths and loops into i386 or x86_64 machine code, with performance often on par with C/C++.

Expressive. Lua began as a data-description language, and continues to benefit from a universal and flexible array/hash data structure. The syntax is largely procedural, however it supports fully functional programming features such as first-class functions, closures and coroutines, and lexically scoped upvalues (granting capabilities similar to Scheme). It also supports various models of inheritence through a prototype inheritence chain (similar to Self and JavaScript). The runtime is also fully re-entrant.

Easy to learn. Really, the majority of the language’s grammar/syntax/typical usage can be learned in one day.

Extensible. These expressive features, plus a dynamic library module system (binary or Lua code) make it easy to extend Lua for a particular project or task.

Small. Lua as a static library is just a few hundred K, making it ideal for embedding within a host program.

Simpler C API. The simplicity of binding Lua to C code is often mentioned as a key attraction of Lua (as opposed to alternative embeddable langauges such as Python andRuby).

Portability. The Lua core is ANSI C compliant: that means it should run on almost any system these days. A lot of effort is currently going into improving Lua on microcontrollers.

Lua is licensed under the MIT license, so there are basically no restrictions on embedding, deriving or modifying it.

What’s not so exciting?

Limited set of standard libraries. Compared to Perl or Python, Lua has very few built-in libraries, and a reasonable but not outstanding set of third-party libraries. The chances of your favourite C library being bound to Lua are much less…

No built-in threading. Deterministic multi-tasking is fully supported using coroutines, but if you want to make use of your processors, you need to make use of third party libraries that might not always be so stable.

Array indexing starts from 1. A feature inherited from its background in data-description, but a gotcha for programmers coming from a background in C…

Documentation and Resources

Programming in Lua

This book is *excellent*. Get it from here:

Programming in Lua (second edition)
by Roberto Ierusalimschy
Lua.org, March 2006
ISBN 85-903798-2-5 (also available as an e-book)

Programming in Lua is also available in German, Korean, Chinese, and Japanese.

The Lua 5.1 Reference Manual

Another excellent resource. Fortunately, it is available online: http://www.lua.org/manual/5.1/.

Lua 5.1 Reference Manual
by R. Ierusalimschy, L. H. de Figueiredo, W. Celes
Lua.org, August 2006
ISBN 85-903798-3-3

In particular, this is the URL that should be bookmarked: http://www.lua.org/manual/5.1/index.html#index

The lua-l Mailing List

http://vlists.pepperfish.net/cgi-bin/mailman/listinfo/lua-l-lists.lua.org

Archive here: http://lua-users.org/lists/lua-l/

Quite a diversity of low to high level discussion, but 500-1000 messages per month!

Lua Wiki

Some tutorials here: http://lua-users.org/wiki/TutorialDirectory

This unofficial FAQ may also be useful: Steve Donovan’s unofficial Lua FAQ

The Lua Language

Hello World

--[[ this is a multi-line comment--]]
-- this is a single-line comment

print("hello world")

Basic Types

All values in Lua are one of the following types; however unlike C, the type does not belong to the variable, but to the value itself. That means that a variable can change type (dynamic typing).

Numbers

All numbers in Lua are 64-bit floating point type (aka ‘double’ for C programmers). There is no distinction between integers and non-integers.

-- these lines are all equivalent:
-- they assign the number value 1 to the variable name x:
x = 1
x = 1.0
x = 100e-2  -- e base10 format
x = 0x1 --hexadecimal format

Strings

Lua strings are immutable: each string operation creates a new string. (strings are hashed internally very efficiently, and garbage collected).

print("a simple string")
print('a simple string')

-- embedding special characters and multi-line strings:
x = "a string isn't stopped by \n a new line";
x = 'a string isn\'t stopped by \n a new line'
x = [[a string isn't stopped by
a new line]]

Boolean and nil

Boolean values are the keywords true and false. The nil value indicates the absence of a value. Assigning nil to a variable effectively marks the variable for garbage collection.

t = true
f = false
n = nil

-- nil evaluates to false for a predicate:
if t then print("t!") end -- prints t!
if f then print("f!") end -- prints nothing
if n then print("n!") end -- prints nothing

Tables

Table is the only structural type in Lua: all data structures are made from tables, and even a large part of Lua’s own implementation relies upon them.

Tables in Lua are associative arrays, mapping keys to values. Both keys and values can be any valid Lua type except nil (booleans, numbers, strings, other tables (or even the same table!), functions, coroutines and so on.)

local t = {}  -- empty table constructor
t[1] = "one"  -- assignment
t["two"] = 2  -- keys and values can be any types
t[true] = function() print("true!") end

-- except nil:
t[x] = nil  -- removes t[x] reference
t[nil] = 1  -- error!
In particular, string key lookup on a table has a useful short-hand form:

local t = {}  -- empty table constructor
t[1] = "one"  -- assignment
t["two"] = 2  -- keys and values can be any types
t[true] = function() print("true!") end

-- except nil:
t[x] = nil  -- removes t[x] reference
t[nil] = 1  -- error!

It’s important to note that a Lua table has two parts; an array portion and a hashtable portion. The array portion is indexed with integer keys, starting from 1 upwards while the integer keys are contiguous. All other keys are stored in the hash (or record) portion.

Array Portion

The array portion gives Lua tables the capability to act as ordered lists, and can grow/shrink as needed (similar to C++ vectors). Sometimes the array portion is called the list portion, because it is useful for creating lists similarly to LISP. In particular, the table constructor will insert numeric keys in order for any values that are not explicitly keyed:

-- these two lines are equivalent
local mylist = { [1]="foo", [2]="bar", [3]="baz" }:
local mylist = { "foo", "bar", "baz" }

print(mylist[2]) -- prints bar

To walk over every key/value in the table, there is the pairs form. Note however, that the order of traversal cannot be predicted:

local t = { "foo", "bar", x = 1, y = 2 }
for key, value in pairs(t) do
  print(key, value) -- prints all keys and values
end

Functions

Unlike C, functions are first-class values, just like numbers, strings and tables. That means that functions can be keys and values in tables, functions can take functions as arguments, and return functions as return values. This might seem exotic at first, but can be very valuable.

A simple function:

-- these are equivalent:
sayhello = function(message)
  print("hello", message)
end

function sayhello(message)
  print("hello", message)
end

-- using the function:
sayhello("me")  -- prints: hello me
sayhello("you") -- prints: hello you

-- replacing the function
sayhello = function(message)
  print("hi", message)
end

sayhello("me")  -- prints: hi me

Of course, a function can have more than one argument… but it can also have more than one return value:

function minmax(a, b)
  return math.min(a, b), math.max(a, b)
end
print(minmax(42, 13)) -- prints: 13 14

A special syntax is available for a table’s member functions that are intended to be used as methods. The use of a colon (:) instead of a period (.) to index the table passes the table itself through as a special ‘hidden’ variable self (similar to the hidden variable this in C++).

local t = { value = 10 }

-- define a method:
t.printvalue = function(self)
  print(self.value)
end

-- equivalent:
function t:printvalue()
  print(self.value)
end

-- use the method:
t.printvalue(t) -- prints: 10

-- equivalent:
t:printvalue()  -- prints: 10

Control Flow

Lua includes a standard set of control flow structures.

-- if blocks:

if x == 1 then
  print("one")
  -- as many elseifs as desired can be chained
elseif x == 2 then
  print("two")
elseif x == 3 then
  print("three")
else
  print("many")
end

-- while loops:
x = 10
while x > 0 do
  print(x)
  x = x - 1
end

repeat
  print(x)
  x = x + 1
until x == 10

Logical Operators

  • Equals: ==
  • Not-equals: ~=
  • Comparatives: < <= >= >
  • Logical combinators: and or not
if x > 12 and x <= 20 then print("teen") end

Variables and Scoping

Variable names in Lua can be any string of letters, digits, and underscores, not beginning with a digit (the same as most other languages). Some names are reserved in the language itself, including:

and       break     do        else      elseif
end       false     for       function  if
in        local     nil       not       or
repeat    return    then      true      until     while

Local and global; lexical scoping

New identifiers (variables) can be defined to be local to a particular block (code chunk), or global. Defining an identifier as local is done using the local keyword. Local identifiers are not visible outside the block in which they were declared, but are visible inside sub-blocks. This is called lexical scoping.

function foo(x)
  -- a function body is a new block
  local y = "mylocal"
  if x == y then
    -- this is a sub-block of the function
    -- y is still visible here
    print(y)  -- prints: mylocal
  end
end
-- y is not visible here:
print(y)    -- prints: nil

Assigning to a variable that has not been declared locally within the current block will search for that name in parent blocks, recursively, up to the top-level. If the name is found, the assignment is made to that variable. If the name is not found, the assignment becomes a global (either creating a new variable, or replacing an existing global). Global identifiers are stored in the implict globals table, which can be explicitly accessed through the name _G.

-- an outer variable:
local x = "outside"
print(x) -- prints: outside

-- sub-block uses a local, which does not affect the outer variable:

function safe()
  local x = "inside"
end
safe()
print(x) -- prints: outside

-- sub-block does not use a local, and overwrites the outer variable:
function unsafe()
  x = "inside"
end
unsafe()
print(x) -- prints: inside

It is good practice to make as many variables local as possible since it greatly improves performance. More importantly, unexpected assignment to non-local variables is probably the most common cause of Lua scripting errors!

Garbage Collection

Objects are never explicitly deleted in Lua (though sometimes resources such as files might have explicit close() methods). Lua has an fast incremental garbage collector that runs in the background, which silently recycles the memory for any values to which no more references remain. For most built-in types (strings, tables etc) this need never concern us. When binding C/C++ objects as full userdata types, the garbage collect event can be handled through a special metamethod.

Going Deeper

Higher Order Functions

Functions as return values:

function operate(operator)
  if operator == "add" then
    return function(a, b)
      return a + b
    end
  elseif operator == "mul" then
    return function(a, b)
      return a * b
    end
  end -- end if
end -- end function operate

local f = operate("mul")
f(2, 10) -- prints: 20

Functions as table values:

local t = {
  add = function(a, b)
    return a + b
  end,
  mul = function(a, b)
    return a + b
  end,
}

t.mul(2, 10) -- prints: 20

In fact, when a function is declared as a global variable, it is in fact just an assignment to an implicit globals table. This table can be explicitly accessed using the special name _G:

-- these are all equivalent:
function foo() print("foo") end
foo = function() print("foo") end
_G.foo = function() print("foo") end

_G["foo"] = function() print("foo") end

Closures

Closures can arise from the mixed use of lexically scoped local variables, and higher order functions. Any function that makes use of non-local variables effectively keeps those variable references alive within it. An example explains this better:

function make_counter()
  local count = 0
  return function()
    count = count + 1
    print(count)
  end
end

-- call to make_counter() returns a function;
-- and 'captures' the local count as an 'upvalue' specific to it
local c1 = make_counter()
c1()  -- prints: 1
c1()  -- prints: 2
c1()  -- prints: 3

-- another call to make_counter() creates a new function,
-- with a new count upvalue
local c2 = make_counter()
c2()  -- prints: 1
c2()  -- prints: 2

-- the two function's upvalues are independent of each other:
c1()  -- prints: 4

This technique can be incredibly powerful. An object constructor and its private data can be entirely ecapsulated through the use of upvalues, for example.

Coroutines

Coroutines are a form of collaborative multi-tasking. You can think of them as functions that can be paused in mid execution, to be resumed at that position at a later time.

The C programmer can think of them as similar to threads, however they are explicitly paused and resumed from within a script (rather than by the operating system), and do not make use of CPU multithreading capabilities.

A coroutine is created from an existing function using coroutine.create(), and is resumed using coroutine.resume(). It can pause itself with coroutine.yield(). In addition, values can be passed back and forth via the arguments to coroutine.resume() and coroutine.yield().

local resume, yield = coroutine.resume, coroutine.yield

-- this function will be used to create a coroutine:

local function loop()
  print("hello!")
  local x = 0
  while true do
    -- pause function here:
    yield(x)
    -- continues here:
    x = x + 1
    print(x)
  end
end

-- create the coroutine:
local c = coroutine.create(loop)

-- the first resume runs from the start of the loop() function to the first yield():
coroutine.resume(c) -- prints: hello!

-- each subsequent resume runs from the last paused yield() to the next yield():
coroutine.resume(c) -- prints: 1
coroutine.resume(c) -- prints: 2

In LuaAV, coroutines are extended for accurate temporal scheduling, using the gowait and now functions.

Advanced Topics

Metatables

Lua does not provide a class-based object-oriented system by default; instead it provides the meta-mechanisms with which many different kinds of object-oriented programming styles can be implemented.

There are several special events that can apply to objects (usually tables and userdata); the default behavior for these events can be overridden by means of a metatable. A metatable is just an ordinary table with some reserved key names bound to functions (metamethods) to specify this variant behavior. Any table or userdata can have its metatable set; some objects may share a metatable.

For example, the __add metamethod defines what happens when two objects are added to each other:

-- a metatable for pairs
local pair_meta = {}

-- a metamethod function for how to add two pairs together:
pair_meta.__add = function(a, b)
local p = {
  a[1]+b[1],
  a[2]+b[2],
}
-- result is also a pair:
setmetatable(p, pair_meta)
  return p
end

-- a constructor for pairs:
function make_pair(x, y)
  local p = { x, y }
  -- tell p to look in pair_meta for how to handle metamethod events:
  setmetatable(p, pair_meta)
  return p
end

-- create two pairs:
local p1 = make_pair(2, 3)
local p2 = make_pair(4, 5)

-- add them (creates a new pair):
local p3 = p1 + p2
print(p3[1], p3[2]) -- prints: 6 8

Arithmetic operator metamethods also exist for __mul, __sub, __div, __mod, __pow, __unm (unary negation).

The __index metamethod is important: if a key cannot be found in a given table, it will try again in whichever object the __index field points to; or call the function if __index points to a function. This is the principal way that inheritence (of both class data and methods) is supported:

local animal = {}

function animal:isalive() print("yes!") end

local dog = {}
function dog:talk() print("bark!") end

-- create metatable for dog, that refers to animal for unknown keys:
local dog_meta = { __index = animal }

-- apply metatable to dog:
setmetatable(dog, dog_meta)

-- test it:
dog:talk()  -- prints: bark!
dog:isalive() -- prints: yes!

animal:talk() -- error!

A corresponding __newindex metamethod exists to handle assignments of new keys to an object.

Other metamethods include __tostring to convert an object to a string (in the print() and tostring() functions), __eq, __lt and __le for logical comparisons, __concat for the .. operator, __len for the # operator, and __call for the () operator.

By combining all of these metamethods, and smart use metatables, various forms of class based inheritance can be designed. Several examples can be found here.

Userdata and the C API

Userdata are objects that encapsulate arbitrary C/C++ data within a Lua interface. Many Lua modules extend the capabilities of Lua by binding external libraries, including the creation of new types as userdata. For example, the built-in module io includes means to create, and modify files on the system, which are represented as userdata.

There are two kinds of userdata: lightuserdata are simply raw pointers, and there is nothing that can be done with them (they have no methods, and are immutable), except for passing them into other C functions. They are rarely used. Full userdata on the other hand behave more like tables, with possible metatables, and local environment data; they are also subject to garbage collection and have a special __gc metamethod for it.

Binding C functions and objects to Lua is done via the C API only, and cannot be done from within Lua itself. It is a complex subject and beyond the scope of this tutorial!

Posted in Tutorials | 2 Comments