LuaAV is an integrated programming environment based upon extensions to the Lua programming language to enable the tight real-time integration of computation, time, sound and space.  It consists of an application for executing Lua scripts and a collection of libraries and Lua modules for sonic, visual and spatial computation.

Core Functionality

  • Interleaved scripting of sound and graphics
  • Precise scheduler with both synchronous and asynchronous timing models
  • Real-time scripting and code generated compilation of audio synthesis routines
  • Cross-platform Lua modules for portable scripting

Essential Lua Modules

  • Array: create and manage large blocks of memory such as images, audio files, geometry data, etc.
  • audio: define and generate audio synthesis routines
  • font: load and process font files using FreeType
  • gui: OpenGL-based GUI toolkit
  • Image: image file I/O
  • midi: MIDI I/O
  • opencl: high-level bindings to the OpenCL framework
  • opengl: bindings to OpenGL C API and high-level abstractions for textures, shaders, meshes, and common drawing routines
  • osc: Open Sound Control network messaging
  • space: spatial computation module containing vector math, quaternions, and spatial partitioning and collision functionality
  • video: video camera control, video file playback, and video recording

Code Generation

One of the major design principles behind LuaAV is to not sacrifice performance for flexibility.  We achieve this using run-time code generation and just-in-time (JIT) compilation.  The audio system in particular uses code generation and JIT compilation heavily so that synthesis routines can be generated at run-time while achieving the performance of pre-compiled code.

The main technology behind the JIT compilation is the Low-Level Virtual Machine (LLVM) and it’s C/C++ frontend project Clang.  Together, they allow you use to generate C/C++ code and immediately compile and link it into LuaAV while it’s running so that new functionality can be added to a Lua script at run-time.    Essentially, can use the JIT capabilities of LuaAV to live code C/C++.


LuaAV attaches a scheduler to each script it executes.  The scheduler models both synchronous and asynchronous notions of time by extending Lua coroutines with temporal semantics.  This makes it easy to descrive both periodic tasks such as “play this not every 1/10th of a second” or event based tasks like “change a color when a note is plated”.

LuaAV Application

The LuaAV application is essentially a runtime environment for Lua scripts.  It can open, execute, and reload multiple scripts at a time, but it does not have a built-in editor.  Instead, has the operating system tell it when a script it is executing is modified to reload it and show the updates, meaning scripts running in LuaAV can be edited with your favorite text editor instead.

The only graphical interface LuaAV provides on its own is a console window to manage executing scripts, display information printed out by any scripts and display error messages.

Leave a Reply