TOI (Thought-Obedient Instruments): Project Specification
“I dream of instruments obedient to my thought and which with their contribution of a whole new world of unsuspected sounds, will lend themselves to the exigencies of my inner rhythm.” - Edgard Varese
Thought-Obedient Instruments (TOI) is a collection of unconventional synthesizers packed into a VST plugin.
It is a mix of experimental synthesis techniques and ideas that each may have varied levels of musical applicability, but in the end it is an experiment to see what might work.
Overview
We, the developers, are making use of some tools that we were already familiar with prior to this project, and we are also learning new things too.
We are somewhat experienced with the JUCE framework that we are using for our user interface, though there are also parts of it we are becoming more familiar with. Some parts of the codebase make use of programs we have written for past courses, such as the envelope code and the general structure we are using to abstract our sound sources and filters. We are also processing MIDI data, which is not very new to us.
Of course, all of our experimental synthesis and filtering techniques are new, and we are learning about them as we go. In addition to the generation and filtering algorithms that we are figuring out, we are also learning about the intricacies of real-time filtering, as in the past we have only programmed filters designed to work on pre-recorded clips. Making TOI a plugin instead of a standalone synthesizer is partially handled by the JUCE framework, but we are learning how to work with this interface effectively.
Algorithm Implementation
Quadratic Spline
This is a relatively simple sound source in which a spline of two quadratic functions is used to replicate a sine wave. The shape of the waveform is not precisely sinusoidal, so there are some harmonics present in the resulting signal. The implementation is very simple, just using the sample sine oscillator structure from the JUCE example code, but replacing calls to the standard sine function with calls to a quadratic spline function, containing two quadratic mathematical functions.
Fourier Scanner
This is, at its core, a modulatable Fourier transform of an input sample, translated back into the time domain as a sum of sine signals. The user will have control over the window span over frequency and time and the transform resolution. Ultimately, these parameters will be modulatable by automated sources, such as envelopes and LFOs. By scanning through the input sample, the user can pick out different tones over the sample’s time or frequency. Short windows will generate sustained playable tones representing miniscule timbres only fleetingly present in the original sound.
With a ramp LFO modulating the window position, this synthesis is similar to time-stretching, as described in DAFX. With alternate modulation shapes of the various parameters, this algorithm will likely generate some interesting musical tones.
Spline Resample
This generator makes a waveform by calculating a 1-dimensional B-spline curve. The spline’s control points are chosen by sampling points in a source waveform, meaning this generator acts as an interpolation of another wave.
User Interface
Being primarily an instrument plugin, user interface is important for TOI. The aim is to have a unified interface with components that allow access to multiple parameter types, of which different combinations will be available for each Generator algorithm.
In addition to on-screen knobs and selection elements, and an on-screen keyboard, TOI supports MIDI input through the host program (DAW, standalone host, etc.). MIDI note messages are supported, and there is basic support for some other MIDI features such as the pitch wheel.
The following is a basic block-out of the planned UI. This figure does not represent the actual final layout.
Figure: Draft layout of the TOI user interface.
The user interface will ultimately consist of the following controls:
Generators:
- Pitch offset
- Pitch track toggle
- Pitch fine tune
- Algorithm selection
- Algorithm parameters
- Attenuator
- Sample selector
Filters:
- Algorithm selection
- Cutoff
- Algorithm parameters
Envelopes:
- ADSR controls
Routing:
- Modulation matrix
Output:
- Level
- Time-signal scope
- Frequency-level scope
- Time-frequency-level scope
Audio Engine
Figure: TOI signal and modulation architecture.
TOI will consist of the following abstract components:
Processor
Handles the overall signal path and audio buffer.
Editor
Hosts the user interface and is the parent that handles the Processor.
Generator
A raw audio-generating signal processing algorithm. Has parameters that are modified by the knobs in the UI.
Sound Source
The interface to host and select Generators. Contains the UI elements for Generators.
Envelope
Generalized ADSR modulation source, routable to an array of parameters.
Filter
An audio-filtering signal processing algorithm, intended to affect Generators.
Modulation Matrix
Interface for routing parameter modulation.
Scope
Visual display of signal over time as well as its frequency magnitude.
Goals
By our first major milestone, the end of week 15, we plan to complete the implementation of 3 sound generation algorithms, MIDI note control, plugin compatibility, and a hardwired amplitude envelope. Currently, the only remaining task among these targets is the addition of 2 generation algorithms, so we are well on track for full milestone completion on time. In case we come across a roadblock that majorly hinders our ability to successfully implement generation algorithms, an alternative goal for us would be to make our plugin an effect plugin instead of an instrument plugin, and only process incoming audio instead of generating sound.
In weeks 16-30, we aim to add more generation algorithms, a filter module, and a modulation system to allow for more dynamic sound shaping.
References
https://docs.juce.com/master/tutorial_code_basic_plugin.html
https://docs.juce.com/master/tutorial_create_projucer_basic_plugin.html
https://azrael.digipen.edu/research/
https://github.com/clone45/EquationComposer
https://azrael.digipen.edu/research/SplineModeling-Klassen-revised.pdf
https://github.com/mattjklassen/TorchAudioSplines