A few weeks ago I started my own re-write of a digital audio synthesis library. I took on this project not to reinvent the wheel, but to understand how this wheel works. Here I want to share some of the things I learned in the process.

I think this is also a fairly good introduction to digital synthesis if, like me, you prefer the hands-on approach to the purely theoretical one.

What is Braids?

Braids is a digital synthesizer Eurorack module from Mutable Instrument.

Synthesizer: Shortcut for audio signal synthesizer, a device that produces sound signals. The signals are usually electrical and require a speaker/headphone to be listened to.
Digital: The audio signal is produced by a computer using numbers and calculation, and then converted to an electrical signal.
Eurorack module: Eurorack is a standard for modular synthesizers. Musicians combine signals from different modules to shape sound.

There are a couple of features that make Braids special:

It’s a digital synthesizer, so the sound is produced with software as opposed to electronic components for an analog synthesizer. Code is far easier to look at and tweak than an electronic circuit, at least that’s my point of view as a software engineer…
It’s open-source (MIT license). Braids, and all the Mutable Instrument modules, are an exception in the audio/synthesizer industry where open-source software is not very common. This means other projects can use Braids code, e.g. Dirtywave’s M8. It also means we can look at the code and understand how it works.
It’s a macro oscillator. That means a single Braids module can produce a variety of relatively complex sounds. As opposed to most modules that will provide one signal and have to be combined with other modules for more complexity.

Listening to Braids

The Braids module is not distributed by Mutable Instruments anymore, but there are couple of options if you want to hear what it sounds like:

VCV Rack: There is a version of Braids available in the Eurorack simulator called VCV Rack. This is the easier/cheapest/fastest way to try Braids.
YouTube: There are plenty of videos showing Braids and its different sounds
Used market: You can get second hand modules on websites likes Reverb.com

Basics of digital audio synthesis

The core principle of any digital sound synthesizer is to generate a series of numbers, which will then be converted to variation of electric current, which in turn will move a speaker cone to produce sound.

The generated numbers are called sample points, they represent signal value from minimum amplitude to maximum amplitude. If you open an audio file with Audacity and zoom in on the waveform, you can see the individual sample points:

There are two important properties for sample points, their bit depth and frequency.

Bit depth is the number of bits used to encode the value. Frequency, usually called sample rate, is the number of samples produced per second. Higher sample rate will produce higher fidelity sound (up to a certain point) but also requires more computing power.

For example CDs are at 16 bit depths 44.1k sample rate, Braids uses 16 bit depth 96k sample rate.

Generating waveforms with lookup tables

Audio synthesis generally starts by producing an oscillating signal at a given frequency, that frequency being the pitch of the note to play. In Braids, this is done by the AnalogOscillator class. Don’t be fooled by the name, Braids is simulating analog synthesis but we are of course in digital territory here. Let’s take, for example, a sinusoidal signal.

Making a sinusoidale signal is very easy using the sin() function. Simple to do, but quite expensive in terms of processor usage. Even more so when the waveform becomes more complex than a sinusoidal. So a very common way to generate signals at a lower CPU cost is to use lookup tables.

A lookup table is a precomputed series of results for a periodic function of arbitrary complexity. This series is computed once during development of the software, and stored in an array in the firmware.

Using a lookup table, getting the result of a function just means reading a value in the corresponding array.

Braids lookup tables are located in ressources.cc and generated by python scripts.

Phase

Looking at Braids’ AnalogOscillator code you will see the phase_ class member. You can think of phase (a.k.a. “phase accumulator”) as the current position of the output signal in the waveform, and therefore as an index in the waveform lookup table.

Since the signal is periodic, phase is a modulo counter. Once the phase reaches the end of the lookup table it starts back from the beginning.

Phase increment

The phase increment value is telling by how much the phase has to move forward between each output sample point.

The higher the phase increment, the faster the phase will step through the lookup table, the higher the output frequency.

In Braids, there is a function that provides the phase increment for a given note/pitch you can find it here. You will notice that the pitch to phase increment conversion itself uses a lookup table…

Interpolation and phase again

Above I said that phase can be seen as an index in the lookup table. It’s actually a little bit more complicated than that.

The waveform lookup tables are discrete representations of continuous signals. In Braids they contain 257 elements. As you can see in the graph above, using a discrete set of values to represent a continuous signal can lead to some distortion. Try to set the phase increment to a low value for instance, the output signal is all squared.

To generate a smooth output signal, we have to find the values of sample points in-between the 257 values of the lookup table. This is called interpolation. Given two points of a signal, find the value of another point in between. Braids uses simple linear Interpolation for that, it’s like drawing a straight line between two points of the lookup table.

For a given phase value, take lookup table values immediately above and below, let’s call them A and B. The linear interpolation is A plus the difference between B and A multiplied by how far the phase is from A’s index.

inline int16_t Interpolate824(const int16_t* table, uint32_t phase) {
  int32_t a = table[phase >> 24];
  int32_t b = table[(phase >> 24) + 1];
  return a + ((b - a) * static_cast<int32_t>((phase >> 8) & 0xffff) >> 16);
}

That means the phase value is actually more than an index in the lookup table. You can think of it as two components: the upper part of phase gives the first index for the interpolation, and the lower part gives the distance from that index.

Macro oscillator

Braids is said to be a macro oscillator, that means it can generate multiple different complex voices, and it does so by combining different synthesis elements. We are going to look at one of those voices, I will let you dive into the code to see how the others are built.

There are three main controls on the Braids module:

Model selection
Timber
Color

Model selection is pretty easy to understand, it’s an encoder to select one of the 49 different voices. Timber and Color have different effects depending on the selected voice.

Let’s say we select the second model: Morph (/\-_ on the screen). This model combines two different waveforms and then runs the result through a filter. The Timber control sweeps through the different waveforms (Triangle + Saw, Square + Saw, Square + Pulse). The Color control, on the other hand, modifies the filter.

Let’s see what it looks like in the code.

Rendering

As I said at the beginning, the purpose of a digital synthesizer is to generate a series of numbers. For Braids this process starts in the Render method of the MacroOscillator class:

void MacroOscillator::Render(
    const uint8_t* sync,
    int16_t* buffer,
    size_t size) {
  RenderFn fn = fn_table_[shape_];
  (this->*fn)(sync, buffer, size);
}

As you can see this method is very small. It takes a buffer and its size, gets a function pointer from an array (fn_table_) using the shape_ value, and then calls that function. Note that I will ignore the sync buffer, it has to do with the synchronization of waveforms and it’s not critical to the understanding of Braids.

The shape_ value is set when turning the model selection encoder, for our example: MACRO_OSC_SHAPE_MORPH. fn_table_ contains a function pointer for each of Braids synthesis models.

So in our case, MacroOscillator::Render calls the MacroOscillator::RenderMorph method.

MacroOscillator::RenderMorph

When looking at this method I will ignore some of the code for the sake of clarity, in particular the part that deals with filtering.

The morph model uses two AnalogOscillator objects and combines their outputs. The first lines of code set the pitch of the oscillators:

  analog_oscillator_[0].set_pitch(pitch_);
  analog_oscillator_[1].set_pitch(pitch_);

As you can see they both have the same pitch here, for other models the oscillators may have different pitch for detune effect.

Then the oscillators are configured based on the Timber control mentioned earlier. The values of Timber and Color controls are stored in parameter_[0] and parameter_[1] respectively in the code. They are members of the MacroOscilaltor class. This piece of code does the morphing between two waveforms:

  uint16_t balance;
  if (parameter_[0] <= 10922) {
    analog_oscillator_[0].set_parameter(0);
    analog_oscillator_[1].set_parameter(0);
    analog_oscillator_[0].set_shape(OSC_SHAPE_TRIANGLE);
    analog_oscillator_[1].set_shape(OSC_SHAPE_SAW);
    balance = parameter_[0] * 6;
  } else if (parameter_[0] <= 21845) {
    analog_oscillator_[0].set_parameter(0);
    analog_oscillator_[1].set_parameter(0);
    analog_oscillator_[0].set_shape(OSC_SHAPE_SQUARE);
    analog_oscillator_[1].set_shape(OSC_SHAPE_SAW);
    balance = 65535 - (parameter_[0] - 10923) * 6;
  } else {
    analog_oscillator_[0].set_parameter((parameter_[0] - 21846) * 3);
    analog_oscillator_[1].set_parameter(0);
    analog_oscillator_[0].set_shape(OSC_SHAPE_SQUARE);
    analog_oscillator_[1].set_shape(OSC_SHAPE_SINE);
    balance = 0;
  }

We first see that there are three different cases depending on the value of Timber (parameter_[0]`):

0 to 10922
10923 to 21845
21846 to 32767

For each case, the shape of the two waveforms is set, as well as the balance value. Later in the code balance will control the mixing of the two waveforms, i.e. which one we hear more than the other.

The third case is a bit different, balance is set to 0 which means only the square wave can be heard, but on the other hand it’s the parameter of analog_oscillator_[0] that varies. This parameter controls the pulse width of the square signal.

The interactive graph below will give you an idea of what the morphing will look like on the output:

The next step is rendering:

  int16_t* shape_1 = buffer;
  int16_t* shape_2 = temp_buffer_;
  analog_oscillator_[0].Render(sync, shape_1, NULL, size);
  analog_oscillator_[1].Render(sync, shape_2, NULL, size);

Calling the Render method of each AnalogOscillator with a different buffer. Render will use the lookup table synthesis explained above to generate audio signal.

And finally, mixing the two different signals (with the filtering stuff removed):

  while (size--) {
    *buffer++ = Mix(*shape_1++, *shape_2++, balance);
  }

Set each sample point output with a mix of the two AnalogOscillators output.

Conclusion

And that’s it! With what we have seen in this post you have all the key elements to understand the AnalogOscillator part of Braids’s models. The other models are just a different combination of settings and mixing for the AnalogOscillators.

If you enjoyed this blog post, let me know and I might do a second part on the DigitalOscillator part of Braids that includes drums sounds, and plucked or fluted instrument simulation.

Bonus section: `A * B >> 15`

This code pattern is very common throughout Braids, two values multiplied and then shifted to the right by 15. This operation is in fact a fixed-point multiplication.

Without explicitly saying it, Braids is heavily relying on fixed-point representation and operations. In particular the 16 bit fixed-point format Q0.15 (sometimes abbreviated Q15). This format represents values from -1 to 1 - 2^-15 (0.999969482421875) using 16 bit 2’s complement signed values (int16_t) that the CPU knows how to manipulate.

To go from the int16_t value to the fixed point value we just have to divide by 2^15:

-32768 / 2^15 = -1.0
-5367 / 2^15 = -0.163787841796875
20000 / 2^15 = 0.6103515625
32767 / 2^15 = 0.999969482421875

In practice we never do this conversion in the code because there’s no need, but this is to give you an idea of how fixed-point works.

Why Q15? Because this is what the 16 bit audio format is in essence, a succession of 16 bit values representing the signal strength from -1 to 1.

Going back to `A * B >> 15`

So when multiplying two Q15 numbers we are just multiplying two factors from -1.0 to 1.0. Conceptually we expect a result from -1.0 to 1.0, but since under the hood we are multiplying two 16 bit signed integers we can very quickly run into problems:

int16_t a = 16384; // 0.5
int16_t b = 22937; // ~0.7
int16_t result = a * b; // Integer overflow! (undefined behavior)

This is why, even though I said Braids uses the 16bit Q15 format, you will see that many operations are done in 32 bit:

int32_t a = 16384; // 0.5
int32_t b = 22937; // ~0.7
int32_t result = a * b; // 375799808

We avoided the integer overflow, but our result still doesn’t fit in 16 bit and therefore is not a valid Q15 number.

To find our number in Q15 format, the result has to be divided by 2 to the power of 15 (see here for more explanations):

int32_t a = 16384; // 0.5
int32_t b = 22937; // ~0.7
int32_t result = a * b >> 15; // 11468 = ~0.35

With this pattern in mind, Braids’ code will be easier to understand.

Acknowledgment

The interactive graphs in this post are based on the excellent: The Design of the Roland Juno oscillators by Thea (Stargirl) Flowers.