::|CONTENTS
- Introduction
- The Basics of FM theory
- Which Formats
- The Building Blocks of FM Synths
- Modulation
- Instrument Design
This article aims to get you up to speed on how FM (Frequency Modulation) synthesis works and how to approach sound design with FM synthesis.
If you prefer video content, check out
FM Synthesis Crash Course (for Chiptune), which should cover most if not all of the same core points in this article.
Prerequisites
It’s assumed that you understand the basic concepts of subtractive synthesis (basic waveforms, ADSR, filtering, amplitude, frequency, amplitude modulation). If you need to brush up on some of those topics, or would like to prepare yourself to discuss FM, check out
Chip Champion’s comprehensive article on the subject!
There are interactive sections within this article using
Furnace Tracker! Go ahead and download it if you don’t have it (it’s free!) If you don’t have much experience with Furnace, no worries! This article will explain all you need to know for our discussion of FM.
If you happen to know some about broadcasting technology, (VHF, UHF, SATCOM, Analog & Digital TV), then you’ll notice how some of that knowledge transfers over to this discussion! If you don’t know too much about it, no problem. In fact, why don’t we start at the beginning…
Introduction
A Brief History of… FM radio?
FM as a concept has been around for more than 100 years. However, its applications in audio synthesis are relatively recent; only within the last 50 or so years has it gained popularity as a way of generating sound…
Let’s take a detour into the world of radio broadcasting, as FM’s first and largest use case was as a means of creating low-noise, high-quality radio broadcasts. Our story starts in 1919, where radio engineer Edwin Howard Armstrong filed the US patent for what would later become known as the “superheterodyne radio receiver".
Though the technology itself is pretty far removed from what we know today as modern FM synthesis, it was the very first step in exploring FM as a concept. Its creation showed that a radio signal could be processed by “vibrating”, or modulating, the signal by what’s known as an “intermediate frequency”. You can essentially think of it as “shifting the pitch” of the original signal. In doing so, it becomes a lot easier to spot and filter out noise, and increases the transmission’s audio fidelity.
That gave engineers another idea in their quest for noise-free broadcasts: instead of transmitting audio in the strength, or amplitude, of a radio signal, what if the broadcast itself encoded the amplitude as tiny “wobbles”, or shifts, in the known carrier frequency? That way, the receiver could measure and convert these rapid shifts in frequency back into an amplitude signal; In other words, raw audio! This is essentially what FM radio is, even up to the modern day. But why go through all this trouble?
As in all areas of life, noise is constant. Even in the electromagnetic spectrum. The good thing about it, though, is that it’s
consistently constant. Everywhere in radioland is covered in background noise, and the only way to be heard above it is for your signal to be more powerful than the noise. That can be a challenge, though, when the thing you’re trying to broadcast must be louder
and quieter than that base line of noise. That makes FM especially useful, because it doesn’t
need to be quiet! All of the “loudness” data of the original broadcast is no longer tied to only the carrier frequency itself, but deviates in time within range of frequencies around that single carrier frequency. So the signal can be broadcast as strongly as possible with little to no effect on audio quality. This also means that whatever you receive out of this FM signal is
almost as it was broadcasted, with much less noise!
You may be wondering about now “Ok, radio. Cool… What does this have to do with FM synthesizers???” FM radio demonstrates the two most basic elements of FM synthesis: the modulation wave, and the carrier wave. You know your favorite radio station, 96.5 The Dingle? That 96.5 stands for 96.5MHz, that station’s carrier wave. The audio that comes from that station? That’s the modulator wave. Normally in radio, the carrier wave is so high-pitched that it’s impossible to hear, but believe me, it’s there. Buried inside your radio somewhere.
When using an FM synthesizer, though, the carrier wave is usually in the audible range and you’ll
DEFINITELY hear it. We’ll dig into that more later in this article.
The Basics of FM theory
The Carrier and Modulators
Unlike with FM radio, the carrier frequency is actually audible in FM synthesis. Oftentimes it acts as the fundamental frequency, the part of a sound that defines the pitch or note we hear (there’s some exceptions to this, but we’ll cover that eventually). A carrier wave can be any type of wave – sine, saw, square, triangle, pulse… it doesn’t matter! If we were to play a melody with just the carrier wave, however, nothing special happens. The real magic is when we use another wave to modulate the carrier wave. Remember, though, FM stands for
frequency modulation. That means the amplitude of our carrier wave stays the same, but the frequency changes. What does that look like? Let’s take a look:
Here we have carrier wave that we’ll say represents 440Hz (A4)
+1 - █████ █████ -
- ███ ███ ███ ███ -
- - ██ ██ ██ ██ -
- ██ ██ ██ ██ -
0 —█ █ █ █ -
- ██ ██ ██ -
- - ██ ██ ██-
- ███ ███ -
-1 - █████ -
Next, we will make a modulator wave at ~880Hz (2x our carrier wave)
+1 - ▄■▄ ▄■▄ ▄■▄ -
- ▄▀ ▀▄ ▄▀ ▀▄ ▄▀ ▀▄ -
- - █ █ █ █ █ █ -
- █ █ █ █ █ █ -
0 —█ █ █ █ █ █ —
- █ █ █ █ █ -
- - █ █ █ █ █ -
- ▀▄ ▄▀ ▀▄ ▄▀ ▀▄-
-1 - ▀■▀ ▀■▀ -
To modulate wave 1 by wave 2, we need a way to set how much we modulate the signal, essentially the sensitivity of our carrier wave to influence from the modulation wave. This is determined by what is known as the “modulation index” and “deviation constant” in FM theory. It’s not necessary to know for our purposes, but the formula for determining this is here for any who are curious:
deviation_constant * modulator_wave_amplitude
B (mod. index) = —----------------------------------------------
modulator_wave_frequency
With both carrier and modulator waves set at equal amplitude, and a deviation factor of 1000, the resulting waveform will look something like this:
+1 - ▄■▄ -
- ▄▀ ▀▄ -
- - ▄■▄ ▄■▄ █ █ -
- ▄▀ ▀▄ ▄▀ ▀▄ █ █ -
0 — █ █ █ █ █ █ █ -
-▄ ▄▀ █ █ ▀▄ ▄▀ ▀▄ ▄▀ -
- - ▀■▀ █ █ ▀■▀ ▀■▀ -
- ▀▄ ▄▀ -
-1 - ▀■▀ -
Pretty neat, huh? Those peaks and valleys that ripple in our waveform are the result of our modulation operation. It’s not exactly clear here how these different shapes change the way our wave sounds, but we will get to that soon.
In the meantime, check this out: just because the carrier wave amplitude stays the same, that doesn’t mean our modulator wave’s amplitude has to! Check out what happens when we triple it, but keep the deviation factor the same:
+1 - ▄ ▄ ▄ ▄ ▄▄ -
- █ █ █ █ █ █ █ █ █ █ -
- - █ █ █ █ █ █ █ █ █ █ -
- █ ■ █ █ ■ █ █ █ -
0 — █ █ █ █ █ █ -
- ▄ █ █ █ █ ▄ █ █ ▄ -
- - █ █ █ █ █ █ █ █ █ █ █-
- █ █ █ █ █ █ █ █ █ █ -
-1 - ▀ ▀▀ ▀ ▀ ▀ -
Increasing the amplitude, the “volume”, of our modulator makes the wave start to fold back in on itself, increasing the intensity of the modulation applied to the carrier wave! Doubling the amplitude of our modulation wave has the same effect as doubling our deviation factor; they’re proportional to each other. This means one of those values can be constant. Indeed, many FM synths take advantage of this relationship to simplify how many values need to be tweaked in order to increase our modulation intensity. Keep this in mind, because we’ll be using this in an upcoming interactive assignment!
INTERACTIVE LESSON I: CARRIERS AND MODULATORS
• Open up Furnace, and you should immediately be put into a Sega Genesis / Mega Drive workspace. The Sega Genesis / Mega Drive uses an FM sound chip called the Yamaha YM2612, sometimes called an “OPN” chip.
• Go ahead and look at your instruments panel in the top right of the screen. You should see a white circle icon with “-None-” next to it. Right above that icon is a “+” button. Select that, and click on FM (OPN). You should now see a new instrument in your panel! Double click on the instrument listing, and a new window should pop up
• This new instrument editor has a lot of settings that can get complicated really fast. The main thing we’ll take a look at is the operators. These are our modulation waves, and they can be stacked in a bunch of different ways for different sounds. But let’s just focus on operator 4 for now. Click each “Operator” button except Operator 4 to disable them.
• Now try pressing a key on your keyboard. What do you hear? Operator 4 is currently acting as our carrier wave. We aren't modulating it with any other waves, so it just sounds like a simple sine wave.
• What happens when we add a modulator wave? Let’s click on Operator 3 to re-enable it, but before playing a note, let’s move its “TL” slider all the way to the bottom (127). Now play another note. Has anything changed? Well, it shouldn’t have. The “TL” slider essentially acts as our “modulation index” from before; it’s the intensity of operator 3, which we’ve basically just silenced.
• Let’s change that! And to better see the changes, try this: with one hand, repeatedly press down a note over and over again. Then with your mouse hand, slowly bring Operator 3’s “TL” slider back up. Notice how the sound changes? It seems to become a lot more sharp-sounding the more you increase the slider. Has the note’s pitch changed? Not really, but it sounds different, doesn’t it? This is an excellent demonstration of FM’s greatest strength: creating harmonics and sidebands! We will discuss what that means in the following section.
Harmonics and Sidebands
Harmonics and Sidebands are two of the most notoriously difficult things to grasp with FM. Part of the reason why is because it’s not intuitive at first to see how changing the shape of a wave creates them. But with some help from our old friends the mathematicians (James Cooley, John Turkey, Joseph Fourier) we can see exactly what they are and how they behave.
As you may know, all audio can be represented as a spectrum of the frequencies that they are made of. This is extremely useful for determining the qualities of a certain sound.
Let’s use the last wave we looked at as an example: the 440Hz carrier wave with the triple-amplified 880Hz modulator wave. Back in ASCII we go!
+1 - ▄ ▄ ▄ ▄ ▄▄ -
- █ █ █ █ █ █ █ █ █ █ -
- - █ █ █ █ █ █ █ █ █ █ -
- █ ■ █ █ ■ █ █ █ -
0 — █ █ █ █ █ █ -
- ▄ █ █ █ █ ▄ █ █ ▄ -
- - █ █ █ █ █ █ █ █ █ █ █-
- █ █ █ █ █ █ █ █ █ █ -
-1 - ▀ ▀▀ ▀ ▀ ▀ -
Using quick maths™, we can plot the frequencies needed to make this wave on what’s called a spectrogram:
╔════════════════════════════════════════════════════════════════════╗
║0db ║
║ █ ║
║ █ █ ║
║ █ █ █ ║ █ ║
║-12db █ █ █ █ ║ █| █ ║
║ █ █ █ █ ║ █| █ ║
║ █ █ █ █|║ █|║█ █ █ ║
║ █ █ █ █|║ █|║█ █ ║█ █ ║
║-24db █ █ █ █|║ █|║█ █|║█ █ █ ║
╠════════════════════════════════════════════════════════════════════╣
║20Hz |50 |100 |200 |500 |1kHz |2k |5k |10k ║
╚════════════════════════════════════════════════════════════════════╝
Although there’s a little extra noise(and the resolution this text gives isn’t helping anything, either), notice how the loudest frequencies that make up this wave seem to be spaced evenly apart by a certain distance? These are our harmonics! You can think of them as “accents” to our fundamental frequency (remember, 440Hz!) that change the timbre, or quality, of our wave. Any complex, tonal (pitched) sound with a given fundamental frequency will always have harmonics in the same places on the frequency spectrum, it’s just a matter of how loud each harmonic is.
Ever wonder why plucking a guitar string doesn’t sound like a clean sine wave? It’s because the physical properties of the materials in the guitar and its strings are, in effect,
modulating the sound of the string vibrating, which both amplifies and dampens the string's harmonic vibrations. Although the metaphor isn't 1:1, the same kind of amplifying and dampening of harmonics happens with FM, but synthetically. And unlike making sounds in the real world, we have fine control over the types of new resonance we can add to our instrument! How so?
Well, what controls the distance between each harmonic? You may notice (although it’s admittedly a little hard to tell in this diagram), that each harmonic is spaced out by around 880Hz -- our modulation frequency! Yes, our modulation frequency is what determines how far apart each harmonic is. Depending on the relationship of our modulation frequency to our carrier, we can make the harmonics be, well, harmonious with our carrier wave, or we can also make them very dissonant from our carrier wave. It’s all in our control!
This control comes, though, with a slight caveat, due to how FM mathematically affects our carrier wave. Notice all of those smaller spikes of higher frequencies in our spectrogram? Those are some of what are called sidebands. You can essentially think of them as “echoes” across the frequency spectrum that result from how fast we modulate our carrier wave.
Interestingly, they sometimes will show up where a harmonic is, and sometimes they won’t. Why? Because technically, those harmonics
are sidebands. They just so happened to line up at a frequency that is an even multiple of our fundamental frequency, which makes it sound harmonious. If the sidebands didn't line up on these harmonic frequencies, it would make them dissonant from our fundamental frequency, which would add more "noise" to our sound. In the example we used above, our modulation wave is exactly 2 times our carrier wave, so the ratio is an even 2:1. Therefore, the most dominant frequencies in the resulting waveform will line up exactly on the closest multiples of our carrier wave frequency.
Sidebands become increasingly more common as the carrier intensity increases, and are unavoidable and ever-present. That’s good, though, as we can use them to our advantage to simulate both the natural resonation of a physical instrument, and even some pretty unnatural-feeling sounds! They also can be quite handy when recreating other primitive waveforms like square, saw, and pulse waves out of the simple sine waves that most FM Synthesizers are limited to. In fact, why don’t we try replicating one? We’ll use the frequency spectrum of this 440Hz Square wave as an example.
+1 -█████████████████████ █████████████████████ -
-█ █ █ █ -
- -█ █ █ █ -
-█ █ █ █ -
0 —█ █ █ █ -
-█ █ █ █ -
- -█ █ █ █ -
-█ █ █ █ -
-1 -█ █████████████████████ █████-
The spectrogram for this sound looks about like this:
╔════════════════════════════════════════════════════════════════════╗
║0db █ ║
║ █ ║
║ █ ║
║ █ ║
║-12db █ ║
║ █ █ ║
║ █ █ █ ║
║ █ █ █ █ ║
║-24db █ █ █ █ █ █ █ ████ █ █ █ ║
╠════════════════════════════════════════════════════════════════════╣
║20Hz |50 |100 |200 |500 |1kHz |2k |5k |10k ║
╚════════════════════════════════════════════════════════════════════╝
The biggest harmonics we can see are in equal intervals of around 880Hz… wait a second, that sounds familiar! We might actually be able to create something pretty close-sounding to a square wave just by messing around with our current parameters a little more! Let’s try it out.
INTERACTIVE LESSON II: HARMONICS AND SIDEBANDS
• Let’s continue on with our experiment in Furnace from Lesson I. Open up the instrument editor again if it’s not already, and let’s take a look for a second at the section in the top right corner. You should see a diagram that looks something like [1->2->3->4]. This is our operator’s algorithm, essentially how each modulation wave is applied to the carrier wave (the operator at the end of the chain). If you’d like some more info on how exactly this works, take a look at the “Building Blocks of FM Synths” section below.
• Before moving on, let’s do a little prep work first. On all of the operators, even the disabled ones, set their “decay” (D) sliders to 0 (all the way up). This will allow us to hear the operators at work without our envelopes getting in our way. After doing this, next set all operator’s “Detune” sliders to 0 (dead center). This ensures all operators are perfectly in sync with each other, and won’t create any phasing effects.
• Up until this point it hasn’t been explained how to change the frequency of our modulator wave. This is a feature of digital FM synthesizers, like the YM2612 we are using: instead of setting the exact frequency manually, operators are instead set as multiples of a base “reference” frequency. We can take a look at how this multiplier is configured by looking at the “multiplier” slider under the ADSR envelope for that operator. By default, both operator 3 and 4 are set with a multiplier of 1, meaning the modulator and the carrier wave are at the same frequency.
• But let’s think back on how we can recreate this square wave. If you recall, the square wave’s harmonics were spaced in intervals about twice as large as our carrier frequency (440Hz with harmonics at 880Hz intervals in our example) so if we want our operators to have the same harmonics, our modulation wave’s frequency needs to be twice as high as our carrier wave. We can accomplish this by keeping operator 4’s multiplier at 1, and setting operator 3’s multiplier to 2.
• Now try playing a note! Chances are, it sounds sort of like a square wave with some weird ringing distortion. In his experiments, your author has found that setting operator 3’s TL slider to around 30 produces the cleanest wave. It’s still not perfect though, it sounds like a square wave that’s being filtered or muffled. This is to be expected, as we’re only replicating the largest harmonics of a square wave. There’s still some smaller ones we’re missing. To fix this, let’s finally enable operator 2 and set its multiplier to 4. This will divide our harmonics into even smaller intervals that we can shape to create a sharper, fuller sound
• Now, play around with operator 4’s TL slider to your liking. And that’s it! We’ve successfully recreated a square wave using just sine waves, and a little understanding of harmonics and sidebands!
Which Formats
These formats are based on some FM synth sound chip:
-
opl2 (YM3812)
-
opl3 (YMF262)
-
pc-x801 (YM2608/OPNA, or YM2203/OPN)
-
sgen (YM2612/OPN2)
-
ym2151 (YM2151/OPM)
Some formats may optionally allow for FM synthesis as part of the format, like:
- NSF expansion based formats by using VRC7 (YM2413/OPLL) or FDS, e.g.
nsfplus,
famitracker,
0cc,
famiplus
- multi-system formats by using an FM sound chip or system, e.g.
deflemask,
furnace,
vgm
-
mptm by using OPL instruments (note that while .s3m in the DOS module formats does support OPL instruments, they are not allowed as part of the format rules)
- other miscellaneous formats like
jummbox,
klystrack,
snibbetracker,
sunvox
The Building Blocks of FM Synths
Operators
Operators are the building blocks of an FM channel or instrument. They output a waveform at a certain input frequency and level. They're the oscillators of the synth if you're familiar with that terminology. Most of the time, the waveform will be a sine wave (e.g. on the Sega Genesis/Mega Drive's YM2612, or the YM2151), although in some cases like the OPL chips, you can use other types of waveforms for the operator.
ADSR Envelopes
The term "envelope" refers to how the volume of a note changes over time as the note key is pressed on and off.
ADSR stands for
Attack -
Decay -
Sustain -
Release and refer to the different parameters or parts of the envelope. There is also a
TL or
Total Level that scales the total volume of the envelope.
How the envelope works:
1. When the key is pressed, the envelope starts increasing at the
Attack rate up to max level.
2. Once the envelope hits the max level, the envelope starts decreasing at the
Decay rate down to the
Sustain level.
3. If sustain is enabled, the envelope stays at the sustain level until the key is released. If sustain is disabled, the envelope skips to the next step.
4. Once the key is released (or if sustain is disabled), the envelope starts decreasing at the
Release rate until it hits 0 level.
Each operator has its own set of ADSR parameters, though key on/off events for a note apply across all operators.
Note that some chips/synths (e.g. the OPN chips like the YM2612, but not the OPL chips like in the AdLib cards) may have a 2nd decay parameter (
D2) that allows the envelope to decay while in the sustain stage.
The envelope allows for some rudimentary modeling of instruments. For example, a piano instrument would have relatively fast attack and decay and lower sustain and release. A flute or other woodwind instrument would have a more moderate attack and decay, a medium sustain level, and a quick release. A pad/chord instrument would probably have a long attack and delay to let the sound slowly fade in and out.
Some chips/synths may also have an
RS or
Rate Scale parameter which controls how much higher notes increase the attack, decay, and release rates on the envelope. This can be used for things like modeling faster attack/decay on strings played at higher notes, but it can also essentially ignored and left alone.
Multiple Operators and Multipliers
You might have noticed that each instrument/channel has multiple operators. These operators are chained together in certain ways to make those neat FM sounds. In most cases, each operator works in sync with all the other operators in the instrument/channel, meaning each operator uses the same base frequency and key on/off triggers. Some chips/synths (e.g. YM2612, YM2608) allow for independent control of each operator on one of the channels so you can set different frequencies and trigger key on/off on each operator independently.
Operators have an
ML or
Multiplier parameter. This parameter will run the operator at a multiple of the base frequency for the channel, such as 0.5x or 2x. What's important is the ratio of multiplier values between the different operators on a channel, so for example an operator with 1x multiplier chained to another operator with 2x multiplier will produce the same timbre as an operator with 2x multiplier chained to another operator with 4x multiplier.
For the Yamaha FM chips, each ML value results in the same multiplier (i.e. ML=1 means 1x frequency, ML=2 means 2x frequency, etc.), except for ML=0 which means 0.5x frequency.
Some chips/synths will also have
DT or
Detune parameter(s) which will adjust the multiplier on the operator by a small amount. Again, what matters is the detune ratios/combinations across the operators. These can be used to create a chorus effect on your instrument if you set different detune values across multiple operators.
Modulation
This is the fun part that goes over how frequency modulation actually works!
First off, a disclaimer: the Yamaha chips mentioned above and most other chips technically use
phase modulation and not frequency modulation. They essentially achieve the same effect, but in different ways. This section will explain how phase modulation works since that's what's actually implemented in most chips/synths here but will continue to refer to things as FM.
Also, some more terms: for an operator, the
phase describes how far along the waveform the operator is, and the
frequency describes how fast the operator advances the phase forward through the waveform. An operator that outputs directly out to sound is called a
carrier, and an operator that is used as an input into another operator for modulation it is called a
modulator.
I won't go into the exact math here (the phase modulation article linked above does a good job of explaining and visualizing that), but essentially how phase modulation works is it takes the carrier wave and shifts its phase forwards or backwards based on the position of the modulator. Note that this means if the modulator level is 0, then it does not affect the carrier wave, and the higher the modulator level goes, the more it distorts the carrier wave since it adjusts the phase much more.
Now let's add the ADSR envelopes into the picture. For the carrier, the envelope controls the resulting volume of the note as time goes on from when the key is pressed and when it is released. For the modulator, the envelope controls the level of the modulating wave, which allows you to control how much modulation gets applied to the carrier wave as time goes on from key press to release.
Algorithms
The "algorithms" in FM synthesis refer to how each of the operators are arranged in relation to which operators are carrier(s) and which operators are the modulator(s) for other operators. For anything past 2OP FM (2 operators), there will be multiple possible arrangements of operators, e.g. multiple carriers, operators modulating each other in a chain or series, multiple operators modulating another operator in parallel, and other arrangements in between these possibilities. Refer to a manual or diagram for the relevant chip to see what arrangements are available.
Some arrangements will also allow for feedback, where an operator's output "feeds back" into itself to modulate its own output. The same modulation principles above apply, only with the same operator. Note that this means the amount of modulation that the operator applies to itself through feedback usually ramps up very quickly as the operator gets louder.
Instrument Design
To understand how to design the sound of an instrument using the FM parameters, I encourage you to play around with an FM instrument editor to really understand how each operator parameter works. The crash course video at the top of this guide may also be useful in demonstrating how each of the parameters works
Here are some general tips to guide you:
- Usually you want to start with turning up the carrier level and turning down the levels for all the modulators, and then set the multipliers and envelope parameters for the carrier operator and work your way back to the modulator operators.
- Generally, you will want less modulation for purer tones, so either less total level or fast decay/lower sustain on your modulators. Similarly, grittier or harsher tones should have more modulation.
- If you're familiar with
overtones, you can use a spectrogram of a waveform as a reference for which overtones are most prominent and try to set the modulator multipliers and levels based on that.
- If you want a chorusy effect, try setting the detune parameters so that each operator has a different detune value. The larger the difference in detune, the faster/wider the effect goes.
You can also try taking some preset FM instruments and playing around with them. For example, you can try increasing or decreasing certain parameters to see how it affects the sound of the instrument. In some programs you can toggle the output of certain operators, so you can also try turning off the modulators and then turning them back on one by one to see how each modulator in the chain affects the carrier wave.
