Playing with MIDI in JavaScript
Machines supported musicians since the beginning of the computer era. It was back in 1951 when the University of Manchester was able to record God Save The Queen on it’s Ferranti Mark 1. But was not until 32 years later, in 1983, when the MIDI standard was created. And despite its many limitations and emergence of other competitive technologies (like OpenSound Control) it is still, after over 30 years, the dominating music creation tool.
Midi in depth
As the MIDI specification is quite vast and enables a very fine-tuned communication between music device and digital audio workstations (DAWs), we will be playing with just a small subset of its features. Using MIDI and Audio API we’ll generate simple oscillator and connect it to our device. If you don’t have the device, you can still try the example — our Oscillator in the last snippet can play Tetris theme by itself!
What you need to know for a start, is that when you play a note on your MIDI-enabled device, it sends a message to the recipient with the identifier of the key, velocity (the strength of a keystroke) and information about the state (i.e. key down, key up).
Connecting to a real MIDI device
JavaScript, since March 2015, has official support for MIDI standard. It includes access to both input and output MIDI devices, hooks for their availability and an EventEmiter to receive MIDI commands.
In order to find an available device, we need to call navigator.requestMidiAccess()
. The method returns a promise that will resolve to a MIDIAccess object. We can easily access its inputs
property. MIDIAccess also provides the onstatechange
hook that will be fired every time a new device is connected or disconnected from our system. If you don't have any MIDI device available, you can still test the demo using virtual MIDI devices: I've set it up using IAC Device on MacOS:
After we locate our MIDI device, we can listen to the data bus by subscribing to onmidimessage
. Every time the key is pressed (or a potentiometer knob value has changed if the device has them as well), a new MIDIMessageEvent is sent. It contains regular event data and, most importantly, data
property. The data property is an array of 3 numbers containing all the information we need to create the sound. The following page is a great resource for all possible codes, in this example we will just listen on one channel sending note on/off events.
Generating sound in JavaScript
Now let’s take our minds off MIDI for a bit and think about the sound generation. To play a note, we can either play a pre-recorded sample or synthesize it manually. It turns out that the second option is not hard at all, so we’ll give it a try.
To generate sound we will be using WebAudio API. This whole library consists of an overwhelming number of features including sound generation, many audio nodes implementations, a precise scheduling system and more. We will be using the most basic features just to show-off the capabilities of current JavaScript in sound generation. The API itself is very powerful and its full potential is used by libraries such as Tone.js. In this example though I would like to use just the native capabilities.
The code above creates a single oscillator, sets its sound frequency to 440Hz (known as a Stuttgart pitch or simply A4) and plays it. The older, deprecated way of setting the frequency was to write code like oscillator.frequency.value = 440
and since January 2018 it is removed from Chrome and the setTargetAtTime
is the correct way of doing that. Full example:
Converting MIDI keys
When the MIDI device sends the message about the note being played, it sends the note identifier called the MIDI number. The C4 note has the number 60 and for every semitone higher you just add 1 to this number. To shift by an octave you just have to add 12.
To be able to play the proper sound generated by a MIDI key, we need to convert the MIDI number to the proper frequency. Fortunately, there’s a quite handy equation for such conversion:
m is our MIDI number and f is the result frequency.
Using this equation, we can easily generate the sound using a JavaScript oscillator:
Below you can find a working example. If you don't have any MIDI devices, you can still run the example - keys q,w,e,r,t,y,u,i are mapped to C4-C5 notes.
Using different waveforms
The sound generated by our program is very generic. In fact what you’re hearing is the sine wave. To generate much more interesting sounds, we can change the waveform. The following waveforms are the building blocks of electronic music. Using and oscillating them we can generate more complex music, so we need to learn how to use these first:
Our OscillatorNode has type
property which can be set to various modes, including custom one. The following example shows the difference between simple oscillator types:
Of course, using this method we can exhaust our soundbank quite quickly. To generate more realistic sounds we should look for more advanced techniques of audio processing as Karplus–Strong string synthesis for example. It can be done using these simple blocks though!
Summary
This is just the beginning of what we can do with the combination of JavaScript and MIDI. We can use them not only to generate music but also to control specific parts of the page for example. We could use knobs and buttons which are also in the specification. We can then record what a user played and save it to file or blend it with a pre-recorded section of music. The possibilities are almost limitless. See the references below and start coding your own music.
Reference
- https://newt.phys.unsw.edu.au/jw/notes.html
- https://www.musicradar.com/news/tech/a-brief-history-of-computer-music-177299
- http://marcgg.com/blog/2016/11/01/javascript-audio/
- https://www.w3.org/TR/webmidi/
- https://www.midi.org/specifications/item/the-midi-1-0-specification
- http://www.keithwhor.com/music/
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API#Creating_a_waveformoscilloscope
- https://www.chromestatus.com/feature/5287995770929152
- https://help.ableton.com/hc/en-us/articles/209774225-Using-virtual-MIDI-buses
- https://developer.mozilla.org/en-US/docs/Web/API/MIDIAccess
- http://computermusicresource.com/MIDI.Commands.html
- https://en.wikipedia.org/wiki/Wave#/media/File:Waveforms.svg