rtpMIDI History

In 2004, I learned about a project of John Lazzaro at Berkeley university. He was working on defining a specification on how to encapsulate MIDI-data in RTP-packets.

Since our company had started to develop our own VoIP-stack including an RTP-stack for media-transfer, I had once again a field where my business-interests overlapped with my private hobbies.

John Lazarro had asked the music-community for feedback on his specification and I was happy to provide some.

The work of John has since been released as an official IETF-specification: RFC-4695

The idea to have a standards-based transport of MIDI over LAN and WAN networks and not to use some proprietary protocols was intriguing, so I thought about implementing such a driver myself.

A year later I heard through some online-community, that Apple had released a new version of their OS X operating-system.  This version 10.4 (Tiger) had a complete network-MIDI-stack integrated, so two Macs could be interconnected via MIDI, tunneled via IP.

This stack used the draft-paper of RTP-MIDI to implement the MIDI-transfer over RTP-packets, but used a proprietary protocol for session-establishment.  On top, one could find peers on the network providing such virtual MIDI-interfaces via Bonjour – so plug & play!

I was annoyed that such a cool thing did not exist for the Windows platform, so in 2006, I started to develop my own network MIDI stack that was supposed to be compatible with the Apple implementation.  I did not know that this topic would keep me occupied for the next couple of years, though 😉

First I needed to get to know Bonjour.  Fortunately Apple had released an SDK for Windows as well and I could use that to enumerate and annouce MIDI-services on the local net pretty fast.

Later on, I even implemented my own “lightweight” mDNS to be able to use Bonjour, even if the Apple-framework had not been installed on the PC.  Actually this was not a small feat – it took quite some time to get this running properly.

The next step was the Apple-proprietary session-establishment.  It was not too complex, but took some time nevertheless.  To help me in my own development, I had created a Wireshark-plugin to be able to analyze network-session easier – this plugin is now a standard-component of the Wireshark-distribution.

More time went into the actual RTP MIDI stack.  I also went the way to create a Wireshark plugin – and actually I had spent more time on this than on the actual protocol implementation itself.

Finally, my RTP-MIDI dissector is now part of the standard-Wireshark-distribution since version 1.8.  So now you can analyze the whole network-MIDI conversation starting from Bonjour service discovery, going into session establishment up to the actual data-exchange between peers…

During the initial tests a nice guy from L.A. who beta-tested a very early version of my driver noticed, that my implementation actually “peeked into the future” 😉

This came from the fact, that Apple sent their RTP-MIDI-packets with a time-stamp a little in the future to be able to work around network-latency & jitter.  So I needed to implement my own MIDI-command-scheduler, to fire MIDI-events at the actual time that they happend – and not some mili-seconds early.

In this phase I found that a company from the Netherlands had implemented a small device that was able to communicate with Macs using the RTP-MIDI protocol: the Kiss-Box.

Of course I had to get such a cool device at once 🙂

Unfortunately I could not get the session-establishment with this hardware-device going for a long time, even though the Wireshark-logs indicated that everything was proceeding according to specification.  Finally the problem was solved by the Dutch company themselves – there was a minor glitch in their firmware that prevented the session to be established.  So take care to use at least R4.1of their firware with my driver 🙂

The initial release of my driver was no driver at all.  The interface to the music-applications was a user-mode driver-dll (whose architecture came from way back from Windows 3.x 😉

It was pretty easy to use, but the installation was not really nice and it was not certain that this architecture would work with future revisions of Windows.

So I decided to go the long way: create a kernel-streaming device-driver that would create the virtual MIDI-endpoints.  So this way the virtualMIDI-project came up along the way.

Actually this part took the longest time to implement properly and I have just finished this component in late August 2010.

After the installer of the virtualMIDI-driver and the rtpMIDI-stack were completed, I could finally release the complete MIDI-driver to the public.

So this whole project took close to 4 years, but since I just had little time to spare over those years and had to create quite a few seperate sub-projects, I think it’s ok 😉

Since Apple has finally included Core MIDI into iOS 4.2, you will also be able to use my driver with applications using this framework for WiFi MIDI connections from your iPhone, iPod Touch and  iPad!