SampleSumo @ Classical Music Hack Day Vienna

We provide the following 3 APIs at Classical Music Hackday:

Real-time music following API

What is it?

In short: it synchronizes something to a live music performance.
In a score following setup, this system essentially listens to a live music performance of a musical score and keeps track of the current location in that score based on what it is hearing.
(more info)

What can you do with it?

  • turn digital sheet music pages automatically at the appropriate moment
  • synchronize projected visuals with a live music performance
  • synchronize a slideshow of images with the music
  • display lyrics in sync with (live) music
  • trigger lighting effects (or other events) during a live music performance
  • control a 3D or 2D animation so that it plays in sync with a live music performance

How do you get access to it?

Come to our workshop at 4:30 PM in the Clara Wieck-Schumann-Saal.
Or contact us directly (see below).

Technical documentation

Our MusicFollower can send out its internal "following along" position to your application in two ways:

Open Sound Control

Via Open Sound Control a.k.a. OSC. There are many, many OSC implementations available, so no matter what programming language you use you can listen to the messages. Many audio and visualization software packages also support OSC.
If you are running the MusicFollower on your own computer the easiest is to send the osc messages to IP 127.0.0.1 and a port of your choice, and then listen on that port where you want to receive the messages.
Because not everyone wants to be running their own MusicFollower (this might result in quite a bit of sonic chaos) we were planning to run one central matcher and send OSC messages to those who want via OSCgroups, a system for routing OSC messages between a group of collaborating users. OSCgroups works like this: we run an OSCgroups server for which we will publish the IP address and port number. On your computer you run the OSCgroups client which connects to our server and passes on the messages to your application of choice.
So in this case the routing would be: SampleSumo's computer [MusicFollower -> OSCGroupsClient -> OSCGroupsServer] -> MHD network -> Your computer [OSCGroupsClient -> Your application]. A lot more detials can be gathered from the OSCgroups readme.

Notes:

  • We will publish the IP address and port number on which we will send out messages at MHD.
  • We will also try to set up an eternal loop of messages going round-and-round (an eternally looping music piece) for easy development and testing.
  • Currently we only support messages at measure level - you get a message when the measure changes - but at music hack day we will try to hack our own software to send out more detailed information (i.e. lower level than measures, for example a message every 50 miliseconds)
  • If you use C++, we highly advise you to use the splendid oscpack library by the equally splendid Ross Bencina.

HTTP

Via HTTP Get calls: the MusicFollower can be told to do an HTTP GET call on each measure. If you are running the MusicFollower on your own computer you can use an asynchronous (or synchronous...) server like Node.js to listen to callbacks from the MusicFollower.
However we will be running a Node.js server on the SampleSumo computer and you can listen to the messages from this server too. I.e. you can use Faye like such:

var client = new Faye.Client('http://#{hostname}/faye');
var subscription = client.subscribe('/#{sessionId}/action', function(message) {
  if (message.command == "gotomeasure") {
    var measureNumber = message.mn;
    // do something with the neasureNumber!
  }
});

Notes:

  • Even though you can use this system to listen for messages there is an even cooler way to combine Node.js and OSC by using node-osc. This exercise is left to the hacker in you. ;-)
  • We will publish the sessionid and hostname at MHD.
  • Currently we only support messages at measure level - you get a message when the measure changes - but at music hack day we will try to hack our own software to send out more detailed information (i.e. lower level than measures, for example a message every 50 miliseconds)

Vocal melody transcription API

Upload a recording of a monophonic sung or hummed melody and get back a transcription result in XML, json or MIDI, or a simple resynthesized audio rendering.
More info here: http://api.samplesumo.com

Granular effect API

Upload any audio recording and apply some weird granular effect on the sound. Includes some presets, but you can also use your own settings.
More info here: http://api.samplesumo.com

Feedback / contact / interaction

Talk to us directly: Michael, Bram and Koen are here in Vienna.
Email contact: support_email.png