Rodan server infrastructure
Posted by Alastair Porter on July 04, 2012
As part of the Simssa project we have been developing Rodan, a software application for helping people to convert images of music into a searchable symbolic format. A main aspect of the Rodan project that I have been working is the integration of the software and supporting infrastructure. Rodan is a complex tool, with six different pieces of software working together in the background to perform the recongition process.
The main server application is written using the Django web framework. We use the PostgreSQL database server to store all data generated by the application. We use a python library called celery to perform long-running operations like music recognition. This helps us to keep the interactive webpage responsive. The main site sends a message to celery that requests an action to be performed on an image by adding the message to a queue, run by RabbitMQ. An example of an operation to perform might be to remove the stafflines from an image. A separate application reads this message off the queue and uses methods provided by the Gamera toolkit to perform most of these operations. The architecture of celery and RabbitMQ allows us to perform these operations on any computer connected to our recognition network. This way, as more people start to use Rodan we can keep up with demand by starting more machines.
Finally, we use the Solr search engine software to convert our recognised music into a form that allows us to very quickly search for sequences of notes in every page that has been recognised. We do this by splitting up the notes on each page of music into n-grams, or lists of every sequence of notes on the page. For instance, if a page contained the notes edcdeee we would store the fact that the following sequences of 3-note phrases (3-grams) appeared on the page: edc, dcd, cde, dee, eee. We do this for all sequences of length 2 up to length 9. By performing this splitting operation before people search for notes we do not need to search through (potentially) thousands of pages of music to find a match for a sequence of notes entered as a search query. Solr will give a list of the pages that each query appears on in under 10 milliseconds.