I've been working on an "online" version of the DOME, which is an interactive music engine for video games that I've been developing on and off for a number of years (for more info see Dometechnics). By "online" I mean a version of the engine that could be embedded in a web page, so that it could be made available for developers of Flash and Javascript games.
The engine itself is a Java applet, which is scriptable via Javascript. Java, for all its faults, is still the best platform for producing MIDI-based synthesized music in a web page. Thanks for nothin', Adobe. Keeping an eye on Google's NativeClient, though.
This blog post will take a look at the challenges of getting Javascript and Java to talk to one another in a web page, but mostly, I want to look at the design decisions that I've been making about the API for doing music programming and whether or not it would be more appropriate to call it a "DSL" (although the distinction might just be syntactic sugar).
In order to make sense of this post, I should explain a little bit about how the DOME works. It's based on the idea of creating a "score" in the sense of a theme and variations on that theme, followed perhaps by another theme. Loosely, it's designed to enable "cinema-like" scores in interactive contexts: "Films often have different themes for important characters, events, ideas or objects, taking the idea from Wagner's use of leitmotif. These may be played in different variations depending on the situation they represent, scattered amongst incidental music." (source Wikipedia).
It sort of looks like this (from Javascript):
dome.setTheme( _JSON-formatted-theme-data_ );
dome.play();
var minor_variation = 'E->Eb,B->Bb|!bar:4'; // C major to C minor, but not in bar 4
dome.bind( _some-dom-element_, 'onclick', function(){ dome.vary(minor_variation) } );
See how easy!?!
The interesting part in terms of this API vs DSL discussion is the line where the variation is defined. Because of limitations in passing anything other than strings or numbers from Javascript to Java, we might have ended up with something like this:
dome.vary('pitch', '+', 5); // not a very attractive looking command to use (and operators as strings? ...bleh)
Instead, we can do this:
dome.vary('pitch+5'); // better. now it's like a mini-script (will require some parsing magic, though)
...Great. (gratuitous Fast Show reference)
That's all well and good, but in fact, the API approach works better in certain situations. Namely when there's a lot of data to pass (for instance, when the initial theme is established), because an API call can just take a JSON-ified Javascript object as a string. That's because we can take advantage of existing libraries for parsing JSON. But having a DSL for creating variations, in this case, allows for the music programming to occur in a more natural-language type of way.
"The key point is that at each layer the API/DSL should allow the user ... to express the intent of what they want to do as easily as possible." (http://www.testdrivensoftware.com/?p=85)
As a side note, if you're the type of person who gets a kick out of making fun of DSL fan boys, I would direct you to chromatic's amusing discourse on the topic.
So, I've got an API to dump a Javascript object on Java and a DSL for scripting that object, once it's deserialized and appropriately instantiated in Java. In a sense this is still API, because it's essentially a function call on an object, passing in a string. There is no interpreter in the traditional sense of a scripting language. But because the string format for defining variations is a mini-language, it kind of becomes a DSL.
...Great.
Another option might have been to use Java 1.6's scripting API (javax.script), but I had made the decision early on that I wanted this system to be compatible with older versions of Java. I believe the difference is that with Java 1.6, one could theoretically write Javascript code and have that evaluated in the Java application (thereby allowing Javascript functions to manipulate the Java objects). It's a nice idea, but, not critical, and Java 1.5 (and previous) compatibility seemed more important. Why? Because at the time of this writing, the MIDI functionality in Java 1.6 on Mac OS X is broken... that's good enough for me!
Another fun point I'd like to make: At one point during the development of this API/DSL, I fell afoul of a little difference between scripting an applet via the Rhino shell and doing it in a web page. In Rhino, you can return a Java object to Javascript from a method in the applet, and pass that object back to the applet in another function call. Not so via LiveConnect!
For example:
var applet = document.MyApplet;
var obj = applet.getSomeObject(); // return a Java object
applet.doSomethingWithObject(obj); // OK in Rhino! Will cause 40 days and nights of floods if you try this in a browser!
So, one kind of work around, is to have Java use a method on the object to execute the relevant code. This could involve having to share state between the applet and the object in question.
var applet = document.MyApplet;
var obj = applet.getSomeObject(); // return a Java object
obj.doSomethingWithMe(); // works OK in a browser
...Great.
So, let's summarize: Embed the DOME in a web page as an applet and make it "scriptable" via Javascript, and support Java 1.5 for unbroken MIDI. With the goal being the ability to create a theme and define variations on that theme, and invoke those variations by binding them to Javascript events (hence making interactive music possible in Flash and Javascript based games). Done! Alright, not so much done, but it is a working prototype.
Because this is not really ready for prime time, I have produced the following screencast to demonstrate the ideas discussed in this blog. I am actively looking for development resource and industry contacts to bring this technology to fruition. Please come back to follow further progress.
No comments:
Post a Comment