OK, so my last post (part 2) was basically a whole bunch of history of the evolution of my EverSequence light sequencing software.
This year, I figured I would put together some hardware (getting to that in the next post) that I would drive using the sequencer. Instead of relay boards, for the meshed Light Suits I’ll have an Arduino micro-controller that is connected via USB (which actually looks like a serial port) to the PC. The Arduino Duemilanove has an ATmega368 MCU on-board that includes an Atmel microprocessor and 32KB of flash memory. Arduino comes with a simple development tool in which you write code using a java-like syntax, then you upload it onto the Arduino using the same tool. The program basically has to include, at a minimum, a setup() and a loop() function. The setup() gets called at boot-up, then the Arduino goes into a loop that just calls loop() indefinitely.
The first step in controlling lights from the PC via the Arduino was to write a little program for the Arduino that listens to the serial port. Then I wrote a plugin for the sequencer that that talks to the Ardiono using a simple protocol I concocted. Basically, to change the state of a pin on the Ardiuno card, I send a period, followed by a hexadecimal number, 2 though B, representing the Arduino pin number I want to control, then a three digit value, 000 – 255. So, for example, “.A255” would turn pin 10 on. Right now, only 000 (off) or 255 (on) matter, but eventually I might support dimming using values in-between (6 of the Aurduino I/O pins can be used for PWM, or pulse-width-modulated output…basically what your wall dimmer does to dim 120V lights.)
Actually, there’s a little more to the program than serial-driven on/off. I added some support for some simple built-in sequences, and for selecting one of the sequences – all-on, all-off, or serial mode – by using a button connected to analog input in pin0. In built-in sequence mode, I use bit patterns (one per output pin), that I just rotate through, shifting left every 100ms, and turning on or off based on the leftmost bit. The actual program is here.
This worked like a charm, as I suspected it would, but something was niggling at me while I was planning it. The problem with EverSequence is: it is a royal pain to design sequences of significant complexity. It’s just plain time consuming to lay out a lot of spans timed correctly to the accompanying music, especially if you want nice patterns.
At one point I had some thoughts about adding some sort of beat detection which would detect the beats in a music span added to a sequence and automatically place some markers or relay spans on the sequence. That might help automate the process some, but I didn’t really find any C# libraries that worked, and even with beat detection, I’d still have to invest a lot of time in polishing sequences. I had another idea though.
Most MP3 players for the PC, including Windows Media Player(WMP), WinAmp, and iTunes, support Visualizations – graphic modes where all kind of cool graphics are rendered on-the-fly and in coordination with the music. Visualizations can be as simple as bar graphs like on your stereo, or as complicated as 3D whirling clouds and other crazy stuff. I knew that it is possible to write custom visualization plugins for WMP or WinAmp, so I figured it might be possible to write one that, instead a drawing a bar graph representing the power level at certain frequencies, would send serial control codes to my Arduino.
The big hurdle to this was that with both WMP and WinAmp, you have to code you plugin in C++. My capability level in C++ is pretty minimal – I’m far more adept at C# development. Fortunately, I found a “wrapper” by this guy in C#. Actually, to be more technical, it’s a C# implementation of the IWmpEffects and IWmpEffects2 interfaces that must be implemented, along with a sample implementation of those interfaces in C#. This was a great starting point and made my job a piece of cake – thanks, Nanook!
Basically, to write a custom plugin, you implement the IWmpEffects2 interface, which includes fleshing out the RenderWindowed method, then registering your plugin with Windows. Then, when you choose your plugin in WMP, it will call the RenderWindowed method many times per second, passing a structure that contains two sets of arrays; one contains stereo frequency power values, divided across 1024 bands, the other contains stereo waveform values, divide into 1024 time slots. Think of the Frequency values as a bar graph with 1024 bars. The Waveform values represent more of an oscilloscope visualization. See the IWmpEffects2 interface and TimedLevel structure docs on MSDN and elsewhere for more detail.
So – how to use 1024 channels of analog values to drive 12 on/off pins on the Arduino? Averaging them into bar graphs is relatively easy (and is a good visualization of the power values), but a bar graph is a visual representation of what is essentially an analog value. With my light suits, I just want On/Off values, so I needed to come up with some sane approach of converting something like 20 samples per second of values ranging from 0 to 255. First, I average the 1024 bands and both stereo channels together in clumps to get 12 channel values that I use to control the 12 pins I use on the Arduino. So the first 85 x 2 array values get averaged together into the lowest (bass) value, representing from 20hz to 1700hz, and so on. Next, I track two sets of thresholds, which I’ll call the ceiling and the floor, one of each per channel. The ceiling thresholds are initialized to 255 and the floors are initialized to 0. As play starts, I have the ceiling and floor values converge towards each other – the ceiling value starts falling and the floor starts rising. At the same time, I’m comparing the current value of a given channel to its ceiling. If its value exceeds the ceiling threshold, a command is sent over the serial port to turn the corresponding pin On and the ceiling value is set, or “pushed up”, to the current channel value. Likewise, if the channel value falls under the floor threshold, an Off signal is sent.
The result is pretty good, but there were two things I didn’t like. One was that the lower (bass) channels, not surprisingly, flash more to the beat. The higher channels are more frenetic. Since my hardware will be set up with channels 1-4 going to one Light Suit, and channels 9-12 going to another, one suit would actually appear to be more in sync with the beat. Since I don’t want to screw either myself, or LakewayLightGuy, I reverse/invert the mapping of sound channels to pins every ten seconds.
Another thing I noticed is that the correlation between the low channels and the beat of the music isn’t actually all that great to begin with. I think this might be because Microsoft is doing fourier transforms that aren’t all that great at determining the power level of each frequency, but it could also have to do with the fact that I’m averaging many frequencies together. In any case, on a whim I switched the code to use the waveform array instead of the frequency array. On the bench, it looks like this might actually look better in terms of flashing to the beat in some cases. I wound up making the plugin configurable, so I can adjust the com port, number of channels, and whether to use frequency or waveform data through a properties page. Once I have it all assembled, I’ll make a decision about which looks best for ALG appearances.
All in all, I think the visualization plugin is going to be great for the meshed light suits. I’ll be able to have a more diverse playlist, and probably spent less time developing the plugin than I would creating multiple sequences in EverSequences. That said, I also wrote a driver for EverSequence that talks to the Ardiuno, so in the future, I’ll probably be incorporating Arduinos into my home light setup, and I may incorporate sequences into light suit appearances, as well.
In my next post, I’ll <finally> get to describing the hardware in more detail.
I should mention that I ended up going a completely different direction for octave analysis and sound-synchronization on the Light Suit. Using the WMP plugin would have meant lugging a laptop around, which I tried the first time out, but wound up being a big pain..also, the little Dell Mini I was using died.
A much more compact solution was the incorporation of an MSGEQ7 chip into my Ardiuno circuitry. This little 8 pin chip does a great job of doing FFT octave analysis internally, and provides 8 octaves worth of level values as an output. A good blog that describes how to use it with an Arduino is here. I basically ended up using the prototype area of the XBee shield on top of my Arduin to hold the pretty simple circuitry and I just pipe plain old line-in level music into it from my phone or even an external mic. This worked pretty well, although I still have just basic on-off synchronization…an issue that I since addressed in Light Suit 2.0.