This was made possible by Steven Whittens ThreeAudio Library. It looks like it hasn't been updated in a couple years so I had to update some of the WebAudio API calls. This probably only works in Chrome. The library has some nice beat detection and FFT implementations that are better quality than what I was getting out of the box with the AnalyzerNode.
Since this uses a vertex shader to displace the plane to frequency data, it's infinitely faster than the other frequency demo I have. One of these days I'll port the old one to use vertex shaders, or maybe this library if I can figure out how to hook up Soundcloud/ mic inputs.
I also decided to grow up and stop using HSL value scale for everything. This is a Lab color space meaning it is indexed by two (not necessarily orthogonal) major axes in the RGB color space, and a lightness component. This amounts to a linear transformation in RGB space, but is more pleasing to the eye because you can keep the lightness of a dataset constant, and convey information through color only.