This Week’s Program: Apr 11 - Apr 15

I’ve got some new subscribers (👋) and you folks will want to read last week’s entry to catch up on what I’ve been working on.

Good. This week I had every intention of getting sonic-sketches running in a production environment. I had no idea how much eldritch computing I would end up doing.

9fb31f700ca7fbfa0bddf7bd8acaf4cb7f92bf88

Innocuous enough. I set up the drummachine to use the core.async based clock-signal. Big whoop.

😖

Things took a turn here and it’s not really reflected in the code at all. There’s a commit where I fix a typo but that’s the only artifact of a bunch of work chasing down how exactly I’ll run this thing in the cloud.

I had in my head it would be neat to try out AWS Lambda for this. This is just a simple command-line program that generates a song and uploads it to S3. I start digging into running Clojure on Lambda. It’s certainly possible. I’m not familiar at all with Lambda so it looks like you just need yourself a -handler function and you’re set. I read up and track down some libraries that show off better ways of dealing with processing AWS events in Clojure and I keep seeing people say things along the lines of “that increased start-up time to get Clojure running on the JVM tho” and I thought what’s the big deal? Only then did I realize that Lambda charges based on how many milliseconds your program runs. This is a dealbreaker because my program has to start up Overtone and then play a song. Lambda isn’t going to work.

After that exploration (which kind of took the better part of an afternoon), it became clear that I would need a real server to run this on. Time to get sonic-sketches running on Linux…

The first thing I try to do is figure out how to package my program up in an uberjar — a big package wrapping up both my program and its dependencies. This doesn’t work. I do a whole lot of reading about AOT compilation in Clojure and Lein and tweak my project.clj in all kinds of subtle and repetitive ways. Something about how Overtone uses native libraries to link to SuperCollider. For those who are interested in this journey this discussion on overtone#140 did manage to help out, involving compiling the uberjar without AOT and then compiling the main class and adding that to the jar later. That worked, but my goodness that’s convoluted. I resolved that I would just have my source code on the box and lein trampoline run because that works reliably. The uberjar exploration took the better part of a morning.

I was going to use HashiCorp’s newer Otto project to manage my development and deployment. Love those HashiCorp folks, they have so many cool tools!

Otto doesn’t have a supported “app type” for Clojure so I went with “custom” which is meant to point to a Vagrantfile. This wasn’t working until I found otto#158 which points out a schema change not reflected in the documentation.

1a03673bc453d5e57724cde6ef8aed68e8bcc3e5

Adds a major mode to my Emacs config for working with the HashiCorp Configuration Language.

607afce251df3252e9c7761fc1fc7a5beeaa5f47

Adds an Otto Appfile and Vagrantfile to the project.

Running otto dev does not sync my project to the vagrant directory on the virtual machine which is what I’ve come to expect with Vagrant. Otto does not treat the Vagrantfile as a “templatized file” as it claims to do in the documentation. I punt on Otto and decide I’ll just stick with Vagrant for now.

dabb87a13fb5bfcc05f3f15d117eabff95a2071f

I set up my Vagrant box to use hashicorp/precise64 because that’s what it uses in all of the documentation and I’d prefer to stay far away from anything that would push me to have an opinion about Linux distros. I have no opinion on Linux distros and I’d like to keep it that way (Arch seems prtty cool tho). I provision the box to have OpenJDK 7 (I guess Precise Pangolin doesn’t care too much for Java 8) and download lein onto the box. Cool I have a Clojure environment now.

lein run doesn’t work in my Vagrant environment. Unsurprising. Something about those native libs and jack.

😱

I then decide I’m going to try to set up my virtual machine to support audio. I thought 💭 “Heck I’m making audio I might as well ensure that I can hear the thing this machine is doing how else will I know if it works?” I don’t know why I thought this but I explored it and my god it just about did me in.

First of all I had to have Vagrant make sure that VirtualBox knew that this machine needed to support audio. After some googling and documentation reading, I found this handy bit of Vagrant configuration:

  config.vm.provider "virtualbox" do |vb|
    vb.customize ["modifyvm", :id, "--audio", "coreaudio", "--audiocontroller", "ac97"]
  end

This allows VirtualBox to emulate a soundcard on the virtual machine and route it into Apple’s coreaudio. I think that’s what this does, at least.

Then, I start searching for ways to play a sound from within the Vagrant machine. I read about two utilities: aplay and speaker-test. Where do I get these utilities? I read on that these are part of the alsa-utils package. ALSA is the Advanced Linux Sound Architecture. Apparently it’s used by Linux to play sounds. There’s also Open Sound System (or OSS) and PulseAudio. I bring these up because when searching for “play audio on Ubuntu” all of these topics appear and all of them are confusing and nothing makes sense to me. All of these maybe conflict with each other (or don’t) but all of them will do something sinister to your system that forces you to dive into system configuration files as root to untangle. I have no idea what I’m doing.

aplay -l

Shows that I have no sound card.

sudo aplay -l

Oh actually I do have sound cards. I have to add the vagrant user to the audio group. Cool. No sound plays. Oh, apparently there’s an alsa-mixer that controls system mixing and volume. I just need to turn the volume up! alsa-mixer is a curses program. I have no idea what I’m doing. I google around and learn that even though I thought I turned the volume up, the channels were still muted. What channels need to be turned up and unmuted? I’m googling and googling and googling and I have so many tabs open and I need to use the ac97 audio controller in VirtualBox and amixer allows you to control volume without the curses interface and finally…

fbef7f98ff4210d3c0a9a85aba9d3889478fb978

Finally I am able to play a wav file with aplay in my Vagrant machine that I can hear through the headphones plugged into my host Macbook Pro.

It dawns on me now that I probably didn’t need to do that. I’m generating wav files why do I need to play them?

Also, lein run still doesn’t work.

I guess I need to install the JACK Audio Connection Kit as mentioned in the Overtone wiki.

jackd won’t start for a variety of reasons. Googling and trials and errors continue.

export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/dbus/system_bus_socket

This is because jackd thinks it needs X11 to run. I have no idea what the above environment variable is doing but it does something. There’s something wrong with the “security policies in the configuration file” and adding

<policy group="audio">
  <allow own="org.freedesktop.ReserveDevice1.Audio0"/>
</policy>

to the /etc/dbus-1/system.d configs at last results in jackd starting. I have no idea what I’m doing.

I vagrant destroy and vagrant up in a loop over and over because I keep stopping daemons that shouldn’t have been stopped. I can’t apt-get install jackd2 in my Vagrantfile provisioning because this big curses dialog box opens up in the middle of installation asking me something about a realtime kernel and that makes the output totally wack.

311e6274968f0c90480bbdda7c4a47c3a7fdb00b

lein run still doesn’t work. I create a separate Overtone project from scratch to see what I can get to work inside a REPL.

SEVERE: Unable to load native libs c and scsynth. Please try an
external server with (use 'overtone.core)

Severe.

user=> (boot-external-server)
--> Booting external SuperCollider server...
--> Connecting to external SuperCollider server: 127.0.0.1:16931
Exception in thread "Thread-10" java.io.IOException: Cannot run
program "scsynth" (in directory "."): error=2, No such file or
directory

This is where I am now. I am determined to make this work. If you are reading this and you have experience running Overtone or SuperCollider on Linux please get at me. I am tempted to rack a Mac Mini somewhere but I’d really rather not. I want to get my program to run in a Linux environment so I can host it in the cloud to generate music. I want to make a Vagrantfile (or a Dockerfile if it comes to that) for others to use so that they can run Overtone projects in the cloud.

I have no idea what I’m doing.

I have no idea what I’m learning.

There’s some kind of moral to this story or lesson to be learned but I have no idea what that is.

🤔 Mark