This Week’s Program: July 10 - July 14

Last week, I said I wanted to focus in on two areas:

• Start making the gstreamer module something with actual utility.
• Documentation with Scribble.

This week, I did some spikes into doing those very things, and encountered more friction than I anticipated.

f6c517e38794e9b16baf81afe4b4bac4b4112ac5

I write a new function: (gi-repository->ffi-lib repo)

This function takes a repo (which the contract asserts must be a gi-repository) and returns an ffi-lib. This creates a symmetry between Racket’s Foreign Interface functions and GObject Introspection. This function allows me to take an introspected Repository (like that returned when I call (introspection 'Gst)), and instead of using the introspected bindings, use it just like any other foreign library. I do this through the use of the C function g_irepository_get_shared_library, which is renamed to be more Racket-like in this code. Given a GIR namespace, this will return a path to the .so or .dylib or .dll that defines it. I can then load this as I would any other C library using Racket’s FFI mechanims.

This is especially helpful for a C function like gst_element_link_many. This function is not actually provided to GObject Introspection. But now I can actually load this function through plain old FFI. Feels like a good thing to keep in the back pocket when I need to step out from the GIR indirection.

d1161aa4deb01bcc21d2f7f4be1980b6c9d4e34f

Here I extract out GStreamer elements into their own module: element.rkt. This is just a stub to start, and the intention is to “start making the gstreamer module something with actual utility.” I’m not quite sure of what form that ought to take, and the aforementioned ffi-lib support was a stab in a direction. I decided to mull over this a bit more and switch to focusing in on documentation.

Scribble

Scribble is the name of Racket’s Documentation tool. To save you a click:

Scribble is a collection of tools for creating prose documents—papers, books, library documentation, etc.—in HTML or PDF (via Latex) form. More generally, Scribble helps you write programs that are rich in textual content, whether the content is prose to be typeset or any other form of text to be generated programmatically.

Scribble is a fascinating expression of Racket’s power in creating domain specific languages. Every bit of Racket documentation you see was generated by Scribble. You might remember last week’s haphazard explorations in LaTeX. Now it all comes full circle. Scribble has piqued my curiosity since my interest in Racket first began (see the work of Matthew Butterick and Pollen). This week I took my first earnest steps in using Scribble to document Overscan.

I found it difficult.

Scribble is effectively a new language. Yeah there’s Racket in there, but the conventions for calling into it as well as the available functions require an additional learning curve. I had a hell of a time learning about the interactions between Racket package management, raco setup, and package documentation. Thankfully, the folks in the #racket freenode channel were more than happy to answer my questions.

First of all, I had to reorganize my docs. Overscan is a multi-collection package and the scribblings field in package metadata is only well-understood with single-collection packages. Something like that… I read this in the Racket documentation somewhere and now can’t find it. Your average Racket installation generates documentation in two different places: a “main” installation directory and a “user-specific” directory. On my Mac that means I can find documentation in /Applications/Racket v6.9/doc and in ~/Library/Racket/6.9/doc. This was so confusing to me but I finally managed to get most of my documentation process squared away.

To build my documentation and put it in the right place I run:

raco setup --doc-index --pkgs overscan

My writing process means I write a bit, save it, load it, and view it in a browser. I want to do this quickly. It occurred to me that it’s been a while since I looked at the landscape of do something when the filesystem changes utilities and it’s real out in these streets.

I decided to download and use entr(1) to do what I needed to do:

find overscan/scribblings -name *.scrbl | entr -d raco setup --doc-index --pkgs overscan

I run this command and now entr is waiting for changes from the list of Scribble files piped in and will run my raco setup command whenever something changes.

Here’s what I sorted out this week w.r.t. Scribble:

• Where to put the scribblings directive in an info.rkt.
• What a 'multi-page Scribble document looks like.
• The difference between the main and user-specific directories vis-à-vis where documentation is accessed from.
• raco docs does not work in the current version of MacOS. This is a known issue and should be fixed in the forthcoming Racket v6.10.
• A productive process for compiling and viewing scribble documents as the code changes.
• The Scribble functions: @table-of-contents, @include-section and @margin-note.

Special thanks to Alexis King (aka lexi-lambda) for being a good sport about answering my questions in #racket and for writing the great Megaparsack documentation, which has been a great example to follow.

Next week I plan on writing some in-depth documentation and will hopefully figure out how to build said documentation at regular intervals and host them on the Web.

– Mark