Say hello to my brand new little friend: punchline

There’s sometimes an io.js / node app. How many languages does an io.js / node app have? In general you would think one: JavaScript (a bit of an oversimplification, but you know what I mean.) But often there’s another language in the mix: shell script.

Quick Summary

JS > shell. Shell CLI cmd length < JS API script length. But what if it wasn't?:

'(a.js b.js c.js).d.r(x.js:X).o(bundle.js)'
Proof of concept demo with browserify API.

Shell CLI

Quite often node apps ship with CLI scripts. People seem to love these things (and some projects even document them better than their JS API!). Me, not so much. I know, I know: there are situations where a CLI provides advantages over a JS API. But in general I’d rather work with JS than the shell, especially when using a JS app. The more repetitive / less trivial an operation is, the more that holds, but even once-off one-liners can get pretty gnarly in the shell.

I’m often-enough a fan of command line interfaces in general, as opposed to GUI’s, and I use the linux shell all the time, but I find linux shells somewhat painful to work with. Idiosyncratic, fussy, complicated (in a way that’s often surprising and / or requires a lot of mental energy to review, parse, and keep track of complicated rules — for the shell in general and specific commands). All programming languages are complicated and require a lot of mental energy, but when I say this about the shell I mean it in comparison to JS. When I want to subject myself to that much weirdness and inconsistency I’ll watch Mulholland Drive again.

JS certainly has plenty of quirks, but in my opinion it’s a far more sane scripting language than the shell (take your pick which one), and in any case it’s the native language of node apps. I’ve done various degrees of scripting in a bunch of languages, some of them pretty bad, and I think shell scripting is quite possibly the most idiosyncratic, least consistent. And that includes just executing ad hoc commands, not only scripts that are extensive or repetitive enough to persist in files.

When utilizing a node app you know you have a functioning JS environment at your disposal. So to me it’s unfortunate and a little crazy to have to revert to the shell for utilizing these apps, especially in the case of some of the gnarlier commands I see people putting together. So why do people do it, what’s the incentive? I believe the main reason is that it’s often the quickest, most compact way to perform ad hoc operations. You don’t need to use an editor, create a file, open the repl, declare variables, or require modules, often you don’t need to quote strings (e.g. file paths), and often you can use short option names that are much more compact than the equivalent API methods. Well, what if those same benefits could be provided for JS:

Punchline

punchline is an experiment in enabling node apps to implement a JS API that is nearly as compact as the CLI (perhaps in some cases more) and that can be invoked in the same way (shell one-liners) and share the same benefits, such as command history. How does it do this?

  • Provides an API for setting options that mimics CLI options. Short and punchy. For example, methods with the same names as short options instead of the regular API method names and a compact syntax for boolean options (expressed as properties instead of method calls: .e(x).d.e(y) sets boolean option d).

  • Eschews the need to quote strings for common cases.

  • Eschews the need to comma delimit lists for common cases (so instead of the typical comma+space between items, just space).

  • Methods that collect space delimited lists of arguments into arrays.

  • All option setting methods are chainable.

  • Eschews the need to require() modules.

  • Reduces the role of the shell to passing a string of (almost) JS to a node CLI.

So punchline commands are written in JS?
Well, sort of. The punchline “language” is neither a strict subset nor superset of JS: it’s a JS-like dialect, pseudo-JS, almost-JS. Sort of a DSL for using JS as a replacement for CLI scripts. How does it differ? For input conforming to certain requirements you can skip quotes for strings and skip commas for delimiting lists (like function args and array elements). It does not support arbitrary JS expressions: it’s optimized to allow the foregoing to support CLI-like use at the expense of supporting the full expressiveness of JS. For example, you could not use the + operator to concatenate strings in a punchline command.

Here’s an example of what a punchline command could look like, using browserify (exactly how to integrate punchline with modules and invoke it is a work in progress).

browserify CLI:

browserify a.js b.js c.js -d -r x.js:X -o bundle.js

punchline equivalent:

punchline browserify '(a.js b.js c.js).d.r(x.js:X).o(bundle.js)'

In this case the part in the single quotes is only 1 char longer than the equivalent part of the CLI command. Some sort of aliasing scheme to shorten the beginning part of the command, e.g. to something like the following, would bring punchline commands closer to length parity with CLI commands (or exactly / shorter in these cases):

punch.b '(a.js b.js c.js).d.r(x.js:X).o(bundle.js)'
// or
punch 'b(a.js b.js c.js).d.r(x.js:X).o(bundle.js)'

How does it work?

The basic architecture is this:

  • Punchline is invoked with some indication of what node program you want to run and a string of pseudo-JS to interpret and convert to real JS.

  • Punchline invokes an adapter for the node program you want to run. The adapter specifies option names and types and an options collecting object is instantiated using that data.

  • The real JS that punchline produced is executed in the context of the options collecting object (thereby populating it).

  • The adapter is invoked with the populated options object so that it can process the data appropriately, e.g. massage the input, call a constructor, set options, call methods.

Punchline ships with an options collection implementation that handles several common types of options:

Boolean

Say a node CLI has a boolean option -d|--debug. In punchline, that option can be set in the following ways:

// True
.d
.d(1)
.d(true)
.debug
.debug(1)
.debug(true)

// False
.d0
.d(0)
.d(false)
.debug0
.debug(0)
.debug(false)
Simple assignment

Assign a single value to a parameter.

.s(whatever)
Accumulation (array)

Collect multiple values for a parameter. Say a node CLI has an option -e|--entry that can occur multiple times. The following are equivalent in punchline:

.e(one).entry(two).e(three)
.e(one two).entry(three)
Subarg

Browserify supports a subarg syntax like: $ browserify -t [ foo --bar=555 ] main.js for passing options to transforms. Punchline has 2 ideas for addressing this situation:

Stock
punchline browserify '(main.js).t(foo {bar: 555})'
Punchlines within punchlines within punchlines

If the transform also has a punchline adapter, then you could do:

punchline browserify '(main.js).t(foo.bar(555))'

It’s worth noting that issuing punchline commands as illustrated here won’t accommodate shell expansions like globbing, brace expansion, tilde expansion, variable expansion, etc. You could handle this by concatenating in your shell command, e.g.:

punchline browserify '.e(' *.js ')'

Needless to say, that gets ugly fast. Probably a better general solution would be for an app to choose to handle some of that in its adapter if it’s important. For example, run the e values in the options object through a globbing module.

At the time of this writing the lexing is a horror show, I know. In other words, it is not horrorshow.

Demo using the browserify API: https://github.com/jmm/punchline-demo.

Leave a Reply

Your email address will not be published. Required fields are marked *

Note: Comments are moderated. Spam comments will never be published.

Is this comment spam?