New make

GNU Make is a fabulous, underrated tool.

I use it a lot for documenting my data organization — sort of as a lab notebook. Even if I keep all the derived files in source control, it’s a handy way to keep track of where they came from, because shell history is relatively ephemeral (I wish it were less so, but that’s a topic for another day, and not the proper solution to documenting data provenance anyway).

Make has a couple of warts that drive people away:

• Significant whitespace

  This isn’t so bad, really. I tell Sublime Text to {"draw_white_space": "all"} while in Makefile syntax mode, and make is quick to complain about invalid indentation. I’ve never run into issues with unintended scoping / level changes, as I have with, for example, Python.

• Whitespace in filenames

  This is a legitimate issue. In the ideal world, whitespace would never appear in filenames that are ever going to be touched at the command line. Instead, we live in the polar opposite: in the software development world.

• Arcane symbols

  I.e., syntax like $@, $<, $+, $*, $(FILES:%=%.bak), $(@D), etc. I’ll admit, it would be nicer if there were long-form versions for these targets. But honestly, this isn’t too far from bash/shell syntax. And I never (?) use $^ and $?. $@ and $< alone will get you 90% of the functionality you need.

Remakes

There are lots of language-specific Make remakes with giveaway names like rake (for Ruby) and jake (for JavaScript). But these are not intended to be general purpose tools, as GNU Make is. I see lots of projects that use Grunt or Gulp as general purpose build tools, even beyond JavaScript-targeted projects, but the syntax for those configurations is overly expressive — it’s just JavaScript.

There have been a couple efforts to re-make Make in a more general sense:

• drake
  • Written in Clojure, “Drake is a simple-to-use, extensible, text-based data workflow tool that organizes command execution around data and its dependencies”, i.e., a “Make for data”.
  • The general design principles seem sound, but the syntax is obscure, and development seems to have trailed off, leaving a partial implementation of a 60-page Google Docs specification that is very much still a WIP. It’s interesting to see the discussion, but it’s no substitute for proper documentation.
  • On the JVM, plus Clojure, so: long start-up times.
  • Does too much, perhaps? Support for HDFS and S3 seems handy, but smells of too much, too quickly.

  • No support for pattern matching on filenames!

    That’s, like, the best part of Make.

    Want to describe how to create a plain text file out of a PDF?

    • In Make: write a %.txt: %.pdf rule. Easy-peasy.
    • In Drake: sorry you want to do what? Lol, no.
  • Would be a good reference for a modern implementation, but at its core it’s a bit too purpose-specific.
• mach
  • “A remake of make (in ClojureScript)”
  • Leads with a lot of ranting about how Make syntax is “baroque” and a barrage of scare quotes that are actually decently descriptive of various things Make should be proud of.
  • Their config files are pure EDN, which is cute, but I’m not sure that’s appropriate. “Data notation” is a broad term, but I don’t think I’d say Makefiles qualify.
  • More imperative than declarative, unfortunately. Feels like a Webpack config. More declarative than Grunt or Gulp, but still not the clean syntax of Make.
• mk
  • “Mk: a Successor to Make”
  • The syntax is very, very similar to Make’s. But it’s unclear what it does better, or what the differences are.
    • IMO it’s kind of weird to write a “successor” but not include in the documentation why the predecessor needs succeeding?
  • Written for Unix, ported to Plan 9, then ported back to Unix.
• ninja
  • Low-level counterpart for a higher-level config-file-generating build system.
• tup
  • “file-based build system” – but it gets its improved file watching by mounting a temporary FUSE filesystem.
  • Author is fuller of himself than Zed Shaw. Nevertheless, looks interesting.
  • Syntax looks minimal. |> — F#-inspired?

Feature requests

make has far too much of a backwards compatibility obligation to fix its quirks, but here are a few things I think are painfully missing functionality:

• Embedding shell scripts / short functions into Make + better shell integration
  • In lots of cases I have a few bash scripts huddled around my Makefile because writing a little snippet of bash in a Make target is tedious. Each line of a Make target creates a new shell, so any inline shell scripts must be on one line or peppered with newline escapes. Also, the similar but not-identical nature of variables in Make vs. bash makes using variables in this environment potentially confusing.
• Multiple patterns
  • Right now, you can only use one % pattern in the left side of a target, and one % on the right. This simplification is handy and sufficient for 95% of use-cases, but helpless when you need to manipulate both a prefix and a suffix of a filename.
  • Relatedly, you cannot use a % pattern twice on the right side, even if you intend for it to be the same thing.
• Filename flexibility
  • There are people out there who put spaces in filenames. I don’t know who they are, and we will never be friends, but sometimes I want to or have to use their data, and I’d rather not have to rename it first.
• Globbing and filtering (find-like functionality) should be first-class
  • I can shell out to find, or my shell’s globstar implementation, but this is basic stuff that Make should handle.
• Built-in fswatch (etc.) functionality
  • Make is lightweight, so it’s not hard to fswatch src.code | xargs -I % make src.bin, but I need one of those processes hanging around in my terminal for each target I’m watching.
• The first target being the default (the inferred target when calling make without arguments) is a nice shortcut if you’re aware of it, but it can bite you if you forget. Perhaps better to output list of targets when calling with no argument, unless a default is explicitly defined in the configuration?