Why I use Nix and make(1) to develop

{{< alert class="info" >}} Initially, this blog post was called "Developing with Nix and make(1)", however, I thought to rename it after a brief discussion with soywod on using Nix as the solo build system. {{< /alert >}}

Today, Magnus, a friend of mine, posted a love letter to make. In his post he talks about how powerful and reliable make(1)¹ is, I really recommend you go there and read it too!

In the meantime, he also posted his blog post to lobste.rs. It was a really interesting thread as people discussed from alternatives (such as just for a command runner) to lesser known features as running python as the default shell. I went ahead and made a comment about using Nix and Makefiles as development tools and u/dkl made a question that really made me think:

I don’t understand why Nix alone isn’t sufficient. Can you elaborate on your setup a little?

---u/dkl

I admit that my message was not clear: "using make(1) with Nix to make dependencies easier to deal with". When I wrote this, I was thinking about the concept of prerequisites make has on its rules. Which is nothing more than "in order to generate this file, this other thing has to be built successfully".

Meanwhile, this made me wonder where I found Nix to not be sufficient on my setup, more specifically, on my blog setup. As mentioned down the same thread, this blog has a relatively simple build process:

1. Content is written on a Org file
2. GNU Emacs transforms this file into multiple hugo-compliant Markdown files
3. Hugo generates a website from the Markdown files
4. A gzipped archive is generated
5. The archive is sent to sourcehut pages

If each step was represented in a shell line, it would look like this:

# edit the Org file
emacs
# generate Markdown files
emacs $(pwd) --batch -load export.el
# generate website
hugo
# create gzipped archive
tar -cvzf site.tar.gz -C public .
# publish website to sourcehut pages
hut pages publish site.tar.gz --domain glorifiedgluer.com --not-found 404.html

Building and publishing with Nix

This is the perfect scenario for Nix as I don't have any external dependencies to be fetched during the build steps. OK, how would one build the website with Nix? It would probably look like this:

packages.website = stdenv.mkDerivation {
  name = "glorifiedgluercom";
  src = lib.cleanSource ./.;

  buildInputs = [
    emacs-nox
    hugo
  ];

  configurePhase = ''
    emacs $(pwd) --batch -load export.el
  '';

  buildPhase = ''
    hugo
    tar -cvzf site.tar.gz -C public .
  '';

  installPhase = ''
    mkdir -p $out
    cp -r site.tar.gz $out
  '';
};

Running nix build .#website on this derivation would get you a site.tar.gz file in a result directory. More than just declaring the build steps, this will also ensure that all the dependencies needed to build the website would be the same byte-by-byte.

What about the developing experience?[^fn:2] How would I run hugo serve to get a feedback loop while I write and how would I publish it? You can tell that my feedback loop is not going to happen inside that derivation above as I have to keep running nix build in order to see how things look like. Fortunately, this is easy to fix with nix run.

# this is using the helper function from github:numtide/flake-utils
apps.run = utils.lib.mkApp {
  drv = pkgs.writeShellScriptBin "run" ''
    ${emacs}/bin/emacs $(pwd) --batch -load export.el
    ${hugo}/bin/hugo serve
  '';
};

Now I can just run the command nix run .#run and I'll have hugo serving my website locally. The only missing step now is publishing it with hut, the sourcehut CLI.

apps.publish = utils.lib.mkApp {
  drv = pkgs.writeShellScriptBin "publish" ''
    ${hut}/bin/hut pages ${website} site.tar.gz \
        --domain glorifiedgluer.com \
        --not-found 404.html
  '';
};

Running nix run .#publish will now publish the website to sourcehut pages. Note that I'm using the derivation we created above, this will ensure that Nix actually built the package before publishing. How cool is that? It's really cool, but we are missing something here: dependencies!

Makefile to the rescue

As previously mentioned, make has a concept of prerequisites. This is exactly what we need to ensure all dependencies are met before publishing the website.

Makefiles have something called target, it is supposed to be mapped to a file. If not, it is called a PHONY target. You can understand PHONY targets as command runners. Let's write our Makefile to take care of our website build steps:

all: publish

	content:
	emacs $(pwd) --batch -load export.el

public: content
	hugo

site.tar.gz: public
	tar -cvzf site.tar.gz -C public .

.PHONY: publish
publish: site.tar.gz
	hut pages publish site.tar.gz \
		--domain glorifiedgluer.com \
		--not-found 404.html

.PHONY: run
run: content
	hugo serve

As you can see, we have targets such as public and content defined in our Makefile, this means that make will generate the public directory if it notices that something changed. Our publish target is also interesting, it takes site.tar.gz as a prerequisite, and site.tar.gz takes public as one too. This will ensure that every dependency on our build is met before actually publishing.

Leveraging Nix for package dependencies

One of the nicest things Nix provides, if not the nicest one, is the nix develop command. This can be used to give us a shell environment with all the packages needed to build our project.

devShells.default = mkShell {
  buildInputs = [
    emacs
    gnumake
    hut
    hugo
  ];
};

If you run nix develop, you'll be thrown at a shell session with these packages on $PATH. However, you can also run nix develop -c <cmd> to run commands without entering the shell. Do you see where we are heading? We can run nix develop -c make and have all the needed packages available for make to use.

user@host:~/glorifiedgluercom$ nix develop -c make
rm -rf content
rm -rf public
rm -rf site.tar.gz
emacs  --batch -load export.el
...
hugo
...
tar -cvzf site.tar.gz -C public .
...
hut pages publish site.tar.gz \
	--domain glorifiedgluer.com \
	--not-found 404.html

Published site at glorifiedgluer.com

Wrapping up

After some more thinking about this subject, it was clear to me that using make the way it is presented here might give you the following advantages:

1. You have a standard way to build your system that Non-Nix users can leverage
2. It's relatively easier to reason about the build steps

ON THE OTHER HAND, this also gave me a really interesting idea that got me excited to try. What are the advantages of using Nix as a make replacement?

1. You have a single tool to manage dependencies and build steps

In this case, build steps is such a broad thing on Nix that it would add a lot more points here and I don't think it's fair. You get a discoverable CLI with auto-completion (this is also true for make), caching dependencies/build steps is rather straightforward and you can share everything between your projects with Nix Flakes.

What about the interesting idea? Some time ago I had another idea and wrote about my [personal monorepo]({{< relref "starting-a-personal-monorepo" >}}), this didn't get too far[^fn:3]. Anyway, most of the problems I faced resolved around tooling and how weird it was to switch between multiple languages and tools. I think, however, that it might be able possible to overcome this with the ideas presented here. Let's see how it goes! 😊