{{< alert class="info" >}} tl;dr: check out the entire code for the module and the integration test. {{< /alert >}}
This website was once hosted at sourcehut pages, however, I shifted its content a bit and started adding more and more media. It felt a bit abusive to keep using the service this way. The current bundle is already sitting at about 45 MB of disk size1.
At the moment, it is being served at my own server behind a reverse proxy alongside other websites I own. This is the right time to share how I manage their infrastructure with the help of Nix modules!
Well, Caddy is already stupidly easy to use for this use case, but some things were still missing from the setup:
- Automatically create the website directory
- Making it easier to update the website files
- Allowing me to select between Caddy and Nginx (just for fun ☺️)
- Always redirect HTTP traffic to HTTPS
Alright, in order to start our module, let's recap how a Nix module look like. According to the unofficial wiki, they have the following structure:
{
imports = [
# paths to other modules
];
options = {
# option declarations
};
config = {
# option definitions
};
}
This is a rather simple structure to follow, we define our new options
inside the options attribute and the result of it inside the
config. For a practical example, if you were to create a meta
module that configures the domain of your computer, you would do this:
# ommited definitions ...
{
options = {
glorifiedgluer.meta = {
domain = mkOption {
type = types.str;
default = "gluer.org";
};
};
};
config = {
networking.domain = config.glorifiedgluer.meta.domain;
};
}
Considering this, a good starting point is to define the "API" of your module before writing it. I mean, how should it look like? In my case, when being used, I want it to look like the following:
{
ermo.services.webserver = {
enable = true;
webserver = "caddy"; # can also be `nginx`
dataDir = "/var/lib/www"; # this will be the default value
user = "my-user";
websites = [
{ domain = "gluer.org"; }
];
};
}
If I were to put into words what each option should represent:
enable : Whether this module is on or not
webserver : Which web server to use by default
dataDir : Where to store the data
user
: The user that will own the dataDir
websites : Which websites are going to be served
Implementing the options
In order to implement the module, let's start this out with its options:
options.ermo.services.webserver = {
enable = mkEnableOption (lib.mdDoc "ermo webserver");
dataDir = mkOption {
type = types.path;
default = "/var/lib/www";
description = lib.mdDoc "The directory where webserver will look for static websites.";
};
user = mkOption {
type = types.str;
default = config.services."${cfg.webserver}".user;
description = lib.mdDoc "User account under which dataDir will be created.";
};
group = mkOption {
type = types.str;
default = config.services."${cfg.webserver}".group;
description = lib.mdDoc "Group account under which dataDir will be created.";
};
websites = mkOption {
default = [ ];
description = lib.mdDoc "Websites that will be setup for static hosting.";
type = types.listOf (types.submodule {
options = {
domain = mkOption {
type = types.nonEmptyStr;
default = null;
description = lib.mdDoc "Website domain.";
};
};
});
};
webserver = mkOption {
type = types.enum [ "caddy" "nginx" ];
default = "caddy";
description = lib.mdDoc "Which webserver to use.";
};
};
Whoa, this is a lot to unpack at once! Let's break it down and see what we did so far.
First, this has defined the options
ermo.services.webserver.<options>, one of which is the enable
option that receives the mkEnableOption function. This function
basically generates the code for us to turn this option into a toggle
on-off.
Next, we have the dataDir option that expects a value of type
path, has the default value of /var/lib/www and also a pretty nice
description.
Now, this is the interesting part, the user option expects a value
of type str and its default value is a weird attribute... What's it?
Do you recall that at this article's beginning I showed you the
structure of a module? Well, this is that structure in action!
There's a certain convention of declaring a variable called cfg that
contains the value of your own module on the top of the file. In this
case, we have this:
let
cfg = config.ermo.services.webserver;
in
#...
This cfg variable has the values set on our own module, so, in the
case of the user option that has
config.services."${cfg.webserver}".user as the value, we are
basically saying: "By default, this option will have the same value as
the user of our webserver of choice". The group option follows
the same logic.
For the websites option, we declared that it expects a list of
submodules, of which expects an option called domain that expects a
non empty string.
As mentioned before, the webserver options is just expecting either
a string caddy or nginx as its value. In this sense, Nix will
check at runtime if we passed a valid value contained on the enum
type.
Implementing the configuration
What good does a module do if it has no actual impact? Well, let's
change this and implement the configuration part. For this, we need to
write the code inside the config attribute of our module:
{
# the options are here ...
config = mkIf cfg.enable (mkMerge [
# our code will be inside of this list
]);
}
Here we use the mkIf function that does exactly what it says. If
cfg.enable is true, then return the value returned by mkMerge.
Alright, let's start the first portion of our configuration that will
be enabling the selected web server and creating the directories for
the websites.
{
services."${cfg.webserver}".enable = true;
systemd.tmpfiles.rules = [
"d ${cfg.dataDir} 770 ${cfg.user} ${cfg.group} - -"
]
++ map
(website:
"d ${generateWebsiteRoot website} 770 ${cfg.user} ${cfg.group} - -")
cfg.websites;
}
This will enable the web server with
services."${cfg.webserver}".enable. As previously shown,
cfg.webserver is either caddy or nginx, so this string will be
evaluated as services."caddy".enable or services."nginx".enable.
The other option called systemd.tmpfiles.rules receives a list of
directories and its permissions that will be created.
The missing part is the specific configuration for each web server as
Caddy and Nginx have different options on their official NixOS module.
However, we should first ensure that we only configure the selected
web server, not both at the same time. One way to do it is to use
mkIf again:
{
config = mkIf cfg.enable (mkMerge [
(mkIf (cfg.webserver == "caddy") {})
(mkIf (cfg.webserver == "nginx") {})
]);
}
Well, the boring part is the configuration itself:
(mkIf (cfg.webserver == "caddy") {
services."${cfg.webserver}".virtualHosts =
foldr
(website: acc:
recursiveUpdate
{
"${website.domain}".extraConfig = ''
encode gzip
root * ${generateWebsiteRoot website}
file_server
'';
}
acc)
{ }
cfg.websites;
})
(mkIf (cfg.webserver == "nginx") {
services."${cfg.webserver}".virtualHosts = foldr
(website: acc:
recursiveUpdate
{
"${website.domain}" = {
forceSSL = true;
enableACME = true;
root = "${generateWebsiteRoot website}";
};
}
acc)
{ }
cfg.websites;
security.acme.acceptTerms = true;
})
That's it, this will do exactly what I wanted. It will enable the chosen web server, create the directories for the domains I want to host and configure the web server to serve them. Now, to upload the website you can just run this command:
$ rsync -azP --delete <folder> user@host:/var/lib/www/<domain>
Writing tests for the module
That's not a lot of code but it has a lot of moving parts, writing NixOS integration tests is a way to ensure that all of them are working properly. One of the caveats of redirecting all HTTP traffic to HTTPS is that testing becomes harder as we can't really retrieve certificates for our testing Virtual Machines. Well, we can work this out by only disabling HTTPS on the web server test. First, let's declare the machines, one with Caddy and the other with Nginx
nodes = {
caddy = { self, ... }: {
imports = [ self.nixosModules.default ];
ermo.services.webserver = {
enable = true;
webserver = "caddy";
websites = [{ domain = "${domain}"; }];
};
# this test will not ensure if the acme certificate is
# retrieved. In this case, caddy is serving plain http
services.caddy.globalConfig = ''
auto_https off
'';
};
nginx = { pkgs, lib, self, ... }: {
imports = [ self.nixosModules.default ];
ermo.services.webserver = {
enable = true;
webserver = "nginx";
websites = [{ domain = "${domain}"; }];
};
services.nginx = {
virtualHosts = {
"${domain}" = {
# this test will not ensure if the acme certificate is
# retrieved. In this case, ACME is disabled.
enableACME = lib.mkForce false;
forceSSL = lib.mkForce false;
};
};
};
};
};
This will configure two virtual machines, one called caddy and the
other called nginx. Inside them we configure our
ermo.services.webserver for each possible webserver. Now, let's
write the test itself.
The testing library for NixOS allows you to write Python code to validate your virtual machine. How does it look like?
testScript = ''
start_all()
# caddy tests
# ===========
caddy.wait_for_unit("caddy")
# test if directory is created
caddy.succeed("test -d /var/lib/www/${domain}")
# nginx tests
# ===========
nginx.wait_for_unit("nginx")
# test if directory is created
nginx.succeed("test -d /var/lib/www/${domain}")
'';
The code is really easy to grasp, we first start both virtual machines
with start_all(), then with objects that have the name of our
machines. We wait for caddy's unit called caddy to be active
through the method wait_for_unit("<unit>"), then we expect the
directory to exist with this line:
caddy.succeed("test -d /var/lib/www/${domain}")
It's the same for nginx, nothing new there. That's it, we made a Nix
module and wrote integration tests to ensure its behavior stays the
same in future modifications!
-
This is mostly due to generating different image sizes for different screen sizes.↩