Nix – putting the nix in Unix
Are you tired of package managers who leave your system feeling like a clown car overflowing with mismatched dependencies?
Ever yearn for a package manager so reliable it builds software in a pristine, bubble-wrapped environment, ensuring clean results every time? Want to compose your application requirements for a project declaratively in a file and not ad hoc, even if they include different versions of apps you already have installed?
Then I’d like to introduce you to the Nix package manager suite.
Nix is different from your average package manager – because it isn’t one! Unlike traditional package managers that install software into a central location on a system that’s in your Path, Nix installs software into a directory outside of your Path, called Nix Store. Inside are sub-directories with unique cryptographic hashes for each package. These hashes take into account all of a software’s unique and version-specific dependencies, ensuring that the installation is complete, reproducible, identical, and functional. This is later symlinked (if and when you instantiate) to a special profile maintained by Nix – and all of this supports versioning inherently, so rollback’s a breeze. In layman’s terms, Nix takes your requested package with all dependencies and their respective versions, “puts it somewhere” outside your environment (Path), and when you call upon it, “adds” it to your environment via a symlink. Because of these unique properties, Nix enables you to install different versions of the same package without hassle or dependency hell.
The NIXtory
Initially conceived as a thesis project in 2003, its inception predates the smartphone revolution and even the rise of social media, Nix is first and foremost a complete build system comprised of a couple of components. It is a dynamically typed, domain-specific, functional programming language, an interpreter, and a set of tools that are focused on declaratively describing/building everything: OSs, software, shells, containers – and more.
Nix – the package manager – is a unique package manager suite of tools that offers a reliable, reproducible, and portable way to manage software using the Nix language. In this article, Nix and the package management aspect of Nix (comprised of several tools, here mainly nix-shell) will be referred to collectively as just Nix for simplicity.
Because of its unique and novel approach to reproducibility and caching, among other things, Nix has seen widespread success and is also a full distribution, with 5000 active maintainers and counting. But let’s backtrack for a bit and explore where does this Nix we’ve been talking about all this long get packages.
The NIX package repository
I sometimes liken Nix to Terraform for software (although the hardcore Nix enthusiasts might not like that comparison!) – when you declare you want a version of a software, Nix interprets that and gets it done for you “automagically”, from its Nixpgks package repository.
Nixpkgs is a large repository of software packages that are built for Nix, constantly updated, and includes the source code for NixOS, the Linux distribution that uses Nix for its entire build and system configuration. With over 80,000 packages (according to repology), Nixpkgs boasts the largest collection of packages out of all the tracked Linux distros.
This collection, in turn, makes it easy to find nearly any software you need – including the very latest. The collection is comprised of .nix files written in the Nix language (called Nix expressions) that have instructions for the build system on how to build the package; most of the time a build (i.e compiling) will not be required as Nix has a sophisticated system of trusted cache registries – If someone already compiled that specific lolcat version 1.3.2 for the RiscV processor – Nix will know and bring you that Binary without needing to compile again (unless you want to!).
Note that NixOS, the Nix-based distro, is a very interesting OS; however, let’s focus on the power Nix brings to your favorite, existing Linux distro or Mac. NixOS is in itself a huge subject that deserves a full-length feature starring Ryan Gosling.
A word on docker
Before we proceed, some of you might ask – why not docker?
While Docker has a lot of merits, complete reproducibility is not one of them. Docker images tend to start with a “FROM” base image, with a lot of inter-dependencies, and if you need an exact copy in time – docker can’t easily facilitate that. In contrast, Nix always builds from source, and has a complete snapshot of all dependencies with their respective versions, for the version of the application you want. There’s also a slight overhead, and not everyone’s got the Docker Command Bible tattooed on their forearm. Nix – even, or especially, for simple package management – is more akin to apt / yum and mixes the ease of “simply installing” with unique “powers” such as PATH automation, “concurrent” conflicting versions of the same app, etc. At the very least, both are great tools that can live together and are not necessarily competing.
Knowledge of the actual Nix language or functional programming is not required but highly recommended. Nothing better illustrates Nix’s usability than a demo, so let’s set the scene.
Installing Nix and setting up (the package manager)
Before we can embark on our quest of discovery, we need to have Nix installed.
The installation should be simple – this link will bring you to the installation page, where it’s just one short
sh <(curl -L https://nixos.org/nix/install) --no-daemon for us Linux users, and following instructions away to set it up.
Note that Nix supports both Single-user and Multi-user installations, and goes into detail explaining the difference between the two on the linked page. I have personally chosen to install as a Single-user installation, as the enhanced security and isolation afforded by Multi-User installation currently matter less to me, and I appreciate the easier uninstall and less involved install.
Also note that for you Mac users out there, Nix offers better integration with system components than on regular, non-NixOS Linux distributions. If you’d like to not only use the package management aspect of Nix this article describes, but also declaratively manage system components and “hard” installations (Imagine that installing docker is just putting a system.virtualisation.docker = enabled in a configuration file, applying it, and Nix will do all the heavy lifting for you!) – Follow this guide.
With that out of the way, let’s get started.
Trying it before installing it
Say you’re working on a project at work, and you have some one-off components you need to manage with gcloud, the CLI tool for the Google Cloud Platform. You know that you’ll need them rarely – if ever, besides this one time, and depending on your platform of choice, getting it installed can be a drag. On Ubuntu, you can follow a long, arduous guide that has you adding a source to your apt configuration (and good luck remembering you did that, future you!), or use Snaps – but they are a hot potato within the community. On Mac, as usual, you can follow an even longer manual process – or use Brew, while bracing yourself for that familiar “Did you know you have 23 packages out of date? Let me S L O W L E Y update all of them, then myself, and then I MIGHT remember to download and install gcloud E V E N S L O W E R”…
Or, we can just use Nix!
Since we already have Nix installed and just want to get into a terminal session where gcloud is available only for that session, all we need to do is make sure Nix has our package, discover that the package that provides this is called “google-cloud-sdk”, and instantiate a new shell with gcloud by using nix-shell -p google-cloud-sdk and that’s it, we have gcloud! (Notice that I’ve already had this previously installed so there’s no copying of paths).
Once we Ctrl-D ourselves out of that shell, gcloud will no longer exist in our system. Amazing. What’s happening behind the scenes is that Nix downloads, builds (if no cache for your architecture exists), and stores the application outside of your PATH at the Nix Store. When we want to use that package again (like if we instantiate a new shell with the same command from above), Nix just symlinks that package into our Path. Garbage collection for such packages happens when you restart, or manually with nix-collect-garbage.
How about we make it last
“OK” I hear you say, “that’s neat and all but what if I want to install things and have them stay? Also, what’s this business about more than one version of an app at a time like you said at the intro?”. First, get out of my head, and second – let’s get to demonstrating that, too!
Come, drink your cup of Joe, and sit down to work on a personal project. This project requires some complicated bash commands, some boilerplates, and even – heaven forbid! – some regexs. You then decide you want to try the new “Copilot from the command line” to assist. Supposing you already are a paying member of the AI revolution, this should be a simple issue of installing gh – the Github cli tool – and starting your journey to AI-assisted productivity.
Ah, but there’s a snag. You already have gh installed on your machine, but it’s connected to your work account, which does not enable Copilot! It’s also a very old version required by IT, that can’t even support Copilot! There’s the temporary Nix package installation we already learned about, but Nix can also save the day in this case!
At the root folder of the project you plan to oh so productively work on for the better part of the coming months, create a new file called shell.nix. The best way to do this is to use your editor-of-choice (I use Vim btw), and enter this Nix expression inside:
{ pkgs ? import <nixpkgs> }:pkgs.mkShell {
packages = with pkgs; [
gh
];
}
Save and exit this lovely file. Now, every time you visit this folder, simply issue nix-shell,
And voila! You’ll be dropped into a shell session with all the packages you’ve specified in the file. Like last time, once you close or Ctrl-D out of that shell, those apps will be gone! No
More headache of manual management of multiple versions.
But let’s take it even further: Having multiple versions of Python on the same system is notoriously messy. Yes, there are ways around it, but aren’t we freshly anointed clerics of Nix?
If – for whatever bad reason – you also need to use the very up-to-date version 2.7.18.7 of Python, we need to tweak our shell.nix file a bit:
let
pkgs = import <nixpkgs> { config = {
permittedInsecurePackages = [
python-2.7.18.7"
];
};
};
python = pkgs.python27;
in
pkgs.mkShell {
packages = with pkgs;[
python
gh
];
}And… YES! Marvel away at having a deprecated, years-old version of Python, at your disposal. Give it a whirl – issue some print text commands and marvel that they do not require () as with Python3. A note about this specific use case: One, it’s a bad idea to use a deprecated version of a language; and two – in my test the cached version of Python2 was stale, meaning Nix compiled it for me. Be aware that it may take some time.
Ah, Worried about those already installed versions of gh you have on your system? Or that we messed up Python? Fret not. From within the nix-shell, a simple check with which gh , for example, yields that it is NOT using the one from your normal system; rather the one from the Nix store is used. If a conflict arises – say you have Python on your system as Python 3.X and you drop into a nix-shell with Python 2.X, Nix will make sure only the one in Nix is on your Path, meaning Python 2.
Keeping different shell.nix files for different projects, each holding all the apps you need for that particular project – with ease of usage and management – is, for me, what Nix is all about! It enables you to just get things done, with all requirements defined and obscured away in a file; you can also trust that your defined environment will be the exact same each time you call on it, with the same versions – and if you send that Nix file to someone else on your team, so as long as he has Nix installed they can reproduce your exact set up to a Tee.
On that note, two additional and highly recommended tools work in tandem with Nix – Devenv and direnv. They complement the per-project management and make it even easier and more automatic to define services, env variables, and more!
Alas, those will have to wait for another time.
What do you think? Have you fallen for the magic of Nix, or will you keep your hand and just keep using Brew for your next project?