NixLang | Christopher Coverdale Blog

Nix 101: Part 3

Wed, 25 Sep 2024 00:00:00 +0000

The third part of Nix 101 will explain and go through nix flakes.

Flakes - what are they and why

Nix Flakes are another way to create reproducible environments. They are arguably more reproducible than shells and are the community favoured strategy in creating environments.

Flakes improve on Nix Shells by addressing the problem inherent in pulling dependencies from Nixpkgs.

In Nix shells, the upstream version of Nixpkgs is not locked, meaning two engineers using the same shell file might use different versions of Nixpkgs (e.g. nixpkgs 1.2.3 vs nixpkgs 1.2.4). This could result in different versions of the dependencies being pulled, leading to breaking builds or inconsistencies between the different environments.

With Nix Flakes, the version of nixpkgs and source is explicitly declared, this generates a flake.lock file that locks in the version.

Example Flake

Using our zig shell in the previous blog post, we’ll create a flake to lock in the nixpkg dependency version.

{
 inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

 outputs = { self, nixpkgs, ... }:
 let
 system = "x86_64-linux";
 pkgs = import nixpkgs { inherit system; };

 # Helper function to build different versions of each zig version.
 zigDerivation = { version, hash } :
 pkgs.stdenv.mkDerivation {
 pname = "zig";
 version = version;

 src = pkgs.fetchurl {
 url = "https://ziglang.org/download/${version}/zig-linux-x86_64-${version}.tar.xz";
 sha256 = hash;
 };

 buildPhase = ''
 mkdir -p $out/bin
 cp ./zig $out/bin
 '';
 };
 in
 {

 packages = {
 x86_64-linux.zig0130 = zigDerivation { version = "0.13.0"; hash = "1FMS5h68xIAyt3vEz3/WkVwR+hbkqtEWtmyUaCESMOo="; };
 x86_64-linux.zig0120 = zigDerivation { version = "0.12.0"; hash = "x66Ga4p2pWji1c/TH+ic22Kb3RYf3VAYsppKChcEXK0="; };
 };

 devShells.x86_64-linux.default = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0130 ];
 };

 devShells.x86_64-linux.zig0130 = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0130 ];
 };

 devShells.x86_64-linux.zig0120 = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0120 ];
 };

 };
}

There’s a lot more going on than the shell, we’ll step through each line:

inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; - This line declares the upstream source which will contain the version of nixpkgs for this flake
We create the derivation and build phase for the flake. We are downloading and going to use the parameters of version and hash when creating this derivation
This will download the binary using fetchurl and copy the binary to the $out/bin directory, which is the environment for the shell
Important to note, this is still not totally reproducible since this flake would only work correctly on systems that are x86_64-linux

 outputs = { self, nixpkgs, ... }:
 let
 system = "x86_64-linux";
 pkgs = import nixpkgs { inherit system; };

 # Helper function to build different versions of each zig version.
 zigDerivation = { version, hash } :
 pkgs.stdenv.mkDerivation {
 pname = "zig";
 version = version;

 src = pkgs.fetchurl {
 url = "https://ziglang.org/download/${version}/zig-linux-x86_64-${version}.tar.xz";
 sha256 = hash;
 };

 buildPhase = ''
 mkdir -p $out/bin
 cp ./zig $out/bin
 '';
 };
 in
 {

After definig the inputs (the derivation, system and pkgs) we create the zig derivation for certain versions (0.13.0, 0.12.0)
We create the shells for each version, using the packages built from the derivations

 in
 {

 packages = {
 x86_64-linux.zig0130 = zigDerivation { version = "0.13.0"; hash = "1FMS5h68xIAyt3vEz3/WkVwR+hbkqtEWtmyUaCESMOo="; };
 x86_64-linux.zig0120 = zigDerivation { version = "0.12.0"; hash = "x66Ga4p2pWji1c/TH+ic22Kb3RYf3VAYsppKChcEXK0="; };
 };

 devShells.x86_64-linux.default = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0130 ];
 };

 devShells.x86_64-linux.zig0130 = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0130 ];
 };

 devShells.x86_64-linux.zig0120 = pkgs.mkShell {
 buildInputs = [ self.packages.${system}.zig0120 ];
 };

 };
}

And finally, to run the dev shell within the flake:

// example: nix develop <path-to-flake-folder>/#<shell-name>

nix develop ./zig-flake/#zig0130

A flake.lock file will be generated after running the flake:

{
 "nodes": {
 "nixpkgs": {
 "locked": {
 "lastModified": 1722185531,
 "narHash": "sha256-veKR07psFoJjINLC8RK4DiLniGGMgF3QMlS4tb74S6k=",
 "owner": "NixOS",
 "repo": "nixpkgs",
 "rev": "52ec9ac3b12395ad677e8b62106f0b98c1f8569d",
 "type": "github"
 },
 "original": {
 "owner": "NixOS",
 "ref": "nixos-unstable",
 "repo": "nixpkgs",
 "type": "github"
 }
 },
 "root": {
 "inputs": {
 "nixpkgs": "nixpkgs"
 }
 }
 },
 "root": "root",
 "version": 7
}

Summary

Flakes solve a reproducibility issue in nix shells by pinning the upstream nixpkg version, allowing different engineers to run the same flake and ensure they have the exact same dependencies.

Nix 101: Part 2

Tue, 30 Jul 2024 00:00:00 +0000

The second part of Nix 101 will use the language features described in Part 1, and create a real life use case.

Creating a Zig package

In this example, we are going to create a derivation of the Zig language.

Specifically, we want to be able to build a derivation where a certain version of Zig can be built e.g. “Give me Zig 0.13.0”.

Derivation

First lets define a derivation.

A derivation is a Nix file that represents the build process of software.

A derivation is a Nix expression that declares the dependencies and required build steps to compile/build the sofware.

The process is reproducible meaning, the same inputs always evaluate to the same outputs. The Nix derivation can also be used as an import in shells or other derivations as a dependency.

Shells

Nix Shells are reproducible environments.

This means that we can use the Nix Language to declaretively express the dependencies of a shell and any required hooks in a reproducible manner.

The shell should evaluate to the same output given the same declared input.

These shells can also be isolated from the underlying environment, creating a pure development environment.

Creating the Derivation

We are going to create an example derivation of the Zig language.

First we are going to start with getting Zig version 0.13.0.

This derivation is in a file named zig.nix.

We import stdenv to be able to call mkDerivation to create the derivation.

fetchurl will download the binary and note the hash sha256. This is the hash of the binary. This is important since it provides a reproducible step, if the downloadable content were to change upstream, we can tell that it wouldn’t evaluate to the same hash. This could be due to a malicious upstream change to the software or an accidental one, either way, this protects our derivation and subsequent use of this package as a dependency.

Note the buildPhase. This is a hook in the derivation, and we are saying at this phase, make a directory at $out/bin and copy the downloaded binary to that path.

$out is a Nix environment variable for the designated output path for the result of the output build.

zig.nix

{ stdenv, fetchurl, }:

stdenv.mkDerivation {
 pname = "zig";
 version = "0.13.0";

 src = fetchurl {
 url = "https://ziglang.org/download/0.13.0/zig-linux-x86_64-0.13.0.tar.xz";
 sha256 = "1FMS5h68xIAyt3vEz3/WkVwR+hbkqtEWtmyUaCESMOo="";
 };

 buildPhase = ''
 mkdir -p $out/bin
 cp ./zig $out/bin
 '';
}

We can build this derivation nix-build ./zig.nix and this will build or derivation with a resulting output for Zig 0.13.0.

But let’s say we wanted the option to request different versions of Zig. We can achieve this by passing args to our a derivation.

Let’s create a new nix file, default.nix.

default.nix will allow us to paramertize the build when calling the derivation.

We’re going to pass x2 args:

zigVersion
zigHash

This will allow us to set the zigVersion we want and it’s corresponding hash.

Notice that we call the package with different versions and hashes. We then assign the evaluated result to a named variable: zig0130 and zig0120.

default.nix

let
 nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-24.05";
 pkgs = import nixpkgs { config = {}; overlays = []; };
in
 {
 zig0130 = pkgs.callPackage ./zig.nix { zigVersion = "0.13.0"; zigHash = "1FMS5h68xIAyt3vEz3/WkVwR+hbkqtEWtmyUaCESMOo="; };
 zig0120 = pkgs.callPackage ./zig.nix { zigVersion = "0.12.0"; zigHash = "x66Ga4p2pWji1c/TH+ic22Kb3RYf3VAYsppKChcEXK0="; };
 }

We’ll update the zig.nix derivation file to accept the args as variables.

The derivation can now use and build according to the passed zigVersion and corresponding zigHash.

We can also specify which version we want to build: nix-build -A zig0120

zig.nix

{ stdenv, fetchurl, zigVersion, zigHash }:

stdenv.mkDerivation {
 pname = "zig";
 version = zigVersion;

 src = fetchurl {
 url = "https://ziglang.org/download/${zigVersion}/zig-linux-x86_64-${zigVersion}.tar.xz";
 sha256 = zigHash;
 };

 buildPhase = ''
 mkdir -p $out/bin
 cp ./zig $out/bin
 '';
}

We can take this another step further by creating a development shell environment where we can use the local derivation: nix-shell --argstr zigVersion "0.13.0"

We can import the local derivation using import ./default.nix and then call the named variable in the evaluated attribute. This will grab the built version we require as a dependency.

shell.nix

{ pkgs ? import <nixpkgs> {},
 zigVersion ? "0.13.0",
}:

let
 customPackages = import ./default.nix;
 selectedZig = if zigVersion == "0.13.0" then
 customPackages.zig0130
 else if zigVersion == "0.12.0" then
 customPackages.zig0120
 else
 customPackages.zig0130;
in
 pkgs.mkShell {
 buildInputs = [ selectedZig ];

 shellHooks = ''
 echo "Running Zig shell with version: ${zigVersion}"
 '';
}

Summary

Nix allows us to declaretively create expressions that evaluate to reproducible build packages and development environments.

Nix 101: Part 1

Mon, 29 Jul 2024 00:00:00 +0000

This post goes over the basics of the Nix Language and in later posts will cover the applications of Nix.

What is Nix?

Nix is a system to create reproducible builds and shell environments. Nix has a purely functional language to express the building of reproducible shell environments.

This post will go over the basics of the language.

Variables

Nix has basic variables found in most high level languages.

string = "Hello, World!";
integer = 1;
bool = true;
list = [1 2 3];

Variables - let..in..

Variables can be defined using the let ... in ... syntax.

The variables are defined in the let scope and then used for evaluation in the in scope.

let
 x = 1;
 y = 2;
in
 x + y

>> 3

Strings

String interpolation is achieved using ${variable}

let
 name = "Foo";
in
 "hello ${name}"

Lists

Lists can be declared using [].

[1 2 3];

Attributes

Attributes are a datastructure, similar to JSON. They are defined using {} and have named key values.

The values can be accessed using dot notation to refer to a key name.

let
 attributes = { x = 1; y = 2; };
in
 attributes.x + attributes.y

>> 3

Recursive Attributes

Recursive Attributes allows the variables within a scope to be referenced using the rec keyword.

rec {
 one = 1;
 two = one + 1;
 three = two + 1;
}

With

Atrributes can be accessed more easily using the with keyword.

with allows access to the attirbutes values without needing to use dot notation.

let
 a = {
 x = 1;
 y = 2;
 z = 3;
 };
in
 with a; [x y z]

Functions

Functions in Nix are lambda functions with a : separating arguments from the function body.

x: x + 1

Attribute sets can be used as arguments.

{a, b}: a + b

Default/optional arguments can be provided using ?.

{a, b ? 0}: a + b

Named arguments can be given using the @ symbol.

args@{a, b, ...}: a + b + args.c

Using let..in.., we can define a function and then call it.

let
 f = x: x + 1;
in
 f 1

We can also pass an attribute set as an argument, which seems to be a common pattern in Nix.

let
 f = x: x.a + 1;
in
 f { a = 1; }

We can also declare functions and variables in the let scope and then apply them.

let
 f = x: x.a + 1;
 v = { a = 1; };
in
 f v

Functions can also be immediately callable by wrapping it around ()

(x: x + 1) 1

Functions can be passed to list and have the function evaluated and the result stored in the list.

let
 f = x: x + 1;
 a = 1;
in [ (f a) ]

Functions can also be passed with out being evaluated along side the arguments.

let
 f = x: x + 1;
 a = 1;
in [ f a ]

Single attribute sets with multiple values can be used as the argument, as mentioned earlier, this is a common pattern.

let
 f = {a, b}: a + b
in
 f { a = 1; b = 1; };

Inherit

This keyword is shorthand for assigning variables from the previous scope into the new scope.

The below is the equivalent of:

x = a.x;
y = a.y;

let
 a = { x = 1; y = 2; };
in
 {
 inherit (a) x y;
 }

Summary

Nix is a reproducible build system for shells and environments. Nix uses a functional programming language to express and evaluate reproducible builds.

NixLang | Christopher Coverdale Blog

Nix 101: Part 3

Flakes - what are they and why

Example Flake

Summary

Did you find this page helpful? Consider sharing it 🙌

Nix 101: Part 2

Creating a Zig package

Derivation

Shells

Creating the Derivation

Summary

Did you find this page helpful? Consider sharing it 🙌

Nix 101: Part 1

What is Nix?

Variables

Variables - let..in..

Strings

Lists

Attributes

Recursive Attributes

With

Functions

Inherit

Summary

Did you find this page helpful? Consider sharing it 🙌