Documentation
Nix (and garnix) avoids rebuilding anything that's been built before at the package level, but sometimes we want caching at the module (file-object file) level. We want, in other words, incremental builds.
garnix has support for this. In order to use it:
- Add https://github.com/garnix-io/incrementalize/ as a flake input named garnix-incrementalize (it must be named this).
- Wrap your entire flake output in a call to garnix-incrementalize.lib.withCaches.
- Make the attributes that you want to incrementalize be a function taking a caches argument. When run, this will be either the empty directory if no cache is available, or a nix store path with the previous cache if it is.
- Make your derivations outputs an extra intermediates output with your cache. This will become the cache argument of the next commit!
- Add lines to your garnix.yaml stating when you want to incrementalize. We recommend:
incrementalizeBuilds: excludeBranches: - main
(Substituting main for whatever branch is closest to production.)
Here is an example flake file that uses incremental builds to keep a log of the consecutive incremental runs (in practice you'd do something more useful with the cache!):
{ inputs.garnix-incrementalize.url = "github:garnix-io/incrementalize"; outputs = { nixpkgs, garnix-incrementalize, ...} : garnix-incrementalize.lib.withCaches { packages.x86_64.default = cache: pkgs.mkDerivation { name = "inc" outputs = [ "out" "intermediates" ]; buildPhase = '' mkdir $intermediates cat ${cache}/run-logs > "$intermediates"/run-logs echo "I ran again" > "$intermediates"/run-logs echo normal-build-stuff > $out ''; } ; }; }
A few notes:
- garnix currently only searches up to 5 parent commits for an incremental candidate;
- garnix ignores any commits that haven't finished building, or failed in any way;
- if you have one flake output reference another, do it via the self attribute rather than via the rec keyword so that what you get is already the proper output rather than the function.