Upcoming release

Breaking Changes

  • Lix no longer speaks the Nix remote-build worker protocol to clients or servers older than CppNix 2.3 fj#325 cl/1207 cl/1208 cl/1206 cl/1205 cl/1204 cl/1203 cl/1479

    CppNix 2.3 was released in 2019, and is the new oldest supported version. We will increase our support baseline in the future up to a final version of CppNix 2.18 (which may happen soon given that it is the only still-packaged and thus still-tested >2.3 version), but this step already removes a significant amount of dead, untested, code paths.

    Lix speaks the same version of the protocol as CppNix 2.18 and that fact will never change in the future; the Lix plans to replace the protocol for evolution will entail a complete incompatible replacement that will be supported in parallel with the old protocol. Lix will thus retain remote build compatibility with CppNix as long as CppNix maintains protocol compatibility with 2.18, and as long as Lix retains legacy protocol support (which will likely be a long time given that we plan to convert it to a frozen-in-time shim).

    Many thanks to jade for this.

Improvements

  • Trace which part of a foo.bar.baz expression errors [cl/1505, 1506](https://gerrit.lix.systems/c/lix/+/1505, 1506)

    Previously, if an attribute path selection expression like linux_4_9.meta.description it wouldn't show you which one of those parts in the attribute path, or even that that line of code is what caused evaluation of the failing expression. The previous error looks like this:

    pkgs.linuxKernel.kernels.linux_4_9.meta.description

error:
       … while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … while calling the 'throw' builtin
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Now, the error will look like this:

    pkgs.linuxKernel.kernels.linux_4_9.meta.description

error:
       … while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … while evaluating 'pkgs.linuxKernel.kernels.linux_4_9' to select 'meta' on it
         at «string»:1:1:
            1| pkgs.linuxKernel.kernels.linux_4_9.meta.description
             | ^

       … caused by explicit throw
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Not only does the line of code that referenced the failing attribute show up in the trace, it also tells you that it was specifically the linux_4_9 part that failed.

    This includes if the failing part is a top-level binding:

    let
  inherit (pkgs.linuxKernel.kernels) linux_4_9;
in linux_4_9.meta.description
error:
       … while evaluating 'linux_4_9' to select 'meta.description' on it
         at «string»:3:4:
            2|   inherit (pkgs.linuxKernel.kernels) linux_4_9;
            3| in linux_4_9.meta.description
             |    ^

       … while evaluating the attribute 'linux_4_9'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … caused by explicit throw
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Many thanks to Qyriad for this.

  • Add a build-dir setting to set the backing directory for builds cl/1514

    build-dir can now be set in the Nix configuration to choose the backing directory for the build sandbox. This can be useful on systems with /tmp on tmpfs, or simply to relocate large builds to another disk.

    Also, XDG_RUNTIME_DIR is no longer considered when selecting the default temporary directory, as it's not intended to be used for large amounts of data.

    Many thanks to Robert Hensing and Tom Bereknyei for this.

  • Distinguish between explicit throws and errors that happened while evaluating a throw cl/1511

    Previously, errors caused by an expression like throw "invalid argument" were treated like an error that happened simply while some builtin function was being called:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"

error:
       … while calling the 'throw' builtin
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg "linuz"

       error: linuz isn't the right package

    But the error didn't just happen "while" calling the throw builtin — it's a throw error! Now it looks like this:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"

error:
       … caused by explicit throw
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg "linuz"

       error: linuz isn't the right package

    This also means that incorrect usage of throw or errors evaluating its arguments are easily distinguishable from explicit throws:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg { attrs = "error when coerced in string interpolation"; }

error:
       … while calling the 'throw' builtin
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg { attrs = "error when coerced in string interpolation"; }

       … while evaluating a path segment
         at «string»:2:24:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                        ^
            3| in throwMsg { attrs = "error when coerced in string interpolation"; }

       error: cannot coerce a set to a string: { attrs = "error when coerced in string interpolation"; }

    Here, instead of an actual thrown error, a type error happens first (trying to coerce an attribute set to a string), but that type error happened while calling throw.

    Many thanks to Qyriad for this.

  • Hash mismatch diagnostics for fixed-output derivations include the URL cl/1536

    Now, when building fixed-output derivations, Lix will guess the URL that was used in the derivation using the url or urls properties in the derivation environment. This is a layering violation but making these diagnostics tractable when there are multiple instances of the AAAA hash is too significant of an improvement to pass it up.

    error: hash mismatch in fixed-output derivation '/nix/store/sjfw324j4533lwnpmr5z4icpb85r63ai-x1.drv':
        likely URL: https://meow.puppy.forge/puppy.tar.gz
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-a1Qvp3FOOkWpL9kFHgugU1ok5UtRPSu+NwCZKbbaEro=

    Many thanks to jade for this.

  • Add log formats multiline and multiline-with-logs cl/1369

    Added two new log formats (multiline and multiline-with-logs) that display current activities below each other for better visibility.

    These formats attempt to use the maximum available lines (defaulting to 25 if unable to determine) and print up to that many lines. The status bar is displayed as the first line, with each subsequent activity on its own line.

    Many thanks to kloenk for this.

  • Eliminate some pretty-printing surprises #11100 cl/1616 cl/1617 cl/1618

    Some inconsistent and surprising behaviours have been eliminated from the pretty-printing used by the REPL and nix eval:

    • Lists and attribute sets that contain only a single item without nested structures are no longer sometimes inappropriately indented in the REPL, depending on internal state of the evaluator.
    • Empty attribute sets and derivations are no longer shown as «repeated», since they are always cheap to print. This matches the existing behaviour of nix-instantiate on empty attribute sets. Empty lists were never printed as «repeated» already.
    • The REPL by default does not print nested attribute sets and lists, and indicates elided items with an ellipsis. Previously, the ellipsis was printed even when the structure was empty, so that such items do not in fact exist. Since this behaviour was confusing, it does not happen any more.

    Before:

    nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
  [
    3
  ]
  [ 3 ]
]

nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
  «repeated»
]

nix-repl> let x = {}; in [ x ]
[
  { ... }
]

    After:

    nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
  [ 3 ]
  [ 3 ]
]

nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
]

nix-repl> let x = {}; in [ x ]
[
  { }
]

    Many thanks to alois31 and Robert Hensing for this.

  • nix registry add now requires a shorthand flakeref on the 'from' side cl/1494

    The 'from' argument must now be a shorthand flakeref like nixpkgs or nixpkgs/nixos-20.03, making it harder to accidentally swap the 'from' and 'to' arguments.

    Registry entries that map from other flake URLs can still be specified in registry.json, the nix.registry option in NixOS, or the --override-flake option in the CLI, but they are not guaranteed to work correctly.

    Many thanks to delan for this.

  • Allow automatic rejection of configuration options from flakes cl/1541

    Setting accept-flake-config to false now respects user choice by automatically rejecting configuration options set by flakes. The old behaviour of asking each time is still available (and default) by setting it to the special value ask.

    Many thanks to alois31 for this.

  • nix repl now allows tab-completing the special repl :colon commands cl/1367

    The REPL (nix repl) supports pressing <TAB> to complete a partial expression, but now also supports completing the special :colon commands as well (:b, :edit, :doc, etc), if the line starts with a colon.

    Many thanks to Qyriad for this.

Fixes

  • Define integer overflow in the Nix language as an error fj#423 cl/1594 cl/1595 cl/1597 cl/1609

    Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but probably manifested as wrapping around on overflow.

    Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around. Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it is safe to turn into an error.

    Some other overflows were fixed:

    • builtins.fromJSON of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
    • nixConfig in flakes will no longer accept negative values for configuration options.

    Integer overflow now looks like the following:

    » nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1

    Many thanks to jade for this.

  • Fix nix-collect-garbage --dry-run fj#432 cl/1566

    nix-collect-garbage --dry-run did not previously give any output - it simply exited without even checking to see what paths would be deleted.

    $ nix-collect-garbage --dry-run
$

    We updated the behaviour of the flag such that instead it prints out how many paths it would delete, but doesn't actually delete them.

    $ nix-collect-garbage --dry-run
finding garbage collector roots...
determining live/dead paths...
...
<nix store paths>
...
2670 store paths deleted, 0.00MiB freed
$

    Many thanks to Quantum Jump for this.

  • nix copy is now several times faster at querying info about /nix/store/... fj#366 cl/1462

    We fixed a locking bug that serialized querying info about /nix/store/... onto just one thread such that it was eating O(paths to copy * latency) time while setting up to copy paths to s3 and other stores. It is now nproc times faster.

    Many thanks to jade for this.

Development

  • Lix now supports building with UndefinedBehaviorSanitizer cl/1483

    You can now build Lix with the configuration option -Db_sanitize=undefined and it will both work and pass tests. AddressSanitizer support is also coming soon.

    For a list of undefined behaviour fixed by sanitizer usage, see the gerrit topic "undefined-behaviour".

    Many thanks to jade for this.