Lix 2.91 "Dragon's Breath" (2024-08-12)

Lix 2.91.1 (2024-10-18)

Fixes

  • <nix/fetchurl.nix> now uses TLS verification #11585

    Previously <nix/fetchurl.nix> did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from netrc and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does not check the hash. Therefore we have now enabled TLS verification. This means that downloads by <nix/fetchurl.nix> will now fail if you're fetching from a HTTPS server that does not have a valid certificate.

    <nix/fetchurl.nix> is also known as the builtin derivation builder builtin:fetchurl. It's not to be confused with the evaluation-time function builtins.fetchurl, which was not affected by this issue.

    Many thanks to Eelco Dolstra for this.

Lix 2.91.0 (2024-08-12)

Breaking Changes

  • Block io_uring in the Linux sandbox cl/1611

    The io_uring API has the unfortunate property that it is not possible to selectively decide which operations should be allowed. This, together with the fact that new operations are routinely added, makes it a hazard to the proper function of the sandbox.

    Therefore, any access to io_uring has been made unavailable inside the sandbox. As such, attempts to execute any system calls forming part of this API will fail with the error ENOSYS, as if io_uring support had not been configured into the kernel.

    Many thanks to alois31 for this.

  • The build-hook setting is now deprecated

    Build hooks communicate with the daemon using a custom, internal, undocumented protocol that is entirely unversioned and cannot be changed. Since we intend to change it anyway we must unfortunately deprecate the current build hook infrastructure. We do not expect this to impact most users—we have not found any uses of build-hook in the wild—but if this does affect you, we'd like to hear from you!

  • Lix no longer speaks the Nix remote-build worker protocol to clients or servers older than CppNix 2.3 fj#325 cl/1207 cl/1208 cl/1206 cl/1205 cl/1204 cl/1203 cl/1479

    CppNix 2.3 was released in 2019, and is the new oldest supported version. We will increase our support baseline in the future up to a final version of CppNix 2.18 (which may happen soon given that it is the only still-packaged and thus still-tested >2.3 version), but this step already removes a significant amount of dead, untested, code paths.

    Lix speaks the same version of the protocol as CppNix 2.18 and that fact will never change in the future; the Lix plans to replace the protocol for evolution will entail a complete incompatible replacement that will be supported in parallel with the old protocol. Lix will thus retain remote build compatibility with CppNix as long as CppNix maintains protocol compatibility with 2.18, and as long as Lix retains legacy protocol support (which will likely be a long time given that we plan to convert it to a frozen-in-time shim).

    Many thanks to jade for this.

Features

  • Pipe operator |> (experimental) fj#438 cl/1654

    Implementation of the pipe operator (|>) in the language as described in RFC 148. The feature is still marked experimental, enable --extra-experimental-features pipe-operator to use it.

    Many thanks to piegames and eldritch horrors for this.

Improvements

  • Trace which part of a foo.bar.baz expression errors cl/1505 cl/1506

    Previously, if an attribute path selection expression like linux_4_9.meta.description it wouldn't show you which one of those parts in the attribute path, or even that that line of code is what caused evaluation of the failing expression. The previous error looks like this:

    pkgs.linuxKernel.kernels.linux_4_9.meta.description

error:
       … while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … while calling the 'throw' builtin
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Now, the error will look like this:

    pkgs.linuxKernel.kernels.linux_4_9.meta.description

error:
       … while evaluating the attribute 'linuxKernel.kernels.linux_4_9.meta.description'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … while evaluating 'pkgs.linuxKernel.kernels.linux_4_9' to select 'meta' on it
         at «string»:1:1:
            1| pkgs.linuxKernel.kernels.linux_4_9.meta.description
             | ^

       … caused by explicit throw
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Not only does the line of code that referenced the failing attribute show up in the trace, it also tells you that it was specifically the linux_4_9 part that failed.

    This includes if the failing part is a top-level binding:

    let
  inherit (pkgs.linuxKernel.kernels) linux_4_9;
in linux_4_9.meta.description
error:
       … while evaluating 'linux_4_9' to select 'meta.description' on it
         at «string»:3:4:
            2|   inherit (pkgs.linuxKernel.kernels) linux_4_9;
            3| in linux_4_9.meta.description
             |    ^

       … while evaluating the attribute 'linux_4_9'
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:5:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |     ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       … caused by explicit throw
         at /nix/store/dk2rpyb6ndvfbf19bkb2plcz5y3k8i5v-source/pkgs/top-level/linux-kernels.nix:278:17:
          277|   } // lib.optionalAttrs config.allowAliases {
          278|     linux_4_9 = throw "linux 4.9 was removed because it will reach its end of life within 22.11";
             |                 ^
          279|     linux_4_14 = throw "linux 4.14 was removed because it will reach its end of life within 23.11";

       error: linux 4.9 was removed because it will reach its end of life within 22.11

    Many thanks to Qyriad for this.

  • Confusing 'invalid path' errors are now 'path does not exist' cl/1161 cl/1160 cl/1159

    Previously, if a path did not exist in a Nix store, it was referred to as the internal name "path is invalid". This is, however, very confusing, and there were numerous such errors that were exactly the same, making it hard to debug. These errors are now more specific and refer to the path not existing in the store.

    Many thanks to julia for this.

  • Add a build-dir setting to set the backing directory for builds gh#10303 gh#10312 gh#10883 cl/1514

    build-dir can now be set in the Nix configuration to choose the backing directory for the build sandbox. This can be useful on systems with /tmp on tmpfs, or simply to relocate large builds to another disk.

    Also, XDG_RUNTIME_DIR is no longer considered when selecting the default temporary directory, as it's not intended to be used for large amounts of data.

    Many thanks to Robert Hensing and Tom Bereknyei for this.

  • Better usage of colour control environment variables cl/1699 cl/1702

    Lix now heeds NO_COLOR/NOCOLOR for more output types, such as that used in nix search, nix flake metadata and similar.

    It also now supports CLICOLOR_FORCE/FORCE_COLOR to force colours regardless of whether there is a terminal on the other side.

    It now follows rules compatible with those described on https://bixense.com/clicolors/ with CLICOLOR defaulted to enabled.

    That is to say, the following procedure is followed in order:

    • NO_COLOR or NOCOLOR set

      Always disable colour

    • CLICOLOR_FORCE or FORCE_COLOR set

      Enable colour

    • The output is a tty; TERM != "dumb"

      Enable colour

    • Otherwise

      Disable colour

    Many thanks to jade for this.

  • Distinguish between explicit throws and errors that happened while evaluating a throw cl/1511

    Previously, errors caused by an expression like throw "invalid argument" were treated like an error that happened simply while some builtin function was being called:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"

error:
       … while calling the 'throw' builtin
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg "linuz"

       error: linuz isn't the right package

    But the error didn't just happen "while" calling the throw builtin — it's a throw error! Now it looks like this:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg "linuz"

error:
       … caused by explicit throw
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg "linuz"

       error: linuz isn't the right package

    This also means that incorrect usage of throw or errors evaluating its arguments are easily distinguishable from explicit throws:

    let
  throwMsg = p: throw "${p} isn't the right package";
in throwMsg { attrs = "error when coerced in string interpolation"; }

error:
       … while calling the 'throw' builtin
         at «string»:2:17:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                 ^
            3| in throwMsg { attrs = "error when coerced in string interpolation"; }

       … while evaluating a path segment
         at «string»:2:24:
            1| let
            2|   throwMsg = p: throw "${p} isn't the right package";
             |                        ^
            3| in throwMsg { attrs = "error when coerced in string interpolation"; }

       error: cannot coerce a set to a string: { attrs = "error when coerced in string interpolation"; }

    Here, instead of an actual thrown error, a type error happens first (trying to coerce an attribute set to a string), but that type error happened while calling throw.

    Many thanks to Qyriad for this.

  • nix flake metadata prints modified date cl/1700

    Ever wonder "gee, when did I update nixpkgs"? Wonder no more, because nix flake metadata now simply tells you the times every locked flake input was updated:

    <...>
Description:   The purely functional package manager
Path:          /nix/store/c91yi8sxakc2ry7y4ac1smzwka4l5p78-source
Revision:      c52cff582043838bbe29768e7da232483d52b61d-dirty
Last modified: 2024-07-31 22:15:54
Inputs:
├───flake-compat: github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33
│   Last modified: 2023-10-04 06:37:54
├───nix2container: github:nlewo/nix2container/3853e5caf9ad24103b13aa6e0e8bcebb47649fe4
│   Last modified: 2024-07-10 13:15:56
├───nixpkgs: github:NixOS/nixpkgs/e21630230c77140bc6478a21cd71e8bb73706fce
│   Last modified: 2024-07-25 11:26:27
├───nixpkgs-regression: github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2
│   Last modified: 2022-01-24 11:20:45
└───pre-commit-hooks: github:cachix/git-hooks.nix/f451c19376071a90d8c58ab1a953c6e9840527fd
    Last modified: 2024-07-15 04:21:09

    Many thanks to jade for this.

  • Hash mismatch diagnostics for fixed-output derivations include the URL cl/1536

    Now, when building fixed-output derivations, Lix will guess the URL that was used in the derivation using the url or urls properties in the derivation environment. This is a layering violation but making these diagnostics tractable when there are multiple instances of the AAAA hash is too significant of an improvement to pass it up.

    error: hash mismatch in fixed-output derivation '/nix/store/sjfw324j4533lwnpmr5z4icpb85r63ai-x1.drv':
        likely URL: https://meow.puppy.forge/puppy.tar.gz
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-a1Qvp3FOOkWpL9kFHgugU1ok5UtRPSu+NwCZKbbaEro=

    Many thanks to jade for this.

  • Add log formats multiline and multiline-with-logs cl/1369

    Added two new log formats (multiline and multiline-with-logs) that display current activities below each other for better visibility.

    These formats attempt to use the maximum available lines (defaulting to 25 if unable to determine) and print up to that many lines. The status bar is displayed as the first line, with each subsequent activity on its own line.

    Many thanks to kloenk for this.

  • Lix will now show the package descriptions in when running nix flake show. cl/1540

    When running nix flake show, Lix will now show the package descriptions, if they exist.

    Before:

    $ nix flake show
path:/home/isabel/dev/lix-show?lastModified=1721736108&narHash=sha256-Zo8HP1ur7Q2b39hKUEG8EAh/opgq8xJ2jvwQ/htwO4Q%3D
└───packages
    └───x86_64-linux
        ├───aNoDescription: package 'simple'
        ├───bOneLineDescription: package 'simple'
        ├───cMultiLineDescription: package 'simple'
        ├───dLongDescription: package 'simple'
        └───eEmptyDescription: package 'simple'

    After:

    $ nix flake show
path:/home/isabel/dev/lix-show?lastModified=1721736108&narHash=sha256-Zo8HP1ur7Q2b39hKUEG8EAh/opgq8xJ2jvwQ/htwO4Q%3D
└───packages
    └───x86_64-linux
        ├───aNoDescription: package 'simple'
        ├───bOneLineDescription: package 'simple' - 'one line'
        ├───cMultiLineDescription: package 'simple' - 'line one'
        ├───dLongDescription: package 'simple' - 'abcdefghijklmnopqrstuvwxyz'
        └───eEmptyDescription: package 'simple'

    Many thanks to kjeremy and isabelroses for this.

  • Eliminate some pretty-printing surprises #11100 cl/1616 cl/1617 cl/1618

    Some inconsistent and surprising behaviours have been eliminated from the pretty-printing used by the REPL and nix eval:

    • Lists and attribute sets that contain only a single item without nested structures are no longer sometimes inappropriately indented in the REPL, depending on internal state of the evaluator.
    • Empty attribute sets and derivations are no longer shown as «repeated», since they are always cheap to print. This matches the existing behaviour of nix-instantiate on empty attribute sets. Empty lists were never printed as «repeated» already.
    • The REPL by default does not print nested attribute sets and lists, and indicates elided items with an ellipsis. Previously, the ellipsis was printed even when the structure was empty, so that such items do not in fact exist. Since this behaviour was confusing, it does not happen any more.

    Before:

    nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
  [
    3
  ]
  [ 3 ]
]

nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
  «repeated»
]

nix-repl> let x = {}; in [ x ]
[
  { ... }
]

    After:

    nix-repl> :p let x = 1 + 2; in [ [ x ] [ x ] ]
[
  [ 3 ]
  [ 3 ]
]

nix-repl> let inherit (import <nixpkgs> { }) hello; in [ hello hello ]
[
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
  «derivation /nix/store/fqs92lzychkm6p37j7fnj4d65nq9fzla-hello-2.12.1.drv»
]

nix-repl> let x = {}; in [ x ]
[
  { }
]

    Many thanks to alois31 and Robert Hensing for this.

  • nix registry add now requires a shorthand flakeref on the 'from' side cl/1494

    The 'from' argument must now be a shorthand flakeref like nixpkgs or nixpkgs/nixos-20.03, making it harder to accidentally swap the 'from' and 'to' arguments.

    Registry entries that map from other flake URLs can still be specified in registry.json, the nix.registry option in NixOS, or the --override-flake option in the CLI, but they are not guaranteed to work correctly.

    Many thanks to delan for this.

  • Allow automatic rejection of configuration options from flakes cl/1541

    Setting accept-flake-config to false now respects user choice by automatically rejecting configuration options set by flakes. The old behaviour of asking each time is still available (and default) by setting it to the special value ask.

    Many thanks to alois31 for this.

  • nix repl now allows tab-completing the special repl :colon commands cl/1367

    The REPL (nix repl) supports pressing <TAB> to complete a partial expression, but now also supports completing the special :colon commands as well (:b, :edit, :doc, etc), if the line starts with a colon.

    Many thanks to Qyriad for this.

  • :editing a file in Nix store no longer reloads the repl fj#341 cl/1620

    Calling :edit from the repl now only reloads if the file being edited was outside of Nix store. That means that all the local variables are now preserved across :edits of store paths. This is always safe because the store is read-only.

    Many thanks to goldstein for this.

  • :log in repl now works on derivation paths fj#51 cl/1716

    :log can now accept store derivation paths in addition to derivation expressions.

    Many thanks to goldstein for this.

Fixes

  • Define integer overflow in the Nix language as an error fj#423 cl/1594 cl/1595 cl/1597 cl/1609

    Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but probably manifested as wrapping around on overflow.

    Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around. Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it is safe to turn into an error.

    Some other overflows were fixed:

    • builtins.fromJSON of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
    • nixConfig in flakes will no longer accept negative values for configuration options.

    Integer overflow now looks like the following:

    » nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1

    Many thanks to jade for this.

  • Fix nix-collect-garbage --dry-run fj#432 cl/1566

    nix-collect-garbage --dry-run did not previously give any output - it simply exited without even checking to see what paths would be deleted.

    $ nix-collect-garbage --dry-run
$

    We updated the behaviour of the flag such that instead it prints out how many paths it would delete, but doesn't actually delete them.

    $ nix-collect-garbage --dry-run
finding garbage collector roots...
determining live/dead paths...
...
<nix store paths>
...
2670 store paths deleted, 0.00MiB freed
$

    Many thanks to Quantum Jump for this.

  • Fix unexpectedly-successful GC failures on macOS fj#446 cl/1723

    Has the following happened to you on macOS? This failure has been successfully eliminated, thanks to our successful deployment of advanced successful-failure detection technology (it's just if (failed && errno == 0). Patent pendingnot really):

    $ nix-store --gc --print-dead
finding garbage collector roots...
error: Listing pid 87261 file descriptors: Undefined error: 0

    Many thanks to jade for this.

  • nix copy is now several times faster at querying info about /nix/store/... fj#366 cl/1462

    We fixed a locking bug that serialized querying info about /nix/store/... onto just one thread such that it was eating O(paths to copy * latency) time while setting up to copy paths to s3 and other stores. It is now nproc times faster.

    Many thanks to jade for this.

Development

  • clang-tidy support fj#147 cl/1697

    clang-tidy can be used to lint Lix with a limited set of lints using ninja -C build clang-tidy and ninja -C build clang-tidy-fix. In practice, this fixes the built-in meson rule that was used the same as above being broken ever since precompiled headers were introduced.

    Many thanks to jade for this.

  • Lix now supports building with UndefinedBehaviorSanitizer cl/1483 cl/1481 cl/1669

    You can now build Lix with the configuration option -Db_sanitize=undefined,address and it will both work and pass tests with both AddressSanitizer and UndefinedBehaviorSanitizer enabled. To use ASan specifically, you have to set -Dgc=disabled, which an error message will tell you to do if necessary anyhow.

    Furthermore, tests passing with Clang ASan+UBSan is checked on every change in CI.

    For a list of undefined behaviour found by tooling usage, see the gerrit topic "undefined-behaviour".

    Many thanks to jade for this.