-
Notifications
You must be signed in to change notification settings - Fork 21
chore(nix): add development shell and CI #2128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
07662f4
e826766
ef3b098
efca305
e90b34e
dfc5642
c2a77fe
1a2c4e4
6084791
9a0cb69
c97e301
788cf17
ede5caa
96ff489
cfb9257
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,86 @@ | ||||||||||||||||||||
| name: Test Nix | ||||||||||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add documentation (maybe here and / or inline in this file) with instructions on what to do if there is a failure? I suspect it'll only come up on MSRV bumps.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not too sure what the expectation of the doc would be: some basic usage examples to onboard the Nix-unaware / a LLM agent maybe? I did not want to be too invasive as a first PR but seems like you folks are quite on board with this so I feel like the doc you pointed to would be a good place. Will add. |
||||||||||||||||||||
|
|
||||||||||||||||||||
| on: # yamllint disable-line rule:truthy | ||||||||||||||||||||
| push: | ||||||||||||||||||||
| branches: | ||||||||||||||||||||
| - main | ||||||||||||||||||||
| - mq-working-branch-* | ||||||||||||||||||||
|
Comment on lines
+4
to
+7
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of triggering it in every PR I think it would be better to have a nightly schedule. That way we can save some capacity for the actual pipelines.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But wouldn't you want to check tings on PRs that change Nix files at least?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Definitely. As is it's not going to trigger in every PR anyway, only those that change nix files or that have an impact on nix (the toolchain files). The schedule can be nice though as a sanity check for when those don't change often. |
||||||||||||||||||||
| # Also run on PRs that touch the devshell or what it reads. Paths mirror the | ||||||||||||||||||||
| # Nix CODEOWNERS entries, plus the toolchain files (read by the flake) and | ||||||||||||||||||||
| # this workflow itself. | ||||||||||||||||||||
| pull_request: | ||||||||||||||||||||
| paths: | ||||||||||||||||||||
| - "*.nix" | ||||||||||||||||||||
| - "flake.*" | ||||||||||||||||||||
| - "rust-toolchain.toml" | ||||||||||||||||||||
| - "nightly-toolchain.toml" | ||||||||||||||||||||
| - ".github/workflows/nix.yml" | ||||||||||||||||||||
|
|
||||||||||||||||||||
| # Default permissions for all jobs | ||||||||||||||||||||
| permissions: {} | ||||||||||||||||||||
|
|
||||||||||||||||||||
| concurrency: | ||||||||||||||||||||
| group: ci-${{ github.ref == 'refs/heads/main' && github.run_id || github.ref }}-nix | ||||||||||||||||||||
| cancel-in-progress: true | ||||||||||||||||||||
|
|
||||||||||||||||||||
| env: | ||||||||||||||||||||
| CARGO_TERM_COLOR: always | ||||||||||||||||||||
|
|
||||||||||||||||||||
| jobs: | ||||||||||||||||||||
| test: | ||||||||||||||||||||
| strategy: | ||||||||||||||||||||
| fail-fast: false | ||||||||||||||||||||
| matrix: | ||||||||||||||||||||
| platform: | ||||||||||||||||||||
| - os: darwin | ||||||||||||||||||||
| cpu: arm64 | ||||||||||||||||||||
| base: macos-15 # always arm64-darwin | ||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is macos strictly needed? I ask because macos runners are scarce and ocuppying one for this might hurt us a bit in terms of merge times
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, it guarantees that it consistently works across OSes for any dev that uses Nix on a macOS machine Should be mostly mitigated by conditional running mentioned above.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could also use a concurrency group (example from our test workflow)and cancel in progress if you don't need commit level granularity. Not necessarily a hill I want to die on, but I think using a concurrency group is a good compromise between getting signal that something broke and not potentially backing up the MergeQueue waiting for a macOS runner for a workflow that doesn't indicate an actual code issue. |
||||||||||||||||||||
| - os: linux | ||||||||||||||||||||
| cpu: x86_64 | ||||||||||||||||||||
| base: ubuntu-24.04 # always x86_64-linux-gnu | ||||||||||||||||||||
| - os: linux | ||||||||||||||||||||
| cpu: aarch64 | ||||||||||||||||||||
| base: ubuntu-24.04-arm # always aarch64-linux-gnu | ||||||||||||||||||||
|
lloeki marked this conversation as resolved.
|
||||||||||||||||||||
|
|
||||||||||||||||||||
| name: Test Nix (${{ matrix.platform.cpu }}-${{ matrix.platform.os }}) | ||||||||||||||||||||
| runs-on: ${{ matrix.platform.base }} | ||||||||||||||||||||
|
|
||||||||||||||||||||
| permissions: | ||||||||||||||||||||
| contents: read | ||||||||||||||||||||
|
|
||||||||||||||||||||
| steps: | ||||||||||||||||||||
| - name: Check CPU arch | ||||||||||||||||||||
| run: | | ||||||||||||||||||||
| test "$(uname -m)" = "${{ matrix.platform.cpu }}" | ||||||||||||||||||||
| - name: Free Disk Space (Linux only) | ||||||||||||||||||||
| if: runner.os == 'Linux' | ||||||||||||||||||||
| uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # 1.3.1 | ||||||||||||||||||||
| with: | ||||||||||||||||||||
| tool-cache: true | ||||||||||||||||||||
| android: true | ||||||||||||||||||||
| dotnet: true | ||||||||||||||||||||
| haskell: true | ||||||||||||||||||||
| large-packages: false | ||||||||||||||||||||
| docker-images: false | ||||||||||||||||||||
| swap-storage: false | ||||||||||||||||||||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||||||||||||||||||||
| with: | ||||||||||||||||||||
| persist-credentials: false | ||||||||||||||||||||
| - uses: cachix/install-nix-action@ab739621df7a23f52766f9ccc97f38da6b7af14f # v31.10.5 | ||||||||||||||||||||
| - name: Print toolchain versions | ||||||||||||||||||||
| run: | | ||||||||||||||||||||
| nix develop --command rustc --version | ||||||||||||||||||||
| nix develop --command cargo --version | ||||||||||||||||||||
| nix develop --command cbindgen --version | ||||||||||||||||||||
| - name: Check nightly formatter toolchain | ||||||||||||||||||||
| run: nix develop .#nightly --command cargo fmt --version | ||||||||||||||||||||
| - name: Build workspace | ||||||||||||||||||||
| run: nix develop --command cargo build --workspace --exclude builder | ||||||||||||||||||||
|
|
||||||||||||||||||||
| complete: | ||||||||||||||||||||
| name: Nix (complete) | ||||||||||||||||||||
| runs-on: ubuntu-24.04 | ||||||||||||||||||||
| needs: | ||||||||||||||||||||
| - test | ||||||||||||||||||||
| steps: | ||||||||||||||||||||
| - run: echo "DONE!" | ||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # flake-compat shim for usage without flakes | ||
| (import | ||
| ( | ||
| let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in | ||
| fetchTarball { | ||
| url = lock.nodes.flake-compat.locked.url or "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; | ||
| sha256 = lock.nodes.flake-compat.locked.narHash; | ||
| } | ||
| ) | ||
| { src = ./.; } | ||
| ).defaultNix |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| { | ||
|
lloeki marked this conversation as resolved.
|
||
| inputs = { | ||
| nixpkgs.url = "github:nixos/nixpkgs/release-25.11"; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unstable is nice to have the very latest version, but I think you won't hit the binary cache, which might mean a lot of full local rebuilds from source. The latest stable makes sense though 👍
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unstable is actually cached, but it moves fast. I vaguely remember situations such as user A updates the flake.lock, 1 month passes, new user B checks the project out from scratch and inherits the same flake.lock, but stuff isn't in the unstable cache anymore. If you are the one updating you shouldn't have issues ofc, because when you check out the package, they are the most recent available and thus in cache. I think. I am not sure. Both latest stable and unstable are probably reasonable. Another reason could be if you run NixOS, you usually target a specific release, and you probably don't want to duplicate many versions of core packages (the stable and the unstable one). |
||
|
|
||
| # cross-platform convenience | ||
| flake-utils.url = "github:numtide/flake-utils"; | ||
|
|
||
| # backwards compatibility with nix-build and nix-shell | ||
| flake-compat.url = "https://flakehub.com/f/edolstra/flake-compat/1.tar.gz"; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: Is this necessary?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ha, I forgot to add the shims, will add. |
||
|
|
||
| # pinned, exact upstream Rust toolchains | ||
| rust-overlay = { | ||
| url = "github:oxalica/rust-overlay"; | ||
| inputs.nixpkgs.follows = "nixpkgs"; | ||
| }; | ||
| }; | ||
|
|
||
| outputs = { self, nixpkgs, flake-utils, flake-compat, rust-overlay }: | ||
| # resolve for all platforms in turn | ||
| flake-utils.lib.eachDefaultSystem (system: | ||
| let | ||
| # packages for this system platform, with the rust-overlay applied | ||
| pkgs = import nixpkgs { | ||
| inherit system; | ||
| overlays = [ (import rust-overlay) ]; | ||
| }; | ||
|
|
||
| # A devshell for a given Rust toolchain (read from a toolchain file via | ||
| # rust-overlay), with the rest of the build dependencies. | ||
| mkDevShell = rust: pkgs.stdenv.mkDerivation { | ||
| name = "libdatadog-devshell"; | ||
|
|
||
| # The stdenv cc-wrapper injects -D_FORTIFY_SOURCE, which glibc rejects | ||
| # when compiling without optimization. Some build scripts (e.g. | ||
| # spawn_worker's trampoline.c) compile C at -O0 with -Werror, so the | ||
| # resulting fortify #warning becomes a hard error. Disable fortify | ||
| # hardening in the shell so those builds succeed. | ||
| hardeningDisable = [ "fortify" "fortify3" ]; | ||
|
|
||
| nativeBuildInputs = [ | ||
| rust # rustc + cargo + rustfmt + clippy, pinned via toolchain file | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One slightly annoying thing here is that we need to
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Coincidentally I just added a nightly toolchain as Maybe we can even work out something that would have a closure over the nightly toolchain as an internal dependency for a nightly-based Indeed probably for a later PR.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To clarify usage as of today:
|
||
| pkgs.rust-cbindgen | ||
|
lloeki marked this conversation as resolved.
|
||
| pkgs.cmake | ||
| pkgs.autoconf | ||
| pkgs.automake | ||
| pkgs.libtool | ||
| ]; | ||
| }; | ||
| in { | ||
| # Default: the pinned stable toolchain (single source of truth is | ||
| # ./rust-toolchain.toml), matching CI and rustup. | ||
| devShells.default = mkDevShell (pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml); | ||
|
|
||
| # Nightly toolchain (./nightly-toolchain.toml) for the jobs that | ||
| # genuinely need a nightly compiler. Use with `nix develop .#nightly`. | ||
| devShells.nightly = mkDevShell (pkgs.rust-bin.fromRustupToolchainFile ./nightly-toolchain.toml); | ||
| } | ||
| ); | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # flake-compat shim for usage without flakes | ||
| (import | ||
| ( | ||
| let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in | ||
| fetchTarball { | ||
| url = lock.nodes.flake-compat.locked.url or "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; | ||
| sha256 = lock.nodes.flake-compat.locked.narHash; | ||
| } | ||
| ) | ||
| { src = ./.; } | ||
| ).shellNix |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TIL there's a Nix guild 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a small but growing group. Join us at
#nix!