Skip to content

Commit 66b70bd

Browse files
redsun82Copilot
redsun82
and
Copilot
committed
Add a Bazel skill for this repository
Agents (and humans) editing BUILD.bazel/*.bzl here had no cross-cutting reference: the only Bazel docs are per-area READMEs. This collects the conventions that are not discoverable from any single file — which construct to reach for, the shared `misc/bazel` macros, the `{CODEQL_PLATFORM}` packaging mechanism, checksum pinning, and what the `semmle_code` stub does and does not cover. A counterpart exists for the internal module that also depends on this one. This is deliberately not a copy of it: it is scoped to the standalone build, so it is useful to someone who only ever sees this repository, and it says nothing that requires internal access. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 7a367c9 commit 66b70bd

1 file changed

Lines changed: 172 additions & 0 deletions

File tree

‎.github/skills/bazel/SKILL.md‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
---
2+
name: bazel
3+
description: Conventions for editing Bazel files in the github/codeql repository — the shared `misc/bazel` helpers, `codeql_platform_select` and the `{CODEQL_PLATFORM}` packaging placeholder, adding `MODULE.bazel` dependencies, and the `semmle_code` stub that keeps the standalone build working. Use when editing any `BUILD.bazel`, `*.bzl`, `MODULE.bazel` or `.bazelrc` here, and before validating such an edit.
4+
---
5+
6+
# Bazel in the codeql repository
7+
8+
A bzlmod module named `ql`, repo name `@codeql` ([`MODULE.bazel`](../../../MODULE.bazel)). It builds standalone, and is
9+
also consumed by an internal module that depends on it; standalone builds replace that module with a stub.
10+
11+
## Traps
12+
13+
Things that will waste your time or produce a wrong edit here. Read this section even if you skip the rest.
14+
15+
* **`//...` does not work.** `bazel build //...`, and even `bazel query //...`, fail at the repo root — the patched
16+
modules under `misc/bazel/registry/modules/*/*/overlay` are real packages referencing repos that are not visible from
17+
the main repo, and [`.bazelrc`](../../../.bazelrc) notes that transitions break `...` builds separately. **Validate
18+
the specific target or package you changed**, not a recursive pattern. The error names an unrelated directory and is
19+
very easy to misdiagnose.
20+
* **There is no `MODULE.bazel.lock`, deliberately.** [`.bazelrc`](../../../.bazelrc) sets `--lockfile_mode=off` because
21+
the workspace-relative module override makes a lockfile unstable. Do not add one, and do not "fix" its absence.
22+
* **`linux_arm64` vs `linux-arm64`.** The keyword argument and config setting use an underscore; the platform *string*
23+
substituted into paths and zip names uses a hyphen. They are not interchangeable.
24+
* **Do not stub your way out of a missing internal dependency.** See [Building standalone](#building-standalone) for the
25+
one narrow case where extending the stub is correct.
26+
27+
## Rules of thumb
28+
29+
* **Copy a neighbouring target rather than inventing a shape.** Note that packaging is not uniform — some packages use
30+
the `codeql_*` wrappers, others still use `pkg_files` directly — so copy the closest *working* neighbour and prefer
31+
the wrapper for new code.
32+
* **Pin anything fetched over the network** with `sha256` or `integrity`. Bazel only *warns* on an unpinned download, so
33+
nothing fails loudly, but the build stops being reproducible and a retagged upstream release silently changes what you
34+
build. `lfs_archive` is the exception — content is pinned by git object.
35+
* **Format with `bazel run //misc/bazel/buildifier`.** It rewrites in place, so don't hand-tune formatting. Also wired as
36+
a `pre-commit` hook ([`.pre-commit-config.yaml`](../../../.pre-commit-config.yaml)).
37+
38+
## Where new code goes
39+
40+
Bazel's own macro / rule / repository-rule distinction applies as usual. What is repo-specific:
41+
42+
| Adding… | Goes in |
43+
| --- | --- |
44+
| a new packaging shape | extend [`misc/bazel/pkg.bzl`](../../../misc/bazel/pkg.bzl) — don't fork `pkg_files` |
45+
| a new OS or arch split | [`misc/bazel/os.bzl`](../../../misc/bazel/os.bzl) — don't hand-roll a `select()` over `@platforms//` |
46+
| a fetch of something external | a repository rule ([`lfs.bzl`](../../../misc/bazel/lfs.bzl), [`ripunzip.bzl`](../../../misc/ripunzip/ripunzip.bzl)) — not a `genrule` |
47+
| a wrapper used by one language | next to that language ([`swift/rules.bzl`](../../../swift/rules.bzl)) |
48+
| a wrapper used across languages | `misc/bazel/` |
49+
50+
Prefer inline rules in `BUILD.bazel`. A `.bzl` file earns its `load()` only when the shape repeats across packages or a
51+
value must be computed: [`rust.bzl`](../../../misc/bazel/rust.bzl) is worth it because every Rust binary in the repo
52+
must get the same universal-binary wrapper and symbols test, and forgetting either is a release bug. The `_gen_binaries`
53+
list in [`go/BUILD.bazel`](../../../go/BUILD.bazel) is not — it is shared by two targets in one file, so a local
54+
variable does the job.
55+
56+
Macros here are typically a thin public wrapper around a private rule (`codeql_csharp_binary`, `swift_cc_binary`). Keep
57+
the rule narrow and the ergonomics in the macro. Generated targets are named after the macro's `name` (`<name>-all`,
58+
`single_arch/<name>`, `internal/<name>`) and kept private; when an error names a target you cannot find in any source
59+
file, a macro minted it — grep the suffix under `misc/bazel/`.
60+
61+
## Shared helpers
62+
63+
Frequently-used pieces, so you load the existing one instead of rewriting it. Read the file for its actual exports.
64+
65+
| `load()` path | Covers |
66+
| --- | --- |
67+
| `//misc/bazel:pkg.bzl` | CodeQL packs and packaging |
68+
| `//misc/bazel:os.bzl` | platform and architecture selection |
69+
| `//misc/bazel:lfs.bzl` | on-demand git-LFS repositories |
70+
| `//misc/bazel:rust.bzl` | Rust binary wrapper |
71+
| `//misc/bazel:csharp.bzl` | C# binary/library/test wrappers |
72+
| `//misc/bazel:utils.bzl` | `select_os`; prefer `os.bzl`'s `os_select` in new code |
73+
74+
[`defs.bzl`](../../../defs.bzl) at the root exports `codeql_platform` for *dependent* modules — nothing in this repo
75+
loads it, and it is not the way to get the platform string here (use `os.bzl`).
76+
77+
## Platform selection
78+
79+
`codeql_platform_select` discriminates the four platforms CodeQL knows about: `linux64`, `linux_arm64`, `osx64`,
80+
`win64`. `otherwise` supplies the value for whichever of those four you leave unset — it is **not** a
81+
`//conditions:default`. **There is deliberately no fallback from `linux_arm64` to `linux64`**; if you only care about
82+
the OS, use `os_select`, which gives Linux the same value on both architectures and has a `posix` shorthand for the
83+
shared Linux/macOS value.
84+
85+
In a macro (no `ctx`) it returns a `select()`:
86+
87+
```python
88+
load("//misc/bazel:os.bzl", "codeql_platform_select")
89+
load("//misc/bazel:pkg.bzl", "codeql_pkg_files")
90+
91+
codeql_pkg_files(
92+
name = "extractor-arch",
93+
exes = codeql_platform_select(
94+
otherwise = ["//unified/extractor"],
95+
win64 = ["//unified/extractor-unsupported-os:extractor"],
96+
),
97+
prefix = "tools/{CODEQL_PLATFORM}",
98+
)
99+
```
100+
101+
If implementation code needs to *branch* on the value rather than pass it through, pass `ctx` and add
102+
`OS_DETECTION_ATTRS` to the rule's attributes — the value is then resolved eagerly instead of being an opaque `select()`:
103+
104+
```python
105+
load("//misc/bazel:os.bzl", "OS_DETECTION_ATTRS", "os_select")
106+
107+
def _impl(ctx):
108+
ext = os_select(ctx, windows = ".exe", posix = "")
109+
...
110+
111+
my_rule = rule(
112+
implementation = _impl,
113+
attrs = {"src": attr.label()} | OS_DETECTION_ATTRS,
114+
)
115+
```
116+
117+
## Packs
118+
119+
A pack is the unit that becomes an extractor pack. See [`unified/BUILD.bazel`](../../../unified/BUILD.bazel) for a
120+
minimal complete example and [`pkg.bzl`](../../../misc/bazel/pkg.bzl) for the arguments. The non-obvious parts:
121+
122+
* **`{CODEQL_PLATFORM}` in a destination path is the routing mechanism**, not just a substitution. A path containing it
123+
is *arch-specific* and lands in the per-architecture zip; every other path is *common*. So `prefix =
124+
"tools/{CODEQL_PLATFORM}"` both places the file and marks it arch-specific. `arch_overrides` forces named
125+
destinations into the arch-specific part without a placeholder.
126+
* **`codeql_pkg_files` splits `srcs` (plain) from `exes` (mode 755)** and **rejects `attributes =`** with an explicit
127+
error — use `exes` rather than hand-rolling `pkg_attributes(mode = "755")`.
128+
* **`pkg_dirs` and `pkg_symlinks` are unsupported** and fail at analysis time.
129+
* `codeql_pack` also generates an installer and an `install` alias, hence `bazel run //unified:install`. Pass
130+
`installer_alias = None` if one package defines several packs.
131+
* `codeql_pack_group` exists for bundling packs into distribution zips, but nothing in this repo instantiates it.
132+
133+
## Dependencies and `MODULE.bazel`
134+
135+
In order of preference:
136+
137+
1. **A [Bazel Central Registry](https://registry.bazel.build/) module** — a `bazel_dep` in
138+
[`MODULE.bazel`](../../../MODULE.bazel).
139+
2. **A patched upstream module** — add it under
140+
[`misc/bazel/registry`](../../../misc/bazel/registry), which `.bazelrc` puts ahead of the BCR. Put patches in
141+
`modules/<repo>/<version>/patches`, rename the version with a `-codeql.N` suffix, and run
142+
[`fix.py`](../../../misc/bazel/registry/fix.py) to realign the metadata.
143+
3. **A raw archive** — `http_archive` via `use_repo_rule`, or a repository rule. Copy an adjacent declaration and keep
144+
its checksum field populated.
145+
146+
Vendored Rust crates under [`misc/bazel/3rdparty`](../../../misc/bazel/3rdparty) are generated — regenerate with
147+
`update_cargo_deps.sh` rather than editing, and keep the `use_repo` lists in sync (`bazel mod tidy` handles several).
148+
149+
## Building standalone
150+
151+
[`MODULE.bazel`](../../../MODULE.bazel) declares `semmle_code` with a `local_path_override` pointing at `..`, which
152+
resolves when this repo is checked out inside the internal module. [`.bazelrc`](../../../.bazelrc) — read when Bazel is
153+
invoked in *this* workspace — overrides that with a stub, and this line is the whole reason a standalone build resolves:
154+
155+
```
156+
common --override_module=semmle_code=%workspace%/misc/bazel/semmle_code_stub
157+
```
158+
159+
Do not change either the override path or the `local_path_override`; they work as a pair.
160+
161+
[`misc/bazel/semmle_code_stub`](../../../misc/bazel/semmle_code_stub) is an otherwise empty module supplying no-op
162+
versions of the internal helpers that shared `.bzl` files load *unconditionally*. That is its only job, and it is small
163+
enough to read.
164+
165+
**Extend the stub only when a `.bzl` file every standalone target loads gains a new internal `load()`** — that breaks
166+
package loading outright, for everyone. Prefer not needing one. **Never add a stub so that an internal-only target
167+
appears to build**: some targets depend on internal libraries (`grep -rl @semmle_code --include=*.bazel` finds them) and
168+
are correctly unbuildable here. Unlike a `load()`, such a dependency only fails when that target is actually requested.
169+
170+
[`.bazelrc.internal`](../../../.bazelrc.internal) is **not** read here; it carries settings for the internal build. A
171+
setting needed by both has to be written in both files, with paths differing because this repo sits at a different depth
172+
there.

0 commit comments

Comments
 (0)