diff --git a/README.md b/README.md index b7cdbaa..119e37e 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,10 @@ Heavily inspired by Rust's [Iterator trait](https://doc.rust-lang.org/std/iter/t pip install itrx ``` +> **Using an AI coding agent?** `itrx` ships an installable [agent skill](#agent-skill) — run +> `uv run itrx-skill --install` and your agent gets a built-in reference for writing `Itr` chains +> correctly, without needing this whole README in context. + ## Quick Start & Core Functionality ### Chaining @@ -105,6 +109,27 @@ When working with `Itr`, keep these points in mind: * **No Rewinding:** It's not possible to rewind an `Itr` to an earlier state. You can "preview" the next value using the `peek()` method, and conditionally consume it with `next_if()`. * **Infinite Iterators:** Be cautious with open-ended iterators (e.g., those from `itertools.count()` or custom generators). Eager evaluation methods (like `collect()`, `count()`, `reduce()`) will attempt to consume the entire sequence, potentially leading to infinite loops or out-of-memory errors if applied to an infinite source. +## Agent skill + +`itrx` ships a `SKILL.md` for AI coding agents (e.g. Claude Code) covering the `Itr` API, the +lazy/eager split described above, and the common pitfalls (single-pass iterators, infinite +sources, `groupby` vs `chunk_by`), so an agent doesn't need this whole README in context to write +correct `Itr` chains. + +Install it into a project as a symlink to the version installed in the current environment: + +```sh +uv run itrx-skill --install [PATH] # default PATH: .agents +uv run itrx-skill --remove [PATH] # default PATH: .agents +``` + +(Or, without `uv`, activate the virtualenv `itrx` is installed in and drop the `uv run` prefix — +`itrx-skill` is a normal console-script entry point, so it's only on `PATH` while that environment +is active.) + +This creates (or removes) `PATH/skills/itrx`, symlinked to the skill bundled inside the installed +`itrx` package, so it always matches the version in use. + ## API Reference For a complete list of `Itr` methods and their detailed descriptions, please refer to the [API documentation](./doc/apidoc.md). diff --git a/pyproject.toml b/pyproject.toml index 081b916..2379c62 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -20,6 +20,9 @@ classifiers = [ "Homepage" = "https://github.com/virgesmith/itrx" "Bug Tracker" = "https://github.com/virgesmith/itrx/issues" +[project.scripts] +itrx-skill = "itrx.skill_cli:main" + [build-system] requires = ["hatchling"] build-backend = "hatchling.build" diff --git a/relnotes.md b/relnotes.md index a8b5af8..3496152 100644 --- a/relnotes.md +++ b/relnotes.md @@ -1,3 +1,9 @@ +## Unreleased + +### New features + +- Installable **agent skill**: the package now bundles a `SKILL.md` reference for AI coding agents, plus an `itrx-skill` console script to symlink it into a project (`itrx-skill --install [PATH]` / `--remove [PATH]`, default `PATH=.agents`, creating `PATH/skills/itrx`). The symlink points at the skill inside the installed `itrx`, so it always matches the version in use. See the "Agent skill" section of the README. + ## 0.4.0 ### Breaking changes diff --git a/src/itrx/skill/SKILL.md b/src/itrx/skill/SKILL.md new file mode 100644 index 0000000..d80de39 --- /dev/null +++ b/src/itrx/skill/SKILL.md @@ -0,0 +1,168 @@ +--- +name: itrx +description: > + Use when writing, editing, or debugging code that uses itrx's `Itr` — a chainable, lazy + iterator adaptor for Python inspired by Rust's Iterator trait. Covers the lazy/eager split, + the full method set by category, single-pass and infinite-iterator pitfalls, and how + `collect` materialises results. Triggers: "itrx", "Itr", chainable/fluent iterator, a + Rust-style `.map().filter().collect()` chain in Python, replacing nested `itertools` calls. +--- + +# Developing with itrx + +`itrx` wraps any Python iterable, iterator, or generator in an `Itr[T]` that exposes a fluent, +chainable, mostly-lazy API modelled on Rust's [`Iterator`](https://doc.rust-lang.org/std/iter/trait.Iterator.html) +trait. It is zero-dependency: in most cases each method is thin syntactic sugar over `itertools` +or a builtin, so the runtime cost over hand-written `itertools` is negligible. + +```py +from itrx import Itr + +Itr(range(100)).rev().step_by(4).skip(10).map(lambda x: x * x).filter(lambda x: x % 10 == 9).for_each(print) +``` + +`Itr` implements the iterator protocol itself (`__iter__`/`__next__`), so an `Itr` can be passed +anywhere an iterator is accepted — `for x in itr`, `list(itr)`, `sum(itr)`, or as the `other` +argument to `zip`/`chain` on another `Itr`. + +This skill is a quick reference, not the full API. For a method's exact signature, parameters, +raises and doctest, consult the generated per-method reference: + + + +Read it whenever you need a detail this file doesn't state — it is generated from the source +docstrings and covers every public method. Note it tracks `main`, so it may describe methods +newer than the installed version; `Itr..__doc__` (or `help(Itr.)`) is the +authoritative check for what the *installed* version actually has. In a checkout of the itrx repo +itself the same file is at `doc/apidoc.md`, and the narrative docs are in `README.md`. + +## When reaching for `Itr` is worth it + +The reason to use it is **readability**, not speed — it will not make a pipeline faster than the +equivalent `itertools` code, because it *is* that code underneath. Reach for it when: + +- **A pipeline reads inside-out as builtins.** `filter(lambda .., map(lambda .., islice(islice(reversed(x), None, None, 4), 10, None)))` + has to be read from the innermost call outwards; the `Itr` chain reads left to right in the + order the data actually flows. +- **You want laziness by default without writing generators.** Every intermediate step returns a + new `Itr` and pulls items only on demand, so an infinite source stays workable right up to the + terminal call. +- **The operation exists in Rust's `Iterator` but not as a Python builtin** — `fold`, `inspect`, + `partition`, `position`, `intersperse`, `interleave`, `dedup`, `chunk_by`, `unzip`, `map_while`, + `next_chunk`, `step_by`, `rolling`. + +Conversely, it is **not** worth it for a single `map`/`filter` (a comprehension is clearer), for +code already dominated by numpy/pandas vectorised calls, or where the data is a materialised +`list` you are indexing into rather than streaming. + +## Constructing + +`Itr(it)` accepts any `Iterable[T]` and immediately calls `iter()` on it. Wrapping a **sequence** +(list, tuple, str, range) is safe to do repeatedly — each `Itr` gets its own fresh iterator — but +wrapping a **generator or iterator** hands over ownership: consuming the `Itr` consumes it. + +## Lazy vs eager — the single most important distinction + +**Lazy** (return a new `Itr` and consume nothing until pulled, so they are safe on infinite +sources — with the exception of `product`, which materialises `other` up front like +`itertools.product` does): + +`accumulate`, `batched`, `chain`, `chunk_by`, `copy`, `cycle`, `dedup`, `enumerate`, `filter`, +`flat_map`, `flatten`, `inspect`, `interleave`, `intersperse`, `map`, `map_dict`, `map_while`, +`pairwise`, `partition`, `product`, `repeat`, `rolling`, `skip`, `skip_while`, `step_by`, `take`, +`take_while`, `tee`, `unzip`, `zip`, `zip_longest` + +**Eager** (consume the iterator, return a concrete value; **never** on an infinite source): + +- Collection: `collect`, `last`, `next`, `next_chunk`, `next_if`, `nth`, `position`, `peek` +- Aggregation: `all`, `any`, `consume`, `count`, `find`, `fold`, `for_each`, `max`, `min`, `prod`, + `reduce`, `sum` +- Whole-input reordering: `groupby`, `sorted_by`, `value_counts`, `rev` + +Note that some of these consume only as far as they need to: `next`, `next_chunk`, `nth`, +`next_if`, `peek`, `find`, `position`, `any` and `all` short-circuit, so they *are* safe on an +infinite source. `collect`, `count`, `last`, `consume`, `fold`, `reduce`, `sum`, `prod`, `max`, +`min`, `for_each`, `rev`, `groupby`, `sorted_by` and `value_counts` are not. + +## Outputs + +`collect(container=tuple)` materialises into `tuple` (default), `list`, `set`, or `dict`: + +```py +>>> from itrx import Itr +>>> Itr(("apple", "banana", "carrot")).groupby(len).collect(dict) +{5: ('apple',), 6: ('banana', 'carrot')} +``` + +- `collect(dict)` requires the iterator to yield 2-tuples — pair up with `zip`, `enumerate`, + `groupby`, `chunk_by`, `value_counts` or a `map` that returns pairs first. +- `collect(set)` is the "unique" operation (and drops order). +- `for_each(func)` is the side-effecting terminal; `consume()` exhausts for side effects alone + (pair it with `inspect`); `fold(init, func)` is the general terminal reduction. + +## Method notes and gotchas + +- **Single-pass, no rewinding.** Like any Python iterator, an `Itr` is consumed once. To use a + sequence twice, use `copy()` (splits at the current position into an independent `Itr`), + `tee(n)`, or re-wrap the original source. There is no `reset`. +- **`peek()` does not advance**, and returns the same value on repeated calls; it raises + `StopIteration` when exhausted. `next_if(predicate)` consumes the next item **only** if it + satisfies the predicate, otherwise leaves it in place and returns `None` (also `None` when + exhausted) — the correct tool for conditional lookahead parsing. +- **`tee(n)` and `copy()` share a buffer.** After `tee`, do not consume the original `Itr` — use + the returned ones. If one tee lags far behind another the buffer grows to hold the gap, so + memory can blow up on a large or infinite source. `tee(n)` raises `ValueError` for `n < 1`. +- **`nth(n)` is 0-based** (like Rust's `Iterator::nth`): `nth(0)` is the first item. It raises + `ValueError` for `n < 0` and `StopIteration` if the iterator is shorter than `n + 1`. +- **`next()` and `last()` raise `StopIteration`** on an empty iterator, they do not return `None`. + `find(predicate)` *does* return `None` when nothing matches. +- **`groupby(key)` and `sorted_by(key)` sort the whole input up front**, so they reorder output, + require mutually-orderable keys, and must not touch an infinite source. Use the lazy + **`chunk_by(key)`** to group *consecutive* runs without sorting (the `itertools.groupby` + semantics) — it preserves order and works on infinite iterators. +- **`value_counts()` is eager** and yields `(value, count)` most-common-first with ties by first + appearance (pandas semantics, not sorted by key). Items must be hashable. Collect it into a + `dict` for a lookup table. +- **`dedup()` removes only *adjacent* duplicates**, keeping the first of each run. It compares by + equality (items need not be hashable) and stays lazy — it is not "unique". For global + uniqueness use `collect(set)`, accepting the loss of order. +- **`rev()` materialises the entire remaining sequence** into memory before yielding — unavoidable, + but never call it on an unbounded source. +- **`repeat(n)` tees the iterator `n` times**, so it buffers the whole sequence for large `n`; it + always returns a new `Itr` and leaves the original exhausted. `cycle()` repeats indefinitely. +- **`flatten()` removes exactly one level** of nesting and every item must itself be iterable + (note that `str` is, which flattens strings into characters). `flat_map(f)` is + `map(f).flatten()`. +- **`zip(other)` stops at the shorter input** (no `strict=True` mode); `zip_longest(other, + fillvalue=...)` runs to the longer, padding. `interleave(other)` alternates and then yields the + tail of whichever is longer. +- **`starmap(f)` unpacks each item as `*args`** — use it after `zip`, `enumerate`, `pairwise` or + `product` instead of `map(lambda t: f(*t))`. +- **`map_dict(mapping)`** looks each item up in a `dict` (a `defaultdict` works); a missing key + raises `KeyError`. +- **`batched(n)` yields non-overlapping tuples** (the last possibly short); **`rolling(n)`** yields + overlapping windows of exactly `n` (a generalisation of `pairwise`); **`step_by(n)`** keeps every + n-th item starting with the first. +- **`partition(predicate)`** returns `(matching, non_matching)` as two lazy `Itr`s built on a + `copy()`, so the predicate runs twice per item and the same buffering caveat as `tee` applies. +- **`product(other)` materialises `other`** (as `itertools.product` does), so the *other* iterable + must be finite even though the chain stays lazy in `self`. +- **`inspect(func)` is the lazy debugging hook** — it calls `func` on each item and passes it + through unchanged, so you can drop it mid-chain without altering results. + +## Typing + +`Itr[T]` is generic and every method is fully annotated, so type checkers track the element type +through a chain: `Itr(range(3)).map(str)` is `Itr[str]`, `.enumerate()` is `Itr[tuple[int, str]]`, +`.collect(list)` is `list[str]`. The package ships a `py.typed` marker. When a chain's inferred +type is wrong, the fix is usually to annotate the lambda or the source, not to cast the `Itr`. + +## Working on itrx itself (not just using it) + +If you are editing this library's own source (`src/itrx/itr.py`) rather than using it in a +downstream project, the repo's `AGENTS.md` governs the workflow. In short: no new runtime +dependencies (it is zero-dependency by design), new non-terminal methods must return an `Itr` and +stay lazy, every public method needs a doctest, the full gate suite is +`uv run ruff check && uv run ruff format --check && uv run ty check src && uv run pytest`, and +coverage is enforced at 100%. Regenerate `doc/apidoc.md` with +`uv run python src/scripts/introspect.py` whenever the public API changes. diff --git a/src/itrx/skill_cli.py b/src/itrx/skill_cli.py new file mode 100644 index 0000000..28ca84c --- /dev/null +++ b/src/itrx/skill_cli.py @@ -0,0 +1,100 @@ +import argparse +import os +from pathlib import Path + +_SKILL_NAME = "itrx" +_DEFAULT_PATH = ".agents" + + +def _source_dir() -> Path: + """The bundled skill directory, shipped inside this package.""" + return Path(__file__).parent / "skill" + + +def _target_path(root: str) -> Path: + return Path(root) / "skills" / _SKILL_NAME + + +def _is_ours(target: Path, source: Path) -> bool: + if not target.is_symlink(): + return False + try: + return target.resolve() == source.resolve() + except OSError: + return False + + +def _link_target(source: Path, target: Path) -> str: + """A relative link target, or an absolute one if there is no relative path (different windows drives).""" + try: + return os.path.relpath(source.resolve(), target.parent.resolve()) + except ValueError: + return str(source.resolve()) + + +def _install(root: str) -> int: + source = _source_dir() + target = _target_path(root) + target.parent.mkdir(parents=True, exist_ok=True) + + if target.is_symlink(): + if _is_ours(target, source): + print(f"already installed and up to date: {target}") + return 0 + print(f"refusing to overwrite existing symlink not managed by {_SKILL_NAME}-skill: {target}") + return 1 + if target.exists(): + print(f"refusing to overwrite existing file or directory: {target}") + return 1 + + target.symlink_to(_link_target(source, target), target_is_directory=True) + print(f"installed: {target} -> {source}") + return 0 + + +def _remove(root: str) -> int: + source = _source_dir() + target = _target_path(root) + + if not target.exists() and not target.is_symlink(): + print(f"not installed: {target}") + return 0 + if not _is_ours(target, source): + print(f"refusing to remove {target}: not a symlink managed by {_SKILL_NAME}-skill") + return 1 + + target.unlink() + print(f"removed: {target}") + return 0 + + +def main(argv: list[str] | None = None) -> int: + """Entry point for the `itrx-skill` console script.""" + parser = argparse.ArgumentParser( + prog=f"{_SKILL_NAME}-skill", + description=f"Install or remove the '{_SKILL_NAME}' agent skill as a symlink into a project.", + ) + group = parser.add_mutually_exclusive_group(required=True) + group.add_argument( + "--install", + nargs="?", + const=_DEFAULT_PATH, + metavar="PATH", + help=f"Symlink the skill into PATH/skills/{_SKILL_NAME} (default PATH: {_DEFAULT_PATH}).", + ) + group.add_argument( + "--remove", + nargs="?", + const=_DEFAULT_PATH, + metavar="PATH", + help=f"Remove the symlink from PATH/skills/{_SKILL_NAME} (default PATH: {_DEFAULT_PATH}).", + ) + args = parser.parse_args(argv) + + if args.install is not None: + return _install(args.install) + return _remove(args.remove) + + +if __name__ == "__main__": + raise SystemExit(main()) # pragma: no cover - console script entry point diff --git a/src/test/test_skill_cli.py b/src/test/test_skill_cli.py new file mode 100644 index 0000000..dd37ff7 --- /dev/null +++ b/src/test/test_skill_cli.py @@ -0,0 +1,115 @@ +from pathlib import Path + +import pytest + +from itrx import skill_cli + + +def test_install_creates_symlink(tmp_path: Path) -> None: + assert skill_cli.main(["--install", str(tmp_path)]) == 0 + + target = tmp_path / "skills" / "itrx" + assert target.is_symlink() + assert target.resolve() == skill_cli._source_dir().resolve() + assert (target / "SKILL.md").is_file() + + +def test_install_default_path(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + monkeypatch.chdir(tmp_path) + assert skill_cli.main(["--install"]) == 0 + + target = tmp_path / ".agents" / "skills" / "itrx" + assert target.is_symlink() + + +def test_install_across_windows_drives(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + # os.path.relpath raises when the installed package and the target project are on different drives + def raise_valueerror(*_args: object, **_kwargs: object) -> str: + raise ValueError("path is on mount 'D:', start on mount 'C:'") + + monkeypatch.setattr(skill_cli.os.path, "relpath", raise_valueerror) + assert skill_cli.main(["--install", str(tmp_path)]) == 0 + + target = tmp_path / "skills" / "itrx" + assert target.is_symlink() + assert target.resolve() == skill_cli._source_dir().resolve() + + +def test_is_ours_ignores_unresolvable_symlink(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + target = tmp_path / "link" + target.symlink_to(tmp_path, target_is_directory=True) + + def raise_oserror(*_args: object, **_kwargs: object) -> Path: + raise OSError("cannot resolve") + + monkeypatch.setattr(Path, "resolve", raise_oserror) + assert not skill_cli._is_ours(target, skill_cli._source_dir()) + + +def test_install_idempotent(tmp_path: Path) -> None: + assert skill_cli.main(["--install", str(tmp_path)]) == 0 + assert skill_cli.main(["--install", str(tmp_path)]) == 0 + + target = tmp_path / "skills" / "itrx" + assert target.is_symlink() + + +def test_install_refuses_existing_directory(tmp_path: Path) -> None: + target = tmp_path / "skills" / "itrx" + target.mkdir(parents=True) + (target / "keepme.txt").write_text("do not delete") + + assert skill_cli.main(["--install", str(tmp_path)]) == 1 + assert not target.is_symlink() + assert (target / "keepme.txt").read_text() == "do not delete" + + +def test_install_refuses_foreign_symlink(tmp_path: Path) -> None: + foreign = tmp_path / "elsewhere" + foreign.mkdir() + target = tmp_path / "skills" / "itrx" + target.parent.mkdir(parents=True) + target.symlink_to(foreign, target_is_directory=True) + + assert skill_cli.main(["--install", str(tmp_path)]) == 1 + assert target.resolve() == foreign.resolve() + + +def test_remove_deletes_owned_symlink(tmp_path: Path) -> None: + skill_cli.main(["--install", str(tmp_path)]) + target = tmp_path / "skills" / "itrx" + assert target.is_symlink() + + assert skill_cli.main(["--remove", str(tmp_path)]) == 0 + assert not target.exists() + assert not target.is_symlink() + + +def test_remove_missing_is_noop(tmp_path: Path) -> None: + assert skill_cli.main(["--remove", str(tmp_path)]) == 0 + + +def test_remove_refuses_foreign_directory(tmp_path: Path) -> None: + target = tmp_path / "skills" / "itrx" + target.mkdir(parents=True) + (target / "keepme.txt").write_text("do not delete") + + assert skill_cli.main(["--remove", str(tmp_path)]) == 1 + assert (target / "keepme.txt").read_text() == "do not delete" + + +def test_remove_default_path(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: + monkeypatch.chdir(tmp_path) + skill_cli.main(["--install"]) + target = tmp_path / ".agents" / "skills" / "itrx" + assert target.is_symlink() + + assert skill_cli.main(["--remove"]) == 0 + assert not target.exists() + + +def test_mutually_exclusive_args_required() -> None: + with pytest.raises(SystemExit): + skill_cli.main([]) + with pytest.raises(SystemExit): + skill_cli.main(["--install", ".", "--remove", "."])