Skip to content

Progress reporting on long-running scans and index builds #957

Closed
1 of 1 issue completed
Closed

Description

@thekevinbot

Problem

A cold scan over a large corpus gives the user nothing to look at. Core has no progress reporting; the only bar anywhere is the Hugging Face model download bar in the embeddings plugin (embedding/progress.py), TTY-gated.

This becomes acute with #956: a first vector-index build runs embed() once per ingested row via a worker round trip (41k rows on the reference corpus — minutes of wall time, silent). Trigger-fired scalar calls cannot batch, so the cost is irreducible; the answer is visibility, not policy — show the user what they are paying rather than deciding for them whether it is acceptable. The embeddings disk cache makes re-runs cheap; this is about first builds and full rebuilds.

Proposed

Progress on the scan path in core, and on the embedding path in the plugin. TTY-gated, consistent with the existing download bar. No new dependencies in core.

Activity

  1. added a commit that references this issue on Aug 20, 2026
    eed9548
  2. added 2 commits that reference this issue on Aug 21, 2026
    9dec8bd
    349e391
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions