Skip to content

[MP][Refactor] Move subclass_discovery, banner, api_key, bearer and periodic_thread to common - #6763

Open
thegoldenflow wants to merge 7 commits into
LMCache:devfrom
thegoldenflow:mp-common-subclass-discovery-banner
Open

thegoldenflow wants to merge 7 commits into
LMCache:devfrom
thegoldenflow:mp-common-subclass-discovery-banner

Conversation

@thegoldenflow

@thegoldenflow thegoldenflow commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Part of #6758.

Moves six pieces that the client and the server share to lmcache/v1/common/ as is. Only import paths change, and the old paths are removed.

What From To
discover_subclasses v1/utils/subclass_discovery.py v1/common/subclass_discovery.py
print_banner_once lmcache/banner.py v1/common/banner.py
api_key.py lmcache/security/api_key.py v1/common/security/api_key.py
bearer.py lmcache/security/bearer.py v1/common/security/bearer.py
periodic_thread.py v1/periodic_thread.py v1/common/periodic_thread.py
IrrecoverableException v1/exceptions/__init__.py v1/common/errors.py; the package is removed

The client reaches each of them: import lmcache builds the device registry with discover_subclasses, the vLLM MP connector prints the banner, the ZMQ and gRPC request clients load the API key and send it as a Bearer header, and HeartbeatThread subclasses PeriodicThread. The three groups (subclass_discovery and banner; api_key and bearer; periodic_thread) are independent and in separate commits. lmcache/security/http.py stays, since it imports Starlette and only the HTTP apps use it.

Outside this repo, LMCache/LMCache_inprocess imports the old paths; the matching change is LMCache/LMCache_inprocess#37. vllm-project/vllm#68540 (draft) and rannnayy/LMCache@get-query-tensor-v2, which snapkv_colab.ipynb installs, import lmcache.banner. Open PRs #6774, #6596, #5473, #5867 and #6421 add old-path imports; whichever lands second updates them.

Special notes for your reviewers:

AI-assisted change; the submitter reviews every line before this leaves draft. The explainer below covers the placement reasons, the code walkthrough and the open PRs.

Check Result
git diff -M dev 62 files, +112/−108; four renames at 100% similarity, periodic_thread.py at 99% (one import line)
Backend registry in a fresh interpreter, import lmcache first and import torch first Same as dev both ways
pytest --collect-only tests 9081 tests, same IDs as dev
pytest on the 204 test files that reference a moved module or its users 3424 passed, 278 skipped; same per test ID as dev
SKIP=rust-fmt,rust-clippy pre-commit run --all-files Pass
Clean git archive export: import each of the six old modules ModuleNotFoundError
git grep for the old paths, including tests/, docs/, examples/ and notebooks None left

For #6582: lmcache.v1.utils.subclass_discovery, lmcache.banner, lmcache.security.__init__, lmcache.security.api_key, lmcache.security.bearer and lmcache.v1.periodic_thread can leave the client wheel's temporary list, and lmcache.v1.exceptions* its package list.

AI-generated PR explainer (required when AI-generated code is included):

pr-explainer-31d3b8a2.html

If applicable:

  • this PR contains user facing changes - docs added
  • this PR contains unit tests
Before marking ready for review
  • I reviewed every changed line since the latest update, including generated or
    agent-assisted changes.
  • I ran the relevant local checks and recorded the commands and results above.
  • I confirmed the description matches the final diff and documents any missing coverage
    or compatibility concerns.
  • I confirmed every commit has a DCO Signed-off-by trailer.
  • I stated whether AI assistance was used in Special notes for your reviewers.
  • If AI-generated code is included, this PR is a draft and its title, description, and
    bilingual HTML explainer all match the current head.
  • The HTML explainer is detailed enough for a reviewer without prior context, and its
    diagrams, optional motion, code walkthrough, and test rationale match the implementation.

Move lmcache/v1/utils/subclass_discovery.py to
lmcache/v1/common/subclass_discovery.py and update the imports. The
platform device registry runs it on import lmcache, and the CLI and the
server module loaders use it too. It only depends on the standard
library and lmcache.logging. The code is unchanged apart from import
paths, and the old path is not re-exported.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/banner.py to lmcache/v1/common/banner.py and update the
imports. The vLLM MP connector and the CLI both print the banner, and it
only depends on the standard library and lmcache._version. The code is
unchanged apart from import paths, and the old path is not re-exported.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
No conflicts. LMCache#6503 changed two lines of
lmcache/integration/vllm/lmcache_mp_connector.py, away from the import
block where this branch switches the banner import.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/security/api_key.py and lmcache/security/bearer.py to
lmcache/v1/common/security/ and update the imports. The client's gRPC
transport, the server's request transports and HTTP apps, and the CLI
all use them, and they only depend on the standard library and
typing_extensions. lmcache/security/http.py stays where it is because
it imports Starlette. The code is unchanged apart from import paths,
and the old paths are not re-exported.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/v1/periodic_thread.py to lmcache/v1/common/ and
IrrecoverableException from lmcache/v1/exceptions/ to
lmcache/v1/common/errors.py, then update the imports. The client
heartbeat subclasses PeriodicThread, and the MP server modules, the
periodic-threads HTTP API and usage telemetry use the base class, the
registry and the factory. lmcache/v1/exceptions/ only held
IrrecoverableException, so it is removed. The code is unchanged apart
from import paths, and the old paths are not re-exported.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
@thegoldenflow thegoldenflow changed the title [MP][Refactor] Move subclass_discovery and banner to common [MP][Refactor] Move subclass_discovery, banner, api_key, bearer and periodic_thread to common Nov 19, 2026
…urity modules

- snapkv_colab.ipynb: import print_banner_once from lmcache.v1.common.banner,
  like the other token_dropping notebooks.
- lmcache/security/__init__.py: the package now only holds the Starlette
  middleware; point to lmcache.v1.common.security for the primitives.
- mp_coordinator/http_auth.py: the shared primitives now live in
  lmcache.v1.common.security.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
No conflicts. LMCache#6782 and LMCache#6710 change docs, the operator and the k3
multiprocess CI scripts, none of which this branch touches.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant