Repository navigation
[MP][Refactor] Move subclass_discovery, banner, api_key, bearer and periodic_thread to common - #6763
Open
thegoldenflow wants to merge 7 commits into
Open
thegoldenflow wants to merge 7 commits into
thegoldenflow wants to merge 7 commits into
Conversation
Move lmcache/v1/utils/subclass_discovery.py to lmcache/v1/common/subclass_discovery.py and update the imports. The platform device registry runs it on import lmcache, and the CLI and the server module loaders use it too. It only depends on the standard library and lmcache.logging. The code is unchanged apart from import paths, and the old path is not re-exported. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/banner.py to lmcache/v1/common/banner.py and update the imports. The vLLM MP connector and the CLI both print the banner, and it only depends on the standard library and lmcache._version. The code is unchanged apart from import paths, and the old path is not re-exported. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
This was referenced Nov 19, 2026
No conflicts. LMCache#6503 changed two lines of lmcache/integration/vllm/lmcache_mp_connector.py, away from the import block where this branch switches the banner import. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/security/api_key.py and lmcache/security/bearer.py to lmcache/v1/common/security/ and update the imports. The client's gRPC transport, the server's request transports and HTTP apps, and the CLI all use them, and they only depend on the standard library and typing_extensions. lmcache/security/http.py stays where it is because it imports Starlette. The code is unchanged apart from import paths, and the old paths are not re-exported. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Move lmcache/v1/periodic_thread.py to lmcache/v1/common/ and IrrecoverableException from lmcache/v1/exceptions/ to lmcache/v1/common/errors.py, then update the imports. The client heartbeat subclasses PeriodicThread, and the MP server modules, the periodic-threads HTTP API and usage telemetry use the base class, the registry and the factory. lmcache/v1/exceptions/ only held IrrecoverableException, so it is removed. The code is unchanged apart from import paths, and the old paths are not re-exported. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
…urity modules - snapkv_colab.ipynb: import print_banner_once from lmcache.v1.common.banner, like the other token_dropping notebooks. - lmcache/security/__init__.py: the package now only holds the Starlette middleware; point to lmcache.v1.common.security for the primitives. - mp_coordinator/http_auth.py: the shared primitives now live in lmcache.v1.common.security. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
No conflicts. LMCache#6782 and LMCache#6710 change docs, the operator and the k3 multiprocess CI scripts, none of which this branch touches. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
thegoldenflow
marked this pull request as ready for review
November 20, 2026 01:53
thegoldenflow
requested review from
ApostaC,
DongDongJu,
KuntaiDu,
VincyZhang,
YaoJiayi,
deng451e,
hickeyma,
hlin99,
maobaolong,
ruizhang0101,
sammshen and
zhengfeihe
as code owners
November 20, 2026 01:53
1 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Part of #6758.
Moves six pieces that the client and the server share to
lmcache/v1/common/as is. Only import paths change, and the old paths are removed.discover_subclassesv1/utils/subclass_discovery.pyv1/common/subclass_discovery.pyprint_banner_oncelmcache/banner.pyv1/common/banner.pyapi_key.pylmcache/security/api_key.pyv1/common/security/api_key.pybearer.pylmcache/security/bearer.pyv1/common/security/bearer.pyperiodic_thread.pyv1/periodic_thread.pyv1/common/periodic_thread.pyIrrecoverableExceptionv1/exceptions/__init__.pyv1/common/errors.py; the package is removedThe client reaches each of them:
import lmcachebuilds the device registry withdiscover_subclasses, the vLLM MP connector prints the banner, the ZMQ and gRPC request clients load the API key and send it as a Bearer header, andHeartbeatThreadsubclassesPeriodicThread. The three groups (subclass_discoveryandbanner;api_keyandbearer;periodic_thread) are independent and in separate commits.lmcache/security/http.pystays, since it imports Starlette and only the HTTP apps use it.Outside this repo,
LMCache/LMCache_inprocessimports the old paths; the matching change is LMCache/LMCache_inprocess#37.vllm-project/vllm#68540(draft) andrannnayy/LMCache@get-query-tensor-v2, whichsnapkv_colab.ipynbinstalls, importlmcache.banner. Open PRs#6774,#6596,#5473,#5867and#6421add old-path imports; whichever lands second updates them.Special notes for your reviewers:
AI-assisted change; the submitter reviews every line before this leaves draft. The explainer below covers the placement reasons, the code walkthrough and the open PRs.
git diff -M devperiodic_thread.pyat 99% (one import line)import lmcachefirst andimport torchfirstdevboth wayspytest --collect-only testsdevpyteston the 204 test files that reference a moved module or its usersdevSKIP=rust-fmt,rust-clippy pre-commit run --all-filesgit archiveexport: import each of the six old modulesModuleNotFoundErrorgit grepfor the old paths, includingtests/,docs/,examples/and notebooksFor #6582:
lmcache.v1.utils.subclass_discovery,lmcache.banner,lmcache.security.__init__,lmcache.security.api_key,lmcache.security.bearerandlmcache.v1.periodic_threadcan leave the client wheel's temporary list, andlmcache.v1.exceptions*its package list.AI-generated PR explainer (required when AI-generated code is included):
pr-explainer-31d3b8a2.html
If applicable:
Before marking ready for review
agent-assisted changes.
or compatibility concerns.
Signed-off-bytrailer.bilingual HTML explainer all match the current head.
diagrams, optional motion, code walkthrough, and test rationale match the implementation.