Skip to content

[Refactor] Fall back to the new LMCache paths for moved modules and names - #37

Draft
thegoldenflow wants to merge 4 commits into
LMCache:mainfrom
thegoldenflow:common-imports-fallback
Draft

thegoldenflow wants to merge 4 commits into
LMCache:mainfrom
thegoldenflow:common-imports-fallback

Conversation

@thegoldenflow

@thegoldenflow thegoldenflow commented Nov 19, 2026 •

Copy link
Copy Markdown

Summary

LMCache is moving the modules that its client side shares with the server into lmcache/v1/common/ (LMCache/LMCache#6758) and doesn't keep the old paths. This PR adapts every import of them here, one commit per group:

Commit LMCache PR What moves there Files here
1 LMCache/LMCache#6763 lmcache/v1/periodic_thread.py → v1/common/periodic_thread.py; IrrecoverableException → v1/common/errors.py 4
2 LMCache/LMCache#6763 v1/utils/subclass_discovery.py and lmcache/banner.py → v1/common/ 5
3 LMCache/LMCache#6781 lmcache/logging.py → v1/common/logging.py 141
4 Not open yet; it follows LMCache/LMCache#6781 EngineType, _lmcache_nvtx_annotate and CacheStoreEvent leave lmcache/utils.py for v1/common/custom_types.py, v1/common/decorators.py and integration/vllm/utils.py 26

That is 147 files, +1661/−211. Several files carry more than one fallback.

  • As in [Refactor] Move cache engine keys to in-process utils #17, each file keeps the old import for the pinned snapshot and falls back to the new module only when the old path is gone. So this can merge before or after any of the LMCache PRs, in any order. Every fallback is its own try, except in v1/health_monitor/base.py, where the three modules that move together in [MP][Refactor] Move subclass_discovery, banner, api_key, bearer and periodic_thread to common LMCache#6763 share one.

  • Which exception each fallback catches depends on how the old import fails:

    • ModuleNotFoundError for single-file modules (logging, periodic_thread, subclass_discovery, banner).
    • ImportError for lmcache.v1.exceptions. It is a package, so in a source checkout updated in place the ignored __pycache__/ keeps the directory as a namespace package, and importing the class from it raises ImportError.
    • ImportError for the names leaving lmcache.utils. The module stays, so the old import fails with cannot import name 'EngineType' from 'lmcache.utils'.

    All of them re-raise unless exc.name is the old module, so a missing dependency inside an existing module still raises its original error.

  • Names that stay in lmcache.utils (DiskCacheMetadata, cdiv, compress_slot_mapping, convert_tokens_to_list, start_loop_in_thread_with_exceptions) keep a plain import. Five statements are split in two for that.

  • Commit 3 is the bulk: 142 seven-line blocks for one moved module. The alternative is a one-line switch per file (+142/−142) together with raising the pinned LMCache revision, which can only merge after [MP][Refactor] Move logging to common LMCache#6781. Tell me if you prefer that and I'll swap the commit.

No new compatibility module or tests are added. AGENTS.md and UPSTREAM_REVISION are unchanged.

Companion: LMCache/LMCache#6763 and LMCache/LMCache#6781. The LMCache PR for commit 4 isn't open yet.

Validation

Current head: 43b474df77367cc06ce751574a40818bfee53628.

LMCache trees used below, all as clean git archive exports with the native extensions from a dev build: dev (e0e64d0e), LMCache/LMCache#6763 (e66e6d46), LMCache/LMCache#6781 (439c9de7), and the lmcache/utils.py split on top of LMCache/LMCache#6781. No single tree has every move yet, so each fallback is exercised on the tree that removes its old path.

  • pytest --collect-only tests with this repo's lmcache/, tests/ and pyproject.toml overlaid on each tree (Python 3.12, torch 2.13 CPU, no vLLM):

    LMCache main (0aebefd0) This PR
    dev 1305 tests, 7 collection errors Same test IDs, same 7 errors
    LMCache#6763 4 tests: No module named 'lmcache.v1.utils.subclass_discovery' Same as main on dev
    LMCache#6781 0 tests: No module named 'lmcache.logging' Same as main on dev
    utils split 0 tests: No module named 'lmcache.logging' Same as main on dev

    The 7 files that don't collect need vllm or triton, or import lmcache.v1.gpu_connector.* and raw_block.decode_legacy_key, which LMCache dev no longer has.

  • Importing the 131 changed package modules in the same setups: 121 import with main on dev and with this PR on all four trees. The other 10 need vllm, tensorrt_llm, triton, habana_frameworks, eic, infinistore, maru or nixl, or hit the decode_legacy_key import. With main, 53 import on LMCache#6763 and 1 on the other two trees. The moved names come from the old modules where those still exist and from the new modules elsewhere.

  • The same import sweep in an environment with vLLM 0.32.0: 124 of 131 with main on dev and with this PR on all four trees, including integration/vllm/vllm_v1_adapter.py and vllm_service_factory.py. With main on LMCache#6763, vllm_v1_adapter.py fails with No module named 'lmcache.banner'.

  • Six test files that open no sockets (tests/smoke/test_distribution.py, both health-monitor files, test_gpu_connector_factory.py, test_mock_gpu_connector.py, cache_controller/test_config.py): 56 passed with main on dev and with this PR on all four trees, same per test ID. A full pytest tests comparison is not included.

  • AST check (nothing executed) of every lmcache.* import in the repo: against all four trees, the unguarded top-level imports that don't resolve are the same 5 as main against dev. main adds 12 against LMCache#6763, 140 against LMCache#6781 and 169 against the utils split. For each of the 176 new try blocks, the old import resolves where the old path exists; where it doesn't, the old module is in the exc.name set and the fallback imports resolve.

  • pre-commit run on the 147 changed files: isort, ruff, ruff-format, codespell, end-of-file-fixer and trailing-whitespace pass.

  • Open PRs: 21 of the 23 touch a file changed here. git merge-tree shows new conflicts for three of them: #18 and #19 (2 files each) and #35 (4 files). They are the same files the lmcache.logging commit conflicts with on its own. None of the 23 adds an import of an old path.

…ache.v1.common

LMCache is moving lmcache/v1/periodic_thread.py to
lmcache/v1/common/periodic_thread.py and IrrecoverableException from
lmcache/v1/exceptions/ to lmcache/v1/common/errors.py, without keeping
the old paths. Keep the old imports for the pinned snapshot and fall
back to the new modules only when the old module paths are absent.
Missing dependencies inside an existing module still raise their
original errors. The lmcache.v1.exceptions fallbacks catch ImportError:
in a checkout updated in place, the ignored __pycache__/ can keep
lmcache/v1/exceptions/ as a namespace package, and importing the class
from it then raises ImportError rather than ModuleNotFoundError.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
LMCache is moving lmcache/v1/utils/subclass_discovery.py to
lmcache/v1/common/subclass_discovery.py and lmcache/banner.py to
lmcache/v1/common/banner.py, without keeping the old paths. Keep the
old imports for the pinned snapshot and fall back to the new modules
only when the old modules are absent. In health_monitor/base.py the
import joins the existing try for periodic_thread and
IrrecoverableException, which move in the same LMCache PR.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
LMCache is moving lmcache/logging.py to lmcache/v1/common/logging.py
without keeping the old path. Keep the existing import for the pinned
snapshot and fall back to the new module only when lmcache.logging is
absent. A missing dependency inside an existing module still raises its
original error.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>
…che.utils

LMCache is moving EngineType, _lmcache_nvtx_annotate and CacheStoreEvent
out of lmcache/utils.py, to lmcache/v1/common/custom_types.py,
lmcache/v1/common/decorators.py and lmcache/integration/vllm/utils.py,
without re-exporting them. Keep the old imports for the pinned snapshot
and fall back to the new modules only when lmcache.utils no longer has
the name. lmcache/utils.py itself stays, so the old import then raises
ImportError ("cannot import name") rather than ModuleNotFoundError. The
fallbacks catch ImportError and re-raise unless it was raised for
lmcache.utils, so a missing dependency inside an existing module still
raises its original error. Names that stay in lmcache.utils keep a plain
import.

Signed-off-by: Jason Yao <wsyjh8@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant