Repository navigation
[Refactor] Fall back to the new LMCache paths for moved modules and names - #37
Draft
thegoldenflow wants to merge 4 commits into
Draft
thegoldenflow wants to merge 4 commits into
thegoldenflow wants to merge 4 commits into
Conversation
…ache.v1.common LMCache is moving lmcache/v1/periodic_thread.py to lmcache/v1/common/periodic_thread.py and IrrecoverableException from lmcache/v1/exceptions/ to lmcache/v1/common/errors.py, without keeping the old paths. Keep the old imports for the pinned snapshot and fall back to the new modules only when the old module paths are absent. Missing dependencies inside an existing module still raise their original errors. The lmcache.v1.exceptions fallbacks catch ImportError: in a checkout updated in place, the ignored __pycache__/ can keep lmcache/v1/exceptions/ as a namespace package, and importing the class from it then raises ImportError rather than ModuleNotFoundError. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
LMCache is moving lmcache/v1/utils/subclass_discovery.py to lmcache/v1/common/subclass_discovery.py and lmcache/banner.py to lmcache/v1/common/banner.py, without keeping the old paths. Keep the old imports for the pinned snapshot and fall back to the new modules only when the old modules are absent. In health_monitor/base.py the import joins the existing try for periodic_thread and IrrecoverableException, which move in the same LMCache PR. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
LMCache is moving lmcache/logging.py to lmcache/v1/common/logging.py without keeping the old path. Keep the existing import for the pinned snapshot and fall back to the new module only when lmcache.logging is absent. A missing dependency inside an existing module still raises its original error. Signed-off-by: Jason Yao <wsyjh8@gmail.com>
…che.utils
LMCache is moving EngineType, _lmcache_nvtx_annotate and CacheStoreEvent
out of lmcache/utils.py, to lmcache/v1/common/custom_types.py,
lmcache/v1/common/decorators.py and lmcache/integration/vllm/utils.py,
without re-exporting them. Keep the old imports for the pinned snapshot
and fall back to the new modules only when lmcache.utils no longer has
the name. lmcache/utils.py itself stays, so the old import then raises
ImportError ("cannot import name") rather than ModuleNotFoundError. The
fallbacks catch ImportError and re-raise unless it was raised for
lmcache.utils, so a missing dependency inside an existing module still
raises its original error. Names that stay in lmcache.utils keep a plain
import.
Signed-off-by: Jason Yao <wsyjh8@gmail.com>
9 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LMCache is moving the modules that its client side shares with the server into
lmcache/v1/common/(LMCache/LMCache#6758) and doesn't keep the old paths. This PR adapts every import of them here, one commit per group:lmcache/v1/periodic_thread.py→v1/common/periodic_thread.py;IrrecoverableException→v1/common/errors.pyv1/utils/subclass_discovery.pyandlmcache/banner.py→v1/common/lmcache/logging.py→v1/common/logging.pyEngineType,_lmcache_nvtx_annotateandCacheStoreEventleavelmcache/utils.pyforv1/common/custom_types.py,v1/common/decorators.pyandintegration/vllm/utils.pyThat is 147 files, +1661/−211. Several files carry more than one fallback.
As in [Refactor] Move cache engine keys to in-process utils #17, each file keeps the old import for the pinned snapshot and falls back to the new module only when the old path is gone. So this can merge before or after any of the LMCache PRs, in any order. Every fallback is its own
try, except inv1/health_monitor/base.py, where the three modules that move together in [MP][Refactor] Move subclass_discovery, banner, api_key, bearer and periodic_thread to common LMCache#6763 share one.Which exception each fallback catches depends on how the old import fails:
ModuleNotFoundErrorfor single-file modules (logging,periodic_thread,subclass_discovery,banner).ImportErrorforlmcache.v1.exceptions. It is a package, so in a source checkout updated in place the ignored__pycache__/keeps the directory as a namespace package, and importing the class from it raisesImportError.ImportErrorfor the names leavinglmcache.utils. The module stays, so the old import fails withcannot import name 'EngineType' from 'lmcache.utils'.All of them re-raise unless
exc.nameis the old module, so a missing dependency inside an existing module still raises its original error.Names that stay in
lmcache.utils(DiskCacheMetadata,cdiv,compress_slot_mapping,convert_tokens_to_list,start_loop_in_thread_with_exceptions) keep a plain import. Five statements are split in two for that.Commit 3 is the bulk: 142 seven-line blocks for one moved module. The alternative is a one-line switch per file (+142/−142) together with raising the pinned LMCache revision, which can only merge after [MP][Refactor] Move logging to common LMCache#6781. Tell me if you prefer that and I'll swap the commit.
No new compatibility module or tests are added.
AGENTS.mdandUPSTREAM_REVISIONare unchanged.Companion: LMCache/LMCache#6763 and LMCache/LMCache#6781. The LMCache PR for commit 4 isn't open yet.
Validation
Current head:
43b474df77367cc06ce751574a40818bfee53628.LMCache trees used below, all as clean
git archiveexports with the native extensions from adevbuild:dev(e0e64d0e), LMCache/LMCache#6763 (e66e6d46), LMCache/LMCache#6781 (439c9de7), and thelmcache/utils.pysplit on top of LMCache/LMCache#6781. No single tree has every move yet, so each fallback is exercised on the tree that removes its old path.pytest --collect-only testswith this repo'slmcache/,tests/andpyproject.tomloverlaid on each tree (Python 3.12, torch 2.13 CPU, no vLLM):main(0aebefd0)devNo module named 'lmcache.v1.utils.subclass_discovery'mainondevNo module named 'lmcache.logging'mainondevNo module named 'lmcache.logging'mainondevThe 7 files that don't collect need
vllmortriton, or importlmcache.v1.gpu_connector.*andraw_block.decode_legacy_key, which LMCachedevno longer has.Importing the 131 changed package modules in the same setups: 121 import with
mainondevand with this PR on all four trees. The other 10 needvllm,tensorrt_llm,triton,habana_frameworks,eic,infinistore,maruornixl, or hit thedecode_legacy_keyimport. Withmain, 53 import on LMCache#6763 and 1 on the other two trees. The moved names come from the old modules where those still exist and from the new modules elsewhere.The same import sweep in an environment with vLLM 0.32.0: 124 of 131 with
mainondevand with this PR on all four trees, includingintegration/vllm/vllm_v1_adapter.pyandvllm_service_factory.py. Withmainon LMCache#6763,vllm_v1_adapter.pyfails withNo module named 'lmcache.banner'.Six test files that open no sockets (
tests/smoke/test_distribution.py, both health-monitor files,test_gpu_connector_factory.py,test_mock_gpu_connector.py,cache_controller/test_config.py): 56 passed withmainondevand with this PR on all four trees, same per test ID. A fullpytest testscomparison is not included.AST check (nothing executed) of every
lmcache.*import in the repo: against all four trees, the unguarded top-level imports that don't resolve are the same 5 asmainagainstdev.mainadds 12 against LMCache#6763, 140 against LMCache#6781 and 169 against the utils split. For each of the 176 newtryblocks, the old import resolves where the old path exists; where it doesn't, the old module is in theexc.nameset and the fallback imports resolve.pre-commit runon the 147 changed files:isort,ruff,ruff-format,codespell,end-of-file-fixerandtrailing-whitespacepass.Open PRs: 21 of the 23 touch a file changed here.
git merge-treeshows new conflicts for three of them:#18and#19(2 files each) and#35(4 files). They are the same files thelmcache.loggingcommit conflicts with on its own. None of the 23 adds an import of an old path.