Skip to content

perf: lazy-load inference backends on import - #41

Open
VijayKumarMidde wants to merge 1 commit into
opendatalab:mainfrom
VijayKumarMidde:lazy-optional-transformers
Open

perf: lazy-load inference backends on import#41
VijayKumarMidde wants to merge 1 commit into
opendatalab:mainfrom
VijayKumarMidde:lazy-optional-transformers

Conversation

@VijayKumarMidde

Copy link
Copy Markdown

Lazy-load inference backends so importing mineru_html no longer eagerly pulls in transformers / torch.

  • Use PEP 562 getattr for public exports in mineru_html, implementations, and inference
  • Move annotation-only AutoTokenizer imports behind TYPE_CHECKING in base_backend / transformers_api

Install deps and the public API are unchanged. Constructing a Transformers / vLLM / OpenAI backend still loads the usual stack on first use.

@VijayKumarMidde

Copy link
Copy Markdown
Author

@darkrush any chance this could get a look when you have a moment? It lazy-loads inference backends so import mineru_html no longer eagerly pulls in heavy deps. Public API and install deps are unchanged. Backends still load on first use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant