Skip to content

perf: import package re-exports on first use - #724

Merged
ChuckBuilds merged 5 commits into
mainfrom
perf/lazy-package-imports
Oct 3, 2026
Merged

ChuckBuilds merged 5 commits into
mainfrom
perf/lazy-package-imports

Conversation

@ChuckBuilds

@ChuckBuilds ChuckBuilds commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Summary

Resolve the re-exports in src/common/__init__.py and src/plugin_system/__init__.py lazily (PEP 562 __getattr__), and import numpy in sync_manager only where send_frame uses it. The web process stops loading numpy, freetype helpers and PluginManager just to import path_safety, store_manager or schema_manager. Importing web_interface.app on a Pi 4 drops from ~67 MB to ~54 MB RSS. The display process ends up with the same modules loaded.

Type of change

  • Refactor (no functional change)

Test plan

  • Ran the test suite (pytest): full suite passes, apart from 8 test_live_status_fields.py tests that need psutil and fail the same way on main. New test/test_lazy_package_imports.py pins that each name resolves to the same object as before, __all__ is unchanged, and the web import path doesn't load numpy.
  • Ran on a real Raspberry Pi with hardware (not yet soaked)

Documentation

  • I updated the relevant doc in docs/ if developer behavior changed (src/common/README.md, CHANGELOG.md)

Plugin compatibility

  • No plugin breakage expected (from src.common import X and from src.plugin_system import X keep working)

Notes for reviewer

__all__ and __dir__ are preserved and TYPE_CHECKING imports keep mypy seeing the real types.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Performance
    • Package imports are lighter, with utilities and plugin classes loaded only when first accessed. Image-processing dependencies are also deferred until frame transmission, reducing work during startup.
  • Documentation
    • Added guidance on lazy imports and how to maintain package exports.
  • Tests
    • Added coverage for import behavior, exported names, and handling of unknown names.

ChuckBuilds and others added 2 commits October 1, 2026 22:49
The web interface's API blueprint imports src.common.sync_manager only for
STATUS_FILE and SYNC_PORT, which loaded numpy into the web process. numpy is
used in one place, the leader's send_frame, so import it there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uc5DAbSwGGUTm3MrCtpC2m
src/common/__init__.py and src/plugin_system/__init__.py imported every
re-exported helper eagerly, so any submodule import -- the web interface's
path_safety, store_manager, schema_manager -- also loaded ScrollHelper
(numpy), LogoHelper, APIHelper, the adaptive layout helpers (freetype) and
PluginManager. Resolve those names through a PEP 562 module __getattr__
instead, with __dir__, an unchanged __all__ and TYPE_CHECKING imports so
mypy still sees the real types. Each name resolves to the same object as
before, and is cached in the module namespace on first access.

With the sync_manager change, importing web_interface.app on a Pi 4 drops
from ~67 MB to ~54 MB RSS and no longer loads numpy. The display process
ends up with the same modules loaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uc5DAbSwGGUTm3MrCtpC2m
@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 9 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 71128199-5153-45c7-a3b4-08e7fdc3dd39
📥 Commits

Reviewing files that changed from the base of the PR and between d3ef625 and e9792a0.

📒 Files selected for processing (6)
  • CHANGELOG.md
  • src/common/README.md
  • src/common/__init__.py
  • src/common/sync_manager.py
  • src/plugin_system/__init__.py
  • test/test_lazy_package_imports.py

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 4505be49-de04-463d-93e0-4bb58d2cb7c8

📥 Commits

Reviewing files that changed from the base of the PR and between 5aa7a63 and d3ef625.

📒 Files selected for processing (6)
  • CHANGELOG.md
  • src/common/README.md
  • src/common/__init__.py
  • src/common/sync_manager.py
  • src/plugin_system/__init__.py
  • test/test_lazy_package_imports.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

src.common and src.plugin_system now load declared exports on first access. sync_manager imports NumPy inside send_frame. New tests check import behavior, export compatibility, and unknown-name errors.

Changes

Lazy imports

Layer / File(s) Summary
Lazy package exports
src/common/__init__.py, src/plugin_system/__init__.py
Both packages map declared exports to their modules and load and cache each export on first access. Unknown package attributes raise AttributeError.
Deferred NumPy import
src/common/sync_manager.py
send_frame imports NumPy when it reaches the frame-conversion path. Its RGB encoding and UDP send behavior remain unchanged.
Import validation and documentation
test/test_lazy_package_imports.py, src/common/README.md, CHANGELOG.md
Tests check module loading, export identity, import forms, and unknown-name errors. The README and changelog describe the lazy exports and import behavior.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant Package as src.plugin_system
  participant Importlib
  participant ExportModule as mapped module
  Caller->>Package: access BasePlugin
  Package->>Importlib: import mapped module
  Importlib->>ExportModule: load module
  Package->>Caller: return cached BasePlugin
Loading

Merge Risk: ⚪ Minimal · up to d3ef6

The lazy exports and deferred NumPy import appear ready to merge after normal checks; hardware testing remains outstanding.

Security Architecture Review

Security architecture risk: 🔵 Low · up to d3ef6

The exported objects and frame protocol are preserved, and no new security exposure was established. Risk is low because dependency failures now occur later, while interrupted imports and simultaneous first access have not been directly validated.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The inspected change affects process-local package bindings and dependency initialization along an existing leader-to-peer frame path. It does not establish expanded attacker-controlled reachability, but the available evidence does not enumerate every runtime caller.

Trust Boundaries and Controls

  • observed — The changed frame path preserves its existing role and connected-peer checks. Moving the NumPy import does not bypass those checks or change the inspected transmission destination.

Resilience and Maintainability Implications

  • inferred — Both loaders publish a cached package value only after import and attribute resolution succeed, limiting partial package-level publication. Failed-import cleanup, interruption recovery, and concurrent initialization remain delegated to Python import machinery and were not directly validated.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files. (2 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: package re-exports now load on first use to improve import performance.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

codacy-production Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 4 complexity

Metric Results
Complexity 4

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

ChuckBuilds and others added 3 commits October 3, 2026 12:24
…Semgrep

Codacy flagged importlib.import_module(module_name) as a non-literal
import; module_name only ever comes from the module's own _LAZY map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ChuckBuilds
ChuckBuilds merged commit ef69201 into main Oct 3, 2026
15 checks passed
@ChuckBuilds
ChuckBuilds deleted the perf/lazy-package-imports branch October 3, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant