MILESTONE - SWAP -> SHARED & 长期容器 & 测试重构#26
Open
ChesterNotChest wants to merge 13 commits into
Open
Conversation
Collaborator
Author
|
完成了内存 SWAP -> SHARED |
- ENHANCEMENT 优化了日志产出，减少非必要日志占盘
- ADDED 补充了邮件提醒逻辑
- ADDED 完成了测试命令与测试安全运行预检功能
- TEST 新增 user / machine 的 task、API、repository-backed 测试
- FIX remove_collaborator() 对 ROOT 绑定的判断在枚举对象场景下会失效，可能误删 ROOT owner，现在改为统一 role value 判断。 - FIX create_container_api() 局部 reason map 漏了 machine_permission_denied，导致本应 403 的权限错误返回 500，现在统一走 REASON_STATUS_MAP。
ChesterNotChest
changed the title
There was a problem hiding this comment.
Pull request overview
This PR advances the Ctrl kernel “swap → shared memory” transition, introduces long-term container capabilities and pre-cleanup email reminders, and establishes a safer/refactored pytest + CI testing platform intended to run without touching production dependencies.
Changes:
- Rename container/machine resource fields from swap to shared memory across models/repos/services/APIs.
- Add long-term container support plus container cleanup reminder persistence + email reminder flow.
- Refactor and expand tests into a “safe-by-default” pytest suite (SQLite + external-service blocking) and add a GitHub Actions workflow + coverage config.
Reviewed changes
Copilot reviewed 90 out of 93 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/logging_config.py | Adds daily rotating file logging + redirects stdout/stderr to loggers. |
| utils/Container.py | Replaces swap_memory with shared_memory in container config DTO. |
| services/container_tasks.py | Implements shared-memory validation usage, long-term container state, cleanup metadata in listing/detail, and new APIs/helpers. |
| services/machine_tasks.py | Switches capacity field to max_shared_gb, adds heartbeat error handling and shared validation. |
| services/user_tasks.py | Adds long-term container count fields and expands allowed registration domains. |
| repositories/containers_repo.py | Renames container creation arg to shared_gb and adds validate_shared_request. |
| repositories/machine_repo.py | Renames max_swap_* to max_shared_* and updates update allowlist. |
| repositories/usercontainer_repo.py | Extends bindings to include granted_at for snapshots/auditing. |
| repositories/long_term_container_repo.py | Adds CRUD + counting APIs for long-term containers. |
| repositories/container_ssh_login_repo.py | Adds get_by_container helper. |
| repositories/container_cleanup_reminder_repo.py | Adds reminder persistence (dedupe via unique constraint) + ensure_table. |
| models/containers.py | Renames persisted column swap_gb → shared_gb. |
| models/machine.py | Renames persisted column max_swap_gb → max_shared_gb. |
| models/long_term_container.py | Adds long_term_containers table model. |
| models/container_cleanup_reminder.py | Adds container_cleanup_reminders table model. |
| models/init.py | Registers new models for metadata/import side effects. |
| blueprints/container_api.py | Adds long-term container endpoint + shared memory parsing + status mappings + list payload extensions. |
| blueprints/machine_api.py | Renames max swap field to max shared in machine APIs. |
| schemas/container_cleanup_task.py | Adds reminder scheduling logic, snapshot logging, and long-term skip behavior. |
| config.py | Adds env-backed config for long-term limit + reminder hours. |
| init.py | Adds create_app(overrides=...), config override injection, logging configuration, and disables schedulers in tests. |
| requirements.txt | Adds cryptography + pytest/pytest-cov to requirements. |
| pytest.ini | Sets safe default selection (not integration and not legacy) and registers markers. |
| .coveragerc | Adds coverage config with initial fail threshold and test/migrations omission. |
| docs/testing.md | Documents safe default test suite, markers, and rules. |
| docs/testing_legacy_migration.md | Documents legacy/integration test segregation and replacement mapping. |
| .github/workflows/ctrl-tests.yml | Adds GitHub Actions job for safe test suite with coverage + artifacts. |
| .gitignore | Ignores /logs/* but keeps logs/.gitkeep. |
| logs/.gitkeep | Keeps logs directory in repo. |
| .env.example | Documents new env vars for long-term limit + reminder hours. |
| tests_refactor_phase1_plan.md | Adds phase plan documentation (testing refactor). |
| tests_refactor_phase3_plan.md | Adds phase plan documentation (testing maturity + CI). |
| test/conftest.py | Introduces SQLite-only test app fixture, DB reset, and external service blocking. |
| test/assertions.py | Adds JSON assertion helpers. |
| test/factories.py | Adds ORM factories for users/machines/containers/bindings/tokens. |
| test/mocks.py | Adds shared mocks for auth, node send, crypto, and mail. |
| test/README.md | Documents test layout and safe-suite rules. |
| test/test_safe_pytest_app.py | Tests app factory safety contract (no background tasks + SQLite). |
| test/test_testing_safety_contract.py | Enforces marker registration, safe defaults, legacy/integration isolation, and CI config invariants. |
| test/test_testing_platform.py | Tests factories/mocks/assertion helpers baseline behavior. |
| test/utils/test_logging_config.py | Tests logging handler duplication prevention. |
| test/utils/test_mail.py | Tests mail send wrapper via mocked SMTP behavior. |
| test/test_mail.py | Marks legacy mail test file as integration. |
| test/test_api_web.py | Marks legacy API web tests as legacy. |
| test/test_api_web_machine.py | Marks legacy machine API web tests as legacy. |
| test/test_api_web_containers.py | Marks legacy containers API web tests as legacy. |
| test/test_user_sql.py | Marks legacy user SQL tests as legacy. |
| test/test_machine_sql.py | Marks legacy machine SQL tests as legacy. |
| test/test_container_sql.py | Marks legacy container SQL tests as legacy and updates swap→shared fields in legacy expectations. |
| test/user/test_user_tasks_auth.py | Adds task-level tests for login/password reset/change. |
| test/user/test_user_tasks_profile.py | Adds task-level tests for register/delete/update/detail/list behavior. |
| test/user/test_user_tasks_registration_code.py | Adds task-level tests for registration code flow. |
| test/user/test_user_api_auth.py | Adds API contract tests for register/request_code/login. |
| test/user/test_user_api_profile.py | Adds API contract tests for profile endpoints. |
| test/user/test_user_repository_integration.py | Adds small repository-backed integration tests (SQLite). |
| test/machine/test_machine_tasks_crud.py | Adds machine task CRUD tests. |
| test/machine/test_machine_tasks_status.py | Adds machine task status/heartbeat/listing tests. |
| test/machine/test_machine_tasks_permission.py | Adds machine permission task tests. |
| test/machine/test_machine_api_crud.py | Adds machine API CRUD contract tests. |
| test/machine/test_machine_api_listing.py | Adds machine API listing/token resolution tests. |
| test/machine/test_machine_api_permission.py | Adds machine API permission contract tests. |
| test/machine/test_machine_repository_integration.py | Adds small repository-backed machine integration tests. |
| test/container/conftest.py | Adds container test fixtures + node/crypto/heartbeat mocking helpers. |
| test/container/test_container_common.py | Adds baseline container test invariants for safe suite. |
| test/container/test_container_tasks_lifecycle.py | Adds container task lifecycle tests (create/remove/start/stop/restart). |
| test/container/test_container_tasks_collaborators.py | Adds collaborator add/remove/update role tests. |
| test/container/test_container_tasks_information.py | Adds detail/listing behavior tests incl cleanup + long-term fields. |
| test/container/test_container_tasks_long_term.py | Adds long-term container service tests and limit enforcement. |
| test/container/test_container_tasks_ssh.py | Adds last-ssh fetch/persist behavior tests. |
| test/container/test_container_tasks_node_errors.py | Adds node error mapping tests. |
| test/container/test_container_tasks_helpers.py | Adds parsing + cleanup-info helper tests. |
| test/container/test_container_api_lifecycle.py | Adds container API lifecycle contract tests. |
| test/container/test_container_api_collaborators.py | Adds container API collaborator contract tests. |
| test/container/test_container_api_information.py | Adds container API info/list/status/ssh refresh contract tests. |
| test/container/test_container_api_long_term.py | Adds long-term container API contract tests. |
| test/container/test_container_cleanup_task.py | Adds cleanup task unit tests (skip long-term, snapshot, removal). |
| test/container/test_container_cleanup_reminders.py | Adds cleanup reminder behavior tests (dedupe + mark sent). |
| test/container/test_container_ssh_refresh_task.py | Adds ssh-refresh scheduler + paging behavior tests. |
| test/repository/test_authentications_repo.py | Adds repository tests for auth token lifecycle. |
| test/repository/test_registration_code_repo.py | Adds repository tests for registration code lifecycle. |
| test/repository/test_container_ssh_login_repo.py | Adds repository tests for SSH login upsert/get. |
| test/repository/test_container_cleanup_reminder_repo.py | Adds repository tests for reminder dedupe semantics. |
| test/e2e/test_ctrl_user_machine_container_flow.py | Adds Ctrl-internal E2E flow tests for login→create→list and long-term toggling. |
| test/e2e/test_ctrl_cleanup_reminder_flow.py | Adds Ctrl-internal E2E tests for reminder sending + long-term skip. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # 非管理员用户必须先通过机器权限表过滤可见机器 | ||
| if user_id and not _is_operator_user(user_id): | ||
| if not _is_operator_user(request_user_id): | ||
| allowed = set(machine_permission_repo.list_machine_ids_by_user(user_id)) |
Comment on lines
+139
to
+143
| e = ValueError(f"max_shared_gb ({ss}) cannot be greater than max_memory_gb ({mm})") | ||
| setattr(e, 'error_reason', 'create_failed') | ||
| raise e | ||
|
|
||
| create_machine( | ||
| create_machine( |
|
|
||
| allowed = {"machine_name", "machine_ip", "machine_type", "machine_status", "cpu_core_number", | ||
| "memory_size_gb", "gpu_number", "gpu_type", "disk_size_gb", "machine_description", "swap_size_gb", "max_swap_gb", | ||
| "memory_size_gb", "gpu_number", "gpu_type", "disk_size_gb", "machine_description", "shared_size_gb", "max_shared_gb", |
|
|
||
| memory_gb: int = db.Column(db.Integer, nullable=False) | ||
| swap_gb: int = db.Column(db.Integer, nullable=False) | ||
| shared_gb: int = db.Column(db.Integer, nullable=False) |
| gpu_number: int = db.Column(db.Integer, nullable=True) | ||
| gpu_type: str = db.Column(db.String(120), nullable=True) | ||
| max_swap_gb: int = db.Column(db.Integer, nullable=False, default=2) | ||
| max_shared_gb: int = db.Column(db.Integer, nullable=True) |
Comment on lines
1294
to
1297
| try: # 理论不会报错 但是被建议保留 | ||
| total_count = count_containers(machine_id=machine_id) | ||
| total_page = max(1, math.ceil(total_count / page_size)) | ||
| except Exception: |
Comment on lines
259
to
264
| containers=container_ids, | ||
| amount_of_container=total, | ||
| amount_of_functional_container=functional, | ||
| amount_of_managed_container=managed, | ||
| amount_of_long_term_container=long_term_container_repo.count_by_user(u.id), | ||
| )) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.