Skip to content

fix: strip paired shell quotes from env vars and consolidate legacy env compat - #1734

Merged
SAKURA-CAT merged 9 commits into
mainfrom
fix/env-quotes
Aug 21, 2026
Merged

fix: strip paired shell quotes from env vars and consolidate legacy env compat#1734
SAKURA-CAT merged 9 commits into
mainfrom
fix/env-quotes

Conversation

@SAKURA-CAT

@SAKURA-CAT SAKURA-CAT commented Aug 11, 2026

Copy link
Copy Markdown
Member

Summary

修复 shell 脚本注入环境变量时字面引号导致解析异常的问题,并将遗留环境变量兼容逻辑下沉到独立的 compat 包统一管理。

当通过 shell 脚本(eval、命令替换、/etc/environment)注入环境变量时,字面引号字符会保留在值中(如 SWANLAB_API_KEY='"abc123"' 实际值为 "abc123"),导致字符串字段携带多余引号、数值/布尔字段解析失败。本 PR 在 Settings 环境变量加载阶段统一剥离首尾成对引号,同时将各 settings 模块中直接读取 os.environ 的遗留环境变量 factory 集中到 compat 包,统一经 getenv 剥离引号。

Changes

  • 新增 swanlab/sdk/internal/settings/compat/

    • env.pystrip_env_quotes / getenv 工具函数,仅当首尾字符相同且为 "' 时剥离
    • experiment.py / core.py / integration.py:各域遗留环境变量 factory(含 project_name_factorymap_resume_value 等)
    • __init__.py:聚合导出与 log_dir_factory
  • swanlab/sdk/internal/settings/__init__.py

    • 新增 _QuoteAwareEnvSettingsSource,在 __init__ 中清洗 self.env_vars(取代原覆写 _load_env_vars 的实现),并替换默认 EnvSettingsSource
    • SECRETS_DIR / CONFIG_DIR / get_user_config_dir 改用 getenv
  • core.py / experiment.py / integration.py:移除本地 factory 定义,改为从 compat 导入

  • project_name_factory:先剥离引号再校验 SWANLAB_PROJ_NAME,非法时告警并回退默认项目名

  • 测试

    • 新增 TestEnvQuoteStripping(10 个场景)与 TestLegacyEnvQuoteStripping(覆盖遗留命名环境变量)
    • 新增 test_project_name_factory_env_vartests/unit/sdk/internal/pkg/constraints/test_pkg_constraints.py

Testing

未运行测试。建议执行:

  • pytest tests/unit/sdk/internal/settings/test_settings.py
  • pytest tests/unit/sdk/internal/pkg/constraints/test_pkg_constraints.py

Notes

  • 剥离规则仅当首尾字符相同且为 "' 时生效,避免误伤单侧引号或不匹配场景
  • 后续可考虑通过环境变量开关控制剥离行为(见 _QuoteAwareEnvSettingsSource 注释)
  • 关联 PR:add env double quote remove support #1732(已关闭的前序尝试)

Implement a custom EnvSettingsSource that strips matching leading and
trailing quotes from environment variable values before they are
processed by pydantic-settings. This handles shell-injected values like
`SWANLAB_API_KEY='"abc123"'` that retain literal quote characters from
eval, command substitution, or /etc/environment.
@SAKURA-CAT SAKURA-CAT added the 🐛 bug Something isn't working label Aug 11, 2026
@SAKURA-CAT SAKURA-CAT self-assigned this Aug 11, 2026

@SAKURA-CAT SAKURA-CAT left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审核结论: 要求修改

注:平台不允许对自己的 PR 提交 REQUEST_CHANGES,故以 COMMENT 形式提交,结论以本节为准。合并前请先处理阻塞与必须修改项。

本次 PR 解决的问题

修复 shell 脚本注入的环境变量值携带成对引号(如 SWANLAB_API_KEY='"abc123"')导致 Settings 解析失败或字段携带多余引号的问题。

审核范围

  • 配置、认证与 HTTP 客户端: settings/__init__.py 自定义 env source 与配置源优先级、settings/experiment.py factory 校验
  • 安全与隐私: 环境变量凭据泄露(发现阻塞问题,见 inline)
  • 横切质量与代码规范: 调试残留、无关代码移动
  • 测试与验证: 新增 10 个引号剥离用例 + 2 个 PROJ_NAME factory 用例

未挂行发现

  • 说明: 覆写私有方法 EnvSettingsSource._load_env_vars:已在 pydantic-settings 2.8.1(声明最低版本)与 2.13.1(当前安装)确认签名与调用点一致,且新增测试可兜底行为变化,无需行动。
  • 说明: 引号剥离启发式会改变合法地以成对同类引号开头/结尾的值,PR 描述已声明此权衡,可接受。

验证

  • 相关测试: uv run pytest tests/unit/sdk/internal/settings/ tests/unit/sdk/internal/pkg/constraints/ → 197 passed
  • uv run ruff check . → 通过(注:print 调试残留未被现有规则覆盖)
  • uv run basedpyright(变更文件)→ 0 errors;CI Type check (Python 3.9) 通过
  • uv run pytest tests/unit → 1575 passed, 46 skipped;32 failed + 1 收集错误均为缺少可选依赖(soundfile/rdkit),已在 base main 复现相同失败,与 PR 无关
  • PR Actions 全部通过(ubuntu/macos/windows × Python 3.9-3.14 + type check)

Comment thread swanlab/sdk/internal/settings/__init__.py Outdated
Comment thread swanlab/sdk/internal/settings/experiment.py Outdated
Comment thread swanlab/sdk/internal/settings/__init__.py Outdated

@SAKURA-CAT SAKURA-CAT left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充评审:关于覆写私有方法 _load_env_vars 的替代方案

补充上一轮 review 中"说明"项(覆写私有方法 EnvSettingsSource._load_env_vars)的进一步分析与实测。结论:存在不依赖私有方法签名的等效实现,建议改用方案 A(详见 inline)。两种替代方案均在本地以等价模型验证(API key / mode / host / 嵌套字段 / 布尔字段共 5 项断言全部通过,pydantic-settings 2.13.1)。

Comment thread swanlab/sdk/internal/settings/__init__.py Outdated
@SAKURA-CAT SAKURA-CAT changed the title fix: strip paired shell quotes from env var values in settings fix: strip paired shell quotes from env vars and consolidate legacy env compat Aug 21, 2026
@SAKURA-CAT
SAKURA-CAT merged commit 9c29b50 into main Aug 21, 2026
20 checks passed
@SAKURA-CAT
SAKURA-CAT deleted the fix/env-quotes branch August 21, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐛 bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant