Skip to content

scm: make ScmOverride hashable with nested values - #704

Open
mahaase wants to merge 1 commit into
BobBuildTool:masterfrom
mahaase:feature/url-scm-credentials
Open

scm: make ScmOverride hashable with nested values#704
mahaase wants to merge 1 commit into
BobBuildTool:masterfrom
mahaase:feature/url-scm-credentials

Conversation

@mahaase

@mahaase mahaase commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Problem

scmOverrides allows arbitrary values in match and set (schema {str: object}).
A natural use case is injecting authentication headers into a url SCM so that
downloads from a private server work — matched by host and kept out of the
version-controlled recipe:

scmOverrides:
   - match:
       url: "https://gitea.example.com/*"
     set:
       headers:
         Authorization: "token <personal access token>"

This crashes as soon as the override actually matches:

  File ".../bob/scm/scm.py", line 58, in __hash__
    frozenset(self.__set.items()), frozenset(self.__replace.items())))
TypeError: unhashable type: 'dict'

ScmOverride.__hash__ builds a frozenset directly from the raw match/set
items. A nested dict/list value (like headers) is unhashable, and since
matched overrides are collected into a set() in the builder
(builder.getActiveOverrides()), any build that hits such an override fails.

Fix

Recursively freeze nested dict/list values into frozenset/tuple before
hashing. __eq__ already compares the raw structures correctly and is left
unchanged.

Tests

Added two regression tests to test/unit/test_input_scmoverride.py:

  • a set with a nested headers dict is hashable and behaves correctly in a
    set() (equal overrides collapse, unequal ones stay distinct);
  • the nested value is injected verbatim into the matched SCM.

Note

This came out of a discussion about a proposed urlCredentials config option.
With this fix, authenticated url SCMs can be handled entirely through the
existing headers attribute + scmOverrides (kept in a git-ignored
user.yaml), so no dedicated credentials setting is needed.

@mahaase
mahaase force-pushed the feature/url-scm-credentials branch from 3c20b19 to a652b60 Compare July 30, 2026 15:36
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.25%. Comparing base (7b00892) to head (356c155).

Files with missing lines Patch % Lines
pym/bob/scm/scm.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #704      +/-   ##
==========================================
+ Coverage   89.23%   89.25%   +0.01%     
==========================================
  Files          50       50              
  Lines       16450    16456       +6     
==========================================
+ Hits        14679    14687       +8     
+ Misses       1771     1769       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mahaase
mahaase force-pushed the feature/url-scm-credentials branch from 2140c86 to d6c21cc Compare July 31, 2026 12:42
@jkloetzke

Copy link
Copy Markdown
Member

Wouldn't it be more natural to add support for "user:password" standard basic authentication in the URL?

Regarding the "token" field, this seems to be a non standard header. You can already add custom headers to an URL SCM. This should even work through scmOverride...

ScmOverride.__hash__ built a frozenset from the raw 'match' and 'set'
items. Both allow arbitrary values (schema '{str: object}'), e.g. the
'headers' dict of an url SCM. A nested dict/list value made the override
unhashable, so as soon as such an override matched, collecting it into
the builder's active-overrides set() crashed with:

    TypeError: unhashable type: 'dict'

Recursively freeze nested dicts/lists into frozensets/tuples before
hashing. This makes it possible to inject e.g. authentication headers
into a url SCM via scmOverrides:

    scmOverrides:
       - match:
           url: "https://gitea.example.com/*"
         set:
           headers:
             Authorization: "token <personal access token>"

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mahaase
mahaase force-pushed the feature/url-scm-credentials branch from d6c21cc to 356c155 Compare August 3, 2026 09:43
@mahaase mahaase changed the title scm: add urlCredentials for authenticated url SCM access scm: make ScmOverride hashable with nested values Aug 3, 2026
@mahaase

mahaase commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Wouldn't it be more natural to add support for "user:password" standard basic authentication in the URL?

Regarding the "token" field, this seems to be a non standard header. You can already add custom headers to an URL SCM. This should even work through scmOverride...

unluckely it doesn't work like expected. i switched this PR to fix the remaining issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants