Skip to content

resource: Expand PingOne SSO export coverage - #143

Open
samir-gandhi wants to merge 43 commits into
mainfrom
pingone-core-resources
Open

samir-gandhi wants to merge 43 commits into
mainfrom
pingone-core-resources

Conversation

@samir-gandhi

@samir-gandhi samir-gandhi commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Expands PingOne SSO export coverage from 12 to 25 resource types across three waves: the application family, the supporting/child resources batch, and the infrastructure the batch rides on (E2E fixtures, export pipeline extraction, and a review-bundle generator for resource PRs). Also includes export-correctness fixes surfaced by live UAT exports.

Resource coverage (13 new resource types)

Application family

  • pingone_application (OIDC/SAML applications)
  • pingone_application_attribute_mapping
  • pingone_application_sign_on_policy_assignment
  • pingone_sign_on_policy (with pingone_sign_on_policy_action as nested blocks)

Resource scope family

  • pingone_resource
  • pingone_resource_attribute (CORE/PREDEFINED variants filtered — no exportable API value)
  • pingone_resource_scope (built-in PingOne API scopes the provider schema cannot manage are filtered)
  • pingone_resource_scope_openid
  • pingone_resource_scope_pingone_api
  • pingone_resource_secret

Supporting resources

  • pingone_password_policy
  • pingone_group_nesting
  • pingone_population_default
  • pingone_population_default_identity_provider

Intentionally excludes user and user-assignment resources because individual end-user identities are not promotable environment configuration.

Export correctness

  • Resolves UUIDs embedded in plain SCIM filter strings, including pingone_group.user_filter (plain-string mode alongside the existing JSON-blob @@TFREF@@ mode; both share one resolution strategy in internal/core/embedded_references.go).
  • Filters CORE/PREDEFINED resource attributes that have no exportable API value.
  • Filters built-in PingOne API scopes that the provider schema cannot manage.
  • Silently skips deterministic PingOne system applications and other deterministic bootstrap objects; keeps warnings for actionable, environment-specific conditions and deduplicates them.
  • Documents system-managed/bootstrap resource analysis for future resource work (skill Phase 1a.5).

E2E tier: provisioned-environment fixtures

  • Expands the throwaway-environment fixture set to the full SSO family: 14 fixtured modules in terraform-test-data/sso/ wired through terraform-test-data/root/main.tf in dependency order (output → input cross-module wiring); resource_secret/application_secret are auto-covered by their parents.
  • Fixture provider pinned to pingidentity/pingone 1.21.1.
  • Fixture-module contract documented in tests/regression-provisioned/README.md (coverage table: 14 fixtured / 2 auto-covered / known gaps / follow-ups).
  • Live validation: fixtures apply cleanly against a real environment; base-vs-PR export compare exercises them.

Review bundle: human-reviewable report for resource PRs

Every resource-adding or resource-changing PR now carries review evidence generated by tools/review-bundle (internal/review/ + make review-bundle):

  • Changed-field detection: diffs definitions/ + internal/platform/ against the base ref, maps handlers to types, and produces a per-attribute semantic field diff (added/removed/changed YAML keys, nested-aware).
  • Definition-driven summary: API surface, full attribute tables (flags, references, transforms, source paths), dependency tables (exported vs fallback-variable), embedded reference rules, custom code paths, variables.
  • Restricted live export + raw API snapshots: runs the shared internal/export pipeline against a dependency-filtered registry, so the SSO-only throwaway environment never aborts on absent DaVinci services (the Orchestrator: per-type list failure aborts entire export — SSO-only environments cannot export #163 abort-on-list-failure behavior; Orchestrator: per-type list failure aborts entire export — SSO-only environments cannot export #163 remains the proper product fix — BOM gating + --fail-fast). Snapshots capture the raw API JSON per reviewed type.
  • Checklist: E2E fixture + regression-env coverage verified mechanically; computed/sensitive/provider-syntax prompts; export evidence (non-empty files, unresolved UUID counts, fallback variables); PASS/CHECK/SKIPPED/WARN statuses.
  • Security invariants: the tool never provisions (callers provision via tools/tf-regression-provision); the finished bundle is byte-scanned for the client secret and deleted on a hit; meta.json records the target environment ID only.
  • Surfaces: make review-bundle (auto-detects; dry-run without credentials), make review-bundle-live (provisions + keeps the environment for inspection), and a PR workflow (.github/workflows/review-bundle.yaml) that posts the bundle artifact + checklist highlights on PRs touching definitions/** or internal/platform/**.
  • Supporting refactor: the export pipeline was extracted verbatim from cmd/export.go into internal/export/pipeline.go so both the CLI and the review tool share one implementation.

Verification

  • go build ./...
  • go vet ./...
  • go fmt ./... (no diffs)
  • go test ./... -count=1
  • go run ./tools/validate-definitions definitions/
  • Live UAT export completed successfully; generated configuration passed terraform validate.
  • Provisioned-environment fixture batch: terraform init -input=false && terraform validate in terraform-test-data/root (credential-free), then full fixture apply + base-vs-PR compare against a live throwaway environment.
  • Review bundle: dry-run verified (--resource-types pingone_group → correct 5-type dependency closure, composition-root and fixture rows PASS); detect-only verified against this PR's own diff (23-type closure, 2 DaVinci types skipped live with the Orchestrator: per-type list failure aborts entire export — SSO-only environments cannot export #163 reason); 17 unit tests in internal/review/.
  • make review-bundle (auto-detect dry-run) produces the full 23-type bundle.

One pre-existing intermittent map-iteration test (TestResolveEmbeddedReferences_DisambiguatedName_StableAcrossEnvironments) may fail nondeterministically; reruns pass and the failure is unrelated to this change.

🤖 Generated with Claude Code

samir-gandhi and others added 25 commits July 24, 2026 15:34
…sources

pingone-go-client only covers DaVinci, Environments, Connectors, and
ConfigurationManagement APIs, so the ~92 open Platform/SSO/Authorize/MFA/
Protect/Verify export-support issues cannot be implemented against it.
Add patrickcping/pingone-go-sdk-v2/management (the SDK terraform-provider-
pingone itself uses) as a second client on Client, built lazily since it
performs an OAuth exchange at construction time rather than on first
request.
The go:embed directive listed pingone/base and pingone/davinci explicitly,
so every new resource category (sso, platform, mfa, ...) would require
editing this file — violating the "new resource requires zero edits to
existing files" invariant. A single pingone/*/*.yaml wildcard covers any
category subdirectory automatically.
Adds a schema-driven definition and Go handler for pingone_group,
covering name, description, population_id (referencing
pingone_population), user_filter, external_id, and custom_data
(jsonencode_raw). Uses the pingone-go-sdk-v2/management SDK via
Client.management(), following the list-then-iterate pagination
pattern documented in the new-resource-pingone skill.

Also bumps the hardcoded definition-count assertions in
internal/schema/integration_test.go and the sorted resource-type
list in internal/platform/pingone/dispatch_test.go, both of which
track the total number of registered definitions/resources and
needed updating for the new pingone_group entry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds the pingone_population YAML definition and Go handler using the
management SDK's list-then-paginate/get pattern, per issue #81. Both
password_policy.id and theme.id nested references degrade gracefully
to Terraform variable fallbacks since pingone_password_policy and
pingone_branding_theme are not yet exported by this tool; resolution
auto-promotes once those resource types land, with no changes needed
here.

Updates dispatch_test.go and schema integration_test.go resource-count
assertions to account for the new registered type, and flips the
pingone_population row in RESOURCE_COVERAGE.md to supported.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds Terraform export support for pingone_resource (OAuth 2.0 custom
resources) via the management SDK. Built-in resources (OPENID_CONNECT,
PINGONE_API) are filtered out of the list since only CUSTOM resources
are manageable through the pingone_resource Terraform resource.

Closes #84.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a schema-driven definition and Go handler for pingone_application,
covering OIDC, SAML, External Link, and WS-Fed application types (the
four the pingone_application Terraform resource schema actually supports).
The management SDK's ReadOneApplication200Response is a discriminated
union with 7 possible variants; a custom handler unwraps it via
GetActualInstance() and projects the result into a flat struct with one
nilable field per supported variant, letting the generic object/
nested_attributes engine emit exactly one *_options block per resource.

The 3 built-in PingOne system application types (Admin Console, Portal,
Self Service) have no corresponding schema block in the provider and are
skipped with a warning (c.AddWarning) rather than exported incorrectly.

Also bumps the hardcoded definition-count assertions in
internal/schema/integration_test.go and the sorted resource-type list in
internal/platform/pingone/dispatch_test.go, both of which track the total
number of registered definitions/resources and needed updating for the
new pingone_application entry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…olaris/pilot-combined-manual-test

# Conflicts:
#	internal/platform/pingone/dispatch_test.go
…aris/pilot-combined-manual-test

# Conflicts:
#	internal/platform/pingone/dispatch_test.go
#	internal/schema/integration_test.go
…polaris/pilot-combined-manual-test

# Conflicts:
#	internal/schema/integration_test.go
Merging the group/population/resource pilot branches together surfaced
a symbol collision: three independent agents each wrote an identically
named test helper for building a fake management.APIClient. Renamed
each to be resource-specific so all three test files can coexist.
…branch

Flips enabled: true on definitions/pingone/base/environment.yaml so the
resource participates in export alongside the pilot resources. This was
deliberately disabled in commit 9eb4335 ("not ready for publish") and
re-enabling it changes environment_id from a variable reference to a
real resource reference across every other exported resource -- a
breaking change for existing users, confirmed via make regression-local.
Not intended to ship; local manual-testing branch only.
Resolves the fallback-variable case for pingone_population.theme and
the DaVinci flow showForm theme reference (resource_flow.go), both of
which previously fell back to a Terraform variable because
pingone_branding_theme had no exported resource type.
pingone-go-client only covers DaVinci/Environments/Connectors; the
patrickcping/pingone-go-sdk-v2/management SDK added for the Platform/
SSO/Authorize backlog (issue #119) was undocumented outside a local,
gitignored skill file. Records which SDK backs which resource category,
why the management client is built lazily, and the shape differences
that affect handler code.
…source_scope family, application children)

Completes the application-family batch of PingOne SSO resource export
support: pingone_sign_on_policy (new base resource), pingone_resource_scope,
pingone_resource_scope_openid, pingone_resource_scope_pingone_api (children
of pingone_resource / built-in resources), and pingone_application_secret,
pingone_application_attribute_mapping, pingone_application_resource,
pingone_application_flow_policy_assignment,
pingone_application_sign_on_policy_assignment, and
pingone_application_resource_grant (children of pingone_application).

pingone_sign_on_policy and the resource_scope family were added ahead of
their dependents (application_sign_on_policy_assignment,
application_attribute_mapping's oidc_scopes, application_resource_grant's
scopes) so those references resolve to real resources instead of fallback
variables.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Three bugs surfaced by exporting against a real environment:

- resource/pingone_application_attribute_mapping: mapping names (e.g. the
  reserved "sub"/"saml_subject" core mappings) are only unique within a
  single application, not across the environment, causing duplicate-label
  export failures whenever multiple applications shared a core mapping name.
  Added label_fields: [name, application_id].

- resource/pingone_resource_scope, resource/pingone_application_resource:
  import_id_format used "{resource_id}", which the import generator treats
  as a reserved placeholder for the resource's own ID (data.ID), not as a
  reference to the "resource_id" *attribute*. This silently duplicated the
  scope/app-resource's own ID into both UUID segments of the import ID
  instead of including the parent resource's ID. Added a distinctly-named
  shadow attribute (parent_resource_id_for_import) for the import path.

- resource/pingone_application_resource_grant: SAML/external-link/WS-Fed
  applications have no resource-grants endpoint and return 404, aborting
  the whole export. Grants only apply to OIDC applications, so non-OIDC
  applications are now skipped without attempting the call.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ute CORE/PREDEFINED bug

Adds pingone_password_policy, pingone_group_nesting, pingone_resource_secret,
pingone_population_default, and pingone_population_default_identity_provider.
password_policy resolves the fallback-variable placeholder left on
pingone_population.password_policy.id from the prior batch.

Also fixes pingone_resource_attribute to export only user-authored (Type ==
CUSTOM) attributes — CORE/PREDEFINED attributes (e.g. "sub", "given_name")
exist on every resource automatically with no real API-returned value, which
was surfacing as spurious "required field value is missing" errors.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…up.user_filter

pingone_group.user_filter is a SCIM filter string (e.g. population.id eq
"<uuid>") rather than a jsonencode(...) JSON blob, so the existing
EmbeddedReferenceRule machinery (which walks JSON structures) couldn't reach
the embedded population UUID, and it was exported as a raw, environment-
specific ID.

Adds EmbeddedReferenceRule.PlainStringPattern: a single-capture-group regex
that locates a UUID inside an ordinary string attribute and resolves it via
the dependency graph exactly like the JSON-blob path (same Strategy
semantics), producing a new core.InterpolatedString type. Both formatters
render it correctly — HCL escapes everything except the embedded "${...}"
expression (avoiding the "$${...}" double-escaping that cty.StringVal would
otherwise produce), and tfjson passes it through as-is.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds terraform-test-data/ (hand-authored Terraform fixtures per resource),
tools/tf-regression-provision (creates/destroys a throwaway PingOne
environment via Terraform itself, using org-admin credentials scoped to
this run's lifecycle), and a `make e2e` entry point that applies fixtures
into that environment and runs the existing base-vs-PR export/compare
pipeline against it. Additive to tests/regression/, which continues
comparing export output against a static environment unchanged.

Closes #140.
Previously silently defaulted to NA, inheriting the same fallback used for
the unrelated PINGCLI_PINGONE_REGION_CODE export credential elsewhere in
this repo. The org-admin credential set should always be complete and
specified on its own terms.
Adds regression-provisioned.yaml, a single-job workflow that runs the
existing tests/regression-provisioned/run-local.sh on workflow_dispatch
only (no schedule yet - real environment creation/deletion per run needs
cost/quota sign-off first). Reuses the exact locally-proven script rather
than decomposing into a multi-job pipeline, avoiding untested cross-job
credential/state passing.

Also fixes a report-persistence gap in run-local.sh found while wiring
this up: reports were written to TMPDIR_LOCAL but never copied out before
the cleanup trap deleted that directory, so the workflow's artifact-upload
step would have found nothing. Added copy_reports(), mirroring the
existing pattern in tests/regression/run-local.sh.

Verified live end-to-end locally: provision, apply, all 5 matrix entries
pass, reports land in regression-reports/, teardown confirmed via a
follow-up API read (404).
Silently skip PingOne system applications and built-in API scopes that the Terraform provider cannot manage. Deduplicate actionable warnings, document the exclusions, and add guidance for identifying system-managed resources during new resource analysis.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

1 similar comment
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

samir-gandhi and others added 3 commits September 20, 2026 23:21
Expands terraform-test-data with three minimal SSO fixture modules
following the existing population fixture pattern: pingone_group,
pingone_resource, and pingone_sign_on_policy, wired into the
composition root in dependency order. Part of the fixture-coverage
rollout from the provisioned-environment E2E tier.

Co-Authored-By: Claude Code <noreply@anthropic.com>
tests: Add provisioned-environment E2E test tier
# Conflicts:
#	Makefile
#	internal/core/embedded_references.go
#	internal/core/embedded_references_test.go
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

Extract the strategy-resolution tail shared by the JSON-blob path
(processRawHCLValue) and the plain-string path (resolvePlainStringReference)
into one resolveReferenceToTFRef helper: UUID-format guard, variable-strategy
fallback allocation, graph lookup with reference_with_fallback fallback, and
dependency-graph edge creation. The two call sites differ only in how the
resolved token is embedded (bare-ref sentinel substitution into the blob vs
"${...}" splicing into an InterpolatedString) and in the edge annotation path
and jsonData context, both now parameters.

Behavior is unchanged; the plain-string, sentinel, and blob test suites
pass untouched.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

samir-gandhi and others added 3 commits October 20, 2026 13:10
Documents the goreleaser release checklist, including the
homebrew_casks migration (PR #164) and the tap_migrations.json
coordination with pingidentity/homebrew-tap#25, so the cask-related
release steps are caught during future releases.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Restores the reverted group/resource/sign-on-policy fixture batch
(aa6b011, reverted in 5b81a7d pending a release that has since shipped)
and expands terraform-test-data to cover every remaining fixturable SSO
resource: password_policy, population_default, group_nesting,
resource_scope, resource_attribute, resource_scope_openid,
resource_scope_pingone_api, application (OIDC WEB_APP with
response_types per the live-API AUTHORIZATION_CODE validation),
application_sign_on_policy_assignment, and application_attribute_mapping.

resource_secret and application_secret are auto-covered by their parent
fixtures; population_default_identity_provider and
application_flow_policy_assignment remain documented gaps in the
SSO-only throwaway environment.

Cross-module dependencies use the output→input pattern: producer
modules expose outputs.tf (group IDs, resource_id, sign_on_policy_id,
application_id) and consumers take them as input variables, wired in
dependency order in terraform-test-data/root/main.tf.

Bumps the fixture provider pin from 1.16.0-beta.2 to 1.21.1 (matching
the static regression tier's locked version). Adds
tests/regression-provisioned/README.md documenting the fixture-module
contract and coverage table. Drops two stale go.sum entries left by the
pingone-go-client v0.13.0 bump.

Verified: terraform init + validate clean against 1.21.1 (including the
p1:read:user:e2e name validator and resource_attribute CUSTOM
cross-validation); live E2E run provisions all 14 fixture resources and
tears down cleanly.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

docs: Add release process documentation
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

samir-gandhi and others added 2 commits October 20, 2026 14:46
Lifts the schema-driven export pipeline (orchestrator setup, variable
extraction/injection, formatting, import-block and output-block building,
module generation) out of cmd/export.go into internal/export.Run, along
with its helper functions and their tests (moved to
internal/export/pipeline_test.go). cmd/export.go now parses flags,
constructs the client, loads the full registry via
export.LoadFullRegistry, and delegates.

The resource review-bundle tool (next commit) reuses this pipeline with a
pre-filtered registry to export a dependency-closed subset of resource
types — impossible through the CLI's --include-resources flag, which
filters after listing and therefore cannot avoid the DaVinci 403 abort
in SSO-only environments (issue #163).

Behavior-preserving: the pipeline body is moved verbatim; the full unit
suite including the relocated helper tests passes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
internal/review provides the pure-logic half of the resource review
bundle (the tool wires it up next):

- detect.go: maps git-diff'd definition/handler files to resource types
  and computes semantic field diffs between the base and PR versions of
  each changed definition (per-attribute, recursing nested attributes;
  section-level diffs for api/dependencies/variables/custom_handlers).
- diff.go: DiffDefinitions — the structural comparison engine.
- summary.go: definition-driven markdown summary (attribute tables with
  flags/refs/transforms, API surface, dependency + reference-target
  status, embedded-reference rules, custom code paths, variables,
  changed-fields report).
- checklist.go: review checklist with mechanically checkable rows
  (E2E fixture wiring, regression env presence, export evidence,
  unresolved UUIDs, fallback variables) and CHECK prompts for human
  verification (computed/sensitive declarations, provider syntax).
- html.go: single self-contained HTML review page (stdlib html/template,
  no external JS/CSS).
- meta.go: provenance record (base ref, mode, environment ID — never
  credentials).
- export.go: dependency-closure computation over the embedded
  definitions with the SSO-only guard excluding DaVinci types from the
  live set (issue #163 workaround), plus fallback-target detection.

Unit tests cover the diff engine and the closure computation against
the real embedded definitions (17 tests).

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

samir-gandhi and others added 2 commits October 20, 2026 14:53
tools/review-bundle is the entry point for the resource review bundle:
detects changed types from the git diff, computes the dependency-closed
review set, runs a restricted live export through the shared
internal/export pipeline against a filtered registry (the #163
workaround for SSO-only environments), snapshots raw API responses per
type, and renders meta.json / summary.md / checklist.md / review.html.

- internal/review/snapshot.go: per-type ListResources marshal with a
  recover() guard writing <type>.json.error instead of failing the bundle
- internal/review/bundle.go: SecretScanGuard byte-scans the finished
  bundle for the client secret (delete + exit non-zero on hit),
  CredentialsFromEnv, DetectOnlyJSON
- checklist: pingone_environment no longer warns about a missing E2E
  fixture — it IS the composition root; fixture rows now name their
  fixture category
- html: HTMLPageData exported for the CLI

The tool never provisions; callers (tests/review/run-live.sh, CI)
provision via tools/tf-regression-provision and pass the five
PINGCLI_PINGONE_* variables.

Co-Authored-By: Claude Code <noreply@anthropic.com>
- make review-bundle: dry-run (or live with PINGCLI_PINGONE_* set)
  bundle generation into review-bundle/ (gitignored)
- make review-bundle-live: tests/review/run-live.sh provisions a
  throwaway environment via tools/tf-regression-provision, runs the
  tool against it, and keeps the environment by default (REVIEW_DESTROY=1
  tears it down) — the review's point is a live environment that still
  exists and contains the change
- .github/workflows/review-bundle.yaml: on PRs touching definitions/**
  or internal/platform/**, a detect job (no credentials) computes the
  changed types from the diff against the base SHA, and a bundle job —
  gated by the regression-provision environment required-reviewer
  gating — provisions, exports, snapshots, uploads the bundle artifact
  (14-day retention), and posts a checklist-highlights PR comment
- contributing/pr-checklist.md: review-bundle checklist item
- .changelog/pr-143.txt: feature entry for the generator

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

…sponse

GET /environments/{envID}/groups/{groupID}/memberOfGroups returns
_embedded.groupMemberships ([]GroupMembership, where Id is the nested
group's ID), not _embedded.groups. The handler read GetGroupsOk(), which
is never populated for this endpoint, so every page was skipped and the
resource always exported zero instances. Surfaced by the review bundle's
live verification: the fixture creates a nesting (confirmed in terraform
state) while the export produced no group_nesting.tf.

- list: parse GetGroupMembershipsOk(); single-get path (ReadOneGroupNesting,
  decoding into *GroupNesting{Id, Type}) is unchanged
- tests: the fixture server served the same wrong _embedded.groups shape;
  rebuilt it to serve the real groupMemberships contract

Also tightens the review bundle's DaVinci SSO-only guard: the
resource-type prefix (pingone_davinci_) is authoritative, with the SDK
package check kept as defense in depth —
pingone_davinci_application_flow_policy lists via the management SDK but
still 403s in an SSO-only environment (#163).

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

make regression-local wrote exports into a mktemp directory that the
EXIT trap deleted, copied only the small JSON reports at the very end of
main, and aborted before any reporting when an export or the compare
tool failed — a crashed regression-compare even read as a pass. The same
shape existed in the provisioned E2E runner.

- Both runners now write reports, export outputs, and per-entry logs to
  regression-reports/run-<timestamp>/ (gitignored, symlinked from
  regression-reports/latest), which persists after the run
- Summary table + verdict + per-entry report paths print in the EXIT
  trap, so they appear even when a build or export aborts the loop
- Export/compare failures are classified ERROR (distinct from BREAKING),
  recorded in the entry's report file as an error field instead of
  silently passing
- Export logs are captured per entry to keep the console readable;
  compare output streams and is captured too
- READMEs document the output layout

Verified end-to-end with stubbed binaries/compare against a three-entry
matrix covering PASS, BREAKING, and ERROR paths.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

The make target was a separate inline copy of the base-vs-PR pipeline
that predates the runner script and kept the old behavior after the
runner's rework: exports into a mktemp dir deleted on exit, reports
copied only at the end, no output at all when the PR export failed. The
stale base worktree from such a failed run even survived as a dangling
git-worktree registration.

The target now delegates to tests/regression/run-local.sh, which
persists everything under regression-reports/run-<timestamp>/ and always
prints the summary (see the tests: Persist regression reports commit).

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

The persisted regression reports are raw JSON, and the compare tool's
stdout summary enumerates every diff line individually — a real run
produced 192 breaking diffs that were all one repeated change
(environment_id moving from var.pingone_environment_id to a resource
reference). The report needs to group by pattern, not list every line,
locally and in the CI PR comment.

- internal/compare/report.go: Report/FileReport/DiffReport moved here
  from the tool (identical JSON schema — a CI contract), plus a small
  markdown table helper
- internal/compare/render.go: RenderMarkdown and RenderHTML with
  breaking diffs grouped by pattern (kind+attribute+expected→actual):
  one row per pattern with a resource count, affected resources in a
  collapsed list (capped at 20 with an overflow line); count-1 diffs
  render individually; acceptable additions as a compact table; the
  runners' error-shape report renders the error prominently. HTML
  mirrors internal/review/html.go: self-contained, auto-escaped,
  inline styles, dark-mode aware
- regression-compare: --markdown/--html flags writing the friendly
  reports; JSON output, exit codes, and stdout summary unchanged
- Both regression runners: --markdown/--html per entry, error paths
  write a markdown twin, finalize points non-PASS entries at the .md
- regression.yaml: compare step also emits the markdown report (uploaded
  with the JSON), and the PR comment's breaking-changes block splices
  the grouped markdown section instead of dumping every raw diff

Verified against the real run's reports: 192 breaking diffs collapse to
one pattern row in all three failing matrix entries; stub-run of the
runner confirms per-entry .md/.html and error-twin files.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Regression Test Results

Matrix Entry Status Breaking Acceptable

✅ No regressions detected

All export configurations produced compatible output.


Generated by regression workflow • View run

This branch had an error being deployed

1 failed deployment
regression-provision — 58a70754 Deployed Oct 21, 2026 by samir-gandhi via bundle #5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant