Skip to content

feat(rum): add rum_operation_strong_links resource - #720

Open
michael-richey wants to merge 7 commits into
michael.richey/add-rum-operationsfrom
michael.richey/add-rum-operation-strong-links
Open

michael-richey wants to merge 7 commits into
michael.richey/add-rum-operationsfrom
michael.richey/add-rum-operation-strong-links

Conversation

@michael-richey

Copy link
Copy Markdown
Collaborator

Summary

RUM operation strong links — keyed by the composite (operation_id, feature_id). Depends on rum_operations (and rum_applications).

Stacked on #719 (rum_operations).

Design

  • Create requires application_id and operation_name which are not in the response. pre_resource_action_hook derives them from the parent source operation (it runs before connect_resources remaps operation_id, so the operation_id is still the source id and state.source["rum_operations"] is keyed by it).
  • Update sends only status (the sole updatable field); the composite key is read from state.destination.
  • Delete uses the composite key from state.destination.
  • application_id and operation_name are create-only (not in the response), so they're excluded from diffs via deep_diff_config.exclude_regex_paths (must survive prep_resource so create can read them).
  • resource_connections: rum_operations (remap attributes.operation_id), rum_applications (remap attributes.application_id).

Changes

  • datadog_sync/model/rum_operation_strong_links.py (new)
  • datadog_sync/models/__init__.py — register
  • tests/unit/test_rum_operation_strong_links.py (new) — 7 unit tests
  • README.md — add rum_operation_strong_links (depends on rum_operations, rum_applications)

Testing

  • pytest tests/unit/test_rum_operation_strong_links.py → 7 passed
  • Full unit suite: no new failures.

Assumptions (to verify during integration-test recording)

  • operation_name in the create payload maps to the parent operation's attributes.name.
  • Update accepts a payload containing only status (per RUMOperationStrongLinkUpdateAttributes).

Follow-up

Integration tests + VCR cassettes deferred (require sandbox-org API access).

@michael-richey michael-richey left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strong implementation overall; I like the way the composite key behavior is encoded for update/delete and how tests validate status-only updates.

One non-blocking robustness suggestion:

  • pre_resource_action_hook silently no-ops if rum_operations source state is missing, which can lead to create payloads lacking application_id / operation_name and producing less-actionable API errors later.
  • Consider raising SkipResource (or a clear exception) when required derived fields cannot be populated, so operators get an explicit dependency/error message.

No blocking issues in this diff.

@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 936598e to 893649f Compare September 25, 2026 20:53
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from 2e1c183 to 6dd8547 Compare September 25, 2026 20:53
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Good suggestion — fixed in 893649fb. pre_resource_action_hook now raises SkipResource with an explicit message when the parent operation can't be found in source state or when operation_id is missing, so operators get an immediate, actionable error instead of a silent no-op that produces an incomplete create payload. Two new tests cover both cases.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Pagination is incomplete, and read-only attribute differences can cause perpetual ineffective updates.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds synchronization support for RUM operation strong links and their parent-resource mappings.

Changes:

  • Implements list, create, update, delete, and dependency remapping.
  • Registers and documents the resource.
  • Adds unit coverage for resource operations.
File Description
datadog_sync/​model/​rum_operation_strong_links.py Implements the resource model.
datadog_sync/​models/​__init__.py Registers the model.
tests/​unit/​test_rum_operation_strong_links.py Adds unit tests.
README.md Documents support and dependencies.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +55 to +57
resp = await client.get(self.resource_config.base_path)

return resp["data"]
Comment on lines +44 to +48
deep_diff_config={
"ignore_order": True,
# application_id and operation_name are create-only (not in the
# response), so a source-vs-destination diff would always flag them.
"exclude_regex_paths": [r".*\['application_id'\]", r".*\['operation_name'\]"],
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 893649f to 381b14d Compare September 25, 2026 21:25
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from c279302 to 608bc96 Compare September 28, 2026 14:10
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 381b14d to 2fbb6d5 Compare September 28, 2026 14:10
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fixed in 2fbb6d5e (two issues):

  1. Pagination: get_resources now uses client.paginated_request with PaginationConfig(page_size=100, page_size_param="page[limit]", page_number_param="page[offset]") so all strong links are imported, not just the first page. Test updated to verify paginated_request is used.

  2. Non-updatable fields in diff: Added description, tags, feature_id, and operation_id to deep_diff_config.exclude_regex_paths (alongside the existing application_id and operation_name). These fields are in the response but NOT updatable via PUT (only status is), so including them in the diff caused a non-converging update loop. They're kept in the resource for create but excluded from comparison.

@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from 608bc96 to f73ec4c Compare October 1, 2026 15:54
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 2fbb6d5 to d58b3b5 Compare October 1, 2026 15:54
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Integration Test Findings (EU1 → US5 sync)

Ran a full integration test with a rum_operation_strong_links resource created in the EU1 internal testing org. The import fails because the list endpoint requires query parameters that the model doesn't provide.

Import failure — 400 Bad Request

ERROR - Paginated request TRUNCATED for /api/v2/rum/operations/strong_links at page[offset]=0 page[limit]=100 after collecting 0 resource(s).
Error: 400 Bad Request - {"errors":[{"title":"At least one of operation_id or feature_id must be provided"}]}

Root cause

The get_resources method calls GET /api/v2/rum/operations/strong_links without any query parameters. However, the API requires at least one of operation_id or feature_id to be provided as a filter. Without these, the endpoint returns 400.

Suggested fix

The get_resources method in RUMOperationStrongLinks needs to iterate over the imported rum_operations and fetch strong links per operation ID, e.g.:

async def get_resources(self, client: CustomClient) -> List[Dict]:
    # Get all operations from source state
    operations = self.config.state.source.get("rum_operations", {})
    all_strong_links = []
    for op_id in operations:
        resp = await client.get(f"{self.resource_config.base_path}?operation_id={op_id}")
        all_strong_links.extend(resp["data"])
    return all_strong_links

Alternatively, if the API supports listing all strong links without filters in some environments, the model should handle the 400 gracefully and fall back to per-operation fetching.

What was tested

A strong link was successfully created in EU1 via POST /api/v2/rum/operations/strong_links with type: "strong_links" and status: "CONFIRMED" (note: the API expects DRAFT, CONFIRMED, or REJECTED — not "enabled"). The resource exists in the source org but cannot be imported by sync-cli due to the list endpoint issue.

@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fixed in 4440f025. The strong_links list endpoint requires at least one of operation_id or feature_id as a query parameter (the OpenAPI spec marks them optional, but the API returns 400 without one).

get_resources now iterates over rum_operations in state (source for import, destination for apply) and fetches strong links per operation_id using a query parameter. Falls back to an empty list when no operations are in state yet. Removed the unused PaginationConfig since per-operation fetches are unlikely to need pagination.

Two new tests: one verifying per-operation iteration with operation_id query param, one verifying empty result when no operations exist.

@michael-richey

Copy link
Copy Markdown
Collaborator Author

Re-test Results (after fix 4440f02)

Re-ran the full integration test with the updated branch. The list endpoint fix works — no more 400 error on the get_resources call. However, 0 strong links were imported.

What works

The get_resources method no longer crashes. The fix to iterate over rum_operations and fetch strong links per operation_id is correct in approach.

What doesn't work

During the import discovery phase, all 10 resource types call get_resources in parallel. When rum_operation_strong_links.get_resources() runs, state.source["rum_operations"] is empty because rum_operations hasn't been imported yet (it's being discovered in the same phase). The code falls back to an empty list, so 0 strong links are found.

Suggested fix

The get_resources method for strong_links needs to fetch operations directly from the API (not from state) during the discovery phase. For example:

async def get_resources(self, client: CustomClient) -> List[Dict]:
    if client is not self.config.source_client:
        # destination apply: use state as before
        state_map = getattr(self.config.state, "destination", {})
        operations = state_map.get("rum_operations", {})
        ...
    else:
        # source import: fetch operations from API since state isn't populated yet
        operations_resp = await client.get("/api/v2/rum/operations/search", ...)
        operations = {op["id"]: op for op in operations_resp}
        ...

Alternatively, ensure rum_operations is fully imported before rum_operation_strong_links discovery begins (dependency ordering in the import phase).

@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from f73ec4c to 67daf21 Compare October 1, 2026 18:13
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 4440f02 to 2498ccf Compare October 1, 2026 18:13
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fixed in 2498ccfb. get_resources used state.source['rum_operations'] to find operation IDs, but during import discovery all resource types run in parallel, so state wasn't populated yet → 0 strong links imported.

Fix: for the source client (import), get_resources now fetches operations directly from the API search endpoint instead of relying on state. For the destination client (apply), it still uses state (rum_operations is already synced by then). Tests updated to verify both paths.

@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 2498ccf to 0215115 Compare October 1, 2026 18:48
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fuzz Test Finding: rum_operation_strong_links needs destination reconciliation

During fuzz testing with 52 strong links (including 50 from an operation with 100 feature_ids), the sync failed with 50 x 409 Conflict errors:

409 Conflict - {"errors":[{"status":"409","detail":"Strong link for this operation and feature already exists"}]}

Root cause

Same issue as rum_operations (PR #719): rum_operation_strong_links has skip_resource_mapping=True, so the pre-apply listing phase is skipped. When strong links already exist at the destination from a previous sync, create_resource tries to POST them again, and the API returns 409.

The create_resource method does a plain POST without checking for existing strong links:

async def create_resource(self, _id: str, resource: Dict) -> Tuple[str, Dict]:
    destination_client = self.config.destination_client
    resource.pop("id", None)
    payload = {"data": resource}
    resp = await destination_client.post(self.resource_config.base_path, payload)
    return _id, resp["data"]

Suggested fix

Add destination reconciliation to create_resource, similar to the fix applied to rum_operations (PR #719). Before POSTing, search for an existing strong link with the same operation_id + feature_id at the destination and adopt it via update instead of creating a duplicate.

What worked in the fuzz test

Despite this issue, the fuzz test confirmed sync-cli handles many edge cases correctly:

  • ✅ Special characters in operation names (unicode, HTML injection, backslashes, brackets, zero-width chars, mixed CJK/Cyrillic scripts) — all imported and synced correctly
  • ✅ Duplicate playlist names — 3 playlists with the same name all imported and synced
  • ✅ Pagination stress — 56 metrics all imported and synced (including 25 fuzz metrics)
  • ✅ 504 Gateway Timeout — sync-cli handled API timeouts with backoff and retry, all metrics eventually synced
  • ✅ Orphaned dependencies — teams-ownership mappings with non-existent application_id were properly skipped with "missing connections" error
  • ✅ 27 operations with various edge-case names all synced
  • ✅ 5 of 7 teams-ownership mappings synced (2 with non-existent app IDs correctly skipped)

@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fixed in 80c65f87. rum_operation_strong_links has skip_resource_mapping=True, so the pre-apply listing phase is skipped. When strong links already exist at the destination from a previous sync, create_resource POSTed unconditionally and got 409 Conflict.

Fix: create_resource now searches for an existing strong link with the same operation_id + feature_id at the destination before POSTing. If found, it hydrates state and delegates to update. New test verifies the reconciliation path.

@michael-richey

Copy link
Copy Markdown
Collaborator Author

Re-test: Reconciliation fix works but introduces type field mismatch

The destination reconciliation fix (80c65f87) resolved the 409 Conflict — strong links that already exist at the destination are now adopted via update instead of creating duplicates. However, the create path now fails with a 400 Bad Request for 50 of 80 strong links:

400 Bad Request - {"errors":[{"status":"400","title":"Bad Request","detail":"got type \"rum_operation_strong_links\" expected one of \"strong_links\""}]}

Root cause

The create_resource method sends the resource body as-is, which includes "type": "rum_operation_strong_links" (the sync-cli resource type name). But the API expects "type": "strong_links" (the API's type name). The original create_resource had this right (it was just a plain POST), but the reconciliation code likely re-sets the type field to the sync-cli resource type name when hydrating state.

The fix needs to ensure the type field in the POST body is "strong_links" (the API type), not "rum_operation_strong_links" (the sync-cli resource type). This is the same pattern as rum_operations where the API type is "operations" not "rum_operations".

Lifecycle test results

Scenario Result Notes
Create ❌ FAIL 50 of 80 fail with type mismatch (the 30 that succeed are new creates with correct type)
Update ✅ PASS Reconciliation + update works correctly
Skip ✅ PASS No diff detected
Delete ✅ PASS Resources deleted at destination

Fuzz sync results

The fuzz sync (171 resources) now passes with 0 failures:

  • 169 successes
  • 2 skipped (orphaned dependencies — correct behavior)
  • 0 failures

All special chars, duplicates, pagination stress, and 504 timeout handling work correctly.

@michael-richey

Copy link
Copy Markdown
Collaborator Author

Fixed in 111428f2. The create_resource method was sending the resource body with type='rum_operation_strong_links' (the sync-cli resource type), but the API expects type='strong_links'. The update_resource PUT payload had the same bug.

Fix: create_resource now sets resource['type'] = 'strong_links' before POSTing. update_resource uses 'strong_links' in the PUT payload type field instead of self.resource_type. Test updated to verify the correct API type.

@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from fbd95da to 3a336dd Compare October 1, 2026 20:37
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 111428f to 99755b9 Compare October 1, 2026 20:37
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from 3a336dd to bcf4158 Compare October 1, 2026 21:17
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 99755b9 to 456a32c Compare October 1, 2026 21:17
RUM operation strong links are keyed by the composite (operation_id,
feature_id). The create payload requires application_id and operation_name
which are NOT in the response, so pre_resource_action_hook derives them from
the parent source operation (it runs before connect_resources remaps
operation_id). Only status is updatable, so update sends only status and the
composite key is read from state.destination. application_id and
operation_name are create-only, excluded from diffs via deep_diff_config
(must survive prep_resource so create can read them).

- datadog_sync/model/rum_operation_strong_links.py (new)
- datadog_sync/models/__init__.py -- register
- tests/unit/test_rum_operation_strong_links.py (new) -- 7 unit tests
- README.md -- add rum_operation_strong_links (depends on rum_operations,
  rum_applications)

Integration tests + VCR cassettes deferred (require sandbox-org API access).
Per review: pre_resource_action_hook silently no-ops when rum_operations
source state is missing, which can lead to create payloads lacking
application_id/operation_name and producing less-actionable API errors
later. Now raises SkipResource with an explicit message when the parent
operation can't be found or operation_id is missing, so operators get an
immediate, actionable error.
…rom diff

Per review (two comments):

1. The list endpoint is paginated but get_resources only returned the first
   page. Switch to client.paginated_request with PaginationConfig so all
   strong links are imported. Test updated to verify paginated_request is used.

2. The diff included response attributes that update_resource cannot change
   (description, tags, feature_id, operation_id). If any differed while
   status was equal, the handler repeatedly scheduled an update but the PUT
   sends only status, creating a non-converging loop. Now all non-updatable
   response fields are excluded from the diff via deep_diff_config
   (keeping them in the resource for create, but excluding from comparison).
Per integration test findings (EU1 -> US5 sync):

The strong_links list endpoint (GET /api/v2/rum/operations/strong_links)
requires at least one of operation_id or feature_id as a query parameter.
Without it, the API returns 400 'At least one of operation_id or feature_id
must be provided'. The OpenAPI spec marks them optional, but the API enforces
at least one.

Fix: get_resources now iterates over rum_operations in state (source for
import, destination for apply) and fetches strong links per operation_id
using a query parameter. Falls back to an empty list when no operations are
in state yet. Removed the unused PaginationConfig since per-operation fetches
are unlikely to need pagination.

Two new tests: one verifying per-operation iteration, one verifying empty
result when no operations exist.
Per integration test re-test: get_resources used state.source['rum_operations']
to find operation IDs, but during import discovery all resource types run in
parallel, so state isn't populated yet -> 0 strong links imported.

Fix: for the source client (import), get_resources now fetches operations
directly from the API search endpoint instead of relying on state. For the
destination client (apply), it still uses state (rum_operations is already
synced by then). Tests updated to verify both paths.
Per fuzz test findings: rum_operation_strong_links has
skip_resource_mapping=True, so the pre-apply listing phase is skipped.
When strong links already exist at the destination from a previous sync,
create_resource POSTs unconditionally and gets 409 Conflict.

Fix: create_resource now searches for an existing strong link with the
same operation_id + feature_id at the destination before POSTing. If
found, it hydrates state and delegates to update. New test verifies the
reconciliation path.
…ayloads

Per integration test re-test: create_resource sent the resource body as-is
with type='rum_operation_strong_links' (the sync-cli resource type), but the
API expects type='strong_links'. This caused 400 Bad Request on 50 of 80
strong links. update_resource had the same bug in its PUT payload.

Fix: create_resource now sets resource['type'] = 'strong_links' before
POSTing. update_resource uses 'strong_links' in the PUT payload type field
instead of self.resource_type. Test updated to verify the correct API type.
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operations branch from bcf4158 to dcf0952 Compare October 2, 2026 01:45
@michael-richey
michael-richey force-pushed the michael.richey/add-rum-operation-strong-links branch from 456a32c to 258282d Compare October 2, 2026 01:45
@michael-richey

Copy link
Copy Markdown
Collaborator Author

Thanks for adding strong-link support — this is close, but I see two correctness issues to fix:

1) get_resources() is unpaginated for both operations and strong links

  • Operations are fetched with a single GET to /api/v2/rum/operations/search (paginated endpoint).
  • Strong links are fetched per operation with a single GET to /api/v2/rum/operations/strong_links?operation_id=... (also paginated: page[offset], page[limit]).

Result: larger orgs can silently miss resources during import/sync.

2) Destination branch uses source ids instead of destination ids

In destination mode, operation_ids = set(operations.keys()) uses state keys from rum_operations. If keying/normalization changes (or differs from raw API ids), this can query strong links with the wrong ids. Safer to derive ids from values (e.g., op["id"]) rather than dict keys.

Suggested fix:

  • Paginate both operation and strong-link listing.
  • Build destination operation id set from operation payload values, not mapping keys.

I’d treat these as blockers for completeness/correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants