Skip to content

Full integration test - #167

Closed
kevihan-te wants to merge 7 commits into
mainfrom
CP-2453-full-integration-tests
Closed

Full integration test#167
kevihan-te wants to merge 7 commits into
mainfrom
CP-2453-full-integration-tests

Conversation

@kevihan-te

@kevihan-te kevihan-te commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds hand-written core test infrastructure (MockApiServer, exception status mapping) in thousandeyes-sdk-core
  • Syncs generated OAS-driven integration test artifacts across 21 SDK packages (154 new test files)
  • Each package includes mock_manifest.py, integration_test_utils.py, conftest.py, and per-operation test_*_integration.py modules

Scope

Package count Integration test modules
21 91

Excluded: thousandeyes-sdk-bgp-monitors (missing OAS spec in generator)

Status

Do not merge — This PR is only for preview the final test result

Related generator PR: https://github.com/thousandeyes/thousandeyes-sdk-generator/pull/79

Test plan

  • Install packages locally (pip install -e thousandeyes-sdk-core + target packages)
  • Run core mock server unit tests: pytest thousandeyes-sdk-core/test/test_mock_server.py
  • Spot-check generated integration tests on representative packages (alerts, tags, tests)
  • Review generated test volume and CI runtime impact before merge

Made with Cursor

kevihan-te and others added 2 commits July 24, 2026 15:34
Move the integration mock server into test-only sdk_test_support helpers
and expose ApiException.exception_class_for_http_status for generated
integration error assertions.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sync OAS-driven integration test artifacts (mock manifest, test base, conftest, and per-operation integration tests) across 21 packages for CP-2453 full rollout evaluation.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kevihan-te kevihan-te changed the title CP-2453 Full integration test rollout (do not merge) CP-2453 Full integration test Jul 28, 2026
kevihan-te and others added 4 commits July 28, 2026 12:13
Success response bodies now use valid JSON quotes instead of HTML entities,
and optional expand enum params are omitted when OpenAPI examples are invalid.

Co-authored-by: Cursor <cursoragent@cursor.com>
Treat Z and +00:00 as equivalent when comparing serialized SDK request
bodies to OAS examples, fixing alert suppression window integration tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
Recursively ignore read-only fields in nested request objects and
regenerate dashboards/streaming integration tests for list responses.

Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate endpoint-agents and event-detection integration tests with Python booleans, sync get_events ongoing param support, and compare responses using only fields present in constructed models.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
93373 76677 82% 40% 🟢

New Files

File Coverage Status
thousandeyes-sdk-administrative/test/conftest.py 100% 🟢
thousandeyes-sdk-administrative/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-administrative/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-administrative/test/test_account_groups_api_integration.py 100% 🟢
thousandeyes-sdk-administrative/test/test_permissions_api_integration.py 98% 🟢
thousandeyes-sdk-administrative/test/test_roles_api_integration.py 100% 🟢
thousandeyes-sdk-administrative/test/test_user_events_api_integration.py 99% 🟢
thousandeyes-sdk-administrative/test/test_users_api_integration.py 100% 🟢
thousandeyes-sdk-agents/test/conftest.py 100% 🟢
thousandeyes-sdk-agents/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-agents/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-agents/test/test_agent_proxies_api_integration.py 98% 🟢
thousandeyes-sdk-agents/test/test_cloud_and_enterprise_agent_notification_rules_api_integration.py 99% 🟢
thousandeyes-sdk-agents/test/test_cloud_and_enterprise_agents_api_integration.py 100% 🟢
thousandeyes-sdk-agents/test/test_enterprise_agent_cluster_api_integration.py 99% 🟢
thousandeyes-sdk-agents/test/test_local_problems_api_integration.py 99% 🟢
thousandeyes-sdk-agents/test/test_tests_assignment_on_agents_api_integration.py 100% 🟢
thousandeyes-sdk-alerts/test/conftest.py 100% 🟢
thousandeyes-sdk-alerts/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-alerts/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-alerts/test/test_alert_rules_api_integration.py 100% 🟢
thousandeyes-sdk-alerts/test/test_alert_suppression_windows_api_integration.py 100% 🟢
thousandeyes-sdk-alerts/test/test_alerts_api_integration.py 99% 🟢
thousandeyes-sdk-connectors/test/conftest.py 100% 🟢
thousandeyes-sdk-connectors/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-connectors/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-connectors/test/test_credential_vault_operations_api_integration.py 100% 🟢
thousandeyes-sdk-connectors/test/test_cyber_ark_conjur_connectors_api_integration.py 100% 🟢
thousandeyes-sdk-connectors/test/test_generic_connectors_api_integration.py 100% 🟢
thousandeyes-sdk-connectors/test/test_operation_connectors_api_integration.py 98% 🟢
thousandeyes-sdk-connectors/test/test_webhook_operations_api_integration.py 100% 🟢
thousandeyes-sdk-core/test/conftest.py 80% 🟢
thousandeyes-sdk-core/test/sdk_test_support/init.py 100% 🟢
thousandeyes-sdk-core/test/sdk_test_support/mock_server.py 84% 🟢
thousandeyes-sdk-core/test/sdk_test_support/mock_server_types.py 100% 🟢
thousandeyes-sdk-core/test/test_exceptions.py 100% 🟢
thousandeyes-sdk-core/test/test_mock_server.py 100% 🟢
thousandeyes-sdk-credentials/test/conftest.py 100% 🟢
thousandeyes-sdk-credentials/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-credentials/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-credentials/test/test_credentials_api_integration.py 100% 🟢
thousandeyes-sdk-dashboards/test/conftest.py 100% 🟢
thousandeyes-sdk-dashboards/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-dashboards/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-dashboards/test/test_dashboard_snapshots_api_integration.py 100% 🟢
thousandeyes-sdk-dashboards/test/test_dashboards_api_integration.py 100% 🟢
thousandeyes-sdk-dashboards/test/test_dashboards_filters_api_integration.py 100% 🟢
thousandeyes-sdk-emulation/test/conftest.py 100% 🟢
thousandeyes-sdk-emulation/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-emulation/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-emulation/test/test_emulation_api_integration.py 99% 🟢
thousandeyes-sdk-endpoint-agents/test/conftest.py 100% 🟢
thousandeyes-sdk-endpoint-agents/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-endpoint-agents/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-endpoint-agents/test/test_endpoint_agent_log_items_api_integration.py 99% 🟢
thousandeyes-sdk-endpoint-agents/test/test_endpoint_agents_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-agents/test/test_endpoint_agents_transfer_api_integration.py 98% 🟢
thousandeyes-sdk-endpoint-agents/test/test_endpoint_proxies_api_integration.py 98% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/conftest.py 100% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/test_agent_to_server_endpoint_instant_scheduled_tests_api_integration.py 99% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/test_http_server_endpoint_instant_scheduled_tests_api_integration.py 99% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/test_run_endpoint_instant_scheduled_tests_api_integration.py 99% 🟢
thousandeyes-sdk-endpoint-labels/test/conftest.py 100% 🟢
thousandeyes-sdk-endpoint-labels/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-endpoint-labels/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-endpoint-labels/test/test_endpoint_agent_labels_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/conftest.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_http_server_endpoint_scheduled_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_local_network_endpoint_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_network_dynamic_endpoint_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_network_endpoint_scheduled_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_real_user_endpoint_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/conftest.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/test_agent_to_server_endpoint_dynamic_tests_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/test_agent_to_server_endpoint_scheduled_tests_api_integration.py 100% 🟢
thousandeyes-sdk-endpoint-tests/test/test_endpoint_real_user_tests_api_integration.py 98% 🟢
thousandeyes-sdk-endpoint-tests/test/test_endpoint_scheduled_tests_api_integration.py 98% 🟢
thousandeyes-sdk-endpoint-tests/test/test_http_server_endpoint_scheduled_tests_api_integration.py 100% 🟢
thousandeyes-sdk-event-detection/test/conftest.py 100% 🟢
thousandeyes-sdk-event-detection/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-event-detection/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-event-detection/test/test_events_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/conftest.py 100% 🟢
thousandeyes-sdk-instant-tests/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-instant-tests/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-instant-tests/test/test_agent_to_agent_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_agent_to_server_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_api_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_dns_server_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_dns_trace_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_dnssec_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_ftp_server_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_http_page_load_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_http_server_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_instant_tests_api_integration.py 77% 🟢
thousandeyes-sdk-instant-tests/test/test_sip_server_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_voice_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-instant-tests/test/test_web_transaction_instant_tests_api_integration.py 99% 🟢
thousandeyes-sdk-internet-insights/test/conftest.py 100% 🟢
thousandeyes-sdk-internet-insights/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-internet-insights/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-internet-insights/test/test_internet_insights_catalog_providers_api_integration.py 99% 🟢
thousandeyes-sdk-internet-insights/test/test_internet_insights_outages_api_integration.py 100% 🟢
thousandeyes-sdk-snapshots/test/conftest.py 100% 🟢
thousandeyes-sdk-snapshots/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-snapshots/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-snapshots/test/test_test_snapshots_api_integration.py 99% 🟢
thousandeyes-sdk-streaming/test/conftest.py 100% 🟢
thousandeyes-sdk-streaming/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-streaming/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-streaming/test/test_streaming_api_integration.py 99% 🟢
thousandeyes-sdk-tags/test/conftest.py 100% 🟢
thousandeyes-sdk-tags/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-tags/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-tags/test/test_tag_assignment_api_integration.py 100% 🟢
thousandeyes-sdk-tags/test/test_tags_api_integration.py 100% 🟢
thousandeyes-sdk-test-results/test/conftest.py 100% 🟢
thousandeyes-sdk-test-results/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-test-results/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-test-results/test/test_api_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_dns_server_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-test-results/test/test_dns_trace_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_dnssec_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_network_bgp_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_network_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-test-results/test/test_voice_rtp_server_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_voice_sip_server_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_web_ftp_server_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_web_http_server_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_web_page_load_test_results_api_integration.py 99% 🟢
thousandeyes-sdk-test-results/test/test_web_transactions_test_results_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/conftest.py 100% 🟢
thousandeyes-sdk-tests/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-tests/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-tests/test/test_agent_to_agent_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_agent_to_server_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_api_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_bgp_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_dns_server_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_dns_trace_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_dnssec_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_ftp_server_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_http_server_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_page_load_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_path_visualization_interface_groups_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_sip_server_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_tests_api_integration.py 99% 🟢
thousandeyes-sdk-tests/test/test_voice_tests_api_integration.py 100% 🟢
thousandeyes-sdk-tests/test/test_web_transaction_tests_api_integration.py 100% 🟢
thousandeyes-sdk-usage/test/conftest.py 100% 🟢
thousandeyes-sdk-usage/test/integration_test_utils.py 100% 🟢
thousandeyes-sdk-usage/test/mock_manifest.py 100% 🟢
thousandeyes-sdk-usage/test/test_quotas_api_integration.py 100% 🟢
thousandeyes-sdk-usage/test/test_usage_api_integration.py 99% 🟢
TOTAL 99% 🟢

Modified Files

File Coverage Status
thousandeyes-sdk-administrative/test/test_utils.py 91% 🟢
thousandeyes-sdk-agents/test/test_utils.py 91% 🟢
thousandeyes-sdk-alerts/test/test_utils.py 91% 🟢
thousandeyes-sdk-bgp-monitors/test/test_utils.py 91% 🟢
thousandeyes-sdk-connectors/test/test_utils.py 91% 🟢
thousandeyes-sdk-core/src/thousandeyes_sdk/core/exceptions.py 52% 🟢
thousandeyes-sdk-credentials/test/test_utils.py 91% 🟢
thousandeyes-sdk-dashboards/test/test_utils.py 91% 🟢
thousandeyes-sdk-emulation/test/test_utils.py 91% 🟢
thousandeyes-sdk-endpoint-agents/test/test_utils.py 91% 🟢
thousandeyes-sdk-endpoint-instant-tests/test/test_utils.py 91% 🟢
thousandeyes-sdk-endpoint-labels/test/test_utils.py 91% 🟢
thousandeyes-sdk-endpoint-test-results/test/test_utils.py 91% 🟢
thousandeyes-sdk-endpoint-tests/test/test_utils.py 91% 🟢
thousandeyes-sdk-event-detection/src/thousandeyes_sdk/event_detection/api/events_api.py 81% 🟢
thousandeyes-sdk-event-detection/test/test_utils.py 91% 🟢
thousandeyes-sdk-instant-tests/test/test_utils.py 91% 🟢
thousandeyes-sdk-internet-insights/test/test_utils.py 91% 🟢
thousandeyes-sdk-snapshots/test/test_utils.py 91% 🟢
thousandeyes-sdk-streaming/test/test_utils.py 91% 🟢
thousandeyes-sdk-tags/test/test_utils.py 91% 🟢
thousandeyes-sdk-test-results/test/test_utils.py 91% 🟢
thousandeyes-sdk-tests/test/test_utils.py 91% 🟢
thousandeyes-sdk-usage/test/test_utils.py 91% 🟢
TOTAL 89% 🟢

updated for commit: a8102f1 by action🐍

@kevihan-te kevihan-te changed the title CP-2453 Full integration test Full integration test Jul 28, 2026
@kevihan-te kevihan-te closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant