Testing

This guide covers testing strategies, tools, and practices for TMI development including unit tests, integration tests, API tests, and end-to-end tests.

Testing Philosophy

TMI follows a comprehensive testing approach:

Unit Tests - Fast tests with no external dependencies
Integration Tests - Tests with real database and services
API Tests - Complete API workflow testing with Postman/Newman
E2E Tests - Full user journey testing with Playwright

Test Pyramid

        /\
       /E2E\          Few, slow, expensive
      /------\
     /  API  \        Some, medium speed
    /----------\
   /Integration\     More, medium speed
  /--------------\
 /   Unit Tests  \   Many, fast, cheap
/------------------\

Testing Principles

Test business logic thoroughly - Unit test all business rules
Test integration points - Verify components work together
Test user workflows - Ensure complete features work end-to-end
Automate everything - All tests should be automated
Fast feedback - Unit tests run in seconds
Realistic testing - Integration tests use real databases

Unit Testing

Server Unit Tests (Go)

TMI server uses Go's built-in testing framework.

Running Unit Tests

# Run all unit tests
make test-unit

# Run specific test
go test -v ./api -run TestCreateThreatModel

# Run with coverage
make test-coverage-unit

Writing Unit Tests

Test File Naming: *_test.go

Example Test:

// api/threat_model_test.go
package api

import (
    "testing"
    "github.com/stretchr/testify/assert"
)

func TestCreateThreatModel(t *testing.T) {
    // Arrange
    tm := ThreatModel{
        Name:        "Test Threat Model",
        Description: stringPtr("Test description"),
    }

    // Act
    result, err := createThreatModelLogic(tm)

    // Assert
    assert.NoError(t, err)
    assert.NotEmpty(t, result.ID)
    assert.Equal(t, tm.Name, result.Name)
}

Test Patterns

Table-Driven Tests:

func TestAuthorizationRoles(t *testing.T) {
    tests := []struct {
        name     string
        role     string
        canRead  bool
        canWrite bool
        canDelete bool
    }{
        {"owner", "owner", true, true, true},
        {"writer", "writer", true, true, false},
        {"reader", "reader", true, false, false},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            assert.Equal(t, tt.canRead, canRead(tt.role))
            assert.Equal(t, tt.canWrite, canWrite(tt.role))
            assert.Equal(t, tt.canDelete, canDelete(tt.role))
        })
    }
}

Mocking External Dependencies:

type MockDatabase struct {
    mock.Mock
}

func (m *MockDatabase) GetThreatModel(id string) (*ThreatModel, error) {
    args := m.Called(id)
    return args.Get(0).(*ThreatModel), args.Error(1)
}

func TestWithMock(t *testing.T) {
    // Create mock
    mockDB := new(MockDatabase)
    mockDB.On("GetThreatModel", "123").Return(&ThreatModel{
        ID: "123",
        Name: "Test",
    }, nil)

    // Use mock in test
    tm, err := mockDB.GetThreatModel("123")

    assert.NoError(t, err)
    assert.Equal(t, "123", tm.ID)
    mockDB.AssertExpectations(t)
}

Web App Unit Tests (Angular/TypeScript)

TMI-UX uses Vitest for unit testing.

Running Unit Tests

# Run all tests
pnpm run test

# Run in watch mode
pnpm run test:watch

# Run with UI
pnpm run test:ui

# Run specific test
pnpm run test -- src/app/pages/tm/tm.component.spec.ts

# Coverage report
pnpm run test:coverage

Writing Unit Tests

Test File Naming: *.spec.ts

Example Component Test (using Vitest):

// src/app/pages/tm/tm.component.spec.ts
import '@angular/compiler';
import { vi, expect, beforeEach, describe, it } from 'vitest';
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TmComponent } from './tm.component';
import { ApiService } from '../../core/services/api.service';
import { of } from 'rxjs';

describe('TmComponent', () => {
  let component: TmComponent;
  let fixture: ComponentFixture<TmComponent>;
  let mockApiService: {
    getThreatModels: ReturnType<typeof vi.fn>;
  };

  beforeEach(async () => {
    vi.clearAllMocks();

    // Create mock using Vitest
    mockApiService = {
      getThreatModels: vi.fn()
    };

    await TestBed.configureTestingModule({
      imports: [TmComponent],
      providers: [
        { provide: ApiService, useValue: mockApiService }
      ]
    }).compileComponents();

    fixture = TestBed.createComponent(TmComponent);
    component = fixture.componentInstance;
  });

  it('should create', () => {
    expect(component).toBeTruthy();
  });

  it('should load threat models on init', () => {
    // Arrange
    const mockThreatModels = [
      { id: '1', name: 'TM 1' },
      { id: '2', name: 'TM 2' }
    ];
    mockApiService.getThreatModels.mockReturnValue(of(mockThreatModels));

    // Act
    component.ngOnInit();

    // Assert
    expect(mockApiService.getThreatModels).toHaveBeenCalled();
    expect(component.threatModels).toEqual(mockThreatModels);
  });
});

Service Test:

// src/app/core/services/api.service.spec.ts
import { TestBed } from '@angular/core/testing';
import { HttpClientTestingModule, HttpTestingController } from '@angular/common/http/testing';
import { ApiService } from './api.service';

describe('ApiService', () => {
  let service: ApiService;
  let httpMock: HttpTestingController;

  beforeEach(() => {
    TestBed.configureTestingModule({
      imports: [HttpClientTestingModule],
      providers: [ApiService]
    });

    service = TestBed.inject(ApiService);
    httpMock = TestBed.inject(HttpTestingController);
  });

  afterEach(() => {
    httpMock.verify();
  });

  it('should fetch threat models', () => {
    const mockThreatModels = [{ id: '1', name: 'TM 1' }];

    service.getThreatModels().subscribe(tms => {
      expect(tms).toEqual(mockThreatModels);
    });

    const req = httpMock.expectOne('/api/threat_models');
    expect(req.request.method).toBe('GET');
    req.flush(mockThreatModels);
  });
});

DFD Service Integration Testing (Vitest)

For DFD services with complex interdependent behavior, TMI-UX uses an integration testing approach with Vitest instead of mocks that would require duplicating business logic.

Key Principles:

Use Real Service Instances: Replace mocks with actual service instances for business logic services
Mock Cross-Cutting Concerns: Keep LoggerService mocked since it's a cross-cutting concern
Test Real Integration: Verify actual service integration and behavior
Eliminate Logic Duplication: No need to replicate service logic in mocks

Example - Testing DFD Edge Service:

```typescript // src/app/pages/dfd/infrastructure/services/infra-edge.service.spec.ts import { Graph } from '@antv/x6'; import { InfraEdgeService } from './infra-edge.service'; import { InfraEdgeQueryService } from './infra-edge-query.service'; import { InfraPortStateService } from './infra-port-state.service'; import { InfraX6CoreOperationsService } from './infra-x6-core-operations.service'; import { createTypedMockLoggerService, type MockLoggerService } from '../../../../../testing/mocks';

describe('InfraEdgeService - X6 Integration Tests', () => { let service: InfraEdgeService; let queryService: InfraEdgeQueryService; let portStateManager: InfraPortStateService; let x6CoreOps: InfraX6CoreOperationsService; let mockLogger: MockLoggerService;

beforeEach(() => { // Create mock for LoggerService only (cross-cutting concern) mockLogger = createTypedMockLoggerService();

// Create REAL service instances for integration testing
queryService = new InfraEdgeQueryService(mockLogger as unknown as LoggerService);
portStateManager = new InfraPortStateService(
  queryService,
  mockLogger as unknown as LoggerService,
);
x6CoreOps = new InfraX6CoreOperationsService(mockLogger as unknown as LoggerService);

// Create service under test with real dependencies
service = new InfraEdgeService(
  mockLogger as unknown as LoggerService,
  portStateManager,
  x6CoreOps,
);

}); }); ```

Service Dependency Chain:

``` InfraEdgeService +-- InfraPortStateService (REAL) | +-- InfraEdgeQueryService (REAL) | +-- LoggerService (MOCK) +-- InfraX6CoreOperationsService (REAL) | +-- LoggerService (MOCK) +-- LoggerService (MOCK) ```

When to Use Integration Testing:

Services have complex interdependent behavior
Mocking would require duplicating significant business logic
You need to verify actual integration between services
The services are part of the same bounded context (e.g., DFD infrastructure layer)

When to Use Unit Testing with Mocks:

Testing isolated units of functionality
Dependencies are simple or cross-cutting concerns (like logging)
You want to test error conditions that are hard to reproduce with real services
Performance is a concern for test execution speed

Testing Strategy Decision Tree:

``` Is this a high-level orchestrator/coordinator? +- YES: Mock all dependencies (test orchestration logic only) | Example: AppDfdOrchestrator with 12 mocked dependencies +- NO: Use integration testing approach +- Create real service instances +- Only mock cross-cutting concerns (LoggerService) +- Test actual service integration Example: InfraEdgeService with real InfraPortStateService ```

Angular + Vitest Setup:

The test setup in `src/test-setup.ts` handles Angular JIT compilation globally via `src/testing/compiler-setup.ts`. Zone.js is loaded globally via `src/testing/zone-setup.ts`. TestBed state is NOT serializable across Vitest's forked processes, so each test file must initialize TestBed in its own `beforeAll()` hook if needed.

For complete implementation examples, see:

`src/app/pages/dfd/infrastructure/services/infra-edge.service.spec.ts`
`src/app/pages/dfd/infrastructure/services/infra-port-state.service.spec.ts`

Integration Testing

Integration tests verify that components work correctly with real databases and services.

Server Integration Tests (Go)

Running Integration Tests

# Run all integration tests (automatic setup and cleanup)
make test-integration

# This automatically:
# 1. Starts PostgreSQL container
# 2. Starts Redis container
# 3. Runs migrations
# 4. Starts server
# 5. Runs tests
# 6. Cleans up everything

Test Configuration

Integration tests use dedicated ports to avoid conflicts:

PostgreSQL: Port 5433 (vs 5432 for development)
Redis: Port 6380 (vs 6379 for development)
Server: Port 8080

Writing Integration Tests

Test File Naming: *_integration_test.go

Example:

// api/threat_model_integration_test.go
package api

import (
    "testing"
    "net/http"
    "net/http/httptest"
    "github.com/stretchr/testify/assert"
)

func TestDatabaseThreatModelIntegration(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    // Create threat model
    threatModelData := map[string]interface{}{
        "name": "Integration Test TM",
        "description": "Test with real database",
    }

    req := suite.makeAuthenticatedRequest("POST", "/threat_models", threatModelData)
    w := suite.executeRequest(req)

    assert.Equal(t, http.StatusCreated, w.Code)

    // Verify in database
    var tm ThreatModel
    err := suite.db.First(&tm).Error
    assert.NoError(t, err)
    assert.Equal(t, "Integration Test TM", tm.Name)
}

Test Data Management

Predictable Test Users (using login hints):

func createTestUser(hint string) (*User, string) {
    // Create specific test user 'alice@test.tmi' instead of random
    resp, _ := http.Get(
        "http://localhost:8080/oauth2/authorize?idp=test&login_hint=" + hint
    )

    // Parse token from response
    token := parseTokenFromResponse(resp)
    return &User{Email: hint + "@test.tmi"}, token
}

func TestMultiUserScenario(t *testing.T) {
    alice, aliceToken := createTestUser("alice")
    bob, bobToken := createTestUser("bob")

    // Test with both users
}

Test Patterns

Complete Entity Lifecycle:

func TestThreatModelLifecycle(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    // 1. Create
    createReq := suite.makeAuthenticatedRequest("POST", "/threat_models", data)
    createW := suite.executeRequest(createReq)
    assert.Equal(t, http.StatusCreated, createW.Code)
    tmID := parseID(createW.Body)

    // 2. Read
    getReq := suite.makeAuthenticatedRequest("GET", "/threat_models/" + tmID, nil)
    getW := suite.executeRequest(getReq)
    assert.Equal(t, http.StatusOK, getW.Code)

    // 3. Update
    updateReq := suite.makeAuthenticatedRequest("PUT", "/threat_models/" + tmID, updatedData)
    updateW := suite.executeRequest(updateReq)
    assert.Equal(t, http.StatusOK, updateW.Code)

    // 4. Delete
    deleteReq := suite.makeAuthenticatedRequest("DELETE", "/threat_models/" + tmID, nil)
    deleteW := suite.executeRequest(deleteReq)
    assert.Equal(t, http.StatusNoContent, deleteW.Code)

    // 5. Verify deletion
    verifyReq := suite.makeAuthenticatedRequest("GET", "/threat_models/" + tmID, nil)
    verifyW := suite.executeRequest(verifyReq)
    assert.Equal(t, http.StatusNotFound, verifyW.Code)
}

Authorization Testing:

func TestAuthorizationMatrix(t *testing.T) {
    suite := SetupIntegrationTest(t)
    defer suite.TeardownIntegrationTest(t)

    alice, aliceToken := createTestUser("alice")
    bob, bobToken := createTestUser("bob")

    // Alice creates threat model
    tm := createThreatModel(aliceToken)

    // Test reader permissions
    addAuthorization(tm.ID, bob.Email, "reader", aliceToken)

    // Bob can read
    getReq := makeRequestWithToken("GET", "/threat_models/" + tm.ID, nil, bobToken)
    assert.Equal(t, http.StatusOK, suite.executeRequest(getReq).Code)

    // Bob cannot write
    updateReq := makeRequestWithToken("PUT", "/threat_models/" + tm.ID, data, bobToken)
    assert.Equal(t, http.StatusForbidden, suite.executeRequest(updateReq).Code)

    // Bob cannot delete
    deleteReq := makeRequestWithToken("DELETE", "/threat_models/" + tm.ID, nil, bobToken)
    assert.Equal(t, http.StatusForbidden, suite.executeRequest(deleteReq).Code)
}

OpenAPI-Driven Integration Test Framework

TMI uses an OpenAPI-driven integration test framework located in test/integration/. The framework provides:

OAuth Authentication: Automated OAuth flows via the OAuth callback stub
Request Building: Type-safe request construction with fixtures
Response Validation: OpenAPI schema validation for all responses
Assertion Helpers: Specialized assertions for API responses

Framework Structure

test/integration/
├── framework/
│   ├── client.go       # HTTP client with authentication
│   ├── oauth.go        # OAuth authentication utilities
│   ├── fixtures.go     # Test data fixtures
│   ├── assertions.go   # Test assertion helpers
│   └── database.go     # Database utilities
├── spec/
│   ├── schema_loader.go    # OpenAPI schema loading
│   └── openapi_validator.go # Response validation
└── workflows/
    ├── example_test.go          # Framework demonstration
    ├── oauth_flow_test.go       # OAuth tests
    ├── threat_model_crud_test.go # Threat model CRUD
    ├── diagram_crud_test.go     # Diagram CRUD
    ├── user_operations_test.go  # User operations
    ├── user_preferences_test.go # User preferences
    └── admin_promotion_test.go  # Admin promotion

Writing Framework Tests

package workflows

import (
    "os"
    "testing"
    "github.com/ericfitz/tmi/test/integration/framework"
)

func TestResourceCRUD(t *testing.T) {
    // Skip if not running integration tests
    if os.Getenv("INTEGRATION_TESTS") != "true" {
        t.Skip("Skipping integration test")
    }

    serverURL := os.Getenv("TMI_SERVER_URL")
    if serverURL == "" {
        serverURL = "http://localhost:8080"
    }

    // Ensure OAuth stub is running
    if err := framework.EnsureOAuthStubRunning(); err != nil {
        t.Fatalf("OAuth stub not running: %v", err)
    }

    // Authenticate
    userID := framework.UniqueUserID()
    tokens, err := framework.AuthenticateUser(userID)
    framework.AssertNoError(t, err, "Authentication failed")

    // Create client
    client, err := framework.NewClient(serverURL, tokens)
    framework.AssertNoError(t, err, "Client creation failed")

    // Use subtests for each operation
    t.Run("Create", func(t *testing.T) {
        fixture := framework.NewThreatModelFixture().
            WithName("Test Model")

        resp, err := client.Do(framework.Request{
            Method: "POST",
            Path:   "/threat_models",
            Body:   fixture,
        })
        framework.AssertNoError(t, err, "Request failed")
        framework.AssertStatusCreated(t, resp)
    })
}

Test Coverage Plan

TMI aims for 100% API coverage (178 operations across 92 paths) organized in three tiers:

Tier	Purpose	Run Frequency	Time Budget
Tier 1	Core workflows (OAuth, CRUD)	Every commit	< 2 min
Tier 2	Feature tests (metadata, webhooks, addons)	Nightly	< 10 min
Tier 3	Edge cases & admin operations	Weekly	< 15 min

For the complete integration test plan including implementation roadmap and coverage tracking, see the source documentation at docs/migrated/developer/testing/integration-test-plan.md.

API Testing

TMI uses Postman collections and Newman for comprehensive API testing.

Running API Tests

# Run all API tests
make test-api

# Or run manually
cd test/postman
./run-tests.sh

Test Collections

Located in test/postman/ directory:

comprehensive-test-collection.json - Main test suite
unauthorized-tests-collection.json - 401 error testing
threat-crud-tests-collection.json - Threat CRUD operations
metadata-tests-collection.json - Metadata operations
complete-metadata-tests-collection.json - Full metadata operations
permission-matrix-tests-collection.json - Authorization testing
bulk-operations-tests-collection.json - Batch operations
collaboration-tests-collection.json - WebSocket collaboration
advanced-error-scenarios-collection.json - 409, 422 edge cases
oauth-complete-flow-collection.json - OAuth flow testing
saml-tests-collection.json - SAML testing
webhooks-tests-collection.json - Webhook operations
addons-tests-collection.json - Addon operations
survey-tests-collection.json - Survey operations
ownership-transfer-tests-collection.json - Ownership transfer

Test Coverage

API tests cover:

70+ endpoints
91 workflow methods
All HTTP status codes (200, 201, 204, 400, 401, 403, 404, 409, 422, 500)
Authentication and authorization
CRUD operations for all entities
Metadata operations
Batch operations
Error scenarios

Threat Model API Coverage Analysis

A comprehensive coverage analysis tracks test coverage across all 41 threat model paths (105 operations):

Metric	Value
Total Threat Model Paths	41
Total Operations	105
Operations with Success Tests	~85 (81%)
Operations with 401 Tests	~25 (24%)
Operations with 403 Tests	~15 (14%)
Operations with 404 Tests	~35 (33%)
Operations with 400 Tests	~30 (29%)

Key Gap Areas:

Sub-resource 401 tests (threats, diagrams, metadata endpoints)
Authorization 403 tests for writer/reader role scenarios
Rate limit (429) and server error (500) tests

Collection Files (test/postman/):

comprehensive-test-collection.json - Full workflow tests
unauthorized-tests-collection.json - 401 authentication tests
permission-matrix-tests-collection.json - Multi-user authorization
threat-crud-tests-collection.json - Threat entity CRUD
collaboration-tests-collection.json - WebSocket collaboration
advanced-error-scenarios-collection.json - 409, 422 edge cases

For complete coverage matrix, gap analysis, and implementation recommendations, see docs/migrated/developer/testing/postman-threat-model-coverage.md.

Writing Postman Tests

Basic Test:

pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

pm.test("Response has threat models", function () {
    const response = pm.response.json();
    pm.expect(response).to.be.an('array');
    pm.expect(response.length).to.be.above(0);
});

Advanced Test with Setup:

// Pre-request Script
const data = {
    name: "Test Threat Model",
    description: "Created by test"
};
pm.collectionVariables.set("threat_model_data", JSON.stringify(data));

// Test Script
pm.test("Threat model created", function () {
    pm.response.to.have.status(201);
    const response = pm.response.json();

    pm.expect(response).to.have.property('id');
    pm.expect(response.name).to.equal("Test Threat Model");

    // Save ID for subsequent tests
    pm.collectionVariables.set("threat_model_id", response.id);
});

End-to-End Testing

TMI-UX uses Playwright for E2E testing.

Running E2E Tests

# Run all E2E tests
pnpm run test:e2e

# Run in headed mode (visible browser)
pnpm run test:e2e:headed

# Run with Playwright UI
pnpm run test:e2e:ui

# Run in debug mode
pnpm run test:e2e:debug

# Run specific browser
pnpm run test:e2e:chromium
pnpm run test:e2e:firefox
pnpm run test:e2e:webkit

Playwright Configuration

E2E tests are configured in playwright.config.ts and test files live under e2e/. The configuration runs tests against three browser projects (Chromium, Firefox, WebKit) and starts a local dev server automatically.

Writing E2E Tests

Test File Naming: *.spec.ts (in e2e/tests/)

Example Login Test:

// e2e/tests/login.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Login Flow', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/');
  });

  test('should display login page', async ({ page }) => {
    await expect(page.getByText('Sign In')).toBeVisible();
  });

  test('should login with test provider', async ({ page }) => {
    await page.getByText('Test Login').click();
    await expect(page).toHaveURL(/.*dashboard/);
    await expect(page.getByText('Threat Models')).toBeVisible();
  });
});

Example Diagram Test:

// e2e/tests/diagram.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Diagram Editor', () => {
  test.beforeEach(async ({ page }) => {
    // Login and navigate to diagram
    await page.goto('/threat-models/123/diagrams/456');
  });

  test('should add process to diagram', async ({ page }) => {
    // Open shape palette
    await page.getByTestId('shape-palette').click();

    // Select process shape
    await page.getByTestId('shape-process').click();

    // Click on canvas to add
    await page.getByTestId('diagram-canvas').click({ position: { x: 200, y: 200 } });

    // Verify process added
    await expect(page.locator('[data-shape=process]')).toBeVisible();
  });

  test('should edit process label', async ({ page }) => {
    await page.locator('[data-shape=process]').first().dblclick();
    await page.getByTestId('label-input').fill('Authentication Service');
    await page.getByTestId('label-save').click();

    await expect(page.locator('[data-shape=process]').first())
      .toContainText('Authentication Service');
  });
});

DFD Component Integration Test Plan

This section defines a comprehensive browser-based integration test plan for the DFD (Data Flow Diagram) component. The plan prioritizes catching selection styling persistence issues while providing complete coverage of all DFD features.

Note: This is a planned test suite. Existing DFD integration tests use Vitest and are located in src/app/pages/dfd/integration/. Those tests are currently skipped pending conversion to browser-based Playwright E2E tests due to Angular CDK JIT compilation issues in the Vitest environment.

Testing Philosophy

The DFD component requires browser-first integration testing because:

Real browser environments test actual user interactions
DOM verification inspects actual SVG elements and CSS properties
Visual regression detection catches styling issues that unit tests miss
Performance monitoring measures actual browser rendering performance

Critical Issues Priority

Priority	Issue	Impact	Test Focus
1 (Highest)	Selection Styling Persistence	After undo operations, restored cells retain selection styling (glow effects, tools)	Verify clean state restoration after undo/redo
2	Visual Effects State Management	Visual effects may accumulate or persist across operations	State transitions maintain correct styling throughout workflows
3	History System Integration	Visual effects may pollute undo/redo history	History contains only structural changes, not visual effects

Proposed Test Structure

``` e2e/tests/dfd/ ├── critical/ │ ├── selection-styling-persistence.spec.ts # Priority 1 bug │ ├── visual-effects-consistency.spec.ts # Visual state management │ └── history-system-integrity.spec.ts # History filtering ├── core-features/ │ ├── node-creation-workflows.spec.ts # Node creation and styling │ ├── edge-creation-connections.spec.ts # Edge creation and validation │ ├── drag-drop-operations.spec.ts # Movement and positioning │ └── port-management.spec.ts # Port visibility and connections ├── user-workflows/ │ ├── complete-diagram-creation.spec.ts # End-to-end workflows │ ├── context-menu-operations.spec.ts # Right-click operations │ ├── keyboard-interactions.spec.ts # Keyboard shortcuts │ └── multi-user-collaboration.spec.ts # Real-time collaboration ├── advanced-features/ │ ├── z-order-embedding.spec.ts # Layer management │ ├── export-functionality.spec.ts # Export to various formats │ ├── label-editing.spec.ts # In-place text editing │ └── performance-testing.spec.ts # Large diagram performance └── browser-specific/ ├── responsive-behavior.spec.ts # Window resize, zoom, pan ├── cross-browser-compatibility.spec.ts # Browser-specific behaviors └── accessibility-testing.spec.ts # Keyboard navigation, a11y ```

Critical Test: Selection Styling Persistence

The highest priority test verifies that deleted and restored cells have clean state:

```typescript import { test, expect } from '@playwright/test'; import { DfdPage } from '../../helpers/dfd-page';

test.describe('Selection Styling Persistence Bug', () => { test('should restore deleted nodes without selection styling', async ({ page }) => { const dfd = new DfdPage(page);

// Create node
await dfd.createNode('actor', { x: 100, y: 100 });
await expect(dfd.getNode('actor')).toBeVisible();

// Select node and verify selection styling
await dfd.selectNode('actor');
await dfd.verifySelectionStyling('actor', true);
await dfd.verifyTools('actor', ['button-remove', 'boundary']);

// Delete selected node
await dfd.deleteSelected();
await expect(dfd.getNodes()).toHaveCount(0);

// Undo deletion - CRITICAL VERIFICATION
await dfd.undo();
await expect(dfd.getNodes()).toHaveCount(1);

// VERIFY: No selection styling artifacts
await dfd.verifySelectionStyling('actor', false);
await dfd.verifyTools('actor', []);
await dfd.verifyCleanState('actor');

// VERIFY: Graph selection is empty
await expect(dfd.getSelectedCells()).toHaveCount(0);

}); }); ```

Styling Constants Reference

Tests should use centralized styling constants from `src/app/pages/dfd/constants/styling-constants.ts`:

Constant	Value	Usage
`DFD_STYLING.SELECTION.GLOW_COLOR`	`rgba(255, 0, 0, 0.8)`	Selection glow effect
`DFD_STYLING.HOVER.GLOW_COLOR`	`rgba(255, 0, 0, 0.6)`	Hover glow effect
`DFD_STYLING.CREATION.GLOW_COLOR`	`rgba(0, 150, 255, 0.9)`	Creation highlight (blue)
`DFD_STYLING.CREATION.FADE_DURATION_MS`	`500`	Fade animation duration
`DFD_STYLING.SELECTION.GLOW_BLUR_RADIUS`	`8`	Selection blur radius

Implementation Priority

Phase	Week	Focus
1	Week 1	Selection styling persistence tests, visual effects state management, basic Playwright infrastructure
2	Week 2	Node/edge creation workflows, history system integration, port management and connections
3	Week 3	Complete user workflows, performance testing, browser-specific behaviors
4	Week 4	Multi-user collaboration, export functionality, cross-browser compatibility

Success Criteria

Selection styling persistence eliminated - no selection artifacts after undo
Visual effects state management - clean state transitions
History system integrity - only structural changes in history
Complete workflow testing - end-to-end diagram creation
Performance validation - large diagrams handle smoothly
Real DOM verification - actual SVG and CSS inspection

For the complete test plan with all test scenarios and implementation details, see `docs/migrated/agent/dfd-integration-test-plan.md`.

WebSocket Testing

Manual WebSocket Testing

TMI provides a WebSocket test harness for manual testing:

# Run 3-terminal test (alice as host, bob and charlie as participants).
# This builds the harness first; there is no separate build target.
make wstest

# Run monitor mode
make monitor-wstest

# Clean up
make clean-process

Automated WebSocket Testing

Test File: postman/collaboration-tests-collection.json

Tests WebSocket functionality:

Session creation and joining
Diagram operations broadcast
Presenter mode
Cursor sharing
User join/leave events

CATS Security Fuzzing

CATS (Contract-driven Automatic Testing Suite) is a security fuzzing tool that tests API endpoints for vulnerabilities and spec compliance.

What is CATS

CATS automatically generates, runs, and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.

Features:

Boundary testing (very long strings, large numbers)
Type confusion testing
Required field validation
Authentication bypass testing
Malformed input handling

Running CATS

Fuzzing runs through the portable cats plugin (cats@efitz-skills), which is installed as a Claude Code plugin and also wrapped by the Makefile. Both routes run the same engine:

From Claude Code: /cats:run (campaign), /cats:report (results schema and ad-hoc queries), /cats:analyze (triage findings), /cats:fp (false-positive rules), /cats:init (bootstrap a new repo).
From a shell: the make targets below.

The Makefile resolves CATS_TOOL to the installed plugin first and falls back to a ~/Projects/skills/cats development checkout — the same copy the skills reach through ${CLAUDE_PLUGIN_ROOT} — so make cats-fuzz and /cats:run cannot diverge. Override with make ... CATS_TOOL=/path/to/cats_tool.py. Don't hardcode either path; two copies of the run-validity gates disagreeing is exactly the failure those gates exist to catch.

The Makefile wraps the plugin's subcommands:

Target	Purpose
`make cats-seed`	Seed test data (calls `scripts/run-dbtool.py` → `tmi-dbtool --import-test-data`)
`make cats-seed-oci`	Seed against Oracle ADB (requires `scripts/oci-env.sh` sourced)
`make e2e-seed`	Seed E2E test data from the tmi-ux seed spec
`make cats-fuzz`	Seed (via the plugin's `seed` hook), fuzz, parse, and classify results. Accepts `ENDPOINT=/path` and `BLACKBOX=true`
`make cats-fuzz-oci`	Same, seeded against Oracle ADB first
`make query-cats-results`	Query the most recent results database
`make analyze-cats-results`	Alias for `query-cats-results`
`make cats-report`	Render and open a self-contained HTML report from the latest run

# Full fuzzing pipeline: seed, fuzz, parse, classify
make cats-fuzz

# Fuzz a single endpoint
make cats-fuzz ENDPOINT=/addons

# Analyze the most recent run
make analyze-cats-results

CATS_USER, CATS_SERVER, and CATS_PROVIDER only steer seeding (cats-seed/cats-seed-oci/e2e-seed). They have no effect on cats-fuzz and there is no FUZZ_USER/FUZZ_SERVER equivalent. Fuzzing identity and target server come entirely from .local/cats/config.yaml (gitignored, per-repo). To fuzz as a different identity, add a named entry under identities: in that file — its token_cmd must print a bearer token on stdout — then either:

run /cats:run --identity alice from Claude Code, or
make it the default identity in .local/cats/config.yaml and use make cats-fuzz.

There is no make variable for this; --identity is a plugin flag, and the make targets don't forward it.

Configuration (`.local/cats/config.yaml`)

The plugin is configured per-repo by a gitignored file discovered by walking up from the working directory. Key settings:

Key	Purpose
`server`	Target server URL — must be the cluster NodePort, not a `kubectl port-forward` (see Fuzzing Topology below)
`results_dir`	Where run databases and reports land (`test/results/cats`)
`false_positives`	Path to the false-positive rule set (`test/cats/false-positives.yaml`)
`retain_raw_report`	Keep the raw CATS report after parsing (default `false` — see Results & Database)
`allow_suppressing_5xx`	Must stay `false`; guards the Zero-500 policy by refusing any FP rule that would suppress a 5xx
`allow_port_forward`	Escape hatch permitting a run through a detected `kubectl port-forward` (normally a fatal preflight error)
`keep_runs`	How many per-run databases to retain (default `5`, `0` disables). Pruning removes each dropped run's `report-<run_id>` companion artifacts too, and only ever runs after a valid campaign — so a bad run can't delete good history. Whatever `latest.db` points at is always protected, including when the symlink is dangling.
`max_connection_error_pct`	Connection-error threshold (default `1.0`%) above which a completed run is marked invalid — see Run Validity
`max_unauthenticated_pct`	Non-false-positive 401 threshold (default `5.0`%) above which a completed run is marked invalid — see Run Validity
`identities`	Map of named identities, each with a `token_cmd` that prints a bearer token on stdout
`hooks`	Shell commands for `pre_run`, `seed`, `post_run` pipeline stages, run in that order
`cats.*`	Passed to the CATS binary: `max_requests_per_minute`, `skip_fuzzers`, `skip_fuzzers_for_extension`, `skip_paths`, `headers`, `ref_data`, `skip_field_format`, `skip_field`, `extra_args`

cats.headers must include If-Match: "*" so CATS satisfies optimistic-locking preconditions on mutating requests; without it, most PUT/PATCH/DELETE calls return spurious 400s (#581). It used to ride in cats.extra_args as a raw -H; the first-class key (#599) merges it into the same generated headers file as the bearer token and refuses to let a header name collide with the auth header.

cats.skip_paths excludes paths from the campaign entirely. Reserve it for endpoints that destroy the campaign's own ability to keep testing — TMI skips /me/logout, which blacklists whatever bearer token is in the Authorization header (see Run Validity). Coverage is not lost: an explicit --path overrides the skip list, so such an endpoint can be fuzzed in its own short campaign where losing the token at the end costs nothing.

make cats-fuzz ENDPOINT=/me/logout    # or: /cats:run --path /me/logout

Seeding runs over loopback, not the NodePort. hooks.seed targets http://localhost:8080 behind a kubectl port-forward, unlike the campaign itself. tmi-dbtool is a freshly built, unsigned Go binary, and macOS local-network restrictions can block such a binary from opening any TCP connection to a LAN host — every dial to the cluster node times out while curl, nc and the Homebrew-installed cats reach the same host and port instantly (#595). Loopback is exempt. Seeding is low volume, so the throughput limits that make a port-forward unacceptable for fuzzing do not apply. Both forwards must be up:

kubectl --context k3s-rp -n tmi-platform port-forward svc/tmi-server 8080:8080
kubectl --context k3s-rp -n tmi-platform port-forward svc/postgres 5432:5432

Fuzzing Topology

CATS drives a sustained request rate (max_requests_per_minute, default 3000 ≈ 50 req/s) against the dev server. Fuzz the dev cluster's NodePort directly (e.g. http://<cluster-node>:30080 — the base server.yml manifest exposes the tmi-server Service on NodePort 30080), never through kubectl port-forward. A userspace port-forward silently drops connections under CATS's rate — routing a full campaign through one lost ~46% of requests to connection errors in past runs (#463, #578), and those losses get absorbed by the CONNECTION_ERROR_999 false-positive rule, so an under-fuzzed run can look clean at a glance.

This is now enforced, not just documented:

Preflight: the plugin aborts before running if it detects the target is a kubectl port-forward, unless allow_port_forward: true in the config or --allow-port-forward is passed on the command line.
Post-run: after a campaign completes, if the connection-error rate (response codes 953/999) exceeds max_connection_error_pct (default 1%), the run is marked INVALID, the tool exits with code 3, and latest.db is not repointed at the new run — so a bad run can't silently become the source of truth for the next analysis.

Run Validity

Before drawing any conclusion from a run, check two things. Both are now enforced automatically — a run failing either exits 3 and never becomes latest.db — but the queries are here because an explicitly-named older database (--db <file>) bypasses the gate:

Connection-error rate (response codes 953/999). Enforced automatically per Fuzzing Topology above, but worth checking directly:

SELECT response_code, COUNT(*) FROM tests t
JOIN responses r ON r.test_id = t.id
WHERE r.response_code IN (953, 999)
GROUP BY response_code;

Non-false-positive 401 rate. Only the BypassAuthentication fuzzer is expected to produce 401/403 — it's the sole fuzzer that deliberately strips the Authorization header, and the AUTH_BYPASS_401_403 false-positive rule accounts for exactly that case (#583). Any other fuzzer returning 401 means the request's auth token stopped working. A high sustained non-false-positive 401 rate across many fuzzers means the campaign lost its authentication partway through (e.g. an expiring token, or the fuzzed identity being mutated by another test earlier in the same run — see #591) — its results are not trustworthy and should not be used to draw conclusions about the API. Check with:
```
SELECT t.is_false_positive, COUNT(*) FROM tests t
JOIN responses r ON r.test_id = t.id
WHERE r.response_code = 401
GROUP BY t.is_false_positive;
```
If a large share of 401s have is_false_positive = 0, treat the run as invalid regardless of what the connection-error check says, and re-run before analyzing. Enforced by max_unauthenticated_pct (default 5%); a healthy TMI campaign reads well under 1%.

The usual cause is the campaign logging itself out. In run 20260727T204514Z, test 25485 was POST /me/logout, it returned 204, and every one of the 52,340 authenticated requests after it got a 401 — while the run reported as complete and became latest.db. MeLogout takes the bearer token straight from the Authorization header and blacklists it, which is correct server behaviour; the campaign was simply asking the server to revoke its own credential. Any endpoint that can revoke the caller's token belongs in cats.skip_paths. To find the culprit in a contaminated run, look for the cliff (where the 401 rate jumps) rather than the first 401 — isolated early 401s from BypassAuthentication are normal — then read the successful mutations immediately before it.

Public and Cacheable Endpoint Handling

TMI marks certain operations with OpenAPI vendor extensions so CATS skips fuzzers that would otherwise flag intended behavior as a finding:

Extension	Fuzzer Skipped	Reason
`x-public-endpoint: true` (21 operations)	`BypassAuthentication`	OAuth, OIDC, and SAML endpoints intentionally accessible without authentication per RFC (all other security fuzzers still run)
`x-cacheable-endpoint: true` (7 operations)	`CheckSecurityHeaders`	Discovery endpoints intentionally cacheable (`Cache-Control: public, max-age=...`) per RFC 8414/7517/9728
`x-skip-deleted-resource-check: true`	`CheckDeletedResourcesNotAvailable`	e.g. `/me` — users can't delete themselves, so the "deleted resource" precondition can't be set up
`x-skip-idor-check: true`	`InsecureDirectObjectReferences`	e.g. `/oauth2/revoke` — valid per RFC 7009

Filter parameters (like threat_model_id, addon_id) returning empty results for non-matching values are not IDOR vulnerabilities — they narrow results, not authorize access; the false-positive rule set marks these accordingly.

Results & Database

Results land in test/results/cats/:

cats-results-<run_id>.db — one SQLite database per run
latest.db — symlink to the most recently valid completed run (see Run Validity)
report-<run_id>.html — self-contained HTML report (make cats-report)
cats-test-data.json / cats-test-data.yml — seeded entity IDs and CATS --refData reference file

The raw CATS report directory is deleted after a successful parse by default (retain_raw_report: false) since it's large and redundant once results are in SQLite.

Perform all analysis by querying the SQLite database — don't read the HTML or JSON report files directly.

The tests table carries is_false_positive (boolean) and fp_rule (the matching rule id, or null). Useful views:

View	Purpose
`test_results_view`	All tests, with false-positive flag
`test_results_filtered_view`	Excludes false positives (recommended for triage)
`true_positives_view`	Non-false-positive findings only
`fp_rule_stats_view`	Counts and percentages per false-positive rule
`fuzzer_stats_view`	Counts per fuzzer × result type
`path_error_analysis_view`	Errors grouped by API path
`response_code_stats_view`	Distribution of HTTP response codes

-- Query actual errors (excluding false positives)
SELECT * FROM test_results_filtered_view WHERE result = 'error';

-- View false positives with their matching rule
SELECT fp_rule, COUNT(*) FROM tests WHERE is_false_positive = 1 GROUP BY fp_rule;

Analyzing CATS Results

When reviewing CATS results, categorize each finding as:

Category	Description	Action
Should Fix (High)	Security vulnerability	Fix immediately
Should Fix (Medium)	API contract violation	Fix in next sprint
Should Fix (Low)	Minor compliance issue	Add to backlog
Should Investigate	Unclear behavior	Review with team
False Positive	Expected/correct behavior	Mark as ignored
Should Ignore	By design or not applicable	Document reason

Example Classifications:

False Positive: Server returns 200 for GET / without authorization — this endpoint is intentionally public (security: [])
Should Investigate: Unexpected Accept-Language header handling — needs design decision
Should Fix (Low): Server returns 400 instead of 405 for unsupported HTTP method
Should Fix (Medium): Response Content-Type doesn't match OpenAPI schema

For every true-positive 500, follow the Zero 500-Error Policy (file a bug-labeled issue, prioritize for the current milestone). For any undocumented status code (including 4xx not listed for that operation), follow the Documented-Status-Code Policy — either add the code to the OpenAPI spec or change the handler.

CATS False Positives

CATS can flag legitimate API responses as "errors" due to expected behavior patterns. These are not security vulnerabilities — they are correct, RFC-compliant responses or intended API behavior.

False positives are classified declaratively in test/cats/false-positives.yaml (48 rules as of 2026-07-28, down from 65: six over-broad rules were tightened to match the server's actual rejection text and 17 provably-dead ones removed), evaluated in file order — first match wins. Rule order is load-bearing; do not reorder without re-verifying against the documented baseline. CONNECTION_ERROR_999 is rule 1, so a transport-layer failure (response code 953/999) can never be claimed by an application-layer rule. allow_suppressing_5xx: false in the plugin config guards the Zero-500 policy by refusing any rule that would suppress a 5xx.

Manage rules through the /cats:fp skill (add/review/reclassify workflows) — the legacy Python detect_false_positive() implementation these rules were ported from no longer exists.

test/cats/rule-baseline.json is the golden per-rule baseline, carrying both matched (records a rule matches in isolation) and fired (records where it is the first match under first-match-wins) counts. Regenerate it after any deliberate rule change. Take the baseline from a corpus that passes both validity gates — the pre-2026-07-28 baseline came from the June legacy corpus, which was later found to have lost its bearer token ~21% of the way in, so its 52,804 OAUTH_AUTH_401_403 count was the contamination rather than genuine auth coverage.

Quick Reference (illustrative, not exhaustive — see the rule file for the full set):

Scenario	Is False Positive?	Reason
401/403 from the `BypassAuthentication` fuzzer	Yes	`AUTH_BYPASS_401_403` — that fuzzer deliberately strips the Authorization header; rejection is the correct security outcome
401/403 from any other fuzzer	No	Every other fuzzer preserves auth headers; a 401/403 here means something is actually broken, or the run's identity failed mid-campaign (see Run Validity)
401 `insufficient_user_authentication` on step-up-gated operations	Yes	`STEP_UP_AUTH_401` — sensitive operations require recent re-authentication that a seeded token can't satisfy
403 on `/automation/*` or admin client-credentials endpoints for a non-automation identity	Yes	`AUTOMATION_ONLY_ENDPOINT_403` / `ADMIN_CLIENT_CREDENTIALS_AUTOMATION_ONLY_403` — those endpoints are automation-account-only by design
409 on duplicate-name creation from fuzzed values	Yes	`CONFLICT_409` — correct REST semantics for duplicate resources
400 from header fuzzers	Yes	`HEADER_VALIDATION_400` — correct rejection of malformed headers
429 rate limit	Yes	`RATE_LIMIT_429` — infrastructure protection, not API behavior
404 from boundary fuzzers using invalid IDs	Yes	`NOT_FOUND_404` — expected with random/invalid resource IDs
"NoSQL injection" findings (e.g. `$where` payloads) on 2xx	Yes	TMI uses PostgreSQL; NoSQL operators have no effect and are stored as literal strings
XSS findings on GET query parameters	Yes	`XSS_QUERY_PARAMS` — TMI is a JSON API; responses don't render HTML
500 with a stack trace or similar	No	Actual server error — see the Zero 500-Error Policy

Template and Expression Injection Protection

TMI validates string fields for template/expression injection patterns as defense-in-depth (api/html_injection_checker.go):

Pattern	Description	Example
`{{` / `}}`	Handlebars, Jinja2, Angular, Go templates	`{{constructor.constructor('alert(1)')()}}`
`${`	JavaScript template literals, Freemarker	`${alert(1)}`
`<%` / `%>`	JSP, ASP, ERB server templates	`<%=System.getProperty('user.home')%>`
`#{`	Spring EL, JSF EL expressions	`#{T(java.lang.Runtime).exec('calc')}`
`${{`	GitHub Actions context injection	`${{github.event.issue.title}}`

NoSQL syntax is allowed since it's harmless in a SQL (PostgreSQL) context.

Fuzzer Coverage

The set of fuzzers CATS runs, and which ones TMI deliberately skips, is driven entirely by .local/cats/config.yaml's cats.* block. Fuzzer counts (registered vs. running) vary by installed CATS version and are not worth hardcoding here — query the results database for current numbers:

# List all fuzzers that appeared in the most recent run
sqlite3 test/results/cats/latest.db "SELECT DISTINCT name FROM fuzzers ORDER BY name"

# Test counts for specific fuzzers
sqlite3 test/results/cats/latest.db "
SELECT f.name, COUNT(*) as tests
FROM tests t JOIN fuzzers f ON t.fuzzer_id = f.id
WHERE f.name IN ('SSRFInUrlFields','MassAssignment','SqlInjectionInStringFields')
GROUP BY f.name"

Skipped Fuzzers (Documented False Positives)

These fuzzers are explicitly skipped via cats.skip_fuzzers because TMI correctly rejects the malformed input (returns 400), but CATS expects 2XX responses:

Fuzzer	Reason Skipped
`DuplicateHeaders`	TMI ignores duplicate headers (valid per HTTP spec)
`LargeNumberOfRandomAlphanumericHeaders`	TMI ignores extra headers (valid behavior)
`EnumCaseVariantFields`	TMI uses case-sensitive enum validation (stricter is valid)
`BidirectionalOverrideFields`	Unicode BiDi chars correctly rejected (security protection)
`ResponseHeadersMatchContractHeaders`	Flags missing optional headers as errors
`PrefixNumbersWithZeroFields`	Leading zeros in JSON numbers correctly rejected (RFC 8259)
`ZalgoTextInFields`	Exotic Unicode correctly handled
`HangulFillerFields`	Korean filler characters correctly handled
`AbugidasInStringFields`	Indic script characters correctly handled
`FullwidthBracketsFields`	CJK brackets correctly handled
`ZeroWidthCharsInValuesFields`	Zero-width characters correctly handled

Conditionally Skipped Fuzzers (Via Vendor Extensions)

Extension	Fuzzer Skipped	Endpoints	Reason
`x-public-endpoint=true`	`BypassAuthentication`	21 public endpoints	RFC-compliant public access
`x-cacheable-endpoint=true`	`CheckSecurityHeaders`	7 discovery endpoints	RFC 8414/7517/9728 caching
`x-skip-deleted-resource-check=true`	`CheckDeletedResourcesNotAvailable`	e.g. `/me`	Users can't delete themselves
`x-skip-idor-check=true`	`InsecureDirectObjectReferences`	e.g. `/oauth2/revoke`	Valid per RFC 7009

Optional Fuzzer Categories (Not Enabled)

CATS supports optional fuzzer categories that are not enabled for TMI:

Category	Flag	Why Not Enabled
Emoji	`--includeEmojis`	High false positive risk; JSON APIs handle emojis correctly
Control Chars	`--includeControlChars`	Already skip similar fuzzers; TMI rejects control chars correctly
Whitespace	`--includeWhitespaces`	Already skip ZeroWidthChars; high false positive risk

These categories generate false positives because TMI's input validation correctly rejects malformed Unicode (returns 400), but CATS expects 2XX responses.

CATS Test Data Setup

CATS fuzzing can report false positives when testing endpoints that require prerequisite objects (e.g., testing GET /threat_models/{id}/threats/{threat_id} fails with 404 when no threat model exists). TMI addresses this by pre-creating a complete object hierarchy before fuzzing.

Test data is defined declaratively in test/seeds/cats-seed-data.json (a JSON seed file) rather than being hardcoded in Go source. tmi-dbtool processes this file in import-test-data mode to create all required entities. See Database-Tool-Reference for full documentation of the tool and seed file format.

Object Hierarchy

threat_model (root)
├── threats
│   └── metadata
├── diagrams
│   └── metadata
├── documents
│   └── metadata
├── assets
│   └── metadata
├── notes
│   └── metadata
├── repositories
│   └── metadata
└── metadata

addons (independent root)
└── invocations

webhooks (independent root)
└── deliveries

client_credentials (independent root)

Creating Test Data

# Full fuzzing (seeds automatically via the plugin's seed hook)
make cats-fuzz

# Or seed test data separately
make cats-seed

make cats-seed runs scripts/run-dbtool.py, which builds tmi-dbtool and runs it in import-test-data mode against the CATS seed file. It authenticates as the configured user through the OAuth callback stub — the stub must already be running (make start-oauth-stub); seeding does not start it for you. It then:

Authenticates via OAuth as the configured user (--user/--provider/--server)
Creates one of each object type via the TMI API
Generates a JSON reference file (test/results/cats/cats-test-data.json)
Generates YAML reference data for CATS (test/results/cats/cats-test-data.yml)

CATS Reference Data Format

CATS uses the --refData parameter to substitute path parameters with real IDs. The YAML file uses the all: key for global substitution:

# CATS Reference Data - Path-based format for parameter replacement
all:
  id: <threat_model_uuid>
  threat_model_id: <threat_model_uuid>
  threat_id: <threat_uuid>
  diagram_id: <diagram_uuid>
  document_id: <document_uuid>
  asset_id: <asset_uuid>
  note_id: <note_uuid>
  team_note_id: <team_note_uuid>
  project_note_id: <project_note_uuid>
  feedback_id: <feedback_uuid>
  triage_note_id: <triage_note_uuid>
  repository_id: <repository_uuid>
  webhook_id: <webhook_uuid>
  addon_id: <addon_uuid>
  client_credential_id: <credential_uuid>
  key: cats-test-key
  # Admin resource identifiers
  group_id: <group_uuid>
  internal_uuid: <user_internal_uuid>

The file also carries per-path sections, which override all: for a single contract path. TMI needs them because /admin/groups/{internal_uuid} and /admin/users/{internal_uuid} share a parameter name across two unrelated resource types, and refData is keyed by name — so a single global value can only ever satisfy one family, leaving the other permanently 404ing (#603). The global value points at a user; per-path sections point the group routes back at a seeded group:

/admin/groups/{internal_uuid}:
  internal_uuid: <group_uuid>
/admin/groups/{internal_uuid}/members/{member_uuid}:
  internal_uuid: <group_uuid>
  member_uuid: <member_user_uuid>

This is a workaround. The parameter name describes the storage shape rather than the resource it identifies, which is what makes the collision possible; renaming to {group_id}/{user_id} in the spec is the real fix.

Removing fields CATS cannot generate correctly

The magic value cats_remove_field deletes a body property from every generated payload. TMI uses it for four arrays, because CATS 13.8.0 cannot produce a valid item for any of them and so made every POST/PUT to /projects and /teams return 400 on every single test:

Removed	Why
`related_projects`, `related_teams`	CATS drops a nested property whose name shares an underscore-delimited token with an ancestor property's name — a self-reference guard matching on name similarity rather than actual schema recursion. `related_projects[].related_project_id` collides with the array containing it and is dropped; `responsible_parties[].user_id` does not collide and survives. Renaming the parent array to `linked_projects` makes the field appear. Since `related_project_id` is required, every generated item failed validation (#596).
`responsible_parties`, `members`	CATS ignores `readOnly: true` on request-body properties and emits the server-populated `user` object, which the OpenAPI validation middleware correctly rejects (#604).

Note cats_remove_field reaches top-level body properties only — it cannot remove a nested field, which is why the whole array goes rather than just the offending property. --skipFields is not an alternative: it means "do not fuzz this field", not "do not send it".

Both are upstream CATS defects, so the related_team_id/related_project_id entries and the corresponding spec example values are deliberately left in place — they start working the moment CATS is fixed.

For complete reference data format documentation, see CATS Reference Data File.

Documentation

Coverage Reporting

Server Coverage

The TMI server provides comprehensive test coverage reporting with both unit and integration test coverage.

Quick Start

# Generate full coverage report (unit + integration + merge + reports)
make test-coverage

# Run only unit tests with coverage
make test-coverage-unit

# Run only integration tests with coverage
make test-coverage-integration

# Generate reports from existing profiles
make generate-coverage

Output Files

Coverage Directory (coverage/):

unit_coverage.out - Raw unit test coverage data
integration_coverage.out - Raw integration test coverage data
combined_coverage.out - Merged coverage data
unit_coverage_detailed.txt - Detailed unit test coverage by function
integration_coverage_detailed.txt - Detailed integration test coverage
combined_coverage_detailed.txt - Detailed combined coverage
coverage_summary.txt - Executive summary with key metrics

HTML Reports Directory (coverage_html/):

unit_coverage.html - Interactive unit test coverage report
integration_coverage.html - Interactive integration test coverage report
combined_coverage.html - Interactive combined coverage report

View HTML Report:

open coverage_html/combined_coverage.html

Coverage Goals

Unit Tests: Target 80%+ coverage for core business logic
Integration Tests: Target 70%+ coverage for API endpoints and workflows
Combined: Target 85%+ overall coverage

Key Areas of Focus

High priority areas for coverage:

API Handlers - All HTTP endpoints should be tested
Business Logic - Core threat modeling functionality
Authentication & Authorization - Security-critical code
Database Operations - Data persistence and retrieval
Cache Management - Performance-critical caching logic

Prerequisites

Go 1.26 or later
Docker (for integration tests with PostgreSQL and Redis)
gocovmerge tool (automatically installed if missing)

Test Database Configuration

Coverage integration tests use the same test infrastructure as integration tests:

PostgreSQL: localhost:5433 (container: tmi-postgresql-test)
Redis: localhost:6380 (container: tmi-redis-test)

These ports avoid conflicts with development databases (5432, 6379).

Troubleshooting

Docker Not Available:

# Start Docker on macOS
open -a Docker

# Verify Docker is running
docker info

Database Connection Issues:

# Clean up any existing containers
make clean-everything

# Or manually clean up
docker stop tmi-postgresql-test tmi-redis-test 2>/dev/null
docker rm tmi-postgresql-test tmi-redis-test 2>/dev/null

Coverage Tool Missing:

go install github.com/wadey/gocovmerge@latest

Advanced Usage

Custom Coverage Profiles:

# Test specific packages
go test -coverprofile=custom.out ./api/...

# Test with race detection
go test -race -coverprofile=race.out ./...

# Generate HTML from custom profile
go tool cover -html=custom.out -o custom.html

Coverage Analysis:

# Find functions with zero coverage
go tool cover -func=coverage/combined_coverage.out | awk '$3 == "0.0%" {print $1}'

# Show files sorted by coverage
go tool cover -func=coverage/combined_coverage.out | sort -k3 -n

Web App Coverage

# Generate coverage report
pnpm run test:coverage

# View report
open coverage/index.html

Coverage Configuration: vitest.config.ts

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8',
      reporter: ['text', 'html'],
      exclude: [
        'node_modules/',
        'src/test-setup.ts',
        '**/*.d.ts',
        '**/*.spec.ts',
        '**/environments/**',
        'unused/**'
      ]
    }
  }
});

Testing Best Practices

1. Test Organization

One test file per source file
Group related tests with describe blocks
Use clear, descriptive test names
Follow AAA pattern: Arrange, Act, Assert

2. Test Data

Use factories for test data
Create minimal test data
Clean up after tests
Use predictable test users (login hints)

3. Isolation

Tests should be independent
Don't rely on test order
Clean up between tests
Mock external dependencies

4. Assertions

Test one thing per test
Use specific assertions
Test both happy path and error cases
Verify side effects

5. Performance

Keep unit tests fast (<1s each)
Use before/after hooks efficiently
Parallelize tests when possible
Cache test fixtures

6. Maintainability

DRY - Don't Repeat Yourself
Use helper functions
Keep tests simple
Update tests with code

Continuous Integration

GitHub Actions

The TMI server has two CI workflows:

Security & Quality (.github/workflows/security.yml) - Runs on pushes and PRs to main:
- Lint & Static Analysis - golangci-lint with gosec
- Build Verification - Ensures the server compiles
- Unit Tests - Runs go test -v -race -short ./...
- OpenAPI Validation - JSON syntax check and vacuum linting with OWASP rules
- Dependency Review - Checks for high-severity vulnerabilities on PRs
CodeQL (.github/workflows/codeql.yml) - Deep semantic code analysis:
- Runs on pushes and PRs to main
- Scheduled weekly (Sunday at midnight UTC)
- Analyzes Go code for security vulnerabilities

Security & Quality Workflow (.github/workflows/security.yml):

name: "Security & Quality"

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version-file: go.mod
      - uses: golangci/golangci-lint-action@v9

  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version-file: go.mod
      - run: go build -o bin/tmiserver ./cmd/server

  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version-file: go.mod
      - run: go test -v -race -short ./...

  openapi-validation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - run: jq empty api-schema/tmi-openapi.json
      - run: vacuum lint -r vacuum-ruleset.yaml api-schema/tmi-openapi.json

  dependency-review:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
      - uses: actions/checkout@v6
      - uses: actions/dependency-review-action@v4
        with:
          fail-on-severity: high

Troubleshooting Tests

Integration Tests Fail

# Clean everything and retry
make clean-everything
make test-integration

# Check container logs
docker logs tmi-postgresql-test
docker logs tmi-redis-test

# Verify ports are free
lsof -ti :5433  # PostgreSQL
lsof -ti :6380  # Redis

API Tests Fail

# Check server is running
curl http://localhost:8080/

# Check authentication
curl -H "Authorization: Bearer TOKEN" http://localhost:8080/threat_models

# Run specific collection
newman run test/postman/comprehensive-test-collection.json

E2E Tests Fail

# Run in headed mode to see what's happening
pnpm run test:e2e:headed

# Run with debug mode
pnpm run test:e2e:debug

# Run with Playwright UI for interactive debugging
pnpm run test:e2e:ui

TMI-UX Testing Utilities

TMI-UX provides standardized testing utilities in the src/testing/ directory to make testing easier, more consistent, and more maintainable.

TMI-UX uses Vitest with promise-based async patterns (no done() callbacks). This section documents established patterns and utilities for service unit testing.

Service Test Coverage

Metric	Value
Total Services	72
Services with Tests	66 (91.7%)
Services Needing Tests	6 (8.3%)

Services still requiring tests:

client-credential.service.ts - Client credential management
user-preferences.service.ts - User preferences storage
threat-model-report.service.ts - Report generation
import-orchestrator.service.ts - Multi-step import workflow (high complexity)
app-websocket-event-processor.service.ts - WebSocket event processing
ui-presenter-cursor-display.service.ts - Cursor rendering

Unit tests have been implemented across the following categories:

Core Services (src/app/core/services/):

Dialog direction, theme, operator, server connection
Addon, administrator, quota, webhook management
Collaboration session and DFD collaboration state

TM Services (src/app/pages/tm/services/):

Authorization checking and role management
Import utilities: ID translation, field filtering, reference rewriting
Provider adapters and authorization preparation

DFD Application Services (src/app/pages/dfd/application/services/):

Diagram loading, state management, history (undo/redo)
Export and SVG optimization
Operation state management and broadcasting
Event handling and rejection recovery

DFD Presentation Services (src/app/pages/dfd/presentation/services/):

Tooltip content and positioning
Presenter coordination with WebSocket
Cursor tracking and selection broadcasting

Shared Services (src/app/shared/services/):

Notification with spam prevention
Form validation with multiple validators
Framework JSON loading
Cell data extraction from X6 and threat models

I18N Services (src/app/i18n/):

Language switching and direction management
Translation file loading via HTTP

Key Testing Patterns

Mock Setup

Use typed mocks with ReturnType<typeof vi.fn>
Cast to service types with as unknown as Type
For properties (not methods), include them directly in mock object

let mockAuthService: {
  userEmail: string;
  isAuthenticated: ReturnType<typeof vi.fn>;
};

beforeEach(() => {
  mockAuthService = {
    userEmail: 'test@example.com',  // property, not vi.fn()
    isAuthenticated: vi.fn().mockReturnValue(true)
  };
});

Timer Testing

vi.useFakeTimers() in beforeEach
vi.useRealTimers() in afterEach
vi.advanceTimersByTimeAsync(1) to trigger scheduled operations
Avoid vi.runAllTimersAsync() (causes infinite loops with intervals)

Known Issues and Solutions

Issue	Problem	Solution
Timer Infinite Loops	`vi.runAllTimersAsync()` causes infinite recursion	Use `vi.advanceTimersByTimeAsync(milliseconds)`
Mock Property Access	Services accessing properties not methods	Include properties directly in mock object
Event Handler Types	ESLint complains about `Function` type	Use explicit signature: `(event: EventType) => void`

DFD Service Mock Factories

Shared mock factories in src/app/pages/dfd/application/services/test-helpers/mock-services.ts:

LoggerService, AppStateService, AppHistoryService
AppOperationStateManager, AppDiagramService
InfraNodeConfigurationService, InfraX6GraphAdapter
Graph (with batchUpdate), DfdCollaborationService
InfraDfdWebsocketAdapter, AppDiagramResyncService

Next Steps

Contributing -- Learn the contribution workflow
Getting-Started-with-Development -- Set up your development environment
API-Integration -- Learn API patterns

Home

Releases

Getting Started

Deployment

Planning Your Deployment
Terraform Deployment (AWS, OCI, GCP, Azure)
Deploying TMI Server
OCI Container Deployment
Certificate Automation
Deploying TMI Web Application
Setting Up Authentication
Database Setup
Bootstrapping Production
Component Integration
Post-Deployment
Branding and Customization

Testing

Testing

Table of Contents

Testing Philosophy

Test Pyramid

Testing Principles

Unit Testing

Server Unit Tests (Go)

Running Unit Tests

Writing Unit Tests

Test Patterns

Web App Unit Tests (Angular/TypeScript)

Running Unit Tests

Writing Unit Tests

DFD Service Integration Testing (Vitest)

Integration Testing

Server Integration Tests (Go)

Running Integration Tests

Test Configuration

Writing Integration Tests

Test Data Management

Test Patterns

OpenAPI-Driven Integration Test Framework

Framework Structure

Writing Framework Tests

Test Coverage Plan

API Testing

Running API Tests

Test Collections

Test Coverage

Threat Model API Coverage Analysis

Writing Postman Tests

End-to-End Testing

Running E2E Tests

Playwright Configuration

Writing E2E Tests

DFD Component Integration Test Plan

Testing Philosophy

Critical Issues Priority

Proposed Test Structure

Critical Test: Selection Styling Persistence

Styling Constants Reference

Implementation Priority

Success Criteria

WebSocket Testing

Manual WebSocket Testing

Automated WebSocket Testing

CATS Security Fuzzing

What is CATS

Running CATS

Configuration (.local/cats/config.yaml)

Fuzzing Topology

Run Validity

Public and Cacheable Endpoint Handling

Results & Database

Analyzing CATS Results

CATS False Positives

Template and Expression Injection Protection

Fuzzer Coverage

Skipped Fuzzers (Documented False Positives)

Conditionally Skipped Fuzzers (Via Vendor Extensions)

Optional Fuzzer Categories (Not Enabled)

CATS Test Data Setup

Object Hierarchy

Creating Test Data

CATS Reference Data Format

Removing fields CATS cannot generate correctly

Documentation

Coverage Reporting

Server Coverage

Quick Start

Output Files

Coverage Goals

Key Areas of Focus

Prerequisites

Test Database Configuration

Troubleshooting

Advanced Usage

Web App Coverage

Testing Best Practices

Configuration (`.local/cats/config.yaml`)