Skip to content

Add the GCP import web API - #36

Merged
STRRL merged 1 commit into
masterfrom
conductor/lapp-gcp-import-web
Jul 26, 2026
Merged

Add the GCP import web API#36
STRRL merged 1 commit into
masterfrom
conductor/lapp-gcp-import-web

Conversation

@STRRL

@STRRL STRRL commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Problem

The web server has no API to start or inspect GCP log imports. Users can only run imports through the CLI.

Solution

Add asynchronous ImportRun APIs with saved progress, shared run exclusion, startup recovery, and recent query recall.

This PR covers the backend only. The import form, progress view, and history UI remain separate work.

Major Changes

  • Import run API
    • Add create, get, list, and recent query RPCs
    • Validate requests and expose saved structured state
  • Run safety
    • Share one workspace guard across imports, discovery, log changes, and workspace deletion
    • Recover interrupted queued and running imports
  • Coverage
    • Test validation, async success and failure, truncation, ordering, exclusion, recovery, and query recall

Note

Medium Risk
New async import path touches workspace mutation, GCP fetch injection, and shared run locking; misconfiguration or lock bugs could block workspaces or leave stale run state, though behavior is heavily tested.

Overview
Adds asynchronous GCP log import to the web backend so clients can start imports, poll status, and reuse recent query parameters without the CLI.

API & codegen: WorkspaceService gains CreateImportRun, GetImportRun, ListImportRuns, and ListRecentImportQueries, with new ImportRun / ImportRunState types wired through Go Connect and TypeScript protobuf stubs.

Server behavior: pkg/webapp/import_service.go validates import requests (project, time range, limit defaulting to 100k), writes a queued run record, and runs workspace.RunImport in a goroutine via an injected ImportFetcher. Imports share the existing per-workspace run lock with discovery and log/workspace mutations. On web startup, queued/running import runs are marked failed with IMPORT_INTERRUPTED (documented alongside discovery recovery in CLAUDE.md).

Tests: Broad coverage for validation, async success/failure/truncation, list ordering, mutual exclusion, startup recovery, and recent-query aggregation (capped at 20).

Reviewed by Cursor Bugbot for commit 48de5d3. Bugbot is set up for automated code reviews on this repo. Configure here.

@STRRL
STRRL marked this pull request as ready for review July 26, 2026 00:25
@STRRL
STRRL merged commit 7bf7ddf into master Jul 26, 2026
2 checks passed
@STRRL
STRRL deleted the conductor/lapp-gcp-import-web branch July 26, 2026 00:25

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 48de5d3. Configure here.

slog.Error("ImportRun failed", "workspace", id, "run", record.ID, "error", err)
return
}
}()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import run stuck queued after failure

Medium Severity

When CreateImportRun writes a QUEUED record, subsequent errors—either during importRunMessage or early failures within the RunImport goroutine (e.g., before fetch handling)—do not update the record to a terminal FAILED state. This leaves a stale QUEUED record visible to pollers, despite the workspace lock being released and the error being logged.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 48de5d3. Configure here.

records, err := workspace.ListImportRunRecords(s.workspaceDir(entry.Name()))
if err != nil {
return nil, connect.NewError(connect.CodeInternal, err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recent queries abort on bad record

Low Severity

ListRecentImportQueries walks every directory under the workspace root and returns an internal error if ListImportRunRecords fails for any one of them. A single workspace with a corrupt or unreadable import run record disables recent-query recall for all workspaces, not just the affected one.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 48de5d3. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant