feat: import GCP Cloud Logging into a workspace - #35
Merged
Conversation
New workspace import gcp command pulls a query plus time range through ADC and lands enveloped NDJSON plus an import-runs provenance record, per ADR 0004. Validation errors leave no record; credential failures leave a FAILED record. Conversion is a pure function with fixture pairs including sanitized real GCP entries.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 661b50f. Configure here.
| if err != nil { | ||
| return FetchResult{}, err | ||
| } | ||
| result.Lines = append(result.Lines, line) |
There was a problem hiding this comment.
Limit keeps oldest log entries
High Severity
FetchLines lists Cloud Logging entries without requesting newest-first order, then stops after --limit rows from the iterator. The Logging API returns entries in ascending timestamp order by default, so a capped import keeps the earliest part of the window—not the recent logs typical for --since incident pulls.
Reviewed by Cursor Bugbot for commit 661b50f. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Problem
Logs had to already exist as local files before LAPP could help: to
investigate an incident on GCP you had to remember the right gcloud
invocation, pipe its output around, and lose track of what query produced
which file. ADR 0004 decided imports should be recorded snapshots.
Solution
workspace import gcppulls a query plus time range from GCP Cloud Loggingthrough the machine's ADC credentials, lands it as one enveloped NDJSON file
in
logs/, and records provenance underimport-runs/<run-id>/record.json.The imported file flows through the existing discovery pipeline like any
other log file; the command never starts discovery (ADR 0005).
Major Changes
ConvertEntrymapping a GCP entry to the ADR 0006 envelope(jsonPayload → payload as-is, textPayload → payload.message, missing
severity → DEFAULT, timestamps normalized to RFC3339 UTC)
audit, embedded text log) alongside synthetic edge cases
credential-failure hint pointing at gcloud auth application-default login
RunImportorchestration behind an injectable fetch seam: ImportRunrecord lifecycle (import-runs//record.json, symmetric to
discovery-runs), zero-entries success without a file, orchestration-level
limit cap, duplicate run id rejection, no stuck RUNNING records
workspace import gcp --topic --project [--filter] (--since | --from/--to) [--limit]; validation errors create no record, run failures leave aFAILED record; success output points at
workspace discoverNote
Medium Risk
New external GCP dependency and credential path; import writes workspace files and records but is isolated from discovery/analyze until the user runs discover.
Overview
Adds
workspace import gcpso logs can be pulled from GCP Cloud Logging (ADC) into a workspace as a single enveloped NDJSON file inlogs/, with query/time-range provenance inimport-runs/<run-id>/record.json. Import does not start discovery; users still runworkspace discoverafterward, same asadd-log.New
pkg/gcplogmaps GCP entries to the ADR 0006 envelope (ConvertEntry), builds time-bounded Cloud Logging filters, and fetches via logadmin with limit/truncation and ADC error hints.pkg/workspaceadds ImportRun persistence andRunImportorchestration behind an injectable fetcher (success with zero entries skips writing a file; failures persist FAILED records; duplicate run IDs rejected).workspace createnow createsimport-runs/; docs andgo.modadd GCP logging/API dependencies and notegcloud auth application-default login.Reviewed by Cursor Bugbot for commit 661b50f. Bugbot is set up for automated code reviews on this repo. Configure here.