Skip to content

feat: add structured logging fields (log_template, log_args) to GELF output#96

Open
soustruh wants to merge 1 commit into
mainfrom
devin/1779367613-structured-logging
Open

feat: add structured logging fields (log_template, log_args) to GELF output#96
soustruh wants to merge 1 commit into
mainfrom
devin/1779367613-structured-logging

Conversation

@soustruh
Copy link
Copy Markdown
Contributor

Summary

Adds a custom logging filter (StructuredLoggingFilter) that preserves Python's raw message template and arguments as separate GELF fields, enabling pattern-based filtering and aggregation in Datadog.

Why This Matters

Even with AI agents that can crawl unstructured logs, structured logging provides critical value:

  • Precision: Structured fields are deterministic and unambiguous vs semantic text parsing
  • Performance: Pattern matching on indexed fields is orders of magnitude faster than full-text search at scale
  • Aggregation: Enables reliable analytics like count by log_template or grouping by argument values
  • Agent Efficiency: Reduces token usage and enables agents to build reliable monitoring rules
  • Cross-System Consistency: Group same error patterns across distributed systems for root cause analysis

Changes

  • Added StructuredLoggingFilter class before CommonInterface to extract record.msg and record.args before interpolation
  • Modified set_gelf_logger to attach the filter to the GELF handler
  • New fields are included via existing include_extra_fields=True mechanism
  • Removed stale # noqa: E301 comment from set_gelf_logger signature

Example

logging.warning("The Answer to the Ultimate Question of Life, the Universe, and Everything: %i", 42)

Results in GELF output with:

  • short_message: "The Answer to the Ultimate Question of Life, the Universe, and Everything: 42"
  • log_template: "The Answer to the Ultimate Question of Life, the Universe, and Everything: %i"
  • log_args: [42]

Backward Compatibility

This is an additive change — existing log fields remain unchanged. Components adopting the new version will have additional structured data available for advanced filtering in Datadog.

Review & Testing Checklist for Human

  • Verify StructuredLoggingFilter correctly extracts record.msg as template and record.args as list
  • Confirm that when record.args is None (no arguments), log_args is set to None and doesn't break GELF serialization
  • Test with a real GELF endpoint (or stdout=True) to verify log_template and log_args appear as extra fields in the output

Notes

  • The filter is only attached to the GELF handler (not the default stdout logger), so local development logging is unaffected
  • log_template and log_args are surfaced as GELF extra fields because include_extra_fields=True is the default

Link to Devin session: https://app.devin.ai/sessions/6eb5ae7e18be4f48ac4a1070c2fead88
Requested by: @soustruh

…output

Co-Authored-By: martin.struzsky <martin@struzsky.cz>
@devin-ai-integration
Copy link
Copy Markdown

devin-ai-integration Bot commented May 21, 2026

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@soustruh soustruh marked this pull request as draft May 21, 2026 14:36
@soustruh soustruh marked this pull request as ready for review May 27, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant