feat: add structured logging fields (log_template, log_args) to GELF output#96
Open
soustruh wants to merge 1 commit into
Open
feat: add structured logging fields (log_template, log_args) to GELF output#96soustruh wants to merge 1 commit into
soustruh wants to merge 1 commit into
Conversation
…output Co-Authored-By: martin.struzsky <martin@struzsky.cz>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a custom logging filter (
StructuredLoggingFilter) that preserves Python's raw message template and arguments as separate GELF fields, enabling pattern-based filtering and aggregation in Datadog.Why This Matters
Even with AI agents that can crawl unstructured logs, structured logging provides critical value:
count by log_templateor grouping by argument valuesChanges
StructuredLoggingFilterclass beforeCommonInterfaceto extractrecord.msgandrecord.argsbefore interpolationset_gelf_loggerto attach the filter to the GELF handlerinclude_extra_fields=Truemechanism# noqa: E301comment fromset_gelf_loggersignatureExample
Results in GELF output with:
short_message: "The Answer to the Ultimate Question of Life, the Universe, and Everything: 42"log_template: "The Answer to the Ultimate Question of Life, the Universe, and Everything: %i"log_args: [42]Backward Compatibility
This is an additive change — existing log fields remain unchanged. Components adopting the new version will have additional structured data available for advanced filtering in Datadog.
Review & Testing Checklist for Human
StructuredLoggingFiltercorrectly extractsrecord.msgas template andrecord.argsas listrecord.argsisNone(no arguments),log_argsis set toNoneand doesn't break GELF serializationstdout=True) to verifylog_templateandlog_argsappear as extra fields in the outputNotes
log_templateandlog_argsare surfaced as GELF extra fields becauseinclude_extra_fields=Trueis the defaultLink to Devin session: https://app.devin.ai/sessions/6eb5ae7e18be4f48ac4a1070c2fead88
Requested by: @soustruh