Repository navigation
Conversation
…rners Adds resolve_conflicting_exposed_keys_iter, eval_key_keep_mask_iter, SingleLabelKeepCache::keep_str, SingleLabelTransformCache::transform_str, SingleLabelTransformCache::eval_single_str, and retained_memory_bytes helpers so arena-backed interners like ArrowDecoder's StringInterner can run Stage 2 relabeling without building temporary key/value slices. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…rowDecoder Adds MetricRelabelPlan, DecodeOptions::honor_labels (default false), and DecodeOptions::metric_relabel_plan / with_metric_relabel_configs to ArrowDecoder. - Stage 1 pushes leading __name__-only keep/drop/keepequal/dropequal rules into ArrowDecoder::ingest_entries, caching per contiguous wire series name so dropped series runs skip label unescaping, string interning, and family assembly while preserving cooperative budget accounting. - Stage 2 resolves honor_labels = false collisions once across key_interner (leaving exemplar attributes untouched), executes labeldrop/labelkeep and single-label keep/drop/replace/lowercase/uppercase/hashmod rules on dictionary interners, and falls back to exact prometheus_relabel::process_in_place for general multi-label, labelmap, keepequal/dropequal, dynamic target_label, and __name__-mutating rules. - Compacts dropped data points, child exemplars, and unused metric families in place, skips empty chunks in decode_stream, and updates ResourceAttrs only when Stage 2 mutates or drops target labels. Towards #35. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…relabeling Add unit tests covering honor_labels collision handling and exemplar preservation, Stage 1 __name__ pushdown (including histograms and cooperative budgets), Stage 2 dictionary-vectorized rules, general scalar fallback with a prometheus-relabel oracle, and empty-chunk dropping. Add OTAP/OTLP round-trip coverage and enable MetricRelabelPlan in the dhat bounded-memory test, add a Stage 1 + Stage 2 decode benchmark case, and update specs/SPEC.md. Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #62. Towards #35.
Wires
honor_labelscollision resolution and two-stagemetric_relabel_configsexecution intoArrowDecoder. Thearrowfeature ofprometheus-expfmtenablesprometheus-relabeland re-exports it asprometheus_expfmt::relabel, whiledefault-features = falsekeeps the text parser dependency-free.When
honor_labelsisfalseand exposed metric labels collide with target labels, colliding keys are remapped inO(unique_keys)across the batch key dictionary toexported_<key>without touching exemplar attributes. Notice thatexported_<key>occupancy is checked batch-wide across the dictionary rather than per series, which matchesspecs/SPEC.mdand avoids per-sample key lookups.MetricRelabelPlansplits leading__name__-only filter rules into Stage 1 (evaluated at wire series-name transitions before float parsing or interning) and executes remaining rules in Stage 2 over finished chunks (dictionary-vectorized forlabeldrop,labelkeep, and single-labelkeep/drop/replace/lowercase/uppercase/hashmod, with a general scalar fallback for multi-source,labelmap, or__name__-rewriting rules). Chunks whose data points are all dropped are skipped bydecode_stream. Dropping__name__while other labels remain returnsDecodeError::MissingMetricName, matching PrometheuserrNameLabelMandatory.Commits in reading order:
373a832(relabel: add str-slice and iterator dictionary helpers for arena interners) adds iterator and&strentry points plus retained-memory accounting onprometheus-relabelcaches soArrowDecodercan feed its arena interners without copying keys intoVec<String>.8a19730(arrow: wire honor_labels and two-stage metric_relabel_configs into ArrowDecoder) addsMetricRelabelPlan,DecodeOptions::{with_honor_labels, with_metric_relabel_plan, with_metric_relabel_configs},DecodeError::MissingMetricName, Stage 1 pushdown, Stage 2 relabeling, and table compaction.87efe0a(tests, benches, docs: verify ArrowDecoder honor_labels and two-stage relabeling) adds unit tests (including aprometheus-relabeloracle), OTAP/OTLP round-trip coverage,dhatbounded-memory verification with active relabeling, a Stage 1 + Stage 2 decode benchmark, andspecs/SPEC.mdupdates.Benchmarks (before: 16ce7f9 vs after: 87efe0a)
Comparing 16ce7f9 (base PR #62) against 87efe0a:
cargo bench -p prometheus-expfmt --bench decode -- -bench '^BenchmarkDecode' -benchtime 1s -count 6 -cpu 2 -benchmem