Skip to content

Exposition format: Termination marker for scrapes #546

Open

Description

@beorn7

Taken from the original issue prometheus/common#14 :

To positively confirm the completion of a scrape (rather than only negatively detect an incomplete scrape by a connection error), a termination marker is needed.

Proposal:

If an exposition is meant to be terminated by a termination marker, it's content-type contains an additional parameter termination=marked. (It will still be possible to not use termination markers so that trivial one-line expositions are still possible by default.) Termination can be requested via the usual content-type negotiation, and should only be applied if requested.

With termination=marked, the following termination marker has to end the payload:

  • A line # END\n for the text format.
  • A uvarint-encoded value of MaxUint32 in case of protobuf. (Has a length of 5 bytes - protobufs of that size cannot be encoded anymore but the value is any waymeant to be larger than a protobuf can reasonably be. Some protobuf implementation limit the size of a proto message at 64MiB anyway.)

An exposition with termination=marked and no termination marker is considered incomplete, which should be treated the same as an ingestion interrupted by a connection error.

Activity

  1. RichiH commented on Sep 9, 2016

    @RichiH
    Member

    Just to be explicit, am I correct in assuming that anything after # END\n MUST NOT be exposed and MUST be discarded by the receiving side?

    To play devils' advocate: May there be multiple whitespace before/after END?

  2. RichiH commented on Jul 18, 2017

    @RichiH
    Member

    As this is implementation-specific, moving this here: It could be implemented as an end metric.

    up == 1 && end == 0 and you have everything that's not right.

  3. brian-brazil commented on Jul 18, 2017

    @brian-brazil
    Contributor

    The problem is that users would not remember to do this, which is why this should fail in a really obvious way. Partial scrapes cause subtle failures, which is what we're trying to avoid here.

  4. lasomethingsomething commented on Nov 18, 2026

    @lasomethingsomething

    Is this problem still valid and something to take action on?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions