Skip to content

Exposition format: Specify whitespace handling in text format #543

Open

Description

@beorn7

So far, the text format allows whitespace liberally. This was done to make generation of metrics in text format easy. The assumption was that there are way more generator implementations than parser implementations.

However, some implementers of parsers requested to have stricter rules for whitespace to make parser implementation easier.

We need to come up with a decision before carving the format into stone.

Activity

  1. beorn7 commented on Sep 9, 2016

    @beorn7
    ContributorAuthor

    Just FTR: My personal opinion is still that we should make generation as easy as possible and keep being liberal with whitespace. I don't even see how parser implementation gets so much more complicated with liberal whitespace.

  2. RichiH commented on Sep 9, 2016

    @RichiH
    Member

    I agree with @beorn7. I suggest exporters/libraries SHOULD NOT print more than one whitespace behind each other, while consumers MUST accept multiples and MAY print a warning/info message.

  3. brian-brazil commented on Sep 9, 2016

    @brian-brazil
    Contributor

    Allowing whitespace e.g. between the metric name and the { requires more FSM states and transitions, and in no way makes it easier to generate the text format. Similarly after the {, before the =, after the = etc.

  4. brian-brazil commented on Sep 10, 2016

    @brian-brazil
    Contributor

    By contrast, allowing a trailing comma in the labels and empty {} both make writing generators easier as both avoid a special case.

  5. RichiH commented on Sep 10, 2016

    @RichiH
    Member

    This is actually a good point. The failure to allow a trailing comma is my main gripe with JSON.

  6. brian-brazil commented on Sep 10, 2016

    @brian-brazil
    Contributor

    We already expanded the format to allow the trailing comma.

  7. beorn7 commented on Sep 15, 2016

    @beorn7
    ContributorAuthor

    One of the main reasons to introduce the text format was to make it easy to generate it on the fly with a shell script or other not very powerful tools to generate a text string. By being tolerant with whitespace, the user, who might not be very familiar with the format, doesn't have to waste brain cycles to know where whitespace is required or disallowed. Also, the tool might insist on adding whitespace between components (e.g. fmt.Println() in Go).

  8. brian-brazil commented on Sep 15, 2016

    @brian-brazil
    Contributor

    I've not seen any client library or other custom exposition format put extra whitespace in the places we allow it, so empirically it doesn't help generation. On the other hand this doubles the complexity of a parser.

    I've yet to come across a language that doesn't offer a way to control whitespace. Almost everything modern has a form of sprintf, and the rest have a way to do it.

    If this is to be an IEFT standard there will be far more than the existing 2 parser implementations, and the chances of implementors not implementing this is high as noone actually uses this whitespace. It's better to have a smaller simpler standard.

  9. beorn7 commented on Sep 15, 2016

    @beorn7
    ContributorAuthor

    I've not seen any client library or other custom exposition format put extra whitespace in the places we allow it, so empirically it doesn't help generation.

    I'm not talking about client libraries. Obviously, it's easy to teach libraries to follow restricted rules. I'm talking about ad-hoc generation of lines of text format.

    On the other hand this doubles the complexity of a parser.

    That's a bold claim. The Go implementation treats whitespace correctly. I have my doubts it would become half as complex if we removed that feature from it.

    But even if it's true, I estimate about ten parser implementations to exist worldwide, but there will be thousands of occasions where users create text-format lines ad hoc. So I still believe it's worth it.

    It's better to have a smaller simpler standard.

    Meticulously specifying where which whitespace is allowed, required, or disallowed makes the standard more complicated. Also, most languages allow arbitrary whitespace between components. We don't have to go back to FORTRAN77 for the sake of simplifying parser implementation. It will make it more difficult to understand the standard.

  10. brian-brazil commented on Sep 15, 2016

    @brian-brazil
    Contributor

    I'm talking about ad-hoc generation of lines of text format.

    I included that in my statement. I've seen none of them that had extra whitespace.

    That's a bold claim.

    It roughly doubles the number of parser states. It'd halve the complexity of the Python parser not to have to support extra whitespace.

    Meticulously specifying where which whitespace is allowed, required, or disallowed makes the standard more complicated.

    I disagree, we need to specify that anyway and anywhere something is optional makes it more complicated and less likely to be followed.

    Saying that the format is just

    name{label_name="value"} 0
    name{label_name="value",} 0
    name{label_name="value",name2="value2"} 0
    name{,} 0
    name{} 0
    name 0
    name 0 0
    

    and not

    name {label_name="value",name2="value2"} 0
    name{ label_name="value",name2="value2"} 0
    name{label_name ="value",name2="value2"} 0
    name{label_name= "value",name2="value2"} 0
    name{label_name="value" ,name2="value2"} 0
    name{label_name="value", name2="value2"} 0
    name{label_name="value",name2="value2" } 0
    etc.
    

    makes it simpler.

  11. beorn7 commented on Sep 15, 2016

    @beorn7
    ContributorAuthor

    I've seen none of them that had extra whitespace.

    I have seen many. Especially blanks after the comma between label pairs.
    In fact, every machine at SoundCloud has at least one file for the text file exporter with that kind of whitespace usage.

    It'd halve the complexity of the Python parser not to have to support extra whitespace.

    As said, even if that's the case, I'd vote for allowing whitespace for the reasons already stated.

    About the specification: Specifications don't happen by example. Allowing whitespace essentially adds one sentence to the spec. In practice, it won't make it longer because, for the non-whiteespace spec, you probably want to explicitly state that no spurious whitespace is allowed (as it is so common to allow it).

  12. brian-brazil commented on Sep 15, 2016

    @brian-brazil
    Contributor

    Based on other RFCs I'd expect the spec to be via EBNF, with examples (as EBNF while precise is hard for humans to interpret).

  13. beorn7 commented on Sep 15, 2016

    @beorn7
    ContributorAuthor

    Specifying optional whitespace in EBNF is actually quite verbose. So for shortness of EBNF, I'd even buy the argument that optional whitespace bloats the spec.

    However, as you say, EBNF is not what humans will see as relevant for the perceived complexity of the spec. Humans will perceive it as complex if they have to unlearn that whitespace between tokens is fine.

  14. brian-brazil commented on Sep 15, 2016

    @brian-brazil
    Contributor

    I'm not following your argument here. The EBNF form is a "pure" mathematical form of the spec, so if that's bloated then the spec is bloated as that's what implementors actually have to implement to be correct.

    I'd be okay with a space after the commas.

    Humans will perceive it as complex if they have to unlearn that whitespace between tokens is fine.

    I don't think we should presume anything of people reading the spec.

  15. beorn7 commented on Sep 15, 2016

    @beorn7
    ContributorAuthor

    The EBNF form is a "pure" mathematical form of the spec, so if that's bloated then the spec is bloated as that's what implementors actually have to implement to be correct.

    The fact that it is "pure" doesn't mean it's the shortest possible description.

    But I anyway don't think it is relevant to come up with a spec that is describable in the theoretically shortest form.

    I think it is relevant to come up with a spec that is easy to understand and memorize for humans.

    I'd be okay with a space after the commas.

    In which case we would introduce a special case, which would indeed "bloat" the spec in both senses, the mathematical "pure" one and the one concerning the cognitive load for humans.

    I don't think we should presume anything of people reading the spec.

    Then we will have a hard time formulating a spec that is readable by humans.

    But we are losing ourselves in fairly irrelevant details. A slightly more or less complex description of the format is hardly of any relevance for the actual decision we have to come up with here.

  16. grobie commented on Sep 15, 2016

    @grobie
    Member

    In general I believe we should prioritize ease of creating a text format representation over the ease of writing a parser. A parser is written once and while it might be annoying having to consume extra whitespace, I don't think it's a big deal. Ensuring the correctness of a parser is easy by providing a reference file implementors can check against.

    I have seen people putting whitespaces anywhere inside of curly braces. I expect peole to do the same in ad-hoc generated outputs.

    Though, the only thing I actually feel strongly about is allowing a a single optional whitespace after a comma.

  17. juliusv commented on Sep 15, 2016

    @juliusv
    Member

    Leaning towards allowing whitespace. Only not a big fan of allowing it between the metric name and {...}, but then again, PromQL also allows that...

  18. lasomethingsomething commented on Nov 18, 2026

    @lasomethingsomething

    What steps or todo's would help drive a decision and resolution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions