Add a forMode attribute to several elements

[msdemlei]

This is an alternative to PR #8 and was developed in the discussion there.

[mbtaylor]

Yes, I think this would be a good solution.

As I noted in my comments on #8 there is some scope for older clients getting confused by multiple limit declarations differentiated by an attribute they don't know about. In existing versions of TOPCAT, I think this would result in multiple apparently contradictory entries in the TAP window Max Rows selector (only really apparent if you actually click on the thing, which probably most users don't most of the time). That could look a bit weird to users, but I wouldn't say it's catastrophic, and arguably is more informative than the existing state where you see a single limit that may be invalid for one of sync/async. I can't speak for other clients.

[stvoutsin]

I think this approach looks great and does indeed seem to work for to the various use-cases.
Just for completeness I was wondering if we want to clarify a couple edge cases:

Should we clarify how clients should choose if both a general declaration and a mode-specific declaration exist for the same element type?

For example say that the service declares:

<executionDuration>
  <default>30</default>
  <hard>120</hard>
</executionDuration>

<executionDuration forMode="sync">
  <default>60</default>
</executionDuration>

<executionDuration forMode="async">
  <default>3600</default>
  <hard>86400</hard>
</executionDuration>

What would be the default for sync? I'm assuming it sould it be 60, but is this clear what the clients should do based on the document? And is the hard-limit for sync 120 or undefined? In other words should clients scan all elements and merge them, or use first-match?

The second edge case is that the schema allows multiple declarations for the same mode.
For example I think this would be valid:

<outputLimit forMode="async">
  <default unit="row">100000</default>
  <hard unit="row">1000000</hard>
</outputLimit>

<outputLimit forMode="async">
  <default unit="row">200000</default>
  <hard unit="row">5000000</hard>
</outputLimit>

Should we outline how clients should handle this, and should the document prohibit this somehow?
Since XSD constraints seem difficult here unless there is an easy way I'm not aware of, should we add explicit guidance in the specification:

Something like:

Uniqueness of Mode-Specific Declarations

For each limit type, servers MUST not include more than one element with the same forMode attribute value.

Servers MAY include:

• At most one element without forMode (the general declaration)
• At most one element with forMode="sync"
• At most one element with forMode="async"

If the capabilities document violates this constraint, clients SHOULD/MUST(?) use the first matching element.

[gmantele]

For example say that the service declares:

<executionDuration>
  <default>30</default>
  <hard>120</hard>
</executionDuration>

<executionDuration forMode="sync">
  <default>60</default>
</executionDuration>

<executionDuration forMode="async">
  <default>3600</default>
  <hard>86400</hard>
</executionDuration>

What would be the default for sync? I'm assuming it sould it be 60, but is this clear what the clients should do based on the document? And is the hard-limit for sync 120 or undefined? In other words should clients scan all elements and merge them, or use first-match?

My understanding for this example is the following for sync mode: default=60 and max=120.

I agree this corner case should be explicitly described. I'd say that the default block (i.e. the one with no for-mode attribute) applies by default and is overwritten by the most specific one (if any is present). A such example can then be added in the document to be more concrete for readers.

[gmantele]

The second edge case is that the schema allows multiple declarations for the same mode. For example I think this would be valid:

<outputLimit forMode="async">
  <default unit="row">100000</default>
  <hard unit="row">1000000</hard>
</outputLimit>

<outputLimit forMode="async">
  <default unit="row">200000</default>
  <hard unit="row">5000000</hard>
</outputLimit>

Should we outline how clients should handle this, and should the document prohibit this somehow? Since XSD constraints seem difficult here unless there is an easy way I'm not aware of, should we add explicit guidance in the specification:

Something like:

Uniqueness of Mode-Specific Declarations

For each limit type, servers MUST not include more than one element with the same forMode attribute value.
Servers MAY include:

• At most one element without forMode (the general declaration)
• At most one element with forMode="sync"
• At most one element with forMode="async"

If the capabilities document violates this constraint, clients SHOULD/MUST(?) use the first matching element.

Here, I'd say there are two approaches:

1. Allow multiple occurrences for the same mode and take into account only the last one into account (but what happens if one defines the hard limit and not the other?)
2. As you suggest, explicitly say that at most one occurrence of each mode must be included.

My preference goes for the 2nd option, as @stvoutsin suggests, as the 1st one will be clearly more confusing for everyone. If this one is adopted, may I suggest to reflect this in the XML schema by setting the attribute maxOccurs to 3 instead of unbounded?

[mbtaylor]

I'd say that the text in the proposed section 2.2 Mode-dependent Declarations covers this in sufficient detail:

Elements without a forMode attribute apply to all modes for which no more specific specification is given. Elements with a forMode attribute override elements without one. That is, to find the applicable declaration for a particular access mode, clients should first look for an element with the current access mode and then for one without forMode. Whatever is found first is the relevant declaration.

Admittedly it doesn't explicitly forbid adding multiple contradictory limits; that could be added but it seems pretty obvious to me how that would work (services shouldn't do it, I don't know why they would, and if clients find such things they'd be within their rights to use whichever one they like).

[gmantele]

My bad, I did not read the diff of the .tex part...sorry. Indeed the 1st corner case of @stvoutsin seems to be covered there.

But as you said, not the 2nd one. I'd say it should because of the behavior confusion that it would bring, on both server and client sides.

[mbtaylor]

I'm happy with the revised text.

[gmantele]

It looks good to me too.
I can review and approve this PR whenever @stvoutsin and you are also happy with this version (unless you want to add anything).

[msdemlei]

On Thu, Oct 02, 2025 at 02:30:48AM -0700, Grégory Mantelet wrote: gmantele left a comment (ivoa-std/TAPRegExt#10) But as you said, not the 2nd one. I'd say it should because of the behavior confusion that it would bring, on both server and client sides.

Hm... regrettably, I have a bit of a hard time to put this into words in an understandable way. The main trouble is that multiple, say, outputFormat elements with the same forMode are ok, but for limits we definitely don't want that. Saying that makes me realise that my current recipe in the section on Mode-dependent Declarations doesn't work. I'm trying again in commit #7165975 and immediately dislike the language. That this is so complicated is instilling doubts in me whether that's actually what we should do. Hm. Better text solicited, either way.

[mbtaylor]

I'm trying again in commit #7165975 and immediately dislike the language. That this is so complicated is instilling doubts in me whether that's actually what we should do. Hm. Better text solicited, either way.

I think you're being too hard on your text. It looks comprehensible to me.

[stvoutsin]

This version looks good to me, but I still wonder if there might be ambiguity in the edge case that I mentioned.

The text specifies that

limits given in elements with forMode override limits given in elements without one for the mode specified.

However override could potentially be interpreted as a complete replacement, or a merge/patch.
My interpretation would be a complete replacement because we say "limits given in elements" override rather than "child values in elements", which suggests to me the entire element is the unit of replacement.
However @gmantele interpretation reads it as a merge. So perhaps it would be useful to clarify?

Also one thought is should we change the text structure a bit based on the two distinctive behaviours?
I'm thinking something like this:

Elements without a forMode attribute apply to all modes.
The forMode attribute controls two different behaviors:

Additive declarations: outputFormat, uploadMethod
Mode-specific declarations add to the general set. All declarations
apply, mode-specific ones don't remove or replace general ones.

Override declarations: (limit elements: retentionPeriod, executionDuration, outputLimit, uploadLimit)
Mode-specific declarations completely replace general ones for that mode.
When a mode-specific limit element is found, only its child elements apply; missing child elements are not inherited from the general declaration.
At most one occurrence per mode is allowed (including at most one without forMode).

Having said that if you disagree that the edge case is ambiguous or don't like this structure, what you have is also perfectly fine from my point of view so I'm happy if you want to merge either way.

[msdemlei]

On Fri, Oct 03, 2025 at 08:50:29AM -0700, stvoutsin wrote: stvoutsin left a comment (ivoa-std/TAPRegExt#10) However override could potentially be interpreted as a complete replacement, or a merge/patch. My interpretation would be a complete replacement because we say "limits given in elements" override rather than "child values in elements", which suggests to me the entire element is the unit of replacement.

The intention definitely is merge; if you only override a hard limit, the default limit would stay in place.

Also one thought is should we change the text structure a bit based on the two distinctive behaviours? I'm thinking something like this:

That is probably clearer, yes. Would you care to prepare a PR with that text? That would be wonderful...

[stvoutsin]

On Fri, Oct 03, 2025 at 08:50:29AM -0700, stvoutsin wrote: stvoutsin left a comment (ivoa-std/TAPRegExt#10) However override could potentially be interpreted as a complete replacement, or a merge/patch. My interpretation would be a complete replacement because we say "limits given in elements" override rather than "child values in elements", which suggests to me the entire element is the unit of replacement.
The intention definitely is merge; if you only override a hard limit, the default limit would stay in place.
Also one thought is should we change the text structure a bit based on the two distinctive behaviours? I'm thinking something like this:
That is probably clearer, yes. Would you care to prepare a PR with that text? That would be wonderful...

I've added a PR here: msdemlei#1
Let me know if you'd like me to make any amendments to it.

[stvoutsin]

As far as I can see this looks good to go from my point of view, with only a couple minor comments/questions upon re-reading this

[stvoutsin]

Do we want to add a sentence specifying what a client should do when it does encounter duplicates? (Take first? Error? Merge?)

[stvoutsin]

Can we think of any case where a service would want to restrict Language per mode? Like restricting for example geometry support or UDFs for async-only? I'm fine with omitting if not, perhaps a brief not of why if so, but we also could not say anything, not a major issue.

[stvoutsin]

Should this be sect.~\ref{sect:modedep} ?
If so, it also shows up this way in the sentence:

Operators can restrict individual upload methods

[msdemlei]

On Fri, May 29, 2026 at 01:42:28PM -0700, stvoutsin wrote: @stvoutsin approved this pull request. +are inherited from the general declaration. For these elements, only zero +or one occurrences are allowed each without \xmlel{forMode} and per +\xmlel{forMode} value\footnote{This constraint is hard to express in XML +Schema and is thus not enforced by it; its purpose is to make sure that Do we want to add a sentence specifying what a client should do when it does encounter duplicates? (Take first? Error? Merge?)

In some other standards, I've tried to require certain behaviours in similar situations, and in general some client author came around and felt that I was overreaching what a standard should say. Hence, I'm reluctant. FWIW, I'd say a typical client should show something like a modal error and discard the entire limits record. But then that's not an option for something like pyVO, which perhaps gives you a bit of sympathy for my reluctance.

+\paragraph{Override declarations} +For limit elements (\xmlel{retentionPeriod}, \xmlel{executionDuration}, Can we think of any case where a service would want to restrict Language per mode? Like restricting for example geometry support or UDFs for async-only? I'm fine with omitting if not, perhaps a brief not of why if so, but we also could not say anything, not a major issue.

We have briefly brainstormed about this. I can certainly think of situations like that, but language has such a complex content model that semantics would quickly become ugly. And think of TOPCAT that would have to re-do a lot of its UI when you flip to async. In short, there was fairly universal dislike of that idea, and the scenarios we could think of weren't compelling enough to overfide that dislike.

> @@ -600,11 +637,15 @@ \subsection{Output Formats} with a BINARY2 child \end{description} +Operators can restrict individual output formats to either sync or async +requests using the \xmlel{forMode} attribute as explained in +\ref{sect:modedep}. Should this be sect.`~\ref{sect:modedep}` ? If so, it also shows up this way in the sentence:

Yes it should. Thanks.