Repository navigation
9.0 (fix): Spring format negotiation follow-ups: problem+xml ordering, fallback wording - #17102
codeconsole wants to merge 7 commits into
Conversation
With grails.mime.negotiation.spring, a format grails.mime.types does not list is written in the type Spring MVC's content negotiation manager maps it to. The guide, the upgrade notes, the upgrade skill and the Javadoc said only spring.mvc.contentnegotiation.media-types fed that fallback, but the manager also holds the defaults Spring MVC registers for its message converters (json, and xml, atom, rss, smile, cbor and yaml where their libraries are present), and it matches a format whatever its case. Say so, and pin the json default, also as JSON, in SpringFormatNegotiationSpec.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 9.0.x #17102 +/- ##
==================================================
+ Coverage 65.6431% 65.6514% +0.0083%
Complexity 33022 33022
==================================================
Files 2446 2446
Lines 121408 121411 +3
Branches 22524 22526 +2
==================================================
+ Hits 79696 79708 +12
+ Misses 31647 31632 -15
- Partials 10065 10071 +6
🚀 New features to boost your workflow:
|
…s do The Grails rules read application/problem+xml, where no configured type names it, as application/xml, and so put a later +xml type of equal quality ahead of it: Accept: application/problem+xml, application/atom+xml is the atom format. With grails.mime.negotiation.spring the +xml preference looked only for application/xml and text/xml, so the same header was the xml format. preferringSuffixedXml becomes a method of the strategy, which knows the configured types, and also takes an unconfigured problem+xml as the XML type. SpringFormatNegotiationSpec gains the header at equal and lower quality, and FormatNegotiationComparisonSpec sends it under every configuration: only what Spring MVC writes for it differs, as it does for application/xml, application/atom+xml.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A configured problem type is treated as if it were not configured.
matrei
left a comment
There was a problem hiding this comment.
Reviewed at 3364f39 (base 9.0.x, merge base 733397b, the merge of #16996 and the current tip of 9.0.x). 9.0.x is the right base: grails.mime.negotiation.spring exists only there, and preferringSuffixedXml came with #16996 and is in no release yet, so it can stop being static.
What I ran:
:grails-mimetypes:testwith the build cache off: 206 tests, all pass.- The new rows against changes to the code:
- In
springMediaTypesOf, a lookup that keeps the case of the format fails theJSONrow, and no fallback at all fails both rows. Withgrails.mime.typesofallandxml, the manager maps onlyjsonandxml, so thejsonrow does come from Spring MVC's default. - In
preferringSuffixedXml, dropping theproblem+xmlclause fails the new row ofSpringFormatNegotiationSpecandFormatNegotiationComparisonSpec.
- In
:grails-doc:publishGuidebuilds (at e7fa993, whose docs 3364f39 leaves as they are), and the changed passages read as intended insingle.html.
Only the canary jobs fail, as they do on 9.0.x at 733397b.
To fix
-
upgrading90x.adoc:1587: "A format that Spring MVC's content negotiation manager maps, such as ... one of the defaults Spring MVC registers for its message converters (json, andxml, ...), whatever its case, decides what Spring MVC writes, ..., whilerespondstill answers406for it." The examples now includejsonandxml, whichgrails.mime.typeslists by default, so the sentence reads as ifrespondanswered?format=jsonwith406. The guide ("a formatgrails.mime.typesdoes not list and ...") and the skill ("A format only ...") keep the scope, and "A format only Spring MVC's content negotiation manager maps" would do it here too. -
The guide (
contentNegotiation.adoc:309), the upgrade note, the skill and the Javadoc ofspringMediaTypesOflist what the manager maps as the formatsspring.mvc.contentnegotiation.media-typesadds and Spring MVC's defaults. My point on #16996, which this follows, got two things wrong:- The manager also maps the data formats Grails registers with it as format aliases:
json,xml,hal,atom,rss,csvandtext, wheregrails.mime.typeslists them (section 15 of the upgrade notes). Since the lookup ignores case,?format=HAL,?format=CSVand?format=Textresolve toapplication/hal+json,text/csvandtext/plainfor what Spring MVC writes, whileresponse.formatisnull, sorespondanswers406, as for?format=JSON. I checked this with temporary features inSpringFormatNegotiationSpec. With the defaultgrails.mime.types, the manager mapsatom,csv,hal,json,rss,textandxml. - Spring MVC 7.1.0 registers its defaults for the media types its message converters support, not for the libraries on the classpath:
WebMvcConfigurationSupport.getDefaultMediaTypes()collects them fromgetMessageConverters(). It also registersyamlwhen a converter supportsapplication/atom+xml, a Spring bug with an open fix, spring-projects/spring-framework#37725. Soyamlcurrently comes with Rome rather than with Jackson's YAML module.
Naming Spring MVC's defaults by example, rather than listing them, would keep the text right as Spring changes them. For the guide, something like: "The manager maps the data formats Grails registers with it, those among
json,xml,hal,atom,rss,csvandtextthatgrails.mime.typeslists, the formatsspring.mvc.contentnegotiation.media-typesadds, and those Spring MVC registers for the types its message converters support, such asjson. It matches a format whatever its case, whichgrails.mime.typesdoes not, so?format=JSONand?format=HALare written asapplication/jsonandapplication/hal+json." The upgrade note can point at section 15 for the first part, and the skill and the Javadoc can say the same in short. - The manager also maps the data formats Grails registers with it as format aliases:
-
3364f39: no test covers a
grails.mime.typesthat namesapplication/problem+xml. Taking the!problemXmlConfiguredout, so that a configuredproblem+xmlcounts as the XML type too, fails neitherSpringFormatNegotiationSpecnorFormatNegotiationComparisonSpec. A row ofSpringFormatNegotiationSpecwith such agrails.mime.typeswould cover it: thereAccept: application/problem+xml, application/atom+xmlis theproblemformat,response.mimeTypesis[application/problem+xml, application/atom+xml], and Spring MVC keeps that order.
On the inline comment about an explicit problem mapping
A problem: application/problem+xml mapping in spring.mvc.contentnegotiation.media-types alone does not change the format Grails resolves for Accept: application/problem+xml, application/atom+xml: it is atom, with response.mimeTypes of [application/atom+xml, application/xml], with the setting and without it, and with the setting Spring MVC gets [application/atom+xml, application/problem+xml]. Keeping the order of the header there would have Spring MVC prefer problem+xml while response.format and withFormat answer atom, which is the disagreement preferringSuffixedXml exists to prevent. The check follows DefaultAcceptHeaderParser, which reads problem+xml as application/xml unless a type of grails.mime.types has that name, and where one does, the format is problem and the order is kept (item 3). I checked each of these with temporary features. So I think the commit is right as it is.
Optional
- A
HALrow in the new feature of e7fa993, with the defaultgrails.mime.typesand[application/hal+json]expected, would pin the case of a format Grails registers, if the text names it.
Verified as correct
springMediaTypesOflooks the format up in lower case, whilemediaTypesOfmatches it exactly, as the guide and the Javadoc now say.- The new feature of e7fa993 sends
Accept: application/xml, which itsgrails.mime.typeslists, so a pass cannot come from theAcceptheader. - The table row of the guide and the Javadoc of
GrailsFormatContentNegotiationStrategygive their examples with "such as", so they hold with the fuller list. - Spring Framework 7.1.0, which 9.0.x resolves, registers
atom,rss,xml,json,smile,cborandyamldefaults, the formats the PR lists. - 3364f39: the configured check matches the one
DefaultAcceptHeaderParsermakes, a type of that name ignoring case, andFormatNegotiationComparisonSpecshows the Grails formats of the new header now agree under every configuration it compares.
… maps preferringSuffixedXml takes application/problem+xml as the XML type a +xml type comes ahead of only where no type of grails.mime.types names it, as DefaultAcceptHeaderParser reads it. No test covered the configured case: with grails.mime.types naming application/problem+xml, Accept: application/problem+xml, application/atom+xml is the problem format and keeps the order of the header, for Grails and for what Spring MVC writes. A problem mapping in Spring MVC alone leaves it the atom format, with application/problem+xml behind atom, as Grails reads it as application/xml.
Grails registers the data formats of grails.mime.types (json, xml, hal, atom, rss, csv and text) with Spring MVC's content negotiation manager as format aliases, and the fallback looks a format up there whatever its case. With the default grails.mime.types, ?format=HAL, ?format=CSV and ?format=Text are written as application/hal+json, text/csv and text/plain for what Spring MVC writes, while Grails resolves no format for them.
The fallback takes a format grails.mime.types does not list from Spring MVC's content negotiation manager. Besides the formats spring.mvc.contentnegotiation.media-types adds, the manager maps the data formats Grails registers with it (json, xml, hal, atom, rss, csv and text, where grails.mime.types lists them), which the case-insensitive lookup also finds as ?format=HAL, and those Spring MVC registers for the types its message converters support, which depend on the converters, not on the libraries on the classpath. The guide, the upgrade note, the upgrade skill and the Javadoc of springMediaTypesOf now say so, naming Spring MVC's defaults by example. The upgrade note says "a format only" the manager maps, so it no longer reads as if respond answered ?format=json with 406, and it links the format resolution section by a new anchor, upgradingGrailsFormatResolution.
|
@matrei thanks, all three held, and I added the HAL row too.
|
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. The format and the written type disagree when the request also accepts plain xml.
For a request a URL mapping handles, preferringSuffixedXml runs once for what Spring MVC writes and again when Grails resolves its formats. An unconfigured application/problem+xml was both a +xml type the first pass moved ahead of application/xml and the XML type the second pass anchored on, so Accept: application/xml, application/atom+xml, application/problem+xml had Spring MVC write problem+xml while the format was atom. DefaultAcceptHeaderParser reads such a problem type as application/xml, one of the XML types, which the +xml preference never moves. Treat it the same way: it is still the XML type a +xml type comes ahead of, but it is not moved itself, so a second pass changes nothing and Spring MVC writes application/atom+xml for that header, the atom format. The Spring-only problem feature of SpringFormatNegotiationSpec becomes data-driven over that header, the upgrade note's text/xml example, application/xml, application/problem+xml and problem+xml between xml and atom, and FormatNegotiationComparisonSpec sends the header under every configuration, where only what Spring MVC writes differs.
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A repeated type in the accept header still makes the written type and the format disagree.
preferringSuffixedXml took each +xml type behind the XML type out of the list with remove(Object), which removes the first equal type. Where the header lists a type twice, that was the one ahead of the XML type, so the first pass, which decides what Spring MVC writes, left the type behind it, and the second, when Grails resolves its formats, moved it again: with the default grails.mime.types, Accept: application/atom+xml, application/xml, application/atom+xml had Spring MVC write application/xml while the format was atom, and application/xml, application/atom+xml, application/atom+xml was the xml format. With problem+xml mapped in Spring MVC only, the same happened with an unconfigured problem+xml as the XML type. apache#16996 brought the method in on 9.0.x. Each type is now removed from its own index. After one pass no +xml type of the quality of the XML type is left behind it, so a second pass changes nothing, and the format and the type Spring MVC writes agree. SpringFormatNegotiationSpec covers the repeated type with the default types and with the Spring-only problem mapping, and FormatNegotiationComparisonSpec sends application/atom+xml, application/xml, application/atom+xml under every configuration, where only what Spring MVC writes differs.
🚨 TestLens detected 1 failed test 🚨Failed Jobs without Test Failures❌ Groovy Snapshot Canary Build / Build Grails (shard 0) Test SummaryTip Inspect all details with the TestLens CLI: CI / Build Grails-Core (macOS JDK 21) > :grails-async-groovy-concurrent:test
🏷️ Commit: 26fb584 Test FailuresAwaitablePromiseFactorySpec > single and aggregate error callbacks receive the same original failure > single and aggregate error callbacks receive the same original failure [checked: true, #1] (:grails-async-groovy-concurrent:test in CI / Build Grails-Core (macOS JDK 21))Rerun ControlsSelect tests to mute in this pull request:
Reuse successful test results:
Click the checkbox to trigger a rerun:
Learn more about TestLens at testlens.app/docs. |
jamesfredley
left a comment
There was a problem hiding this comment.
The items this review found have been addressed. A repeated type is removed from its own place, so the written type and the format agree.
Follow-ups to #16996 (
grails.mime.negotiation.spring).An unconfigured
application/problem+xmlis one of the XML types a+xmltype comes ahead ofThe Grails rules read
application/problem+xml, where no type ofgrails.mime.typesnames it, asapplication/xml, so a later+xmltype of equal quality comes ahead of it:Accept: application/problem+xml, application/atom+xmlis theatomformat. With the setting, the+xmlpreference looked only forapplication/xmlandtext/xml, so the same header wasxml. It now takes an unconfiguredapplication/problem+xmlas an XML type too, for the Grails formats and for what Spring MVC writes for a request a URL mapping handles.As in
DefaultAcceptHeaderParser, such a problem type is one of the XML types: a+xmltype comes ahead of it, but it is never moved ahead of another XML type itself.Accept: application/xml, application/atom+xml, application/problem+xmlisatom, and Spring MVC writesapplication/atom+xml, as does the upgrade note'stext/xml, application/atom+xml, application/problem+xml.What counts as configured is what
DefaultAcceptHeaderParserchecks: a type ofgrails.mime.typeswith that name, ignoring case. Wheregrails.mime.typesnamesapplication/problem+xml, the header is theproblemformat and keeps its order. Aproblemmapping inspring.mvc.contentnegotiation.media-typesalone leaves itatom, withapplication/atom+xmlahead ofapplication/problem+xmlfor Spring MVC, because Grails still reads the type as XML. Keeping the header's order there would have Spring MVC writeproblem+xmlwhileresponse.formatandwithFormatansweratom.The format and the type Spring MVC writes agree for a repeated
+xmltypeFor a request a URL mapping handles, the
+xmlpreference runs twice: once for what Spring MVC writes and once when Grails resolves its formats. As #16996 brought it in, it took each+xmltype behind the XML type out of the list as the first equal type, so where the header repeats a type it removed the copy ahead of the XML type, and the second pass reordered again. With the defaultgrails.mime.types,Accept: application/atom+xml, application/xml, application/atom+xmlhad Spring MVC writeapplication/xmlwhile the format wasatom, andapplication/xml, application/atom+xml, application/atom+xmlwas thexmlformat. Each type is now moved from its own place, so after one pass no+xmltype of the quality of the XML type is left behind it, a second pass changes nothing, and both headers areatom, with Spring MVC writingapplication/atom+xml.Tests of the
+xmlpreferenceSpringFormatNegotiationSpeccovers the problem type at equal and at lower quality, withproblem+xmlconfigured ingrails.mime.types, and with it mapped in Spring MVC only. In the second case each of these headers has the same format and written type:application/problem+xml, application/atom+xml,application/xml, application/atom+xml, application/problem+xml,text/xml, application/atom+xml, application/problem+xml,application/xml, application/problem+xml,application/problem+xml, application/xml, application/atom+xml,application/atom+xml, application/problem+xml, application/atom+xmlandapplication/problem+xml, application/atom+xml, application/atom+xml. The repeated type is also covered with the default types.FormatNegotiationComparisonSpecsendsapplication/problem+xml, application/atom+xml,application/xml, application/atom+xml, application/problem+xmlandapplication/atom+xml, application/xml, application/atom+xmlunder every configuration it compares: the Grails formats agree, and only what Spring MVC writes differs, as it already does forapplication/xml, application/atom+xml. The problem type was found while flipping the default for Grails 10, whereWithFormatSpec's Grails 8 feature forapplication/problem+xml, application/atom+xmlfailed.The format fallback takes every format Spring MVC's manager maps
From @matrei's approving review of #16996 (optional point 1) and the review of this pull request. A format that
grails.mime.typesdoes not list is written in the type Spring MVC's content negotiation manager maps it to. The guide, the upgrade notes, the upgrade skill and the Javadoc said that fallback takes a formatspring.mvc.contentnegotiation.media-typesmaps. It takes any format the manager maps:json,xml,hal,atom,rss,csvandtextthatgrails.mime.typeslists;spring.mvc.contentnegotiation.media-typesadds;json.The lookup ignores case, which
grails.mime.typesdoes not. So?format=JSONand?format=HALare written asapplication/jsonandapplication/hal+jsonfor what Spring MVC writes, while Grails resolves no format for them andrespondanswers406.The wording is corrected in
contentNegotiation.adoc,upgrading90x.adoc, thegrails-upgradeskill, and the Javadoc ofGrailsContentNegotiationStrategy.springMediaTypesOfandGrailsFormatContentNegotiationStrategy. Spring MVC's defaults are named by example rather than listed, since they depend on the converters. The upgrade note limits the point to a format only the manager maps, and links the format resolution section through a newupgradingGrailsFormatResolutionanchor.SpringFormatNegotiationSpecpins the cases:grails.mime.typeslackingjson,?format=jsonand?format=JSONresolve toapplication/json;grails.mime.types,?format=HAL,?format=CSVand?format=Textresolve toapplication/hal+json,text/csvandtext/plain, with no Grails format.:grails-mimetypes:test, itscodeStyleand:grails-doc:publishGuidepass, as do the negotiation specs ofgrails-test-suite-webandgrails-xml.Merge order
#17106, which makes the setting the default on 10.0.x, needs the
problem+xmlfix: merge this pull request, then merge 9.0.x up through 9.1.x into 10.0.x, and #17106 can then be marked ready for review.