Add langstring support to the metamodel

[niegrzybkowski]

Issue linkml/linkml#3548 and a dozen others

This PR adds the definitions and the documentation of a new langstring type which will provide the necessary infrastructure for implementing multi-lingual support for schema metadata. This PR does not yet change the metamodel metadata slots to use the type - but title and description are marked as candidates.

Justification
Lack of first class support for multilinguality in LinkML has been a big pain point for us. Adding external tooling would work for now, and I can see it being a useful feature on its own, but this is something that should be enabled by the metamodel, not done in spite of it. Multi-linguality is a hard requirement in the EU, often enforced by law. Web has moved on to being multi-lingual by default: multi-linguality is required by W3C in all standards. For LinkML to be useful for ontology writing, this is badly needed.

Functionality
langstring values are a union of plain strings (non-language-tagged), and multi-language strings (language tagged, with at least one language). plain strings are treated as such - they do not specify language information and are assumed to always be something that should be displayed to the user. multi-language strings are a mapping of a language tag to the appropriate text, localized in the specified language.

Serializations
Copied from the notes:

• In JSON serializations, plain strings should be mapped to JSON strings, and multi-language strings should be represented as a dictionary object of a language tag key to a text value in that language.
• In RDF serializations, plain strings should be mapped to xsd:string, and multi-language strings to rdf:langString with appropriate language tags.

Examples:

• JSON "some text" becomes RDF "some text"^^xsd:string
• JSON { "en": "some text", "pl": "jakiś tekst" } becomes RDF "some text"@en and "jakiś tekst"@pl

Runtime behavior
Copied from the notes:

• plain() yields a string that is populated from: (1) the plain text if available, (2) the English text if available, or (3) the text of the alphabetically first language tag.
• in_language(tag) yields a string that is populated from: (1) the plain text if available, (2) the text of the specified language if available, (3) the English text if available, or (4) the text of the alphabetically first language tag.

Rationale for making langstring a magic type
We've seen the approaches proposed by linkml-adjacent projects, but we have opted to go for the simplest option that would work for metadata. Given the constraints:

• not break every single LinkML schema in existence,
• not force people to use multi-lingual metadata if they don't need it,
• provide generators a clean migration path.

To implement something that would satisfy all of these constraints generically (while allowing it to be used in "regular" schemas as well) would require a lot more effort in the metamodel, which would probably not be useful outside of this specific application. Let's not add 5 new boolean slots for this

Additional context
We have this implemented fully in LinkML-Scala. See the demo in the playground here and the code here

I think an alternative approach would be to make langstrings a magic class like linkml:Any, as opposed to a type, which would preserve the "types are always atomic values" assumption.

Future considerations
I think in_language slot should control the language of the plain string, but it would require careful scoping and other considerations for this to work well. This would allow for example

in_language: en
title: My thing

to mean "My thing"@en in RDF

[satra]

ping @djarecka @puja-trivedi (also see the linkml playground)

[ddooley]

I see how this kind of option could be useful in data holdings. I'll just note that for the schema itself we used the LinkML schema.extensions structure to enable an overlay of all textual content of a schema, such that after loading a schema into an application, a given language overlay can be placed right on top of it to facilitate locale-friendly applications.

schema...
  classes ...
    CanCOGeNCovid19 ...
  slots ...
  ...
  extensions:
    locales:
      tag: locales
      value:
        fr:
          id: https://example.com/CanCOGeN_Covid-19
          name: CanCOGeN_Covid-19
          version: 3.0.0
          in_language: fr
          classes:
            CanCOGeNCovid19:
              title: [French variation of title]
              description: [French variation of description]
              see_also:
                - templates/canada_covid19/SOP_fr.pdf  [French variation on URL content]
                ...