Automatic embeding of latest version numbers of modules in docs through special syntax

[silviubogan]

but the replacement syntax used works only on full pages such as the home page or the system requirements page, and this is a problem because we should be able to use the replacements in files in the _inc directory too (currently, when using them there, we get either no content visible for what we want or, in code blocks, the replacement not done at all

• I signed and returned the Plone Contributor Agreement, and received and accepted an invitation to join a team in the Plone GitHub organization.
• I verified there aren't other open pull requests for the same change.
• I followed the guidelines in Contributing to Aurora.
• I succesfully ran code linting checks on my changes locally.
• I succesfully ran unit tests on my changes locally.
• I succesfully ran acceptance tests on my changes locally.
• If needed, I added new tests for my changes.
• If needed, I added documentation for my changes, either in the Storybook or narrative documentation.
• I included a change log entry in my commits.

---

• Bug: the replacement feature works well with the latest versions as retrieved from the GH API, but only on full pages in the docs, not in partial content inside the docs/_inc directory.
  • The source-read event from Sphinx is called before the {include} Sphinx directive is applied to include the _incs.
• Disable cache for docname that contains one of the {{version smth}} syntaxes (the issue was that the _inc files were not generating rebuild on change)
• Make the specified change in System requirements should be updated #179 in the System Requirements page of the docs, that specify the latest NVM release's version.
• The GH API should be used to not only retrieve the latest release, but the latest stable release, because we might use this new feature for other modules with different release names. (already done because the used GH API endpoint already returns only the latest stable release)
• Solve the other problem in System requirements should be updated #179 . (it is already solved, the issue author did not use the recommended LTS version of Node.js)

Closes #179.

[silviubogan]

@stevepiercy I am currently chewing the idea that the syntax used to import the _incs is not ideal.

[silviubogan]

@stevepiercy I also experimented with standard MyST Parser substitutions (Janji2). The advantage is that we get working substitutions in _inc parts (although not in code blocks, which is a requirement). And the disadvantage is that the syntax is uglier, we have to translate the string version, the owner and the name of the GH repo, all three in a single variable name.

[stevepiercy]

@silviubogan any script should also work with plone/documentation and any other Plone documentation for maintainability and consistency. The latest commit appears not to do that and goes a different route.

I also don't know whether a separate script outside of conf.py will help with maintainability.

Finally, there are failures in the CI checks that need to be addressed.

Please let me know. Thank you!

[stevepiercy]

Please see my comment at #183 (comment).

Eventually this will be merged into plone/documentation, and I don't want to wipe out its current functionality. This PR would do that. Is there a way to ensure compatibility between the two?

[silviubogan]

@stevepiercy

Please see my comment at #183 (comment).

Eventually this will be merged into plone/documentation, and I don't want to wipe out its current functionality. This PR would do that. Is there a way to ensure compatibility between the two?

Currently, the new substitution syntax is implemented in a separate Python module, next to conf.py. This module can be reorganized into a Sphinx plugin.

[silviubogan]

@stevepiercy There is no package manager available for Sphinx plugins. Can we use a simple copy-paste? Should we use a separate GitHub repo as a template for it?

[stevepiercy]

@silviubogan let's keep it simple. A copy-paste is sufficient. Although Sphinx extensions can be Python packages, for something this basic I think that making it a package would be more work than copy-paste.

My greater concern is whether the existing replacements in plone/documentation would integrate with this modification. I want to reuse the method from there in this modification. Please take a look at the original and how it's handled there.

https://github.com/plone/documentation/blob/dbbaca6b070cc98ef2ffcd391597241a8842c6e9/docs/conf.py

[silviubogan]

@stevepiercy

The failures in the CI checks are there because the new Python module is not found. In the latest commit I've put:

- name: Print Python module paths
  run: python3 -c "import sys;print(sys.path)"

It gives the output:

['', '/opt/hostedtoolcache/Python/3.12.14/x64/lib/python312.zip', '/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12', '/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/lib-dynload', '/opt/hostedtoolcache/Python/3.12.14/x64/lib/python3.12/site-packages']

On my laptop, '' lets Python see the new module.

Do you have any advice?

[silviubogan]

@stevepiercy Bug above solved. Now another CI failure: https://github.com/silviubogan/aurora/actions/runs/36948968346/job/110657400436.

The issue is that the coockieplone GH workflow clones the upstream repo, not the fork.
The output of the workflow, near the error, is:

Cloning core from https://github.com/plone/aurora.git...
✓ cloned core at core
✓ update core to tag 1.0.0-alpha.15
fatal: couldn't find remote ref system-requirements-should-be-updated

I searched the web but did not find how to get the fork owner and name, but I believe this part of the action/checkout action docs should work, if we have the correct info:

- uses: actions/checkout@v7
  with:
    # Repository name with owner. For example, actions/checkout
    # Default: ${{ github.repository }}
    repository: ''

[silviubogan]

@stevepiercy

My greater concern is whether the existing replacements in plone/documentation would integrate with this modification. I want to reuse the method from there in this modification. Please take a look at the original and how it's handled there.

https://github.com/plone/documentation/blob/dbbaca6b070cc98ef2ffcd391597241a8842c6e9/docs/conf.py

Yes, they would integrate. Just uncomment these:

# import os
# import sys
# sys.path.insert(0, os.path.abspath("."))

and add "latest_gh_version_substitution" to the list "extensions".

The "myst_enable_extensions" list contains "substitution", but these substitutions are not as flexible as we need: they do not have parameters.

source_replace and source_replacements work well, but they use a very limited regex. My Sphinx extension has grown from these.

[stevepiercy]

A few formatting suggestions, and it's good to merge.

[stevepiercy]

@silviubogan I tried this in plone/documentation as you said, and it worked perfectly. Very cool!

That test helped me better understand what's going on, too. This is a complement to existing replacements. Would you like to make a PR in both the Volto and Documentation repos as well?

It could also be a nice Python package, maybe in the collective organization, if you want to go that route.

Please let me know. Thank you!

[silviubogan]

@stevepiercy CI fails only in the fork. After plone/aurora CI is done for the PR, the PR shows "All checks have passed".

[silviubogan]

@stevepiercy

Would you like to make a PR in both the Volto and Documentation repos as well?

Yes.

It could also be a nice Python package, maybe in the collective organization, if you want to go that route.

Yes.