Skip to content

tags.rb: substitute AsciiDoc attributes in tagged inline text - #260

Merged
Bill Traynor (wmat) merged 2 commits into
mainfrom
169-substitute-asciidoc-attributes-in-tagged-text-without-rewriting-other-inline-markup
Sep 1, 2026
Merged

Bill Traynor (wmat) merged 2 commits into
mainfrom
169-substitute-asciidoc-attributes-in-tagged-text-without-rewriting-other-inline-markup

Conversation

@wmat

@wmat Bill Traynor (wmat) commented Sep 1, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #169.

Two commits: the attribute-substitution fix, and the removal of
detect_tag_changes.py per Jordan Carlin (@jordancarlin)'s review.

1. The problem

Asciidoctor applies substitutions at the block level, in this order:
specialcharacters, quotes, attributes, replacements, macros,
post_replacements. Inline nodes are produced by the quotes pass, so their
text has only had specialcharacters applied — an attribute reference such as
{ge} reached the tags JSON unresolved, and there is no built-in way to ask an
inline node for its substituted text.

PR #131 fixed that by replaying the parent block's whole sub list over the
inline text. That also ran the three passes after attributes, which is what
made it a net loss and got it reverted in PR #170:

  • replacements rewrote apostrophes to ’
  • [.underline] was dropped
  • <<xref>> cross references were deleted outright

The fix applies only the attributes substitution. Nested formatting is
unaffected, because bold, italics, underline and friends are separate inline
nodes that reach the converter in their own right — they never pass through
this code path as raw text.

Verified against riscv-isa-manual

Built make build-tags on riscv-isa-manual at main with the submodule on
each revision. No warnings under --failure-level=WARN. Of 2923 tags, 38
change, all of them attribute expansions
; the tag set is identical and
nothing else moves. Cross references survive (&lt;&lt;translation&gt;&gt; in
norm:sfence_vma_asid_only), apostrophes stay literal (two's, entry i's),
and superscripts and insn:/csr:/ext: macros are untouched:

norm:elen    ELEN {ge} 8       ->  ELEN ≥ 8
norm:vwsll   2{times}SEW       ->  2×SEW
norm:c-j_op  a {pm}2 KiB range ->  a ±2 KiB range

A note on escaping

{ge} expands to ≥ because symbols.adoc defines it as literal Unicode.
{endash}, {emdash} and {asterisk} are defined as character references
instead, so they expand to the &amp;-escaped form — &amp;#8205;&amp;#8211;&amp;#8205;.
That looks like over-escaping but is the consistent result: every string this
backend emits sits at the post-specialcharacters stage (&lt;&lt;, &amp;),
and an author-written &#8805; already produces &amp;#8805; today — see
norm:unicode_decimal_value. Both spellings render identically through
create_normative_rules.py.

Tests

Three cases added to tests/norm-rule/test-ch1.adoc, pinning all three
regressions from #169 plus the attribute-missing default:

"norm:attribute_in_formatting":  "&amp;#8805; and &amp;#8805;"
"norm:attribute_with_hyperlink": "It isn't 100, it's &amp;#8805; &lt;&lt;norm:superscript&gt;&gt;"
"norm:attribute_undefined":      "200 {no-such-attribute} 100"

Attribute expands through nested formatting; apostrophe stays literal; cross
reference survives; an undefined attribute is left alone.

2. Deleting detect_tag_changes.py

riscv/riscv-isa-manual#3334 removed the only consumer — the
check-normative-tags workflow, the committed ref/riscv-spec-norm-tags.json
and the supporting scripts — because the reference copy existed only to raise
automated issues, and those are no longer wanted. Nothing else calls the
script: no importers, no workflow here, and the only remaining references were
this Makefile and the tools README.

Normative rules are still built in CI and shipped in every release. That path
goes through create_normative_rules.py and is untouched.

This also retires test-tag-changes-update, which has been failing since
PR #248 removed the --update-reference flag the target still passed.

Result

make test exits 0 from a clean build/ — 16 assertions, 0 failures, no make
errors, for the first time since #248.

🤖 Generated with Claude Code

Asciidoctor applies substitutions at the block level in the order
specialcharacters, quotes, attributes, replacements, macros,
post_replacements. Inline nodes are produced by the quotes pass, so their
text has only had specialcharacters applied and an attribute reference such
as `{ge}` reached the tags JSON unresolved.

PR #131 fixed that by replaying the parent block's whole sub list over the
inline text, which also ran the three passes after `attributes`: apostrophes
became `&#8217;`, `[.underline]` was dropped, and `<<xref>>` cross references
were deleted outright. That was a net loss and was reverted in PR #170.

Apply only the attributes substitution instead. Nested formatting is
unaffected because bold, italics, underline and friends are separate inline
nodes that reach the converter in their own right.

The expanded value keeps the post-specialcharacters escaping the rest of this
file already uses, so `:ge: &#8805;` yields `&amp;#8805;` exactly as an
author-written `&#8805;` does today (see norm:unicode_decimal_value), and
create_normative_rules.py renders both as the same character.

Adds three test cases covering attribute expansion inside nested formatting,
attribute expansion alongside an apostrophe and a cross reference, and an
undefined attribute being left alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Bill Traynor <wmat@greenmonkeylabs.com>
@wmat

Copy link
Copy Markdown
Collaborator Author

Jordan Carlin (@jordancarlin) would you mind checking this one? I think both James and Tim have moved on from this.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think long term we should think about simplifying a lot of this, but this seems fine for now.

Comment thread tools/detect_tag_changes.py Outdated
@wmat
Bill Traynor (wmat) force-pushed the 169-substitute-asciidoc-attributes-in-tagged-text-without-rewriting-other-inline-markup branch from 65f13a8 to c4bb4b5 Compare September 1, 2026 20:38
riscv-isa-manual removed the only consumer in riscv/riscv-isa-manual#3334:
the check-normative-tags workflow, the committed ref/riscv-spec-norm-tags.json
and the supporting scripts all went, because the reference copy existed only
to raise automated issues and those are no longer wanted. Nothing else calls
the script - it has no importers, appears in no workflow here, and the only
remaining references were this Makefile and the tools README. Normative rules
are still built in CI and shipped in every release; that path goes through
create_normative_rules.py and is untouched.

This also retires test-tag-changes-update, which has been failing since PR #248
removed the --update-reference flag the target still passed. `make test` now
exits 0.

Recoverable from history if the review aid is ever wanted back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Bill Traynor <wmat@greenmonkeylabs.com>
@wmat
Bill Traynor (wmat) merged commit b5df347 into main Sep 1, 2026
7 checks passed
@wmat
Bill Traynor (wmat) deleted the 169-substitute-asciidoc-attributes-in-tagged-text-without-rewriting-other-inline-markup branch September 1, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Substitute AsciiDoc attributes in tagged text without rewriting other inline markup

2 participants