feat: regenerate the syntax from the tree-sitter grammar dictionary#2
Merged
Conversation
Rewrites the plugin around keywords.json, a pinned copy of the dictionary the tree-sitter-gnuplot grammar emits, so both projects recognise the same 724 keywords with the same abbreviation minimums. The abbreviation system now costs no regexes at all. Vim's `syn keyword` already understands `xran[ge]` as "xran plus an optional tail", which is exactly what a (keyword, min_chars) pair means, so the whole table becomes plain keyword lists. The previous file expressed the same idea with variable-width lookbehind — patterns like `\v(((un)?set|show) .*)@<=(<angles>)` — roughly 150 of which are replaced by generated lists. Two bugs the rewrite removes: - ftdetect never worked. `au ... *.pal, *.plt, ...` is a comma-separated pattern list, so the space after each comma started a new pattern; Vim parsed the remainder as a command and raised `E20: Mark not set`. No filetype was set for any extension. - Four `syn match` patterns opened with `"\n(` instead of `"\v(`, so they matched a literal newline and never fired: two style boxplot rules, one terminal-name rule and one png option rule. Also adds ftplugin (commentstring, comments, formatoptions, iskeyword), datablock and format-specifier highlighting, `:help gnuplot`, and a headless test suite built on Vim's own assert_* that runs under both Vim and Neovim. Groups are emitted in reverse-priority order because Vim resolves a keyword defined twice to the last definition: `p` is both `p[lot]` and `p[oints]`, and the command verb is the more valuable of the two to get right.
vint reports thirteen style problems: autocmds outside an augroup, the abbreviated `cpo` in place of `cpoptions`, a missing `scriptencoding` in files containing multibyte characters, and `nocompatible` in the test runner. All are fixed. The job itself also needed pinning. vint imports pkg_resources, which setuptools removed in 81 and Python no longer bundles from 3.13 on, so `python-version: 3.x` resolved to an interpreter where vint cannot start — the failure was environmental rather than a lint result.
Honouring every gnuplot abbreviation turned most of the alphabet into a keyword: fifty-one of them reduce to a single letter, so `a = 42`, a loop counter `i` and a variable named `w` all lit up as keywords. Abbreviations are now opt-in, limited to the forms that are near-universal in real scripts — plot, splot, replot, terminal, output, using, with, title, notitle and the three `with` values. Everything else matches as a whole word. Four bugs found while comparing the old and new files token by token: - `transparent`, `extend` and `start` never registered at all. Vim reads trailing words on a `:syn keyword` line as syntax ARGUMENTS, and all three are spelled like one, so they were silently swallowed. They are emitted as matches instead. - The datablock region never opened, leaving `$data << EOD` bodies to be highlighted as code. A `$name` match defined after the region out-ranked its start pattern, since Vim prefers the later item at the same position. - Style attribute short spellings (`ps`, and any other whose long form also names a set/show option) were dropped: the generator read `alt` off the winning tier entry alone, and `pointsize` resolves to the option tier, which carries no short form. `alt` and `no` now merge across every entry. - `,` was not an operator, and `$` needs its own match for column references rather than joining the operator class, where it would out-rank datablocks. Adds the sub-keywords the upstream dictionary skips on purpose, since the parse tables already describe them: rgb/rgbcolor, fill, separator, via, name, nooutput, background, above/below/closed/between and shell. Coverage over a representative script goes from 83.9% to 91.6% of non-whitespace characters, with no keyword now claiming a bare variable name.
Under Neovim the highlight links now point at the same captures queries/highlights.scm uses in tree-sitter-gnuplot — @Keyword for a command verb, @variable.member for an option or suboption, @keyword.directive for a toggle, @constant for an enumerated value, @Property for a plot-element modifier, @Attribute for a plot style — so a buffer highlighted by this plugin and one highlighted by the grammar take identical colours from the colorscheme. Neovim defines defaults for every standard capture, so this holds with no parser installed. Vim keeps the standard groups. Measured by resolving every character's foreground twice under one colorscheme, once through the grammar and once through this file, agreement went from 84.5% to 92.5% of the characters both highlight. Closing the gap needed the groups the plugin had no equivalent for at all: - brackets (@punctuation.bracket) were not matched - connectors at/via/from/to/by (@keyword.function) - the `variable` style slot (@variable.parameter) and the named colours viridis/black/bgnd/background (@variable.parameter.builtin) - newhistogram/newspiderplot/keyentry (@Keyword) - `eq`/`ne` split from the symbolic operators (@keyword.operator) - a datablock reference (@module) split from the heredoc delimiters (@Label) - an open range end `[*:*]` as @character.special rather than multiplication and three misclassifications: fill words such as `solid` and `transparent` are @Attribute rather than enumerated values, `nooutput` is a toggle, and a bare `title columnheader` is the builtin function. The remaining difference is two deliberate choices: a datablock body stays inert, and `!` marks only the shell escape because Vim cannot inject bash.
The section still described every keyword as abbreviable, which stopped being true when abbreviations became opt-in.
…pace Rewriting the ftdetect comment introduced a multibyte character, which vint requires a scriptencoding declaration for. The comment now also states why Neovim goes through vim.filetype.add() rather than an autocmd: ftdetect scripts are sourced after the builtin detection, and :setfiletype does nothing once a filetype is set.
`vim -es` discards :echo, so a failing assertion produced exit code 1 and no output at all in CI. Failures now go to stderr, which both editors pass through.
Vim's runtime assigns .gp to PARI/GP, and :setfiletype does nothing once a filetype is set, so the extension stayed PARI/GP under Vim while the Neovim side already claimed it through vim.filetype.add(). Vim now assigns it outright, which is what installing a gnuplot plugin implies and keeps the two editors consistent. Every other extension still yields to whatever detected first.
`.gp` is assigned to PARI/GP by both stock runtimes. Claiming it meant forcing the filetype under Vim and registering ahead of the runtime under Neovim, which takes the extension away from another language for everyone who installs this plugin. Detection now yields it, and the help file shows the one-liner for claiming it in a personal config instead. Also drops the note about `.pal`. Palette files with that extension are a local habit rather than anything gnuplot defines, so the plugin has no opinion worth documenting.
Merging a Conventional-Commit PR to main now opens a release PR carrying the CHANGELOG and the version bump; merging that creates the tag and the GitHub Release. Nothing is published — plugin managers install from the git repository, so the tag is the release. That lets people pin a version instead of tracking main. `release-type: simple`, since this is not a package: there is no manifest to bump beyond version.txt, and none of the registry machinery the grammar repo needs. CITATION.cff is listed under `extra-files` and carries `# x-release-please-version` and `# x-release-please-date` on its version and date lines, so `Cite this repository` tracks the latest tag. Listing it is the part that matters: a file absent from `extra-files` is silently never updated. tree-sitter-gnuplot has that exact bug today, with its CITATION.cff pinned at 2.0.4 while the manifest reads 3.0.0. The App token falls back to GITHUB_TOKEN when the secrets are absent; the release still works, the release PR just carries no checks and needs an admin merge.
Checked against the CFF 1.2.0 schema: `url` is defined as a landing page, which here was just the repository again, so `repository-code` alone says it. Adds the author's GitHub handle as `alias`, a `vim-plugin` keyword, and an abstract that says what the plugin actually does rather than restating the title. Validates clean against the published schema. An ORCID on the author entry and a DOI under `identifiers` are the two remaining additions that would improve citability; both need information this repository does not have.
Ties the software to a persistent researcher identifier, which matters here because a Spanish double surname is indexed inconsistently: López Jacinto, D. and López, D.J. and Jacinto, D.L. would otherwise become separate author profiles. The schema requires the full URI form rather than the bare identifier.
keywords.json is a vendored copy, and nothing noticed when its source changed: CI only checked that the syntax file matched the local copy, so the pair could stay perfectly consistent with each other while drifting from the grammar. The plugin would just quietly stop knowing about new gnuplot keywords. A weekly job now compares the two and opens a PR when they differ, regenerating the syntax file in the same commit. The PR runs normal CI, so a sync that breaks the generated-file check or the assertions cannot merge, and the fetch is validated before it overwrites anything so a truncated or error response cannot land. It follows the grammar's default branch rather than its releases. The dictionary is generated data about gnuplot, not an API surface, so newer means better coverage — and releases lag: v3.0.0 carries 772 entries against main's 794, so tracking tags would have rolled the plugin backwards on its first run.
keywords.json only covers tokens the grammar resolves through its scanner
tiers; its generator deliberately skips self-named key() aliases and bare
string literals because the parse tables already describe them. Those words
therefore never reach the generated block, and nothing noticed that the
hand-maintained list standing in for them had fallen well behind.
Mining every STRING and PATTERN terminal out of the grammar's grammar.json
(545 words, after expanding the abbreviation regexes) and subtracting what
this file already names left 99 unhighlighted: the pm3d lighting and
corners2color bodies, the binary and matrix datafile modifiers, the plot
element filters, the axis names, and assorted option heads. They are added
here by hand, grouped by where the grammar puts them.
Four of the 99 are left out on purpose. bmargins/lmargins/rmargins/tmargins
are artifacts of expanding the margin family's abbreviation regex to its
maximal form; gnuplot's spelling is singular, which the dictionary already
carries.
Two words need more than a keyword line. `skip` and `end` are spelled like
:syn arguments, so they are matches, the same workaround the generated block
applies to `extend` and `start`. `system` is both a command and a string
function: a keyword would out-rank the function match and break
`system("ls")`, so it is a match with a lookahead that excludes a call.
Also completes the number rule's unit suffixes to the grammar's full
typographic set, with `inch` ordered before `in` so the longer spelling can
match at all.
Abbreviations are untouched. `contour`, `cluster`, `wall`, `isosample`,
`zerror`, `whiskerbar` and `norangelimit` still need their maximal spelling,
since honouring them means relaxing the allowlist for every keyword at once.
The repo has no tags yet, so release-please would fall back to
initial-version. Pinning it here instead makes the first release exactly
1.0.0 regardless of how that bootstrap path resolves.
Release-As: 1.0.0
The report block writes to '/dev/stderr', which does not exist on Windows. writefile() fails there with E482, and because that error aborts the script, the `qall!` on the next line never runs — so the headless editor sits there until GitHub kills the job at its six-hour limit: Error in command line..script D:\a\gnuplot.vim\gnuplot.vim\tests\run.vim: E482: Can't open file /dev/stderr for writing: no such file or directory Nvim: Caught deadly signal 'SIGINT' The error lands six seconds into the run, after every assertion has already passed, so this was never a real test failure — the reporting line alone was burning six hours of Windows runner time per job and leaving the two windows-latest checks permanently pending. No run on this branch has ever completed them. writefile() has no Windows path that reaches stderr, so keep it on Unix, where -es genuinely needs it, and fall back to :echomsg elsewhere. Vim's silent-ex mode discards that too, so on Windows+vim the text is lost and only the exit code survives — which is what fails the job anyway. Neovim prints it. Verified by forcing the fallback branch: both editors exit 0 on success and 1 on failure, neither hangs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds the plugin around
keywords.json, a pinned copy of the dictionary thetree-sitter-gnuplot grammar
emits, so both projects recognise the same 724 keywords with the same
abbreviation minimums.
No regexes for the abbreviations
Vim's
syn keywordalready understandsxran[ge]as "xran plus an optionaltail", which is exactly what a (keyword, min_chars) pair means. The whole
abbreviation table therefore becomes plain keyword lists. Roughly 150
variable-width lookbehind patterns of the shape
are gone, replaced by generated lists. CI regenerates and diffs the file, so
the committed block cannot drift from the dictionary.
Two bugs removed
Filetype detection never worked.
au BufNewFile,BufRead *.pal, *.plt, …isa comma-separated pattern list, so the space after each comma began a new
pattern and Vim parsed the remainder as a command:
No filetype was set, for any extension.
Four
syn matchpatterns opened with"\n(instead of"\v(, so theylooked for a literal newline and never fired: two
set style boxplotrules,one terminal-name rule and one
pngoption rule.Also added
ftplugin/gnuplot.vim— commentstring, comments, formatoptions, iskeyword,with
b:undo_ftplugin$data << EOD), format specifiers inside strings, andgnuplot's unit-suffixed numbers (
10cm,3.0in):help gnuplotassert_*, running under both Vim andNeovim across Linux, macOS and Windows
conventional-title check, and the community health files
Notes
Groups are emitted in reverse-priority order because Vim resolves a keyword
defined twice to the last definition.
pis bothp[lot]andp[oints]; thecommand verb wins. The consequences of context-free keyword matching are listed
under "Known limits" in the README.
set xrgoes unhighlighted because the dictionary recordsxrangewith aminimum of four characters while gnuplot accepts two. That is an upstream
property, left to be fixed there rather than patched in generated output.