Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/build-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,6 @@ jobs:

- name: Build docs
run: npm run docs:build

- name: Check the build output and the docs for citation markers
run: npm run check:citation-markers
2 changes: 2 additions & 0 deletions dev-docs/BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ The generator lives in `docs/script/` rather than beside the engine build script

Both are git-ignored, and both are regenerated as the first step of `docs:dev` and `docs:build`. `docs:generate-function-docs` is a gate, not a formatter: it fails the build on a missing catalogue entry or a `'Custom'` category. See [`FUNCTION-CATALOGUE.md`](../hyperformula/dev-docs/FUNCTION-CATALOGUE.md).

After `docs:build`, the `build-docs` workflow runs `npm run check:citation-markers`. It fails when a citation marker from a draft document (the patterns are listed in the script) appears in the engine's build output or in `docs/api/` and `docs/guide/`, and when one of those directories is missing or empty.

## Deploying the portal

The portal is served by a Cloudflare Worker, and **deployment is driven by Workers Builds on push, not from a developer's machine**: `master` deploys to production, every other branch uploads a preview version. The commands below exist for debugging that pipeline. The triggers, the Worker name, and the dashboard build settings are in [`docs/README.md`](../docs/README.md#deployment).
Expand Down
2 changes: 1 addition & 1 deletion dev-docs/STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Each script lives with whatever invokes it, and every one has exactly one caller

| Directory | Holds |
|---|---|
| `script/` | `release/` and `check-licenses.mjs` — both span the whole repository |
| `script/` | `release/`, `check-licenses.mjs` and `check-citation-markers.mjs` — all three span the whole repository |
| `hyperformula/script/` | `check-file.js`, `check-publish-package.js`, `if-ne-env.js` — called by the engine's build; `check-license-key-parser-drift.js` — called by `vendored-parser.yml`; `postinstall-skills-message.js` — shipped in the package and run by npm when a user installs it, silent inside this repository |
| `docs/script/` | the built-in-functions generator and `prepare-cf-assets.js` — called by the portal |
| `hyperformula-ui/scripts/` | `generateCssText.mjs` and `generateApiDocs.mjs` — called by the UI core's `build:styles` and `docs` |
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"docs:preview:cf": "npm --prefix docs run preview:cf --",
"audit": "npm audit --omit=dev",
"check:licenses": "node script/check-licenses.mjs",
"check:citation-markers": "node script/check-citation-markers.mjs",
"release": "bash script/release/release.sh"
},
"devDependencies": {
Expand Down
3 changes: 2 additions & 1 deletion script/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ Repository-level scripts. Each package keeps its own in `hyperformula/script/` a
|---|---|---|
| `check-licenses.mjs` | `npm run check:licenses` | Asserts every production dependency is permissively licensed |
| `release/` | `npm run release` | The release procedure, across the packages and the sibling repositories |
| `check-citation-markers.mjs` | `npm run check:citation-markers`, which `build-docs.yml` runs after `docs:build` | Fails when a citation marker from a draft document appears in the engine's build output or the docs |

- What the build produces, and where the licence gate fits: [`dev-docs/BUILD.md`](../dev-docs/BUILD.md)
- Everything else: [`dev-docs/README.md`](../dev-docs/README.md)

No workflow runs these; they are run by hand from the repository root. `release/release.sh` needs an operator at a terminal on purpose — after `--real-run`, its typed confirmation is the last guard before `npm publish` — so never make it run without one. Nothing here is covered by tests; a change is verified by running it.
Only `check-citation-markers.mjs` runs in CI; the others are run by hand from the repository root. `release/release.sh` needs an operator at a terminal on purpose — after `--real-run`, its typed confirmation is the last guard before `npm publish` — so never make it run without one. Nothing here is covered by tests; a change is verified by running it.
109 changes: 109 additions & 0 deletions script/check-citation-markers.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
#!/usr/bin/env node
/**
* Fails when a citation marker from a draft document appears in the engine's
* build output or in the documentation.
*
* The markers are `[V<n>]`, `[<prefix>_<n>]` with the prefix `vrf`, `dec`,
* `con`, `que`, `wrg` or `crf`, and the footer heading `§AuditSources`. A source
* comment survives into `commonjs/`, `es/`, the development bundle in `dist/`
* and the `sourcesContent` of the source maps in `dist/`, so every file of the
* output is scanned, not only the minified bundles.
*
* Run it after `npm run docs:build`, which builds both the engine and the docs.
* A path that is missing or holds no files fails the check instead of being
* skipped, so a moved output directory cannot turn it into a scan of nothing.
*/
import { readdirSync, readFileSync, statSync } from 'node:fs'
import path from 'node:path'
import { fileURLToPath } from 'node:url'

const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..')

/** The engine's build output and the documentation, relative to the repository root. */
const SCANNED = [
'hyperformula/commonjs',
'hyperformula/dist',
'hyperformula/es',
'hyperformula/languages',
'hyperformula/typings',
'docs/api',
'docs/guide',
]

const MARKER = /\[(?:V\d+|(?:vrf|dec|con|que|wrg|crf)_\d+)\]|§\s*AuditSources/g

/** Checked before every scan, so a broken pattern fails instead of finding nothing. */
const MUST_MATCH = ['[V3]', '[vrf_1]', '[dec_2]', '[con_3]', '[que_4]', '[wrg_5]', '[crf_6]', '§AuditSources', '§ AuditSources']
const MUST_NOT_MATCH = ['[V]', '[v1]', '[vrf_]', '[vrf_1a]', '[abc_1]', 'AuditSources', 'cells[V1']

/**
* Prints an error and exits with code 2, which means the scan could not run.
*
* @param {string} message What went wrong.
*/
function abort(message) {
console.error(`ERROR: ${message}`)
process.exit(2)
}

/**
* Lists every file under a directory.
*
* @param {string} dir Absolute path of the directory.
* @returns {string[]} Absolute paths of the files.
*/
function listFiles(dir) {
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
const full = path.join(dir, entry.name)
if (entry.isDirectory()) {
return listFiles(full)
}
return entry.isFile() ? [full] : []
})
}

for (const sample of MUST_MATCH) {
if (!sample.match(MARKER)) {
abort(`the marker pattern does not match ${sample}`)
}
}
for (const sample of MUST_NOT_MATCH) {
if (sample.match(MARKER)) {
abort(`the marker pattern matches ${sample}, which is not a marker`)
}
}

const hits = []
let fileCount = 0
for (const relative of SCANNED) {
const dir = path.join(REPO_ROOT, relative)
let isDirectory = false
try {
isDirectory = statSync(dir).isDirectory()
} catch {
abort(`${relative} does not exist. Build the engine and the docs first: npm run docs:build`)
}
if (!isDirectory) {
abort(`${relative} is not a directory`)
}
const files = listFiles(dir)
if (files.length === 0) {
abort(`${relative} contains no files`)
}
console.log(`${relative}: ${files.length} files`)
fileCount += files.length
for (const file of files) {
const text = readFileSync(file, 'utf8')
for (const match of text.matchAll(MARKER)) {
const line = text.slice(0, match.index).split('\n').length
hits.push(`${path.relative(REPO_ROOT, file)}:${line}: ${match[0]}`)
}
}
}

if (hits.length > 0) {
console.error(hits.join('\n'))
console.error(`\nERROR: ${hits.length} citation markers found. Remove them from the source comments or strings they come from.`)
process.exit(1)
}
console.log(`No citation markers in ${fileCount} files.`)
Loading