Converts a directory of Word or Markdown documents into more-accessible HTML pages, packages them as an IMS Common Cartridge for import into Brightspace or another LMS, and can assemble them into an EPUB.
Licensed GPL 3.0. See LICENSE for more info.
The initial release of these scripts was created by Robert Szarka and supported by a grant from the West Virginia Higher Education Policy Commission.
Reads a book's sources, one file per page (Word, Markdown, AsciiDoc, or HTML), into one intermediate per page, after unpacking any that arrive another way (an EPUB, a site saved from the web or captured in a WARC or WACZ, a Jekyll site, a Common Cartridge), makes each page more accessible on the way, and writes every output the configuration asks for from the same intermediates: HTML pages, an EPUB, Markdown or AsciiDoc source, Word files, and a Common Cartridge for import into an LMS. The book's structure, declared once as contents or guessed from the files and the publisher's PDF, is the cartridge's module tree, the EPUB's table of contents, and the generated contents page alike.
Conversion runs each source through Pandoc and a Lua filter that makes the page more accessible: figures get real captions tied to their images, data tables get captions, header cells, and a focusable scroll region, images get their alt text checked and their layout spacers marked, equations stay equations, and cross-references that Word's export left dangling land. Where the source doesn't say something a screen reader needs, the run reports it, and a sidecar file holds what you decide; after a Markdown round trip, the decisions are in the source itself.
Packaging turns the pages into a Common Cartridge with the book's structure as the module tree, validated against the IMS schemas, and into an EPUB 3 that validates with epubcheck and says what it can claim about itself.
Every run checks what it wrote: dead links and fragments, missing alt text, heading order, invalid ids, tables without headers or caption, and, when the validators are installed, epubcheck and the Nu HTML checker. The same checks, plus what a Word, Markdown, or PDF file says about itself, run on any file without converting it: audit.py writes a findings CSV and a report.
Everything a run decides is written down: reports name what to fix, sidecar files hold what you decided, and the configuration file lists every setting with a sentence explaining it.
You need Pandoc 3.9 or later, Python 3.9 or later, and PyYAML; see Installation for the rest.
T=/path/to/tools # where you cloned this
cd /path/to/your/docx/files
python3 $T/bin/convert.py # 1. convert: html/, one page per source, plus reportsThe first run converts everything, then stops and writes packaging-sample.yaml, because a manifest needs two things only you can supply. Set identifier and title near the top of that file and rename it:
mv packaging-sample.yaml packaging.yaml
python3 $T/bin/convert.py # 2. builds imsmanifest.xml
python3 $T/bin/convert.py --zip # 3. ... and the .imscc archiveIf you have the book's PDF or EPUB, run python3 $T/bin/convert.py --toc book.pdf (or --toc book.epub) before renaming the sample: it orders the pages from the book's own table of contents, with chapters as modules. To build more than one output, declare targets in conversion.yaml:
targets:
html:
format: html # writes html/
epub:
format: epub3 # writes epub/<identifier>.epub
src:
format: markdown # writes src/, the book as Markdown sourceEach target writes into a directory of its own; the content directory keeps the sources, the intermediates, the sidecars, and the reports. See Configuration for targets, editions, and the book's structure, and Markdown sources for Markdown in and out.
| Page | What it covers |
|---|---|
| Installation | Prerequisites and how to check them |
| A first run | Start to finish, what each run writes, exit codes |
| Worked examples | Two real books from start to finish: an OpenStax book from its Word files to a course cartridge, and a book website from a WARC to an EPUB and an HTML round trip. |
| Formats and packaging | Every input, every way it can arrive, every output, how far each path is tested and what it loses, and how the output is packaged |
| How it works | The pieces, what conversion does to a page, running the filter alone |
| Configuration | How the configuration files fit together, precedence, contents, migration from v0.1 |
| Project settings | Reference, generated from the schema |
| Conversion settings | Reference, generated from the schema |
| Packaging settings | Reference, generated from the schema |
| Sidecars and reports | Each report, each sidecar, and what goes in them |
| Bare links | Links whose text is their own address: the sidecar that replaces them with a shortDOI or text or gives them a title, the shortDOI helper, and why a screen-reader description can't do what it seems it should |
| Building the cartridge | The packager, ordering from a PDF, validating the manifest |
| Building an EPUB | One EPUB per book from the same pages and the same contents, and what it claims about itself |
| Checking the output | What every run checks about the pages and EPUBs it wrote, and what it doesn't |
| Auditing | audit.py: what is wrong with a Word, Markdown, HTML, EPUB, or PDF file, without converting it; the findings format every check shares |
| HTML sources | Every .html beside the sources is a source, and a finished page goes in _pt/: what is read from it, what isn't, and why converting this pipeline's own pages changes nothing |
| A book saved from the web | unpack-site.py: browser saves or .mhtml into pages whose every reference is local, the generator recognized, the order read from the site's own menus |
| AsciiDoc sources | The AsciiDoc target, and .adoc chapters as sources, a master file that includes them as the book's order, and what is done on reading that the reader leaves undone |
| Adopting pages as sources | adopt-pages.py: a split book's pages renamed without --, links rewritten, as the sources of a new book |
| A site's own source | unpack-jekyll.py: a Jekyll site's Markdown (just-the-docs) into a book directory, contents from its front matter, code left alone |
| Making a WARC | Installing and running wget or Browsertrix Crawler to capture a book's website for unpack-site.py |
| A course cartridge as the source | unpack-cartridge.py: a Common Cartridge exported from an LMS or a publisher into pages, Word sources, and files, with its modules as contents and what a course holds that a book doesn't reported |
| An EPUB as the source | unpack-epub.py: a publisher's EPUB into pages, media, and a project.yaml with its metadata and its navigation as contents; a book that is one file |
| Markdown sources | A .md beside the sources is a page: what it can declare, how its images travel, and a book set up for a Pandoc PDF build |
| Splitting pages | One page per heading from a source that arrived as one file per chapter, or one file |
| Brightspace | What importing and deleting actually do there, confirmed by D2L |
| Testing | The test suites and what each is load-bearing for |
| Utilities | The tools in util/: surveying a corpus, checking a source, comparing runs |
| Troubleshooting and known limits | Cloud drives, common failures, validator defects, what isn't fixed |
CHANGELOG.md records what changed in each version; ROADMAP.md records what is planned and why.