docs: add module README guides

README.md

@@ -18,6 +18,7 @@ extensive test suite, and has undergone
 ## Table Of Contents
 
 *  [Getting Started](#getting-started)
+*  [Repository Layout](#repository-layout)
 *  [Prepackaged Policies](#prepackaged-policies)
 *  [Crafting a policy](#crafting-a-policy)
 *  [Custom policies](#custom-policies)
@@ -33,6 +34,29 @@ extensive test suite, and has undergone
 [Getting Started](docs/getting_started.md) includes instructions on
 how to get started with or without Maven.
 
+## Repository Layout
+
+Building requires JDK 11 or newer.  From the repository root, build all Maven
+modules and run their tests with:
+
+```sh
+./mvnw clean verify
+```
+
+The Maven wrapper pins the required Maven version.  If you select a module
+with `-pl`, also use `-am` so Maven builds the modules it depends on.
+
+* [owasp-java-html-sanitizer/](owasp-java-html-sanitizer/README.md) contains the
+  published library, its tests, and its Java module descriptor.
+* [java8-shim/](java8-shim/README.md) and
+  [java10-shim/](java10-shim/README.md) implement
+  compatibility code that is bundled into the library JAR.
+* [examples/](examples/README.md) contains sample policies to read and copy;
+  it is tested but not published.
+* [empiricism/](empiricism/README.md) contains the browser experiments used
+  to generate the sanitizer's HTML element tables.
+* [docs/](docs/README.md) contains user, security, and historical documentation.
+
 ## Prepackaged Policies
 
 You can use

change_log.md

@@ -2,6 +2,8 @@
 
 Most recent at top.
   * Next release
+    * Docs: Add focused README files for each Maven module and an index for
+      supporting documentation, including safe build and regeneration steps.
     * Table parts inside cell content now return to the existing table, and a
       row after a column closes the column group before continuing the table.
       When an orphan table part needs an implied table, that table closes a

docs/README.md

@@ -0,0 +1,42 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Documentation
+
+This directory holds supporting documentation for the OWASP Java HTML
+Sanitizer.  Start with the [project README](../README.md) for a short API
+example and links to the current Javadoc.
+
+## Using the sanitizer
+
+* [Getting started](getting_started.md) explains how to obtain the library and
+  identifies the main policy-building APIs.
+* [Using with Maven](maven.md) gives the dependency coordinates and JPMS
+  module name.
+* [Why sanitize when you can validate?](html-validation.md) explains why the
+  library returns normalized, sanitized output instead of declaring arbitrary
+  input safe.
+* [Examples](../examples/README.md) introduces the sample policies and their
+  tests.
+
+## Security
+
+* [Known public vulnerabilities](vulnerabilities.md) lists published
+  advisories and the first fixed versions.
+* [Attack review ground rules](attack_review_ground_rules.md) defines the
+  scope for adversarial testing and links to the private reporting process.
+* [CVE-2011-4457](cve20114457.md) and
+  [CVE-2021-42575](cve202142575.md) provide historical notes for those
+  issues.
+
+Do not open a public issue for a suspected sanitizer bypass.  Follow the
+repository's [security policy](../SECURITY.md) so maintainers can coordinate a
+fix and disclosure.
+
+## Research and project history
+
+The [client-side template notes](client-side-templates.md) collect research
+on template-language syntax, including open questions and examples from
+other sanitizers.
+
+The [credits](credits.md) recognize project contributors.  Release-by-release
+changes are recorded in the repository [change log](../change_log.md).

empiricism/README.md

@@ -0,0 +1,65 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Browser Parser Experiments
+
+This module measures how browsers nest and balance HTML tags.  It converts
+those observations into the element tables used by the sanitizer.
+
+This is an internal maintenance tool.  It is part of the Maven reactor so its
+generator is compiled and checked, but it is not published.
+
+## Files and data flow
+
+1. [html-containment.html](html-containment.html) and
+   [html-containment.js](html-containment.js) probe a browser's parser.
+2. The probe emits JSON that is saved as [canned-data.json](canned-data.json).
+3. [rebuild.sh](rebuild.sh) parses and formats that JSON into
+   [canned-data.js](canned-data.js), then runs the
+   [Java generator](src/main/java/org/owasp/html/empiricism/JsonToSerializedHtmlElementTables.java).
+4. The generator writes `target/HtmlElementTablesCanned.java`, and the script
+   copies it into the library as
+   [HtmlElementTablesCanned.java](../owasp-java-html-sanitizer/src/main/java/org/owasp/html/HtmlElementTablesCanned.java).
+
+`canned-data.json`, `canned-data.js`, and `HtmlElementTablesCanned.java` are
+different representations of the same browser observations.  Never hand-edit
+`HtmlElementTablesCanned.java`.
+
+## Regenerating the tables
+
+You need a browser with JavaScript enabled, Bash, Python 3, Perl, and
+[JDK 11 or newer](../README.md#repository-layout).  The script uses the
+repository's Maven wrapper.
+
+1. Open the local `empiricism/html-containment.html` file in your browser and
+   follow its `?rerun` link.  The full probe can take a long time; wait for it
+   to finish.  Without `?rerun`, the page displays the saved observations.
+2. Copy the complete JSON dump at the bottom of the page into
+   `canned-data.json`.  Record the browser version used when describing the
+   change.
+3. From the repository root, run:
+
+   ```sh
+   ./empiricism/rebuild.sh
+   ```
+
+4. Review the diffs in `canned-data.json`, `canned-data.js`, and the generated
+   Java file.  Explain changes in browser observations and add corresponding
+   sanitizer tests where behavior changes.
+5. Run the complete reactor before proposing the update:
+
+   ```sh
+   ./mvnw clean verify
+   ```
+
+For a quick diagnostic run, use `?rerun&shortlist`.  Its output covers only a
+subset of elements; do not use it to replace the complete `canned-data.json`.
+
+The rebuild script first installs the reactor artifacts into your local Maven
+repository with tests skipped, then generates and copies the new tables.  The
+final `clean verify` above compiles and tests the updated library.  A normal
+Maven build uses the checked-in tables and does not run browser experiments.
+
+Because these tables affect parsing and balancing of attacker-controlled HTML,
+treat every data change as security-sensitive.  Include adversarial cases for
+raw-text elements, misnested tags, foreign SVG and MathML content, case
+differences, and serialize/reparse mutation where relevant.

examples/README.md

@@ -0,0 +1,39 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Examples
+
+This module contains worked examples of building and applying sanitizer
+policies.  The sources are meant to be read and adapted; the module is tested
+as part of the reactor but is not published, so applications should not import
+`org.owasp.html.examples`.
+
+## Included examples
+
+* [EbayPolicyExample.java](src/main/java/org/owasp/html/examples/EbayPolicyExample.java)
+  demonstrates a rich-text policy with CSS and attribute rules.
+* [SlashdotPolicyExample.java](src/main/java/org/owasp/html/examples/SlashdotPolicyExample.java)
+  demonstrates a smaller policy, custom element names, and normalized attributes.
+* [UrlTextExample.java](src/main/java/org/owasp/html/examples/UrlTextExample.java)
+  demonstrates a postprocessor that appends a URL's
+  authority as visible text after links and images.
+
+The eBay and Slashdot programs read UTF-8 HTML from standard input and write
+sanitized HTML to standard output; each buffers the complete input before
+sanitizing.  `UrlTextExample` accepts HTML as command-line arguments.
+
+## Build and test
+
+The [tests](src/test/java/org/owasp/html/examples/) exercise both accepted
+markup and hostile input.  With JDK 11 or newer, run them from the repository
+root so Maven also builds the sanitizer and shim modules:
+
+```sh
+./mvnw -pl examples -am test
+```
+
+Treat these policies as examples, not universal security profiles.  Start from
+default deny, allow only the markup an application needs, restrict URL
+protocols explicitly, and add negative tests for the application's threat
+model.  The root [README](../README.md) and
+[getting-started guide](../docs/getting_started.md) cover the supported public
+API.

java10-shim/README.md

@@ -0,0 +1,29 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Java 10 Compatibility Shim
+
+This internal module provides
+[ForJava10AndLater](src/main/java/org/owasp/shim/ForJava10AndLater.java), the
+implementation of the [java8-shim](../java8-shim/README.md) adapter that uses
+newer JDK collection factories and copy operations.  Its `setCopyOf` method
+uses an unmodifiable `LinkedHashSet` to preserve encounter order.
+
+It is compiled with `maven.compiler.release` set to 10 and may use Java 9 and
+Java 10 APIs, but no newer APIs.
+
+The core library loads this implementation when the running JDK supports it
+and otherwise uses the Java 8 fallback.  Both shim modules are shaded into the
+published sanitizer JAR; this module is not independently published and is not
+a supported consumer API.
+
+## Build
+
+The [build requires JDK 11 or newer](../README.md#repository-layout).
+Run from the repository root to build this module and its dependencies:
+
+```sh
+./mvnw -pl java10-shim -am package
+```
+
+Changes to the shim should also be checked in the final shaded artifact with
+the full `./mvnw clean verify` build.

java8-shim/README.md

@@ -0,0 +1,28 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Java 8 Compatibility Shim
+
+This internal module adapts collection APIs introduced in Java 9 and 10 for
+the sanitizer's Java 8-compatible source.
+[Java8Shim](src/main/java/org/owasp/shim/Java8Shim.java) defines the adapter
+API, and [ForJava8](src/main/java/org/owasp/shim/ForJava8.java) provides
+the fallback implementation.
+
+At runtime `Java8Shim` first tries the implementation from
+[java10-shim](../java10-shim/README.md) and falls back to `ForJava8` when that
+newer class cannot load, including on Java 8 and 9.  Both shim modules are
+shaded into the published sanitizer JAR; neither is a separate dependency for
+library users or a supported public API.
+
+## Build
+
+Keep this module compatible with the repository's Java 8 source and bytecode
+target.  Follow the [build prerequisites](../README.md#repository-layout)
+(JDK 11 or newer), and run from the repository root:
+
+```sh
+./mvnw -pl java8-shim -am package
+```
+
+Changes to the shim should also be checked in the final shaded artifact with
+the full `./mvnw clean verify` build.

owasp-java-html-sanitizer/README.md

@@ -0,0 +1,58 @@
+<!-- SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause -->
+
+# Core Library Module
+
+This module builds the published
+`com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer` artifact.
+It combines HTML parsing, tag balancing, allowlist policy enforcement, and
+rendering to produce sanitized HTML.  See the
+[getting-started guide](../docs/getting_started.md) for using the public API.
+
+## Layout
+
+* [Library sources](src/main/java/org/owasp/html/) contain the Java 8 library
+  and its public API.  `HtmlPolicyBuilder`, `PolicyFactory`, and `Sanitizers`
+  are the usual entry points.
+* [Tests](src/test/java/org/owasp/html/) include JUnit 5 unit, regression, and
+  fuzzer tests.
+* [Test resources](src/test/resources/) include lexer fixtures, benchmark
+  input, and checks for the packaged artifact.
+* The [module descriptor](src/main/java9/module-info.java) is compiled
+  separately for Java 9 and put in `META-INF/versions/9/` of the multi-release
+  JAR.  It exports only `org.owasp.html`.
+* The [JPMS consumer](src/it/jpms-consumer/) is compiled for Java 9 and run
+  against the packaged JAR during the `integration-test` phase, included in
+  `verify`.  It checks the descriptor and confirms that the bundled shim
+  package remains encapsulated.
+
+`HtmlElementTablesCanned.java` is generated from browser observations.  Do
+not edit it directly; update the experiment or data in
+[the browser experiments module](../empiricism/README.md) and follow its
+regeneration instructions.
+
+## Build and test
+
+Use JDK 11 or newer to build, and run from the repository root:
+
+```sh
+./mvnw clean verify
+```
+
+To select this module while still building its in-repository dependencies,
+use:
+
+```sh
+./mvnw -pl owasp-java-html-sanitizer -am test
+```
+
+`test` runs the unit, regression, and fuzzer tests.  `verify` also checks the
+packaged JAR and module descriptor.  Ordinary library and test sources must
+remain compatible with Java 8; the descriptor and JPMS consumer use Java 9.
+
+## Security-sensitive changes
+
+Parser, policy, URL, CSS, encoding, balancing, and rendering changes need
+hostile regression cases as well as positive tests.  Preserve default-deny
+behavior and public API compatibility.  Report a possible bypass privately as
+described in the [security policy](../SECURITY.md); do not publish a regression
+payload before the coordinated fix is released.