diff --git a/README.md b/README.md index 9ba150ca..5ed52d18 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ extensive test suite, and has undergone ## Table Of Contents * [Getting Started](#getting-started) +* [Repository Layout](#repository-layout) * [Prepackaged Policies](#prepackaged-policies) * [Crafting a policy](#crafting-a-policy) * [Custom policies](#custom-policies) @@ -33,6 +34,29 @@ extensive test suite, and has undergone [Getting Started](docs/getting_started.md) includes instructions on how to get started with or without Maven. +## Repository Layout + +Building requires JDK 11 or newer. From the repository root, build all Maven +modules and run their tests with: + +```sh +./mvnw clean verify +``` + +The Maven wrapper pins the required Maven version. If you select a module +with `-pl`, also use `-am` so Maven builds the modules it depends on. + +* [owasp-java-html-sanitizer/](owasp-java-html-sanitizer/README.md) contains the + published library, its tests, and its Java module descriptor. +* [java8-shim/](java8-shim/README.md) and + [java10-shim/](java10-shim/README.md) implement + compatibility code that is bundled into the library JAR. +* [examples/](examples/README.md) contains sample policies to read and copy; + it is tested but not published. +* [empiricism/](empiricism/README.md) contains the browser experiments used + to generate the sanitizer's HTML element tables. +* [docs/](docs/README.md) contains user, security, and historical documentation. + ## Prepackaged Policies You can use diff --git a/change_log.md b/change_log.md index e0079519..9f8970f1 100644 --- a/change_log.md +++ b/change_log.md @@ -2,6 +2,8 @@ Most recent at top. * Next release + * Docs: Add focused README files for each Maven module and an index for + supporting documentation, including safe build and regeneration steps. * Table parts inside cell content now return to the existing table, and a row after a column closes the column group before continuing the table. When an orphan table part needs an implied table, that table closes a diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..68fade75 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,42 @@ + + +# Documentation + +This directory holds supporting documentation for the OWASP Java HTML +Sanitizer. Start with the [project README](../README.md) for a short API +example and links to the current Javadoc. + +## Using the sanitizer + +* [Getting started](getting_started.md) explains how to obtain the library and + identifies the main policy-building APIs. +* [Using with Maven](maven.md) gives the dependency coordinates and JPMS + module name. +* [Why sanitize when you can validate?](html-validation.md) explains why the + library returns normalized, sanitized output instead of declaring arbitrary + input safe. +* [Examples](../examples/README.md) introduces the sample policies and their + tests. + +## Security + +* [Known public vulnerabilities](vulnerabilities.md) lists published + advisories and the first fixed versions. +* [Attack review ground rules](attack_review_ground_rules.md) defines the + scope for adversarial testing and links to the private reporting process. +* [CVE-2011-4457](cve20114457.md) and + [CVE-2021-42575](cve202142575.md) provide historical notes for those + issues. + +Do not open a public issue for a suspected sanitizer bypass. Follow the +repository's [security policy](../SECURITY.md) so maintainers can coordinate a +fix and disclosure. + +## Research and project history + +The [client-side template notes](client-side-templates.md) collect research +on template-language syntax, including open questions and examples from +other sanitizers. + +The [credits](credits.md) recognize project contributors. Release-by-release +changes are recorded in the repository [change log](../change_log.md). diff --git a/empiricism/README.md b/empiricism/README.md new file mode 100644 index 00000000..b9a5f2d2 --- /dev/null +++ b/empiricism/README.md @@ -0,0 +1,65 @@ + + +# Browser Parser Experiments + +This module measures how browsers nest and balance HTML tags. It converts +those observations into the element tables used by the sanitizer. + +This is an internal maintenance tool. It is part of the Maven reactor so its +generator is compiled and checked, but it is not published. + +## Files and data flow + +1. [html-containment.html](html-containment.html) and + [html-containment.js](html-containment.js) probe a browser's parser. +2. The probe emits JSON that is saved as [canned-data.json](canned-data.json). +3. [rebuild.sh](rebuild.sh) parses and formats that JSON into + [canned-data.js](canned-data.js), then runs the + [Java generator](src/main/java/org/owasp/html/empiricism/JsonToSerializedHtmlElementTables.java). +4. The generator writes `target/HtmlElementTablesCanned.java`, and the script + copies it into the library as + [HtmlElementTablesCanned.java](../owasp-java-html-sanitizer/src/main/java/org/owasp/html/HtmlElementTablesCanned.java). + +`canned-data.json`, `canned-data.js`, and `HtmlElementTablesCanned.java` are +different representations of the same browser observations. Never hand-edit +`HtmlElementTablesCanned.java`. + +## Regenerating the tables + +You need a browser with JavaScript enabled, Bash, Python 3, Perl, and +[JDK 11 or newer](../README.md#repository-layout). The script uses the +repository's Maven wrapper. + +1. Open the local `empiricism/html-containment.html` file in your browser and + follow its `?rerun` link. The full probe can take a long time; wait for it + to finish. Without `?rerun`, the page displays the saved observations. +2. Copy the complete JSON dump at the bottom of the page into + `canned-data.json`. Record the browser version used when describing the + change. +3. From the repository root, run: + + ```sh + ./empiricism/rebuild.sh + ``` + +4. Review the diffs in `canned-data.json`, `canned-data.js`, and the generated + Java file. Explain changes in browser observations and add corresponding + sanitizer tests where behavior changes. +5. Run the complete reactor before proposing the update: + + ```sh + ./mvnw clean verify + ``` + +For a quick diagnostic run, use `?rerun&shortlist`. Its output covers only a +subset of elements; do not use it to replace the complete `canned-data.json`. + +The rebuild script first installs the reactor artifacts into your local Maven +repository with tests skipped, then generates and copies the new tables. The +final `clean verify` above compiles and tests the updated library. A normal +Maven build uses the checked-in tables and does not run browser experiments. + +Because these tables affect parsing and balancing of attacker-controlled HTML, +treat every data change as security-sensitive. Include adversarial cases for +raw-text elements, misnested tags, foreign SVG and MathML content, case +differences, and serialize/reparse mutation where relevant. diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 00000000..289d8017 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,39 @@ + + +# Examples + +This module contains worked examples of building and applying sanitizer +policies. The sources are meant to be read and adapted; the module is tested +as part of the reactor but is not published, so applications should not import +`org.owasp.html.examples`. + +## Included examples + +* [EbayPolicyExample.java](src/main/java/org/owasp/html/examples/EbayPolicyExample.java) + demonstrates a rich-text policy with CSS and attribute rules. +* [SlashdotPolicyExample.java](src/main/java/org/owasp/html/examples/SlashdotPolicyExample.java) + demonstrates a smaller policy, custom element names, and normalized attributes. +* [UrlTextExample.java](src/main/java/org/owasp/html/examples/UrlTextExample.java) + demonstrates a postprocessor that appends a URL's + authority as visible text after links and images. + +The eBay and Slashdot programs read UTF-8 HTML from standard input and write +sanitized HTML to standard output; each buffers the complete input before +sanitizing. `UrlTextExample` accepts HTML as command-line arguments. + +## Build and test + +The [tests](src/test/java/org/owasp/html/examples/) exercise both accepted +markup and hostile input. With JDK 11 or newer, run them from the repository +root so Maven also builds the sanitizer and shim modules: + +```sh +./mvnw -pl examples -am test +``` + +Treat these policies as examples, not universal security profiles. Start from +default deny, allow only the markup an application needs, restrict URL +protocols explicitly, and add negative tests for the application's threat +model. The root [README](../README.md) and +[getting-started guide](../docs/getting_started.md) cover the supported public +API. diff --git a/java10-shim/README.md b/java10-shim/README.md new file mode 100644 index 00000000..c2303e28 --- /dev/null +++ b/java10-shim/README.md @@ -0,0 +1,29 @@ + + +# Java 10 Compatibility Shim + +This internal module provides +[ForJava10AndLater](src/main/java/org/owasp/shim/ForJava10AndLater.java), the +implementation of the [java8-shim](../java8-shim/README.md) adapter that uses +newer JDK collection factories and copy operations. Its `setCopyOf` method +uses an unmodifiable `LinkedHashSet` to preserve encounter order. + +It is compiled with `maven.compiler.release` set to 10 and may use Java 9 and +Java 10 APIs, but no newer APIs. + +The core library loads this implementation when the running JDK supports it +and otherwise uses the Java 8 fallback. Both shim modules are shaded into the +published sanitizer JAR; this module is not independently published and is not +a supported consumer API. + +## Build + +The [build requires JDK 11 or newer](../README.md#repository-layout). +Run from the repository root to build this module and its dependencies: + +```sh +./mvnw -pl java10-shim -am package +``` + +Changes to the shim should also be checked in the final shaded artifact with +the full `./mvnw clean verify` build. diff --git a/java8-shim/README.md b/java8-shim/README.md new file mode 100644 index 00000000..28995aba --- /dev/null +++ b/java8-shim/README.md @@ -0,0 +1,28 @@ + + +# Java 8 Compatibility Shim + +This internal module adapts collection APIs introduced in Java 9 and 10 for +the sanitizer's Java 8-compatible source. +[Java8Shim](src/main/java/org/owasp/shim/Java8Shim.java) defines the adapter +API, and [ForJava8](src/main/java/org/owasp/shim/ForJava8.java) provides +the fallback implementation. + +At runtime `Java8Shim` first tries the implementation from +[java10-shim](../java10-shim/README.md) and falls back to `ForJava8` when that +newer class cannot load, including on Java 8 and 9. Both shim modules are +shaded into the published sanitizer JAR; neither is a separate dependency for +library users or a supported public API. + +## Build + +Keep this module compatible with the repository's Java 8 source and bytecode +target. Follow the [build prerequisites](../README.md#repository-layout) +(JDK 11 or newer), and run from the repository root: + +```sh +./mvnw -pl java8-shim -am package +``` + +Changes to the shim should also be checked in the final shaded artifact with +the full `./mvnw clean verify` build. diff --git a/owasp-java-html-sanitizer/README.md b/owasp-java-html-sanitizer/README.md new file mode 100644 index 00000000..75258cbb --- /dev/null +++ b/owasp-java-html-sanitizer/README.md @@ -0,0 +1,58 @@ + + +# Core Library Module + +This module builds the published +`com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer` artifact. +It combines HTML parsing, tag balancing, allowlist policy enforcement, and +rendering to produce sanitized HTML. See the +[getting-started guide](../docs/getting_started.md) for using the public API. + +## Layout + +* [Library sources](src/main/java/org/owasp/html/) contain the Java 8 library + and its public API. `HtmlPolicyBuilder`, `PolicyFactory`, and `Sanitizers` + are the usual entry points. +* [Tests](src/test/java/org/owasp/html/) include JUnit 5 unit, regression, and + fuzzer tests. +* [Test resources](src/test/resources/) include lexer fixtures, benchmark + input, and checks for the packaged artifact. +* The [module descriptor](src/main/java9/module-info.java) is compiled + separately for Java 9 and put in `META-INF/versions/9/` of the multi-release + JAR. It exports only `org.owasp.html`. +* The [JPMS consumer](src/it/jpms-consumer/) is compiled for Java 9 and run + against the packaged JAR during the `integration-test` phase, included in + `verify`. It checks the descriptor and confirms that the bundled shim + package remains encapsulated. + +`HtmlElementTablesCanned.java` is generated from browser observations. Do +not edit it directly; update the experiment or data in +[the browser experiments module](../empiricism/README.md) and follow its +regeneration instructions. + +## Build and test + +Use JDK 11 or newer to build, and run from the repository root: + +```sh +./mvnw clean verify +``` + +To select this module while still building its in-repository dependencies, +use: + +```sh +./mvnw -pl owasp-java-html-sanitizer -am test +``` + +`test` runs the unit, regression, and fuzzer tests. `verify` also checks the +packaged JAR and module descriptor. Ordinary library and test sources must +remain compatible with Java 8; the descriptor and JPMS consumer use Java 9. + +## Security-sensitive changes + +Parser, policy, URL, CSS, encoding, balancing, and rendering changes need +hostile regression cases as well as positive tests. Preserve default-deny +behavior and public API compatibility. Report a possible bypass privately as +described in the [security policy](../SECURITY.md); do not publish a regression +payload before the coordinated fix is released.