Understanding OpenSCAP architecture for building a modern web-based compliance auditing interface

[Yesser-Belhajali]

Hello everyone,

First of all, thank you in advance to anyone who takes the time to read this and help me.

I am currently working on a school project that is also related to an internship application I will be making next semester. The goal of the project is to audit a machine, identify security weaknesses, and eventually provide remediation solutions.

I am relatively new to the cybersecurity field, so I have been putting a lot of effort into learning and understanding the different concepts involved. When I came across SCAP, OpenSCAP, and SCAP Workbench, it took me some time to understand the ecosystem and how all the different components fit together (SCAP, XCCDF, OVAL, datastreams, etc.).

SCAP Workbench was very close to what I had in mind, but since it is now in an end-of-life state and no longer actively maintained, I started thinking about building a more modern solution based on the OpenSCAP and SCAP ecosystem.

My idea is to develop a modern web interface where OpenSCAP would act as the compliance scanning engine, while security content such as ComplianceAsCode/SCAP Security Guide would provide the rules, benchmarks, and remediation information.

However, I have been struggling to understand the internal architecture. I have spent time studying XCCDF, OVAL, datastreams, and the OpenSCAP source code. I understand their general purpose and how they are related, but I am having difficulties understanding how everything is connected internally and how the scanning process flows through the codebase.

I would like to better understand:

• How the OpenSCAP architecture is organized internally.
• Where the XCCDF and OVAL processing happens during a scan.
• How datastreams are handled internally.
• How scan results are generated and linked back to individual rules.
• What would be the recommended approach for building an external web interface on top of OpenSCAP.
• Whether it would be better to simply use the OpenSCAP executable/API after building the project, or if modifying parts of the source code would make sense for this type of project.

My goal is not to simply copy existing work, but to properly understand the ecosystem and add my own contribution on top of it.

If anyone could provide guidance, point me toward relevant parts of the codebase or documentation, or explain the general architecture, I would greatly appreciate it.

Thank you very much for your time and for the great work you have already done on these projects.

[oldnco]

For a first version, I would keep OpenSCAP behind a narrow worker interface and use the oscap CLI rather than modifying the scanner. That gives you a stable process boundary, makes upgrades easier, and lets the web application remain independent of OpenSCAP internals.

A useful mental model for the scan flow is:

1. A source data stream is the package. It contains or references the XCCDF benchmark, OVAL definitions, CPE/applicability content, and related components.
2. The selected XCCDF profile determines which rules and values apply. Tailoring can refine that selection without editing the original benchmark.
3. OpenSCAP evaluates platform applicability, then processes each selected XCCDF rule. Rules normally reference an OVAL definition (or another supported check engine).
4. The OVAL engine collects system characteristics and evaluates the referenced definitions. Those results are mapped back to XCCDF rule-result elements through the rule/check references.
5. OpenSCAP exports XCCDF results and, when requested, OVAL results or an ARF/result data stream containing the input context, assets, and detailed results.

For an external web application, I would start with a job architecture like this:

• The API stores a scan request: approved content ID/version, profile, tailoring file, target, and requested output formats.
• A separate, least-privileged worker validates the content and runs a fixed command template. Do not pass arbitrary user-supplied command-line fragments to a shell.
• Run each scan in an isolated temporary directory with time/resource limits and an immutable audit record.
• Use oscap info to discover profiles/components and oscap ds sds-validate to validate source data streams before evaluation.
• Run an evaluation such as oscap xccdf eval --profile PROFILE --results-arf results.xml --report report.html CONTENT.xml.
• Parse the machine-readable XCCDF/ARF XML into your own normalized result model. Keep the original result artifact as evidence. Link a result to a rule by the XCCDF rule/result identifiers, not by display title.

ARF is a good archival format because it can include the source context, asset information, XCCDF results, and lower-level check results. XCCDF result XML is often simpler when the UI only needs rule status, severity, identifiers, and references. HTML reports are useful for humans but should not be your application's data API.

I would keep remediation as a separate, explicitly authorized workflow. The first release can display the fix content associated with failed rules, but should not execute it automatically. If you later add execution, require an approval step, preserve the exact generated remediation, constrain privileges, and rescan afterward.

If the CLI boundary eventually becomes limiting, the high-level C API is the next place to look. xccdf_session encapsulates loading XCCDF/data streams, CPE and OVAL content, evaluation, and exporting XCCDF/OVAL/ARF results: https://github.com/OpenSCAP/openscap/blob/main/src/XCCDF/public/xccdf_session.h

The current project README also has concrete validation, evaluation, and report-generation examples: https://github.com/OpenSCAP/openscap/blob/main/README.md

So my recommendation is: wrap the executable first, model the results cleanly, and modify OpenSCAP only after you have identified a specific missing capability that cannot be provided through the CLI or public API. That approach is also much more manageable for a school project.