Skip to content

Commit 718d47c

Browse files
wurstbrotwurstbrot
authored
BREAKING CHANGE: add agentic AI (#80)
BREAKING CHANGE: add agentic AI (#80)
1 parent ca6e517 commit 718d47c

44 files changed

Lines changed: 18511 additions & 1074 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎ARCHITECTURE.md‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
# Architecture of the DSOMM YAML Model
2+
3+
This document describes the structure of the YAML files in `src/assets/YAML/default`,
4+
which define the DSOMM maturity model (dimensions, subdimensions, activities and
5+
implementation hints).
6+
7+
## Processing pipeline
8+
9+
```
10+
src/assets/YAML/default/<Dimension>/<Subdimension>.yaml (model source, this document)
11+
src/assets/YAML/default/implementations.yaml (shared tool/reference catalog)
12+
src/assets/YAML/custom/<Dimension>/*.yaml (optional org-specific overlay)
13+
│
14+
▼ yaml-generation/generateDimensions.php (run via generateDimensions.bash, dockerized)
15+
generated/model.yaml (merged model consumed by the application)
16+
```
17+
18+
The generator:
19+
20+
- merges all `src/assets/YAML/default/*/*.yaml` files (skipping `_meta.yaml`),
21+
- overlays `src/assets/YAML/custom/*/*.yaml` if present (when a custom folder exists,
22+
only activities defined there are kept),
23+
- fails on duplicate `uuid`s and duplicate activity names (both must be globally unique),
24+
- fails on missing `uuid` or `level`,
25+
- resolves `dependsOn` entries (by activity name or uuid) across the *whole* model — cross-dimension dependencies are allowed,
26+
- defaults missing `tags` to `["none"]` and auto-generates an `openCRE` reference,
27+
- sorts activities by level.
28+
29+
## Directory and file structure
30+
31+
Each **dimension** is a directory; each **subdimension** is one YAML file in it.
32+
Inside a file the nesting repeats dimension and subdimension as map keys:
33+
34+
```yaml
35+
# yaml-language-server: $schema=../../schemas/dsomm-schema-<dimension>.json
36+
---
37+
Build and Deployment: # dimension name (must match across all files in the folder)
38+
Build: # subdimension name (one per file)
39+
Defined build process: # activity name (globally unique)
40+
uuid: ...
41+
...
42+
```
43+
44+
| Path | Purpose |
45+
|---|---|
46+
| `default/<Dimension>/_meta.yaml` | Display metadata of the dimension |
47+
| `default/<Dimension>/<Subdimension>.yaml` | Activities of one subdimension |
48+
| `default/implementations.yaml` | Catalog of tools/references, targeted by `$ref` |
49+
| `schemas/dsomm-schema-*.json` | JSON schemas (one per dimension) for editor validation |
50+
| `../meta.yaml` | UI strings, labels for reference systems, level descriptions |
51+
| `../teams.yaml` | Team names and team groups for per-team assessment |
52+
53+
Current dimensions: `AI`, `BuildAndDeployment`, `CultureAndOrganization`,
54+
`Implementation`, `InformationGathering`, `TestAndVerification`.
55+
56+
## `_meta.yaml` attributes
57+
58+
| Attribute | Required | Description |
59+
|---|---|---|
60+
| `_meta.label` | yes | Display name of the dimension (e.g. "Build and Deployment") |
61+
| `_meta.icon` | no | Icon file name; may be empty |
62+
| `_meta.description` | no | Markdown description of the dimension and its subdimensions |
63+
64+
## Activity attributes
65+
66+
"Schema" = required by the JSON schemas in `src/assets/YAML/schemas/`.
67+
"Generator" = hard-enforced by `generateDimensions.php` (build fails without it).
68+
In practice several schema-required attributes are omitted in existing files (see
69+
"Known inconsistencies" below); when creating **new** activities, treat every
70+
schema-required attribute as mandatory.
71+
72+
### Mandatory
73+
74+
| Attribute | Enforced by | Description |
75+
|---|---|---|
76+
| `uuid` | schema + generator | Globally unique, stable identifier (UUID v4). Never reuse or change; used to track assessment state across model updates. |
77+
| `risk` | schema | What can go wrong if the activity is *not* performed. Shown as motivation for the activity. |
78+
| `measure` | schema | What to do: the concrete countermeasure/practice the activity consists of. |
79+
| `level` | schema + generator | Maturity level 1–5 at which the activity is expected. |
80+
| `difficultyOfImplementation` | schema | Effort estimate object with three sub-attributes, each rated 1–5: `knowledge` (required expertise; disciplines involved), `time` (effort over time), `resources` (systems/licenses needed). All three are required. |
81+
| `usefulness` | schema | Security benefit of the activity, rated 1–5. |
82+
| `implementation` | schema | List of `- $ref: src/assets/YAML/default/implementations.yaml#/implementations/<key>` pointers to tools/guides that help implement the activity. May be an empty list (`[]`). Only `$ref` entries are allowed. |
83+
| `references` | schema | Mappings to other standards. `samm2` (OWASP SAMM v2 stream IDs), `iso27001-2017` and `iso27001-2022` (control numbers) are required arrays; free-text entries are used to note missing mappings. `openCRE` is added automatically by the generator — do not maintain it manually. Other standards (e.g. OWASP AISVS categories) are linked as `implementation` `$ref`s (entries `aisvs-c01` … `aisvs-appc` in `implementations.yaml`), not as reference arrays. |
84+
| `description` | schema | Markdown long-form explanation of the activity (background, benefits, guidelines). Required by schema, though older activities often omit it. |
85+
| `comments` | schema¹ | Assessment comments placeholder, initialize with `""`. |
86+
87+
¹ Optional in `dsomm-schema-test-and-verification.json` and `dsomm-schema-agentic-ai.json`;
88+
required by the other dimension schemas.
89+
90+
The former assessment-state attributes `isImplemented` and `evidence` have been
91+
removed from the schemas and from all activity YAML files; assessment state is
92+
not maintained in the model source.
93+
94+
### Optional
95+
96+
| Attribute | Description |
97+
|---|---|
98+
| `assessment` | Markdown checklist describing how an assessor verifies the activity (what to show/demonstrate). |
99+
| `dependsOn` | List of activity *uuids* that should be implemented first, with the activity name appended as a YAML comment for readability (`- <uuid> # <name>`). Resolved globally, so activities in other dimensions can be referenced. The generator also resolves plain activity names (used in older files) and fails on unknown references. |
100+
| `tags` | List of free-form tags used for filtering in the UI (e.g. `inventory`, `sca`, `ai`). Defaults to `["none"]`. |
101+
| `meta.implementationGuide` | Practical how-to hints for implementing the measure. |
102+
| `credits` | Attribution when an activity was adopted from another project (e.g. AppSecure-nrw Security Belts). |
103+
| `teamsImplemented`, `teamsEvidence` | Per-team assessment state (used in team mode; see `dsomm-schema-implementation.json` and `teams.yaml`). |
104+
105+
YAML anchors/aliases (`&name` / `*name`) are used in existing files to share
106+
identical `risk` texts between activities and are safe to use.
107+
108+
## `implementations.yaml` attributes
109+
110+
Every entry under the top-level `implementations:` key:
111+
112+
| Attribute | Required | Description |
113+
|---|---|---|
114+
| `uuid` | yes | Unique identifier of the implementation |
115+
| `name` | yes | Display name of the tool/resource |
116+
| `tags` | yes | List of tags (may be empty `[]`) for filtering |
117+
| `url` | yes (schema) | Link to the tool/resource; omitted in a few legacy entries |
118+
| `description` | yes (schema) | Short markdown description; omitted in many legacy entries |
119+
120+
Activities reference these entries via `$ref`, so a tool is described once and
121+
reused everywhere. `TEST_REFERENCED_URLS=true` (or `generateDimensions.bash --test-urls`)
122+
checks all URLs.
123+
124+
## Known inconsistencies
125+
126+
- The JSON schemas declare `additionalProperties: false` but existing files use
127+
attributes not present in every schema (`tags`, `credits`, `comment`); the
128+
generator accepts them.
129+
- Schema-required attributes (`description`, `comments`) are missing in many
130+
existing activities; only `uuid` and `level` make the generator fail.
131+
- The per-dimension schemas differ slightly (`dsomm-schema-test-and-verification.json`
132+
requires fewer attributes; `dsomm-schema-implementation.json` uses
133+
`teamsImplemented`/`teamsEvidence`).
134+
135+
## The Agentic AI dimension
136+
137+
The `Agentic AI` dimension (`src/assets/YAML/default/AgenticAI/`, schema
138+
`schemas/dsomm-schema-agentic-ai.json`) covers the secure use of AI in software
139+
development and the security of AI-based features:
140+
141+
| Subdimension | Activity | Level |
142+
|---|---|---|
143+
| Isolation | Usage of sandboxing for AI agents | 1 |
144+
| Isolation | Permission management for AI agents | 1 |
145+
| Isolation | Least privilege on external systems for AI agents | 2 |
146+
| Isolation | Untrusted workspace handling for AI agents | 2 |
147+
| Isolation | Network isolation for AI agents | 3 |
148+
| Guidance | Basic secure coding rules for AI assistants | 1 |
149+
| Guidance | AI usage policy | 2 |
150+
| Guidance | Security requirements for AI-assisted development | 2 |
151+
| Guidance | Spec-driven development | 2 |
152+
| Guidance | Inventory of AI agents | 2 |
153+
| Guidance | Language and framework specific secure coding rules for AI assistants | 3 |
154+
| Guidance | Evaluation of the trust of used AI components | 3 |
155+
| Verification | Continuous detection of compromised AI components | 4 |
156+
| Guidance | Threat modeling of AI components | 3 |
157+
| Guidance | Loading security rules at the right development step | 4 |
158+
| Red Teaming | Basic AI red teaming | 2 |
159+
| Red Teaming | Regular automated AI red teaming | 4 |
160+
| Data Protection | Basic data leak prevention | 1 |
161+
| Data Protection | Input validation for AI systems | 2 |
162+
| Data Protection | Secure output handling in AI applications | 4 |
163+
| Data Protection | Protection of agent memory against poisoning | 3 |
164+
| Data Protection | Automated data leak detection for AI interactions | 4 |
165+
| Data Protection | Hallucination detection for AI responses | 4 |
166+
| Verification | Human review of AI generated code | 1 |
167+
| Verification | Validation of AI-suggested dependencies | 2 |
168+
| Verification | Self-verification of AI generated changes | 2 |
169+
| Verification | No verification bypass for AI generated code | 2 |
170+
| Verification | Static and dynamic analysis of AI generated code | 3 |
171+
| Verification | Security test generation with AI | 3 |
172+
173+
All Agentic AI activities carry the tag `ai` plus a subdimension tag
174+
(`isolation`, `guidance`, `red-teaming`, `data-protection`, `verification`).
175+
176+
## Handling duplicates (e.g. SAST/DAST for AI-generated code)
177+
178+
AI-generated code must be verified with SAST/DAST — but static and dynamic
179+
analysis are already covered by the *Test and Verification* dimension
180+
(`StaticDepthForApplications`, `DynamicDepthForApplications`, …). Copying those
181+
activities into the Agentic AI dimension would break the model: the generator rejects
182+
duplicate uuids and duplicate activity names, and a copy would fork the
183+
assessment state (an organization would have to answer the same question twice).
184+
185+
The proposed rules, applied in the Agentic AI dimension:
186+
187+
1. **One canonical activity per topic.** A practice lives in exactly one
188+
dimension/subdimension. SAST/DAST stay in *Test and Verification*.
189+
2. **Reference, don't copy — use `dependsOn`.** `dependsOn` resolves activity
190+
names across the whole model. The AI activity
191+
*"No verification bypass for AI generated code"* depends on
192+
`Defined build process`, `Static analysis for important server side components`
193+
and `Simple Scan` instead of redefining them. The AI activity itself only
194+
describes the **delta**: making sure AI-generated changes cannot skip those
195+
existing gates.
196+
3. **Use `tags` for cross-cutting views.** Tagging existing canonical activities
197+
with `ai` (e.g. the SAST/DAST/SCA activities most relevant for AI-generated
198+
code) allows filtering an "AI security" view in the UI without duplicating
199+
content or uuids.
200+
4. **Only create a new activity for a genuine delta.** If AI introduces a new
201+
aspect of an existing topic, create a new activity with its own uuid that is
202+
scoped to that aspect (e.g. *"Human review of AI generated code"* — review
203+
accountability and no self-approving agents), and link the generic activity
204+
via `dependsOn`.
205+
5. **Mention the relationship in `description`.** The activity text should state
206+
explicitly that the generic practice is defined elsewhere, so readers and
207+
assessors are not confused about scope.

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,10 @@ If you prefer Podman over Docker, set the environement variable `DOCKER_CMD` to
6464
- ISO 27001:2017 mapping by Andre Baumeier.
6565
- Other inspirations and contributions are acknowledged in the original README.
6666

67+
## AI Assistance
68+
69+
AI tools were used to assist in drafting and refining texts in this model (e.g. activity descriptions). All AI-assisted content has been reviewed by maintainers.
70+
6771

6872
## License
6973

0 commit comments

Comments
 (0)