Skip to content

fix updated PROV-JSONLD via specific serializer for JSON-LD - #2349

Open
fmigneault wants to merge 12 commits into
common-workflow-language:mainfrom
crim-ca:prov-jsonld-fix-associations
Open

fmigneault wants to merge 12 commits into
common-workflow-language:mainfrom
crim-ca:prov-jsonld-fix-associations

Conversation

@fmigneault

@fmigneault fmigneault commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Description

Replace the previous manual RDF-graph-patching workaround (vendored prov-jsonld-context.json + rdflib json-ld compaction) with a direct call to ProvDocument.serialize(format="jsonld"), now that prov>=3.1.0 implements the PROV-JSONLD submission natively and produces a proper {"@context": [...], "@graph": [...]} document.

  • Bump prov==3.1.0 prov==3.2.2 dependency (requirements.txt, setup.py).
  • Keep emitting the direct prov:wasAssociatedWith shortcut triple in RDF-based serializations (ttl/nt/jsonld): prov 3.1.0's RDF serializer only emits it for associations with no extra qualifying attributes, so add an unqualified association alongside the plan-qualified one.
  • Replace mentionOf/Mention (unsupported by the PROV-JSONLD submission) with an equivalent
    specializationOf plus a plain prov:asInBundle attribute, valid across all serializations.
  • Bump CWLPROV_VERSION to 0.7.0 (breaking change to the emitted provenance graph shape for associations/directory mentions).
  • Add more test coverage for CWL prov under certain code paths that were not evaluted.

References

Replace the previous manual RDF-graph-patching workaround (vendored
prov-jsonld-context.json + rdflib json-ld compaction) with a direct
call to `ProvDocument.serialize(format="jsonld")`, now that prov>=3.1.0
implements the PROV-JSONLD submission natively and produces a proper
``` {"@context": [...], "@graph": [...]} ``` document.

- Bump the `prov` dependency from 1.5.1 to 3.1.0 (requirements.txt,
  setup.py).
- Keep emitting the direct `prov:wasAssociatedWith` shortcut triple in
  RDF-based serializations (ttl/nt/jsonld): prov 3.1.0's RDF serializer
  only emits it for associations with no extra qualifying attributes,
  so add an unqualified association alongside the plan-qualified one.
- Replace `mentionOf`/`Mention` (unsupported by the PROV-JSONLD
  submission) with an equivalent `specializationOf` plus a plain
  `prov:asInBundle` attribute, valid across all serializations.
- Bump CWLPROV_VERSION to 0.7.0 (breaking change to the emitted
  provenance graph shape for associations/directory mentions).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.62500% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.56%. Comparing base (2655097) to head (8d62110).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
cwltool/cwlprov/provenance_profile.py 90.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2349      +/-   ##
==========================================
+ Coverage   85.27%   85.56%   +0.29%     
==========================================
  Files          46       46              
  Lines        8629     8640      +11     
  Branches     2022     2022              
==========================================
+ Hits         7358     7393      +35     
+ Misses        806      783      -23     
+ Partials      465      464       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fmigneault

Copy link
Copy Markdown
Contributor Author

@mr-c
What do you think of this update? It aligns with latest W3C PROV JSON-LD. Current contents are generated in a way that are not proper representations.

@stain
Would you be interested in back-porting this as cwlprov==0.7.0 since the reference will not exist otherwise.

@mr-c

mr-c commented Sep 12, 2026

Copy link
Copy Markdown
Member

@fmigneault Can you open a draft PR to https://github.com/common-workflow-language/cwlprov/ for your proposed 0.7.0 version?

@mr-c

mr-c commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

@fmigneault Here is a patch to fix the Python 3.15 build; I guess you didn't check the "Allow edits from maintainers" box?

From c70987670773866adb7025be1e77ab1a4544710c Mon Sep 17 00:00:00 2001
From: "Michael R. Crusoe" <michael.crusoe@gmail.com>
Date: Sat, 12 Sep 2026 11:43:29 +0200
Subject: [PATCH] adjust requirements to match
 https://prov.readthedocs.io/en/latest/upgrading-3.0.html

---
 requirements.txt | 4 ++--
 setup.py         | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/requirements.txt b/requirements.txt
index b4efc578..e837cabb 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,8 +1,8 @@
 requests>=2.32.2
 ruamel.yaml>=0.16.0,<0.20
-rdflib>=4.2.2,<7.7
+rdflib>=7,<7.7
 schema-salad>=8.10,<9
-prov==3.1.0
+prov[xml,rdf]>=3.1.0,<3.2
 mypy-extensions
 psutil>=5.6.6
 coloredlogs
diff --git a/setup.py b/setup.py
index 58ed3e48..7023ed6d 100644
--- a/setup.py
+++ b/setup.py
@@ -151,9 +151,9 @@ setup(
         "requests >= 2.6.1",  # >= 2.6.1 to workaround
         # https://github.com/ionrock/cachecontrol/issues/137
         "ruamel.yaml >= 0.16, < 0.20",
-        "rdflib >= 4.2.2, < 7.7.0",
+        "rdflib >= 7, < 7.7.0",
         "schema-salad >= 8.10, < 9",
-        "prov == 3.1.0",
+        "prov[xml,rdf] >= 3.1.0, < 3.2",
         "mypy-extensions",
         "psutil >= 5.6.6",
         "coloredlogs",
-- 
2.47.3

@fmigneault

Copy link
Copy Markdown
Contributor Author

I guess you didn't check the "Allow edits from maintainers" box?

The option is not there. Maybe because it is in draft?
{B431F531-CA12-4325-895B-80973E18078C}

@mr-c

mr-c commented Sep 15, 2026

Copy link
Copy Markdown
Member

I guess you didn't check the "Allow edits from maintainers" box?

The option is not there. Maybe because it is in draft? {B431F531-CA12-4325-895B-80973E18078C}

As @CGuichard discovered, that option isn't available when your fork is from an organization.

Can you apply the changes from #2349 (comment) and merge/rebase with the latest from main?

@fmigneault
fmigneault marked this pull request as ready for review September 15, 2026 14:42
@fmigneault

Copy link
Copy Markdown
Contributor Author

@mr-c
Ready for re-review.
I've opened corresponding:

Let me know if anything else needs adjustment.

@fmigneault

Copy link
Copy Markdown
Contributor Author

@mr-c
I've added some extra coverage tests for the CWL PROV module. There is 1 failing job in the CI that I am not sure if it is a sporadic docker pull problem or another actual problem to address. Can you check?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Typing needs refinement for additional attributes

2 participants