From 17219073c14bc5ace51262604b04462b446284f6 Mon Sep 17 00:00:00 2001 From: Scott Sims Date: Thu, 13 Aug 2026 22:05:48 -0400 Subject: [PATCH 1/2] Document latency, failure behaviour, and attestation scope MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The example constructed RubricClient without background_queue, which is off by default and submits synchronously inside the component's run() — a slow API blocks the pipeline run for up to timeout seconds (default 15). Both examples now set it, and the default is explained. Documents behaviour an audit-tool adopter needs and the page did not state: attestation failures are caught and logged at WARNING, so a pipeline succeeds with no evidence for that run (on_dead_letter surfaces the gap); only the first reply is attested, truncated to 2000 characters; the reply text is transmitted to the API rather than hashed client-side. Also adds a verification example, reads the API key from the environment rather than a literal, notes the package covers frameworks beyond AutoGen, and replaces "court-grade evidence" with the claim that is actually supportable. --- integrations/rubric-protocol.md | 37 ++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/integrations/rubric-protocol.md b/integrations/rubric-protocol.md index 051d6232..c2c2d4c4 100644 --- a/integrations/rubric-protocol.md +++ b/integrations/rubric-protocol.md @@ -17,7 +17,7 @@ toc: true ## Overview -[Rubric Protocol](https://rubric-protocol.com) produces court-grade evidence of what your AI pipelines did and when. Every attestation is signed with ML-DSA-65 (FIPS 204) post-quantum cryptography and anchored to Hedera Consensus Service — a public, neutral ledger — so anyone can independently verify a pipeline's output years later at [rubric-protocol.com/verify](https://rubric-protocol.com/verify), without trusting your logs or your vendor. +[Rubric Protocol](https://rubric-protocol.com) produces independently verifiable evidence of what your AI pipelines did and when. Every attestation is signed with ML-DSA-65 (FIPS 204) post-quantum cryptography and anchored to Hedera Consensus Service — a public, neutral ledger — so anyone can verify a pipeline's output years later at [rubric-protocol.com/verify](https://rubric-protocol.com/verify), without trusting your logs or your vendor. This matters wherever two parties need to agree on what an AI system did: regulatory examinations (EU AI Act Annex IV, SR 26-2, Illinois SB 315), insurance claims, vendor disputes, and internal audit. Unlike mutable application logs, an anchored attestation cannot be silently edited after the fact. @@ -27,15 +27,21 @@ This matters wherever two parties need to agree on what an AI system did: regula pip install autogen-rubric ``` +The package instruments several agent frameworks, Haystack among them — the name predates the others. + ## Usage Add the component to any pipeline. It attests each run's replies and passes them through unchanged: ```python +import os from haystack import Pipeline from autogen_rubric import RubricClient, RubricHaystackComponent -client = RubricClient(api_key="YOUR_RUBRIC_API_KEY") +client = RubricClient( + api_key=os.environ["RUBRIC_API_KEY"], + background_queue=True, # keeps attestation off the pipeline's critical path +) pipeline = Pipeline() # ... add your generator, retriever, etc. ... @@ -50,15 +56,40 @@ pipeline.connect("llm.replies", "rubric.replies") Or attest ad-hoc from anywhere in your app with the callback form: ```python +import os from autogen_rubric import RubricClient, rubric_haystack_callback -client = RubricClient(api_key="YOUR_RUBRIC_API_KEY") +client = RubricClient(api_key=os.environ["RUBRIC_API_KEY"], background_queue=True) attest = rubric_haystack_callback(client, agent_id="support-rag") attest(result) ``` Each attestation returns an ID resolvable on the public verifier, with its Hedera sequence number and ML-DSA-65 signature — evidence that stands on its own. +## Latency and failure behaviour + +`background_queue=True` hands each attestation to a background sender, so your pipeline is never waiting on network I/O. It is **off by default**: a client constructed as `RubricClient(api_key=...)` submits synchronously inside the component's `run()`, and will block that pipeline run for up to `timeout` seconds (default 15) if the API is slow to respond. Set it explicitly in production. + +Attestation never breaks a pipeline. Any failure — network, authentication, or API error — is caught, logged at `WARNING`, and the replies are returned unchanged. The corollary is worth stating plainly for an audit tool: when attestation fails the pipeline still succeeds, and the evidence for that run is simply absent. Pass `on_dead_letter=` to be notified which attestations were dropped, rather than discovering the gap when someone asks for the record. + +Attestations marked `risk="high"` are always sent synchronously, queue or not, so a high-risk decision is not lost to an abrupt process exit. + +## What is attested + +The component attests the **first** reply of each run, truncated to 2000 characters, and records the total reply count in metadata. Pipelines that return several replies, or replies longer than 2000 characters, are therefore attested in part rather than in full — worth knowing before relying on an attestation as a complete record of a run. + +The reply text is transmitted to the Rubric API over HTTPS; it is not reduced to a hash on the client. Treat the attested output as leaving your network. + +## Verifying an attestation + +Verification needs neither this package, nor an account, nor Rubric's cooperation — which is the point: + +```bash +npx @rubric-protocol/verify +``` + +The attestation can equally be checked directly against its Hedera Consensus Service record by sequence number, or looked up at [rubric-protocol.com/verify](https://rubric-protocol.com/verify). + ## License MIT From ee996b2c7135cc34832491401200b8f28f8ffbda Mon Sep 17 00:00:00 2001 From: Scott Date: Sat, 15 Aug 2026 23:06:51 +0000 Subject: [PATCH 2/2] Document the callback's input types and 1.10.3 requirement --- integrations/rubric-protocol.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/integrations/rubric-protocol.md b/integrations/rubric-protocol.md index c2c2d4c4..b2693082 100644 --- a/integrations/rubric-protocol.md +++ b/integrations/rubric-protocol.md @@ -61,9 +61,24 @@ from autogen_rubric import RubricClient, rubric_haystack_callback client = RubricClient(api_key=os.environ["RUBRIC_API_KEY"], background_queue=True) attest = rubric_haystack_callback(client, agent_id="support-rag") + +result = pipeline.run({"prompt_builder": {"query": query}}) attest(result) ``` +The callback accepts the dict returned by `Pipeline.run()`, and also a +`PipelineSnapshot` or `PipelineState` if you use +[pipeline breakpoints](https://docs.haystack.deepset.ai/docs/pipeline-breakpoints). +If it cannot resolve the pipeline outputs it logs at ERROR and writes **no** +attestation rather than anchoring an empty one - check your logs for +`[RubricHaystack] no attestation written` if a run you expected is missing from +the verifier. + +Requires `autogen-rubric >= 1.10.3`. Earlier versions resolved outputs only from +an object exposing `.pipeline_outputs` directly, so passing the `Pipeline.run()` +dict silently attested an empty payload. + + Each attestation returns an ID resolvable on the public verifier, with its Hedera sequence number and ML-DSA-65 signature — evidence that stands on its own. ## Latency and failure behaviour