Skip to content

test(harness): restore sinex_prop and failure snapshots #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Sinity wants to merge 4 commits into from
Closed

test(harness): restore sinex_prop and failure snapshots #102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4bde91f
test(harness): reintroduce the sinex_prop baseline
Sinity Nov 17, 2025
4acff99
refactor(test-macros): rebuild the macro-driven property harness
Sinity Nov 18, 2025
03182c4
fix(test-macros): run async property cases on the current Tokio handle
Sinity Nov 18, 2025
740c1e0
test(macros): persist failure snapshots for sinex_test and sinex_prop
Sinity Nov 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions .cargo-machete.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@ sinex-satellite-sdk = ["tonic-build"]
sinex-services = ["sinex-test-utils"]
sinex-terminal-command-canonicalizer = ["sinex-test-utils"]
sinex-health-aggregator = ["sinex-test-utils"]
sinex-db = ["tokio-test"]

# Used by feature flags or conditional compilation
sinex-db = ["sea-orm-migration", "prometheus", "sysinfo", "procfs", "parking_lot", "sinex-macros"]

# Used in binaries that need logging setup
sinex-desktop-satellite = ["tracing-subscriber"]
Expand All @@ -24,4 +20,4 @@ sinex-terminal-satellite = ["tracing-subscriber"]

# Platform-specific or conditional dependencies
sinex-terminal-satellite = ["xterm-query"]
sinex-system-satellite = ["battery", "procfs", "systemstat"]
sinex-system-satellite = ["battery", "procfs", "systemstat"]
8 changes: 4 additions & 4 deletions TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,10 @@ tests are reserved for scenarios that truly span multiple crates or binaries.
.await?;
```
- **Property tests:** place proptest suites alongside the crate they fuzz (see
below). Use `#[sinex_test(proptest)]` so the harness provides tracing,
timeouts, and `TestContext`; add `cases = 256` (or any other number) to fix
the `PROPTEST_CASES` used by `TestRunner`. Capture any new failing seeds in
that crate’s `tests/property/*.proptest-regressions` files.
below). Use `#[sinex_prop]` (or the block-style `sinex_proptest!`) so the
harness provides tracing, timeouts, context wiring, and seed persistence.

@coderabbitai coderabbitai Bot Apr 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add blank line after heading for markdown consistency.

Static analysis indicates a blank line is expected below line 93 per MD022 (blanks-around-headings).

📝 Proposed fix 
   below). Use `#[sinex_prop]` (or the block-style `sinex_proptest!`) so the
   harness provides tracing, timeouts, context wiring, and seed persistence.
+
   Add `cases = 256` (or any number) to lock the runner config. Capture any
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
harness provides tracing, timeouts, context wiring, and seed persistence.
below). Use `#[sinex_prop]` (or the block-style `sinex_proptest!`) so the
harness provides tracing, timeouts, context wiring, and seed persistence.
Add `cases = 256` (or any number) to lock the runner config. Capture any
🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 93-93: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below

(MD022, blanks-around-headings)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@TESTING.md` at line 93, Add a single blank line immediately after the
Markdown heading that precedes or contains the text "harness provides tracing,
timeouts, context wiring, and seed persistence." to satisfy MD022
(blanks-around-headings); edit the TESTING.md heading block so there is one
empty line between the heading and the following paragraph/line.

Add `cases = 256` (or any number) to lock the runner config. Capture any
failing seeds in that crate’s `tests/property/*.proptest-regressions` files.
Comment on lines +94 to +95

Copilot AI Apr 9, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs say failing seeds should be captured in tests/property/*.proptest-regressions, but the restored harness config in #[sinex_prop] sets proptest failure persistence to target/proptest-regressions by default (and optionally SINEX_PROPTEST_DIR). Please align this guidance with the actual persistence path (either update the doc text or change the default persistence location).

Suggested change
Add `cases = 256` (or any number) to lock the runner config. Capture any
failing seeds in that crate’s `tests/property/*.proptest-regressions` files.
Add `cases = 256` (or any number) to lock the runner config. Failing seeds
are persisted under `target/proptest-regressions` by default, or under the
directory specified by `SINEX_PROPTEST_DIR` when that override is set.

Copilot uses AI. Check for mistakes.
Comment on lines +92 to +95

@coderabbitai coderabbitai Bot Apr 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify the default failure persistence path in the macro implementation
rg -n "proptest-regressions" crate/lib/sinex-test-utils/macros/src/lib.rs

Repository: Sinity/sinex

Length of output: 152

🏁 Script executed:

#!/bin/bash
# Read the TESTING.md file around lines 92-95 to see full context
sed -n '85,100p' TESTING.md | cat -n

Repository: Sinity/sinex

Length of output: 866

🏁 Script executed:

#!/bin/bash
# Read the macro implementation around line 367 to see env var handling
sed -n '360,375p' crate/lib/sinex-test-utils/macros/src/lib.rs | cat -n

Repository: Sinity/sinex

Length of output: 784

Documentation path is inconsistent with implementation.

Line 95 states that failing seeds are captured in tests/property/*.proptest-regressions, but the macro implementation (macros/src/lib.rs, lines 365-368) defaults to target/proptest-regressions. The default is configurable via the SINEX_PROPTEST_DIR environment variable.

Update the documentation to reflect the actual default path and document the environment variable override option.

🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 93-93: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below

(MD022, blanks-around-headings)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@TESTING.md` around lines 92 - 95, Update the doc string that currently claims
failing seeds are saved under tests/property/*.proptest-regressions to match the
macro implementation: failing seeds default to target/proptest-regressions (this
default is controlled by the SINEX_PROPTEST_DIR environment variable); mention
both the default path and how to override it via SINEX_PROPTEST_DIR in the
documentation for #[sinex_prop] / sinex_proptest! so readers see the actual
behavior implemented in macros::lib (the code that reads SINEX_PROPTEST_DIR).


## Property Testing Guidelines

Expand Down
1 change: 0 additions & 1 deletion crate/lib/sinex-core/src/types/bin/sinex-schema.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ use std::collections::{BTreeMap, BTreeSet};
use std::fs;
use std::path::{Path, PathBuf};
use tracing::info;
use uuid::Uuid;

#[derive(Parser)]
#[command(author, version, about, long_about = None)]
Expand Down
63 changes: 29 additions & 34 deletions crate/lib/sinex-core/tests/adversarial/enhanced_boundary_test.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
//
// Tests system behavior at boundaries, limits, and edge cases

use sinex_test_utils::prelude::*;
use sinex_test_utils::{prelude::*, sinex_prop};
use proptest::prelude::*;
use std::sync::Arc;
use tokio::time::{Duration, timeout};
Expand Down Expand Up @@ -340,40 +340,35 @@ async fn test_string_length_boundaries(ctx: TestContext) -> Result<()> {
}

/// Property-based testing for boundary conditions
#[sinex_test]
async fn test_property_based_boundaries(ctx: TestContext) -> Result<()> {
proptest!(|(
array_size in 0..1000usize,
string_len in 0..10000usize,
nest_depth in 0..50usize
)| {
let rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async {
let array: Vec<i32> = (0..array_size as i32).collect();
let text = "x".repeat(string_len);

let mut nested = serde_json::json!("leaf");
for _ in 0..nest_depth {
nested = serde_json::json!({"child": nested});
}
#[sinex_prop]
async fn test_property_based_boundaries(
ctx: &TestContext,
#[strategy(0..1000usize)] array_size: usize,
#[strategy(0..10000usize)] string_len: usize,
#[strategy(0..50usize)] nest_depth: usize,
) -> Result<()> {
let array: Vec<i32> = (0..array_size as i32).collect();
let text = "x".repeat(string_len);

let mut nested = serde_json::json!("leaf");
for _ in 0..nest_depth {
nested = serde_json::json!({ "child": nested });
}

let event = ctx.create_test_event(
"property_test",
"boundary",
serde_json::json!({
"array": array,
"text": text,
"nested": nested,
"array_size": array_size,
"string_len": string_len,
"nest_depth": nest_depth
}),
);

// We don't assert success, just that it doesn't panic
let _ = ctx.pool.events().insert(event).await;
});
});
let event = ctx.create_test_event(
"property_test",
"boundary",
serde_json::json!({
"array": array,
"text": text,
"nested": nested,
"array_size": array_size,
"string_len": string_len,
"nest_depth": nest_depth
}),
);

// We don't assert success, just that it doesn't panic
let _ = ctx.pool.events().insert(event).await;
Ok(())
}
Comment on lines +343 to 374

@coderabbitai coderabbitai Bot Apr 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider using TestResult<()> for consistency.

The return type Result<()> resolves to std::result::Result<(), SinexError> per sinex_test_utils. Other migrated tests use TestResult<()> (color_eyre::eyre::Result<()>), which provides richer error context. Consider aligning for consistency.

♻️ Suggested change 
 #[sinex_prop]
 async fn test_property_based_boundaries(
     ctx: &TestContext,
     #[strategy(0..1000usize)] array_size: usize,
     #[strategy(0..10000usize)] string_len: usize,
     #[strategy(0..50usize)] nest_depth: usize,
-) -> Result<()> {
+) -> TestResult<()> {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#[sinex_prop]
async fn test_property_based_boundaries(
ctx: &TestContext,
#[strategy(0..1000usize)] array_size: usize,
#[strategy(0..10000usize)] string_len: usize,
#[strategy(0..50usize)] nest_depth: usize,
) -> Result<()> {
let array: Vec<i32> = (0..array_size as i32).collect();
let text = "x".repeat(string_len);
let mut nested = serde_json::json!("leaf");
for _ in 0..nest_depth {
nested = serde_json::json!({ "child": nested });
}
let event = ctx.create_test_event(
"property_test",
"boundary",
serde_json::json!({
"array": array,
"text": text,
"nested": nested,
"array_size": array_size,
"string_len": string_len,
"nest_depth": nest_depth
}),
);
// We don't assert success, just that it doesn't panic
let _ = ctx.pool.events().insert(event).await;
});
});
let event = ctx.create_test_event(
"property_test",
"boundary",
serde_json::json!({
"array": array,
"text": text,
"nested": nested,
"array_size": array_size,
"string_len": string_len,
"nest_depth": nest_depth
}),
);
// We don't assert success, just that it doesn't panic
let _ = ctx.pool.events().insert(event).await;
Ok(())
}
#[sinex_prop]
async fn test_property_based_boundaries(
ctx: &TestContext,
#[strategy(0..1000usize)] array_size: usize,
#[strategy(0..10000usize)] string_len: usize,
#[strategy(0..50usize)] nest_depth: usize,
) -> TestResult<()> {
let array: Vec<i32> = (0..array_size as i32).collect();
let text = "x".repeat(string_len);
let mut nested = serde_json::json!("leaf");
for _ in 0..nest_depth {
nested = serde_json::json!({ "child": nested });
}
let event = ctx.create_test_event(
"property_test",
"boundary",
serde_json::json!({
"array": array,
"text": text,
"nested": nested,
"array_size": array_size,
"string_len": string_len,
"nest_depth": nest_depth
}),
);
// We don't assert success, just that it doesn't panic
let _ = ctx.pool.events().insert(event).await;
Ok(())
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@crate/lib/sinex-core/tests/adversarial/enhanced_boundary_test.rs` around
lines 343 - 374, Change the test function signature for
test_property_based_boundaries to return TestResult<()> instead of Result<()>
for consistency with other migrated tests; update the signature from async fn
test_property_based_boundaries(...) -> Result<()> to -> TestResult<()>, ensure
TestResult is in scope (import or fully-qualify, e.g., color_eyre::eyre::Result
as TestResult or whatever project alias is used), and keep the existing Ok(())
return so the body requires no further changes.

Loading
Loading