Skip to content

fix: handle empty String fields in to_byte_array #2371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
agentmila wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: handle empty String fields in to_byte_array #2371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions program-libs/hasher/src/to_byte_array.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,13 +180,22 @@ impl ToByteArray for String {

/// Max allowed String length is 31 bytes.
/// For longer strings hash to field size or provide a custom implementation.
///
/// Empty strings are represented with a leading 1 byte at index 0 to avoid
/// an all-zero `[0u8; 32]` representation, which can cause errors in the
/// Poseidon hasher on Solana (see issue #1272).
fn to_byte_array(&self) -> Result<[u8; 32], HasherError> {
let bytes = self.as_bytes();
let mut result = [0u8; 32];
let byte_len = bytes.len();
if byte_len > 31 {
return Err(HasherError::InvalidInputLength(31, bytes.len()));
}
if byte_len == 0 {
// Use a non-zero sentinel to avoid all-zero representation.
// This prevents Poseidon syscall errors on Solana (issue #1272).
result[0] = 1;
}
result[32 - byte_len..].copy_from_slice(bytes);
Comment on lines +194 to 199
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

All-zero output is still reachable via non-empty NUL-only strings.

At Line 194, only byte_len == 0 gets remapped. A non-empty string like "\0" still serializes to [0u8; 32] after Line 199, which can hit the same Poseidon invalid-parameters path you’re trying to avoid.

💡 Minimal fix to block all-zero serialization while preserving current format for normal strings 
         let mut result = [0u8; 32];
         let byte_len = bytes.len();
         if byte_len > 31 {
             return Err(HasherError::InvalidInputLength(31, bytes.len()));
         }
         if byte_len == 0 {
             // Use a non-zero sentinel to avoid all-zero representation.
             // This prevents Poseidon syscall errors on Solana (issue `#1272`).
             result[0] = 1;
+            return Ok(result);
         }
         result[32 - byte_len..].copy_from_slice(bytes);
+        // Non-empty NUL-only strings can also become all-zero; avoid that too.
+        if result == [0u8; 32] {
+            result[0] = 2;
+        }
         Ok(result)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if byte_len == 0 {
// Use a non-zero sentinel to avoid all-zero representation.
// This prevents Poseidon syscall errors on Solana (issue #1272).
result[0] = 1;
}
result[32 - byte_len..].copy_from_slice(bytes);
if byte_len == 0 {
// Use a non-zero sentinel to avoid all-zero representation.
// This prevents Poseidon syscall errors on Solana (issue `#1272`).
result[0] = 1;
return Ok(result);
}
result[32 - byte_len..].copy_from_slice(bytes);
// Non-empty NUL-only strings can also become all-zero; avoid that too.
if result == [0u8; 32] {
result[0] = 2;
}
Ok(result)
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@program-libs/hasher/src/to_byte_array.rs` around lines 194 - 199, The code
only handles byte_len == 0 but still produces an all-zero 32-byte array when
given a non-empty NUL-only input (e.g., "\0"); update the logic around
result/bytes/byte_len so that after copying bytes into result you detect the
all-zero case (e.g., check if result.iter().all(|b| *b == 0) or if
bytes.iter().all(|b| *b == 0)) and then set the sentinel (result[0] = 1) to
avoid the all-zero representation while preserving the existing encoding for
normal strings.

Ok(result)
}
Expand Down
8 changes: 6 additions & 2 deletions program-libs/hasher/tests/to_byte_array.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,11 +216,15 @@ fn test_to_byte_array_u8_arrays() {

#[test]
fn test_to_byte_array_string() {
// Test with empty string
// Test with empty string - should produce a non-zero representation
// to avoid Poseidon errors (issue #1272)
let empty_string = "".to_string();
let result = empty_string.to_byte_array().unwrap();
let expected = [0u8; 32];
let mut expected = [0u8; 32];
expected[0] = 1; // non-zero sentinel for empty strings
assert_eq!(result, expected);
// Ensure it's not all zeros
assert_ne!(result, [0u8; 32]);
Comment on lines +219 to +227
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add a regression test for non-empty NUL-only strings.

This block validates empty strings well, but it should also assert that "\0" (or "\0\0") does not serialize to [0u8; 32]; otherwise the all-zero failure path can regress unnoticed.

✅ Suggested test addition 
     let mut expected = [0u8; 32];
     expected[0] = 1; // non-zero sentinel for empty strings
     assert_eq!(result, expected);
     // Ensure it's not all zeros
     assert_ne!(result, [0u8; 32]);

+    // Test with non-empty NUL-only string - must also avoid all-zero output
+    let nul_only = "\0".to_string();
+    let result = nul_only.to_byte_array().unwrap();
+    assert_ne!(result, [0u8; 32]);
+    assert_ne!(result, expected); // should not collide with empty-string sentinel
+
     // Test with short string
     let short_string = "foobar".to_string();
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@program-libs/hasher/tests/to_byte_array.rs` around lines 219 - 227, Add
assertions to cover NUL-only strings so they don't serialize to all zeros:
create strings like "\0" and "\0\0", call to_byte_array() (e.g., same pattern as
empty_string -> result), unwrap the results, and assert that each result !=
[0u8; 32]; place these new checks in the same test (to_byte_array.rs) near the
empty-string case so they run as part of the regression test for non-empty
NUL-only inputs.


// Test with short string
let short_string = "foobar".to_string();
Expand Down
Loading