Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,22 @@
<!-- markdownlint-disable-file MD025 -->

# 5.3.0

- **BinaryReader**:
- **New Feature**: Added random access read methods — `getUint8(int position)`, `getInt16(int position)`, `getUint16(int position)`, `getInt32(int position)`, `getUint32(int position)`, `getInt64(int position)`, `getUint64(int position)`, `getFloat32(int position)`, `getFloat64(int position)` — read at arbitrary position without advancing offset.
- **Refactoring**: Rewritten using `extension type` for zero-cost abstractions.
- **Refactoring**: Methods organized into semantic extension groups (`BinaryReaderCore`, `BinaryReaderVarInt`, `BinaryReaderNumeric`, `BinaryReaderBytesString`, `BinaryReaderRandomAccess`, `BinaryReaderPosition`, `BinaryReaderInternal`).

- **BinaryWriter**:
- **New Feature**: Added random access read methods — `getUint8(int position)`, `getInt16(int position)`, `getUint16(int position)`, `getInt32(int position)`, `getUint32(int position)`, `getInt64(int position)`, `getUint64(int position)`, `getFloat32(int position)`, `getFloat64(int position)` — read written bytes at arbitrary position.
- **New Feature**: Added random access write methods — `setUint8(int position, int value)`, `setInt16(int position, int value)`, `setUint16(int position, int value)`, `setInt32(int position, int value)`, `setUint32(int position, int value)`, `setInt64(int position, int value)`, `setUint64(int position, int value)`, `setFloat32(int position, double value)`, `setFloat64(int position, double value)` — write at arbitrary position without changing offset.
- **Refactoring**: Rewritten using `extension type` for zero-cost abstractions.
- **Refactoring**: Methods organized into semantic extension groups (`BinaryWriterCore`, `BinaryWriterVarInt`, `BinaryWriterNumeric`, `BinaryWriterBytesString`, `BinaryWriterRandomAccess`, `BinaryWriterPosition`, `BinaryWriterInternal`).

- **Tests**:
- Added `test/unit/binary_reader_get_test.dart` — 41 tests for random access reads.
- Added `test/unit/binary_writer_set_get_test.dart` — 49 tests for random access reads/writes.

# 5.2.0

- **BinaryWriter**:
Expand Down
108 changes: 91 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,55 @@

**High-performance binary serialization and deserialization for Dart.** Optimized for high-frequency network protocols, real-time streaming, and fast local storage. Features zero-copy reads, object pooling, and transactional stream parsing.

## Table of Contents

- [pro\_binary](#pro_binary)
- [Table of Contents](#table-of-contents)
- [Key Features](#key-features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Recipes \& Patterns](#recipes--patterns)
- [1. Efficient Object Serialization](#1-efficient-object-serialization)
- [2. High-Frequency writes (Pooling)](#2-high-frequency-writes-pooling)
- [3. Stream Parsing (Async Binary Messages)](#3-stream-parsing-async-binary-messages)
- [4. Binary Packets (Manual navigation)](#4-binary-packets-manual-navigation)
- [Examples](#examples)
- [API Overview](#api-overview)
- [Performance](#performance)
- [String Encoding (One-Pass vs `utf8.encode`)](#string-encoding-one-pass-vs-utf8encode)
- [Object Serialization \& Deserialization](#object-serialization--deserialization)
- [Object Pooling](#object-pooling)
- [Testing](#testing)
- [Contributing](#contributing)
- [License](#license)

## Key Features

* **Extreme Performance:** Built from the ground up for speed. Leverages Dart Extension Types for zero-overhead abstractions and direct memory manipulation.
* **Zero-Copy Reads:** Deserialization operations return `Uint8List` views instead of allocating new memory arrays, significantly reducing GC (Garbage Collector) pauses.
* **One-Pass String Encoding:** Features a highly optimized `writeVarString` with optimistic size estimation and native memory shifting. Up to **~30% faster** than standard `utf8.encode`.
* **Zero-Allocation Object Pooling:** Includes built-in `BinaryWriterPool` to reuse writer instances. Perfect for high-frequency network packets (e.g., game servers, WebSockets).
* **Compact Encoding:** Native support for VarInt and ZigZag encoding to shrink payload sizes for integers.
* **Transactional Stream Parsing:** Easily process fragmented asynchronous data chunks using `StreamBinaryReader` with `bookmark()` and `rollback()` capabilities.
* **Cross-Platform:** 100% pure Dart. Works seamlessly across Native (AOT/JIT) and Web (WASM/JS) with a consistent, predictable API.
- **Extreme Performance:** Built from the ground up for speed. Leverages Dart Extension Types for zero-overhead abstractions and direct memory manipulation.
- **Zero-Copy Reads:** Deserialization operations return `Uint8List` views instead of allocating new memory arrays, significantly reducing GC (Garbage Collector) pauses.
- **One-Pass String Encoding:** Features a highly optimized `writeVarString` with optimistic size estimation and native memory shifting. Up to **~30% faster** than standard `utf8.encode`.
- **Zero-Allocation Object Pooling:** Includes built-in `BinaryWriterPool` to reuse writer instances. Perfect for high-frequency network packets (e.g., game servers, WebSockets).
- **Compact Encoding:** Native support for VarInt and ZigZag encoding to shrink payload sizes for integers.
- **Transactional Stream Parsing:** Easily process fragmented asynchronous data chunks using `StreamBinaryReader` with `bookmark()` and `rollback()` capabilities.
- **Cross-Platform:** 100% pure Dart. Works seamlessly across Native (AOT/JIT) and Web (WASM/JS) with a consistent, predictable API.

## Installation

Add `pro_binary` to your `pubspec.yaml` manually:

```yaml
dependencies:
pro_binary: ^5.2.0
pro_binary: <latest_version>
```

Or add it using the command line:

```bash
# For Dart projects
dart pub add pro_binary

# For Flutter projects
flutter pub add pro_binary
```

## Quick Start
Expand All @@ -34,7 +68,7 @@ final writer = BinaryWriter()
..writeVarString('Dart 🚀')
..writeBool(true);

final bytes = writer.takeBytes();
final bytes = writer.takeBytes(); // takes the buffer and resets the writer

// Deserialize
final reader = BinaryReader(bytes);
Expand All @@ -59,8 +93,8 @@ class User {
User(this.id, this.name);

void encode(BinaryWriter w) => w
..writeVarUint(id)
..writeVarString(name);
..writeVarUint(id) // compact integer encoding
..writeVarString(name); // fast one-pass UTF-8 encoding

factory User.decode(BinaryReader r) =>
User(
Expand Down Expand Up @@ -160,9 +194,9 @@ if (reader.hasBytes(4)) {

Explore the [example](example/) directory for complete, runnable projects:

* [Basic Usage](example/basic/): Simple serialization and deserialization.
* [File Streaming](example/file_streaming/): Reading and writing large binary files using streams.
* [Network Streaming](example/network_streaming/): Implementing a custom protocol for TCP/Socket data.
- [Basic Usage](example/basic/): Simple serialization and deserialization.
- [File Streaming](example/file_streaming/): Reading and writing large binary files using streams.
- [Network Streaming](example/network_streaming/): Implementing a custom protocol for TCP/Socket data.

## API Overview

Expand All @@ -175,12 +209,37 @@ Explore the [example](example/) directory for complete, runnable projects:
| **StreamBinaryReader** | Handles async data chunks seamlessly with a transactional `bookmark`/`rollback` model for partial data. |
| **BinaryStreamTransformer** | The easiest way to parse a `Stream<List<int>>` into a stream of typed messages or objects. |
| **BinaryWriterPool** | Object pool for `BinaryWriter` to eliminate GC pressure during high-frequency write operations. |
| **getUtf8Length** | High-speed utility to calculate UTF-8 byte length without encoding (O(n) but heavily optimized). |
| **TransactionalReader** | Base interface for custom transactional readers. Used internally by `StreamBinaryReader`. |

## Performance

Run benchmarks to see it in action:
`pro_binary` is built for extreme performance. Our AOT benchmarks show massive improvements over standard Dart approaches:

### String Encoding (One-Pass vs `utf8.encode`)

Our highly optimized one-pass string encoder is **up to 2.7x faster** than standard `utf8.encode`.

| Payload | `pro_binary` (One-Pass) | Standard (`utf8.encode`) | Speedup |
| :--- | :--- | :--- | :--- |
| **ASCII** | 0.79 μs | 2.15 μs | **2.7x** |
| **Mixed UTF-8** | 1.15 μs | 2.62 μs | **2.28x** |
| **Emoji / Complex** | 1.91 μs | 4.17 μs | **2.18x** |

### Object Serialization & Deserialization

Extremely low overhead for serializing and deserializing Dart objects.

| Scenario | Serialization | Deserialization |
| :--- | :--- | :--- |
| **Simple Message** | 0.31 μs | 0.14 μs |
| **Complex Profile** | 1.62 μs | 1.73 μs |
| **10K integers array** | 403.5 μs | 284.5 μs |

### Object Pooling

Using `BinaryWriterPool` reduces allocation overhead and virtually eliminates GC (Garbage Collector) pauses during high-frequency writes (like game servers or real-time trading).

---
Run these benchmarks yourself to see it in action:

```bash
# Serialization (Writer)
Expand Down Expand Up @@ -208,6 +267,21 @@ dart test
dart test --coverage=coverage
```

## Contributing

Contributions are welcome! Please ensure that all tests pass and code is formatted before submitting a Pull Request.

```bash
# Formatter
dart format .

# Analyzer
dart analyze

# Tests
dart test
```

## License

MIT License. See [LICENSE](LICENSE) for details.
3 changes: 2 additions & 1 deletion example/basic/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ void main() {
_log('\nPool API');
final pooledBytes = BinaryWriterPool.withWriter((w) {
user.encode(w);
return w.toBytes(); // toBytes() returns a zero-copy view
// safely copy data, keeping internal buffer pooled
return w.takeBytes(copy: true);
});
_log(' Pooled serialization done: ${pooledBytes.length} bytes');

Expand Down
6 changes: 2 additions & 4 deletions example/file_streaming/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,12 @@ void main() async {
).encode(writer);

if (writer.bytesWritten >= 64000) {
ios.add(writer.toBytes());
writer.seek(0);
ios.add(writer.takeBytes(copy: true));
}
}

if (writer.bytesWritten > 0) {
ios.add(writer.toBytes());
writer.seek(0);
ios.add(writer.takeBytes(copy: true));
}

writeWatch.stop();
Expand Down
Loading
Loading