Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 57 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Add `EncodeConfig::new(codec, width, height)`, for a codec chosen at
runtime.
- Add RGB input: `EncodeConfig::with_source(Source::Rgb(..))`. The encoder
converts RGB frames to the YUV its colour description describes, through
`VK_VALVE_video_encode_rgb_conversion` where the device can and the colour
Expand All @@ -21,12 +23,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
waits for them.
- Add `Encoder::input_slot` and `InputSlot`: the encoder's own input image,
to write a frame in place and submit with `Frame::input`.
- Add `InputImage::source`.
- Add `YuvUploader::source`.
- Add `Encoder::repeat`: encode the last frame again from the encoder's own
input image, for a key frame while no new frame arrives. The caller's
source does not have to stay alive for it.
- Add DMA-BUF input (feature `dmabuf`): `Encoder::input_formats`,
`Encoder::import`, `Encoder::allocate_input`, `ExternalImage` and
- Add DMA-BUF input (feature `dmabuf`): `Encoder::dmabuf_format`,
`Encoder::import`, `Encoder::allocate_input`, `DmaBufImage` and
`Frame::external`. The encoder reads a buffer directly where the device
can, and copies it otherwise.
- Add sync_file fences (feature `dmabuf`): `Frame::wait_sync_file`,
Expand Down Expand Up @@ -62,9 +64,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Breaking:** `EncodeConfig::with_b_frames` and the public `b_frame_count`
field are removed. No codec encodes B-frames yet, and every encoder refused
a count above 0 at runtime.
- **Breaking:** `InputImage::upload_yuv420_to` and `upload_yuv444_to` take an
`InputSlot` instead of a `vk::Image`. `InputImage` uploads wait until the
frames that read the image are done with it.
- **Breaking:** `YuvUploader` has two uploads, `upload` to its own image and
`upload_to` an encoder's `InputSlot`, which take a `YuvLayout`: `I420`,
`Nv12`, `I420P10` or `I444`. They replace `upload_yuv420`, `upload_nv12`,
`upload_yuv420_10bit`, `upload_yuv420_to` and `upload_yuv444_to`, which
covered only some of the combinations. A layout the image cannot hold is
an error; `upload_nv12` panicked for a 10-bit or 4:4:4 image. The uploads
wait until the frames that read the image are done with it.
- **Breaking:** `YuvUploader::new` no longer takes a `Codec`, which it did not
use. `YuvUploader::for_encoder` makes one for an encoder's frames.
- **Breaking:** `EncodeConfig::frame_rate` (a `FrameRate`) replaces
`frame_rate_numerator` and `frame_rate_denominator`. `with_frame_rate` is
unchanged.
- **Breaking:** the encoder's `BitDepth` is exported as `BitDepth`, not
`EncodeBitDepth`. `Dimensions` and `FrameRate` are exported from the crate
root.
- **Breaking:** the `DEFAULT_*` encoder constants are no longer public.
`EncodeConfig::new` and `RateControl::default_for` apply them.
- **Breaking:** `PixelForgeError` has fewer, typed variants. A failed
Vulkan call is `Vulkan { call, result }`, with the function's name and its
`vk::Result`, in place of `DeviceCreation`, `VideoSessionCreation`,
`SessionParametersCreation`, `MemoryAllocation`, `ResourceCreation`,
`CommandBuffer`, `Synchronization`, `QueryPool` and `Vulkan(vk::Result)`.
A codec or memory type the device lacks is `NoSuitableDevice`, which
replaces `CodecNotSupported`. New are `Driver`, for a driver that reports
success with no usable result, `Io`, for a failed OS call, and
`Cancelled`, for a frame whose encoder shut down. `From<vk::Result>` is
removed.
- Every failed Vulkan call on a lost device is `PixelForgeError::DeviceLost`.
Some were reported as another variant.
- **Breaking:** `EncodedPacket::is_key_frame` is a method, not a field. An
IDR's `frame_type` is `FrameType::Idr`; it was `FrameType::I`.
`FrameType::Unknown`, which no packet had, is removed.
- **Breaking:** types are renamed for what they are: `InputImage` is
`YuvUploader`, `InputFormat` is `RgbFormat`, `PixelFormat` is
`ChromaFormat` (with `EncodeConfig::chroma_format`, `with_chroma_format`
and `DecodedFrame::chroma_format`), and the DMA-BUF types are
`DmaBufImage`, `DmaBufFormat` and `DmaBufModifier`, from
`Encoder::dmabuf_format`. They were `ExternalImage`, `InputFormats`,
`InputModifier` and `Encoder::input_formats`.
- **Breaking:** `VideoContextBuilder::device_requirements` and
`build_from_existing` adopt a caller's device for the codecs asked for with
`require_encode` and `require_decode`, so one adopted context can encode
and decode. They replace `encode_device_requirements`,
`decode_device_requirements`, `build_from_existing_encode` and
`build_from_existing_decode`. At least one codec must be asked for: an
encode adoption with none no longer takes every codec the device has.
- **Breaking:** `StreamColor` replaces the raw `ColorDescription` in the
encode API: `EncodeConfig::with_color` replaces `with_color_description`,
`Encoder::reconfigure` replaces `set_color_description`, and
Expand Down Expand Up @@ -94,6 +139,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `wall_latency_ns` starts before the encode submit. It started after it, so
a submit that blocked in the driver made it shorter than the GPU time.

### Removed

- The encoder's internal modules are no longer public:
`encoder::{av1, bitwriter, dpb, gop, h264, h265, reorder, resources}` and
`decoder::h264`. Nothing in them was needed to encode or decode.

## [0.10.0] - 19-10-2026

### Added
Expand Down
41 changes: 16 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ H.265 and AV1 encode, and H.264 decode.
- **GPU-native API**: Encode directly from Vulkan images (`vk::Image`).
- **Flexible configuration**: Rate control (CBR, VBR, CQP), quality levels, GOP settings.
- **Multiple input formats**: BGRx, RGBx, BGRA, RGBA, ABGR2101010 (10-bit packed), RGBA16F (FP16).
- **Utility helpers**: [`InputImage`] for easy YUV data upload to GPU.
- **Utility helpers**: [`YuvUploader`] for easy YUV data upload to GPU.
- **Optional DMA-BUF input**: import a producer's buffers, or allocate buffers
for it, read with no copy where the driver allows, with sync_file fences
(Linux only).
Expand Down Expand Up @@ -96,10 +96,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
### Encoding Video

```rust
use pixelforge::{
Codec, EncodeBitDepth, EncodeConfig, Encoder, InputImage, PixelFormat, RateControl,
VideoContextBuilder,
};
use pixelforge::{Codec, EncodeConfig, Encoder, YuvUploader, RateControl, VideoContextBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
let context = VideoContextBuilder::new()
Expand All @@ -115,20 +112,13 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
.with_frame_rate(30, 1)
.with_gop_size(60);

// Create an InputImage helper for uploading YUV data to the GPU.
let mut input_image = InputImage::new(
context.clone(),
Codec::H264,
1920,
1080,
EncodeBitDepth::Eight,
PixelFormat::Yuv420,
)?;
let mut encoder = Encoder::new(context, config)?;
// Create a YuvUploader helper for uploading YUV data to the GPU.
let mut input_image = YuvUploader::for_encoder(&encoder)?;

// For each frame: upload YUV data and encode.
// let yuv_data: &[u8] = ...; // YUV420 frame data
// input_image.upload_yuv420(yuv_data)?;
// input_image.upload(YuvLayout::I420, yuv_data)?;
// let packet = encoder.submit(Frame::image(input_image.source()))?;

Ok(())
Expand Down Expand Up @@ -205,7 +195,7 @@ stream's label cannot disagree, and a colour change through

```rust
use pixelforge::{
Codec, ColorRange, ColorSpec, EncodeConfig, Encoder, InputFormat, RgbSource, Source,
Codec, ColorRange, ColorSpec, EncodeConfig, Encoder, RgbFormat, RgbSource, Source,
StreamColor, VideoContextBuilder,
};

Expand All @@ -215,8 +205,8 @@ let context = VideoContextBuilder::new()

// SDR desktop content, encoded as HDR10 for an HDR streaming session.
let config = EncodeConfig::h265(1920, 1080)
.with_bit_depth(pixelforge::EncodeBitDepth::Ten)
.with_source(Source::Rgb(RgbSource::new(InputFormat::BGRx, ColorSpec::Srgb)))
.with_bit_depth(pixelforge::BitDepth::Ten)
.with_source(Source::Rgb(RgbSource::new(RgbFormat::BGRx, ColorSpec::Srgb)))
.with_color(StreamColor::Hdr10 {
range: ColorRange::Full,
});
Expand Down Expand Up @@ -270,9 +260,9 @@ flight. The pixels are in one of:
- a [`SourceImage`], an image of the caller's. Pixelforge copies or
converts it into the input image after the call returns, and the
`SourceImage` keeps track of that: write the image again only after
[`SourceImage::released`]. [`InputImage`], for frames uploaded from the
[`SourceImage::released`]. [`YuvUploader`], for frames uploaded from the
CPU, waits for that itself.
- an `ExternalImage`, a DMA-BUF (see below).
- a `DmaBufImage`, a DMA-BUF (see below).

### Encoding on your own device

Expand All @@ -281,9 +271,10 @@ device instead of letting it create a second one. Frames then never leave
the device: no external memory, no import, and no CPU wait between the
application's work and pixelforge's.

[`VideoContextBuilder::encode_device_requirements`] lists the queue
families, extensions and features to create the device with, and
[`VideoContextBuilder::build_from_existing_encode`] adopts it. A `VkQueue`
[`VideoContextBuilder::device_requirements`] lists the queue
families, extensions and features to create the device with, for the
codecs asked for with `require_encode` and `require_decode`, and
[`VideoContextBuilder::build_from_existing`] adopts it. A `VkQueue`
may not be submitted to from two threads at once, so give pixelforge queues
the application does not use, with
[`VideoContextBuilder::with_encode_queue`] and friends.
Expand All @@ -305,12 +296,12 @@ let reusable = source.released();
### DMA-BUF input

With the `dmabuf` feature, an encoder reads DMA-BUFs from another process.
`Encoder::input_formats` gives the DRM format it wants and, per DRM
`Encoder::dmabuf_format` gives the DRM format it wants and, per DRM
modifier, whether it reads such a buffer with no copy. Offer those
modifiers to the producer first. `Encoder::import` takes a producer's
buffer; `Encoder::allocate_input` makes one for a producer that writes
into buffers the consumer supplies, such as `ext-image-copy-capture`.
Either is checked once and gives an `ExternalImage`, which
Either is checked once and gives a `DmaBufImage`, which
`Frame::external` submits. A buffer the encoder cannot read directly is
copied on the GPU instead.

Expand Down
17 changes: 5 additions & 12 deletions benches/encode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
//! cargo bench --bench encode

use pixelforge::{
Codec, EncodeBitDepth, EncodeConfig, Encoder, EncoderTuningMode, Frame, InputImage,
PixelFormat, RateControl, VideoContextBuilder,
Codec, EncodeConfig, Encoder, EncoderTuningMode, Frame, RateControl, VideoContextBuilder,
YuvLayout, YuvUploader,
};
use std::fs::File;
use std::io::Read;
Expand Down Expand Up @@ -134,16 +134,9 @@ fn run_encode_test(

println!("Config: {:?}, GOP={}", config.rate_control, config.gop_size);

// Create input image for uploading frames.
let mut input_image = InputImage::new(
context.clone(),
codec,
WIDTH,
HEIGHT,
EncodeBitDepth::Eight,
PixelFormat::Yuv420,
)?;
let mut encoder = Encoder::new(context.clone(), config)?;
// Create input image for uploading frames.
let mut input_image = YuvUploader::for_encoder(&encoder)?;
let mut total_bytes = 0;

let mut gpu_durations: Vec<Duration> = Vec::new();
Expand All @@ -169,7 +162,7 @@ fn run_encode_test(
let frame = &yuv_data[i * frame_size..(i + 1) * frame_size];

// Upload YUV420 data to the input image
input_image.upload_yuv420(frame)?;
input_image.upload(YuvLayout::I420, frame)?;

pending.push_back(encoder.submit(Frame::image(input_image.source()))?);
// drain before submitting when at capacity
Expand Down
21 changes: 7 additions & 14 deletions examples/encode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
//! The codec defaults to `h264`.

use pixelforge::{
Codec, EncodeBitDepth, EncodeConfig, Encoder, Frame, InputImage, PixelFormat, RateControl,
VideoContextBuilder,
ChromaFormat, Codec, EncodeConfig, Encoder, Frame, RateControl, VideoContextBuilder, YuvLayout,
YuvUploader,
};
use std::fs::File;
use std::io::{Read, Write};
Expand Down Expand Up @@ -46,7 +46,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
),
"h265" => (
Codec::H265,
EncodeConfig::h265(WIDTH, HEIGHT).with_pixel_format(PixelFormat::Yuv420),
EncodeConfig::h265(WIDTH, HEIGHT).with_chroma_format(ChromaFormat::Yuv420),
"output.h265",
),
"av1" => (Codec::AV1, EncodeConfig::av1(WIDTH, HEIGHT), "output.av1"),
Expand Down Expand Up @@ -101,16 +101,9 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
config.rate_control, config.gop_size
);

// Create input image for uploading frames.
let mut input_image = InputImage::new(
context.clone(),
codec,
WIDTH,
HEIGHT,
EncodeBitDepth::Eight,
PixelFormat::Yuv420,
)?;
let mut encoder = Encoder::new(context, config)?;
// Create input image for uploading frames.
let mut input_image = YuvUploader::for_encoder(&encoder)?;
let mut output = File::create(output_name)?;
let mut total_bytes = 0;

Expand All @@ -129,7 +122,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
packet.dts,
packet.data.len(),
packet.frame_type,
if packet.is_key_frame { " [KEY]" } else { "" }
if packet.is_key_frame() { " [KEY]" } else { "" }
);
Ok::<(), Box<dyn std::error::Error>>(())
};
Expand All @@ -139,7 +132,7 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
let frame = &yuv_data[i * frame_size..(i + 1) * frame_size];

// Upload YUV420 data to the input image.
input_image.upload_yuv420(frame)?;
input_image.upload(YuvLayout::I420, frame)?;

// Submit the frame (async) and keep the pipeline at most ~2 deep.
pending.push_back(encoder.submit(Frame::image(input_image.source()))?);
Expand Down
2 changes: 1 addition & 1 deletion examples/sample_frame.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ impl Sampler {
frame: &DecodedFrame,
) -> Result<Self, Box<dyn std::error::Error>> {
let device = context.device().clone();
let format = frame.pixel_format;
let format = frame.chroma_format;
let _ = format;
let picture_format = vk::Format::G8_B8R8_2PLANE_420_UNORM;

Expand Down
23 changes: 10 additions & 13 deletions src/converter/device_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,11 @@
mod source;
use source::{SrcImage, create_src_image, host_buffer, one_shot};

use super::{
ColorConverter, ColorConverterConfig, ColorRange, ColorSpec, InputFormat, OutputFormat,
};
use super::{ColorConverter, ColorConverterConfig, ColorRange, ColorSpec, OutputFormat, RgbFormat};
use crate::encoder::input::SourceOwnership;
use crate::{
Codec, ColorDescription, ConversionPreference, EncodeBitDepth, EncodeConfig, Encoder, Frame,
Handoff, RateControl, RgbSource, Source, SourceImage, StreamColor, VideoContext,
VideoContextBuilder,
BitDepth, Codec, ColorDescription, ConversionPreference, EncodeConfig, Encoder, Frame, Handoff,
RateControl, RgbSource, Source, SourceImage, StreamColor, VideoContext, VideoContextBuilder,
};
use ash::vk;

Expand Down Expand Up @@ -194,7 +191,7 @@ fn encoders(
let ten = Encoder::new(
context.clone(),
EncodeConfig::h265(WIDTH, HEIGHT)
.with_bit_depth(EncodeBitDepth::Ten)
.with_bit_depth(BitDepth::Ten)
.with_frame_rate(30, 1),
)
.ok();
Expand Down Expand Up @@ -256,7 +253,7 @@ fn convert(
let config = ColorConverterConfig::new(
WIDTH,
HEIGHT,
InputFormat::BGRA,
RgbFormat::BGRA,
output_format,
source,
target,
Expand Down Expand Up @@ -428,7 +425,7 @@ fn unsupported_conversions_are_refused() -> Result<(), Box<dyn std::error::Error
let config = ColorConverterConfig::new(
WIDTH,
HEIGHT,
InputFormat::BGRA,
RgbFormat::BGRA,
OutputFormat::NV12,
source,
target,
Expand Down Expand Up @@ -470,7 +467,7 @@ fn a_replaced_source_image_is_read_afresh() -> Result<(), Box<dyn std::error::Er
let config = ColorConverterConfig::new(
WIDTH,
HEIGHT,
InputFormat::BGRA,
RgbFormat::BGRA,
OutputFormat::NV12,
ColorSpec::Srgb,
ColorSpec::Srgb,
Expand Down Expand Up @@ -539,7 +536,7 @@ fn by_hand_and_by_encoder(
ColorConverterConfig::new(
ENCODE_WIDTH,
ENCODE_HEIGHT,
InputFormat::BGRA,
RgbFormat::BGRA,
OutputFormat::NV12,
ColorSpec::Srgb,
ColorSpec::Srgb,
Expand All @@ -550,7 +547,7 @@ fn by_hand_and_by_encoder(
context.clone(),
encode_config(codec)
.with_source(Source::Rgb(RgbSource::new(
InputFormat::BGRA,
RgbFormat::BGRA,
ColorSpec::Srgb,
)))
.with_conversion(ConversionPreference::Shader),
Expand Down Expand Up @@ -660,7 +657,7 @@ fn encode_is_deterministic() -> Result<(), Box<dyn std::error::Error>> {
ColorConverterConfig::new(
ENCODE_WIDTH,
ENCODE_HEIGHT,
InputFormat::BGRA,
RgbFormat::BGRA,
OutputFormat::NV12,
ColorSpec::Srgb,
ColorSpec::Srgb,
Expand Down
Loading
Loading