Developer documents

AGENTS.md

@@ -0,0 +1 @@
+@DEVELOPMENT.md

CLAUDE.md

@@ -0,0 +1 @@
+@DEVELOPMENT.md

DEVELOPMENT.md

@@ -0,0 +1,142 @@
+# Development
+
+## Architecture
+
+The project consists of three layers with different levels of abstraction.
+
+- InstantReplay (high-level C# API that depends on Unity)
+  - Asynchronous encoder pipeline
+  - Ring buffer for compressed data
+  - Session management
+  - Implementations of video and audio providers
+- UniEnc (low-level C# API that does not depend on Unity)
+  - Also published on NuGet
+  - Interfaces for encoders and muxers
+- unienc_c (native library written in Rust)
+  - Per-platform abstraction of encoders and muxers
+  - Functions exposed through a C FFI
+  - Details: `InstantReplay.Externals/unienc/README.md`
+
+## Prerequisites
+
+- Unity 2022.3 or later. The exact editor version of the development project is recorded in `ProjectSettings/ProjectVersion.txt`.
+- .NET SDK 8 or later, to build and run the projects in `InstantReplay.Externals.sln`.
+- The toolchains for the native library are only required when building it locally. See `InstantReplay.Externals/unienc/README.md`.
+
+### Cloning the Repository
+
+The Rust workspace depends on git submodules. Clone the repository with the submodules, or initialize them after cloning.
+
+```sh
+git clone --recursive <repository URL>
+# or, in an existing clone
+git submodule update --init --recursive
+```
+
+`Assets/Samples` is a symbolic link. On Windows, enable symbolic links for git (`git config core.symlinks true`, which requires Developer Mode or administrator privileges) before cloning; otherwise the link is checked out as a plain text file.
+
+The prebuilt native binaries are committed to the repository, so the Unity project and the .NET projects can be opened without building the native library.
+
+## Project Structure
+
+- InstantReplay: Root of the Unity project
+  - Assets
+    - Samples: Symbolic link to `Packages/jp.co.cyberagent.instant-replay/Samples~`
+  - Packages
+    - jp.co.cyberagent.instant-replay: Main package
+      - Runtime
+        - InstantReplay.asmdef
+      - UniEnc
+        - Plugins: Native libraries for Unity
+        - Runtime
+          - UniEnc.asmdef
+      - UniEnc.Unity
+        - Runtime
+          - UniEnc.Unity.asmdef: Unity-dependent helpers for UniEnc
+        - Editor
+          - UniEnc.Unity.Editor.asmdef
+      - Suite: Recorder interface suite (see `docs/recorder-interface-suite.md`)
+        - Runtime
+          - InstantReplay.Suite.asmdef
+        - UI
+          - InstantReplay.Suite.UI.asmdef
+        - Sharing
+          - Slack
+            - InstantReplay.Suite.Sharing.Slack.asmdef
+          - Discord
+            - InstantReplay.Suite.Sharing.Discord.asmdef
+      - Resources: Runtime assets such as shaders
+      - Samples~
+      - UniversalRP: Video provider for the Universal Render Pipeline
+      - Cri: Audio provider for CRIWARE
+      - Wwise: Audio provider for Wwise
+    - jp.co.cyberagent.instant-replay.dependencies: Auxiliary package used to install the dependencies through UnityNuGet
+  - InstantReplay.Externals: Directory for source files outside of Unity
+    - src: Sources for pure .NET
+      - InstantReplay.DiskBuffer.Tests: End-to-end tests of the disk-buffered mode
+      - UniEnc: C# project used to publish the NuGet package
+        - Plugins: Native libraries for the NuGet package (Unity-dependent features are omitted)
+      - UniEnc.Example: End-to-end sample of UniEnc
+    - InstantReplay.Externals.sln: Solution for pure .NET
+    - unienc: Rust workspace
+  - docs: Design documents
+  - THIRD-PARTY-NOTICES.md: License notices of the dependencies used by the unienc native library
+  - README.md
+  - README.ja.md
+  - CHANGELOG.md
+  - DEVELOPMENT.md: This document
+
+### EXCLUDE_INSTANTREPLAY
+
+`EXCLUDE_INSTANTREPLAY` is a scripting define symbol that excludes the contents of the Instant Replay package from a build without removing the package from the dependencies. When the symbol is defined, the contents are excluded by the following mechanisms.
+
+| Kind of content | Mechanism |
+| --- | --- |
+| Assemblies | Every asmdef in the package has `!EXCLUDE_INSTANTREPLAY` in `defineConstraints`. |
+| Native plugins | `UniEnc.Unity/Editor/PluginsExcluder.cs` disables every plugin under `UniEnc/Plugins/` for the target platform before the build and restores the setting after the build. |
+| Shaders loaded through Resources | `UniEnc.Unity/Editor/ShaderStripper.cs` strips all variants of the shaders listed in it. |
+| Assets generated at build time | Build hooks such as `UniEnc.Unity/Editor/NativeLogLevelBaker.cs` are compiled only when the symbol is not defined. |
+
+When adding a new asset, confirm that it is covered by one of these mechanisms:
+
+- A new asmdef must have `!EXCLUDE_INSTANTREPLAY` in `defineConstraints`.
+- A new native plugin must be placed under `UniEnc/Plugins/`, or `PluginsExcluder` must be extended to cover it.
+- A new shader or other asset in a `Resources` directory is included in the build regardless of assemblies. A shader must be added to `ShaderStripper`; any other kind of asset needs its own exclusion.
+- A new build hook that generates assets must be enclosed in `#if !EXCLUDE_INSTANTREPLAY`.
+
+## Release
+
+### Procedure
+
+1. Create a branch for the release from the latest `main`, for example `feature/release_v1.2.3`.
+2. Update the `version` field of both `Packages/jp.co.cyberagent.instant-replay/package.json` and `Packages/jp.co.cyberagent.instant-replay.dependencies/package.json`.
+3. Add a section for the new version to `CHANGELOG.md`. The changelog is written for the users of the package, so describe how each change affects them rather than how it is implemented.
+4. If the unienc sources have changed since the last build of the native binaries, run the `build-unienc.yml` workflow on the release branch. See `InstantReplay.Externals/unienc/README.md` for details.
+5. Open a pull request. `check-native-binaries.yml` runs on the pull request because `package.json` has changed, and fails if the native binaries are outdated.
+6. Merge the pull request into `main`.
+7. Create a GitHub release on `main` with a tag named `v<version>`, for example `v1.2.3`. Publishing the release triggers the workflows described below.
+
+### UPM
+
+The Instant Replay package is installed through UPM as a git dependency that references the `release` branch. When a new release is published, the `update-release-branch.yml` workflow automatically moves the `release` branch to the tag of the release.
+
+### NuGet
+
+The `publish-nuget.yml` workflow is triggered by a new release and publishes the NuGet package built from `InstantReplay.Externals/src/UniEnc/UniEnc.csproj`. The package version is taken from the tag name with the leading `v` removed.
+
+## Testing
+
+### Native Library
+
+CI runs only the tests of the native library; the Unity and C# sides are not tested in CI. The tests run on iOS (simulator), Android (emulator), Windows, macOS, Linux and WebAssembly (headless Chrome). See `InstantReplay.Externals/unienc/README.md` for running them locally.
+
+### Disk Buffer
+
+`InstantReplay.Externals/src/InstantReplay.DiskBuffer.Tests` compiles the storage layer of the disk-buffered mode without UnityEngine and exercises it together with the real native encoder. It is a console application that returns a non-zero exit code when a check fails.
+
+```sh
+cd InstantReplay.Externals
+dotnet run --project src/InstantReplay.DiskBuffer.Tests
+```
+
+The native library is loaded from `src/UniEnc/Plugins`, so the test runs only on the platforms for which a NuGet variant of the native library is present there (Windows x64, macOS and Linux x64).

InstantReplay.Externals/unienc/README.md

@@ -12,9 +12,106 @@ A Rust-based unified encoding system that provides cross-platform video and audi
 | **Linux / Unix** | FFmpeg | FFmpeg | FFmpeg | - |
 | **WebAssembly** (Emscripten) | WebCodecs API | WebCodecs API | muxide (MP4) | - |
 
+## Prerequisites
+
+- A stable Rust toolchain installed through rustup, with the target of each platform to build (`rustup target add <target>`).
+- The git submodules under `external/` (`git submodule update --init --recursive` at the root of the repository).
+- Additional tools for each platform:
+
+| Platform | Requirements |
+|----------|--------------|
+| Android | Android NDK (`ANDROID_NDK_HOME`), `cargo-ndk` (`cargo install cargo-ndk`). For the tests: Android SDK (`ANDROID_HOME`), a JDK and the build-tools for `d8`. |
+| iOS | Xcode. For the tests: an iOS simulator runtime. |
+| Linux | `gcc-multilib`. For the tests: the `ffmpeg` executable with libx264 and an AAC encoder. |
+| WebAssembly | A nightly toolchain with `rust-src` (`rustup toolchain install nightly --component rust-src`), an activated emsdk (`source emsdk_env.sh`) and TypeScript (`tsc`). For the tests: Google Chrome. Chromium does not work because it does not ship the H.264 codec. |
+
 ## Build
 
-See .github/workflows/build-unienc.yml at the root of the repository for the CI build configuration. The build process is automated and handles platform-specific compilation and linking.
+### Variants
+
+The native library is built in two variants from `crates/unienc_c`.
+
+| Variant | Cargo features | Destination |
+|---------|----------------|-------------|
+| Unity | `-F unity,mimalloc` | `Packages/jp.co.cyberagent.instant-replay/UniEnc/Plugins/<target>/` |
+| NuGet | default features | `InstantReplay.Externals/src/UniEnc/Plugins/<target>/` |
+
+The NuGet variant omits the Unity-dependent features. The WebAssembly build is produced only for the Unity variant.
+
+### Building Locally
+
+The commands below are run in `crates/unienc_c` and correspond to the jobs of `.github/workflows/build-unienc.yml` at the root of the repository, which is the authoritative build configuration. Replace `--release` with nothing for a debug build, and omit `-F unity,mimalloc` for the NuGet variant.
+
+```sh
+# macOS (each architecture)
+cargo build --target aarch64-apple-darwin --release -F unity,mimalloc
+cargo build --target x86_64-apple-darwin --release -F unity,mimalloc
+
+# iOS
+CFLAGS_aarch64-apple-ios=-fno-common IPHONEOS_DEPLOYMENT_TARGET=10.0 \
+  cargo build --target aarch64-apple-ios --release -F unity,mimalloc
+
+# Android (each ABI)
+cargo ndk -t aarch64-linux-android --platform 26 build --release -F unity,mimalloc
+
+# Windows
+cargo build --target x86_64-pc-windows-msvc --release -F unity,mimalloc
+
+# Linux
+cargo build --target x86_64-unknown-linux-gnu --release -F unity,mimalloc
+```
+
+The WebAssembly build is run in the root of the workspace:
+
+```sh
+RUSTFLAGS=-Ctarget-cpu=mvp cargo +nightly rustc -Z json-target-spec -Z build-std=panic_abort,std \
+  --target wasm32-unknown-emscripten --profile release-wasm --no-default-features --crate-type staticlib -p unienc_c
+```
+
+The outputs are written to `target/<target>/<profile>/`. The workflow applies the following post-processing before placing them in the destination directories:
+
+- macOS (Unity variant): The two architectures are combined into a universal binary with `lipo -create -output libunienc_c.bundle ...` and placed in `apple-darwin/`. The NuGet variant keeps one `.dylib` per architecture.
+- iOS (Unity variant): `.github/scripts/localize-apple-staticlib.sh libunienc_c.a arm64 iphoneos` localizes the non-exported symbols, so that the bundled mimalloc does not conflict with the mimalloc built into Unity when the library is linked statically into UnityFramework. `-fno-common` above is required for the same reason.
+- Windows: `unienc_c.dll` is renamed to `libunienc_c.dll`.
+- Destination directory names drop the vendor and ABI suffix of some targets: `x86_64-pc-windows`, `x86_64-unknown-linux`.
+
+### Updating the Committed Binaries
+
+The native binaries committed to the repository must be produced by the `build-unienc.yml` workflow. Run it with `workflow_dispatch` on the branch to update. The workflow builds every platform and variant, places the binaries, regenerates `THIRD-PARTY-NOTICES.md`, writes the source fingerprint, and commits the result to the branch.
+
+The source fingerprint (`Packages/jp.co.cyberagent.instant-replay/UniEnc/.source-fingerprint`) is computed by `.github/scripts/compute-unienc-fingerprint.sh` from the git object hashes of this workspace (including the submodule pointers), `build-unienc.yml` and `localize-apple-staticlib.sh`. On a pull request that changes the package version, `check-native-binaries.yml` recomputes the fingerprint and fails if it differs from the committed one, which means that the binaries have not been rebuilt since the sources changed.
+
+Binaries built locally are useful for development and debugging, but are not to be committed, because the fingerprint would not be updated together with them.
+
+## Testing
+
+`crates/unienc_testkit` contains a shared end-to-end harness that drives the encoding system of the host platform (video encoder, audio encoder and muxer). The GPU texture path is not covered because it requires a Unity graphics device. Muxed outputs are written under `target/` for inspection.
+
+| Platform | Command (run in the root of the workspace) |
+|----------|--------------------------------------------|
+| macOS | `cargo test -p unienc_testkit -p unienc_common -p unienc_apple_vt` |
+| Windows | `cargo test -p unienc_testkit -p unienc_common -p unienc_windows_mf` |
+| Linux | `cargo test -p unienc_testkit -p unienc_common -p unienc_ffmpeg` |
+| iOS simulator | `scripts/ios-simulator-test.sh [<simulator name or UDID>]` |
+| Android device or emulator | `scripts/android-device-test.sh [--release] [-s <adb serial>]` |
+| WebAssembly | `scripts/web-browser-test.sh [--release]` |
+
+- iOS: The test binary is run directly in a booted simulator through `simctl`, so no code signing is required. The script boots a simulator if none is running. The static library link and the hardware encoder of a real device are not covered.
+- Android: The test binary is run through `app_process` on the device, so no APK is built. The same script works on an emulator and on a real device; the emulator uses a software encoder, so vendor-specific behavior of hardware encoders is covered only on a real device.
+- WebAssembly: The harness is run as a page in Chrome through emrun. Set `UNIENC_TEST_BROWSER` to use a browser at a non-default location.
+
+`cargo test` does not build the FFI layer, so also run `cargo check -p unienc_c` after changing the signatures of a platform backend. CI additionally checks the formatting with `cargo fmt --all -- --check`. See `.github/workflows/ci-unienc.yml` at the root of the repository for the CI configuration.
+
+## Third-Party Notices
+
+`THIRD-PARTY-NOTICES.md` at the root of the repository lists the licenses of the Rust dependencies linked into the Unity variant. It is generated with cargo-about, using `about.toml` (the list of accepted licenses) and `about.hbs` (the template). The `build-unienc.yml` workflow regenerates it on every build. To regenerate it locally, run the following in the root of the workspace:
+
+```sh
+cargo install cargo-about --locked --features cli
+cargo about generate --features unity,mimalloc about.hbs > ../../THIRD-PARTY-NOTICES.md
+```
+
+If a new dependency uses a license that is not in `about.toml`, the generation fails. Review the license before adding it to the accepted list.
 
 ## Architecture
 
@@ -46,8 +143,10 @@ unienc (platform selector via cfg)
 
 ### External Dependencies
 
-- `external/unity-native-plugin-rs/` — Unity native plugin SDK for Rust
+The following crates are included as git submodules and replace the crates.io versions through `[patch.crates-io]` in `Cargo.toml`.
+
 - `external/muxide/` — MP4 multiplexer (used by `unienc_webcodecs`)
+- `external/mimalloc_rust/` — Fork of the mimalloc allocator bindings (used by the `mimalloc` feature)
 
 ### Key Traits