diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..ccbe5501 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +@DEVELOPMENT.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..ccbe5501 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@DEVELOPMENT.md diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 00000000..7f35ec41 --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,142 @@ +# Development + +## Architecture + +The project consists of three layers with different levels of abstraction. + +- InstantReplay (high-level C# API that depends on Unity) + - Asynchronous encoder pipeline + - Ring buffer for compressed data + - Session management + - Implementations of video and audio providers +- UniEnc (low-level C# API that does not depend on Unity) + - Also published on NuGet + - Interfaces for encoders and muxers +- unienc_c (native library written in Rust) + - Per-platform abstraction of encoders and muxers + - Functions exposed through a C FFI + - Details: `InstantReplay.Externals/unienc/README.md` + +## Prerequisites + +- Unity 2022.3 or later. The exact editor version of the development project is recorded in `ProjectSettings/ProjectVersion.txt`. +- .NET SDK 8 or later, to build and run the projects in `InstantReplay.Externals.sln`. +- The toolchains for the native library are only required when building it locally. See `InstantReplay.Externals/unienc/README.md`. + +### Cloning the Repository + +The Rust workspace depends on git submodules. Clone the repository with the submodules, or initialize them after cloning. + +```sh +git clone --recursive +# or, in an existing clone +git submodule update --init --recursive +``` + +`Assets/Samples` is a symbolic link. On Windows, enable symbolic links for git (`git config core.symlinks true`, which requires Developer Mode or administrator privileges) before cloning; otherwise the link is checked out as a plain text file. + +The prebuilt native binaries are committed to the repository, so the Unity project and the .NET projects can be opened without building the native library. + +## Project Structure + +- InstantReplay: Root of the Unity project + - Assets + - Samples: Symbolic link to `Packages/jp.co.cyberagent.instant-replay/Samples~` + - Packages + - jp.co.cyberagent.instant-replay: Main package + - Runtime + - InstantReplay.asmdef + - UniEnc + - Plugins: Native libraries for Unity + - Runtime + - UniEnc.asmdef + - UniEnc.Unity + - Runtime + - UniEnc.Unity.asmdef: Unity-dependent helpers for UniEnc + - Editor + - UniEnc.Unity.Editor.asmdef + - Suite: Recorder interface suite (see `docs/recorder-interface-suite.md`) + - Runtime + - InstantReplay.Suite.asmdef + - UI + - InstantReplay.Suite.UI.asmdef + - Sharing + - Slack + - InstantReplay.Suite.Sharing.Slack.asmdef + - Discord + - InstantReplay.Suite.Sharing.Discord.asmdef + - Resources: Runtime assets such as shaders + - Samples~ + - UniversalRP: Video provider for the Universal Render Pipeline + - Cri: Audio provider for CRIWARE + - Wwise: Audio provider for Wwise + - jp.co.cyberagent.instant-replay.dependencies: Auxiliary package used to install the dependencies through UnityNuGet + - InstantReplay.Externals: Directory for source files outside of Unity + - src: Sources for pure .NET + - InstantReplay.DiskBuffer.Tests: End-to-end tests of the disk-buffered mode + - UniEnc: C# project used to publish the NuGet package + - Plugins: Native libraries for the NuGet package (Unity-dependent features are omitted) + - UniEnc.Example: End-to-end sample of UniEnc + - InstantReplay.Externals.sln: Solution for pure .NET + - unienc: Rust workspace + - docs: Design documents + - THIRD-PARTY-NOTICES.md: License notices of the dependencies used by the unienc native library + - README.md + - README.ja.md + - CHANGELOG.md + - DEVELOPMENT.md: This document + +### EXCLUDE_INSTANTREPLAY + +`EXCLUDE_INSTANTREPLAY` is a scripting define symbol that excludes the contents of the Instant Replay package from a build without removing the package from the dependencies. When the symbol is defined, the contents are excluded by the following mechanisms. + +| Kind of content | Mechanism | +| --- | --- | +| Assemblies | Every asmdef in the package has `!EXCLUDE_INSTANTREPLAY` in `defineConstraints`. | +| Native plugins | `UniEnc.Unity/Editor/PluginsExcluder.cs` disables every plugin under `UniEnc/Plugins/` for the target platform before the build and restores the setting after the build. | +| Shaders loaded through Resources | `UniEnc.Unity/Editor/ShaderStripper.cs` strips all variants of the shaders listed in it. | +| Assets generated at build time | Build hooks such as `UniEnc.Unity/Editor/NativeLogLevelBaker.cs` are compiled only when the symbol is not defined. | + +When adding a new asset, confirm that it is covered by one of these mechanisms: + +- A new asmdef must have `!EXCLUDE_INSTANTREPLAY` in `defineConstraints`. +- A new native plugin must be placed under `UniEnc/Plugins/`, or `PluginsExcluder` must be extended to cover it. +- A new shader or other asset in a `Resources` directory is included in the build regardless of assemblies. A shader must be added to `ShaderStripper`; any other kind of asset needs its own exclusion. +- A new build hook that generates assets must be enclosed in `#if !EXCLUDE_INSTANTREPLAY`. + +## Release + +### Procedure + +1. Create a branch for the release from the latest `main`, for example `feature/release_v1.2.3`. +2. Update the `version` field of both `Packages/jp.co.cyberagent.instant-replay/package.json` and `Packages/jp.co.cyberagent.instant-replay.dependencies/package.json`. +3. Add a section for the new version to `CHANGELOG.md`. The changelog is written for the users of the package, so describe how each change affects them rather than how it is implemented. +4. If the unienc sources have changed since the last build of the native binaries, run the `build-unienc.yml` workflow on the release branch. See `InstantReplay.Externals/unienc/README.md` for details. +5. Open a pull request. `check-native-binaries.yml` runs on the pull request because `package.json` has changed, and fails if the native binaries are outdated. +6. Merge the pull request into `main`. +7. Create a GitHub release on `main` with a tag named `v`, for example `v1.2.3`. Publishing the release triggers the workflows described below. + +### UPM + +The Instant Replay package is installed through UPM as a git dependency that references the `release` branch. When a new release is published, the `update-release-branch.yml` workflow automatically moves the `release` branch to the tag of the release. + +### NuGet + +The `publish-nuget.yml` workflow is triggered by a new release and publishes the NuGet package built from `InstantReplay.Externals/src/UniEnc/UniEnc.csproj`. The package version is taken from the tag name with the leading `v` removed. + +## Testing + +### Native Library + +CI runs only the tests of the native library; the Unity and C# sides are not tested in CI. The tests run on iOS (simulator), Android (emulator), Windows, macOS, Linux and WebAssembly (headless Chrome). See `InstantReplay.Externals/unienc/README.md` for running them locally. + +### Disk Buffer + +`InstantReplay.Externals/src/InstantReplay.DiskBuffer.Tests` compiles the storage layer of the disk-buffered mode without UnityEngine and exercises it together with the real native encoder. It is a console application that returns a non-zero exit code when a check fails. + +```sh +cd InstantReplay.Externals +dotnet run --project src/InstantReplay.DiskBuffer.Tests +``` + +The native library is loaded from `src/UniEnc/Plugins`, so the test runs only on the platforms for which a NuGet variant of the native library is present there (Windows x64, macOS and Linux x64). diff --git a/InstantReplay.Externals/unienc/README.md b/InstantReplay.Externals/unienc/README.md index c2e00af2..ed8ce728 100644 --- a/InstantReplay.Externals/unienc/README.md +++ b/InstantReplay.Externals/unienc/README.md @@ -12,9 +12,106 @@ A Rust-based unified encoding system that provides cross-platform video and audi | **Linux / Unix** | FFmpeg | FFmpeg | FFmpeg | - | | **WebAssembly** (Emscripten) | WebCodecs API | WebCodecs API | muxide (MP4) | - | +## Prerequisites + +- A stable Rust toolchain installed through rustup, with the target of each platform to build (`rustup target add `). +- The git submodules under `external/` (`git submodule update --init --recursive` at the root of the repository). +- Additional tools for each platform: + +| Platform | Requirements | +|----------|--------------| +| Android | Android NDK (`ANDROID_NDK_HOME`), `cargo-ndk` (`cargo install cargo-ndk`). For the tests: Android SDK (`ANDROID_HOME`), a JDK and the build-tools for `d8`. | +| iOS | Xcode. For the tests: an iOS simulator runtime. | +| Linux | `gcc-multilib`. For the tests: the `ffmpeg` executable with libx264 and an AAC encoder. | +| WebAssembly | A nightly toolchain with `rust-src` (`rustup toolchain install nightly --component rust-src`), an activated emsdk (`source emsdk_env.sh`) and TypeScript (`tsc`). For the tests: Google Chrome. Chromium does not work because it does not ship the H.264 codec. | + ## Build -See .github/workflows/build-unienc.yml at the root of the repository for the CI build configuration. The build process is automated and handles platform-specific compilation and linking. +### Variants + +The native library is built in two variants from `crates/unienc_c`. + +| Variant | Cargo features | Destination | +|---------|----------------|-------------| +| Unity | `-F unity,mimalloc` | `Packages/jp.co.cyberagent.instant-replay/UniEnc/Plugins//` | +| NuGet | default features | `InstantReplay.Externals/src/UniEnc/Plugins//` | + +The NuGet variant omits the Unity-dependent features. The WebAssembly build is produced only for the Unity variant. + +### Building Locally + +The commands below are run in `crates/unienc_c` and correspond to the jobs of `.github/workflows/build-unienc.yml` at the root of the repository, which is the authoritative build configuration. Replace `--release` with nothing for a debug build, and omit `-F unity,mimalloc` for the NuGet variant. + +```sh +# macOS (each architecture) +cargo build --target aarch64-apple-darwin --release -F unity,mimalloc +cargo build --target x86_64-apple-darwin --release -F unity,mimalloc + +# iOS +CFLAGS_aarch64-apple-ios=-fno-common IPHONEOS_DEPLOYMENT_TARGET=10.0 \ + cargo build --target aarch64-apple-ios --release -F unity,mimalloc + +# Android (each ABI) +cargo ndk -t aarch64-linux-android --platform 26 build --release -F unity,mimalloc + +# Windows +cargo build --target x86_64-pc-windows-msvc --release -F unity,mimalloc + +# Linux +cargo build --target x86_64-unknown-linux-gnu --release -F unity,mimalloc +``` + +The WebAssembly build is run in the root of the workspace: + +```sh +RUSTFLAGS=-Ctarget-cpu=mvp cargo +nightly rustc -Z json-target-spec -Z build-std=panic_abort,std \ + --target wasm32-unknown-emscripten --profile release-wasm --no-default-features --crate-type staticlib -p unienc_c +``` + +The outputs are written to `target///`. The workflow applies the following post-processing before placing them in the destination directories: + +- macOS (Unity variant): The two architectures are combined into a universal binary with `lipo -create -output libunienc_c.bundle ...` and placed in `apple-darwin/`. The NuGet variant keeps one `.dylib` per architecture. +- iOS (Unity variant): `.github/scripts/localize-apple-staticlib.sh libunienc_c.a arm64 iphoneos` localizes the non-exported symbols, so that the bundled mimalloc does not conflict with the mimalloc built into Unity when the library is linked statically into UnityFramework. `-fno-common` above is required for the same reason. +- Windows: `unienc_c.dll` is renamed to `libunienc_c.dll`. +- Destination directory names drop the vendor and ABI suffix of some targets: `x86_64-pc-windows`, `x86_64-unknown-linux`. + +### Updating the Committed Binaries + +The native binaries committed to the repository must be produced by the `build-unienc.yml` workflow. Run it with `workflow_dispatch` on the branch to update. The workflow builds every platform and variant, places the binaries, regenerates `THIRD-PARTY-NOTICES.md`, writes the source fingerprint, and commits the result to the branch. + +The source fingerprint (`Packages/jp.co.cyberagent.instant-replay/UniEnc/.source-fingerprint`) is computed by `.github/scripts/compute-unienc-fingerprint.sh` from the git object hashes of this workspace (including the submodule pointers), `build-unienc.yml` and `localize-apple-staticlib.sh`. On a pull request that changes the package version, `check-native-binaries.yml` recomputes the fingerprint and fails if it differs from the committed one, which means that the binaries have not been rebuilt since the sources changed. + +Binaries built locally are useful for development and debugging, but are not to be committed, because the fingerprint would not be updated together with them. + +## Testing + +`crates/unienc_testkit` contains a shared end-to-end harness that drives the encoding system of the host platform (video encoder, audio encoder and muxer). The GPU texture path is not covered because it requires a Unity graphics device. Muxed outputs are written under `target/` for inspection. + +| Platform | Command (run in the root of the workspace) | +|----------|--------------------------------------------| +| macOS | `cargo test -p unienc_testkit -p unienc_common -p unienc_apple_vt` | +| Windows | `cargo test -p unienc_testkit -p unienc_common -p unienc_windows_mf` | +| Linux | `cargo test -p unienc_testkit -p unienc_common -p unienc_ffmpeg` | +| iOS simulator | `scripts/ios-simulator-test.sh []` | +| Android device or emulator | `scripts/android-device-test.sh [--release] [-s ]` | +| WebAssembly | `scripts/web-browser-test.sh [--release]` | + +- iOS: The test binary is run directly in a booted simulator through `simctl`, so no code signing is required. The script boots a simulator if none is running. The static library link and the hardware encoder of a real device are not covered. +- Android: The test binary is run through `app_process` on the device, so no APK is built. The same script works on an emulator and on a real device; the emulator uses a software encoder, so vendor-specific behavior of hardware encoders is covered only on a real device. +- WebAssembly: The harness is run as a page in Chrome through emrun. Set `UNIENC_TEST_BROWSER` to use a browser at a non-default location. + +`cargo test` does not build the FFI layer, so also run `cargo check -p unienc_c` after changing the signatures of a platform backend. CI additionally checks the formatting with `cargo fmt --all -- --check`. See `.github/workflows/ci-unienc.yml` at the root of the repository for the CI configuration. + +## Third-Party Notices + +`THIRD-PARTY-NOTICES.md` at the root of the repository lists the licenses of the Rust dependencies linked into the Unity variant. It is generated with cargo-about, using `about.toml` (the list of accepted licenses) and `about.hbs` (the template). The `build-unienc.yml` workflow regenerates it on every build. To regenerate it locally, run the following in the root of the workspace: + +```sh +cargo install cargo-about --locked --features cli +cargo about generate --features unity,mimalloc about.hbs > ../../THIRD-PARTY-NOTICES.md +``` + +If a new dependency uses a license that is not in `about.toml`, the generation fails. Review the license before adding it to the accepted list. ## Architecture @@ -46,8 +143,10 @@ unienc (platform selector via cfg) ### External Dependencies -- `external/unity-native-plugin-rs/` — Unity native plugin SDK for Rust +The following crates are included as git submodules and replace the crates.io versions through `[patch.crates-io]` in `Cargo.toml`. + - `external/muxide/` — MP4 multiplexer (used by `unienc_webcodecs`) +- `external/mimalloc_rust/` — Fork of the mimalloc allocator bindings (used by the `mimalloc` feature) ### Key Traits