diff --git a/.gitignore b/.gitignore index 06ad17f..9e023a0 100644 --- a/.gitignore +++ b/.gitignore @@ -9,3 +9,8 @@ videos/ node_modules/ test-results/ playwright-report/ +# Python test bytecode +__pycache__/ +*.pyc +# local generated/read-only review pages +docs/ diff --git a/README.md b/README.md index 025681a..70e19ab 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ site/work// A slug is never silently replaced; choose another slug if that directory already exists. The publish operation also rebuilds `site/catalog.json`, which the gallery at `site/index.html` reads. Published viewer settings come from embedded configuration rather than browser storage, so one work cannot inherit another work's camera or shader. -All resources used by an individual work are relative to its own directory. You can copy `site/work//` wholesale to an S3 bucket or any other static web server without changing a base URL. It contains no Studio/API dependency and makes no network requests for runtime libraries. Serve the directory over HTTP rather than opening `index.html` as a `file://` URL; ES modules are commonly blocked or restricted from local files. This layout is intended to support a future BrightSign packaging workflow with little or no transformation, though device-specific browser/WebGL validation is still required. +All resources used by an individual work are relative to its own directory. You can copy `site/work//` wholesale to an S3 bucket or any other static web server without changing a base URL. It contains no Studio/API dependency and makes no network requests for runtime libraries. Serve the directory over HTTP rather than opening `index.html` as a `file://` URL; ES modules are commonly blocked or restricted from local files. Published environments also load on plain HTTP origins without Web Crypto; SHA-256 hashing is optional only for this static playback path. Export-capable skybox loading still requires Web Crypto (HTTPS or localhost) to verify content and protect resume identity. This layout is intended to support a future BrightSign packaging workflow with little or no transformation, though device-specific browser/WebGL validation is still required. Preview the gallery locally with: @@ -106,7 +106,30 @@ The collapsed **Video timing** section provides export FPS, loop frame count, du **Export video…** opens a confirmation dialog for MP4 or MKV container, resolution, quality and range. The default is a 30-second MP4 clip starting at the current frame, shortened to fit when the loop is shorter than 30 seconds. Whole-loop export remains available. A clip window uses a start and duration in seconds, `MM:SS`, or `HH:MM:SS`; the window must remain inside one loop. The confirmation lists exact frames, duration and a bitrate-based size estimate before any work begins. If that file name already labels a clip in `videos/`, the viewer asks before adding another. -Rendering and encoding are intentionally split. The active browser renders the exact live WebGL scene one deterministic PNG frame at a time. The server encodes independent H.264/Matroska checkpoint segments, then losslessly concatenates them into the selected MP4 or MKV under `videos/`. Frames are not retained as an image sequence or accumulated in browser memory. Completed videos include an adjacent JSON provenance file. Static published works can display the export UI but cannot encode video without the local API server. +Rendering and encoding are intentionally split. The active browser renders the exact live WebGL scene one deterministic PNG frame at a time. The server encodes independent H.264/Matroska checkpoint segments, then losslessly concatenates them into the selected MP4 or MKV under `videos/`. Frames are not retained as an image sequence or accumulated in browser memory. Capture uses a reusable OffscreenCanvas with promise-based PNG encoding where supported, falling back to synchronous capture for compatibility. The selected skybox must finish loading before export; its actual face hashes are part of the resume signature, and environment changes are locked during rendering. Completed videos include an adjacent JSON provenance file. Static published works can display the export UI but cannot encode video without the local API server. + +New exports explicitly convert full-range RGB to limited-range BT.709 YUV and tag BT.709 matrix, transfer and primaries in H.264 and the container. Checkpoint concatenation preserves these tags, verified with real `ffprobe` tests for MP4 and MKV. Existing checkpoint jobs without a color profile retain their original conversion rather than mixing old and new colors within one movie. Checkpoint encoder failures preserve the last 4 KB of diagnostics in the interrupted job's reason. + +### Video validation commands + +```sh +make test-fast # routine editing: no Chromium, networking or real encoder +make test-python # durable store, API and real ffmpeg recovery tests +make test-browser # isolated capture + complete browser integration +make test-video-capture # pixel integrity and capture-fallback test; no HTTP server +make test-video-encoder # store + real encoder tests without localhost networking +make test-video-soak VIDEO_SOAK_SOURCE="videos/example.mp4" +# Longer encoder validation is opt-in; duration is output video seconds: +make test-video-soak VIDEO_SOAK_SOURCE="videos/example.mp4" VIDEO_SOAK_SECONDS=600 +``` + +Use `make test-fast` during development and select `test-video-capture` or `test-video-encoder` only for relevant changes. Run `make test-browser` once for final integration validation; do not automatically retry failed full browser suites. + +Resource-heavy targets share a per-user nonblocking lock across checkout directories, a wall-clock timeout, and an owned-process cleanup guard. Concurrent runs are refused rather than queued. Network-heavy targets refuse to start at 8,000 or more host `TIME_WAIT` sockets; nested stages recheck pressure before proceeding. Browser tests also use this guard when invoked directly. The guard attempts graceful cleanup, then terminates only the invocation's process group and recorded descendants whose PID/start time/command still match—not the user's server or browser. Test encoders are limited to two threads; production exports keep their normal thread policy. + +Browser cleanup is awaited on success, failure and SIGINT/SIGTERM. The temporary server pauses its jobs before exit, stderr is continuously drained with an 8 KB diagnostic tail, and temporary files are removed only after server exit. The API test helpers reuse an HTTP connection per test and close it during teardown. The export API already supports HTTP/1.1 keep-alive for sequential browser uploads. + +The soak uses three samples from an existing artwork video at 1080p/30 FPS, two store-restart/resume cycles, exact frame count and duration checks, BT.709 verification and encoder-leak checks. It uses temporary storage and does not alter the source. The default six-second run is deliberately bounded; it is not a claim of multi-hour browser endurance. Long hidden-tab/4K exports still need workload-specific endurance validation. ### Long-running video recovery diff --git a/makefile b/makefile index c7f2704..95e4788 100644 --- a/makefile +++ b/makefile @@ -1,4 +1,6 @@ -.PHONY: serve static install sync update test test-python test-browser unstick +.PHONY: serve static install sync update test test-fast test-guard test-python test-browser test-video-capture test-video-encoder test-video-soak _test-all _test-browser unstick + +TEST_GUARD = python3 scripts/test_guard.py serve: python3 studio/server.py --port 1313 @@ -29,11 +31,45 @@ update: $(MAKE) sync $(MAKE) test +# Full validation is explicit and serialized. Prefer test-fast while editing. test: - bun run test + $(TEST_GUARD) --network --timeout 600 -- $(MAKE) _test-all + +_test-all: + $(MAKE) test-fast + $(MAKE) test-python + $(MAKE) test-browser + +# No browser, network server, real encoder, or production assets. +test-fast: + bun run test:vendor + bun run test:syntax + node studio/test_resolution.cjs + node studio/test_lifecycle_test.cjs + node studio/test_skybox_hashing.cjs + python3 -m unittest discover -s studio -p 'test_video_resume.py' + $(MAKE) test-guard + +test-guard: + python3 -m unittest discover -s scripts -p 'test_test_guard.py' test-python: - python3 -m unittest discover -s studio -p 'test_*.py' + $(TEST_GUARD) --network --timeout 180 -- python3 -m unittest discover -s studio -p 'test_*.py' + +# Opt-in production-resolution encoder test. Uses temporary storage, never overwrites source. +VIDEO_SOAK_SECONDS ?= 6 +test-video-soak: + $(TEST_GUARD) --timeout 7200 -- python3 studio/video_soak.py --source "$(VIDEO_SOAK_SOURCE)" --seconds "$(VIDEO_SOAK_SECONDS)" + +test-video-encoder: + $(TEST_GUARD) --timeout 90 -- python3 -m unittest studio.test_video_resume studio.test_video_resume_integration + +test-video-capture: + $(TEST_GUARD) --timeout 60 -- node studio/test_video_capture.cjs test-browser: - bun run test:browser + $(TEST_GUARD) --network --timeout 360 -- $(MAKE) _test-browser + +_test-browser: + $(MAKE) test-video-capture + node studio/test_browser.cjs diff --git a/package.json b/package.json index 020a809..a492ce3 100644 --- a/package.json +++ b/package.json @@ -3,11 +3,12 @@ "private": true, "type": "module", "scripts": { - "test": "bun run test:vendor && bun run test:syntax && bun run test:unit && bun run test:browser", + "test": "make test", + "test:fast": "make test-fast", "test:vendor": "node scripts/sync-vendor.mjs --check", - "test:syntax": "node --check tesseract.js && node --check viewer-skyboxes.js && node --check studio/studio.js && node --check studio/pipeline.js", + "test:syntax": "node --check tesseract.js && node --check viewer-skyboxes.js && node --check studio/studio.js && node --check studio/pipeline.js && node --check studio/test_browser.cjs && node --check studio/test_video_capture.cjs && node --check studio/test_lifecycle.cjs", "test:unit": "node studio/test_resolution.cjs && make test-python", - "test:browser": "node studio/test_browser.cjs", + "test:browser": "make test-browser", "sync:vendor": "node scripts/sync-vendor.mjs" }, "dependencies": { diff --git a/scripts/test_guard.py b/scripts/test_guard.py new file mode 100644 index 0000000..0f3fe48 --- /dev/null +++ b/scripts/test_guard.py @@ -0,0 +1,176 @@ +#!/usr/bin/env python3 +"""Serialize resource-heavy tests, bound runtime, and clean only owned processes (POSIX).""" +import argparse +import fcntl +import json +import os +from pathlib import Path +import signal +import subprocess +import sys +import tempfile +import time +import uuid + +LOCK_PATH = Path(tempfile.gettempdir()) / f'tesseract-heavy-tests-{os.getuid()}.lock' +TOKEN_KEY = 'TESSERACT_TEST_GUARD_TOKEN' + + +def socket_pressure(): + result = subprocess.run(['netstat', '-an'], capture_output=True, text=True, timeout=5) + if result.returncode: + raise RuntimeError('Cannot inspect socket pressure; refusing network-heavy tests') + return sum('TIME_WAIT' in line for line in result.stdout.splitlines()) + + +def process_snapshot(): + result = subprocess.run(['ps', '-axo', 'pid=,ppid=,lstart=,command='], + capture_output=True, text=True, timeout=5) + if result.returncode: + raise RuntimeError('Cannot inspect test-owned processes') + entries = {} + for line in result.stdout.splitlines(): + fields = line.split(None, 7) + if len(fields) == 8 and fields[0].isdigit() and fields[1].isdigit(): + entries[int(fields[0])] = (int(fields[1]), ' '.join(fields[2:7]), fields[7]) + return entries + + +def remember_descendants(root_pid, owned, snapshot): + parents = {root_pid} | {pid for pid, identity in owned.items() + if pid in snapshot and snapshot[pid][1:] == identity} + changed = True + while changed: + changed = False + for pid, (parent, started, command) in snapshot.items(): + if parent in parents and pid not in parents: + parents.add(pid) + owned[pid] = (started, command) + changed = True + + +def signal_owned(owned, sig): + snapshot = process_snapshot() + for pid, identity in owned.items(): + if pid in snapshot and snapshot[pid][1:] == identity: + try: + os.kill(pid, sig) + except ProcessLookupError: + pass + + +def cleanup(child, owned): + try: + remember_descendants(child.pid, owned, process_snapshot()) + except (OSError, RuntimeError, subprocess.SubprocessError) as error: + print(f'Cleanup process inspection failed: {error}', file=sys.stderr) + # Allow JS finally/signal handlers to close Chromium and the server first. + if child.poll() is None: + child.send_signal(signal.SIGTERM) + try: + child.wait(timeout=10) + except subprocess.TimeoutExpired: + pass + # Detached browser processes can have their own groups. Their PID + start + # time + command must still match our recorded descendants before signalling. + try: + signal_owned(owned, signal.SIGTERM) + deadline = time.monotonic() + 3 + while time.monotonic() < deadline: + snapshot = process_snapshot() + if not any(pid in snapshot and snapshot[pid][1:] == identity + for pid, identity in owned.items()): + break + time.sleep(.2) + signal_owned(owned, signal.SIGKILL) + except (OSError, RuntimeError, subprocess.SubprocessError) as error: + print(f'Detached-process inspection failed; terminating only the owned test group: {error}', file=sys.stderr) + # Also cover ordinary children reparented during forced shutdown. This group + # was exclusively created for this test invocation, not the user's server. + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + if child.poll() is None: + child.wait(timeout=5) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--timeout', type=float, default=300) + parser.add_argument('--network', action='store_true') + parser.add_argument('command', nargs=argparse.REMAINDER) + args = parser.parse_args(argv) + command = args.command[1:] if args.command[:1] == ['--'] else args.command + if not command or args.timeout <= 0: + parser.error('A command and positive timeout are required') + + with LOCK_PATH.open('a+') as lock: + # Nested Make targets inherit the outer guard and its process group. + lock.seek(0) + try: + owner = json.loads(lock.read() or '{}') + except ValueError: + owner = {} + if os.environ.get(TOKEN_KEY) and os.environ[TOKEN_KEY] == owner.get('token'): + try: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + # The token is valid only while the outer guard still owns its + # lock. A stale inherited environment must not bypass guarding. + if args.network and socket_pressure() >= 8000: + print('Host socket pressure rose during this run; stopping before launching more network-heavy tests.', file=sys.stderr) + return 2 + os.execvp(command[0], command) + try: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + print('Heavy tests are already running. Wait for that run to finish; do not retry concurrently.', file=sys.stderr) + return 2 + if args.network: + pressure = socket_pressure() + if pressure >= 8000: + print(f'Refusing network-heavy tests: {pressure:,} TIME_WAIT sockets. ' + 'Use make test-fast or make test-video-encoder; retry only after host pressure subsides.', file=sys.stderr) + return 2 + token = uuid.uuid4().hex + lock.seek(0); lock.truncate() + lock.write(json.dumps({'pid': os.getpid(), 'token': token, 'command': command})) + lock.flush() + environment = dict(os.environ, **{TOKEN_KEY: token, 'TESSERACT_TEST_ENCODER_THREADS': '2'}) + child = subprocess.Popen(command, env=environment, start_new_session=True) + owned = {} + interrupted = [] + def stop(sig, _frame): + interrupted.append(sig) + previous = {sig: signal.signal(sig, stop) for sig in (signal.SIGINT, signal.SIGTERM)} + started = time.monotonic() + timed_out = False + try: + while child.poll() is None: + remember_descendants(child.pid, owned, process_snapshot()) + if interrupted: + break + if time.monotonic() - started > args.timeout: + timed_out = True + print(f'Test exceeded {args.timeout:g}s; cleaning up this invocation only.', file=sys.stderr) + break + time.sleep(.5) + finally: + try: + cleanup(child, owned) + finally: + for sig, handler in previous.items(): + signal.signal(sig, handler) + lock.seek(0); lock.truncate(); lock.flush() + if interrupted: + return 128 + interrupted[0] + return 124 if timed_out else child.returncode + + +if __name__ == '__main__': + try: + sys.exit(main()) + except (OSError, RuntimeError, subprocess.SubprocessError) as error: + print(f'Test guard: {error}', file=sys.stderr) + sys.exit(2) diff --git a/scripts/test_test_guard.py b/scripts/test_test_guard.py new file mode 100644 index 0000000..038e2aa --- /dev/null +++ b/scripts/test_test_guard.py @@ -0,0 +1,113 @@ +"""Cheap, network/browser-free regression tests for the heavy-test guard.""" +import fcntl +import json +import os +from pathlib import Path +import signal +import subprocess +import sys +import tempfile +import unittest +from unittest import mock + +import test_guard + + +class TestGuardTest(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.lock_patch = mock.patch.object(test_guard, 'LOCK_PATH', Path(self.temp.name) / 'guard.lock') + self.lock_patch.start() + self.token_patch = mock.patch.dict(os.environ, {test_guard.TOKEN_KEY: ''}) + self.token_patch.start() + + def tearDown(self): + self.token_patch.stop() + self.lock_patch.stop() + self.temp.cleanup() + + def test_refuses_overlapping_run_without_launching_child(self): + with test_guard.LOCK_PATH.open('a+') as lock: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + with mock.patch.object(test_guard.subprocess, 'Popen') as launch: + self.assertEqual(test_guard.main(['--', 'unused']), 2) + launch.assert_not_called() + + def test_refuses_network_run_under_socket_pressure(self): + with mock.patch.object(test_guard, 'socket_pressure', return_value=9000), \ + mock.patch.object(test_guard.subprocess, 'Popen') as launch: + self.assertEqual(test_guard.main(['--network', '--', 'unused']), 2) + launch.assert_not_called() + + def test_nested_network_stage_rechecks_socket_pressure(self): + token = 'outer-test-token' + test_guard.LOCK_PATH.write_text(json.dumps({'token': token})) + with test_guard.LOCK_PATH.open('a+') as lock: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + with mock.patch.dict(os.environ, {test_guard.TOKEN_KEY: token}), \ + mock.patch.object(test_guard, 'socket_pressure', return_value=9000), \ + mock.patch.object(test_guard.os, 'execvp') as execute: + self.assertEqual(test_guard.main(['--network', '--', 'unused']), 2) + execute.assert_not_called() + + def test_preserves_exit_status_and_releases_lock(self): + self.assertEqual(test_guard.main(['--', sys.executable, '-c', 'raise SystemExit(7)']), 7) + with test_guard.LOCK_PATH.open('a+') as lock: + fcntl.flock(lock.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + self.assertEqual(test_guard.LOCK_PATH.read_text(), '') + + def test_timeout_cleans_child_and_keeps_unrelated_process(self): + marker = Path(self.temp.name) / 'child.pid' + script = ''' +import signal,subprocess,sys,time +from pathlib import Path +child=subprocess.Popen([sys.executable,'-c','import time;time.sleep(60)']) +Path(sys.argv[1]).write_text(str(child.pid)) +def stop(*_): + child.terminate() + child.wait(timeout=3) + raise SystemExit(0) +signal.signal(signal.SIGTERM,stop) +time.sleep(60) +''' + unrelated = subprocess.Popen([sys.executable, '-c', 'import time;time.sleep(60)']) + try: + self.assertEqual(test_guard.main(['--timeout', '1', '--', sys.executable, '-c', script, str(marker)]), 124) + child_pid = int(marker.read_text()) + with self.assertRaises(ProcessLookupError): + os.kill(child_pid, 0) + self.assertIsNone(unrelated.poll(), 'Guard touched an unrelated process') + finally: + unrelated.terminate() + unrelated.wait(timeout=5) + + def test_process_inspection_failure_still_stops_owned_child(self): + launched = [] + real_popen = subprocess.Popen + def launch(*args, **kwargs): + child = real_popen(*args, **kwargs) + launched.append(child) + return child + with mock.patch.object(test_guard, 'process_snapshot', side_effect=RuntimeError('injected ps failure')), \ + mock.patch.object(test_guard.subprocess, 'Popen', side_effect=launch): + with self.assertRaisesRegex(RuntimeError, 'injected'): + test_guard.main(['--', sys.executable, '-c', 'import time;time.sleep(60)']) + self.assertEqual(len(launched), 1) + self.assertIsNotNone(launched[0].poll()) + self.assertEqual(test_guard.LOCK_PATH.read_text(), '') + + def test_reparented_owned_process_is_still_signalled(self): + with mock.patch.object(test_guard, 'process_snapshot', return_value={123: (1, 'same time', 'owned browser')}), \ + mock.patch.object(test_guard.os, 'kill') as kill: + test_guard.signal_owned({123: ('same time', 'owned browser')}, signal.SIGTERM) + kill.assert_called_once_with(123, signal.SIGTERM) + + def test_reused_pid_is_not_signalled(self): + with mock.patch.object(test_guard, 'process_snapshot', return_value={123: (1, 'new time', 'unrelated')}), \ + mock.patch.object(test_guard.os, 'kill') as kill: + test_guard.signal_owned({123: ('old time', 'old command')}, signal.SIGKILL) + kill.assert_not_called() + + +if __name__ == '__main__': + unittest.main() diff --git a/studio/server.py b/studio/server.py index bfc526b..9818ba2 100644 --- a/studio/server.py +++ b/studio/server.py @@ -28,10 +28,9 @@ MAX_VIDEO_FRAME_BYTES = 100 * 1024 * 1024 MAX_VIDEO_FRAMES = 10_000_000 VIDEO_FRAME_READ_TIMEOUT = 30 -VIDEO_JOBS = {} -VIDEO_JOBS_LOCK = threading.Lock() VIDEO_STORE = None VIDEO_STORE_LOCK = threading.Lock() +FFMPEG_ENCODER_ERROR = None def hash_file(path): @@ -204,14 +203,30 @@ def video_clip_labels(): return labels +def ffmpeg_encoder_error(ffmpeg): + try: + completed = subprocess.run( + [str(ffmpeg), '-hide_banner', '-encoders'], + capture_output=True, text=True, timeout=10, + ) + except (OSError, subprocess.SubprocessError) as error: + return f'Could not inspect ffmpeg encoders: {error}' + if completed.returncode != 0: + return 'ffmpeg could not list its encoders' + if not re.search(r'^\s*V\S*\s+libx264\s', completed.stdout, re.MULTILINE): + return 'ffmpeg does not provide the required libx264 H.264 encoder' + return None + + def video_capabilities(): ffmpeg = shutil.which('ffmpeg') recovery = { 'recovered': 0, 'alreadyExited': 0, 'refused': 0, 'skippedActive': 0, 'failed': 0, 'indexFailed': False, } - available = bool(ffmpeg) - reason = None if ffmpeg else 'ffmpeg is not installed or is not on the server PATH' + available = bool(ffmpeg) and not FFMPEG_ENCODER_ERROR + reason = (FFMPEG_ENCODER_ERROR if ffmpeg + else 'ffmpeg is not installed or is not on the server PATH') if ffmpeg: store = video_store() if (not store.recovery_running and store.last_recovery['failed'] @@ -275,6 +290,8 @@ def start_video(request): ffmpeg = shutil.which('ffmpeg') if not ffmpeg: raise ValueError('Video export requires ffmpeg on the local server PATH') + if FFMPEG_ENCODER_ERROR: + raise ValueError(FFMPEG_ENCODER_ERROR) store = video_store() if store.recovery_running: raise ValueError('Encoder startup recovery is still running') @@ -320,54 +337,12 @@ def start_video(request): return active -def video_job(job_id): - if not isinstance(job_id, str) or not re.fullmatch(r'[a-f0-9]{32}', job_id): - raise ValueError('Invalid video export id') - with VIDEO_JOBS_LOCK: - job = VIDEO_JOBS.get(job_id) - if not job: - raise ValueError('Video export is missing or already complete') - return job - - -def remove_video_job(job_id): - with VIDEO_JOBS_LOCK: - return VIDEO_JOBS.pop(job_id, None) - - -def cancel_video(job_id): - job = remove_video_job(job_id) - if not job: - return - # Never wait for the frame-write lock here. A disconnected browser can leave - # its request thread blocked in a pipe write; killing ffmpeg must remain able - # to interrupt that write immediately. - process = job['process'] - # Kill only. Closing stdin here deadlocks when the frame thread is blocked - # inside that same buffered write; the broken pipe wakes the writer instead. - if process.poll() is None: - process.kill() - try: - process.wait(timeout=5) - except subprocess.TimeoutExpired: - process.kill() - process.wait(timeout=5) - job['pending'].unlink(missing_ok=True) - - -def cancel_all_videos(): - with VIDEO_JOBS_LOCK: - job_ids = list(VIDEO_JOBS) - for job_id in job_ids: - try: - cancel_video(job_id) - except (OSError, subprocess.SubprocessError): - pass +def pause_all_videos(): if VIDEO_STORE is not None: VIDEO_STORE.pause_all() -atexit.register(cancel_all_videos) +atexit.register(pause_all_videos) def finish_video(job_id): @@ -633,11 +608,16 @@ def do_POST(self): except (ConnectionError, BrokenPipeError): return except (ValueError, KeyError, TypeError, StopIteration) as error: + # Some validation failures occur before a request body is consumed. + # Never reuse that HTTP/1.1 connection: unread bytes could otherwise + # be parsed as the beginning of the next request. + self.close_connection = True try: self.send_json(400, {'error': str(error)}) except (ConnectionError, BrokenPipeError, OSError): return except OSError as error: + self.close_connection = True try: self.send_json(500, {'error': str(error)}) except (ConnectionError, BrokenPipeError, OSError): @@ -659,7 +639,10 @@ def service_actions(self): parser.add_argument('--port', type=int, default=1313) args = parser.parse_args() http = StudioHTTPServer(('localhost', args.port), Handler) - if shutil.which('ffmpeg'): + ffmpeg = shutil.which('ffmpeg') + if ffmpeg: + FFMPEG_ENCODER_ERROR = ffmpeg_encoder_error(ffmpeg) + if ffmpeg and not FFMPEG_ENCODER_ERROR: store = video_store() try: store.claim_server() @@ -677,5 +660,8 @@ def service_actions(self): http.serve_forever() finally: if VIDEO_STORE is not None: - VIDEO_STORE.release_server() + try: + VIDEO_STORE.pause_all() + finally: + VIDEO_STORE.release_server() http.server_close() diff --git a/studio/test_browser.cjs b/studio/test_browser.cjs index 3614db4..ef5cecd 100644 --- a/studio/test_browser.cjs +++ b/studio/test_browser.cjs @@ -1,4 +1,6 @@ // Browser integration test. Uses the Bun-managed Playwright dependency and never touches real raw/ or exports/. +const {ensureGuard,bounded,stopServer,installCleanup}=require('./test_lifecycle.cjs'); +ensureGuard(__filename,{network:true,timeout:300}); const {chromium}=require('playwright'); const fs=require('fs'),os=require('os'),path=require('path'),assert=require('assert/strict'),{spawn}=require('child_process'); (async()=>{ @@ -6,11 +8,39 @@ const fs=require('fs'),os=require('os'),path=require('path'),assert=require('ass fs.cpSync(__dirname,path.join(root,'studio'),{recursive:true});fs.mkdirSync(path.join(root,'raw')); for(const file of ['index.html','tesseract.js','viewer-skyboxes.js','skybox-paths.js'])fs.copyFileSync(path.join(__dirname,'..',file),path.join(root,file)); fs.cpSync(path.join(__dirname,'..','vendor'),path.join(root,'vendor'),{recursive:true}); - const server=spawn('python3',['-c',`import sys;sys.path.insert(0,${JSON.stringify(__dirname)});import server;from pathlib import Path;server.ROOT=Path(${JSON.stringify(root)});http=server.ThreadingHTTPServer(('localhost',0),server.Handler);print(http.server_port,flush=True);http.serve_forever()`]); - let browser; + const fixture=` +import signal,sys +sys.path.insert(0,${JSON.stringify(__dirname)}) +import server +from pathlib import Path +server.ROOT=Path(${JSON.stringify(root)}) +http=server.StudioHTTPServer(('localhost',0),server.Handler) +def stop(*_): raise KeyboardInterrupt() +signal.signal(signal.SIGTERM,stop) +try: + print(http.server_port,flush=True) + http.serve_forever() +except KeyboardInterrupt: + pass +finally: + server.pause_all_videos() + http.server_close() +`; + const server=spawn('python3',['-c',fixture]); + let browser,serverErrors=''; + // Drain stderr continuously. Per-frame HTTP logs must never fill the pipe and + // block the test server; retain only a small diagnostic tail. + server.stderr.on('data',data=>{serverErrors=(serverErrors+data.toString()).slice(-8192);}); + const cleanup=installCleanup(async()=>{ + try{if(browser)await bounded(browser.close(),10000,'Chromium shutdown');} + finally{await stopServer(server);fs.rmSync(root,{recursive:true,force:true});} + }); try{ - const port=await new Promise((resolve,reject)=>{server.stdout.once('data',data=>resolve(Number(data.toString().trim())));server.once('error',reject);}); - browser=await chromium.launch({headless:true,...(process.env.CHROMIUM_EXECUTABLE?{executablePath:process.env.CHROMIUM_EXECUTABLE}:{}),args:['--use-gl=angle','--use-angle=swiftshader']}); + const port=await bounded(new Promise((resolve,reject)=>{ + let output='';server.stdout.on('data',data=>{output+=data.toString();const line=output.split('\n')[0];if(/^\d+$/.test(line))resolve(Number(line));}); + server.once('error',reject);server.once('exit',code=>reject(new Error(`Test server exited (${code}): ${serverErrors}`))); + }),10000,'Test server startup'); + browser=await chromium.launch({headless:true,...(process.env.CHROMIUM_EXECUTABLE?{executablePath:process.env.CHROMIUM_EXECUTABLE}:{}),timeout:15000,args:['--renderer-process-limit=2','--use-gl=angle','--use-angle=swiftshader']}); const context=await browser.newContext({viewport:{width:1700,height:1100}}); const page=await context.newPage(),errors=[];let starting=true,failStartup; const startupFailure=new Promise((_,reject)=>{failStartup=reject;}); @@ -80,6 +110,7 @@ const fs=require('fs'),os=require('os'),path=require('path'),assert=require('ass await viewer.locator('#rotation-xw').fill('10');await viewer.locator('details').filter({hasText:'Video timing'}).locator('summary').click(); await viewer.locator('#open-video-export').click();await viewer.waitForSelector('#video-export-dialog[open]');await viewer.waitForFunction(()=>document.querySelector('#video-export-dialog').dataset.serverAvailable==='yes'||document.querySelector('#video-export-status').classList.contains('error')); assert.deepEqual(await viewer.locator('#video-export-dialog').evaluate(dialog=>JSON.parse(dialog.dataset.encoderRecovery)),{recovered:0,alreadyExited:0,refused:0,skippedActive:0,failed:0,indexFailed:false}); + const renderEnvironment=await viewer.locator('#video-export-dialog').evaluate(dialog=>JSON.parse(JSON.parse(dialog.dataset.renderSignature).environment));assert.equal(renderEnvironment.kind,'export');assert.equal(Object.keys(renderEnvironment.faces).length,6);assert(Object.values(renderEnvironment.faces).every(hash=>/^[a-f0-9]{64}$/.test(hash))); assert.equal(await viewer.locator('#video-export-checkpoint').inputValue(),'60');assert(await viewer.locator('#video-export-scratch').isVisible()); assert.equal(await viewer.locator('#video-export-range').inputValue(),'clip');assert.match(await viewer.locator('#video-export-summary').innerText(),/Clip/);assert.match(await viewer.locator('#video-export-summary').innerText(),/30\.00 s/);assert.match(await viewer.locator('#video-export-summary').innerText(),/estimated MP4 size/); await viewer.locator('#video-export-range').selectOption('full');assert(!(await viewer.locator('#video-export-clip-fields').isVisible()));assert.equal(await viewer.locator('#video-export-range option[value="full"]').innerText(),'Whole loop');assert.doesNotMatch(await viewer.locator('#video-export-summary').innerText(),/[Pp]erfect/); @@ -135,6 +166,16 @@ const fs=require('fs'),os=require('os'),path=require('path'),assert=require('ass }); await viewer.locator('#close-video-export').click();await viewer.locator('#open-video-export').click();const mismatch=viewer.locator(`.video-resume-job[data-job-id="${mismatchId}"]`);await mismatch.waitFor();assert(await mismatch.locator('.resume-video-export').isDisabled());assert.match(await mismatch.innerText(),/settings do not match/i); await mismatch.locator('.discard-video-export').click();await viewer.locator('#confirm-action-accept').click();await mismatch.waitFor({state:'detached'}); + const restorableId=await viewer.evaluate(async()=>{ + const dialog=document.querySelector('#video-export-dialog'),signature=dialog.dataset.renderSignature,viewerState=JSON.parse(localStorage.getItem('tesseract.viewer-settings.v1')); + const request={name:'restore-settings',width:1280,height:720,fps:viewerState.exportFps,frames:1,quality:'draft',format:'mp4',checkpointSeconds:60,sourceUrl:location.pathname+location.search,renderSignature:signature,startFrame:0,loopFrameCount:1,loopPeriod:0,timeStep:0,viewerState}; + const started=await fetch('/api/video/start',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(request)}).then(r=>r.json()); + await fetch('/api/video/pause',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({id:started.id,lease:started.lease})});return started.id; + }); + await viewer.locator('#close-video-export').click();await viewer.locator('#viewer-shader').selectOption('rough');await viewer.locator('#open-video-export').click(); + const restorable=viewer.locator(`.video-resume-job[data-job-id="${restorableId}"]`);await restorable.waitFor();assert(await restorable.locator('.resume-video-export').isDisabled()); + await restorable.locator('.restore-video-export').click();await viewer.waitForFunction(id=>{const button=document.querySelector(`.video-resume-job[data-job-id="${id}"] .resume-video-export`);return button&&!button.disabled;},restorableId);assert.equal(await viewer.locator('#viewer-shader').inputValue(),'chrome'); + await restorable.locator('.discard-video-export').click();await viewer.locator('#confirm-action-accept').click();await restorable.waitFor({state:'detached'}); } if(await viewer.locator('#video-export-dialog').isVisible())await viewer.locator('#close-video-export').click(); const cameraBefore=await viewer.locator('#viewer-camera-info').innerText(); @@ -202,5 +243,5 @@ const fs=require('fs'),os=require('os'),path=require('path'),assert=require('ass assert(fs.existsSync(path.join(root,'exports',folder,'pipeline.json')));assert(fs.existsSync(path.join(root,'raw','a.png'))); await Promise.all([page.waitForEvent('load'),page.locator('#reload-app').click()]);await page.waitForSelector('.photo');assert.equal(await page.locator('.node').count(),1);assert.equal((await capture()).edges.length,0); assert.deepEqual(errors,[]);console.log('PASS: Studio graph/export, viewer gallery, publish-dialog keyboard isolation, offline vendored runtime, failure retention, persistence and Studio reopen.'); - }finally{if(browser)await browser.close();server.kill();fs.rmSync(root,{recursive:true,force:true});} + }finally{await cleanup();} })().catch(error=>{console.error(error);process.exitCode=1;}); diff --git a/studio/test_lifecycle.cjs b/studio/test_lifecycle.cjs new file mode 100644 index 0000000..a851016 --- /dev/null +++ b/studio/test_lifecycle.cjs @@ -0,0 +1,39 @@ +// Common lifecycle helpers. Direct test-file execution also uses the shared guard. +const { spawnSync } = require('child_process'); +function ensureGuard(file, { network = false, timeout = 300 } = {}) { + if (process.env.TESSERACT_TEST_GUARD_TOKEN) return; + const result = spawnSync('python3', [require('path').join(__dirname, '..', 'scripts', 'test_guard.py'), + '--timeout', String(timeout), ...(network ? ['--network'] : []), '--', process.execPath, file], + { stdio: 'inherit' }); + if (result.error) console.error(result.error); + process.exit(result.status ?? 2); +} +async function bounded(promise, milliseconds, label) { + let timer; + try { + return await Promise.race([promise, new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error(`${label} exceeded ${milliseconds / 1000}s`)), milliseconds); + })]); + } finally { clearTimeout(timer); } +} +async function stopServer(server) { + if (!server.pid || server.exitCode !== null || server.signalCode !== null) return; + const exited = new Promise(resolve => server.once('exit', resolve)); + server.kill('SIGTERM'); + try { await bounded(exited, 5000, 'Test server shutdown'); } + catch { + server.kill('SIGKILL'); + await bounded(exited, 5000, 'Forced test server shutdown'); + } +} +function installCleanup(action) { + let promise; + const cleanup = () => promise ||= Promise.resolve().then(action); + for (const [signal, code] of [['SIGINT', 130], ['SIGTERM', 143]]) { + process.once(signal, () => { + cleanup().catch(error => console.error(error)).finally(() => process.exit(code)); + }); + } + return cleanup; +} +module.exports = { ensureGuard, bounded, stopServer, installCleanup }; diff --git a/studio/test_lifecycle_test.cjs b/studio/test_lifecycle_test.cjs new file mode 100644 index 0000000..268675c --- /dev/null +++ b/studio/test_lifecycle_test.cjs @@ -0,0 +1,28 @@ +// No Chromium or networking: exercise awaited server shutdown and cleanup bounds. +const { spawn } = require('child_process'); +const assert = require('assert/strict'); +const { bounded, stopServer, installCleanup } = require('./test_lifecycle.cjs'); +(async () => { + const child = spawn(process.execPath, ['-e', ` + process.on('SIGTERM',()=>setTimeout(()=>process.exit(0),50)); + console.log('ready');setInterval(()=>{},1000); + `]); + child.stderr.resume(); + try { + await bounded(new Promise((resolve, reject) => { + child.stdout.once('data', resolve);child.once('error', reject); + }), 2000, 'Fixture startup'); + let calls=0; + const cleanup=installCleanup(async()=>{calls++;await stopServer(child);}); + await Promise.all([cleanup(),cleanup()]); + assert.equal(calls,1); + assert.equal(child.exitCode,0,'Shutdown must await the server exit, not just send a signal'); + await stopServer(child); // already exited: idempotent + await assert.rejects(bounded(new Promise(()=>{}),10,'injected wait'),/exceeded/); + console.log('PASS: awaited test-server shutdown, idempotent cleanup, bounded waits.'); + } finally { + if (child.exitCode===null && child.signalCode===null) { + const exited=new Promise(resolve=>child.once('exit',resolve));child.kill('SIGKILL');await exited; + } + } +})().catch(error=>{console.error(error);process.exitCode=1;}); diff --git a/studio/test_server.py b/studio/test_server.py index cdaf1b1..94f92de 100644 --- a/studio/test_server.py +++ b/studio/test_server.py @@ -38,8 +38,10 @@ def setUp(self): self.http = ThreadingHTTPServer(('localhost', 0), server.Handler) self.thread = threading.Thread(target=self.http.serve_forever, daemon=True) self.thread.start() + self.connection = http.client.HTTPConnection('localhost', self.http.server_port, timeout=10) def tearDown(self): + self.connection.close() self.http.shutdown() self.http.server_close() self.thread.join() @@ -53,23 +55,22 @@ def tearDown(self): self.temp.cleanup() def request(self, method, path, payload=None, headers=None): - connection = http.client.HTTPConnection('localhost', self.http.server_port) - connection.request(method, path, json.dumps(payload) if payload is not None else None, headers or {}) - response = connection.getresponse() - status, data = response.status, json.loads(response.read()) - connection.close() - return status, data + return self.raw_request(method, path, json.dumps(payload) if payload is not None else None, headers) def raw_request(self, method, path, body, headers=None): - connection = http.client.HTTPConnection('localhost', self.http.server_port) - connection.request(method, path, body, headers or {}) - response = connection.getresponse() - status, data = response.status, json.loads(response.read()) - connection.close() - return status, data + self.connection.request(method, path, body, headers or {}) + response = self.connection.getresponse() + return response.status, json.loads(response.read()) + + def test_request_helpers_reuse_connection(self): + self.request('GET', '/api/video/capabilities') + first_socket = self.connection.sock + self.assertIsNotNone(first_socket) + self.raw_request('GET', '/api/video/capabilities', None) + self.assertIs(self.connection.sock, first_socket) def test_http_connection_is_reused_between_requests(self): - connection = http.client.HTTPConnection('localhost', self.http.server_port) + connection = http.client.HTTPConnection('localhost', self.http.server_port, timeout=10) connection.request('GET', '/api/video/capabilities') first = connection.getresponse() self.assertEqual(first.status, 200) @@ -155,7 +156,7 @@ def test_streamed_export(self): self.assertEqual(code, 201) folder = started['folder'] for face in server.FACES: - connection = http.client.HTTPConnection('localhost', self.http.server_port) + connection = http.client.HTTPConnection('localhost', self.http.server_port, timeout=10) connection.request('POST', f'/api/export/face?folder={folder}&face={face}', PNG, {'Content-Type': 'image/png'}) response = connection.getresponse() self.assertEqual(response.status, 201) @@ -361,39 +362,6 @@ def test_video_cancel_discards_resumable_job(self): self.assertEqual(code, 400) self.assertIn('scratch', result['error'].lower()) - def test_video_cancel_does_not_wait_for_frame_lock(self): - class FakeProcess: - def __init__(self): - self.stdin = io.BytesIO() - self.returncode = None - self.killed = False - - def poll(self): - return self.returncode - - def kill(self): - self.killed = True - self.returncode = -9 - - def wait(self, timeout=None): - return self.returncode - - job_id = 'a' * 32 - frame_lock = threading.Lock() - frame_lock.acquire() - process = FakeProcess() - pending = server.ROOT / 'blocked.pending.mp4' - pending.write_bytes(b'partial') - with server.VIDEO_JOBS_LOCK: - server.VIDEO_JOBS[job_id] = {'process': process, 'lock': frame_lock, 'pending': pending} - cancelled = threading.Thread(target=server.cancel_video, args=(job_id,)) - cancelled.start() - cancelled.join(timeout=1) - frame_lock.release() - self.assertFalse(cancelled.is_alive(), 'Cancellation waited for the blocked frame lock') - self.assertTrue(process.killed) - self.assertFalse(pending.exists()) - def test_stalled_video_frame_upload_unblocks(self): class FakeFFmpeg: def __init__(self, command, **_kwargs): @@ -465,6 +433,24 @@ def test_video_validation_and_capabilities(self): 'width': 1920, 'height': 1080, 'fps': 60, 'frames': 60, 'quality': 'standard', 'format': 'avi'}) + def test_ffmpeg_preflight_requires_libx264(self): + supported = mock.Mock(returncode=0, stdout=' V....D libx264 H.264 encoder') + missing = mock.Mock(returncode=0, stdout=' V....D h264_videotoolbox H.264 encoder') + with mock.patch.object(server.subprocess, 'run', return_value=supported): + self.assertIsNone(server.ffmpeg_encoder_error('/fake/ffmpeg')) + with mock.patch.object(server.subprocess, 'run', return_value=missing): + self.assertIn('libx264', server.ffmpeg_encoder_error('/fake/ffmpeg')) + previous = server.FFMPEG_ENCODER_ERROR + server.FFMPEG_ENCODER_ERROR = 'ffmpeg does not provide the required libx264 H.264 encoder' + try: + with mock.patch.object(server.shutil, 'which', return_value='/fake/ffmpeg'): + code, capabilities = self.request('GET', '/api/video/capabilities') + self.assertEqual(code, 200) + self.assertFalse(capabilities['available']) + self.assertIn('libx264', capabilities['reason']) + finally: + server.FFMPEG_ENCODER_ERROR = previous + def test_video_pause_accepts_bounded_failure_reason(self): with mock.patch.object(server.shutil, 'which', return_value='/fake/ffmpeg'): code, started = self.request('POST', '/api/video/start', { @@ -584,7 +570,7 @@ def test_publish_creates_standalone_work_and_catalog(self): def test_studio_code_bypasses_cache(self): (server.ROOT / 'studio').mkdir() (server.ROOT / 'studio' / 'app.js').write_text('const version = 3;') - connection = http.client.HTTPConnection('localhost', self.http.server_port) + connection = http.client.HTTPConnection('localhost', self.http.server_port, timeout=10) connection.request('GET', '/studio/app.js', headers={'If-Modified-Since': 'Wed, 01 Jan 2099 00:00:00 GMT'}) response = connection.getresponse() self.assertEqual(response.status, 200) @@ -615,7 +601,7 @@ def test_cross_origin_rejected(self): self.assertEqual(self.request('POST', '/api/export', self.payload(), {'Origin': 'https://example.com'})[0], 403) def test_rejected_request_with_unread_body_closes_connection(self): - connection = http.client.HTTPConnection('localhost', self.http.server_port) + connection = http.client.HTTPConnection('localhost', self.http.server_port, timeout=10) connection.request('POST', '/api/export', json.dumps(self.payload()), {'Origin': 'https://example.com'}) response = connection.getresponse() self.assertEqual(response.status, 403) diff --git a/studio/test_skybox_hashing.cjs b/studio/test_skybox_hashing.cjs new file mode 100644 index 0000000..562948e --- /dev/null +++ b/studio/test_skybox_hashing.cjs @@ -0,0 +1,94 @@ +// Published/insecure-origin regression, without Chromium or HTTP connections. +const assert = require('assert/strict'); +const fs = require('fs'); +const path = require('path'); +const vm = require('vm'); +const { webcrypto, createHash } = require('crypto'); +const source = fs.readFileSync(path.join(__dirname, '..', 'viewer-skyboxes.js'), 'utf8') + .replace(/^import .*;\n/gm, '').replace('export function installSkyboxLibrary', 'function installSkyboxLibrary'); +const faces = ['px','nx','py','ny','pz','nz']; +function fixture(crypto) { + let fetches=0,created=0,revoked=0,states=0; + const status={textContent:'',classList:{add(){}}}; + const section={querySelector(selector){return selector==='#skybox-status'?status:{textContent:''};}}; + const canvas={getContext(){return {createLinearGradient(){return {addColorStop(){}};},fillRect(){}};}}; + const bytes=url=>new TextEncoder().encode(url); + let complete; + const done=new Promise(resolve=>{complete=resolve;}); + const context=vm.createContext({ + crypto, Blob, + URL:{createObjectURL(){created++;return `blob:test-${created}`;},revokeObjectURL(){revoked++;}}, + Image:class {constructor(){this.naturalWidth=this.naturalHeight=16;}set src(_value){queueMicrotask(()=>this.onload());}}, + fetch:async url=>{fetches++;return {ok:true,headers:{get(){return 'image/png';}},arrayBuffer:async()=>bytes(url).buffer};}, + document:{createElement(tag){return tag==='canvas'?canvas:section;}}, + THREE:{CubeTexture:class {constructor(images){this.images=images;}}}, + options:{ + mount:{prepend(){}}, renderer:{capabilities:{isWebGL2:true},getContext(){return {MAX_CUBE_MAP_TEXTURE_SIZE:1,getParameter(){return 1024;}};}}, + onTexture(){},onStateChange(state){if(++states>1)complete(state);}, + publication:{schemaVersion:1,slug:'artwork',size:16,skybox:Object.fromEntries(faces.map(face=>[face,`skybox/${face}.png`]))} + } + }); + vm.runInContext(source,context); + return {context,status,done,bytes,counts:()=>({fetches,created,revoked})}; +} +async function published(crypto) { + const value=fixture(crypto); + value.api=vm.runInContext('installSkyboxLibrary(options)',value.context); + let timer; + try { + value.state=await Promise.race([value.done,new Promise((_,reject)=>{ + timer=setTimeout(()=>reject(new Error('Published load did not finish')),2000); + })]); + } finally {clearTimeout(timer);} + return value; +} +(async()=>{ + // Both missing crypto and crypto without subtle occur on restricted origins. + for(const crypto of [undefined,{}]) { + const value=await published(crypto); + assert.equal(value.state.ready,true); + assert.equal(value.state.identity,null,'Unhashed assets must not get a verified render identity'); + assert.match(value.status.textContent,/16 × 16px/); + assert.deepEqual(value.counts(),{fetches:6,created:6,revoked:6}); + assert.throws(()=>value.api.acquireRenderLock(),/verification requires HTTPS or localhost/); + const strict=fixture(crypto); + await assert.rejects(vm.runInContext("loadImageAsset('skybox/px.png')",strict.context),/HTTPS or localhost/); + assert.equal(strict.counts().fetches,0,'Fail export-capable loads before fetching large assets'); + } + const secure=await published(webcrypto); + assert.equal(secure.state.ready,true); + const identity=JSON.parse(secure.state.identity); + for(const face of faces) { + assert.equal(identity.faces[face],createHash('sha256').update(secure.bytes(`skybox/${face}.png`)).digest('hex')); + } + const release=secure.api.acquireRenderLock();release();release(); + const strict=fixture(webcrypto); + const asset=await vm.runInContext("loadImageAsset('skybox/px.png')",strict.context); + assert.equal(asset.sha256,identity.faces.px,'Export-capable loads retain actual-byte hashing'); + // Exercise the actual export planner without a browser: ready is not enough + // when the lock also requires a verified environment identity. + const viewerSource=fs.readFileSync(path.join(__dirname,'..','tesseract.js'),'utf8'); + const planner=viewerSource.slice(viewerSource.indexOf('function videoExportPlan() {'),viewerSource.indexOf('function updateVideoExportSummary()')); + const fields={ + 'video-export-resolution':'1280x720','video-export-format':'mp4', + 'video-export-quality':'standard','video-export-range':'full', + 'video-export-checkpoint':'60','video-export-scratch':'' + }; + const plannerContext=vm.createContext({ + videoExportDialog:{},loopTiming:{exact:true,frameCount:30},exportFps:30, + VIDEO_QUALITY_BITS_PER_PIXEL:{standard:.07}, + document:{getElementById(id){return {value:fields[id]};}} + }); + vm.runInContext(planner,plannerContext); + for(const state of [{ready:false,identity:null},{ready:true,identity:null},{ready:true,identity:secure.state.identity}]) { + plannerContext.skyboxLibrary={getRenderState(){return state;}}; + const plan=vm.runInContext('videoExportPlan()',plannerContext); + if(!state.ready)assert.match(plan.error,/finish loading/); + else if(!state.identity)assert.match(plan.error,/verification requires HTTPS or localhost/); + else assert.equal(plan.error,''); + } + const failed=await published({subtle:{digest:async()=>{throw new Error('injected digest failure');}}}); + assert.equal(failed.state.ready,false,'A failed available digest must not silently downgrade verification'); + assert.match(failed.status.textContent,/injected digest failure/); + console.log('PASS: insecure published skyboxes load; export hashing stays mandatory; secure face hashes remain verified.'); +})().catch(error=>{console.error(error);process.exitCode=1;}); diff --git a/studio/test_video_capture.cjs b/studio/test_video_capture.cjs new file mode 100644 index 0000000..ecc2538 --- /dev/null +++ b/studio/test_video_capture.cjs @@ -0,0 +1,46 @@ +// Isolated capture regression: no HTTP server, skyboxes, or ffmpeg processes. +const { ensureGuard, bounded, installCleanup } = require('./test_lifecycle.cjs'); +ensureGuard(__filename, { timeout: 60 }); +const { chromium } = require('playwright'); +const fs = require('fs'); +const path = require('path'); +const assert = require('assert/strict'); +(async () => { + const source = fs.readFileSync(path.join(__dirname, '..', 'tesseract.js'), 'utf8'); + const helpers = source.slice(source.indexOf('function dataUrlToBlob('), source.indexOf('function screenshotMetadata(')); + let browser; + const cleanup = installCleanup(async () => { + if (browser) await bounded(browser.close(), 10000, 'Chromium shutdown'); + }); + try { + browser = await chromium.launch({ headless: true }); + const page = await browser.newPage(); + await page.addScriptTag({ content: `let videoCaptureSurface=null,videoCaptureContext=null;let renderer,scene,camera;${helpers}` }); + const result = await page.evaluate(async () => { + const canvas = document.createElement('canvas'); + canvas.width = 128; canvas.height = 72; + const ctx = canvas.getContext('2d'); + const paint = color => { ctx.fillStyle=color;ctx.fillRect(0,0,128,72); }; + const read = async blob => { + const bitmap = await createImageBitmap(blob); + const surface = new OffscreenCanvas(128,72); + const context = surface.getContext('2d');context.drawImage(bitmap,0,0);bitmap.close(); + return Array.from(context.getImageData(20,20,1,1).data); + }; + paint('#ff0000');const first = await canvasPNG(canvas); + paint('#00ff00');const second = await canvasPNG(canvas); + let rerenders=0; + renderer={render(){rerenders++;paint('#0000ff');}}; + const original = OffscreenCanvas.prototype.convertToBlob; + OffscreenCanvas.prototype.convertToBlob=async function(){ctx.clearRect(0,0,128,72);throw new Error('injected encoder failure');}; + let fallback; + try { fallback=await canvasPNG(canvas); } finally { OffscreenCanvas.prototype.convertToBlob=original; } + return {first:await read(first),second:await read(second),fallback:await read(fallback),rerenders}; + }); + assert.deepEqual(result.first,[255,0,0,255]); + assert.deepEqual(result.second,[0,255,0,255]); + assert.deepEqual(result.fallback,[0,0,255,255]); + assert.equal(result.rerenders,1); + console.log('PASS: offscreen capture preserves pixels, reuses surface, and redraws on fallback.'); + } finally { await cleanup(); } +})().catch(error => { console.error(error);process.exitCode=1; }); diff --git a/studio/test_video_resume.py b/studio/test_video_resume.py index 8cd5ad6..e640e3a 100644 --- a/studio/test_video_resume.py +++ b/studio/test_video_resume.py @@ -590,6 +590,7 @@ class FakeProcess: def __init__(self, command, **_kwargs): self.stdin = io.BytesIO() self.output = Path(command[-1]) + self.command = command self.returncode = None processes.append(self) @@ -607,13 +608,26 @@ def kill(self): self.store.popen = FakeProcess job = self.store.create(request(fps=2, frames=6, checkpointSeconds=2)) lease = self.store.resume(job['id'])['lease'] - for frame in range(3): - progress = self.store.write_frame(job['id'], frame, b'png', lease) - self.assertEqual(progress['durableFrame'], 0) + with mock.patch.dict('os.environ', {'TESSERACT_TEST_ENCODER_THREADS': '2'}): + for frame in range(3): + progress = self.store.write_frame(job['id'], frame, b'png', lease) + self.assertEqual(progress['durableFrame'], 0) progress = self.store.write_frame(job['id'], 3, b'png', lease) self.assertEqual(progress['durableFrame'], 4) self.assertTrue(progress['checkpointed']) self.assertEqual(len(processes), 1) + command = processes[0].command + thread_flags = [index for index, arg in enumerate(command) if arg == '-threads'] + self.assertEqual(len(thread_flags), 2) + self.assertLess(thread_flags[0], command.index('-i'), 'PNG decoder input scope') + self.assertGreater(thread_flags[1], command.index('-c:v'), 'H.264 encoder output scope') + self.assertEqual([command[index + 1] for index in thread_flags], ['2', '2']) + self.assertEqual(command[command.index('-filter_threads') + 1], '1') + color_filter = command[command.index('-vf') + 1] + self.assertIn('scale=in_range=pc:out_range=tv:out_color_matrix=bt709', color_filter) + self.assertIn('setparams=range=limited:color_primaries=bt709:color_trc=bt709:colorspace=bt709', color_filter) + for flag in ('-color_range', '-colorspace', '-color_primaries', '-color_trc'): + self.assertEqual(command[command.index(flag) + 1], 'tv' if flag == '-color_range' else 'bt709') persisted = json.loads(self.manifest(job).read_text()) self.assertEqual(persisted['nextFrame'], 4) self.assertEqual(persisted['segments'][0]['frames'], 4) @@ -658,10 +672,12 @@ def kill(self): def test_encoder_failure_pauses_at_last_durable_frame(self): class FailingProcess: - def __init__(self, command, **_kwargs): + def __init__(self, command, **kwargs): self.stdin = io.BytesIO() self.output = Path(command[-1]) self.returncode = None + kwargs['stderr'].write(b'libx264 test diagnostic') + kwargs['stderr'].flush() def poll(self): return self.returncode @@ -682,6 +698,8 @@ def kill(self): self.assertEqual(persisted['state'], 'paused') self.assertEqual(persisted['nextFrame'], 0) self.assertEqual(persisted['segments'], []) + self.assertIn('libx264 test diagnostic', persisted['error']) + self.assertEqual(list(self.manifest(job).parent.glob('*.stderr.log')), []) def test_pause_discards_only_active_segment(self): class WaitingProcess: diff --git a/studio/test_video_resume_integration.py b/studio/test_video_resume_integration.py index b14f66e..7b4c591 100644 --- a/studio/test_video_resume_integration.py +++ b/studio/test_video_resume_integration.py @@ -68,7 +68,7 @@ def probe(self, path): result = subprocess.run([ self.ffprobe, '-v', 'error', '-count_frames', '-select_streams', 'v:0', - '-show_entries', 'stream=nb_read_frames:format=duration', + '-show_entries', 'stream=nb_read_frames,color_range,color_space,color_transfer,color_primaries:format=duration', '-of', 'json', str(path), ], check=True, capture_output=True, text=True) value = json.loads(result.stdout) @@ -112,6 +112,10 @@ def test_real_encoder_resumes_after_interruption_and_scratch_remount(self): output = self.root / 'videos' / completed['filename'] stream, media_format = self.probe(output) self.assertEqual(int(stream['nb_read_frames']), 5) + self.assertEqual(stream['color_range'], 'tv') + self.assertEqual(stream['color_space'], 'bt709') + self.assertEqual(stream['color_transfer'], 'bt709') + self.assertEqual(stream['color_primaries'], 'bt709') self.assertAlmostEqual(float(media_format['duration']), 2.5, delta=0.15) self.assertTrue(output.with_suffix('.json').is_file()) self.assertEqual(recreated.list_jobs(), []) @@ -193,6 +197,10 @@ def test_hard_server_crash_reaps_stopped_encoder_and_resumes_exactly(self): completed = recreated.finish(details['id']) stream, media_format = self.probe(self.root / 'videos' / completed['filename']) self.assertEqual(int(stream['nb_read_frames']), 5) + self.assertEqual(stream['color_range'], 'tv') + self.assertEqual(stream['color_space'], 'bt709') + self.assertEqual(stream['color_transfer'], 'bt709') + self.assertEqual(stream['color_primaries'], 'bt709') self.assertAlmostEqual(float(media_format['duration']), 2.5, delta=0.15) finally: if child.poll() is None: diff --git a/studio/video_resume.py b/studio/video_resume.py index cb870e2..49c41e7 100644 --- a/studio/video_resume.py +++ b/studio/video_resume.py @@ -395,6 +395,7 @@ def create(self, request): 'frames': frames, 'checkpointSeconds': checkpoint_seconds, 'scratchPath': str(scratch_base), + 'colorProfile': 'bt709-limited-v1', } job = { 'schemaVersion': SCHEMA_VERSION, @@ -479,6 +480,7 @@ def _validate_job_directory(self, manifest, job_id): removable = re.compile( r'(?:job\.json|\.owner\.lock|\.concat\.txt|' r'segment-\d{6}\.mkv|\.segment-\d{6}\.pending\.mkv|' + r'\.segment-\d{6}\.stderr\.log|' r'\.job\.json\.[a-f0-9]{32}\.tmp)' ) for path in job_dir.iterdir(): @@ -556,6 +558,8 @@ def _verify_segments(self, manifest, job, repair=False): for path in job_dir.glob('.segment-*.pending.mkv'): path.unlink(missing_ok=True) + for path in job_dir.glob('.segment-*.stderr.log'): + path.unlink(missing_ok=True) for path in job_dir.glob('segment-*.mkv'): if path.resolve() not in recorded: path.unlink(missing_ok=True) @@ -660,10 +664,13 @@ def _interrupt_active_segment(self, job_id): pending = runtime.get('pending') if pending: pending.unlink(missing_ok=True) + detail = self._consume_encoder_stderr(runtime) if stopped else '' runtime_job = runtime.get('job') runtime_manifest = runtime.get('manifest') if runtime_job and runtime_manifest: runtime_job.pop('activeEncoder', None) + if detail: + runtime_job['error'] = f'Checkpoint encoder diagnostic: {detail}' try: self._save(runtime_manifest, runtime_job) except OSError: @@ -716,25 +723,59 @@ def _start_segment(self, manifest, job, lease): raise ValueError( f'Not enough scratch space for the next checkpoint ({free_bytes:,} bytes free)' ) + # Existing jobs without a profile keep their original conversion. Mixing + # old untagged checkpoints with new BT.709 checkpoints would change color + # mid-video and concat would advertise only the first segment's metadata. + color_args = [] + if request.get('colorProfile') == 'bt709-limited-v1': + color_args = [ + '-vf', ('scale=in_range=pc:out_range=tv:out_color_matrix=bt709,' + 'setparams=range=limited:color_primaries=bt709:' + 'color_trc=bt709:colorspace=bt709'), + '-color_range', 'tv', '-colorspace', 'bt709', + '-color_primaries', 'bt709', '-color_trc', 'bt709', + ] + # The guarded test runner constrains only test codecs. Normal exports + # retain their usual ffmpeg threading policy. ffmpeg options are scoped: + # input-side -threads bounds PNG decoding; output-side -threads bounds + # libx264 encoding. Both occurrences are necessary, not duplicates. + test_threads = os.environ.get('TESSERACT_TEST_ENCODER_THREADS') + thread_args = [] + if test_threads: + count = int(test_threads) + if not 1 <= count <= 4: + raise ValueError('Test encoder threads must be between 1 and 4') + thread_args = ['-threads', str(count)] command = [ self.ffmpeg, '-hide_banner', '-loglevel', 'error', '-y', + *(['-filter_threads', '1'] if thread_args else []), *thread_args, '-f', 'image2pipe', '-framerate', str(request['fps']), - '-vcodec', 'png', '-i', 'pipe:0', '-an', - '-c:v', 'libx264', '-preset', 'medium', + '-vcodec', 'png', '-i', 'pipe:0', '-an', *color_args, + '-c:v', 'libx264', '-preset', 'medium', *thread_args, '-b:v', str(bit_rate), '-maxrate', str(round(bit_rate * 1.5)), '-bufsize', str(bit_rate * 2), '-pix_fmt', 'yuv420p', '-f', 'matroska', str(pending), ] - process = self.popen( - command, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - ) + stderr_path = manifest.parent / f'.segment-{segment_index:06d}.stderr.log' + stderr_path.unlink(missing_ok=True) + stderr_stream = stderr_path.open('xb') + try: + process = self.popen( + command, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, + stderr=stderr_stream, + ) + except Exception: + stderr_stream.close() + stderr_path.unlink(missing_ok=True) + raise runtime = { 'manifest': manifest, 'job': job, 'lease': lease, 'process': process, 'pending': pending, + 'stderrPath': stderr_path, + 'stderrStream': stderr_stream, 'firstFrame': job['nextFrame'], 'written': 0, } @@ -755,6 +796,27 @@ def _start_segment(self, manifest, job, lease): raise return runtime + @staticmethod + def _consume_encoder_stderr(runtime): + stream = runtime.pop('stderrStream', None) + path = runtime.pop('stderrPath', None) + if stream and not stream.closed: + try: + stream.flush() + except OSError: + pass + stream.close() + detail = '' + if path: + try: + with path.open('rb') as error_stream: + error_stream.seek(max(0, path.stat().st_size - 4096)) + detail = error_stream.read().decode(errors='replace').strip() + except OSError: + pass + path.unlink(missing_ok=True) + return detail + def _fail_active_segment(self, job_id, runtime, message): process = runtime['process'] stopped = process.poll() is not None @@ -772,6 +834,9 @@ def _fail_active_segment(self, job_id, runtime, message): pass if runtime.get('pending'): runtime['pending'].unlink(missing_ok=True) + detail = self._consume_encoder_stderr(runtime) + if detail: + message = f'{message}: {detail}' with self.lock: self.active.pop(job_id, None) self.leases.pop(job_id, None) @@ -793,6 +858,7 @@ def _finalize_segment(self, job_id, runtime): if return_code != 0 or not runtime['pending'].is_file(): self._fail_active_segment(job_id, runtime, 'Checkpoint encoder exited before finalizing the segment') raise ValueError('Checkpoint encoder failed') + self._consume_encoder_stderr(runtime) try: with runtime['pending'].open('rb') as stream: diff --git a/studio/video_soak.py b/studio/video_soak.py new file mode 100644 index 0000000..243e491 --- /dev/null +++ b/studio/video_soak.py @@ -0,0 +1,74 @@ +"""Opt-in production-resolution checkpoint/restart validation, isolated in temporary storage.""" +import argparse +import json +import shutil +import subprocess +import tempfile +from pathlib import Path + +from video_resume import VideoJobStore + + +def run(source, seconds): + ffmpeg, ffprobe = shutil.which('ffmpeg'), shutil.which('ffprobe') + if not ffmpeg or not ffprobe: + raise ValueError('ffmpeg and ffprobe are required') + if not source.is_file(): + raise ValueError('Provide an existing artwork video using VIDEO_SOAK_SOURCE') + fps, width, height = 30, 1920, 1080 + frames = round(seconds * fps) + with tempfile.TemporaryDirectory(prefix='tesseract-video-soak-') as temporary: + root = Path(temporary) + # Only three PNG samples live on disk and only one frame is read at a time. + subprocess.run([ffmpeg, '-hide_banner', '-loglevel', 'error', '-i', str(source.resolve()), + '-vf', 'fps=1,scale=1920:1080', '-frames:v', '3', + str(root / 'sample-%02d.png')], check=True, timeout=60) + samples = sorted(root.glob('sample-*.png')) + if not samples: + raise ValueError('Could not extract artwork frames') + store = VideoJobStore(root, ffmpeg) + job = store.create({'name': 'production-soak', 'width': width, 'height': height, + 'fps': fps, 'frames': frames, 'format': 'mp4', 'quality': 'high', + 'bitRate': 7464960, 'checkpointSeconds': 1, + 'sourceUrl': '/', 'renderSignature': 'soak-v1'}) + try: + lease = store.resume(job['id'])['lease'] + frame = 0 + restarted = set() + while frame < frames: + store.write_frame(job['id'], frame, samples[frame % len(samples)].read_bytes(), lease) + frame += 1 + # Pause mid-checkpoint twice, recreate the store, verify and rerender. + if frame in (max(1, frames // 3), max(2, frames * 2 // 3)) and frame not in restarted: + restarted.add(frame) + store.pause(job['id'], 'soak restart', lease) + store = VideoJobStore(root, ffmpeg) + store.recover_stale_encoders() + resumed = store.resume(job['id']) + frame, lease = resumed['nextFrame'], resumed['lease'] + completed = store.finish(job['id']) + output = root / 'videos' / completed['filename'] + result = subprocess.run([ffprobe, '-v', 'error', '-count_frames', '-select_streams', 'v:0', + '-show_entries', 'stream=nb_read_frames,color_space,color_transfer,color_primaries:format=duration', + '-of', 'json', str(output)], check=True, capture_output=True, text=True, timeout=60) + media = json.loads(result.stdout) + stream = media['streams'][0] + assert int(stream['nb_read_frames']) == frames + assert abs(float(media['format']['duration']) - frames / fps) < .15 + for key in ('color_space', 'color_transfer', 'color_primaries'): + assert stream[key] == 'bt709' + assert not store.list_jobs() + assert not store.processes_for_path(root) + print(f'PASS: {frames} artwork-derived 1080p frames, two restart/resume cycles, exact duration and BT.709; no encoder leaks.') + finally: + store.pause_all() + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--source', required=True, type=Path) + parser.add_argument('--seconds', type=int, default=6) + args = parser.parse_args() + if not 3 <= args.seconds <= 3600: + parser.error('--seconds must be between 3 and 3600; long runs are opt-in') + run(args.source, args.seconds) diff --git a/tesseract.js b/tesseract.js index 7bd6e7c..8626ead 100644 --- a/tesseract.js +++ b/tesseract.js @@ -20,6 +20,8 @@ let videoExportAbort = null; let videoExportStopAction = null; let videoExportStopPromise = null; let videoExportStopJobId = null; +let videoCaptureSurface = null; +let videoCaptureContext = null; let loopTiming = { period: 0, frameCount: 1, timeStep: 0, exact: true }; let lastAnimationTimestamp = null; let frameAccumulator = 0; @@ -250,6 +252,9 @@ function createEnvironmentMap() { updateMaterialsWithEnvMap(); previous?.dispose(); }, + onStateChange: () => { + if (videoExportDialog) updateVideoExportSummary(); + }, confirmAction, onSwitchChrome: () => { const select = document.getElementById('viewer-shader'); @@ -844,6 +849,7 @@ function createControls() { // Lighting type selector const lightingSelect = document.createElement('select'); + lightingSelect.id = 'viewer-lighting'; lightingSelect.style.width = '100%'; lightingSelect.style.padding = '3px'; lightingSelect.style.backgroundColor = '#222'; @@ -872,11 +878,13 @@ function createControls() { // Light distance control const distanceLabel = document.createElement('div'); + distanceLabel.id = 'viewer-light-distance-label'; distanceLabel.textContent = `Light Distance: ${lightDistance.toFixed(1)}`; distanceLabel.style.marginTop = '5px'; distanceLabel.style.marginBottom = '5px'; const distanceSlider = document.createElement('input'); + distanceSlider.id = 'viewer-light-distance'; distanceSlider.type = 'range'; distanceSlider.min = '0.1'; distanceSlider.max = '20'; @@ -966,21 +974,6 @@ function createControls() { lightingContainer.appendChild(distanceSlider); controlPanel.appendChild(lightingContainer); - // Helper function to update all materials - function updateMaterials() { - const previousMaterials = new Set([...faces, ...vertices].map(object => object.material)); - const newMaterial = createShaderMaterial(); - - faces.forEach(face => { - face.material = newMaterial; - }); - - vertices.forEach(vertex => { - vertex.material = newMaterial; - }); - previousMaterials.forEach(material => material.dispose()); - } - // Add rotation controls const rotationControls = document.createElement('div'); rotationControls.id = 'rotationControls'; @@ -1311,6 +1304,41 @@ function createControls() { updateCameraInfo(); } +function updateMaterials() { + const previousMaterials = new Set([...faces, ...vertices].map(object => object.material).filter(Boolean)); + const newMaterial = createShaderMaterial(); + faces.forEach(face => { face.material = newMaterial; }); + vertices.forEach(vertex => { vertex.material = newMaterial; }); + previousMaterials.forEach(material => { + if (material !== newMaterial) material.dispose(); + }); +} + +function syncViewerControlsFromState() { + if (motionStepSlider) motionStepSlider.value = String(rotationSpeed * 1000); + if (motionStepLabel) motionStepLabel.textContent = `Motion Step: ${rotationSpeed.toFixed(3)} / frame`; + const rotationControls = document.getElementById('rotationControls'); + if (rotationControls) { + const sliders = rotationControls.querySelectorAll('input[type="range"]'); + const values = rotationControls.querySelectorAll('span'); + ['xw', 'yw', 'zw'].forEach((axis, index) => { + if (sliders[index]) sliders[index].value = String(rotationCoefficients[axis] * 20); + if (values[index]) values[index].textContent = rotationCoefficients[axis].toFixed(2); + }); + } + const shader = document.getElementById('viewer-shader'); + if (shader) shader.value = currentShader; + const lighting = document.getElementById('viewer-lighting'); + if (lighting) lighting.value = currentLighting; + const distance = document.getElementById('viewer-light-distance'); + if (distance) distance.value = String(lightDistance); + const distanceLabel = document.getElementById('viewer-light-distance-label'); + if (distanceLabel) distanceLabel.textContent = `Light Distance: ${lightDistance.toFixed(1)}`; + const vertexToggle = document.getElementById('vertexToggle'); + if (vertexToggle) vertexToggle.checked = showVertices; + if (exportFpsSelect) exportFpsSelect.value = String(exportFps); +} + const COEFFICIENT_SCALE = 1000000; function greatestCommonDivisor(a, b) { @@ -1501,7 +1529,8 @@ function videoRenderSignature() { position: ['x', 'y', 'z'].map(axis => Number(camera.position[axis].toFixed(9))), target: ['x', 'y', 'z'].map(axis => Number(controls.target[axis].toFixed(9))) }, - fps: exportFps + fps: exportFps, + environment: skyboxLibrary?.getRenderState().identity || null }); } @@ -1703,12 +1732,17 @@ async function loadVideoResumeJobs() { restore.textContent = 'Restore export settings'; restore.addEventListener('click', async () => { applyViewerSettings(request.viewerState); + const restoredFrame = timelineFrame; camera.position.set(savedCameraPosition.x, savedCameraPosition.y, savedCameraPosition.z); controls.target.set(savedCameraTarget.x, savedCameraTarget.y, savedCameraTarget.z); controls.update(); vertices.forEach(vertex => { vertex.visible = showVertices; }); + syncViewerControlsFromState(); updateMaterials(); - refreshLoopTiming(); + skyboxLibrary?.refreshShaderHint(); + refreshLoopTiming({ preserveTime: false }); + setTimelineFrame(restoredFrame); + updateCameraInfo(); persistViewerSettings(); videoExportDialog.dataset.renderSignature = videoRenderSignature(); await loadVideoResumeJobs(); @@ -1760,6 +1794,9 @@ function videoExportPlan() { const scratchPath = document.getElementById('video-export-scratch').value.trim(); if (!loopTiming.exact || loopTiming.frameCount < 1) error = 'The current motion does not have an exportable timeline.'; + const renderState = skyboxLibrary?.getRenderState(); + if (!renderState?.ready) error = 'Wait for the selected skybox to finish loading before exporting.'; + else if (!renderState.identity) error = 'Skybox verification requires HTTPS or localhost before exporting video.'; if (!Number.isInteger(checkpointSeconds) || checkpointSeconds < 0 || checkpointSeconds > 3600) error = 'Checkpoint duration must be between 0 and 3,600 seconds.'; if (range === 'clip') { const startSeconds = parseTimeInput(document.getElementById('video-export-start').value); @@ -1909,9 +1946,38 @@ function dataUrlToBlob(dataUrl) { return new Blob([bytes], { type: 'image/png' }); } -function canvasPNG(canvas) { - // Synchronous read. toBlob's callback is deferred, and dropped, while the - // document is hidden — that left the export and the encoder waiting on each other. +async function canvasPNG(canvas) { + // Snapshot synchronously into a reusable offscreen surface, then let its + // promise-based encoder work without creating a large Base64 string. Unlike + // HTMLCanvasElement.toBlob(), OffscreenCanvas.convertToBlob() does not rely on + // a document callback that browsers may drop while the page is hidden. + if (typeof OffscreenCanvas !== 'undefined') { + if (!videoCaptureSurface || videoCaptureSurface.width !== canvas.width || videoCaptureSurface.height !== canvas.height) { + videoCaptureSurface = new OffscreenCanvas(canvas.width, canvas.height); + videoCaptureContext = videoCaptureSurface.getContext('2d', { alpha: false }); + } + if (videoCaptureContext) { + videoCaptureContext.drawImage(canvas, 0, 0); + let timeout; + try { + const blob = await Promise.race([ + videoCaptureSurface.convertToBlob({ type: 'image/png' }), + new Promise((_, reject) => { + timeout = setTimeout(() => reject(new Error('Offscreen PNG capture timed out')), 30000); + }) + ]); + if (blob?.size) return blob; + throw new Error('Offscreen PNG capture returned an empty frame'); + } catch (error) { + console.warn('Offscreen video capture failed; using the compatibility path', error); + // The WebGL drawing buffer may have been cleared while awaiting + // the encoder. Re-render this same deterministic frame first. + renderer.render(scene, camera); + } finally { + clearTimeout(timeout); + } + } + } return dataUrlToBlob(canvas.toDataURL('image/png')); } @@ -2056,8 +2122,10 @@ async function runVideoExport(plan, resumableJob = null) { const originalPixelRatio = renderer.getPixelRatio(); let lastProgressUpdate = 0; let exportStartedAt = 0; + let releaseEnvironmentLock = null; try { + releaseEnvironmentLock = skyboxLibrary.acquireRenderLock(); const gpuLimit = renderer.getContext().getParameter(renderer.getContext().MAX_RENDERBUFFER_SIZE); if (plan.width > gpuLimit || plan.height > gpuLimit) throw new Error(`This GPU can render video up to ${gpuLimit}px per side`); if (resumableJob) { @@ -2101,7 +2169,7 @@ async function runVideoExport(plan, resumableJob = null) { setTimelineFrame(plan.startFrame + frame); updateTesseractProjection(); renderer.render(scene, camera); - const png = canvasPNG(renderer.domElement); + const png = await canvasPNG(renderer.domElement); await new Promise(resolve => setTimeout(resolve, 0)); if (cancelVideoExportRequested) throw new DOMException('Video export cancelled', 'AbortError'); const frameResult = await videoApi(`/api/video/frame?id=${activeVideoExportJob.id}&frame=${frame}&lease=${activeVideoExportJob.lease}`, { @@ -2175,9 +2243,12 @@ async function runVideoExport(plan, resumableJob = null) { updateTesseractProjection(); renderer.render(scene, camera); animationPaused = originalPaused; + releaseEnvironmentLock?.(); videoExportRunning = false; cancelVideoExportRequested = false; videoExportAbort = null; + videoCaptureSurface = null; + videoCaptureContext = null; videoExportStopPromise = null; videoExportStopJobId = null; fields.forEach(field => { field.disabled = false; }); diff --git a/viewer-skyboxes.js b/viewer-skyboxes.js index 4f84354..24afcef 100644 --- a/viewer-skyboxes.js +++ b/viewer-skyboxes.js @@ -13,8 +13,21 @@ function diagnosticTexture() { }); const texture=new THREE.CubeTexture(images);texture.needsUpdate=true;return texture; } -function loadImage(url) { - return new Promise((resolve,reject)=>{const image=new Image();image.onload=()=>resolve(image);image.onerror=()=>reject(new Error(`Missing or unreadable image: ${url}`));image.src=url;}); +async function loadImageAsset(url,{requireHash=true}={}) { + // Published static works may run on plain HTTP (e.g. signage players). + // Export-capable loads must still hash actual bytes for resume identity. + const subtle=globalThis.crypto?.subtle; + if(!subtle&&requireHash)throw new Error('Skybox verification requires HTTPS or localhost'); + const response=await fetch(url,{cache:'no-store'}); + if(!response.ok)throw new Error(`Missing or unreadable image: ${url}`); + const bytes=await response.arrayBuffer(); + const digest=subtle?await subtle.digest('SHA-256',bytes):null; + const sha256=digest?Array.from(new Uint8Array(digest),byte=>byte.toString(16).padStart(2,'0')).join(''):null; + const objectURL=URL.createObjectURL(new Blob([bytes],{type:response.headers.get('Content-Type')||'image/png'})); + try{ + const image=await new Promise((resolve,reject)=>{const value=new Image();value.onload=()=>resolve(value);value.onerror=()=>reject(new Error(`Missing or unreadable image: ${url}`));value.src=objectURL;}); + return {image,sha256}; + }finally{URL.revokeObjectURL(objectURL);} } function textureFromImages(images,renderer,expectedSize=null) { const side=images[0].naturalWidth; @@ -36,21 +49,33 @@ function slugify(value) { return value.toLowerCase().normalize('NFKD').replace(/[\u0300-\u036f]/g,'').replace(/[^a-z0-9]+/g,'-').replace(/^-|-$/g,'').slice(0,60); } -export function installSkyboxLibrary({mount,renderer,getShader,onTexture,onSwitchChrome,getViewerState,publication,confirmAction}) { +export function installSkyboxLibrary({mount,renderer,getShader,onTexture,onSwitchChrome,getViewerState,onStateChange=()=>{},publication,confirmAction}) { + let renderReady=false,renderIdentity=null,renderLocks=0; + const setRenderState=(ready,identity=renderIdentity)=>{renderReady=ready;renderIdentity=identity;onStateChange({ready:renderReady,identity:renderIdentity});}; + const renderApi={ + getRenderState(){return {ready:renderReady,identity:renderIdentity};}, + acquireRenderLock(){ + if(!renderReady)throw new Error('Wait for the skybox to finish loading before exporting video'); + if(!renderIdentity)throw new Error('Skybox verification requires HTTPS or localhost before exporting video'); + renderLocks+=1;let released=false; + return ()=>{if(!released){released=true;renderLocks=Math.max(0,renderLocks-1);}}; + } + }; if(publication?.schemaVersion===1){ const section=document.createElement('section');section.className='viewer-skybox'; section.innerHTML='
WORK

Loading published environment…

'; mount.prepend(section);section.querySelector('#active-skybox').textContent=publication.title||'Published work'; - const status=section.querySelector('#skybox-status');onTexture(diagnosticTexture()); + const status=section.querySelector('#skybox-status');onTexture(diagnosticTexture());setRenderState(false,null); (async()=>{ try{ if(!publication.skybox||!FACES.every(face=>publication.skybox[face]===`skybox/${face}.png`))throw new Error('Published skybox configuration is invalid'); - const images=await Promise.all(FACES.map(face=>loadImage(publication.skybox[face]))); - const {texture,side,displaySide}=textureFromImages(images,renderer,publication.size??null);onTexture(texture); + const assets=await Promise.all(FACES.map(face=>loadImageAsset(publication.skybox[face],{requireHash:false}))); + const {texture,side,displaySide}=textureFromImages(assets.map(asset=>asset.image),renderer,publication.size??null);onTexture(texture); + setRenderState(true,assets.every(asset=>asset.sha256)?JSON.stringify({kind:'published',slug:publication.slug||'',manifestSha256:publication.source?.manifestSha256||null,faces:Object.fromEntries(FACES.map((face,index)=>[face,assets[index].sha256]))}):null); status.textContent=`${side} × ${side}px${displaySidemanifest.outputs[face]?.file===`${face}.png`))throw new Error('Manifest must identify all six named PNG faces'); name=manifest.pipeline?.name||folder;size=manifest.pipeline?.size; + expectedHashes=Object.fromEntries(FACES.map(face=>[face,manifest.outputs[face].sha256||null])); } const urls=FACES.map(face=>folder==='default'?`/skybox/${face}.png`:exportFileURL(folder,`${face}.png`)); - const images=await Promise.all(urls.map(loadImage)); - const {texture,side,displaySide}=textureFromImages(images,renderer,size); + const assets=await Promise.all(urls.map(url=>loadImageAsset(url))); + if(expectedHashes&&FACES.some((face,index)=>expectedHashes[face]&&expectedHashes[face]!==assets[index].sha256))throw new Error('Skybox image content does not match the export manifest'); + const identity=JSON.stringify({kind:folder==='default'?'default':'export',folder,faces:Object.fromEntries(FACES.map((face,index)=>[face,assets[index].sha256]))}); + const {texture,side,displaySide}=textureFromImages(assets.map(asset=>asset.image),renderer,size); if(version!==requestId){texture.dispose();return false;} - onTexture(texture);active=folder;activeName=name; + onTexture(texture);active=folder;activeName=name;setRenderState(true,identity); $('active-skybox').textContent=name;$('active-skybox').title=folder==='default'?'skybox/':folder; message(`${side} × ${side}px${displaySide