Skip to content

Make the command executor thread count configurable - #222

Open
jordanfelle wants to merge 2 commits into
Chaptarr:developfrom
jordanfelle:perf-command-executor-threads
Open

jordanfelle wants to merge 2 commits into
Chaptarr:developfrom
jordanfelle:perf-command-executor-threads

Conversation

@jordanfelle

@jordanfelle jordanfelle commented Sep 26, 2026 •

Copy link
Copy Markdown

Problem

CommandExecutor runs a hard-coded 3 threads for every command in the application. On a large library a few long RefreshAuthor commands hold every slot for minutes (an author with ~5,000 books took 8-12 minutes), which starves RSS sync, download processing and imports, and makes the Tasks page back up with queued commands.

Fix

Read CHAPTARR_COMMAND_THREADS (1-32; invalid or unset keeps the existing default of 3), so hosts with headroom can raise it. Command exclusivity and the disk-access group still apply.

Results

Running with 10 threads together with #221 on a live instance (24-thread host):

Measured on a live instance (346k books) Before After
Books refreshed per minute 303 2,585
Rows read from the database per minute 3.7 million 0.5 million
Chaptarr CPU 144% 72%
Queued RefreshAuthor commands (285 at start of the run) 285 16

The books-per-minute figure combines this change with a higher command thread count (#222). Per thread the rate rose from about 100 to about 258 books per minute, which comes from #221; the same profile showed exactly one full-catalogue read per book before, and the top statement afterwards is a 2,580-call slug projection reading ~435k narrow rows.

Verification

Unit tests for the parser; core tests pass (3,029 on top of develop).

Note: the higher per-thread rate in the table comes from #221; this PR only raises concurrency.

  • Adds a startup log line with the thread count and CommandQueueConcurrentConsumersFixture: 10 consumers race for the queue and exactly 1 disk-access command starts, exactly 1 type-exclusive command starts, and all 10 per-author refreshes start (3,032 core tests pass). Npgsql's default pool is 100 connections, so up to 32 threads is safe; on SQLite a high count can add writer contention.

The executor runs a fixed 3 threads for every command in the application, so a few long RefreshAuthor commands (an author with ~5,000 books takes 8-12 minutes) hold every slot and starve RSS sync, download processing and imports. Read CHAPTARR_COMMAND_THREADS (1-32, default stays 3).
…th many consumers

The thread count is now configurable, so log it once at startup. Add queue-level tests with 10 racing TryGet consumers: disk-access commands and type-exclusive commands still run one at a time, while per-author refreshes (neither) can occupy every consumer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant