[BUG] Race condition in Dash._setup_server(): guard flag is set before the work it protects

[a-tram]

Describe your context

dash          4.4.1
dash_ag_grid  35.3.0     (only used by the first script, to have a component bundle to fetch;
                          the second script needs dash alone)

Flask 3.1.3 · Werkzeug 3.1.8 · gunicorn 26.2.0 (gthread, 4 workers × 4 threads) · Python 3.13

This is a server-side race, not a frontend bug, although the visible symptom is in the browser.

• OS: Linux aarch64; also reproduced independently on Linux x86-64 in Docker

Describe the bug

Dash._setup_server() runs as a before_request hook and is meant to execute once per
process. It sets its guard flag before performing the work that flag protects:

def _setup_server(self):                                    # dash.py:1702
    if self._got_first_request["setup_server"]:
        return                                              # <-- thread B leaves here
    self._got_first_request["setup_server"] = True          # <-- flag set BEFORE the work
    ...
    _validate.validate_layout(self.layout, self._layout_value())   # 1724
    self._generate_scripts_html()                                  # 1726, fills registered_paths
    ...
    self.callback_map[k] = _callback.GLOBAL_CALLBACK_MAP.pop(k)    # 1737, fills callback_map

This is a TOCTOU. On any WSGI server that serves more than one request at a time in a single
process — gunicorn -k gthread --threads N, waitress, werkzeug with threaded=True, uWSGI
with threads — a second thread can enter while the first is still inside _setup_server, find
the flag already set, return immediately, and then read state the first thread has not finished
writing. It only affects the first page load after a process starts, and a refresh makes it
go away, which is why it is easy to dismiss as a caching or network glitch.

Both symptoms below are reproduced deterministically by the scripts linked further down.

1. registered_paths still empty → every component-suites request 500s. A browser asks for
the index and its JS bundles on parallel connections. The index thread is inside
_setup_server; the bundle threads skip it and reach validate_js_path against an empty
allowlist:

dash.exceptions.DependencyException: Error loading dependency. "dash_ag_grid" is not a
registered library.
Registered libraries are:
[]

The browser gets HTML where it expected JavaScript (Refused to execute script ... MIME type ('text/html')), so grids and clientside code are silently broken on that load. This is what we
hit in production.

2. callback_map still incomplete → callbacks 500 with KeyError. Callbacks registered
with dash.callback (module level — how every pages-based app registers them) are moved from
GLOBAL_CALLBACK_MAP into self.callback_map only inside _setup_server. A thread that skips
the setup carries on to _prepare_callback:

  File "dash/dash.py", line 1628, in _prepare_callback
    cb = self.callback_map[output]
KeyError: 'out.children'
...
KeyError: "Callback function not found for output 'out.children'."

The second script includes the control that makes this meaningful: the same request, replayed
once initialisation has finished, returns 200.

Possibly related, not reproduced. #2925 reports RuntimeError: dictionary changed size during iteration with an identical signature ("occurs on startup and initial access … a full
refresh (Ctrl+F5) resolves the issue … affecting the first user to visit the app after startup
… becomes more frequent as callback numbers increase"). In 4.4.1 the only iteration of
callback_map on that path is validate_background_callbacks (dash.py:1754, called from
inside _setup_server; it was validate_long_callbacks in the 2.x line #2925 reports), so that
error would require two threads inside the body at once — possible in principle, since the
check and the set are separate statements and both threads can pass the check before either one
sets the flag. I could not trigger that: 0 occurrences in 400 trials × 8 threads released
through a barrier with sys.setswitchinterval(1e-6). I mention it because the two symptoms I
did reproduce show that this state really is observable half-written, but I am not claiming
#2925 is proven to be the same bug.

Steps to reproduce. Deterministic, with no monkeypatching of Dash. The only "slow" ingredient
is a layout function that takes a moment — an ordinary application property, and the pattern
Dash documents as app.layout = serve_layout. It matters because _setup_server evaluates the
layout before filling registered_paths and callback_map, so it widens the pre-existing
window. Ordering of the requests is arranged on the client side only.

Reproduction scripts: https://gist.github.com/a-tram/1a522f6f2beb388543de7d5f18644962

pip install dash dash-ag-grid
python repro_setup_server_race.py       # symptom 1
python repro_callback_map_race.py       # symptom 2, with a control

Output on dash 4.4.1, identical on 5 consecutive runs each:

index: 200
bundle x40: {500: 40}
REPRODUCED

index: 200
update-component x20: {500: 20}
control (same request, setup finished): 200
REPRODUCED

Every one of those bundle 500s is the DependencyException quoted above, with Registered libraries are: [].

Note that gunicorn's default worker class is sync (one request at a time per worker), so the
canonical gunicorn app:server -w 4 deployment is immune. Only threaded deployments are
exposed, which probably explains why this is not reported more often. --preload does not help
either: _setup_server is a before_request hook, so it still runs after the fork, in every
worker.

Expected behavior

The first request to a freshly started process should not be able to observe half-initialised
state. Concurrent first requests should either wait for initialisation to finish or all see it
complete — never a raised flag next to an empty registered_paths or a partially filled
callback_map.

Suggested fix. Moving the flag after the work is not sufficient on its own: two threads would
then both run the full setup, and the GLOBAL_CALLBACK_MAP.pop(k) loop is not safe to run
twice. Double-checked locking keeps the warm path lock-free, and also closes the second window
noted above — today the check and the set are separate statements, so the flag does not prevent
two threads from running the whole body at once:

# in __init__
self._setup_lock = threading.Lock()

def _setup_server(self):
    if self._got_first_request["setup_server"]:
        return
    with self._setup_lock:
        if self._got_first_request["setup_server"]:
            return
        ...  # existing body, unchanged
        self._got_first_request["setup_server"] = True   # set LAST

The same check-then-set-then-work shape appears for the "pages" key in dash.py
(router_async and router_sync), so it may be worth auditing that one as well.

Workaround for anyone hitting this now. Force the initialisation at worker start, where no
other thread can race it — for gunicorn, in post_worker_init:

with app.server.test_request_context("/"):
    app._setup_server()

Screenshots

Not applicable — both reproductions are deterministic and print their result.