Degenerate ray direction defeats walkmesh BVH culling - #341
Open
MichaelMoroz wants to merge 2 commits into
Open
Conversation
Game::update already multiplies frameTime by _gameSpeed, but nothing could
write that field except two developer keys clamped to [1, 8]. They exist for
playing faster, so there was no way to ask for a slower world and none at all
for a still one.
gamespeed <multiplier> writes it directly, taking anything in [0, 8]. Zero is
the setting it exists for: every call in the update path still runs, with
dt == 0.
That is deliberately not the same as pausing, which is the obvious-looking
equivalent and is not one. Game::update gates the whole module tick on _paused:
if (updModule && !_paused) { ... _module->update(dt); ... }
so pausing does not hold the world still, it removes the code that moves it.
A world that is asked to update and does not move is where a step that clamps
to zero length - or any other degenerate-timestep behaviour - becomes
observable rather than absent. The commit that follows this one fixes a bug
found exactly that way, and gamespeed 0 is how it is reproduced.
Callers derive a ray direction by normalising a step, and glm::normalize of a
zero-length vector is NaN. That is not merely a miss. AABB::raycast
reciprocates the direction and compares slabs, every comparison against a NaN
is false, so tmax < tmin never rejects and the tree culls nothing:
raycastAABB walks the whole thing and calls raycastFace on all 3,132 triangles
rather than the ~10 the ray crosses. Since the caller repeats the step every
frame, it costs milliseconds of frame time for a ray that cannot hit anything.
A ray with no direction has no intersection, so answer that and return. One
check at Walkmesh::raycast, which is the single entry point testWalk,
testElevation and testLineOfSight all reach.
Two details it depends on. The test is a negated > rather than a comparison
against zero, because every comparison involving a NaN is false and
`dot < 0.0f` would let one through. And the origin is checked as well as the
direction: testElevation passes a fixed downward direction and takes its NaN
in through the position instead, which is how the second query degenerated
alongside the first.
Measured with the preceding commit's gamespeed 0, headless, loadgame 151 /
warp danm14ad, an 85 s capture on each side. Over the sustained region:
before after
SceneGraph::testWalk p50 2,211 us 5 us
p99 2,642 us 8 us
Module::update p50 4,773 us 353 us
mean 4,833 us 367 us
Before the command lands the two builds agree, 16.8-18.7 us per 5 s bucket
either way, so nothing that was working changed.
This fixes the traversal, not the two things that made it expensive rather
than merely wrong: raycastFace still copies its std::set surface filter by
value once per triangle, and Creature::_pathVelocity still stays NaN for the
life of a path once normalize poisons it. Both are real and neither is this.
MichaelMoroz
force-pushed
the
fix/walkmesh-degenerate-ray
branch
from
August 29, 2026 17:50
93af09e to
3e28616
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Degenerate ray direction defeats walkmesh BVH culling
Area::moveCreaturebuilds a step whose length clamps to zero when a creature isalready at its destination, so
SceneGraph::testWalkcomputesglm::normalize(dest - origin)on a zero vector and gets a NaN direction. ThatNaN reaches
AABB::raycastas the reciprocal directioninvDir, where everyslab-test comparison is false and
tmax < tminnever rejects, so no node isculled.
Walkmesh::raycastAABBtherefore sweeps the entire BVH and callsWalkmesh::raycastFaceon all 3,132 triangles rather than the ~10 the rayactually crosses. Because
raycastFacetakes itsstd::set<uint32_t>surfacefilter by value, each of those calls copies a 19-element red-black tree to
perform one
count(), turning an 18 us query into 2,267 us.Warping into the level gives creatures
MoveToObjectorders, and once theyarrive they re-issue the zero-length step every frame, so the degenerate
traversal is permanent rather than transient. The NaN also propagates into
dest, so the followingSceneGraph::testElevationdegenerates identically -with four creatures each paying both queries,
Module::updatesettles at asustained ~19 ms per frame.
Reaching the state
The state needs the zero-length step to repeat. At a normal timestep a
creature's
_pathVelocitypicks up a non-zero steering force within a frame ortwo, so the degenerate traversal fires once and clears. Handing the module a
zero timestep removes that escape: every call still runs,
speedDtis zeroevery frame, and
dest == originforever.gamespeed 0is the control that does this, and it reproduces the bug ondemand. Headless, retro,
loadgame 151/warp danm14ad/gamespeed 0, one85 s capture spanning the switch:
SceneGraph::testWalk, 5 s bucketsOver the sustained region:
testWalkp50 2,211 us / p99 2,642 us across 11,362calls, and
Module::updatep50 4,773 us / mean 4,833 us. It does not decay -every 5 s bucket to the end of the run reads the same. This is the state the
original 19 ms reproduction was in; the frame cost here is lower only because
this module has fewer navigating creatures paying it.
Pausing does not do this, and cannot.
Game::updategates the whole moduletick on
_paused:so a pause removes the code containing the bug rather than holding it still.
Measured the same way,
gamepause onlefttestWalkat p50 16.7 us / p9931.8 us before the switch and called it exactly zero times across the 6,372
frames after it, while
Game::updatekept running all 9,415. That is whygamepausewas replaced bygamespeed, which scales the timestep the way--freezeframedid rather than skipping the update.No script-only reproduction yet
The scenario without any dev control does not reach the sustained state.
loadgame 151/warp danm14ad, headless at a fixed 1/60 timestep, 96.7 s and112,720
testWalkcalls covering the warp and 70 s after it: every 5 s bucketreads a mean of 16.7 - 19.3 us, 17 calls exceed 200 us, 4 exceed 1 ms, and the
tail is 0.51% of total time. Two isolated 2.5-2.9 ms spikes are the transient
firing and clearing. Nothing sustains.
The route that reaches it at a normal timestep is a creature whose steering
force is exactly zero on the first frame of a fresh path -
_pathVelocityisstill
{0, 0, 0},glm::normalizeof it is NaN, and because NaN survives+=the velocity never recovers for the life of that path. A creature standingexactly on its next path point produces that, which is what a restored forced
MoveToObjectsets up: it is constructed withtimeout = 0, soexpiryMillisecondsequals the current world time, the firstexecutetakesthe expired branch, and
setPositionteleports the actor onto its destination.Whether that is what the original session hit has not been confirmed - it needs
a save whose action list contains one, and it has not been reproduced from a
script.
The fix
One check at
Walkmesh::raycast, the single entry pointtestWalk,testElevationandtestLineOfSightall reach. A ray with no direction has nointersection, so that is the answer it gives.
The test is a negated
>rather than a comparison against zero, because everycomparison involving a NaN is false and
dot < 0.0fwould let one through. Theorigin is checked as well as the direction, because
testElevationpasses afixed downward direction and takes its NaN in through the position instead.
Over the sustained region, before and after:
SceneGraph::testWalkp50SceneGraph::testWalkp99Module::updatep50Module::updatemeanBefore the command lands the two builds agree, 16.8-18.7 us per 5 s bucket
either way, so nothing that was working changed.
This fixes the traversal, not the two things that made it expensive rather than
merely wrong:
raycastFacestill copies itsstd::setsurface filter by valueonce per triangle, and
Creature::_pathVelocitystill stays NaN for the life ofa path once
normalizepoisons it. Both are real and neither is this.Notes for review
The walkmesh,
Area::moveCreatureandCreature::navigateTocode theyexercise is the same code, but the absolute frame costs are not this tree's.
--freezeframeandgamepause, named in the text above, are that fork'scontrols and do not exist here. The first commit adds
gamespeed, which isthe one this tree needs to reproduce the bug.