Skip to content

Add a timestamp for the samples and weight the report by the sample duration - #284

Merged
mattip merged 4 commits into
vmprof:masterfrom
mattip:lost-samples
Sep 22, 2026
Merged

mattip merged 4 commits into
vmprof:masterfrom
mattip:lost-samples

Conversation

@mattip

@mattip mattip commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Fixes #232 Fixes #233

vmprof samples via a signal handler driven by one process-wide setitimer. A
pending signal is a single bit, not a queue, so every timer expiry that lands
while the process is off the cpu or the previous signal is still pending is
silently dropped. Stack records carry no timestamp and their count is always
1, so nothing can tell that samples went missing, and every consumer assumes
samples * interval == time.

This PR adds such a timestamp and weights each sample by the time it actually stands for.

Writer: every stack sample ends with a 64-bit nanosecond timestamp, after
the existing fields, and the header version becomes VERSION_SAMPLE_TIME (7).
The clock matches the timer: process cpu time for ITIMER_PROF, the monotonic
wall clock for ITIMER_REAL and for the Windows sampler thread. Using the wall
clock in cpu-time mode would charge idle time to whichever frame ran next.
The frame walk reserves room for the trailer, which also closes a one-slot overrun
when memory profiling was on.

Reader: a sample is worth the gap to the previous sample divided by the
period (at least 1) capped at max_sample_gap (default 1 s) so a process stopped
in a debugger does not attribute minutes to one frame. The previous sample is
tracked per thread in real-time mode, where every registered thread gets its own
signal, and process-wide in cpu-time mode, where there is one timer. Counts become
sums of weights and may be floats. Stats gains n_samples, expected_samples, lost_time and get_lost_fraction(). Older files are read as before with every weight set to 1.

This echoes the approach used in Linux perf, Austin, Scalene, Datadog's python profiler, and pyinstrument. All these statistical profilers take the approach of measuring sample duration, and not just a count.

@mattip
mattip merged commit 81d659b into vmprof:master Sep 22, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recorded profile end time is too late, making it hard to timestamp each sample SIGALRM not always delivered to main thread in practice

1 participant