du: count subdirectories in --total with --separate-dirs, print time on total line by emirmrkaljevic · Pull Request #15455 · uutils/coreutils

emirmrkaljevic · 2026-10-17T13:23:12Z

Summary

du -c dropped subdirectory usage when -S/--separate-dirs was set, and it omitted the time column on the total line.

$ du -b -S -c d          # d/sub holds 4 bytes, d itself 8
4	d/sub
8	d
8	total                # before; GNU prints 12 total

$ du -b --time -c d
2	2024-03-01 18:00	d
2	total                # before; GNU prints 2	2024-03-01 18:00	total

The grand total now adds every directory under --separate-dirs (file sizes stay in their parent, so -a does not count them twice). With --time, the total line uses the same style and the newest atime, ctime, or mtime seen. A time before 1970 does not move that stamp backward from the epoch.

Refs #15254

Test plan

cargo test --features "du touch" --no-default-features --test tests test_du_total — pass
cargo test --features "du touch chmod" --no-default-features --test tests test_du:: — 118 passed
Manual: du --total --separate-dirs and du --total --time on the reported tree match GNU du 9.4 (28 . / 36 total, and 36 2024-03-01 18:00 total)

Found with behave.

codspeed · 2026-10-17T13:55:53Z

Merging this PR will not alter performance

1 warning

⚠️ 4 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 8 untouched benchmarks
⏩ 464 skipped benchmarks¹

_{Comparing emirmrkaljevic:mrki2/du-total-separate-dirs-time-7d60 (939f1e7) with main (366c18c)²}

464 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on main (732b9c5) during the generation of this report, so 366c18c was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

github-actions · 2026-10-17T14:39:40Z

GNU testsuite comparison:

Congrats! The gnu test tests/sort/sort is no longer failing!

xtqqczze · 2026-10-17T22:23:50Z

        }
    }

+    #[inline]


Was this not being inlined anyway? Non-public functions don't usually benefit from the #[inline] attribute.

Good point, thanks! They weren't needed, so I removed both in cc0d2a2. I also checked the CodSpeed result locally: repeated du -a runs on a 5000/500 wide tree show no measurable wall-clock difference vs the base (about -0.2%, 95% CI -1.7% to +1.3%). The bench is mostly syscall time, which simulation mode doesn't count, so the 3.6% looks like noise to me, but happy to dig further if you see it differently.

The CodSpeed result was likely noise, it has disappeared now.

sylvestre · 2026-10-18T07:39:41Z

@codspeedbot fix this regression

sylvestre · 2026-10-18T07:39:54Z

+
+#[cfg(feature = "touch")]
+#[test]
+fn test_du_total_time_column() {


this test is very long, could you please split it (total time / atime-ctime / pre-1970)?

sylvestre · 2026-10-18T07:40:10Z

-            )?;
+        if const { WITH_TOTAL } {
+            let time = if track_total_time {
+                match (saw_stat, grand_time) {


this seems complex, do we need saw_stat at all?

sylvestre · 2026-10-18T07:40:33Z

+        // `--separate-dirs` left them out. Only then does each directory's
+        // own size have to be added back. Files stay inside their directory.
+        let count_separate_dirs = WITH_TOTAL && self.separate_dirs;
+        // GNU's total timestamp starts at the epoch and moves forward only,


please make this comment shorter, and describe the behavior, not how GNU does it internally (you must not read their code)

sylvestre · 2026-10-18T07:40:37Z

    }

    fn print_stats(&self, rx: &mpsc::Receiver<UResult<StatPrintInfo>>) -> UResult<()> {
+        // Monomorphize so a run without `--total` does not track a total,


do we really need the const generic here? a plain if self.total would be much simpler

codspeed · 2026-10-18T07:40:50Z

@codspeedbot fix this regression

I fixed the regression in 44bc8f7 (du: fold drain_stats back into print_stats).

Cause. The du_all_wide_tree slowdown was real, not noise: it ran about 4% more instructions than main. Splitting the printer loop into drain_stats::<const WITH_TOTAL: bool> changed how the compiler inlined the printer thread. Receiver::recv stopped being inlined, and each StatPrintInfo message (which holds a full Metadata) got one extra memcpy per entry. With -a, du sends one message per file, so that copy adds up. The per-entry total and time bookkeeping is just a few branches, and it wasn't the cause.

Fix. I folded drain_stats back into a single print_stats loop. It checks self.total at runtime, as the code did before this PR. The behaviour is unchanged: the --separate-dirs sum, the time on the total line and the epoch clamp all work the same way.

Local simulation results:

Benchmark	`main`	PR before	PR after
`du_all_wide_tree[(5000, 500)]`	14.70–14.78 ms	15.14–15.22 ms	14.71 ms (2 runs)
`du_summarize_balanced_tree`	4.82–4.84 ms	4.90–4.92 ms	4.81 ms

Callgrind confirms it: total instructions are within about 0.3% of main, down from +4.4%. The extra memcpy calls and the out-of-line recv are gone.

Checks: test_du:: gives 115 passed and 3 failed. The 3 failures are permission tests that fail the same way on the PR without my change, because the sandbox runs as root. cargo fmt --check passes, and clippy reports no warnings for uu_du.

If you want another look after CI runs, comment @codspeedbot explain the new CodSpeed report.

Summary

Warming up the performance wizard...
Analyze the du_all_wide_tree regression — Real +4.4% instructions: the const-generic drain_stats split changed inlining in the printer thread, adding an out-of-line recv() and a memcpy of each StatPrintInfo message
Fix the regression if real — Folded drain_stats back into print_stats; du_all_wide_tree back to main's level (14.71 ms vs 14.70–14.78 ms on main locally)
Report results — Report results
Handling comment — Comment handling finished successfully

_{Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.}

sylvestre · 2026-10-18T07:40:50Z

 struct StatPrintInfo {
    stat: Stat,
    depth: usize,
+    /// Real directory bit for `--separate-dirs` with `--total`.


nobody will read such a long comment :) one line is enough

emirmrkaljevic · 2026-10-18T08:20:45Z

Thanks for the review, @sylvestre! Addressed in 329b250:

split the long test into test_du_total_time_column, test_du_total_time_atime_ctime and test_du_total_time_before_epoch
dropped saw_stat: the total time starts at 1970-01-01 and only moves forward
shortened the comments to describe behavior only
the const generic was already gone in 44bc8f7, print_stats uses a plain if self.total

xtqqczze · 2026-10-18T10:30:14Z

@emirmrkaljevic could you please rebase for a clean commit history

emirmrkaljevic · 2026-10-18T13:06:29Z

@xtqqczze rebased on main and squashed into a single commit. The du code is unchanged from the reviewed version; test_du:: passes locally (120 tests).

sylvestre · 2026-10-22T22:00:52Z

        let latest_time = options
            .time
            .and_then(|time| metadata_get_time(&metadata, time));
+        let is_dir = metadata.is_dir();


this local doesn't seem needed anymore, could you please revert this hunk?

emirmrkaljevic · 2026-10-23T06:56:17Z

Reverted that hunk in 30b7ffb, thanks!

github-actions · 2026-10-23T07:08:25Z

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 155.04 MB (-308 KB, -0.19%)

Significant per-binary changes:
  date     2.61 MB ->    2.30 MB  (-316 KB, -11.81%)

…on total line --separate-dirs was leaving subdirectory usage out of the grand total, and --time omitted the timestamp on that line. The total now adds every directory under --separate-dirs (file sizes stay in their parent, so -a does not count them twice), and with --time the total line shows the newest time seen, never moving back before 1970-01-01. The total time and subdirectory sizes are only tracked when --total is given, so other invocations keep the previous code path. Refs uutils#15254 Found with behave (https://github.com/emirmrkaljevic/behave). Co-authored-by: CodSpeed Bot <no-reply@codspeed.io>

xtqqczze reviewed Oct 17, 2026

View reviewed changes

sylvestre reviewed Oct 18, 2026

View reviewed changes

emirmrkaljevic force-pushed the mrki2/du-total-separate-dirs-time-7d60 branch from 329b250 to b371d87 Compare October 18, 2026 13:06

xtqqczze requested a review from sylvestre October 18, 2026 15:47

sylvestre reviewed Oct 22, 2026

View reviewed changes

emirmrkaljevic and others added 2 commits October 25, 2026 10:54

du: drop unneeded is_dir local in Stat::new

939f1e7

sylvestre force-pushed the mrki2/du-total-separate-dirs-time-7d60 branch from 30b7ffb to 939f1e7 Compare October 25, 2026 09:54

Conversation

emirmrkaljevic commented Oct 17, 2026

Summary

Test plan

Uh oh!

codspeed Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

github-actions Bot commented Oct 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sylvestre commented Oct 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emirmrkaljevic commented Oct 18, 2026

Uh oh!

xtqqczze commented Oct 18, 2026

Uh oh!

emirmrkaljevic commented Oct 18, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emirmrkaljevic commented Oct 23, 2026

Uh oh!

github-actions Bot commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codspeed Bot commented Oct 17, 2026 •

edited

Loading

codspeed Bot commented Oct 18, 2026 •

edited

Loading