Repository navigation
Remove dead code; stdout writers are the file writers on another output - #365
Open
BenjaminDEMAILLE wants to merge 17 commits into
Open
BenjaminDEMAILLE wants to merge 17 commits into
BenjaminDEMAILLE wants to merge 17 commits into
Conversation
STAR's trA.nMM is summed while the transcript is built and never recounted. When a junction scan moves the junction back into exon A (jR < 0), STAR counts those bases again against the acceptor without taking back the mismatches A had already counted (stitchAlignToTranscript.cpp, the commented-out nMM--). finalize_transcript now reports the stitched count plus its extensions; the two halves of a pair carry the pair's count once. Fixes nM on chr21 ERR188021.1648513/1662773 and 21 fusion pairs (DIVERGENCE.md 5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STAR's mate loop starts at exons[0], whose EX_iFrag is trOut.Str (ReadAlign_outputTranscriptSAM.cpp, ReadAlign_alignBAM.cpp): mate 1 when it is forward, mate 2 otherwise. rustar-aligner always wrote mate 1 first, about half of STAR's pairs; the per-read multiset comparison of the parity suite did not see it. The transcriptome path already did this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It has no counterpart in STAR, whose window dedup (blocksOverlap in stitchWindowAligns.cpp) compares the whole pair. It collapsed two alignments with the same mate positions but a different indel or splice placement inside mate 1 (fusion fustut000009a03e, fustut000009a03f, fustut0000167ff6). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STAR's assignAlignToWindow recomputes WALrec over the non-anchors of a full window before it looks at the new align; with none it sets MARKER_TOO_MANY_ANCHORS_PER_WINDOW and nW=0, so the read is 'other'. rustar-aligner skipped an incoming non-anchor instead, so the read could map (chr21 ERR188021.1789477) or count as 'too short' (yeast 1, chr21 17, fusion 73 reads). New integration test in tests/unmapped_buckets.rs; it fails on the previous code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STAR's gEem starts from unique plus uniform counts, zeroes values below 0.01 at the start of each round, and stops when no gene moves by 0.01 or after 101 rounds, so it can keep a value below 0.01 (SoloFeature_collapseUMIall.cpp). The UniqueAndMult matrices also list every gene of the cell's unique or multi-gene counts, a zero included (SoloFeature_outputResults.cpp). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STAR records them with their whitelist candidates and resolves the barcode at counting time, as for a unique-gene read (SoloReadFeature_inputRecords.cpp); one it cannot resolve counts as noTooManyWLmatches. rustar-aligner dropped them, which changed GeneFull Features.stats and Summary.csv on the mouse chr19 fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STAR's collapseUMIall filters multi-gene UMIs on every feature. For SJ, a read crossing two junctions is one UMI on two features, a tie MultiGeneUMI_CR drops (4 UMIs on the mouse chr19 fixture). The gene path's filter-then-dedup is now shared by both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DIVERGENCE.md 5, the README comparison tables and residuals, and CHANGELOG. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SpliceJunctionDb is only looked up (is_annotated), never iterated, so the hasher has no effect on output. It is consulted on every junction stitch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every leaf of the recursion compared itself with each transcript already recorded in the window, recomputing their final score (a log2) and mapped length each time. They are now computed once, when the transcript is recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
score1 changes by at most +1 per position, so when score1 plus the positions left plus the best motif score cannot beat the best score so far, the remaining positions cannot be chosen. Ties count for the reverse-strand deletion case only, as in the scan itself. The chosen position, motif and repeat lengths are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Within one window the read, strand and mismatch limits are fixed, so an extension is determined by its start, direction, length cap and the transcript's mismatches and length so far; the recursion's leaves repeat many of them. Windows with fewer than 25 seeds skip the memo, where hashing cost more than it saved (+1.7% instructions on chr21 without the threshold). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exact-duplicate filters (SE transcripts, PE pairs) formatted every CIGAR into a String and hashed it with SipHash. They now key on the positions with FxHashMap and compare the CIGAR ops of transcripts at the same position; the first occurrence is kept, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
stitch_recurse used to clone the working transcript at every include branch (and stitch_align_to_transcript cloned it again), so the exclude branch could keep the original. It now stitches in place on a single transcript and undoes the change afterwards: a stitch only pushes exons and junctions, changes the last exon and updates scalar fields, and a leaf's extensions also change the first exon, so a small WtMark restores it. A leaf takes its mark only when an extension applies, and a recorded transcript is cloned once, when it enters the window list. stitch_align_to_transcript remains, for tests, as a copy-then-stitch wrapper; ExonBlock is now Copy. Output is unchanged: alignments, SJ.out.tab, Chimeric.out.junction, Log.final.out and Solo.out are identical on the yeast, chr21, fusion and STARsolo runs, and the full fusion set still matches STAR 2.7.11b. Single-threaded instructions: fusion 30k pairs -22%, yeast 100k PE -2.7%, chr21 100k SE unchanged (+0.4%, within noise). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 2 of find_best_junction_position read the read base, the upstream and downstream genome bases and the four motif bases position by position, through the genome view. It now copies them into stack buffers first (SeqView::bases_into, one copy per run; the reverse strand's donor run is read backwards), computes the score1 steps in one branch-free pass, and scans the arrays: same positions, same order, same tie rules, same early stop. Positions off the read or the genome become the out-of-range sentinel, which every test there treats as STAR's skip. Scans longer than 512 positions, or whose coordinates would leave the genome, take the position-by-position path, kept as scan_phase2_scalar. buffered_junction_scan_matches_the_scalar_one checks the two on 20,000 random scans (both strands, deletions and introns, N bases, genome ends). Output is unchanged on the yeast, chr21, fusion and STARsolo runs, and the full fusion set and chr21 PE still match STAR 2.7.11b. Single-threaded instructions: chr21 100k SE -16%, yeast 100k PE -13%, fusion -1%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The extension memo added 4 to 6 MB of private memory on read sets with many large windows, though no table exceeded 4,191 entries (about 500 KB). Creating and freeing a table per window is the likely cause, through heap fragmentation. The tables now live for the thread and are cleared between windows, and an extension entry takes 40 bytes instead of 64. As the extension table now outlives the window, it stops storing at 14,336 entries. Fusion PE, 150k pairs, 8 threads: 51 -> 39 MB of private memory (44 MB before the memo), with 0.7 % fewer instructions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removes, each verified unused in production: - the stitcher's scoreSeedBest block, ending in `let _ = best_pre_score`, and the `pre_ext_score` field it filled - the reverse-strand branches of the stitcher and the junction scan: `stitch_cluster.is_reverse` is always false, so `is_reverse` / `n_genome` / `genome_offset` go from those functions - BufferedBamRecords, SamWriter::write_alignment, Seed::find_paired_seeds, Seed::get_genome_positions, combined_transcript_for_projection, parse_gtf, extract_junctions_from_exons, score_gap_with_strand, word_comp_length - the transcriptome/sjdb file tail duplicated in index/mod.rs SamStdoutWriter, BamStdoutWriter and SortedBamStdoutWriter are replaced by SamWriter<W>, BamWriter<W> and SortedBamWriter::create_stdout. --outStd SAM gains the file writer's [SAM-MISMATCH] check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 3, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #354 (finding 20).
Stacked on #346.
perf/hotpathsis on a fork, so GitHub cannot use it as the base of a PR here; the base ismain, and until #346 lands the diff shows its commits too. Only the last commit (bcddd31) is this PR: review withgit diff aeba641 bcddd31. Retarget or rebase after #346 merges.Problem
The tree carried code nothing runs. One block of it cost time: the stitcher scored every seed before extension for every window, then discarded the result with
let _ = best_pre_score. The stdout writers were copies of the file writers that had drifted:SamStdoutWriterlacked the[SAM-MISMATCH]check.Change
12 files, 265 insertions, 1056 deletions. Each item was grepped across
src,tests,examples,benchesandscripts, then confirmed by the compiler once removed.let _ = best_pre_score, and thepre_ext_scorefield it filledstitch_seeds_corereturnsstitch_clusterwithis_reverse: false, the only clusterstitch_recurse,stitch_align_in_placeandfinalize_transcriptreceive.is_reversegoes fromextend_alignment(_cached),score_region,count_mismatches_in_regionfind_best_junction_position_cachedis called only fromstitch.rs, always forward.is_reverse,n_genome,genome_offsetgo fromfind_best_junction_position(_cached)andScanArgs;score_gap_with_strandmerged intoscore_gap.examples/jrbench.rspassed these arguments and is updatedBufferedBamRecords,SamWriter::write_alignmentand its testtest_sam_output_includes_rg_header_and_tagusedwrite_alignment; it now usesbuild_unmapped_recordandwrite_batch, and still checks the RG header and tagSeed::find_paired_seeds,Seed::get_genome_positions,combined_transcript_for_projectionparse_gtf,extract_junctions_from_exonsparse_gtf_configuredandextract_junctions_configuredword_comp_lengthsrc/index/mod.rswrite_transcript_tablesandwrite_sjdb_filesSamStdoutWriter,BamStdoutWriter,SortedBamStdoutWriterSamWriter<W>,BamWriter<W>(default type parameter is the file writer, so existing call sites are unchanged) andSortedBamWriter::create_stdoutEvery item was dead in production; none had to be kept.
stitch_is_reverseandoriginal_is_reverse, which carry the read's real strand, are untouched.These were
pubitems of the library crate, so this is an API removal, marked inCHANGELOG.md.One intended difference:
--outStd SAMnow has the file writer's[SAM-MISMATCH]check. No run below triggers that check, so the check itself on stdout is not exercised; the output of those runs is identical.Conflict with #355
git merge-treereports conflicts withfix/output-completeness(#355) inCHANGELOG.md,src/io/sam.rsandsrc/lib.rs. This branch does not addfinish, because #355 has not landed and this PR changes no behaviour. Whichever lands second must keep the flush: #355'sSamWriter::finishgoes on the genericimpl<W> SamWriter<W>, theAlignmentWriterimpl onimpl<W> AlignmentWriter for SamWriter<W>, and itsSamStdoutWriter::finishis dropped along with that type. I did not carry out or test this resolution.Measured
macOS, aarch64, 16 cores. Baseline: clean build of
aeba641. Comparison:bcddd31.Output. 84 scenarios, each at
--runThreadN1, 4 and 8, every binary under the sameargv[0]and relative prefix so that command lines in headers match. md5 of every output file, raw BAM and SAM included, plus decoded BAM records andLog.final.outwithout its clock lines.Log.out,Log.progress.outand stderr are not in the md5s.The 4 rows are
Solo.out/{Gene,GeneFull}/CellReads.statsat 4 and 8 threads in one scenario. This is existing behaviour: two runs of the baseline differ from each other in the same file, with the same md5 once sorted (64136fbd), so only the row order changes between runs. At 1 thread the file is identical. Not touched here.--outStdin all three forms,Within, TranscriptomeSAM, GeneCounts,Fastx, chimeric junctions andWithinBAM, BySJout, fixed clip, clip to empty, adapter, CellRanger4,bedGraph, WASP, splicing tag, per-file RG, NoQS,None, 2-pass, read limitCB_samTagOut, clip, CR4, RG,None--soloBarcodeMate 1, half-mapped,CB_samTagOut, clip, RG,NoneThe fixture was checked to reach these paths on the baseline: SE 15 965 unique, 1614 multi, 1417 too short, 1003 other, of 20 000 reads; a second fixture gives 2151 (SE) and 2152 (PE) chimeric reads, with as many supplementary records under
WithinBAM; relaxed--outFilter*OverLreadgives 92 records of half-mapped pairs in PE and in solo paired.Scenarios the program refuses:
sp_tag_m(one invalid parameter combination of mine, all 3 thread counts) and 5 of the 9 earlier scenarios (input files the reader rejects; the 3 I compared give the same error text). Same exit code 1 on both binaries; they prove little beyond that.Exit codes, decoded records and non-header bytes are also identical across 1, 4 and 8 threads within each binary, apart from the same
CellReads.statsrows.Speed.
--runThreadN 8. The machine was not idle: unrelatedbwa-mem2andsalmonjobs were running, so wall time is noisy (the same binary varies between 4 and 16 s), and I compare instructions retired and CPU time. From/usr/bin/time -l, binaries alternated.B200.fq, 801 000 reads, 176 MBInstructions: 3 runs for SE and solo, 1 run for PE and solo paired. The reads are synthetic and short; expect a different figure on real data.
Tests
cargo test --releasepasses (740 library tests and all integration tests,tests/determinism.rsandtests/phase9_threading.rsincluded).cargo clippy --all-targetshas no warnings;cargo fmtapplied. No test was added: this removes code.Not checked
test/parity_suite.sh, and did not compare against STAR here; the claim is "same output asaeba641".docs-oldstill namespre_ext_scoreandget_genome_positions; it is historic and left alone.Also dead but not named in the finding, left in place:
AlignmentScorer::score_gapis called only by tests, andstitch_seeds/stitch_seeds_with_jdbhave no production caller beyond a re-export and each other.🤖 Generated with Claude Code