diff --git a/claude.md b/claude.md index 5b08da7f..57969e43 100644 --- a/claude.md +++ b/claude.md @@ -145,12 +145,25 @@ patching. `InlineApplier.ApplyAll` takes patches together: each file is read onc applied in memory in the order given, each to what the one before it left, and it is written once through the same temporary and swap, with the file's lock and mutex held from the read to the write. Every patch is told what `Apply` would have told it in turn, and `Apply` is the one-patch -case of the same code. One thing can only differ: a write that fails fails every patch from the -first edit on. Both batches use it, a file at a time (`AcceptBatch.Together` in the viewer, +case of the same code. One thing can only differ: a write that fails fails every patch that +edited, and any other that would have had to edit the file as it was read, while the rest keep +what is true of the file. Both batches use it, a file at a time (`AcceptBatch.Together` in the viewer, `OwnedInlineHost.AcceptEvery` in the tray), so the moment up to which a snapshot can still be withdrawn from a bulk accept is its file's turn rather than its own. A `SourceScan` rents its map from the pool and is disposed for that reason, and keeps its spans as sorted lists rather than -hash tables: it is built again for every patch, over the whole file. +hash tables. A batch lexes a file once and carries the scan from one editing patch to the next +(`SourceScan.Edited`): lexing starts again at the edit's line, never at a line that follows a +backslash, and rejoins the old scan at the first line start both have as code. `SourceScanTests` +holds it to a whole scan over random edits, and a lexer change that looks across a line break in +a new way has to extend `RestartFor`. `CanAnchor` and `CanApply` are answered from the last dry +run's scan while the file's length and write time stand, with no lock, for a file that had been +left alone three seconds when it was read. + +Two more shapes of what the patcher writes. A statement a Remove takes whole keeps its line, +empty, over a Snapshot call, as a chained call's does. An Append whose recorded line names no +call is refused, to be run again, when more than one call in the member could take it, and +passes over a call given a variable that `Snapshot` is called on: taking the first was the wrong +call half the time, silently. The holes of an F# interpolated string are lexed as code. A passing inline verification clears its staged trio (`InlineStaging.Clear`), which walked the project's whole `obj` tree to find the `VerifyInline` directories. The list of those is now kept a @@ -166,7 +179,7 @@ the comment there about not caching "nothing staged" asks for. - `Definitions` - Static collection of all supported diff tool definitions. Each tool is defined in `Implementation/` folder. - `Definition` - Record type describing a diff tool: executable paths, command arguments, supported extensions, OS support, MDI behavior, auto-refresh capability. - `DiffTool` - Enum of all supported diff tools (BeyondCompare, P4Merge, VS Code, etc.) -- `TextDiff` (`TextDiff/`) - The line diff behind a failure message and behind every text pair the viewer shows, which links these files. Myers in linear space over line ids, with three things in front of the textbook. Lines only one side has are marked changed and taken out first (`LineDiff.DiffShared`), since they cannot be unchanged and Myers costs by edits: a re-indented snapshot was all edits, four seconds for 40,000 lines. A diff has a budget of searching (`MyersDiff.Budget`, about a sixth of a second), counted in work rather than lines so that nothing quick is given up on for being long; past it a search settles for a split, which is still a correct diff and may not be the smallest. And a search that settles having passed nothing looks for where the start of each side is in the other (`TryDisplaced`), because a block moved further than the search went lines up on a diagonal it never reached. Up to 10,000 lines between the two sides a diff is always minimal. +- `TextDiff` (`TextDiff/`) - The line diff behind a failure message and behind every text pair the viewer shows, which links these files. Myers in linear space over line ids, with three things in front of the textbook. Lines only one side has are marked changed and taken out first (`LineDiff.DiffShared`), since they cannot be unchanged and Myers costs by edits: a re-indented snapshot was all edits, four seconds for 40,000 lines. A diff has a budget of searching (`MyersDiff.Budget`, about a sixth of a second), counted in work rather than lines so that nothing quick is given up on for being long; past it a search settles for a split, which is still a correct diff and may not be the smallest. And a search that settles having passed nothing looks for where the start of each side is in the other (`TryDisplaced`), because a block moved further than the search went lines up on a diagonal it never reached. Up to 10,000 lines between the two sides a diff is always minimal. Past the budget, a search that settles first asks `LineAnchors` for the longest run still in order of the lines each side has once, and splits at all of them, so the same lines in another order keep that run rather than whatever the search had reached; texts made mostly of repeated lines have few such lines and gain little. - `ResolvedTool` - A diff tool that was found on the system with its resolved executable path. - `BuildServerDetector` - Detects CI/build server environments to disable diff tool launching. @@ -190,7 +203,11 @@ the comment there about not caching "nothing staged" asks for. replaced, so the same reference is the same screen, and building one a frame was the whole queue's labels and tooltips, megabytes a second, from a window nobody was touching. Handing a head the same `Screen` is also how it learns nothing changed, with no comparison: the WinForms - head and `ScreenPayload` both stop at the reference. So anything a screen depends on has to be + head and `ScreenPayload` both stop at the reference. The queue column is walked to slots that say only where each row is, and the slice that fits + is what is described; labels are decided over the whole queue once and kept against its list, + so a list is never changed after it is put in a state. `ScreenPayload` cuts a row at + `Screen.PaneCells`, half the window and a cell, which holds while both native heads split the + panes equally. So anything a screen depends on has to be in the state. A spinner or a picture landing is a head's own business and redraws on its own. - Windows renders with **WinForms** and loads no native library. It is pumped through `Application.DoEvents` rather than `Application.Run`, so the shared loop stays shared. Only the @@ -207,7 +224,11 @@ the comment there about not caching "nothing staged" asks for. own loop for every part of the scroll bar, not the thumb alone, and `Present` stops that timer. An Alt chord is no command. A held key gives one accept or discard (the repeat flag, read in `ProcessCmdKey`, for what `ViewerSession.ChangesQueue` names), while navigation keeps its - repeats. A minimised window waits between frames as a hidden one does. + repeats. A minimised window waits between frames as a hidden one does. The footer wraps its buttons, and + gives a status with no room beside them lines of its own under them (`LayOutFooter`), as the + other two heads do; the buttons are the footer's own children, and a taller footer comes off the + canvas, which reports the rows it has left. A decoded picture is kept premultiplied, as the + buffer it is painted into is, which took a paint between half and full size from 15 ms to 9. - macOS renders with **AppKit and Core Text** (`native/swift/`), Linux with **raylib and Dear ImGui** (`native/`). Both implement the same C ABI, so the managed interop layer is identical. - macOS took the same treatment as Windows: a real menu bar, an `NSMenu` context menu, `NSView` @@ -238,7 +259,10 @@ the comment there about not caching "nothing staged" asks for. side is not told. A wheel is told from a trackpad by `hasPreciseScrollingDeltas`, and a control-click is taken in `mouseDown`, since the view has no `NSMenu` for AppKit to ask for. A drag reports the frame's own centre on an axis its picture cannot move on, as the Linux head - does: only when the space is what cut the picture short can it move. + does: only when the space is what cut the picture short can it move. A repeat of a held key is + dropped for accept, accept all and discard. A picture both panes name keeps a scaled copy for + each pane (`Picture.spare`), since the panes can be a point apart in width and one copy was + made again for each in turn without end. - Linux draws its own menu, so it keeps that baseline. Its tooltip and pane scrollbar are ImGui's, the scrollbar being `ScrollbarEx` driven in rows rather than pixels so its travel is exactly `ViewerSession`'s clamp. Its footer wraps as the macOS one does (`LayOutFooter`), with two @@ -273,7 +297,15 @@ the comment there about not caching "nothing staged" asks for. under half size, and one past `GL_MAX_TEXTURE_SIZE` is not drawn. The window is scaled by GLFW's content scale (`Xft.dpi` over 96), read once and without `FLAG_WINDOW_HIGHDPI`; a capture never is, and `MeasureGrid` reports the cells of the last window frame. An Xvfb used to check a layout - or a scale needs `-noreset`, or the setting is gone before the viewer connects. + or a scale needs `-noreset`, or the setting is gone before the viewer connects. A held key + repeats only navigation (`Repeats`); accept, discard and the rest act once. A hidden window is + neither built nor drawn: it still takes in decodes and fonts, and the first present after it is + shown draws. Sampling is set by GL filter directly, because raylib's bilinear on a texture with + mipmaps is `GL_LINEAR_MIPMAP_NEAREST` and works out a level for every pixel: plain linear from + three quarters of a picture's size up, and mipmaps made only for a picture drawn smaller than + that, on the decoder thread when first needed. A picture past `GL_MAX_TEXTURE_SIZE` is brought + down to one that fits as it is read. `PixelTests` has two tests that show the window and count + draws through a GL query, `AWindowLeftAloneIsNotDrawn` and `AHiddenWindowIsNotDrawn`. - Group headers fold. `SessionState.Collapsed` holds `QueueItem.GroupKey`s and `QueueProjection` skips their members, so the marker rides in the label and no head or ABI field knows about it. Whether an entry is hidden is always read back out of `VisibleEntries`, never recomputed — the @@ -296,9 +328,14 @@ the comment there about not caching "nothing staged" asks for. (`BeginAcceptGroup`, `AcceptBatch.Only`), not a transition of its own: in a queue of one solution the header's group is the whole queue. So it goes by the batch's rules, a snapshot the applier would not take staying in the queue with what the applier said, and it counts as still needing - review only its own members. Only the bulk discards are still one transition, since a discard - waits on nothing. A batch's record step is the one inline transition that does not rebuild the - list from the queue: it asks `InlineQueue.AcceptInBatch` of a queue holding the claimed entry + review only its own members. The bulk discards are the same batch with `Discarding` set: + snapshots and pending deletes go as it begins, since nothing of theirs is on disk, and each + move's received file is thrown away outside the lock. A discard under way is not on an owner's + listings. No inline transition rebuilds the whole list any more. An arrival, a settle, a + discard and a single accept read what changed off the `InlineQueue` before and after, whose + untouched items come back as the same instances, and edit the list where it stands + (`TryRebuildChanged`), with `RebuildWhole` as the fallback and as what the tests hold it to. + A batch's record step asks `InlineQueue.AcceptInBatch` of a queue holding the claimed entry alone and takes that entry out of the list, or marks it, where it stands. Rebuilt an entry, the bookkeeping grew with the square of the queue, seconds and gigabytes for 2,000 snapshots. For the same reason which entries are visible is found by `QueueProjection`'s one walk without @@ -512,7 +549,12 @@ the comment there about not caching "nothing staged" asks for. offer. An older owner never says the line and gets a connection each, an older client skips it, and a send that finds its connection gone falls back to the ordinary exchange. So a field can only ever be added as a new line. `ViewerServer.Stop` closes kept connections. -- `ViewerServer.Serve` is the accept loop with its accept handed in. A failed accept is retried +- `ViewerServer.Serve` is the accept loop with its accept handed in. Listings have a kept connection of their own, so a slow one never stands in front of a settle, + and a listing never waits for it: with another in flight it takes a connection of its own. + Sends that change the queue stay a connection each, since one written to an owner that had + just gone would be sent again with nothing to say whether it was acted on. On .NET Framework + the kept sockets have their inherit flag cleared, or a child process kept one open after the + test host had gone. A failed accept is retried at once, and from the second in a row after 100 ms: with no descriptors left a failure persists, and the loop took a core. A send whose caller cancelled throws `OperationCanceledException` and records nothing about the port. @@ -638,7 +680,17 @@ the comment there about not caching "nothing staged" asks for. payload's `Exe` (`ProcessEx.TryGetTool`): a library from before `ProcessCleanup.StillRunning` can send an id that has since been reused, and the tray would kill whatever holds it. A tool started through a `.cmd`, or one whose image cannot be read, is tracked with no process. A - move lets go of its process wherever it leaves for good (`Tracker.Release`). + move lets go of its process wherever it leaves for good (`Tracker.Release`). The image alone + does not rule out another copy of the same tool holding a reused id, so the process's command + line has to name the received file as well (`ProcessEx.WasStartedFor`): every definition + passes that path as it was given. A tool started through a script, or one that hands over to + another process and exits, cannot be tracked from here at all. +- A move that writes a file marks the delete pending on it (`TrackedDelete.Written`), however + the move was accepted, and no accept-all carries a marked delete out until a run raises it + again; accepting it on its own still does. `Tracker.HeldReason` is what the menu and the debug + view show. "Discard (n)" runs wholly on a worker. A scan that fails three times running is one + balloon for the run. Every tray keeps the `SessionEndWindow` now, and `Program.SessionEnding` + stages the queue where it is held here and then removes the version marker. - `ITrackedFiles.Version` is what `ListingTag` asks about the tracked files: the identity of the tracked objects, so `TrackedMove` and `TrackedDelete` must stay immutable in everything a listing carries. The scan removes a move by key and value, so one staged again since it looked @@ -674,10 +726,11 @@ the comment there about not caching "nothing staged" asks for. - Tool discovery uses wildcard path matching (`WildcardFileFinder`) to find executables in common install locations. A wildcard whose matches are all version-named folders takes the highest version; anything else takes the most recently written - Tool order can be customized via `DiffEngine_ToolOrder` environment variable - `DisabledChecker` respects `DiffEngine_Disabled` env var -- `ViewerClient` remembers a port found unowned for ten minutes (`RecheckUnownedAfter`), and the library's telling sends - settle, retire, move, delete, the first inline or diff send - skip the connect while that stands. A refused loopback connection costs two seconds on Windows (firewall stealth mode drops the reset), and a green run settles once per inline verification, which was six minutes for a class of 188 inline tests. Probes (`IsOwned`), the hosts and `InlineQueueClient` always ask and correct the memory; so does `SettleAppliedInline`, being one send per accept. Asking, on Windows, is the operating system's listener table first (`ListenerTable`, shared with `PiperClient`): no listener on the port means nobody to connect to, said without the two seconds, and a listener or a table that cannot be read leaves the connect to answer. So the first telling send of a test process, the launch gate's probe and each of its polls no longer wait to be refused. By port alone, whichever address, since the table is only believed when it says nobody is there. Not for a port that accepted a connection in the last second (`TrustOwnerFor`), because reading the table is reading every connection the machine has, and a run of settles to a live owner would pay more for each than the connect costs +- `ViewerClient` remembers a port found unowned for ten minutes (`RecheckUnownedAfter`), and the library's telling sends - settle, retire, move, delete, the first inline or diff send - skip the connect while that stands. A refused loopback connection costs two seconds on Windows (firewall stealth mode drops the reset), and a green run settles once per inline verification, which was six minutes for a class of 188 inline tests. Probes (`IsOwned`), the hosts and `InlineQueueClient` always ask and correct the memory; so does `SettleAppliedInline`, being one send per accept. Asking, on Windows, is the operating system's listener table first (`ListenerTable`, shared with `PiperClient`): no listener on the port means nobody to connect to, said without the two seconds, and a listener or a table that cannot be read leaves the connect to answer. So the first telling send of a test process, the launch gate's probe and each of its polls no longer wait to be refused. By port alone, whichever address, since the table is only believed when it says nobody is there. Not for a port that accepted a connection in the last second (`TrustOwnerFor`), because reading the table is reading every connection the machine has, and a run of settles to a live owner would pay more for each than the connect costs. The table is Windows' listeners alone, by `GetExtendedTcpTable` (`ListenerTable.Listeners`), falling back to .NET's list of every connection where that call is not there: 0.32 ms beside 3,000 connections where it was 13. What the table found empty is asked about again after a second (`RecheckUnlistedAfter`), so a tray started after a test process is found; the ten minutes is only for what a connect found. And only a refusal or an unanswered connect is remembered as nobody being there (`NobodyThere`). A viewer launch that failed gives its `MaxInstance` slot back - `TrayDisabledChecker` respects `DiffEngine_TrayDisabled` env var, behind `DiffRunner.TrayDisabled`. Separate from `Disabled` because tracking a pending move is separate from launching a tool: every exit of `InnerLaunch`, `Disabled` included, still calls `AddMove`. `PendingFiles.TrayAvailable` is the single gate - Tests use TUnit and Verify for snapshot testing -- The three WinForms test and benchmark hosts set `UnhandledExceptionMode.ThrowException` from a module initializer with `threadScope: false`. The one argument overload covers only the thread that calls it, which leaves every test thread showing WinForms' dialog on the desktop of whoever is running the tests, and waiting for a click. Never run a test that throws inside a window message in a build without it. `DiffEngineTray.Tests` also declines the tray's "open an issue" box (`IssueLauncher.Declined`) and records what was asked in `ModuleInitializer.IssuesAsked` +- The three WinForms test and benchmark hosts set `UnhandledExceptionMode.ThrowException` from a module initializer with `threadScope: false`. The one argument overload covers only the thread that calls it, which leaves every test thread showing WinForms' dialog on the desktop of whoever is running the tests, and waiting for a click. Never run a test that throws inside a window message in a build without it. `DiffEngineTray.Tests` also declines the tray's "open an issue" box (`IssueLauncher.Declined`) and records what was asked in `ModuleInitializer.IssuesAsked`. A form shown the ordinary way is the foreground window even parked off screen, and takes the keyboard of whoever is at the machine, so a test that posts input shows its form without activating it (`ViewerForm.Parked`, `ParkedForm`) and says what modifiers its thread holds (`ThreadKeys`) +- CI's runners have two to four cores. A test must not block pool threads while something in its own process needs the pool to answer it, nor assert that something happened inside a fixed short wait: `DOTNET_PROCESSOR_COUNT=2` shows both locally - `MachineSettings.Ignore` clears `DiffEngine_Disabled`, so a snapshot test that is expected to fail is run with a build server variable such as `TEAMCITY_VERSION` set, or Verify tells the real tray about it - `ps` is asked with `-ww`: procps lets an exported `COLUMNS` cut every command line `ProcessCleanup` reads - The native pixel snapshots (`PixelTests`) are opt in through `DIFFENGINE_VIEWER_PIXEL_TESTS`, which `MachineSettings.Ignore` has to leave alone: it clears every `DiffEngine_*` variable without regard to case, and clearing that one skipped them on the CI job that sets it, silently, for as long as nobody looked. Every call into the shim goes through one thread (`OnShimThread`), because on Linux the window's GL context belongs to the thread that made it and each test starts on whichever pool thread picks it up. The Linux baselines reproduce in an `ubuntu:24.04` container set up as the `unix` job in `build.yml` is - the shim built from source, Xvfb, llvmpipe - which is also the only way to run the C++ at all from Windows diff --git a/docs/mdsource/tray.source.md b/docs/mdsource/tray.source.md index 25a1e4f4..ccf4ba07 100644 --- a/docs/mdsource/tray.source.md +++ b/docs/mdsource/tray.source.md @@ -63,7 +63,7 @@ Exiting the tray writes any still-pending inline snapshots back to disk, under t "Accept all" will accept all pending moves, deletes and inline snapshots. Snapshots whose target frameworks disagree about the content are skipped rather than picked between; resolve those in the viewer. -The deletes it carries out are the ones that were pending when it began. A delete can be the last copy of a snapshot that is moving inline, so the deletes are held back, and the tray says so, when a snapshot could not be written or when a viewer that owns the queue did not answer. A delete of a file that a move in the same accept has written is left pending rather than carried out. +The deletes it carries out are the ones that were pending when it began. A delete can be the last copy of a snapshot that is moving inline, so the deletes are held back, and the tray says so, when a snapshot could not be written or when a viewer that owns the queue did not answer. A delete of a file that an accepted move has written is left pending rather than carried out, and stays that way through later "Accept all"s: it is marked `!` in the menu, with the reason. It goes when it is accepted on its own, or when a test run raises the delete again. A long queue takes a while to accept. An open [DiffEngineViewer](/docs/viewer.md) window shows how far it has got, with each snapshot leaving the list as it lands. diff --git a/docs/tray.md b/docs/tray.md index d5559361..b0ac0f29 100644 --- a/docs/tray.md +++ b/docs/tray.md @@ -70,7 +70,7 @@ Exiting the tray writes any still-pending inline snapshots back to disk, under t "Accept all" will accept all pending moves, deletes and inline snapshots. Snapshots whose target frameworks disagree about the content are skipped rather than picked between; resolve those in the viewer. -The deletes it carries out are the ones that were pending when it began. A delete can be the last copy of a snapshot that is moving inline, so the deletes are held back, and the tray says so, when a snapshot could not be written or when a viewer that owns the queue did not answer. A delete of a file that a move in the same accept has written is left pending rather than carried out. +The deletes it carries out are the ones that were pending when it began. A delete can be the last copy of a snapshot that is moving inline, so the deletes are held back, and the tray says so, when a snapshot could not be written or when a viewer that owns the queue did not answer. A delete of a file that an accepted move has written is left pending rather than carried out, and stays that way through later "Accept all"s: it is marked `!` in the menu, with the reason. It goes when it is accepted on its own, or when a test run raises the delete again. A long queue takes a while to accept. An open [DiffEngineViewer](/docs/viewer.md) window shows how far it has got, with each snapshot leaving the list as it lands. diff --git a/native/src/deview.cpp b/native/src/deview.cpp index 0d683521..2011c588 100644 --- a/native/src/deview.cpp +++ b/native/src/deview.cpp @@ -217,18 +217,31 @@ struct CachedTexture /* Some of it can be seen through, so it is drawn over a checkerboard: see SeeThrough. */ bool translucent = false; + + /* + * A texture is made with reduced copies of the picture only for a picture that is drawn at + * under three quarters of its size, which is the only one sampled from them (see WantsCopies): + * they are a third again of a picture's memory, and a picture that is no larger than its pane + * is never drawn that small. One that comes to be, in a + * window made smaller, is decoded a second time with them while the texture it has goes on + * being drawn, which is what `reducing` says is on its way. And `unreducible` is a picture + * that came back from that without them, which is not asked again. See Reduce. + */ + bool reducing = false; + bool unreducible = false; }; /* - * One picture to decode, or decoded: the path, the stamp the decode was asked for, and once it is - * done the pixels, which are empty when raylib could not read the file, and whether any of them - * can be seen through. + * One picture to decode, or decoded: the path, the stamp the decode was asked for, whether its + * reduced copies are wanted with it, and once it is done the pixels, which are empty when raylib + * could not read the file, and whether any of them can be seen through. */ struct Decode { std::string path; std::uintmax_t length = 0; std::filesystem::file_time_type written{}; + bool reduced = false; Image image{}; bool translucent = false; }; @@ -893,7 +906,7 @@ char RowMarker(int kind) /* ---- pictures ---- */ /* - * The three ways a picture's texture is sampled, by the size it is drawn at. + * The four ways a picture's texture is sampled, by the size it is drawn at. * * Smoothed between its own pixels, which is all a picture drawn at its own size or a little under * needs. As the pixels it has, once it is enlarged past its own size, which is what zooming that @@ -904,10 +917,54 @@ char RowMarker(int kind) * them, and its small text came apart. The reduced copies are each half the size of the one * before, every pixel of them an average of the ones it stands for, so a thin line is fainter * there and not gone. + * + * And from whichever one of those copies is nearest in size, between half its size and three + * quarters. Under about seven tenths that is the half size copy, in which a grid of one pixel + * lines is even, where smoothed between its own pixels at two thirds of its size some of its lines + * are dark and some faint. From seven tenths up it is the picture itself, and the same pixels as + * smoothed. + * + * The last is what raylib makes of "bilinear" for any texture that has reduced copies, and every + * picture not under half its size was sampled that way, at its own size as well. Which copy is + * worked out for every pixel drawn, and where the answer is always the picture itself that was a + * tenth of a frame of two pictures under a software rasteriser, for nothing. So smoothed is set + * here by the filters themselves, and is asked for from three quarters up: clear of the seven + * tenths where the answer changes, so what is drawn is what was drawn at every size. */ constexpr int sampleSmoothed = 0; constexpr int sampleAsPixels = 1; constexpr int sampleReduced = 2; +constexpr int sampleNearestCopy = 3; + +/* Whether a picture drawn so wide is sampled from its reduced copies, either way, and so wants a + * texture that has them: under three quarters of the width it has. */ +bool WantsCopies(float drawnWidth, float ownWidth) +{ + return drawnWidth * 4.0f < ownWidth * 3.0f; +} + +int SamplingOf(float drawnWidth, float ownWidth, bool enlarged) +{ + return enlarged && drawnWidth >= ownWidth ? sampleAsPixels : + drawnWidth * 2.0f < ownWidth ? sampleReduced : + WantsCopies(drawnWidth, ownWidth) ? sampleNearestCopy : + sampleSmoothed; +} + +void Filter(const Texture2D& texture, int sampling) +{ + rlTextureParameters( + texture.id, + RL_TEXTURE_MIN_FILTER, + sampling == sampleAsPixels ? RL_TEXTURE_FILTER_NEAREST : + sampling == sampleReduced ? RL_TEXTURE_FILTER_MIP_LINEAR : + sampling == sampleNearestCopy ? RL_TEXTURE_FILTER_LINEAR_MIP_NEAREST : + RL_TEXTURE_FILTER_LINEAR); + rlTextureParameters( + texture.id, + RL_TEXTURE_MAG_FILTER, + sampling == sampleAsPixels ? RL_TEXTURE_FILTER_NEAREST : RL_TEXTURE_FILTER_LINEAR); +} /* * How a picture's texture is sampled as it is made: smoothed. Clamped at its edges rather than @@ -917,13 +974,14 @@ constexpr int sampleReduced = 2; */ void PrepareTexture(CachedTexture& entry) { - SetTextureFilter(entry.texture, TEXTURE_FILTER_BILINEAR); + Filter(entry.texture, sampleSmoothed); SetTextureWrap(entry.texture, TEXTURE_WRAP_CLAMP); entry.sampling = sampleSmoothed; } /* Changed only when it has to be, since it is a texture parameter and this is asked every - * frame. A picture whose reduced copies could not be made is smoothed instead. */ + * frame. A picture whose reduced copies could not be made, or have yet to land, is smoothed + * instead. */ void Sample(const std::string& path, int sampling) { const auto found = state.pictures.find(path); @@ -934,7 +992,7 @@ void Sample(const std::string& path, int sampling) } CachedTexture& entry = found->second; - if (sampling == sampleReduced && + if ((sampling == sampleReduced || sampling == sampleNearestCopy) && entry.texture.mipmaps <= 1) { sampling = sampleSmoothed; @@ -945,21 +1003,17 @@ void Sample(const std::string& path, int sampling) return; } - SetTextureFilter( - entry.texture, - sampling == sampleAsPixels ? TEXTURE_FILTER_POINT : - sampling == sampleReduced ? TEXTURE_FILTER_TRILINEAR : - TEXTURE_FILTER_BILINEAR); + Filter(entry.texture, sampling); entry.sampling = sampling; } /* * The longest side of a texture the window's GL will take, or zero where it would not say. * - * A picture past it cannot be drawn. Handed to GL all the same, it came back as a texture with a - * name and no pixels, which draws as black: a box of it where the picture should be, in place of - * the nothing a picture this head cannot show is drawn as. It is 16384 under Mesa's software - * rasteriser, and a screenshot of the whole of a long page is past that. + * A picture past it cannot be a texture as it is. Handed to GL all the same, it came back as a + * texture with a name and no pixels, which draws as black: a box of it where the picture should + * be. It is 16384 under Mesa's software rasteriser, and a screenshot of the whole of a long page + * is past that. Such a picture is brought down to the limit as it is read: see FitToATexture. * * Asked of GL by name, through GLFW, since rlgl reads this number only to log it. On the thread * that owns the context, once. @@ -1040,18 +1094,50 @@ bool SeeThrough(const Image& image) return false; } +/* + * A picture longer on a side than a texture can be, resampled to the largest size of its own shape + * that a texture can: every pixel of what is left an average of the ones it stands for, as in a + * reduced copy, so a thin line is fainter there and not gone. + * + * It was drawn as nothing, and a screenshot of the whole of a long page is such a picture. What + * the pane then shows is not its pixels, at any zoom: the rows still give the size it has, and + * enlarged far enough it is the pixels of this copy. + * + * On the decoder's thread, for every picture but a capture's, because of what it costs. The copy + * is made beside the picture as it was read, so for as long as that takes both are held: a + * picture 20,000 pixels square with no alpha channel is 1.2 GB read, and 0.8 GB more for a copy + * 16,384 square before the first is given back. Measured where the limit is that, on one thread: + * that picture took 2.1 s to read and 1.2 s more to bring down, one 17,000 by 9,000 with an + * alpha channel 0.4 s and 0.9 s, and a screenshot 1,920 by 30,000 with one 0.3 s and 0.2 s. + */ +void FitToATexture(Image& image, int limit) +{ + if (image.data == nullptr || + FitsATexture(image, limit)) + { + return; + } + + const double factor = static_cast(limit) / static_cast(std::max(image.width, image.height)); + ImageResize( + &image, + std::min(limit, std::max(1, static_cast(std::lround(image.width * factor)))), + std::min(limit, std::max(1, static_cast(std::lround(image.height * factor))))); +} + /* * A picture read off the disk and made ready to be a texture: whether any of it can be seen - * through, and its reduced copies, which are made here because here is off the window's thread - * for every picture but a capture's. The file, stb_image and raylib's resampling, and nothing - * that touches GL. A picture no texture can hold is left as it was read, since nothing will be - * made of it. + * through, brought down to a size a texture can hold if it is past that, and its reduced copies + * where they are wanted. All of it here because here is off the window's thread for every picture + * but a capture's. The file, stb_image and raylib's resampling, and nothing that touches GL. */ -Image ReadPicture(const std::string& path, int textureLimit, bool& translucent) +Image ReadPicture(const std::string& path, int textureLimit, bool reduced, bool& translucent) { Image image = LoadImage(path.c_str()); translucent = SeeThrough(image); - if (image.data != nullptr && + FitToATexture(image, textureLimit); + if (reduced && + image.data != nullptr && FitsATexture(image, textureLimit)) { ImageMipmaps(&image); @@ -1062,8 +1148,9 @@ Image ReadPicture(const std::string& path, int textureLimit, bool& translucent) /* * The texture for a picture that has been read, or false for one there is none for: a file that - * could not be read, a picture longer on a side than a texture can be, or a context that would - * not make one. On the thread that owns the context. + * could not be read, a picture still longer on a side than a texture can be, which is one read + * against another limit than this context's, or a context that would not make one. On the thread + * that owns the context. */ bool MakeTexture(const Image& image, bool translucent, CachedTexture& entry) { @@ -1086,6 +1173,31 @@ bool MakeTexture(const Image& image, bool translucent, CachedTexture& entry) return true; } +/* + * Puts a texture with reduced copies in the place of the one a picture has, from the picture read + * a second time with them. One that came back without them, or that no texture could be made of, + * leaves the picture with the texture it had, smoothed, and is not asked for again. + */ +bool ReplaceTexture(const Image& image, bool translucent, CachedTexture& entry) +{ + CachedTexture made; + if (image.mipmaps <= 1 || + !MakeTexture(image, translucent, made)) + { + entry.unreducible = true; + return false; + } + + UnloadTexture(entry.texture); + entry.texture = made.texture; + entry.translucent = made.translucent; + entry.sampling = made.sampling; + + /* Another texture behind the picture, which may have been given the name of the last. */ + state.stale = true; + return true; +} + void DecodeLoop(std::shared_ptr decoder) { std::unique_lock lock(decoder->mutex); @@ -1101,7 +1213,7 @@ void DecodeLoop(std::shared_ptr decoder) decoder->requests.pop_front(); lock.unlock(); - decode.image = ReadPicture(decode.path, decoder->textureLimit, decode.translucent); + decode.image = ReadPicture(decode.path, decoder->textureLimit, decode.reduced, decode.translucent); lock.lock(); if (decoder->stopping) @@ -1114,7 +1226,7 @@ void DecodeLoop(std::shared_ptr decoder) } } -void RequestDecode(const std::string& path, std::uintmax_t length, std::filesystem::file_time_type written) +void RequestDecode(const std::string& path, std::uintmax_t length, std::filesystem::file_time_type written, bool reduced) { if (!state.decoder) { @@ -1129,6 +1241,7 @@ void RequestDecode(const std::string& path, std::uintmax_t length, std::filesyst decode.path = path; decode.length = length; decode.written = written; + decode.reduced = reduced; state.decoder->requests.push_back(std::move(decode)); } @@ -1221,6 +1334,16 @@ bool TakeDecoded() state.stale = true; } } + else if (found != state.pictures.end() && + found->second.reducing && + /* Not a decode without them that an entry forgotten since had asked for. */ + decode.reduced && + found->second.written == decode.written && + found->second.length == decode.length) + { + found->second.reducing = false; + landed = ReplaceTexture(decode.image, decode.translucent, found->second) || landed; + } UnloadImage(decode.image); } @@ -1241,7 +1364,8 @@ void ForgetPicture(const std::string& path) UnloadTexture(found->second.texture); } - if (found->second.decoding) + if (found->second.decoding || + found->second.reducing) { CancelDecode(path); } @@ -1290,6 +1414,40 @@ bool PicturesRewritten() return false; } +/* + * Asks for the reduced copies of a picture whose texture was made without them, now that a frame + * draws it small enough to want them. Off the window's thread, as the picture was decoded, with + * the texture it has drawn smoothed until they land: making them here would hold the window for + * as long as a large picture takes to be halved over and over. A capture has no later frame for + * them to land in, and makes them there and then, as it decodes. + */ +void Reduce(const std::string& path, CachedTexture& entry) +{ + if (!entry.loaded || + entry.texture.mipmaps > 1 || + entry.unreducible) + { + return; + } + + if (state.capturing) + { + bool translucent = false; + const Image image = ReadPicture(path, TextureLimit(), true, translucent); + ReplaceTexture(image, translucent, entry); + UnloadImage(image); + return; + } + + if (entry.reducing) + { + return; + } + + entry.reducing = true; + RequestDecode(path, entry.length, entry.written, true); +} + /* * The decoded picture for a path, or null when there is none to draw: either this build cannot * read it, or it is still being decoded, which `loading` says so the pane can show that it is coming @@ -1299,8 +1457,11 @@ bool PicturesRewritten() * Invalidated by the file's write time and length, which is the same freshness test the managed * queue poller uses: a re-run that rewrites a received image has to refresh the pane rather than * leave the previous one up. + * + * `reduced` is whether the frame draws it small enough to want its reduced copies: see + * CachedTexture::reducing. */ -const CachedTexture* Picture(const std::string& path, bool& loading) +const CachedTexture* Picture(const std::string& path, bool reduced, bool& loading) { loading = false; if (path.empty()) @@ -1331,6 +1492,11 @@ const CachedTexture* Picture(const std::string& path, bool& loading) !(found->second.decoding && state.capturing)) { found->second.used = true; + if (reduced) + { + Reduce(path, found->second); + } + loading = found->second.decoding; return found->second.loaded ? &found->second : nullptr; } @@ -1346,7 +1512,7 @@ const CachedTexture* Picture(const std::string& path, bool& loading) { /* What the decoder's thread and then TakeDecoded do, here and now. */ bool translucent = false; - const Image image = ReadPicture(path, TextureLimit(), translucent); + const Image image = ReadPicture(path, TextureLimit(), reduced, translucent); MakeTexture(image, translucent, entry); UnloadImage(image); } @@ -1354,7 +1520,7 @@ const CachedTexture* Picture(const std::string& path, bool& loading) { entry.decoding = true; loading = true; - RequestDecode(path, length, written); + RequestDecode(path, length, written, reduced); } const auto inserted = state.pictures.emplace(path, entry).first; @@ -1385,7 +1551,8 @@ void ForgetUnusedPictures() UnloadTexture(entry->second.texture); } - if (entry->second.decoding) + if (entry->second.decoding || + entry->second.reducing) { CancelDecode(entry->first); } @@ -2193,6 +2360,22 @@ bool LayoutTypes(unsigned int letter) return false; } +/* Whether the layout in use has a key that types any of the letters a to z, unshifted. */ +bool LayoutIsLatin() +{ + for (int key = KEY_APOSTROPHE; key <= 162; key++) + { + const unsigned int typed = OnlyCharacter(glfwGetKeyName(key, 0)); + if (typed >= 'a' && + typed <= 'z') + { + return true; + } + } + + return false; +} + /* * Which character a key press is to be read as, in lower case, or zero for none. * @@ -2237,12 +2420,64 @@ unsigned int LetterOf(const State::KeyPress& press) } } + /* + * And the five keys that are not letters, on a layout with no Latin letters at all: Russian + * and Arabic type letters of their own where a US keyboard has [ and ], so a paged document + * had no key for its pages there, and Persian types its own digits, so zoom had no key to + * put it back. Only on such a layout. A German keyboard types u with a diaeresis from the + * key where [ is and has [ somewhere else, and reading it by position would give a letter + * of its user's own language a command. + */ + if (!LayoutIsLatin()) + { + switch (press.key) + { + case KEY_LEFT_BRACKET: return '['; + case KEY_RIGHT_BRACKET: return ']'; + case KEY_ZERO: return '0'; + case KEY_MINUS: return '-'; + case KEY_EQUAL: return '='; + default: break; + } + } + return 0; } -/* What one key press asks for, or none: a key this head has no use for, or a repeat of one that - * acts once however long it is held. */ -int KeyOf(const State::KeyPress& press) +/* + * Whether a command goes on for as long as its key is held, at the rate the window system repeats + * a key: the ones that move through what is being read, and no other. + * + * A letter repeated like an arrow. A held a accepted the entry on screen and then every one that + * took its place, into source, none of them read, and a held d discarded them. The WinForms head + * gives what changes the queue a press each for that reason. Here the rest of what is not + * navigation acts once as well: a held m, r or j is the view flipping thirty times a second, and + * there is nothing more for a held q or 0 to do. + */ +bool Repeats(int key) +{ + switch (key) + { + case DEVIEW_KEY_SCROLL_UP: + case DEVIEW_KEY_SCROLL_DOWN: + case DEVIEW_KEY_PAGE_UP: + case DEVIEW_KEY_PAGE_DOWN: + case DEVIEW_KEY_NEXT_CHANGE: + case DEVIEW_KEY_PREVIOUS_CHANGE: + case DEVIEW_KEY_NEXT_VARIANT: + case DEVIEW_KEY_PREVIOUS_PAGE: + case DEVIEW_KEY_NEXT_PAGE: + case DEVIEW_KEY_ZOOM_IN: + case DEVIEW_KEY_ZOOM_OUT: + return true; + default: + return false; + } +} + +/* What a key going down asks for, or none for a key this head has no use for, whether it is the + * press or one of its repeats: see KeyOf. */ +int Pressed(const State::KeyPress& press) { const unsigned int letter = LetterOf(press); @@ -2319,24 +2554,12 @@ int KeyOf(const State::KeyPress& press) } } - /* The arrows and paging go on for as long as they are held, at the rate the window system - * repeats a key: one row a press was all a held arrow scrolled. */ switch (press.key) { case KEY_UP: return DEVIEW_KEY_SCROLL_UP; case KEY_DOWN: return DEVIEW_KEY_SCROLL_DOWN; case KEY_PAGE_UP: return DEVIEW_KEY_PAGE_UP; case KEY_PAGE_DOWN: return DEVIEW_KEY_PAGE_DOWN; - default: break; - } - - if (press.repeated) - { - return DEVIEW_KEY_NONE; - } - - switch (press.key) - { case KEY_HOME: return DEVIEW_KEY_HOME; case KEY_END: return DEVIEW_KEY_END; case KEY_TAB: return shift ? DEVIEW_KEY_PREVIOUS_ITEM : DEVIEW_KEY_NEXT_ITEM; @@ -2345,6 +2568,14 @@ int KeyOf(const State::KeyPress& press) } } +/* What one key press asks for, or none: a key this head has no use for, or a repeat of one that + * acts once however long it is held. */ +int KeyOf(const State::KeyPress& press) +{ + const int key = Pressed(press); + return press.repeated && !Repeats(key) ? DEVIEW_KEY_NONE : key; +} + /* * The next key waiting that asks for anything, and whether it was Escape. One a poll: the rest * wait for the polls after it, which is what keeps two keys pressed between two polls both acted @@ -2857,20 +3088,6 @@ void DrawPaneImage(const DeviewScreen* screen, const DeviewPane& pane, const Pan return; } - bool loading = false; - const std::string path = Copy(screen, pane.imagePathOffset, pane.imagePathLength); - const CachedTexture* decoded = Picture(path, loading); - if (decoded == nullptr) - { - /* Nothing at all for a picture this build cannot decode: the rows have said what it is. */ - if (loading) - { - DrawSpinner(list, centre, pitch, bounds.width, available); - } - - return; - } - /* * Fitted, and never enlarged past its own size: a snapshot is judged against the pixels it has, * and an eight pixel icon stretched across a pane is an interpolation of them rather than a @@ -2887,6 +3104,23 @@ void DrawPaneImage(const DeviewScreen* screen, const DeviewPane& pane, const Pan const ImVec2 fitted( std::max(1.0f, static_cast(pane.imageWidth) * scale), std::max(1.0f, static_cast(pane.imageHeight) * scale)); + const float drawnWidth = fitted.x * (pane.imageZoom > 1.0f ? pane.imageZoom : 1.0f); + + /* Asked for with its reduced copies only if it is drawn small enough to be sampled from + * them, which is known before it is decoded, from the size the model carries. */ + bool loading = false; + const std::string path = Copy(screen, pane.imagePathOffset, pane.imagePathLength); + const CachedTexture* decoded = Picture(path, WantsCopies(drawnWidth, static_cast(pane.imageWidth)), loading); + if (decoded == nullptr) + { + /* Nothing at all for a picture this build cannot decode: the rows have said what it is. */ + if (loading) + { + DrawSpinner(list, centre, pitch, bounds.width, available); + } + + return; + } /* * Past that only by the reader asking, and then cut off at the edges of the space rather than @@ -2897,13 +3131,8 @@ void DrawPaneImage(const DeviewScreen* screen, const DeviewPane& pane, const Pan ImVec2 size = fitted; ImVec2 uvMin(0.0f, 0.0f); ImVec2 uvMax(1.0f, 1.0f); - const float drawnWidth = fitted.x * (pane.imageZoom > 1.0f ? pane.imageZoom : 1.0f); const float ownWidth = static_cast(decoded->texture.width); - Sample( - path, - pane.imageZoom > 1.0f && drawnWidth >= ownWidth ? sampleAsPixels : - drawnWidth * 2.0f < ownWidth ? sampleReduced : - sampleSmoothed); + Sample(path, SamplingOf(drawnWidth, ownWidth, pane.imageZoom > 1.0f)); if (pane.imageZoom > 1.0f) { const ImVec2 whole(fitted.x * pane.imageZoom, fitted.y * pane.imageZoom); @@ -4245,10 +4474,38 @@ int32_t deview_present(const DeviewScreen* screen) bool arrived = TakeDecoded(); /* And its fonts, for that reason and because a font can only be added between frames. */ arrived = TakeFonts() || arrived; - arrived = Changed(screen) || arrived; + const bool changed = Changed(screen); + arrived = changed || arrived; arrived = Arrived() || arrived; arrived = arrived || state.tooltipDue; + /* + * A hidden window is not built for and not drawn into, whatever has arrived. It was, each time + * its screen changed, and a viewer hidden behind a tray is handed another screen by every + * arrival in its queue: the whole queue laid out and the whole window filled, for nobody. + * + * What arrived has still been taken in, above, and the fonts for what the screen holds are + * still asked for, so that they are there by the time it is shown. Showing it is itself an + * arrival, and marks the window stale, so the first present after it builds the screen it is + * handed and draws it: see deview_set_hidden and Arrived. + * + * Not before a frame has been built for the window at all. The grid the managed side slices + * its rows by is measured from one, and ImGui has no font to measure with until its first. + */ + if (state.hidden && + state.cellWidth > 0.0f) + { + if (changed) + { + FindFontsFor(screen); + } + + Rest(); + MeasureGrid(); + TrackPlacement(); + return 1; + } + /* * A window nothing is happening to is left alone: no frame is built, and nothing is drawn. * It used to be built, drawn and put on the screen sixty times a second, each one the frame @@ -4406,6 +4663,10 @@ int32_t deview_capture(const DeviewScreen* screen, int32_t width, int32_t height ImGui::SetCurrentContext(capture); ImGuiIO& io = ImGui::GetIO(); io.BackendFlags |= ImGuiBackendFlags_RendererHasTextures; + /* As the window's context declares it, and for its reason: both are drawn by RenderTriangles. + * Without it a capture of more than 65,535 vertices in one draw list, which dense text at 4K + * comes to, had its indices wrap, and came out scrambled. */ + io.BackendFlags |= ImGuiBackendFlags_RendererHasVtxOffset; io.IniFilename = nullptr; io.LogFilename = nullptr; ApplyStyle(); diff --git a/native/swift/Sources/Deview/Renderer.swift b/native/swift/Sources/Deview/Renderer.swift index d373f9bf..d6e49e26 100644 --- a/native/swift/Sources/Deview/Renderer.swift +++ b/native/swift/Sources/Deview/Renderer.swift @@ -70,10 +70,16 @@ final class Renderer { private var dirty: CGRect? /// Both panes name the one picture in the draw in progress, as a byte equal pair of documents - /// does: their pages are kept under the hash of the document. There is one scaled copy a - /// picture and the two panes can be a point apart in width, so each would have the copy made - /// for its own size in turn, without end. An enlarged pair is not given the chance: it is - /// drawn from the picture, as every enlarged picture was before it had a copy. + /// does: their pages are kept under the hash of the document. The two panes can be a point + /// apart in width, and so can ask for copies of different sizes. With one copy a picture each + /// had it made for its own size in turn, without end: every one that landed was a redraw, in + /// which the other pane found the copy was not its own and asked again. So a picture both + /// panes name keeps the last two copies made, which is one for each: see `Picture.spare`. + /// An enlarged pair makes none: it is drawn from the picture, as every enlarged picture was + /// before it had a copy. + /// + /// As the last draw to begin left it. `settle` reads it for a copy that has landed, which + /// was asked for by that draw or by one before it. private var shared = false /// Decoded pictures, keyed by the path the screen model handed over and invalidated by the @@ -105,6 +111,10 @@ final class Renderer { /// goes when a copy for another size lands, and with the picture. var scaled: CGImage? + /// The copy `scaled` took the place of, kept only while both panes name this picture: see + /// `shared`. Each pane then has the copy made for its own size, one here and one there. + var spare: CGImage? + /// Being decoded on `work`. The pane shows a spinner until it lands. var decoding = false @@ -407,10 +417,18 @@ final class Renderer { let panesWidth = max(cell.width * 2, content - Renderer.padding - panesLeft) let half = (panesWidth / 2).rounded(.down) - text(frame.title, in: rect(top: Renderer.padding, left: Renderer.padding, width: size.width - Renderer.padding * 2, height: line, size), Palette.text, context) + // A title that would run on under the subtitle stops a character short of it, where the + // subtitle used to be drawn over whatever of the title had got that far. It is the title + // that gives way, as in the Linux head: what it says is also in the pane headers and the + // queue, and which entry of the queue this is is said only by the subtitle. One that fits + // has the whole row, as it always had, so it is drawn exactly as it was. + let titleWidth = size.width - Renderer.padding * 2 + let subtitleWidth = CGFloat(frame.subtitle.count) * cell.width + let runsUnder = !frame.subtitle.isEmpty && CGFloat(frame.title.count) * cell.width > titleWidth - subtitleWidth + let titleRoom = runsUnder ? titleWidth - subtitleWidth - cell.width : titleWidth + text(frame.title, in: rect(top: Renderer.padding, left: Renderer.padding, width: titleRoom, height: line, size), Palette.text, context) if !frame.subtitle.isEmpty { - let width = CGFloat(frame.subtitle.count) * cell.width - text(frame.subtitle, in: rect(top: Renderer.padding, left: size.width - Renderer.padding - width, width: width, height: line, size), Palette.dim, context) + text(frame.subtitle, in: rect(top: Renderer.padding, left: size.width - Renderer.padding - subtitleWidth, width: subtitleWidth, height: line, size), Palette.dim, context) } let firstRule = Renderer.padding + line + Renderer.gap @@ -1049,6 +1067,14 @@ final class Renderer { return scaled } + // The other of the two copies a picture in both panes keeps, which is this pane's when + // the last one to land was made for the other + if let spare = entry.spare, + spare.width == width, + spare.height == height { + return spare + } + let size = Pixels(width: width, height: height) if capturing { guard let scaled = Renderer.scale(picture, to: size) else { @@ -1145,6 +1171,10 @@ final class Renderer { entry.scaling = nil if let image { + // The copy this one takes the place of is the other pane's, when both panes + // name the picture, and is kept for it. Otherwise it was made for a size + // the pane no longer has, and goes. + entry.spare = shared ? entry.scaled : nil entry.scaled = image } else { entry.unscalable = true diff --git a/native/swift/Sources/Deview/ViewerView.swift b/native/swift/Sources/Deview/ViewerView.swift index 7bb6d336..cb000241 100644 --- a/native/swift/Sources/Deview/ViewerView.swift +++ b/native/swift/Sources/Deview/ViewerView.swift @@ -395,9 +395,27 @@ final class ViewerView: NSView, NSViewToolTipOwner { return } + // A key held past the repeat delay arrives again many times a second, and each one is + // queued. That is what a held Down is for. A held a accepted the entry on screen and then + // every one that took its place, into source, none of them read, and the ones queued + // while a frame was slow were still handed over after the key came up. So what changes + // the queue takes a press each, as it does in the WinForms head. Swallowed rather than + // passed on: it is still this key. + if event.isARepeat, ViewerView.changesQueue(key) { + return + } + Runtime.shared.post(.key(key)) } + /// The keys that act on the queue rather than on the view: the ones of + /// `ViewerSession.ChangesQueue` that a key here can be. + private static func changesQueue(_ key: Int32) -> Bool { + key == DEVIEW_KEY_ACCEPT.value || + key == DEVIEW_KEY_ACCEPT_ALL.value || + key == DEVIEW_KEY_DISCARD.value + } + /// Matches ReadKey in deview.cpp and the WinForms head's Map, which is the keymap the docs /// publish. private static func map(_ event: NSEvent) -> Int32 { @@ -450,11 +468,19 @@ final class ViewerView: NSView, NSViewToolTipOwner { // was. German, French, Nordic, Spanish and Italian layouts have them behind Option, and // with the modifiers left out the key is the digit or the letter printed on it, so those // readers could not turn a page from the keyboard. + // + // The zoom keys the same way, for a layout that has one of them behind Option. Shift is + // no part of this: it is kept by both readings, so equals unshifted and plus with shift + // held are each what was typed on a US keyboard, and each zooms in, as they did. switch event.characters { case "[": return DEVIEW_KEY_PREVIOUS_PAGE.value case "]": return DEVIEW_KEY_NEXT_PAGE.value + case "=", "+": + return DEVIEW_KEY_ZOOM_IN.value + case "-": + return DEVIEW_KEY_ZOOM_OUT.value default: break } diff --git a/src/DiffEngine.Benchmarks/InlineAcceptBenchmarks.cs b/src/DiffEngine.Benchmarks/InlineAcceptBenchmarks.cs index 52b61dd8..2aa88e1d 100644 --- a/src/DiffEngine.Benchmarks/InlineAcceptBenchmarks.cs +++ b/src/DiffEngine.Benchmarks/InlineAcceptBenchmarks.cs @@ -39,6 +39,9 @@ public void Setup() source = text; pristine = new UTF8Encoding(false).GetBytes(text); File.WriteAllBytes(path, pristine); + // As a source file is when a test run asks about it: written at the last save, and not + // in the last few seconds, which is the file a probe keeps what it read of + File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddMinutes(-1)); patches = new InlinePatch[CallSites]; appends = new InlinePatch[CallSites]; for (var index = 0; index < CallSites; index++) @@ -113,36 +116,15 @@ public int AnchorEach() return Every(anchored); } - // The patcher alone over the same patches, each applied to what the last one left: what the - // accepts above cost with the file system, the encoding and the locks taken out + // The patcher alone over the same patches, each applied to what the last one left: what + // AcceptTogether costs with the file system, the encoding and the locks taken out [Benchmark] public int PatchInMemory() { var current = source; - var applied = 0; - foreach (var patch in patches) - { - var status = InlinePatcher.TryApply( - SourceLanguage.CSharp, - current, - patch.LineHint, - patch.Mode, - patch.OriginalExpression, - patch.OriginalValue, - patch.MemberName, - patch.EntryPoints, - false, - patch.NewContent, - out var patched, - out _); - if (status == PatchStatus.Applied) - { - current = patched; - applied++; - } - } - - return Every(applied); + var results = new InlineApplyResult[patches.Length]; + InlineApplier.PatchInTurn(SourceLanguage.CSharp, ref current, patches, path, results); + return Every(results.Count(_ => _.Status == InlineApplyStatus.Applied)); } // The lexing every one of those starts with diff --git a/src/DiffEngine.Benchmarks/ListenerTableBenchmarks.cs b/src/DiffEngine.Benchmarks/ListenerTableBenchmarks.cs new file mode 100644 index 00000000..c6bac7a1 --- /dev/null +++ b/src/DiffEngine.Benchmarks/ListenerTableBenchmarks.cs @@ -0,0 +1,72 @@ +using System.Net; +using System.Net.Sockets; +using BenchmarkDotNet.Attributes; + +namespace DiffEngine.Benchmarks; + +// What it costs to ask the operating system whether anything is listening on a port, which is +// asked in front of a connect that would otherwise wait two seconds to be refused: by ViewerClient +// for a port it has not just talked to, and by PiperClient on every move and delete. +// +// The answer is no here, which is the answer that has to look at every row there is. What is +// varied is how many connections the machine has open beside the listeners, since that is what +// the cost was found to follow: each loopback connection made here is two rows, one for each end. +// +// Never the real ports, which a tray on the machine running this owns. The port asked about is +// one this process binds and does not listen on, as in UnownedPortBenchmarks. +[MemoryDiagnoser] +public class ListenerTableBenchmarks +{ + [Params(0, 1500)] + public int Connections; + + Socket holder = null!; + int port; + TcpListener listener = null!; + List ends = []; + + [GlobalSetup] + public void Setup() + { + holder = new(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp) + { + ExclusiveAddressUse = true + }; + holder.Bind(new IPEndPoint(IPAddress.Loopback, 0)); + port = ((IPEndPoint) holder.LocalEndPoint!).Port; + + listener = new(IPAddress.Loopback, 0); + listener.Start(); + var target = ((IPEndPoint) listener.LocalEndpoint).Port; + for (var index = 0; index < Connections; index++) + { + var client = new TcpClient(); + client.Connect(IPAddress.Loopback, target); + ends.Add(client); + ends.Add(listener.AcceptTcpClient()); + } + } + + [GlobalCleanup] + public void Cleanup() + { + foreach (var end in ends) + { + end.Dispose(); + } + + ends.Clear(); + listener.Stop(); + holder.Dispose(); + } + + [Benchmark] + public bool NobodyListening() => + ListenerTable.IsHeld(port); + + // What IsHeld read before the listeners were asked for alone, and still reads where they + // cannot be + [Benchmark] + public bool NobodyListeningOutOfEveryRow() => + ListenerTable.Enumerated(port); +} diff --git a/src/DiffEngine.Benchmarks/TextDiffBenchmarks.cs b/src/DiffEngine.Benchmarks/TextDiffBenchmarks.cs index fe72b4e2..09c53efd 100644 --- a/src/DiffEngine.Benchmarks/TextDiffBenchmarks.cs +++ b/src/DiffEngine.Benchmarks/TextDiffBenchmarks.cs @@ -30,8 +30,25 @@ public void Setup() var lines = expected.Split('\n'); Array.Reverse(lines); reordered = string.Join('\n', lines); + // Every line survives, in runs of fifty, one run in four moved to somewhere else: what + // sorting a list by another key, or moving its sections about, leaves. Reversed, nothing + // is still in order, so there is nothing for a diff to find however it looks. Here most + // of an order survives, and a diff is as good as how much of it is kept. + var random = new Random(7); + var runs = expected.Split('\n').Chunk(50).ToArray(); + var places = Enumerable.Range(0, runs.Length) + .Select(_ => random.Next(4) == 0 ? random.Next(runs.Length) : _) + .ToArray(); + Array.Sort(places, runs); + runsMoved = string.Join('\n', runs.SelectMany(_ => _)); } + string runsMoved = ""; + + [Benchmark] + public int SameLinesWithRunsMoved() => + LineDiff.Build(expected, runsMoved).Entries.Count; + [Benchmark] public int OnePercentChanged() => LineDiff.Build(expected, onePercentChanged).Entries.Count; diff --git a/src/DiffEngine.Tests/FsCompilerRoundTripTests.cs b/src/DiffEngine.Tests/FsCompilerRoundTripTests.cs index f74da477..a0862551 100644 --- a/src/DiffEngine.Tests/FsCompilerRoundTripTests.cs +++ b/src/DiffEngine.Tests/FsCompilerRoundTripTests.cs @@ -121,6 +121,125 @@ static readonly (InlinePatchMode Mode, string Call)[] midLineCalls = "a ` b" ]; + /// + /// Lines of code F# compiles, each holding a tick or an interpolated string the scanner has + /// to step over as the compiler does, or the call under it is taken to be inside a literal. + /// A tick after a name or a type parameter is part of it, and anywhere else opens a char + /// literal, which is where a quote that opens no string is written. A hole of + /// $"..." holds char literals and comments and no string; one of a triple quoted + /// literal holds strings too, a verbatim one that ends in a doubled quote among them. Each is + /// here because fsi was asked, and is asked again by + /// . + /// + internal static readonly string[] Code = + [ + "let a' = 1 in ignore a'", + "let a'b = 1 in ignore a'b", + "let a'' = 1 in ignore a''", + "ignore '\"'", + "ignore ['a';'\"';'\\'']", + "ignore ('\"', '\\\\', '\\\"')", + "ignore (1,'\"')", + "ignore (id'\"')", + "ignore 'a'B", + "ignore ('\\065', '\\u0041', '\\U00000041', '\\x41')", + "let f (x: 'a) (y: 'a) = x in ignore (f '\"' '\"')", + "let x' = '\"' in ignore x'", + "let x = 1 in ignore (x,'\"')", + "ignore $\"{1}\"", + "ignore $\"{{literal}}\"", + "ignore $\"{ {| A = 1 |}.A }\"", + "ignore $\"{'}'}\"", + "ignore $\"{'{'}\"", + "ignore $\"{'\"'}\"", + "ignore $\"{1:N2}\"", + "ignore $\"%d{1}\"", + "ignore $@\"c:\\{1}\\\"", + "ignore @$\"c:\\{1}\\\"", + "ignore $@\"a\"\"{1}\"\"b\"", + "ignore $\"a\\\"{1}\\\"\"", + "ignore $\"{1 (* } *)}\"", + "ignore $\"{1 (* { *)}\"", + "ignore $\"{ [ for c in ['}'] -> c ] }\"", + "ignore $\"}}\"", + "ignore $\"{{\"", + "ignore $\"{1}}}\"", + "ignore $\"\\{1}\"", + "ignore $\"\"\"{\"a\"}\"\"\"", + "ignore $\"\"\"{ \"}\" }\"\"\"", + "ignore $\"\"\"{ \"{\" }\"\"\"", + "ignore $\"\"\"{ '\"' }\"\"\"", + "ignore $\"\"\"{ '{' }\"\"\"", + "ignore $\"\"\"{ $\"{1}\" }\"\"\"", + "ignore $\"\"\"{ @\"a\"\"\" }\"\"\"", + "ignore $\"\"\"{ @\"a\"\"b\" }\"\"\"", + "ignore $\"\"\"{ \"a\\\"\" }\"\"\"", + "ignore $\"\"\"{1}\"\" \"\"\"", + "ignore $\"\"\"{ (1, \"}}\") }\"\"\"", + "ignore $\"\"\"{ {| A = \"}\" |}.A }\"\"\"", + // A backslash escapes no brace, so this is a backslash and then a hole + "ignore $\"\\{'\"'}\"", + "ignore $\"{1}{'\"'}\"", + "let x' = 1 in ignore $\"{x'}{'}'}\"", + "let ``a}b`` = 1 in ignore $\"{ ``a}b`` }\"", + // A line comment in a hole runs to the end of its line, whatever it holds + "ignore $\"{1 // }\n }\"", + "ignore $\"\"\"{ 1 // } \"\"\"\n }\"\"\"", + // Doubled braces are text with one dollar, and what opens a hole with two + "ignore $\"\"\"{{ {1} }}\"\"\"", + "ignore $\"\"\"{{{'\"'}}}\"\"\"", + "ignore $$\"\"\"{{1}} {literal}\"\"\"", + "ignore $$\"\"\"{{ \"}}\" }}\"\"\"", + "ignore $$\"\"\"{ {{'\"'}} }\"\"\"", + "ignore $$\"\"\"{{{'\"'}}}\"\"\"", + "ignore $$$\"\"\"{{ {{{'\"'}}} }}\"\"\"", + "ignore \"a\"B", + "ignore \"\"\"a\"\"\"B" + ]; + + [Test] + [RequiresDotnet] + public async Task TicksAndHolesAreReadAsTheCompilerReadsThem() + { + var expected = Convert.ToBase64String(Encoding.UTF8.GetBytes("found")); + var builder = new StringBuilder(prelude); + // Every line the scanner lost the call under, rather than the first, since one reading + // put right is as likely as not to be the reason for the next + var lost = new List(); + for (var index = 0; index < Code.Length; index++) + { + // The line is the first thing in the body, so a scanner that reads a literal in it as + // longer or shorter than it is loses the call under it, and a compiler that does not + // take the line has no function to call + var snippet = $"let code{index} () =\n {Code[index]}\n Verify(\"x\").Snapshot().ToTask()\n"; + var status = InlinePatcher.TryApply(SourceLanguage.FSharp, snippet, 3, InlinePatchMode.Set, null, null, null, null, false, "found", out var patched, out _); + if (status != PatchStatus.Applied) + { + lost.Add(Code[index]); + continue; + } + + builder.Append(patched); + builder.Append($"check \"code{index}\" (code{index} ()) \"{expected}\"\n\n"); + } + + await Assert.That(lost).IsEmpty(); + builder.Append(footer); + var path = Path.Combine(Path.GetTempPath(), $"DiffEngineFsCode_{Guid.NewGuid():N}.fsx"); + await File.WriteAllTextAsync(path, builder.ToString(), new UTF8Encoding(false)); + try + { + var (exitCode, output) = RunFsi(path); + + await Assert.That(output).Contains("ALL OK"); + await Assert.That(exitCode).IsEqualTo(0); + } + finally + { + File.Delete(path); + } + } + [Test] [RequiresDotnet] public async Task CommentsAndNamesAreReadAsTheCompilerReadsThem() @@ -337,6 +456,9 @@ void Add(string name, string snippet, int lineHint, InlinePatchMode mode) builder.Append($"check \"removedBang\" (removedBang ()) \"{Convert.ToBase64String(Encoding.UTF8.GetBytes("xdup"))}\"\n\n"); builder.Append(Patch("let removedChain () =\n Verify(\"x\")\n .Snapshot(\"dup\")\n .Snapshot(\"kept\").ToTask()\n", 3, InlinePatchMode.Remove, "", "dup")); builder.Append($"check \"removedChain\" (removedChain ()) \"{Convert.ToBase64String(Encoding.UTF8.GetBytes("kept"))}\"\n\n"); + // And where a statement was, taken whole for being a call on a name and nothing else + builder.Append(Patch("let removedStatement () =\n let first = Verify(\"x\")\n first.Snapshot(\"dup\")\n first.Snapshot(\"dup\")\n first.ToTask()\n", 3, InlinePatchMode.Remove, "", "dup")); + builder.Append($"check \"removedStatement\" (removedStatement ()) \"{Convert.ToBase64String(Encoding.UTF8.GetBytes("x"))}\"\n\n"); builder.Append(footer); return builder.ToString(); diff --git a/src/DiffEngine.Tests/InlineApplierBatchTests.cs b/src/DiffEngine.Tests/InlineApplierBatchTests.cs index 8f7bd498..7bc31b18 100644 --- a/src/DiffEngine.Tests/InlineApplierBatchTests.cs +++ b/src/DiffEngine.Tests/InlineApplierBatchTests.cs @@ -134,6 +134,35 @@ public async Task AWriteThatFailsFailsEveryPatchItCarried() await Assert.That(Directory.GetFileSystemEntries(file.Root)).IsEquivalentTo([file.FullName]); } + /// + /// A patch after the first edit that made no edit of its own is told what is true of the file + /// as it still is, and not that the write failed: its snapshot was in the source all along, + /// or its call site was never there. They used to report the write with the rest, so the + /// first stayed queued as a failure and the second was not told to re-run. The duplicate + /// still reports the write, since the literal it found was one the write was carrying. + /// + [Test] + public async Task AWriteThatFailsLeavesAPatchThatDidNotNeedItItsOwnAnswer() + { + using var file = new TempSource(Members(4)); + var before = file.Text; + + var results = InlineApplier.ApplyAll( + [ + Set(file.FullName, 0), + Set(file.FullName, 2, anchor: "\"not in the source\""), + Set(file.FullName, 1, content: "old 1"), + Set(file.FullName, 0), + Set(file.FullName, 3) + ], + (_, _) => throw new IOException("The process cannot access the file.")); + + await Assert.That(Statuses(results)).IsEqualTo("Failed, NotFound, AlreadyApplied, Failed, Failed"); + await Assert.That(results[1].Message!).Contains("Re-run the test"); + await Assert.That(results[3].Message!).Contains("Failed to write"); + await Assert.That(file.Text).IsEqualTo(before); + } + /// /// A batch in which nothing applies writes nothing, as a single patch that is already applied /// writes nothing: the file an editor has open is not touched for no reason. @@ -211,6 +240,163 @@ public async Task EncodingAndLineEndingsSurviveABatch() public async Task AnEmptyBatchIsNoResults() => await Assert.That(InlineApplier.ApplyAll([])).IsEmpty(); + /// + /// A batch lexes its file once and carries the scan from each patch to the next. What each + /// patch is told, and what the source comes to, has to be what lexing the whole text again + /// for every patch gives. Patches of every mode, in an order made at random: literals that + /// grow and move every call site under them, calls appended and removed, the same patch + /// twice, an anchor that is in no call, and hints that are stale from the first edit on. + /// + [Test] + [Arguments("cs")] + [Arguments("fs")] + public async Task ACarriedScanTellsEachPatchWhatLexingAgainWould(string extension) + { + var language = SourceLanguage.ForFile($"Sample.{extension}"); + var path = Path.Combine(Path.GetTempPath(), $"Sample.{extension}"); + var problems = new List(); + var seen = new HashSet(); + for (var seed = 0; seed < 60 && problems.Count < 3; seed++) + { + var random = new Random(seed); + var (source, patches) = MixedBatch(random, extension, path, seed % 2 == 0 ? "\n" : "\r\n"); + + var carried = source; + var results = new InlineApplyResult[patches.Count]; + InlineApplier.PatchInTurn(language, ref carried, patches, path, results); + + var lexedAgain = source; + for (var index = 0; index < patches.Count; index++) + { + var patch = patches[index]; + var status = InlinePatcher.TryApply( + language, + lexedAgain, + patch.LineHint, + patch.Mode, + patch.OriginalExpression, + patch.OriginalValue, + patch.MemberName, + patch.EntryPoints, + false, + patch.NewContent, + out var patched, + out var reason); + if (status == PatchStatus.Applied) + { + lexedAgain = patched; + } + + var expected = status switch + { + PatchStatus.Applied => InlineApplyResult.Applied, + PatchStatus.AlreadyApplied => InlineApplyResult.AlreadyApplied, + _ => InlineApplyResult.NotFound(reason) + }; + if (results[index].Status != expected.Status || + results[index].Message != expected.Message) + { + problems.Add($"seed {seed}, patch {index} ({patch.Mode} at {patch.LineHint}): {results[index].Status} {results[index].Message}, and lexed again {expected.Status} {expected.Message}"); + break; + } + } + + if (carried != lexedAgain) + { + problems.Add($"seed {seed}: the source differs"); + } + + seen.UnionWith(results.Select(_ => _.Status)); + } + + await Assert.That(problems).IsEmpty(); + // Or the batches were all refusals, and agreed about nothing much + await Assert.That(seen).IsEquivalentTo([InlineApplyStatus.Applied, InlineApplyStatus.AlreadyApplied, InlineApplyStatus.NotFound]); + } + + /// + /// A file of members, each with a call that has a snapshot and one that has none, and a + /// shuffled batch of patches for them. + /// + static (string source, List patches) MixedBatch(Random random, string extension, string path, string eol) + { + var fsharp = extension == "fs"; + var builder = new StringBuilder(); + var patches = new List(); + var line = 0; + + void Add(string text) + { + builder.Append(text); + builder.Append(eol); + line++; + } + + Add(fsharp ? "module Tests" : "class Tests"); + Add(fsharp ? "" : "{"); + var members = 6 + random.Next(6); + for (var member = 0; member < members; member++) + { + Add(fsharp ? $"let member{member} () =" : $" async Task Member{member}()"); + Add(fsharp ? " // a comment with a \" in it" : " {"); + Add($" {(fsharp ? "let" : "var")} text{member} = \"a string // with (* things */ in it\"{(fsharp ? "" : ";")}"); + Add($" Verify(first{member})"); + Add($" .Snapshot(\"old {member}\"){(fsharp ? ".ToTask() |> ignore" : ";")}"); + var snapshotLine = line; + Add($" Verify(second{member}){(fsharp ? ".ToTask() |> ignore" : ";")}"); + var verifyLine = line; + Add(fsharp ? "" : " }"); + + var name = fsharp ? $"member{member}" : $"Member{member}"; + InlinePatch Patch(int hint, string? expression, string? value, string content, InlinePatchMode mode) => + new(path, hint, fsharp ? null : expression, content, mode) + { + TestName = null, + MemberName = name, + OriginalValue = fsharp ? value : null + }; + + var content = random.Next(3) switch + { + 0 => $"new {member}", + 1 => $"new {member}\nsecond \"line\"\n third", + _ => $"// {member}\n(* \"\"\" *)\n" + }; + switch (random.Next(5)) + { + case 0: + patches.Add(Patch(snapshotLine, $"\"old {member}\"", $"old {member}", content, InlinePatchMode.Set)); + break; + case 1: + patches.Add(Patch(snapshotLine, $"\"old {member}\"", $"old {member}", "", InlinePatchMode.Remove)); + break; + case 2: + patches.Add(Patch(verifyLine, null, null, content, InlinePatchMode.Append)); + break; + case 3: + // Both of a member's calls, and the first of them twice + patches.Add(Patch(snapshotLine, $"\"old {member}\"", $"old {member}", content, InlinePatchMode.Set)); + patches.Add(Patch(verifyLine, null, null, content, InlinePatchMode.Append)); + patches.Add(Patch(snapshotLine, $"\"old {member}\"", $"old {member}", content, InlinePatchMode.Set)); + break; + default: + patches.Add(Patch(snapshotLine, "\"in no call\"", "in no call", content, InlinePatchMode.Set)); + break; + } + } + + Add(fsharp ? "" : "}"); + + // Queue order is not file order + for (var index = patches.Count - 1; index > 0; index--) + { + var other = random.Next(index + 1); + (patches[index], patches[other]) = (patches[other], patches[index]); + } + + return (builder.ToString(), patches); + } + // Joined, because the order is the point and the collection assertions do not check it static string Statuses(IEnumerable results) => string.Join(", ", results.Select(_ => _.Status)); diff --git a/src/DiffEngine.Tests/InlineApplierTests.cs b/src/DiffEngine.Tests/InlineApplierTests.cs index 5050746c..294d5d2c 100644 --- a/src/DiffEngine.Tests/InlineApplierTests.cs +++ b/src/DiffEngine.Tests/InlineApplierTests.cs @@ -970,6 +970,109 @@ public async Task CanAnchorOnAMissingFileFails() await Assert.That(result.Status).IsEqualTo(InlineApplyStatus.Failed); } + const string twoCallSites = + """ + class C + { + void First() => Verify(first); + void Other() => Verify(other).Snapshot("old"); + } + """; + + /// + /// A run asks once for each call site, and each asking read and lexed the whole file. The + /// file is held open here with nothing shared, so a second asking that reads it fails to: + /// what it is told comes from the first one's reading. CanApply is a dry run too. + /// + [Test] + public async Task ASecondProbeOfAFileDoesNotReadItAgain() + { + var path = WriteTemp(Utf8(twoCallSites, bom: false)); + try + { + // As a source file is by the time a run asks: not written in the last few seconds + File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddMinutes(-1)); + + var first = InlineApplier.CanAnchor(Append(path)); + InlineApplyResult second; + InlineApplyResult set; + using (new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.None)) + { + second = InlineApplier.CanAnchor(Patch(path, 4, null, "new", InlinePatchMode.Append)); + set = InlineApplier.CanApply(Patch(path, 4, "\"old\"", "new")); + } + + await Assert.That(first.Status).IsEqualTo(InlineApplyStatus.Applied); + await Assert.That(second.Status).IsEqualTo(InlineApplyStatus.Applied); + await Assert.That(set.Status).IsEqualTo(InlineApplyStatus.Applied); + } + finally + { + File.Delete(path); + } + } + + /// + /// What was read stands only while the file does. The same length written at another time is + /// another file, and so is one an accept has just written. + /// + [Test] + public async Task AProbeReadsAFileThatHasChangedSinceTheLast() + { + var path = WriteTemp(Utf8(twoCallSites, bom: false)); + try + { + File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddMinutes(-2)); + var set = Patch(path, 4, "\"old\"", "new"); + var before = InlineApplier.CanApply(set); + + // The same length, so only the time says it is not the file that was read + await File.WriteAllTextAsync(path, twoCallSites.Replace("\"old\"", "\"new\"")); + File.SetLastWriteTimeUtc(path, DateTime.UtcNow.AddMinutes(-1)); + var after = InlineApplier.CanApply(set); + + var applied = InlineApplier.Apply(Patch(path, 3, null, "appended", InlinePatchMode.Append)); + var anchored = InlineApplier.CanApply(Patch(path, 3, null, "appended", InlinePatchMode.Append)); + + await Assert.That(before.Status).IsEqualTo(InlineApplyStatus.Applied); + await Assert.That(after.Status).IsEqualTo(InlineApplyStatus.AlreadyApplied); + await Assert.That(applied.Status).IsEqualTo(InlineApplyStatus.Applied); + await Assert.That(anchored.Status).IsEqualTo(InlineApplyStatus.AlreadyApplied); + } + finally + { + File.Delete(path); + } + } + + /// + /// A file written a moment ago is not kept: a second write inside the same tick of the file + /// system's clock leaves it the same length with the same time, which is arranged here. So + /// the second asking reads it, and is told about what it now holds. + /// + [Test] + public async Task AProbeOfAFileJustWrittenIsNotKept() + { + var path = WriteTemp(Utf8(twoCallSites, bom: false)); + try + { + var written = File.GetLastWriteTimeUtc(path); + var set = Patch(path, 4, "\"old\"", "new"); + var before = InlineApplier.CanApply(set); + + await File.WriteAllTextAsync(path, twoCallSites.Replace("\"old\"", "\"new\"")); + File.SetLastWriteTimeUtc(path, written); + var after = InlineApplier.CanApply(set); + + await Assert.That(before.Status).IsEqualTo(InlineApplyStatus.Applied); + await Assert.That(after.Status).IsEqualTo(InlineApplyStatus.AlreadyApplied); + } + finally + { + File.Delete(path); + } + } + static InlinePatch Append(string path) => Patch(path, 3, null, "new", InlinePatchMode.Append); diff --git a/src/DiffEngine.Tests/InlinePatcherTests.cs b/src/DiffEngine.Tests/InlinePatcherTests.cs index 49acbb00..ad8ae3d9 100644 --- a/src/DiffEngine.Tests/InlinePatcherTests.cs +++ b/src/DiffEngine.Tests/InlinePatcherTests.cs @@ -1074,6 +1074,90 @@ await Verify(b) """)); } + /// + /// A call verified through its settings has nothing chained onto it, and with the hint gone + /// stale it was the first call the walk met with no Snapshot call: the snapshot of the call + /// under it was appended to it. A name Snapshot is called on, passed to a call, says that + /// call has one. + /// + [Test] + [Arguments("settings")] + [Arguments("settings: settings")] + public async Task AppendWithAStaleHintPassesOverACallVerifiedThroughItsSettings(string argument) + { + var source = Method( + $""" + var settings = new VerifySettings(); + settings.Snapshot("A"); + await Verify(a, {argument}); + await Verify(b); + """); + + var status = TryApply(source, 14, InlinePatchMode.Append, null, "B", out var newSource, out var reason, memberName: "Test"); + + await Assert.That((status, reason)).IsEqualTo((PatchStatus.Applied, "")); + await Assert.That(newSource).IsEqualTo( + Method( + $""" + var settings = new VerifySettings(); + settings.Snapshot("A"); + await Verify(a, {argument}); + await Verify(b) + .Snapshot("B"); + """)); + } + + const string twoWithNoSnapshot = + """ + class Tests + { + async Task Test() + { + await Verify(a).UseDirectory("files"); + await Throws(() => Verify(b)); + } + } + + """; + + /// + /// Two calls with no Snapshot call, one of them verified through files, and a hint that names + /// neither. An append has no anchor to tell them apart by, and the first was taken, which is + /// the wrong one whenever the file verified call comes first. Refused, for a re-run to say + /// where the call is. The verify call inside the Throws is part of that call, not a third. + /// + [Test] + public async Task AppendWithAStaleHintIsRefusedWhenTwoCallsCouldTakeIt() + { + var status = TryApply(Source(twoWithNoSnapshot), 12, InlinePatchMode.Append, null, "B", out _, out var reason, memberName: "Test"); + + await Assert.That(status).IsEqualTo(PatchStatus.NotFound); + await Assert.That(reason).Contains("more than one verify entry point call in its test has no Snapshot call"); + } + + // The recorded line still names its call, whatever else is in the member + [Test] + public async Task AppendTakesTheCallOnTheRecordedLineAmongSeveral() + { + var status = TryApply(Source(twoWithNoSnapshot), 6, InlinePatchMode.Append, null, "B", out var newSource, out _, memberName: "Test"); + + await Assert.That(status).IsEqualTo(PatchStatus.Applied); + await Assert.That(newSource).Contains("await Verify(a).UseDirectory(\"files\");\n"); + await Assert.That(newSource).Contains("await Throws(() => Verify(b))\n .Snapshot(\"B\");"); + } + + // One call with none, and a call nested in its arguments, is still one call + [Test] + public async Task AppendWithAStaleHintTakesTheOnlyCallWithNoSnapshot() + { + var source = Method(" await Verify(a).Snapshot(\"A\");\n await Throws(() => Verify(b));"); + + var status = TryApply(source, 12, InlinePatchMode.Append, null, "B", out var newSource, out var reason, memberName: "Test"); + + await Assert.That((status, reason)).IsEqualTo((PatchStatus.Applied, "")); + await Assert.That(newSource).Contains("await Throws(() => Verify(b))\n .Snapshot(\"B\");"); + } + /// /// The hint lands on the accepted call this time, which is what a second framework's patch for /// that same call site looks like once the first has been accepted. It names that call, so the @@ -1582,6 +1666,62 @@ public async Task ReapplyingARemoveOfAStatementIsAlreadyApplied() await Assert.That(status).IsEqualTo(PatchStatus.AlreadyApplied); } + /// + /// A statement taken whole brought the line under it up onto the line the patch names, as a + /// chained call taken with its line did, and where that line was a sibling's statement with + /// the same literal the second apply took the sibling. So the line stays here too, empty. + /// + [Test] + [Arguments(" other.Snapshot(\"dup\");")] + [Arguments(" await Verify(value, other).Snapshot(\"dup\");")] + public async Task RemoveOfAStatementAppliedTwiceLeavesTheSiblingUnderIt(string sibling) + { + var source = Method($" settings.Snapshot(\"dup\");\n{sibling}\n await Verify(value, settings);"); + var removed = Method($"\n{sibling}\n await Verify(value, settings);"); + + var first = TryApply(source, 5, InlinePatchMode.Remove, "\"dup\"", "", out var once, out _, memberName: "Test"); + var second = TryApply(once, 5, InlinePatchMode.Remove, "\"dup\"", "", out _, out _, memberName: "Test"); + + await Assert.That(first).IsEqualTo(PatchStatus.Applied); + await Assert.That(once).IsEqualTo(removed); + await Assert.That(second).IsEqualTo(PatchStatus.AlreadyApplied); + } + + /// + /// One line is kept for a statement over several, written with the file's own line break, as + /// it is for a chained call: the first line is the one the patch names. + /// + [Test] + public async Task RemoveOfAStatementOverSeveralLinesAppliedTwiceLeavesTheSiblingUnderIt() + { + var literal = "\"\"\"\n dup\n \"\"\""; + var source = Method($" settings.Snapshot(\n {literal}); // inline for now\n other.Snapshot(\n {literal});").Replace("\n", "\r\n"); + var removed = Method($"\n other.Snapshot(\n {literal});").Replace("\n", "\r\n"); + var anchor = literal.Replace("\n", "\r\n"); + + var first = TryApply(source, 5, InlinePatchMode.Remove, anchor, "", out var once, out _, memberName: "Test"); + var second = TryApply(once, 5, InlinePatchMode.Remove, anchor, "", out _, out _, memberName: "Test"); + + await Assert.That(first).IsEqualTo(PatchStatus.Applied); + await Assert.That(once).IsEqualTo(removed); + await Assert.That(second).IsEqualTo(PatchStatus.AlreadyApplied); + } + + /// + /// The line is kept only for a Snapshot call that would come up onto it. Under anything else + /// the statement's lines go, as they always did. + /// + [Test] + public async Task RemoveOfAStatementWithNoSnapshotCallUnderItKeepsNoLine() + { + var source = Method(" settings.Snapshot(\"dup\");\n var next = 1;\n other.Snapshot(\"dup\");"); + + var status = TryApply(source, 5, InlinePatchMode.Remove, "\"dup\"", "", out var newSource, out _, memberName: "Test"); + + await Assert.That(status).IsEqualTo(PatchStatus.Applied); + await Assert.That(newSource).IsEqualTo(Method(" var next = 1;\n other.Snapshot(\"dup\");")); + } + /// /// Shapes where the call is all its statement does and the statement cannot simply be lifted /// out: an if with no braces would take the next statement for its body, a line shared diff --git a/src/DiffEngine.Tests/KeptConnectionTests.cs b/src/DiffEngine.Tests/KeptConnectionTests.cs index aab1ecae..cf136f41 100644 --- a/src/DiffEngine.Tests/KeptConnectionTests.cs +++ b/src/DiffEngine.Tests/KeptConnectionTests.cs @@ -238,6 +238,207 @@ public async Task AnOwnerTooBusyToAnswerIsNotWaitedOnTwice() await Assert.That(ViewerClient.Tell(settle, owner.Port)).IsTrue(); } + static readonly ViewerMessage list = new(ViewerVerb.ListFull); + + /// + /// A window showing an owner's queue lists five times a second, and each was a connection. + /// They share one as the settles do, and it is not the settles' one: four connections for + /// twenty of each, the first of each kind being the ordinary exchange whose reply says the + /// owner keeps one. + /// + [Test] + public async Task ListingsShareAConnectionOfTheirOwn() + { + using var owner = new Owner(); + + for (var index = 0; index < 20; index++) + { + await Assert.That(ViewerClient.TrySend(list, out _, owner.Port)).IsTrue(); + await Assert.That(ViewerClient.Tell(settle, owner.Port)).IsTrue(); + } + + await Assert.That(owner.Heard.Count).IsEqualTo(40); + await Assert.That(owner.Accepted).IsEqualTo(4); + } + + /// + /// What a listing carries is the whole queue, and it arrives down the kept connection as it + /// does down its own, whatever is in it. + /// + [Test] + public async Task AListingArrivesWhole() + { + var patch = "a patch\n\nwith an empty line"; + using var owner = new Owner(_ => ViewerResponse.Listing( + [new("key\nwith a break", "name", "status", patch), new("second", "name", null, patch)])); + + await Assert.That(ViewerClient.TrySend(list, out var first, owner.Port)).IsTrue(); + await Assert.That(ViewerClient.TrySend(list, out _, owner.Port)).IsTrue(); + await Assert.That(ViewerClient.TrySend(list, out var second, owner.Port)).IsTrue(); + + // The first on a connection of its own, and the one it opened for the two after it + await Assert.That(owner.Accepted).IsEqualTo(2); + await Assert.That(second!.Build()).IsEqualTo(first!.Build()); + await Assert.That(second.Items.Count).IsEqualTo(2); + await Assert.That(second.Items[0].Key).IsEqualTo("key\nwith a break"); + await Assert.That(second.Items[1].Patch).IsEqualTo(patch); + } + + /// + /// An owner answers the requests of one connection in turn, so a listing it is slow over + /// must not be on the connection the settles use, and is not: a settle sent while a listing + /// is still waiting for its answer is answered. And a second listing sent then does not wait + /// for the first either. It goes on a connection of its own, as every listing used to. + /// + /// Nothing is timed. The listing is held by the owner until the settle and the second + /// listing have both come back, so either of them waiting behind it would be this test + /// hanging until the listing's own wait ran out, and the listing then failing. + /// + /// + [Test] + public async Task ASlowListingHoldsUpNeitherASettleNorAnotherListing() + { + using var release = new ManualResetEventSlim(); + using var arrived = new ManualResetEventSlim(); + using var owner = new Owner(_ => + { + if (_ is { Verb: ViewerVerb.List, Key: "slow" }) + { + arrived.Set(); + release.Wait(TimeSpan.FromMinutes(2)); + } + + return ViewerResponse.Success(); + }); + ViewerClient.Tell(settle, owner.Port); + ViewerClient.Tell(settle, owner.Port); + ViewerClient.TrySend(list, out _, owner.Port); + ViewerClient.TrySend(list, out _, owner.Port); + await Assert.That(owner.Accepted).IsEqualTo(4); + + // A thread of its own: it blocks until the owner answers, and the owner answers from the + // pool, being in this process + var slow = Task.Factory.StartNew( + () => ViewerClient.TrySend(new(ViewerVerb.List, "slow"), out _, owner.Port, TimeSpan.FromMinutes(1)), + Cancel.None, + TaskCreationOptions.LongRunning, + TaskScheduler.Default); + try + { + await Assert.That(arrived.Wait(TimeSpan.FromMinutes(1))).IsTrue(); + // Down the kept connection, or there would be a fifth + await Assert.That(owner.Accepted).IsEqualTo(4); + + await Assert.That(ViewerClient.Tell(settle, owner.Port)).IsTrue(); + await Assert.That(ViewerClient.TrySend(list, out _, owner.Port)).IsTrue(); + + await Assert.That(slow.IsCompleted).IsFalse(); + await Assert.That(owner.Accepted).IsEqualTo(5); + } + finally + { + release.Set(); + } + + await Assert.That(await slow).IsTrue(); + } + + /// + /// Only the listings. An accept written to a kept connection whose owner had just gone + /// would have to be sent again without knowing whether the first was carried out, so + /// whatever changes the queue is still a connection each. + /// + [Test] + public async Task ACommandIsStillAConnectionEach() + { + using var owner = new Owner(); + ViewerClient.TrySend(list, out _, owner.Port); + ViewerClient.TrySend(list, out _, owner.Port); + + for (var index = 0; index < 3; index++) + { + await Assert.That(ViewerClient.TrySend(new(ViewerVerb.Accept, "key"), out _, owner.Port)).IsTrue(); + await Assert.That(ViewerClient.TrySend(new(ViewerVerb.Discard, "key"), out _, owner.Port)).IsTrue(); + } + + await Assert.That(owner.Accepted).IsEqualTo(8); + } + + /// + /// An owner from before any of this is listed a connection each, as it is told. + /// + [Test] + public async Task AnOwnerThatPredatesItIsListedAConnectionEach() + { + using var owner = new OlderOwner(); + + for (var index = 0; index < 5; index++) + { + await Assert.That(ViewerClient.TrySend(list, out _, owner.Port)).IsTrue(); + } + + var requests = owner.Requests; + await Assert.That(requests.Count).IsEqualTo(5); + foreach (var request in requests) + { + await Assert.That(request).IsEqualTo(list.Build()); + } + } + + /// + /// A listing is how a window learns its owner has gone, so one sent down a connection the + /// owner closed has to come back saying so, and the one after an owner has taken the port + /// again has to reach it. + /// + [Test] + public async Task AListingFindsItsOwnerGoneAndTheNextOne() + { + int port; + using (var owner = new Owner()) + { + port = owner.Port; + ViewerClient.TrySend(list, out _, port); + await Assert.That(ViewerClient.TrySend(list, out _, port)).IsTrue(); + } + + await Assert.That(ViewerClient.TrySend(list, out _, port)).IsFalse(); + + using var next = new Owner(port: port); + await Assert.That(ViewerClient.TrySend(list, out _, port)).IsTrue(); + await Assert.That(ViewerClient.TrySend(list, out _, port)).IsTrue(); + await Assert.That(next.Heard.Count).IsEqualTo(2); + } + + /// + /// The kept connection is this process's alone. On .NET Framework a socket is inheritable and + /// a process started without ShellExecute is handed every inheritable handle, so a child a + /// test started held the connection too, and the owner went on keeping it until that child + /// exited, long after the test host had. + /// + /// What is asked is whether the handle is one a child would be given. What a child does with + /// it only shows once the host has gone without closing anything, which a test cannot do to + /// the process it runs in: closing the connection here shuts it down, and that reaches the + /// owner whoever else holds the handle. + /// + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task TheKeptConnectionIsNotOneAChildProcessIsGiven() + { + using var owner = new Owner(); + ViewerClient.Tell(settle, owner.Port); + await Assert.That(ViewerClient.Tell(settle, owner.Port)).IsTrue(); + + var handle = ViewerClient.KeptHandle; + await Assert.That(handle).IsNotNull(); + await Assert.That(GetHandleInformation(handle!.Value, out var flags)).IsTrue(); + // HANDLE_FLAG_INHERIT + await Assert.That(flags & 1).IsEqualTo(0); + } + + [DllImport("kernel32.dll")] + static extern bool GetHandleInformation(IntPtr handle, out int flags); + /// /// A queue owner on a port of the test's choosing, recording what it was sent. /// diff --git a/src/DiffEngine.Tests/ListenerTableTests.cs b/src/DiffEngine.Tests/ListenerTableTests.cs new file mode 100644 index 00000000..14103e42 --- /dev/null +++ b/src/DiffEngine.Tests/ListenerTableTests.cs @@ -0,0 +1,116 @@ +/// +/// Windows' table of listeners, read by a call of this library's own rather than out of what .NET +/// lists. The rows are picked apart by hand, so what is held to here is that they are picked +/// apart correctly: a listener of either family is found on its port, and a port with no +/// listener is answered no, rather than not answered. +/// +/// Each test asks about a port it holds itself, so nothing here depends on what else the machine +/// or the tests beside it are listening on. +/// +/// +public class ListenerTableTests +{ + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task AListenerIsFoundOnItsPort() + { + var listener = new TcpListener(IPAddress.Loopback, 0); + listener.Start(); + var port = ((IPEndPoint) listener.LocalEndpoint).Port; + try + { + await Assert.That(ListenerTable.Listeners(port)).IsTrue(); + await Assert.That(ListenerTable.Enumerated(port)).IsTrue(); + await Assert.That(ListenerTable.IsHeld(port)).IsTrue(); + } + finally + { + listener.Stop(); + } + } + + /// + /// The second table, whose rows are laid out another way. A viewer listens on IPv4's loopback, + /// but the table is asked by port alone, and something else on the port is a reason to connect + /// and find out what. + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task AnIPv6ListenerIsFoundOnItsPort() + { + if (!Socket.OSSupportsIPv6) + { + return; + } + + var listener = new TcpListener(IPAddress.IPv6Loopback, 0); + listener.Start(); + var port = ((IPEndPoint) listener.LocalEndpoint).Port; + try + { + await Assert.That(ListenerTable.Listeners(port)).IsTrue(); + await Assert.That(ListenerTable.IsHeld(port)).IsTrue(); + } + finally + { + listener.Stop(); + } + } + + /// + /// A port that is bound and not listening has nobody to accept a connection, which is the + /// answer the caller is spared two seconds by, and it takes both tables being read to their + /// ends to give it. Bound rather than free, so that nothing else can start listening on it + /// while the test asks. + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task ABoundPortThatIsNotListeningIsNotHeld() + { + using var holder = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp) + { + ExclusiveAddressUse = true + }; + holder.Bind(new IPEndPoint(IPAddress.Loopback, 0)); + var port = ((IPEndPoint) holder.LocalEndPoint!).Port; + + await Assert.That(ListenerTable.Listeners(port)).IsFalse(); + await Assert.That(ListenerTable.Enumerated(port)).IsFalse(); + await Assert.That(ListenerTable.IsHeld(port)).IsFalse(); + } + + /// + /// More listeners than the first buffer has room for, which is the path that asks again with + /// the size it was told. The last one opened is the one asked about, since a table cut short + /// would be missing its end. + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task ATableLargerThanTheFirstBufferIsReadWhole() + { + var listeners = new List(); + try + { + // 4096 bytes are 170 rows of 24 + for (var index = 0; index < 200; index++) + { + var listener = new TcpListener(IPAddress.Loopback, 0); + listener.Start(); + listeners.Add(listener); + } + + foreach (var listener in listeners) + { + var port = ((IPEndPoint) listener.LocalEndpoint).Port; + await Assert.That(ListenerTable.Listeners(port)).IsTrue(); + } + } + finally + { + foreach (var listener in listeners) + { + listener.Stop(); + } + } + } +} diff --git a/src/DiffEngine.Tests/SourceScanTests.cs b/src/DiffEngine.Tests/SourceScanTests.cs index dda0073d..984f9079 100644 --- a/src/DiffEngine.Tests/SourceScanTests.cs +++ b/src/DiffEngine.Tests/SourceScanTests.cs @@ -80,6 +80,177 @@ public async Task TheSuitesOwnSourceIsOneConsistentMap(string file) await Assert.That(spans).IsGreaterThan(10); } + /// + /// What an edit is made of here: everything that opens, closes or escapes a comment or a + /// literal in either language, line breaks of both kinds, and a backslash in front of one, + /// which is the one thing a lexer reads across a line break from in front of it. + /// + static readonly string[] pieces = + [ + "\"", "\"\"\"", "\"\"\"\"", "'", "\\", "\n", "\r\n", "\\\n", "'\\\n", "//", "/*", "*/", "(*", "*)", "(*)", + "#", "#if X", "$\"", "$$\"\"\"", "@\"", "{", "}", "{{", "}}", "``", " ", " ", "\t", "x", "'x'", "u8", "B", + "Verify(value)", ".Snapshot(\"old\")", ";" + ]; + + /// + /// A scan made from another by is the scan of the whole + /// text: every answer it gives, at every offset, after each of a run of edits made at random. + /// A batch carries its scan from patch to patch this way, so anything the two disagree on is + /// a patch told something other than what applying it by itself would have told it. + /// + [Test] + [Arguments("cs")] + [Arguments("fs")] + public async Task AScanMadeFromAnotherIsTheScanOfTheWholeText(string extension) + { + var language = SourceLanguage.ForFile($"file.{extension}"); + var problems = new List(); + for (var seed = 0; seed < 400 && problems.Count < 5; seed++) + { + var random = new Random(seed); + // Each language's sample, and the other's, which is mostly not this one's syntax + var other = seed % 4 == 3; + var text = SourceLanguage.NormalizeNewlines(other == (extension == "cs") ? fsharp : csharp); + var scan = language.Scan(text); + try + { + for (var step = 0; step < 25; step++) + { + // The line starts are carried only once they have been asked for + if (random.Next(2) == 0) + { + _ = scan.LineStarts; + } + + var edited = Edit(random, text); + var next = scan.Edited(edited); + scan.Dispose(); + scan = next; + text = edited; + using var whole = language.Scan(text); + if (Difference(scan, whole) is { } difference) + { + problems.Add($"seed {seed}, step {step}: {difference}"); + break; + } + } + } + finally + { + scan.Dispose(); + } + } + + await Assert.That(problems).IsEmpty(); + } + + /// + /// The same over a file of thousands of lines, where most of the scan is carried and the + /// lexer has a long way to go to rejoin it when an edit opens something. + /// + [Test] + [Arguments("InlinePatcherTests.cs")] + [Arguments("FsCompilerRoundTripTests.cs")] + public async Task AScanMadeFromAnotherOfTheSuitesOwnSourceIsTheScanOfTheWholeText(string file) + { + var text = await File.ReadAllTextAsync(Path.Combine(SourceDirectory, file)); + var random = new Random(file.Length); + var problems = new List(); + var scan = SourceLanguage.CSharp.Scan(text); + try + { + for (var step = 0; step < 60; step++) + { + if (step == 20) + { + _ = scan.LineStarts; + } + + var edited = Edit(random, text); + var next = scan.Edited(edited); + scan.Dispose(); + scan = next; + text = edited; + using var whole = SourceLanguage.CSharp.Scan(text); + if (Difference(scan, whole) is { } difference) + { + problems.Add($"step {step}: {difference}"); + break; + } + } + } + finally + { + scan.Dispose(); + } + + await Assert.That(problems).IsEmpty(); + } + + static string Edit(Random random, string text) + { + var start = random.Next(text.Length + 1); + var length = random.Next(3) == 0 ? 0 : random.Next(Math.Min(12, text.Length - start) + 1); + var replacement = new StringBuilder(); + for (var count = random.Next(4); count > 0; count--) + { + replacement.Append(pieces[random.Next(pieces.Length)]); + } + + return text[..start] + replacement + text[(start + length)..]; + } + + /// + /// The first thing two scans of one text answer differently, or null when there is none. + /// + static string? Difference(SourceScan made, SourceScan whole) + { + for (var offset = 0; offset <= whole.Source.Length + 1; offset++) + { + if (made.IsCode(offset) != whole.IsCode(offset)) + { + return $"{offset} is code to one and not the other"; + } + + if (made.TryGetSkip(offset, out var madeEnd) != whole.TryGetSkip(offset, out var wholeEnd) || + madeEnd != wholeEnd) + { + return $"the span starting at {offset} ends at {madeEnd} and at {wholeEnd}"; + } + + if (made.TryGetCommentSkip(offset, out madeEnd) != whole.TryGetCommentSkip(offset, out wholeEnd) || + madeEnd != wholeEnd) + { + return $"the comment starting at {offset} ends at {madeEnd} and at {wholeEnd}"; + } + + if (made.TryGetSkipEndingAt(offset, out var madeStart) != whole.TryGetSkipEndingAt(offset, out var wholeStart) || + madeStart != wholeStart) + { + return $"the span ending at {offset} starts at {madeStart} and at {wholeStart}"; + } + + if (made.TryGetCommentEndingAt(offset, out madeStart) != whole.TryGetCommentEndingAt(offset, out wholeStart) || + madeStart != wholeStart) + { + return $"the comment ending at {offset} starts at {madeStart} and at {wholeStart}"; + } + } + + if (!made.LineStarts.SequenceEqual(whole.LineStarts)) + { + return "the lines start in different places"; + } + + if (made.Eol != whole.Eol || + made.IndentUnit != whole.IndentUnit) + { + return "the line break or the indentation differs"; + } + + return null; + } + static string SourceDirectory { get; } = Path.GetDirectoryName(GetSourceFile())!; static string GetSourceFile([CallerFilePath] string path = "") => path; diff --git a/src/DiffEngine.Tests/TextDiffTests.cs b/src/DiffEngine.Tests/TextDiffTests.cs index d3ed7c63..f1489be7 100644 --- a/src/DiffEngine.Tests/TextDiffTests.cs +++ b/src/DiffEngine.Tests/TextDiffTests.cs @@ -377,6 +377,37 @@ public async Task AFewThousandLinesInAnotherOrderAreStillDiffedMinimally() await Assert.That(lines.Where(_ => _.Kind != DiffLineKind.Removed).Select(_ => _.Text).SequenceEqual(received)).IsTrue(); } + /// + /// Past what a diff may spend searching, lines that are each on both sides once are what it + /// goes by: the longest run of them still in order is kept, where the search alone left + /// nearly all of them changed. With every line on both sides once, as here, that run is the + /// most that can be unchanged, so the diff is minimal again however the lines were moved: + /// every line somewhere else, runs of them moved, and a few large blocks in another order. + /// + [Test] + [Arguments(30000, 1)] + [Arguments(40000, 50)] + [Arguments(60000, 3000)] + public async Task LinesInAnotherOrderKeepTheLongestRunStillInOrder(int count, int run) + { + var random = new Random(13); + var runs = Enumerable.Range(0, count).Chunk(run).ToArray(); + // One run in four somewhere else when there are many, and every run when there are few + var places = Enumerable.Range(0, runs.Length) + .Select(_ => run == 50 && random.Next(4) != 0 ? _ : random.Next(runs.Length)) + .ToArray(); + Array.Sort(places, runs); + var order = runs.SelectMany(_ => _).ToArray(); + var expected = Enumerable.Range(0, count).Select(_ => $"line {_}").ToList(); + var received = order.Select(_ => expected[_]).ToList(); + + var lines = TextDiff.Compute(string.Join("\n", expected), string.Join("\n", received)); + + await Assert.That(lines.Count(_ => _.Kind == DiffLineKind.Unchanged)).IsEqualTo(LongestIncreasingRun(order)); + await Assert.That(lines.Where(_ => _.Kind != DiffLineKind.Added).Select(_ => _.Text).SequenceEqual(expected)).IsTrue(); + await Assert.That(lines.Where(_ => _.Kind != DiffLineKind.Removed).Select(_ => _.Text).SequenceEqual(received)).IsTrue(); + } + // Patience sorting: the piles' tops stay sorted, and there are as many piles as the longest // increasing run is long. static int LongestIncreasingRun(int[] values) @@ -518,6 +549,114 @@ public void ASearchThatSettlesStillDiffsCorrectly() } } + /// + /// And searches that settle where there are elements to line up by, on sequences small enough + /// to check: some elements on each side once, some repeated, some on one side only, and a run + /// of any length taken as worth splitting by. What is left unchanged has to be the same + /// elements in the same order, whatever was split by an anchor and whatever by a search. + /// + [Test] + public void ASearchThatSettlesOnAnchorsStillDiffsCorrectly() + { + var random = new Random(14); + for (var iteration = 0; iteration < 20000; iteration++) + { + var alphabet = 2 + iteration % 80; + var a = RandomIds(random, random.Next(0, 60), alphabet); + var b = RandomIds(random, random.Next(0, 60), alphabet); + var budget = iteration % 40; + var minimumDepth = 1 + iteration % 5; + var anchorRun = 1 + iteration % 3; + var changedA = new bool[a.Length]; + var changedB = new bool[b.Length]; + + MyersDiff.Diff(a, b, changedA, changedB, budget, minimumDepth, anchorRun); + + var unchangedA = a.Where((_, index) => !changedA[index]); + var unchangedB = b.Where((_, index) => !changedB[index]); + if (!unchangedA.SequenceEqual(unchangedB)) + { + Assert.Fail($"budget: {budget} minimum: {minimumDepth} run: {anchorRun} a: {string.Join(",", a)} b: {string.Join(",", b)}"); + } + } + } + + /// + /// The same elements in another order, with nothing to spend on searching: every element is on + /// both sides once, so the longest run of them still in order is the most that can be + /// unchanged, and that is what is kept. + /// + [Test] + public void TheSameElementsInAnotherOrderKeepTheLongestRunWithNothingToSpend() + { + var random = new Random(15); + for (var iteration = 0; iteration < 5000; iteration++) + { + var a = Enumerable.Range(0, random.Next(2, 60)).ToArray(); + var b = a.OrderBy(_ => random.Next()).ToArray(); + var changedA = new bool[a.Length]; + var changedB = new bool[b.Length]; + + MyersDiff.Diff(a, b, changedA, changedB, 0, 1 + iteration % 5, 1); + + var unchangedA = a.Where((_, index) => !changedA[index]).ToList(); + var unchangedB = b.Where((_, index) => !changedB[index]); + if (!unchangedA.SequenceEqual(unchangedB) || + unchangedA.Count != LongestIncreasingRun(b)) + { + Assert.Fail($"kept {unchangedA.Count} of {LongestIncreasingRun(b)} a: {string.Join(",", a)} b: {string.Join(",", b)}"); + } + } + } + + /// + /// What the anchors are: elements on each side exactly once, in ascending places on both + /// sides, and as many of them as the longest such run has, checked against the quadratic way + /// of finding one. + /// + [Test] + public void AnchorsAreTheLongestRunOfUniqueElementsInOrder() + { + var random = new Random(16); + for (var iteration = 0; iteration < 5000; iteration++) + { + var alphabet = 2 + iteration % 80; + var a = RandomIds(random, random.Next(0, 60), alphabet); + var b = RandomIds(random, random.Next(0, 60), alphabet); + var unique = a + .Where(_ => a.Count(value => value == _) == 1 && b.Count(value => value == _) == 1) + .Select(_ => Array.IndexOf(b, _)) + .ToArray(); + var longest = LongestIncreasingRun(unique); + + var count = LineAnchors.Find(a, b, 1, out var xs, out var ys); + + if (count != longest) + { + Assert.Fail($"found {count} of {longest} a: {string.Join(",", a)} b: {string.Join(",", b)}"); + } + + for (var index = 0; index < count; index++) + { + var x = xs![index]; + var y = ys![index]; + if (a[x] != b[y] || + a.Count(_ => _ == a[x]) != 1 || + b.Count(_ => _ == a[x]) != 1 || + (index > 0 && (x <= xs[index - 1] || y <= ys[index - 1]))) + { + Assert.Fail($"anchor {index} at {x},{y} a: {string.Join(",", a)} b: {string.Join(",", b)}"); + } + } + + // One more than the longest is not there to find, which comes back as none + if (LineAnchors.Find(a, b, longest + 1, out _, out _) != 0) + { + Assert.Fail($"a run longer than {longest} a: {string.Join(",", a)} b: {string.Join(",", b)}"); + } + } + } + /// /// The same sequences with more to spend than any of them needs are diffed as they always /// were, so the budget costs nothing where it does not apply. diff --git a/src/DiffEngine.Tests/ViewerClientUnownedTests.cs b/src/DiffEngine.Tests/ViewerClientUnownedTests.cs index 945400d8..7e1bc496 100644 --- a/src/DiffEngine.Tests/ViewerClientUnownedTests.cs +++ b/src/DiffEngine.Tests/ViewerClientUnownedTests.cs @@ -17,19 +17,32 @@ public class ViewerClientUnownedTests // static constructor that runs on first touching a static field, and TheMemoryExpires sets the // value before it touches one, so running first it captured Zero for every test after it. static TimeSpan recheckUnownedAfter; + static TimeSpan recheckUnlistedAfter; [Before(Class)] - public static void Remember() => + public static void Remember() + { recheckUnownedAfter = ViewerClient.RecheckUnownedAfter; + recheckUnlistedAfter = ViewerClient.RecheckUnlistedAfter; + } + /// + /// What the listener table found stands for a second, which a loaded machine can spend + /// between two lines of a test. Most of these are about there being a memory at all, so for + /// them it stands as long as a connect's answer does, and the ones about how long say so. + /// [Before(Test)] - public void Forget() => + public void Forget() + { + ViewerClient.RecheckUnlistedAfter = recheckUnownedAfter; ViewerClient.ForgetUnowned(); + } [After(Test)] public void Restore() { ViewerClient.RecheckUnownedAfter = recheckUnownedAfter; + ViewerClient.RecheckUnlistedAfter = recheckUnlistedAfter; ViewerClient.ForgetUnowned(); } @@ -102,6 +115,7 @@ public async Task AProbeFindingNobodyIsRemembered() public async Task TheMemoryExpires() { ViewerClient.RecheckUnownedAfter = TimeSpan.Zero; + ViewerClient.RecheckUnlistedAfter = TimeSpan.Zero; var port = FreePort(); await Assert.That(ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true)).IsFalse(); @@ -110,6 +124,62 @@ public async Task TheMemoryExpires() await Assert.That(owner.Heard.Count).IsEqualTo(1); } + /// + /// A tray started after the test process, with everything as shipped: the telling sends find + /// it within moments, with nothing asking on their behalf. Where the listener table is what + /// said the port was empty, asking it again costs a tenth of a millisecond, so there is no + /// reason to go ten minutes on the old answer - which is how long the tray used to hear + /// nothing of the run's settles and moves. + /// + /// Waited for rather than timed. What is held to is that it happens while a test would still + /// be running, and the limit is far short of the ten minutes it replaces. + /// + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task AnOwnerStartedLaterIsFoundByTheTellingSends() + { + ViewerClient.RecheckUnlistedAfter = recheckUnlistedAfter; + var port = FreePort(); + await Assert.That(ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true)).IsFalse(); + + using var owner = new Owner(port); + var elapsed = Stopwatch.StartNew(); + var found = false; + while (!found && + elapsed.Elapsed < TimeSpan.FromSeconds(60)) + { + found = ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true); + if (!found) + { + await Task.Delay(50); + } + } + + await Assert.That(found).IsTrue(); + } + + /// + /// The shorter wait is only for what the table said. A port a connect found nobody on - here + /// because the table could not be read - is one where asking again is the connect again, two + /// seconds of it on Windows, so that answer stands as it always did. + /// + [Test] + public async Task APortAConnectFoundEmptyIsNotAskedAboutAgainSoSoon() + { + ViewerClient.RecheckUnlistedAfter = TimeSpan.Zero; + var port = FreePort(); + using var unreadable = new Lookup(port) + { + Unreadable = true + }; + await Assert.That(ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true)).IsFalse(); + + using var owner = new Owner(port); + await Assert.That(ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true)).IsFalse(); + await Assert.That(owner.Heard).IsEmpty(); + } + /// /// Per port, or a test suite's dead ephemeral port would silence the sends to its live one. /// @@ -163,23 +233,30 @@ public async Task AskingIsTheDefault() /// operating system knows nobody is listening without anything being connected, so where that /// refusal is slow it is asked first. /// - /// A bound of half what the connect takes to be refused, around something that takes well - /// under a millisecond. + /// Shown by what the send does rather than by how long it takes, which on a loaded two core + /// machine is whatever the machine says. An owner is listening on the port and the table is + /// made to say nobody is: a send that connected would be answered, and heard, and one that + /// took the table's word is neither. That the real table says so of a port nobody listens on + /// is ListenerTableTests' half. /// /// [Test] [RunOn(TUnit.Core.Enums.OS.Windows)] public async Task APortNobodyHoldsIsNotWaitedOn() { - var port = FreePort(); + using var owner = new Owner(); + using var unlisted = new Lookup(owner.Port) + { + Unlisted = true + }; - var elapsed = Stopwatch.StartNew(); - var sent = ViewerClient.TrySend(settle, out _, port, skipIfUnowned: true); - elapsed.Stop(); + var sent = ViewerClient.TrySend(settle, out _, owner.Port, skipIfUnowned: true); await Assert.That(sent).IsFalse(); - await Assert.That(elapsed.Elapsed).IsLessThan(TimeSpan.FromSeconds(1)); - await Assert.That(ViewerClient.FoundUnowned(port)).IsTrue(); + await Assert.That(unlisted.Asked).IsEqualTo(1); + await Assert.That(owner.Heard).IsEmpty(); + await Assert.That(owner.Accepted).IsEqualTo(0); + await Assert.That(ViewerClient.FoundUnowned(owner.Port)).IsTrue(); } /// @@ -187,15 +264,34 @@ public async Task APortNobodyHoldsIsNotWaitedOn() [RunOn(TUnit.Core.Enums.OS.Windows)] public async Task APortNobodyHoldsIsNotWaitedOnAsync() { - var port = FreePort(); + using var owner = new Owner(); + using var unlisted = new Lookup(owner.Port) + { + Unlisted = true + }; - var elapsed = Stopwatch.StartNew(); - var outcome = await ViewerClient.SendAsync(settle, Cancel.None, port, skipIfUnowned: true); - elapsed.Stop(); + var outcome = await ViewerClient.SendAsync(settle, Cancel.None, owner.Port, skipIfUnowned: true); await Assert.That(outcome).IsEqualTo(SendOutcome.NoOwner); - await Assert.That(elapsed.Elapsed).IsLessThan(TimeSpan.FromSeconds(1)); - await Assert.That(ViewerClient.FoundUnowned(port)).IsTrue(); + await Assert.That(unlisted.Asked).IsEqualTo(1); + await Assert.That(owner.Heard).IsEmpty(); + await Assert.That(owner.Accepted).IsEqualTo(0); + await Assert.That(ViewerClient.FoundUnowned(owner.Port)).IsTrue(); + } + + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task APortNobodyHoldsIsNotProbed() + { + using var owner = new Owner(); + using var unlisted = new Lookup(owner.Port) + { + Unlisted = true + }; + + await Assert.That(ViewerClient.IsOwned(owner.Port)).IsFalse(); + await Assert.That(owner.Accepted).IsEqualTo(0); } /// @@ -304,11 +400,64 @@ await Assert.That(async () => await ViewerClient.SendAsync(settle, cancel.Token, await Assert.That(ViewerClient.FoundUnowned(port)).IsFalse(); } + /// + /// The memory is of ports nobody is on, and only two failures of a connect say that: it was + /// refused, or it was never answered. A machine with no ports left to connect from fails + /// every connect too, with an owner listening the whole time, and remembering that as an + /// empty port silenced every settle and move for ten minutes. + /// + [Test] + public async Task OnlyARefusalSaysNobodyIsThere() + { + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.ConnectionRefused))).IsTrue(); + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.TimedOut))).IsTrue(); + // How the synchronous connect reports it, which waits on a task + await Assert.That(ViewerClient.NobodyThere(new AggregateException(Failed(SocketError.ConnectionRefused)))).IsTrue(); + + // What running out looks like: the first two on Windows, the third on Linux + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.NoBufferSpaceAvailable))).IsFalse(); + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.AddressAlreadyInUse))).IsFalse(); + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.AddressNotAvailable))).IsFalse(); + await Assert.That(ViewerClient.NobodyThere(Failed(SocketError.TooManyOpenSockets))).IsFalse(); + await Assert.That(ViewerClient.NobodyThere(new AggregateException(Failed(SocketError.NoBufferSpaceAvailable)))).IsFalse(); + await Assert.That(ViewerClient.NobodyThere(new ObjectDisposedException("socket"))).IsFalse(); + + static SocketException Failed(SocketError error) => + new((int) error); + } + + /// + /// The same through a real connect that fails without being refused, which a machine out of + /// ports cannot be made to give without running the machine out. Port zero does: Windows + /// fails a connect to it at once, as an address that is not valid, and that is no more a + /// statement about who is listening than a full port table is. The table is made unreadable + /// so that the connect is reached at all. + /// + [Test] + [RunOn(TUnit.Core.Enums.OS.Windows)] + public async Task AConnectThatCouldNotBeMadeSaysNothingAboutThePort() + { + using var unreadable = new Lookup(0) + { + Unreadable = true + }; + + await Assert.That(ViewerClient.TrySend(settle, out _, 0, skipIfUnowned: true)).IsFalse(); + await Assert.That(ViewerClient.FoundUnowned(0)).IsFalse(); + + await Assert.That(await ViewerClient.SendAsync(settle, Cancel.None, 0, skipIfUnowned: true)) + .IsEqualTo(SendOutcome.NoOwner); + await Assert.That(ViewerClient.FoundUnowned(0)).IsFalse(); + + await Assert.That(ViewerClient.IsOwned(0)).IsFalse(); + await Assert.That(ViewerClient.FoundUnowned(0)).IsFalse(); + } + /// /// Stands in front of the listener table for the ports a test names, counting how often each - /// was asked about and, when told to, failing the way a table that cannot be read does. Every - /// other port goes through untouched, since the tests beside this one are asking about theirs - /// at the same time. + /// was asked about and, when told to, failing the way a table that cannot be read does, or + /// saying nobody listens there whoever does. Every other port goes through untouched, since + /// the tests beside this one are asking about theirs at the same time. /// sealed class Lookup : IDisposable { @@ -324,6 +473,8 @@ public Lookup(params int[] ports) public bool Unreadable { get; init; } + public bool Unlisted { get; init; } + public int Asked => Volatile.Read(ref asked); bool Answer(int port) @@ -339,6 +490,11 @@ bool Answer(int port) throw new System.Net.NetworkInformation.NetworkInformationException(); } + if (Unlisted) + { + return false; + } + return previous(port); } @@ -396,6 +552,11 @@ public Owner(int port = 0) public int Port => server.Port; + /// + /// How many connections were made, which a probe is and says nothing down. + /// + public int Accepted => server.Accepted; + public IReadOnlyList Heard { get diff --git a/src/DiffEngine.Tests/ViewerLaunchGateTests.cs b/src/DiffEngine.Tests/ViewerLaunchGateTests.cs index 5c0b526e..5b73640b 100644 --- a/src/DiffEngine.Tests/ViewerLaunchGateTests.cs +++ b/src/DiffEngine.Tests/ViewerLaunchGateTests.cs @@ -375,6 +375,108 @@ public async Task TheDefaultSlotCheckReadsMaxInstance() } } + /// + /// A slot is for a window, and a viewer that could not be run opened none. With the slot + /// kept, a cap of one was used up by the copy that exited on its arguments, and the launch + /// after it - of a viewer that would have run - was told too many diff tools were open. + /// + [Test] + public async Task AFailedLaunchGivesItsSlotBack() + { + var viewer = new FakeViewer(); + try + { + DiffRunner.MaxInstancesToLaunch(1); + MaxInstance.ResetCount(); + + var failed = ViewerLaunchGate.Launch( + retry: () => true, + launch: () => Exited(2), + isOwned: () => false); + var notStarted = ViewerLaunchGate.Launch( + retry: () => true, + launch: () => null, + isOwned: () => false); + var outcome = ViewerLaunchGate.Launch( + retry: () => true, + launch: viewer.Start, + isOwned: viewer.IsUp); + + await Assert.That(failed).IsEqualTo(ViewerLaunchOutcome.Failed); + await Assert.That(notStarted).IsEqualTo(ViewerLaunchOutcome.Failed); + await Assert.That(outcome).IsEqualTo(ViewerLaunchOutcome.Launched); + await Assert.That(viewer.Starts).IsEqualTo(1); + } + finally + { + MaxInstance.ResetAppDomainValue(); + MaxInstance.ResetCount(); + } + } + + /// + [Test] + public async Task AFailedLaunchGivesItsSlotBackAsync() + { + var viewer = new FakeViewer(); + try + { + DiffRunner.MaxInstancesToLaunch(1); + MaxInstance.ResetCount(); + + var failed = await ViewerLaunchGate.LaunchAsync( + retry: () => Task.FromResult(true), + launch: () => Task.FromResult(Exited(2)), + Cancel.None, + isOwned: () => false); + var outcome = await ViewerLaunchGate.LaunchAsync( + retry: () => Task.FromResult(true), + launch: () => Task.FromResult(viewer.Start()), + Cancel.None, + isOwned: viewer.IsUp); + + await Assert.That(failed).IsEqualTo(ViewerLaunchOutcome.Failed); + await Assert.That(outcome).IsEqualTo(ViewerLaunchOutcome.Launched); + await Assert.That(viewer.Starts).IsEqualTo(1); + } + finally + { + MaxInstance.ResetAppDomainValue(); + MaxInstance.ResetCount(); + } + } + + /// + /// A launch that worked keeps its slot, which is what the cap counts. + /// + [Test] + public async Task ALaunchThatOpenedAViewerKeepsItsSlot() + { + var viewer = new FakeViewer(); + try + { + DiffRunner.MaxInstancesToLaunch(1); + MaxInstance.ResetCount(); + + ViewerLaunchGate.Launch( + retry: () => true, + launch: viewer.Start, + isOwned: viewer.IsUp); + var outcome = ViewerLaunchGate.Launch( + retry: () => true, + launch: viewer.Start, + isOwned: () => false); + + await Assert.That(outcome).IsEqualTo(ViewerLaunchOutcome.Capped); + await Assert.That(viewer.Starts).IsEqualTo(1); + } + finally + { + MaxInstance.ResetAppDomainValue(); + MaxInstance.ResetCount(); + } + } + /// /// The real probe, against a real bound port, so the default the call sites rely on is not /// only ever exercised through a stand-in. diff --git a/src/DiffEngine/DiffRunner_Inline.cs b/src/DiffEngine/DiffRunner_Inline.cs index 4b135735..62537248 100644 --- a/src/DiffEngine/DiffRunner_Inline.cs +++ b/src/DiffEngine/DiffRunner_Inline.cs @@ -127,9 +127,10 @@ internal static InlineResult InlineResultFor(ViewerLaunchOutcome outcome) => /// Drops a pending inline snapshot from the viewer's queue, for when a previously failing test /// starts passing. Does nothing when no viewer is running - and cheaply, since this is called /// once per passing inline verification: a port found with nothing listening is taken as - /// still unowned for ten minutes rather than connected to again, because on Windows a refused + /// still unowned for a while rather than connected to again, because on Windows a refused /// loopback connection takes two seconds rather than none, and a green run of a few hundred - /// inline tests was spending minutes on refusals. + /// inline tests was spending minutes on refusals. A second where the operating system's table + /// of listeners is what said so, and ten minutes where it took a connect to find out. /// /// Carries this process's framework so a multi-targeted run only settles its own variant of a /// conflicted entry; the other framework's differing content stays pending. diff --git a/src/DiffEngine/Inline/CsLanguage.cs b/src/DiffEngine/Inline/CsLanguage.cs index 455930f5..2c4a1e6f 100644 --- a/src/DiffEngine/Inline/CsLanguage.cs +++ b/src/DiffEngine/Inline/CsLanguage.cs @@ -22,12 +22,19 @@ public override bool TryParse(string expression, [NotNullWhen(true)] out string? internal override bool IsIdentifierChar(char ch) => char.IsLetterOrDigit(ch) || ch == '_'; - internal override SourceScan Scan(string source) + internal override void Lex(SourceScan scan, int from) { - var scan = new SourceScan(this, source); - var index = 0; + var source = scan.Source; + var rejoinFrom = scan.RejoinFrom; + var index = from; while (index < source.Length) { + if (index >= rejoinFrom && + scan.TryRejoin(index)) + { + return; + } + var start = index; switch (source[index]) { @@ -81,8 +88,6 @@ internal override SourceScan Scan(string source) // Code, which is whatever the map was not told otherwise about index++; } - - return scan; } /// diff --git a/src/DiffEngine/Inline/FsLanguage.cs b/src/DiffEngine/Inline/FsLanguage.cs index 0d10bf98..6708ece5 100644 --- a/src/DiffEngine/Inline/FsLanguage.cs +++ b/src/DiffEngine/Inline/FsLanguage.cs @@ -40,6 +40,9 @@ public override bool TryParse(string expression, [NotNullWhen(true)] out string? internal override bool SuppliesArgumentExpressions => false; internal override bool IsIdentifierChar(char ch) => + IsNameChar(ch); + + static bool IsNameChar(char ch) => char.IsLetterOrDigit(ch) || ch == '_' || ch == '\''; internal override bool IsTypeArgumentChar(char ch) => @@ -47,12 +50,19 @@ internal override bool IsTypeArgumentChar(char ch) => // Tuple types (Foo) and statically resolved type parameters (^T) ch is '*' or '^'; - internal override SourceScan Scan(string source) + internal override void Lex(SourceScan scan, int from) { - var scan = new SourceScan(this, source); - var index = 0; + var source = scan.Source; + var rejoinFrom = scan.RejoinFrom; + var index = from; while (index < source.Length) { + if (index >= rejoinFrom && + scan.TryRejoin(index)) + { + return; + } + var start = index; switch (source[index]) { @@ -124,8 +134,6 @@ internal override SourceScan Scan(string source) // Code, which is whatever the map was not told otherwise about index++; } - - return scan; } internal override bool IsDeclaration(SourceScan scan, int nameStart) @@ -466,8 +474,11 @@ static bool TrySkipStringLike(string source, ref int index) if (quotes >= 3 && !verbatim) { // Triple quoted: verbatim, so there are no escapes to consider and no delimiter to - // widen. It ends at the next run of three quotes, interpolation holes included + // widen. Its text ends at the next run of three quotes. A hole is not text: it may + // hold a string or a comment with three quotes of its own, which end nothing var search = cursor + 3; + // As many braces open a hole as there are dollars in front, and fewer are text + var dollars = interpolated ? cursor - index : 0; while (search < source.Length) { if (source[search] == '"' && @@ -477,6 +488,42 @@ static bool TrySkipStringLike(string source, ref int index) return true; } + if (dollars > 0 && + source[search] == '{') + { + var braces = 1; + while (search + braces < source.Length && + source[search + braces] == '{') + { + braces++; + } + + // With one dollar a doubled brace is a brace, so only an odd one out opens. + // With more, the last of a run that is long enough do, and the ones in front + // of them are text + var opens = dollars == 1 ? braces % 2 == 1 : braces >= dollars; + search += braces; + if (!opens) + { + continue; + } + + search--; + if (!TrySkipHole(source, ref search)) + { + index = source.Length; + return true; + } + + // The rest of the braces that close it + for (var closing = 1; closing < dollars && search < source.Length && source[search] == '}'; closing++) + { + search++; + } + + continue; + } + search++; } @@ -511,8 +558,9 @@ static bool TrySkipStringLike(string source, ref int index) if (!verbatim && ch == '\\') { - // Escape or line continuation: either way the next character is content - cursor += 2; + // Escape or line continuation: either way the next character is content. Except a + // brace where braces open holes, which no backslash escapes + cursor += interpolated && cursor + 1 < source.Length && source[cursor + 1] == '{' ? 1 : 2; continue; } @@ -548,7 +596,17 @@ static bool TrySkipStringLike(string source, ref int index) return true; } - // cursor at '{' of an interpolation hole; skips past the matching '}' + /// + /// Steps past an interpolation hole, from its { to past the } that closes it. + /// + /// A hole is code, and is lexed as code is: a brace in a char literal, a comment, a string or + /// a backticked name is not one of the hole's own, and a quote in a char literal or a comment + /// opens no string. Counting every brace and taking every quote for a string, as this did, + /// $"{'{'}" never closed and $"{'"'}" closed on a string that ran to the end of + /// the file, and the calls under either were inside a literal. FsCompilerRoundTripTests + /// holds each shape to the compiler. + /// + /// static bool TrySkipHole(string source, ref int cursor) { var depth = 1; @@ -565,6 +623,36 @@ static bool TrySkipHole(string source, ref int cursor) cursor++; } + continue; + case '\'': + // The rule code has: a tick after a name is part of the name + if (IsNameChar(source[cursor - 1]) || + !TrySkipCharLiteral(source, ref cursor)) + { + cursor++; + } + + continue; + case '(': + if (!TrySkipBlockComment(source, ref cursor)) + { + cursor++; + } + + continue; + case '/': + if (!TrySkipLineComment(source, ref cursor)) + { + cursor++; + } + + continue; + case '`': + if (!TrySkipQuotedIdentifier(source, ref cursor)) + { + cursor++; + } + continue; case '{': depth++; diff --git a/src/DiffEngine/Inline/InlineApplier.cs b/src/DiffEngine/Inline/InlineApplier.cs index b81d32a0..140bacb1 100644 --- a/src/DiffEngine/Inline/InlineApplier.cs +++ b/src/DiffEngine/Inline/InlineApplier.cs @@ -32,13 +32,14 @@ public static InlineApplyResult Apply(InlinePatch patch) => /// at a time, the whole file is read, lexed and written again for every patch, and the write /// is where the time goes. A file that has just been written is scanned by whatever watches /// the drive before the next thing can open it, and for five hundred snapshots in one ten - /// thousand line file that came to half a minute. The lexing is still once per patch, since - /// each starts from different source. + /// thousand line file that came to half a minute. The lexing is once a file too, and around + /// each edit after that (). /// /// - /// So a write that fails fails every patch it was carrying, and each says so. The patches - /// after the first of those say so too, whatever they were judged to be, because what they - /// were judged against was never written. The file is left as it was. + /// So a write that fails fails every patch it was carrying, and each says so. A patch after + /// the first of those that made no edit was judged against source that was never written, so + /// it is asked again of the file as it was read, and says the write failed only where it would + /// have had to edit that. The file is left as it was. /// /// public static IReadOnlyList ApplyAll(IReadOnlyList patches) => @@ -176,6 +177,14 @@ static bool TryResolve(InlinePatch patch, out string fullPath, [NotNullWhen(fals static InlineApplyResult[] Run(string fullPath, IReadOnlyList patches, bool write, bool anchorOnly, Action replace) { var normalizedPath = fullPath.ToLowerInvariant(); + // A dry run of a file an earlier one read, and nothing has written since, is answered + // from what was read then, with no lock: there is nothing here for one to protect + if (!write && + Probed.Find(normalizedPath, fullPath) is { } lexed) + { + return Judge(lexed, patches, anchorOnly, fullPath); + } + lock (gates.GetOrAdd(normalizedPath, static _ => new())) { // Answered rather than thrown, as everything else here is. A mutex this process may @@ -210,7 +219,7 @@ static InlineApplyResult[] Run(string fullPath, IReadOnlyList patch return All(patches, InlineApplyResult.Failed($"Timed out waiting for the inline patch mutex for: {fullPath}")); } - return LockedApply(fullPath, patches, write, anchorOnly, replace); + return LockedApply(fullPath, normalizedPath, patches, write, anchorOnly, replace); } finally { @@ -233,7 +242,7 @@ static InlineApplyResult[] All(IReadOnlyList patches, InlineApplyRe return results; } - static InlineApplyResult[] LockedApply(string fullPath, IReadOnlyList patches, bool write, bool anchorOnly, Action replace) + static InlineApplyResult[] LockedApply(string fullPath, string normalizedPath, IReadOnlyList patches, bool write, bool anchorOnly, Action replace) { // Asked here rather than before the lock, because the swap at the end of this method takes // the path away for the instant it takes to rename over it. Asked outside, an applier @@ -244,6 +253,8 @@ static InlineApplyResult[] LockedApply(string fullPath, IReadOnlyList + /// What each patch from the first edit on is told when the one write failed. + /// + /// Nothing from that edit on reached the file, and every answer after it was about source + /// that held it: one already applied only because an earlier patch here had written the same + /// literal, one not found only because an earlier patch had taken its anchor. So a patch that + /// edited reports the write, and an entry that reports a failure is kept for another try. + /// + /// + /// One that made no edit is asked again, of the file as it was read, which is the file as it + /// still is. Already applied there, or not found there, is true whatever became of the write + /// and is what it is told: they all used to report the write, so a snapshot that was in the + /// source all along stayed queued as a failure, and one whose call site had gone was not + /// told to re-run. Where it would have had to edit that file, it needed the write as much as + /// the others, and reports it. + /// + /// + static void AfterAFailedWrite( + SourceLanguage language, + string read, + IReadOnlyList patches, + string fullPath, + InlineApplyResult[] results, + int firstToWrite, + InlineApplyResult failed) + { + SourceScan? scan = null; + try + { + for (var index = firstToWrite; index < results.Length; index++) + { + if (results[index].Status == InlineApplyStatus.Applied) + { + results[index] = failed; + continue; + } + + scan ??= language.Scan(read); + var asRead = Judge(scan, patches[index], false, fullPath, out _); + results[index] = asRead.Status == InlineApplyStatus.Applied ? failed : asRead; + } + } + catch (Exception) + { + // The source could not be lexed again, which leaves the write as all there is to say for (var index = firstToWrite; index < results.Length; index++) { results[index] = failed; } } + finally + { + scan?.Dispose(); + } + } + + /// + /// Applies the patches of one file to its source in memory, each to what the one before it + /// left, and says what became of each. Returns the first patch whose edit a write has to + /// carry, or -1 when none made one. + /// + /// The source is lexed once, for the first patch, and the scan is carried from each patch + /// that edits to the next (), which lexes only around the + /// edit. Lexed whole for every patch, five hundred snapshots in a 600 KB file were most of a + /// second of patching around their one write. What each patch is told is what it would have + /// been told over a scan of the whole text, since the scan it is given is that scan. + /// + /// + /// The language the source is in. + /// The source, replaced by each patch that edits. + /// The patches, in the order they are to be applied. + /// The file, for a failure to name. + /// Filled with an outcome for each patch. + internal static int PatchInTurn( + SourceLanguage language, + ref string source, + IReadOnlyList patches, + string fullPath, + InlineApplyResult[] results) + { + // The first patch whose edit the write has to carry, or -1 while there is none + var firstToWrite = -1; + SourceScan? scan = null; + try + { + for (var index = 0; index < results.Length; index++) + { + try + { + scan ??= language.Scan(source); + } + catch (Exception exception) + { + results[index] = InlineApplyResult.Failed($"Failed to patch: {fullPath}", exception); + continue; + } + + results[index] = Judge(scan, patches[index], false, fullPath, out var newSource); + if (results[index].Status != InlineApplyStatus.Applied) + { + continue; + } + + // The next patch is applied to this one's result, as it would have been to the + // file this one had written + source = newSource; + if (firstToWrite < 0) + { + firstToWrite = index; + } + + var previous = scan; + scan = null; + try + { + if (index + 1 < results.Length) + { + scan = previous.Edited(newSource); + } + } + catch (Exception) + { + // Left for the next patch to lex whole, which is what it used to do. A scan + // that could not be carried is no reason to fail a patch that has applied + } + finally + { + previous.Dispose(); + } + } + } + finally + { + scan?.Dispose(); + } + + return firstToWrite; + } + + /// + /// What the patcher makes of one patch, over source that has been lexed, as the outcome a + /// caller is given. is the edited source where the outcome is + /// Applied, which is all a dry run wanted to know and what a write has still to carry out. + /// + static InlineApplyResult Judge(SourceScan scan, InlinePatch patch, bool anchorOnly, string fullPath, out string newSource) + { + PatchStatus status; + string failReason; + try + { + status = InlinePatcher.TryApply( + scan, + patch.LineHint, + patch.Mode, + patch.OriginalExpression, + patch.OriginalValue, + patch.MemberName, + patch.EntryPoints, + anchorOnly, + SourceLanguage.NormalizeNewlines(patch.NewContent), + out newSource, + out failReason); + } + catch (Exception exception) + { + // A patcher defect on some shape of source, reported against the file it met it in + // rather than thrown at whichever surface was accepting + newSource = ""; + return InlineApplyResult.Failed($"Failed to patch: {fullPath}", exception); + } + + // Every reason a patch can be refused for has been asked by this point. For one that + // none of them held for, all that remains is the write, which is the one step a dry run + // may not take + return status switch + { + PatchStatus.AlreadyApplied => InlineApplyResult.AlreadyApplied, + PatchStatus.NotFound => InlineApplyResult.NotFound(failReason), + _ => InlineApplyResult.Applied + }; + } + + /// + /// A dry run: every patch asked of the same source, which none of them changes. + /// + static InlineApplyResult[] Judge(SourceScan scan, IReadOnlyList patches, bool anchorOnly, string fullPath) + { + var results = new InlineApplyResult[patches.Count]; + for (var index = 0; index < results.Length; index++) + { + results[index] = Judge(scan, patches[index], anchorOnly, fullPath, out _); + } return results; } + /// + /// The source files a dry run has read, lexed, for the next dry run of the same file. + /// + /// A test run asks once for each call site it has not seen before, + /// and each asking read, decoded and lexed the whole file: 0.7 s for the five hundred call + /// sites of a 600 KB file, to be told five hundred things about the same text. Nothing is + /// written by a dry run, so what one read stands for as long as the file does, and the file's + /// length and write time say whether it has. + /// + /// + /// A write time says nothing about a second write inside the same tick of the file system's + /// clock, which is two seconds on some. So only a file that had been left alone for longer + /// than that when it was read is kept: any write after the read then has a later time. A + /// file being edited is read each time, as it was. + /// + /// + /// A few files, since a run goes through its source files a class at a time and tests run + /// side by side. A scan that is dropped is left to the collector and not disposed, because + /// another thread may be part way through asking it something. + /// + /// + static class Probed + { + const int capacity = 4; + + static readonly TimeSpan settled = TimeSpan.FromSeconds(3); + + static readonly List entries = []; + + static long uses; + + sealed class Entry(string path, (long Length, DateTime Written) stamp, SourceScan scan) + { + public string Path => path; + public (long Length, DateTime Written) Stamp => stamp; + public SourceScan Scan => scan; + public long Used; + } + + /// + /// What tells one state of a file from the next, or null for a file that cannot be asked. + /// + public static (long Length, DateTime Written)? Stamp(string fullPath) + { + try + { + var info = new FileInfo(fullPath); + if (!info.Exists) + { + return null; + } + + return (info.Length, info.LastWriteTimeUtc); + } + catch (Exception exception) + when (exception is IOException or UnauthorizedAccessException) + { + return null; + } + } + + /// + /// The scan kept for a file, where the file is still as it was when that was read. + /// + public static SourceScan? Find(string normalizedPath, string fullPath) + { + Entry? entry; + lock (entries) + { + entry = entries.Find(_ => _.Path == normalizedPath); + } + + if (entry is null) + { + return null; + } + + // Asked outside the lock, since it is the file system that answers + var stamp = Stamp(fullPath); + lock (entries) + { + if (stamp is null || + stamp.Value != entry.Stamp) + { + entries.Remove(entry); + return null; + } + + entry.Used = ++uses; + return entry.Scan; + } + } + + /// + /// Keeps a scan of a file that was read while it looked like . + /// False when it is not kept, and is still the caller's to dispose. + /// + public static bool Keep(string normalizedPath, string fullPath, (long Length, DateTime Written) stamp, SourceScan scan) + { + // Written too recently to tell a later write by its time, written since it was read, + // or dated in the future, which the same test refuses + if (DateTime.UtcNow - stamp.Written < settled || + Stamp(fullPath) is not { } after || + after != stamp) + { + return false; + } + + // Asked for here, so nothing about the scan changes once other threads can see it + _ = scan.Eol; + _ = scan.IndentUnit; + lock (entries) + { + entries.RemoveAll(_ => _.Path == normalizedPath); + if (entries.Count >= capacity) + { + var oldest = 0; + for (var index = 1; index < entries.Count; index++) + { + if (entries[index].Used < entries[oldest].Used) + { + oldest = index; + } + } + + entries.RemoveAt(oldest); + } + + entries.Add(new(normalizedPath, stamp, scan) { Used = ++uses }); + } + + return true; + } + } + /// /// Writes the patched source through a temporary file beside it and swaps that in, so the file /// on disk is either what it was or what the patch made it and never half of either. diff --git a/src/DiffEngine/Inline/InlinePatcher.cs b/src/DiffEngine/Inline/InlinePatcher.cs index a1f2c0c0..52396975 100644 --- a/src/DiffEngine/Inline/InlinePatcher.cs +++ b/src/DiffEngine/Inline/InlinePatcher.cs @@ -122,13 +122,36 @@ public static PatchStatus TryApply( out string newSource, out string failReason) { - newSource = ""; - failReason = ""; - var eol = DetectEol(source); - var lineStarts = BuildLineStarts(source); // Everything that reads the scan does so before this returns, the searches that are // enumerated lazily included, so its map goes back to the pool on the way out using var scan = language.Scan(source); + return TryApply(scan, lineHint, mode, originalExpression, originalValue, memberName, entryPoints, anchorOnly, newContent, out newSource, out failReason); + } + + /// + /// The same, over source that has been lexed already. For a caller with more than one thing + /// to ask of the same text: a probe of each call site in a file, or a batch, which carries + /// the scan from each patch to the next (). + /// + public static PatchStatus TryApply( + SourceScan scan, + int lineHint, + InlinePatchMode mode, + string? originalExpression, + string? originalValue, + string? memberName, + string[]? entryPoints, + bool anchorOnly, + string newContent, + out string newSource, + out string failReason) + { + newSource = ""; + failReason = ""; + var language = scan.Language; + var source = scan.Source; + var eol = scan.Eol; + var lineStarts = scan.LineStarts; var memberLine = MemberLine(source, scan, lineStarts, lineHint, memberName); if (mode == InlinePatchMode.Remove) @@ -136,13 +159,15 @@ public static PatchStatus TryApply( return TryRemove(language, source, scan, lineStarts, lineHint, memberLine, EntryPoints(entryPoints), originalExpression, originalValue, eol, ref newSource, ref failReason); } - var fileUnit = DetectIndentUnit(source, scan, lineStarts); - if (mode == InlinePatchMode.Append) { - return TryAppend(source, scan, lineStarts, lineHint, memberLine, EntryPoints(entryPoints), anchorOnly, newContent, eol, fileUnit, ref newSource, ref failReason); + // Not asked for a probe, which writes nothing + var appendUnit = anchorOnly ? "" : scan.IndentUnit; + return TryAppend(source, scan, lineStarts, lineHint, memberLine, EntryPoints(entryPoints), anchorOnly, newContent, eol, appendUnit, ref newSource, ref failReason); } + var fileUnit = scan.IndentUnit; + if (!string.IsNullOrEmpty(originalExpression)) { // Located by content: the Snapshot call whose expected argument is still the text the @@ -448,6 +473,14 @@ static bool TryReadArguments(string source, SourceScan scan, int openParen, out /// accepted already. Stopping there answered "already has a Snapshot call" for a patch whose /// own call sat two lines further down, and a single accept dropped the entry. /// + /// + /// And it has to be the only one, where the recorded line names no call. An append carries no + /// anchor, so among several calls with no Snapshot call nothing says which it was for, and the + /// first was taken: a call verified through files that came earlier in the test was given the + /// snapshot of the one after it. That is refused now, and a re-run brings the line the call + /// is on. A call passed a name Snapshot is called on () + /// has its snapshot and is not one of them. + /// /// static PatchStatus TryAppend( string source, @@ -467,6 +500,10 @@ static PatchStatus TryAppend( // Whether a call passed over for having a Snapshot call was holding this very content var held = false; List<(int Open, int Close)>? passedOver = null; + // The call to append to, once the recorded line has turned out not to name one, and the + // names Snapshot is called on in the member, read when the first such call is met + (int NameStart, int InsertAt)? taken = null; + HashSet? snapshotReceivers = null; foreach (var (nameStart, openParen) in FindCalls(source, scan, lineStarts, lineHint, memberLine, entryPoints, true)) { // An entry point in the argument list of a call that was passed over is part of that @@ -479,8 +516,14 @@ static PatchStatus TryAppend( } found = true; - if (!TryScanArguments(source, scan, openParen, out var closeParen, out _)) + if (!TryScanArguments(source, scan, openParen, out var closeParen, out var commas)) { + // One past the call that was taken is not this patch's to report + if (taken is not null) + { + break; + } + failReason = $"Could not parse the argument list of the {entryPointDescription} call near line {lineHint}."; return PatchStatus.NotFound; } @@ -495,8 +538,39 @@ static PatchStatus TryAppend( var insertAt = WalkChain(source, scan, closeParen + 1, methodName, out var chained); if (chained < 0) { - newSource = AppendCall(source, scan, lineStarts, nameStart, insertAt, newContent, eol, fileUnit); - return PatchStatus.Applied; + // The recorded line names its call, and with no member there is nothing to say + // how far the walk may go for another + if (memberLine is null || + IsOnHint(lineStarts, nameStart, lineHint)) + { + newSource = AppendCall(source, scan, lineStarts, nameStart, insertAt, newContent, eol, fileUnit); + return PatchStatus.Applied; + } + + passedOver ??= []; + passedOver.Add((openParen, closeParen)); + snapshotReceivers ??= SnapshotReceivers(source, scan, lineStarts, memberLine.Value); + if (TakesASnapshotReceiver(source, scan, openParen, closeParen, commas, snapshotReceivers)) + { + continue; + } + + if (taken is not null) + { + failReason = $"The call has moved from line {lineHint}, and more than one {entryPointDescription} call in its test has no {methodName} call. Re-run the test."; + return PatchStatus.NotFound; + } + + taken = (nameStart, insertAt); + continue; + } + + // Past the call that was taken, the walk is only looking for a second one like it + if (taken is not null) + { + passedOver ??= []; + passedOver.Add((openParen, closeParen)); + continue; } held |= HoldsContent(source, scan, chained, newContent); @@ -516,6 +590,12 @@ static PatchStatus TryAppend( passedOver.Add((openParen, closeParen)); } + if (taken is { } call) + { + newSource = AppendCall(source, scan, lineStarts, call.NameStart, call.InsertAt, newContent, eol, fileUnit); + return PatchStatus.Applied; + } + if (!found) { // Short, because every surface that shows it is one line: a status bar, a balloon, a @@ -543,6 +623,92 @@ static PatchStatus TryAppend( return PatchStatus.NotFound; } + /// + /// The names a Snapshot call is made on in the member declared at + /// : settings for settings.Snapshot("old");. Plain + /// names only, since that is all can match an argument to. + /// + static HashSet SnapshotReceivers(string source, SourceScan scan, List lineStarts, int memberLine) + { + var receivers = new HashSet(StringComparer.Ordinal); + var lineCount = lineStarts.Count; + var floor = Clamp(memberLine, lineCount); + var ceiling = Math.Min(NextMemberLine(source, scan, lineStarts, floor), lineCount + 1); + for (var line = floor; line < ceiling; line++) + { + foreach (var (nameStart, _) in CallsOnLine(source, scan, lineStarts, line, snapshotName, false)) + { + var dot = PreviousToken(source, scan, nameStart); + if (dot < 0 || + source[dot] != '.') + { + continue; + } + + var end = PreviousToken(source, scan, dot); + if (end < 0 || + !scan.IsCode(end) || + !scan.IsIdentifierChar(source[end])) + { + continue; + } + + var start = scan.WordStart(end); + var before = PreviousToken(source, scan, start); + if (before >= 0 && + source[before] == '.') + { + continue; + } + + receivers.Add(source.Substring(start, end + 1 - start)); + } + } + + return receivers; + } + + /// + /// Whether a call is passed something a Snapshot call is made on in the same member, which is + /// a verification that has its snapshot already: settings.Snapshot("old"); and then + /// Verify(value, settings). + /// + /// Asked of a call the recorded line did not name. A patch that appends is for a call with no + /// snapshot anywhere, so this one is not it, and with nothing chained onto it, it read as the + /// first call still wanting one. + /// + /// + static bool TakesASnapshotReceiver(string source, SourceScan scan, int openParen, int closeParen, List commas, HashSet receivers) + { + if (receivers.Count == 0) + { + return false; + } + + var start = openParen + 1; + for (var index = 0; index <= commas.Count; index++) + { + var end = index < commas.Count ? commas[index] : closeParen; + var argumentStart = start; + var argumentEnd = end; + start = end + 1; + TrimSpan(source, scan, ref argumentStart, ref argumentEnd); + if (argumentStart < argumentEnd) + { + scan.Language.TryStripArgumentName(source, ref argumentStart, out _); + } + + if (argumentStart < argumentEnd && + scan.IsCode(argumentStart) && + receivers.Contains(source.Substring(argumentStart, argumentEnd - argumentStart))) + { + return true; + } + } + + return false; + } + /// /// The source with a Snapshot call holding spliced in at /// , on a line of its own under the call at @@ -678,7 +844,7 @@ static PatchStatus TryRemove( return PatchStatus.NotFound; } - newSource = Splice(source, from, to, ""); + newSource = Splice(source, from, to, KeptStatementLine(source, scan, lineStarts, nameStart, from, to)); return PatchStatus.Applied; } @@ -782,6 +948,41 @@ static string KeepingItsLine(string source, SourceScan scan, int lineBreak, int return builder.ToString(); } + /// + /// What a statement taken whole leaves where it was: nothing, or one empty line when the line + /// under it holds a Snapshot call. + /// + /// for a statement, and for its reason. + /// settings.Snapshot("dup"); over other.Snapshot("dup"); brought the second up + /// onto the line the patch names, and the next apply of the same Remove had the same line and + /// the same anchor to go by and took the sibling's statement. + /// + /// + /// Only where the Snapshot call is on the statement's first line, which is the line the patch + /// names and so the one the empty line has to be on. With what it is called on a line above + /// it, the line that comes up onto the call's is further down than the one under the + /// statement, and one kept line puts neither right. + /// + /// + /// The source the statement is in. + /// The map of that source. + /// Where each line of the source starts. + /// The Snapshot call's name. + /// The start of the statement's first line. + /// The start of the line after its last. + static string KeptStatementLine(string source, SourceScan scan, List lineStarts, int nameStart, int from, int to) + { + if (to >= source.Length || + LineOf(lineStarts, nameStart) != LineOf(lineStarts, from) || + !CallsOnLine(source, scan, lineStarts, LineOf(lineStarts, to), snapshotName, false).Any()) + { + return ""; + } + + // The break the statement's last line ended in, which is the file's own + return to >= 2 && source[to - 2] == '\r' ? "\r\n" : "\n"; + } + /// /// Whether taking a call out would leave nothing of its expression but what it was called on: /// the call hangs off a name rather than off another call, and nothing is chained on after it. @@ -958,6 +1159,11 @@ static bool TryStatementLines(string source, SourceScan scan, List lineStar /// pulls the rest of its statement up onto the line above, so the recorded line then holds /// whatever followed, and the statement it named ends just before it. /// + /// + /// A call on a variable has no verify statement to be read off, and its whole statement goes. + /// So an empty line over a line holding a Snapshot call is read as removed too, which is what + /// leaves, and . + /// /// static bool RemovedAtHint(string source, SourceScan scan, List lineStarts, int lineHint, int? memberLine, string[] entryPoints) { @@ -983,6 +1189,15 @@ static bool RemovedAtHint(string source, SourceScan scan, List lineStarts, return false; } + // An empty line over a Snapshot call is the line a Remove kept, and the only sign there is + // of a statement taken whole: nothing above it need be a verify call + if (lineHint < lineCount && + IsEmptyLine(source, scan, lineStarts, lineHint) && + CallsOnLine(source, scan, lineStarts, lineHint + 1, snapshotName, false).Any()) + { + return true; + } + for (var line = lineHint; line >= floor; line--) { var calls = CallsOnLine(source, scan, lineStarts, line, entryPoints, true).ToList(); @@ -1006,6 +1221,25 @@ static bool RemovedAtHint(string source, SourceScan scan, List lineStarts, return false; } + /// + /// Whether a line holds nothing but whitespace, and is a line of code: an empty line of a + /// literal's content is not one anything was taken from. + /// + static bool IsEmptyLine(string source, SourceScan scan, List lineStarts, int line) + { + var start = lineStarts[line - 1]; + var end = line < lineStarts.Count ? lineStarts[line] : source.Length; + for (var index = start; index < end; index++) + { + if (!char.IsWhiteSpace(source[index])) + { + return false; + } + } + + return scan.IsCode(start); + } + /// /// The calls a Snapshot call has to be appended in front of rather than after: each hands back /// something other than the SettingsTask a Snapshot call is made on, so the chain cannot be @@ -2003,123 +2237,9 @@ static string Splice(string source, int start, int end, string replacement) => .ToString(); #endif - static string DetectEol(string source) - { - var crlf = 0; - var lf = 0; - for (var index = 0; index < source.Length; index++) - { - if (source[index] != '\n') - { - continue; - } - - if (index > 0 && source[index - 1] == '\r') - { - crlf++; - } - else - { - lf++; - } - } - - if (crlf >= lf && crlf > 0) - { - return "\r\n"; - } - - if (lf > 0) - { - return "\n"; - } - - return Environment.NewLine; - } - /// - /// What one level of indentation is made of in this file: the most common run of whitespace a - /// line adds to the one above it. - /// - /// Read off the source rather than taken from a convention, because a splice has to match the - /// code it lands in, and files disagree with their repo's settings often enough - vendored, - /// generated, or last edited by someone configured differently - that following the convention - /// would make the patch look more out of place, not less. It answers the one question a single - /// call site cannot: a line shows which characters it is indented with, but not how wide a - /// level is, and hard coding four spaces is wrong in every two space repo. - /// - /// Returns "" when the file is too small to show a step, which leaves the choice to - /// . - /// - static string DetectIndentUnit(string source, SourceScan scan, List lineStarts) - { - Dictionary counts = new(StringComparer.Ordinal); - var previous = ""; - foreach (var lineStart in lineStarts) - { - // Inside a comment or a literal the leading whitespace is content, not indentation. - // A snapshot literal in particular is arbitrary text, and counting its lines would - // measure the snapshot rather than the file - if (!scan.IsCode(lineStart)) - { - continue; - } - - var index = lineStart; - while (index < source.Length && - (source[index] == ' ' || source[index] == '\t')) - { - index++; - } - - // A blank line has no indentation of its own, and must not break the run either - if (index >= source.Length || - source[index] == '\r' || - source[index] == '\n') - { - continue; - } - - var lead = source.Substring(lineStart, index - lineStart); - // Only a line that indents further than the one above, by adding to what it already - // had. Anything else is a dedent, or whitespace of a different kind, and neither - // measures a step - if (lead.Length > previous.Length && - lead.StartsWith(previous, StringComparison.Ordinal)) - { - var step = lead.Substring(previous.Length); - counts.TryGetValue(step, out var count); - counts[step] = count + 1; - } - - previous = lead; - } - - var best = ""; - var bestCount = 0; - foreach (var pair in counts) - { - if (bestCount == 0 || - pair.Value > bestCount || - pair.Value == bestCount && Closer(pair.Key, best)) - { - best = pair.Key; - bestCount = pair.Value; - } - } - - return best; - - // A tie goes to the shorter step, since a longer one is two levels taken at once, and - // then to ordinal order so the answer cannot depend on enumeration order - static bool Closer(string candidate, string current) => - candidate.Length == current.Length - ? string.CompareOrdinal(candidate, current) < 0 - : candidate.Length < current.Length; - } - - /// - /// One level of indentation for a splice at a site indented with . + /// One level of indentation for a splice at a site indented with , + /// given the file's own (). /// /// The character comes from the site and the width from the file, so a file that indents /// inconsistently still gets a splice consistent with its own surroundings, while a file that @@ -2156,20 +2276,6 @@ static string NormalizeTo(string value, string eol) => .Replace('\r', '\n') .Replace("\n", eol); - static List BuildLineStarts(string source) - { - List starts = [0]; - for (var index = 0; index < source.Length; index++) - { - if (source[index] == '\n' && index + 1 < source.Length) - { - starts.Add(index + 1); - } - } - - return starts; - } - static int LineOf(List lineStarts, int offset) { var low = 0; diff --git a/src/DiffEngine/Inline/SourceLanguage.cs b/src/DiffEngine/Inline/SourceLanguage.cs index 1ea83a0e..f7fa7087 100644 --- a/src/DiffEngine/Inline/SourceLanguage.cs +++ b/src/DiffEngine/Inline/SourceLanguage.cs @@ -63,7 +63,19 @@ public static SourceLanguage ForFile(string path) /// /// Lexes into the map every search then reads. /// - internal abstract SourceScan Scan(string source); + internal SourceScan Scan(string source) + { + var scan = new SourceScan(this, source); + Lex(scan, 0); + return scan; + } + + /// + /// Fills a scan's map from on, which is the start of the source or a + /// line start it has as code, up to the end or to where the scan says the rest is known + /// (). + /// + internal abstract void Lex(SourceScan scan, int from); internal abstract bool IsIdentifierChar(char ch); diff --git a/src/DiffEngine/Inline/SourceScan.cs b/src/DiffEngine/Inline/SourceScan.cs index b6e38785..4b57405b 100644 --- a/src/DiffEngine/Inline/SourceScan.cs +++ b/src/DiffEngine/Inline/SourceScan.cs @@ -86,6 +86,429 @@ public void AddSkip(int start, int end, bool comment) skipped.AsSpan(start, end - start).Fill(true); } + /// + /// The scan this one is being made from, while it is, and how much longer this source is than + /// that one's. See . + /// + SourceScan? before; + + int growth; + + /// + /// The first offset at which the lexer may be told the rest is known: just past the edit, for + /// a scan being made from another, and never for one lexed whole. + /// + public int RejoinFrom { get; private set; } = int.MaxValue; + + /// + /// The scan of this source after an edit, made from this one and not by lexing the whole of + /// again. + /// + /// A batch applies each patch to what the one before it left, so the file was lexed again for + /// every patch, start to end, to find the handful of spans one literal changed. What a lexer + /// makes of a line depends on the text before it only through whether the line starts inside + /// a comment or a literal, and on the text after it not at all once a span has closed. So the + /// spans before the edit are kept, lexing starts again at the line the edit begins on, and it + /// stops at the first line start past the edit that both this scan and the new one take as + /// code (): from there on the two read the same text from the same + /// state, and the rest of this scan's spans are the new one's, moved by the change in length. + /// + /// + /// One thing does look across a line break from in front of it, which is a backslash: C#'s + /// char and string scans step over the character after one, and F#'s char literal reads the + /// one after that. So a line that follows a backslash is not started from, and the line + /// before it is. SourceScanTests holds the result to a scan of the whole text, over + /// edits made at random. + /// + /// + /// This scan is still whole afterwards, and still the caller's to dispose. + /// + /// + public SourceScan Edited(string edited) + { + var limit = Math.Min(Source.Length, edited.Length); + var prefix = CommonPrefix(Source, edited, limit); + var suffix = CommonSuffix(Source, edited, limit - prefix); + var restart = RestartFor(prefix); + var next = new SourceScan(Language, edited); + + // About as many spans as there were, so the lists are sized once and not as they grow + next.starts.Capacity = starts.Count + 16; + next.ends.Capacity = starts.Count + 16; + next.comments.Capacity = starts.Count + 16; + var kept = LowerBound(starts, restart); + for (var index = 0; index < kept; index++) + { + next.starts.Add(starts[index]); + next.ends.Add(ends[index]); + next.comments.Add(comments[index]); + } + + Array.Copy(skipped, next.skipped, restart); + next.before = this; + next.growth = edited.Length - Source.Length; + // Strictly past the edit, so the line break in front of the offset is one both texts have + next.RejoinFrom = edited.Length - suffix + 1; + Language.Lex(next, restart); + next.before = null; + next.RejoinFrom = int.MaxValue; + + if (lineStarts is not null) + { + next.lineStarts = EditedLineStarts(lineStarts, edited, prefix, edited.Length - suffix, next.growth); + } + + return next; + } + + /// + /// Where lexing starts again for an edit that begins at : the start + /// of its line, or of an earlier one where that start is inside a comment or a literal, or + /// follows a backslash. + /// + int RestartFor(int offset) + { + var restart = offset; + while (restart > 0) + { + restart = Source.LastIndexOf('\n', restart - 1) + 1; + if (restart == 0) + { + break; + } + + // The last span to start before the line does. The line starts inside it when it ends + // past there, and when it ran to the end of the source, which is where a comment or + // a literal that was never closed ends: anything added there is inside it too + var span = LowerBound(starts, restart) - 1; + if (span >= 0 && + (ends[span] > restart || ends[span] == Source.Length)) + { + // Back to where that span starts, and then to the start of its line + restart = starts[span]; + continue; + } + + if (restart < 2 || + Source[restart - 2] != '\\') + { + break; + } + + restart--; + } + + return restart; + } + + /// + /// Asked by a lexer at the top of its loop, once it is at or past : + /// whether everything from on is as the scan this one is made from + /// has it. Where it is, the rest of that scan's spans are taken and the lexer is done. + /// + public bool TryRejoin(int index) + { + if (before is null || + Source[index - 1] != '\n') + { + return false; + } + + var from = index - growth; + if (before.skipped[from]) + { + return false; + } + + for (var span = LowerBound(before.starts, from); span < before.starts.Count; span++) + { + starts.Add(before.starts[span] + growth); + ends.Add(before.ends[span] + growth); + comments.Add(before.comments[span]); + } + + Array.Copy(before.skipped, from, skipped, index, before.Source.Length - from); + return true; + } + + static List EditedLineStarts(List old, string edited, int prefix, int editEnd, int growth) + { + var found = new List(old.Count + 8); + // The lines that start before the edit, which the line breaks in front of it decide + var index = 0; + for (; index < old.Count && old[index] < Math.Max(prefix, 1); index++) + { + found.Add(old[index]); + } + + // Those a line break in the edit, or the one in front of it, starts + var search = Math.Max(prefix - 1, 0); + while (search < editEnd) + { + search = edited.IndexOf('\n', search, editEnd - search); + if (search < 0 || + search + 1 >= edited.Length) + { + break; + } + + search++; + found.Add(search); + } + + // And those past it, moved + index = LowerBound(old, editEnd - growth + 1); + for (; index < old.Count; index++) + { + found.Add(old[index] + growth); + } + + return found; + } + + /// + /// The index of the first value not less than in a sorted list. + /// + static int LowerBound(List sorted, int value) + { + var low = 0; + var high = sorted.Count; + while (low < high) + { + var middle = (low + high) / 2; + if (sorted[middle] < value) + { + low = middle + 1; + } + else + { + high = middle; + } + } + + return low; + } + + // Compared a block at a time, which is vectorised on every framework this targets, and then + // a character at a time within the block that differs + const int block = 4096; + + static int CommonPrefix(string left, string right, int limit) + { + var index = 0; + while (index + block <= limit && + left.AsSpan(index, block).SequenceEqual(right.AsSpan(index, block))) + { + index += block; + } + + while (index < limit && + left[index] == right[index]) + { + index++; + } + + return index; + } + + static int CommonSuffix(string left, string right, int limit) + { + var count = 0; + while (count + block <= limit && + left.AsSpan(left.Length - count - block, block).SequenceEqual(right.AsSpan(right.Length - count - block, block))) + { + count += block; + } + + while (count < limit && + left[left.Length - count - 1] == right[right.Length - count - 1]) + { + count++; + } + + return count; + } + + List? lineStarts; + string? eol; + string? indentUnit; + + /// + /// The offset each line starts at. A line break that ends the source starts no line. + /// + /// Kept here, with and , because they are facts + /// about the same text the map is of, and a scan that is carried from one patch to the next + /// carries them with it. Each is worked out when first asked, since a search that finds + /// nothing to write needs only the first. + /// + /// + public List LineStarts => lineStarts ??= BuildLineStarts(); + + List BuildLineStarts() + { + List found = [0]; + var index = 0; + while (true) + { + index = Source.IndexOf('\n', index); + if (index < 0 || + index + 1 >= Source.Length) + { + return found; + } + + index++; + found.Add(index); + } + } + + /// + /// The line break most of the file's lines end in, a tie going to \r\n, and the + /// platform's own for a file with none. + /// + public string Eol => eol ??= DetectEol(); + + string DetectEol() + { + var crlf = 0; + var lf = 0; + var starts = LineStarts; + // Every line but the first starts after a line break, and the last line may end in one + // that starts nothing + for (var line = 1; line <= starts.Count; line++) + { + var end = line < starts.Count ? starts[line] : Source.Length; + if (end == 0 || + Source[end - 1] != '\n') + { + continue; + } + + if (end > 1 && Source[end - 2] == '\r') + { + crlf++; + } + else + { + lf++; + } + } + + if (crlf >= lf && crlf > 0) + { + return "\r\n"; + } + + if (lf > 0) + { + return "\n"; + } + + return Environment.NewLine; + } + + /// + /// What one level of indentation is made of in this file: the most common run of whitespace a + /// line adds to the one above it. + /// + /// Read off the source rather than taken from a convention, because a splice has to match the + /// code it lands in, and files disagree with their repo's settings often enough - vendored, + /// generated, or last edited by someone configured differently - that following the convention + /// would make the patch look more out of place, not less. It answers the one question a single + /// call site cannot: a line shows which characters it is indented with, but not how wide a + /// level is, and hard coding four spaces is wrong in every two space repo. + /// + /// + /// "" when the file is too small to show a step, which leaves the choice to whoever asked. + /// + /// + public string IndentUnit => indentUnit ??= DetectIndentUnit(); + + string DetectIndentUnit() + { + // Counted without a string a line: this is asked once a patch, of every line of the file, + // and a file shows two or three different steps at most + List<(string Step, int Count)> counts = []; + var previousStart = 0; + var previousLength = 0; + foreach (var lineStart in LineStarts) + { + // Inside a comment or a literal the leading whitespace is content, not indentation. + // A snapshot literal in particular is arbitrary text, and counting its lines would + // measure the snapshot rather than the file + if (!IsCode(lineStart)) + { + continue; + } + + var index = lineStart; + while (index < Source.Length && + (Source[index] == ' ' || Source[index] == '\t')) + { + index++; + } + + // A blank line has no indentation of its own, and must not break the run either + if (index >= Source.Length || + Source[index] == '\r' || + Source[index] == '\n') + { + continue; + } + + var length = index - lineStart; + // Only a line that indents further than the one above, by adding to what it already + // had. Anything else is a dedent, or whitespace of a different kind, and neither + // measures a step + if (length > previousLength && + string.CompareOrdinal(Source, lineStart, Source, previousStart, previousLength) == 0) + { + Count(counts, lineStart + previousLength, length - previousLength); + } + + previousStart = lineStart; + previousLength = length; + } + + var best = ""; + var bestCount = 0; + foreach (var (step, count) in counts) + { + if (bestCount == 0 || + count > bestCount || + count == bestCount && Closer(step, best)) + { + best = step; + bestCount = count; + } + } + + return best; + + // A tie goes to the shorter step, since a longer one is two levels taken at once, and + // then to ordinal order so the answer cannot depend on the order they were met in + static bool Closer(string candidate, string current) => + candidate.Length == current.Length + ? string.CompareOrdinal(candidate, current) < 0 + : candidate.Length < current.Length; + } + + void Count(List<(string Step, int Count)> counts, int start, int length) + { + for (var index = 0; index < counts.Count; index++) + { + var (step, count) = counts[index]; + if (step.Length == length && + string.CompareOrdinal(Source, start, step, 0, length) == 0) + { + counts[index] = (step, count + 1); + return; + } + } + + counts.Add((Source.Substring(start, length), 1)); + } + /// /// True when the offset is outside every comment, string and char literal. /// diff --git a/src/DiffEngine/MaxInstance.cs b/src/DiffEngine/MaxInstance.cs index 319e2be2..9c4f14b6 100644 --- a/src/DiffEngine/MaxInstance.cs +++ b/src/DiffEngine/MaxInstance.cs @@ -119,4 +119,13 @@ public static bool Reached() var count = Interlocked.Increment(ref launchedInstances); return count > MaxInstancesToLaunch; } + + /// + /// Returns a slot granted, for a caller whose launch then opened + /// nothing. The cap is on windows, and a slot has to be asked for before there is one, so + /// the launch that fails is the only one that knows it was charged for a window that never + /// came. + /// + public static void GiveBack() => + Interlocked.Decrement(ref launchedInstances); } diff --git a/src/DiffEngine/Protocol/ListenerTable.cs b/src/DiffEngine/Protocol/ListenerTable.cs index 557b9bb5..8c9d4c62 100644 --- a/src/DiffEngine/Protocol/ListenerTable.cs +++ b/src/DiffEngine/Protocol/ListenerTable.cs @@ -7,10 +7,17 @@ namespace DiffEngine; /// /// For a caller about to connect to a local port that may have nobody on it. Such a connect is not /// refused at once everywhere: where the reset a closed port answers with is dropped, as Windows -/// Firewall's stealth mode drops it, the connect runs for two seconds. Reading the table takes a -/// third of a millisecond on a machine with a few dozen connections open. It is filtered out of -/// every connection the machine has, so that grows with them, and a caller with a cheaper way of -/// knowing somebody is there should use it first. +/// Firewall's stealth mode drops it, the connect runs for two seconds. +/// +/// +/// On Windows the listeners are asked for by themselves (): a tenth of a +/// millisecond, and three tenths with three thousand connections open beside them, since Windows +/// still passes over those to find the listeners. What .NET offers is every connection the +/// machine has, handed across and made into objects, with the listeners then picked out: a third +/// of a millisecond at its quietest and thirteen with those three thousand. That is still what is +/// read wherever the listeners cannot be had alone: off Windows, and on a Windows that will not +/// answer the call. So a caller with a cheaper way of knowing somebody is there should still use +/// it first. /// /// /// By port alone, whichever address the listener is bound to. The table is only ever believed @@ -49,6 +56,21 @@ public static bool IsHeld(int port) internal static Func Lookup { get; set; } = Listed; static bool Listed(int port) + { + if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows) && + Listeners(port) is { } held) + { + return held; + } + + return Enumerated(port); + } + + /// + /// The answer out of everything .NET lists, which on Windows is every TCP row the machine has + /// with the listening ones kept. What is asked wherever has no answer. + /// + internal static bool Enumerated(int port) { foreach (var listener in IPGlobalProperties.GetIPGlobalProperties().GetActiveTcpListeners()) { @@ -60,4 +82,99 @@ static bool Listed(int port) return false; } + + /// + /// The answer out of Windows' own table of listeners, which holds a row for each listening + /// socket and none for a connection, so what is handed over and looked through does not grow + /// with what the machine is doing. Null when it could not be had, for whatever reason: the + /// call is missing, it failed, or what came back is not laid out as expected. + /// + /// Both families, since which one a listener is listed under follows how it was bound, and a + /// loopback connect can reach either. A machine with IPv6 off has no second table to ask for. + /// + /// + internal static bool? Listeners(int port) + { + try + { + // MIB_TCPROW_OWNER_PID, whose third field is the local port + var held = Read(port, interNetwork, 24, 8); + if (held != false || + !Socket.OSSupportsIPv6) + { + return held; + } + + // MIB_TCP6ROW_OWNER_PID, where the port follows the address and its scope + return Read(port, interNetworkV6, 56, 20); + } + catch (Exception exception) + when (exception is DllNotFoundException or EntryPointNotFoundException or MarshalDirectiveException) + { + return null; + } + } + + static bool? Read(int port, int family, int rowSize, int portOffset) + { + var size = 4096; + // A table that outgrew the buffer says how much it needs, and can have grown again by + // the time it is asked for with that. Not for ever: a call that keeps saying so is one + // that is not going to answer + for (var attempt = 0; attempt < 4; attempt++) + { + var table = new byte[size]; + var result = GetExtendedTcpTable(table, ref size, 0, family, ownerPidListener, 0); + if (result == insufficientBuffer) + { + size = Math.Max(size, table.Length * 2); + continue; + } + + if (result != 0) + { + return null; + } + + // The count, and then the rows + var count = BitConverter.ToInt32(table, 0); + if (count < 0 || + 4 + (long) count * rowSize > table.Length) + { + return null; + } + + for (var index = 0; index < count; index++) + { + // Network byte order, in the low half of a field twice the size + var at = 4 + index * rowSize + portOffset; + if (((table[at] << 8) | table[at + 1]) == port) + { + return true; + } + } + + return false; + } + + return null; + } + + const int interNetwork = 2; + const int interNetworkV6 = 23; + + // TCP_TABLE_OWNER_PID_LISTENER. The basic class has rows that are four bytes shorter, and is + // not documented for IPv6 + const int ownerPidListener = 3; + + const uint insufficientBuffer = 122; + + [DllImport("iphlpapi.dll")] + static extern uint GetExtendedTcpTable( + [Out] byte[] table, + ref int size, + int order, + int family, + int tableClass, + int reserved); } diff --git a/src/DiffEngine/Protocol/ViewerClient.cs b/src/DiffEngine/Protocol/ViewerClient.cs index 46f317c7..ddcc74c1 100644 --- a/src/DiffEngine/Protocol/ViewerClient.cs +++ b/src/DiffEngine/Protocol/ViewerClient.cs @@ -83,8 +83,9 @@ public static int Port public static readonly TimeSpan ShortTimeout = TimeSpan.FromMilliseconds(500); /// - /// How long a port found with nothing listening stands as unowned, during which the sends - /// that only tell the owner something return without connecting. + /// How long a port a connect found nobody on stands as unowned, during which the sends that + /// only tell the owner something return without connecting. For a port the listener table + /// found empty it is . /// /// A refused loopback connection is not free everywhere. Windows Firewall's stealth mode, on /// by default, drops the reset a closed port would answer with, so the connect sits through @@ -94,27 +95,50 @@ public static int Port /// connecting to nobody, and the same run took under a second with a tray answering. /// /// - /// Long, because a recheck buys almost nothing and is not free: those two seconds again where - /// it has to connect, and a read of the listener table where the operating system can be asked - /// instead (). What the memory can delay is only a message the - /// owner did not have to receive: an entry to settle in a queue that did not exist when the - /// test failed, or a move to track in a tray that was not there to track it. Anything that has - /// to reach an owner - a patch, a delete, a pair - goes through the launch gate, whose - /// probe always asks and corrects the memory with what it finds. Not the - /// life of the process only for a long lived consumer that is not a test host, launching diff - /// tools all day, where a tray started later would otherwise never see its moves until a - /// restart. + /// Long, because a recheck that has to connect buys almost nothing for those two seconds + /// again. What the memory can delay is only a message the owner did not have to receive: an + /// entry to settle in a queue that did not exist when the test failed, or a move to track in + /// a tray that was not there to track it. Anything that has to reach an owner - a patch, a + /// delete, a pair - goes through the launch gate, whose probe always + /// asks and corrects the memory with what it finds. Not the life of the process only for a + /// long lived consumer that is not a test host, launching diff tools all day, where a tray + /// started later would otherwise never see its moves until a restart. + /// + /// + /// This is the wait wherever the connect is what answered: off Windows, on a Windows whose + /// listener table could not be read, and for a port held by something that is not a viewer. /// /// internal static TimeSpan RecheckUnownedAfter { get; set; } = TimeSpan.FromMinutes(10); + /// + /// How long a port stands as unowned when it was the listener table that said nobody is + /// there (), which is how Windows is asked wherever it can be. + /// + /// A second, where the connect's answer stands for ten minutes, because asking again is a + /// read of the table, a tenth of a millisecond, rather than two seconds waiting to be + /// refused. So a tray or a viewer started after the test process is told of the settles and + /// moves that come a second after it is listening, where it was told of none for ten minutes. + /// + /// + /// Not nothing, because a green run settles once per verification, as fast as the tests + /// pass, and each would then read the table to learn what the one before it had: ten + /// thousand settles are over a second of that. At a second the reads are one a second however + /// many settles there are, a ten thousandth of the run. It is the same second an owner that + /// answered is trusted for (), so what was found about a port is + /// believed for as long whichever way it went. + /// + /// + internal static TimeSpan RecheckUnlistedAfter { get; set; } = TimeSpan.FromSeconds(1); + /// /// How long a port that accepted a connection is connected to again without first asking the /// operating system whether anyone is there: see . /// /// For a run of settles to an owner that is there, which is a connection every third of a - /// millisecond. Each renews this, so none of them reads the listener table, which costs more - /// than the connect does. Short, because a send that comes a while after the last is the one + /// millisecond. Each renews this, so none of them reads the listener table, which costs as + /// much as the connect does, and several times that where the listeners cannot be asked for + /// alone. Short, because a send that comes a while after the last is the one /// most likely to find its owner gone, and asking is what spares that send its two seconds. /// /// @@ -124,13 +148,17 @@ public static int Port /// What was last found on each port, and when, as a timestamp. Per /// port because tests talk to ephemeral ports of their own, in parallel, and what happened on /// those says nothing about the one live port on a developer machine. + /// + /// And how it was found, for a port with nobody on it: by the listener table, which is cheap + /// to ask again, or by a connect, which is not. + /// /// - static ConcurrentDictionary lastFound = new(); + static ConcurrentDictionary lastFound = new(); static bool RecentlyUnowned(int port) => lastFound.TryGetValue(port, out var found) && !found.Owned && - Since(found.At) < RecheckUnownedAfter; + Since(found.At) < (found.Unlisted ? RecheckUnlistedAfter : RecheckUnownedAfter); static bool AnsweredLately(int port) => lastFound.TryGetValue(port, out var found) && @@ -146,7 +174,14 @@ static TimeSpan Since(long timestamp) => /// can talk to, and a port any of them found empty is what the memory is for. /// static void Found(int port, bool owned) => - lastFound[port] = (owned, Stopwatch.GetTimestamp()); + lastFound[port] = (owned, Stopwatch.GetTimestamp(), false); + + /// + /// for a port the listener table has nobody on, which stands for less + /// long: see . + /// + static void FoundUnlisted(int port) => + lastFound[port] = (false, Stopwatch.GetTimestamp(), true); /// /// Whether a connect to would only be a wait to be refused: the @@ -167,7 +202,7 @@ static void Found(int port, bool owned) => /// Only on Windows, which is where the refusal is slow. Elsewhere it arrives at once, so the /// connect is the cheaper question as well as the one whose answer cannot be wrong. And not /// for a port that accepted a connection within , where the table - /// would cost more than the connect it stands in front of. + /// would cost as much as the connect it stands in front of. /// /// static bool NothingListening(int port) => @@ -191,9 +226,14 @@ internal static void ForgetUnowned() { lastFound.Clear(); reportedForeign.Clear(); - lock (keptGate) + lock (telling.Gate) + { + telling.Drop(); + } + + lock (listing.Gate) { - DropKept(); + listing.Drop(); } } @@ -260,7 +300,7 @@ public static bool IsOwned(int? port = null) var endpointPort = port ?? Port; if (NothingListening(endpointPort)) { - Found(endpointPort, false); + FoundUnlisted(endpointPort); return false; } @@ -273,13 +313,38 @@ public static bool IsOwned(int? port = null) catch (Exception exception) when (Ignorable(exception)) { - owned = false; + if (NobodyThere(exception)) + { + Found(endpointPort, false); + } + + return false; } Found(endpointPort, owned); return owned; } + /// + /// Whether a connect that failed says nobody is on the port, which is the only failure the + /// memory of an unowned port is for: the port refused it, or never answered. + /// + /// A connect also fails when the machine cannot make one at all - it has no ports left to + /// connect from, or no buffers - and that says nothing of who is listening. Taken for an + /// empty port, it had every settle and move after it skipped for ten minutes with the owner + /// still there, on exactly the machine a kept connection was meant to help. Which error a + /// machine gives when it runs out differs by platform and was never pinned down for Windows, + /// so this names the two that do mean nobody and takes everything else as not known. + /// + /// + internal static bool NobodyThere(Exception exception) + { + var socket = exception as SocketException ?? + exception.InnerException as SocketException ?? + exception.InnerException?.InnerException as SocketException; + return socket?.SocketErrorCode is SocketError.ConnectionRefused or SocketError.TimedOut; + } + /// /// True when the owner acknowledged. A refused connection means nobody owns the queue. /// @@ -303,22 +368,22 @@ public static bool TrySend(ViewerMessage message) => /// internal static bool Tell(ViewerMessage message, int port) { - switch (SendKept(message, port, out var ok)) + switch (SendKept(telling, message, port, timeout, out var response)) { case KeptSend.Answered: - return ok; + return response!.Ok; case KeptSend.Unanswered: return false; } - if (!Exchange(message, out var response, out var keeps, port, null, skipIfUnowned: true)) + if (!Exchange(message, out response, out var keeps, port, null, skipIfUnowned: true)) { return false; } if (keeps) { - KeepConnection(port); + KeepConnection(telling, port); } return response.Ok; @@ -328,7 +393,8 @@ enum KeptSend { /// /// No connection is kept to that port, or the one that was has gone, as it does when its - /// owner exits. The ordinary exchange is what finds out who is there now. + /// owner exits. The ordinary exchange is what finds out who is there now. Or the one + /// kept is in the middle of someone else's exchange and this send does not wait for it. /// NotKept, @@ -345,27 +411,108 @@ sealed class KeptConnection(TcpClient client, int port) : IDisposable { public int Port { get; } = port; + public IntPtr Handle => client.Client.Handle; public NetworkStream Stream { get; } = client.GetStream(); public StreamReader Reader { get; } = new(client.GetStream(), Encoding.UTF8); + /// + /// How long the next request waits to be written and answered, which is its caller's to + /// say: a tray's timer asks with half a second and a window's listing with fifteen. + /// + public void Allow(TimeSpan wait) => + Configure(client, wait); + public void Dispose() => client.Close(); } /// - /// Held for the whole of an exchange on the kept connection, which is one request and its - /// answer at a time. A parallel run's settles take turns at it, each for about as long as - /// the owner takes to answer. + /// A connection that is kept, and whose turn it is on it. + /// + /// The gate is held for the whole of an exchange on the connection, which is one request and + /// its answer at a time. is what a send does when it finds the gate held. + /// + /// + sealed class KeptSlot(bool queues) + { + public readonly object Gate = new(); + + public KeptConnection? Connection; + + /// + /// Whether a send waits its turn, or leaves the connection to whoever has it and makes + /// one of its own. + /// + public bool Queues { get; } = queues; + + /// + /// Called with the gate held. + /// + public void Drop() + { + Connection?.Dispose(); + Connection = null; + } + } + + /// + /// The connection the telling sends go down. A parallel run's settles take turns at it, each + /// for about as long as the owner takes to answer. + /// + static readonly KeptSlot telling = new(queues: true); + + /// + /// The connection the listings go down: see . One of its own, because the + /// owner answers a connection's requests in turn, and a listing an owner is slow to answer + /// would otherwise stand in front of every settle the process sent behind it. And not waited + /// for, because the callers are a timer and a window that each list on their own clock, with + /// waits of their own: a listing that finds another in flight is a connection each, as every + /// listing used to be. /// - static readonly object keptGate = new(); + static readonly KeptSlot listing = new(queues: false); - static KeptConnection? kept; + /// + /// The socket of the kept telling connection, or null with none kept. For the test of whether + /// a child process would be handed it, which is a property of the handle and of nothing an + /// exchange shows. + /// + internal static IntPtr? KeptHandle + { + get + { + lock (telling.Gate) + { + return telling.Connection?.Handle; + } + } + } - static KeptSend SendKept(ViewerMessage message, int port, out bool ok) + static KeptSend SendKept( + KeptSlot slot, + ViewerMessage message, + int port, + TimeSpan wait, + out ViewerResponse? response) { - ok = false; - lock (keptGate) + response = null; + var entered = false; + try { + if (slot.Queues) + { + Monitor.Enter(slot.Gate, ref entered); + } + else + { + Monitor.TryEnter(slot.Gate, ref entered); + } + + if (!entered) + { + return KeptSend.NotKept; + } + + var kept = slot.Connection; if (kept is null) { return KeptSend.NotKept; @@ -373,12 +520,13 @@ static KeptSend SendKept(ViewerMessage message, int port, out bool ok) if (kept.Port != port) { - DropKept(); + slot.Drop(); return KeptSend.NotKept; } try { + kept.Allow(wait); var bytes = Encoding.UTF8.GetBytes($"{message.Build()}\n"); kept.Stream.Write(bytes, 0, bytes.Length); kept.Stream.Flush(); @@ -391,25 +539,31 @@ static KeptSend SendKept(ViewerMessage message, int port, out bool ok) } if (line is not null && - ViewerResponse.TryParse(reply.ToString(), out var response)) + ViewerResponse.TryParse(reply.ToString(), out response)) { Found(port, true); - ok = response.Ok; return KeptSend.Answered; } // Closed by the owner, which is an owner that stopped or exited since the last // send. Whoever holds the port now is for the ordinary exchange to find - DropKept(); + slot.Drop(); return KeptSend.NotKept; } catch (Exception exception) when (Ignorable(exception)) { - DropKept(); + slot.Drop(); return TimedOut(exception) ? KeptSend.Unanswered : KeptSend.NotKept; } } + finally + { + if (entered) + { + Monitor.Exit(slot.Gate); + } + } } static bool TimedOut(Exception exception) => @@ -422,27 +576,46 @@ exception is IOException }; /// - /// Opens the connection the telling sends after this one go down, to an owner whose reply - /// has just said it keeps one. Failing to is nothing: the next send is an ordinary exchange, - /// and tries again on the strength of its own reply. + /// Opens the connection the sends after this one go down, to an owner whose reply has just + /// said it keeps one. Failing to is nothing: the next send is an ordinary exchange, and + /// tries again on the strength of its own reply. So is finding the gate held, on a slot + /// nothing waits for: whoever holds it has the connection, or is opening it. /// - static void KeepConnection(int port) + static void KeepConnection(KeptSlot slot, int port) { - lock (keptGate) + var entered = false; + try { - if (kept is not null) + if (slot.Queues) + { + Monitor.Enter(slot.Gate, ref entered); + } + else + { + Monitor.TryEnter(slot.Gate, ref entered); + } + + if (!entered) + { + return; + } + + if (slot.Connection is not null) { - if (kept.Port == port) + if (slot.Connection.Port == port) { return; } - DropKept(); + slot.Drop(); } var client = new TcpClient(); try { +#if NETFRAMEWORK + KeepFromChildren(client); +#endif if (!Connect(client, port, ShortTimeout)) { client.Close(); @@ -457,7 +630,7 @@ static void KeepConnection(int port) var bytes = Encoding.UTF8.GetBytes($"{ViewerServer.Keep}\n"); connection.Stream.Write(bytes, 0, bytes.Length); connection.Stream.Flush(); - kept = connection; + slot.Connection = connection; } catch (Exception exception) when (Ignorable(exception)) @@ -465,14 +638,69 @@ static void KeepConnection(int port) client.Close(); } } + finally + { + if (entered) + { + Monitor.Exit(slot.Gate); + } + } } - static void DropKept() + /// + /// Whether a message only reads the queue, so that it can go down a kept connection as the + /// telling sends do. + /// + /// The listings are the asking sends there are many of: a window showing an owner's queue + /// lists five times a second for as long as it is open, and a tray driving one lists on a + /// timer, each a connection whose port then waited out TIME_WAIT. And they are the ones that + /// can be asked again. A request written to a kept connection whose owner has just gone is + /// sent a second time as an ordinary exchange, since nothing says whether the first was + /// acted on, and for a listing that is harmless. An accept or a discard asked twice is not, + /// so everything that changes the queue, and the async send, is still a connection each. + /// + /// + static bool Lists(ViewerMessage message) => + message.Verb is ViewerVerb.List or ViewerVerb.ListFull; + +#if NETFRAMEWORK + /// + /// Marks the socket of a connection about to be kept as one a child process does not get. + /// + /// .NET Framework makes its sockets inheritable, and a process started without ShellExecute + /// is given every inheritable handle its parent has. The kept connection is open for as long + /// as the test process is, so any child a test started that way - a server under test, a + /// tool - took a handle to it, and the connection then stayed open in the owner until that + /// child had exited too, however the test host ended. Run with a child that lived eight + /// seconds, the owner saw the connection close eight seconds after the host was gone, and at + /// once with the flag cleared. Later runtimes make their sockets uninheritable themselves. + /// + /// + /// Before the connect, on the socket the parameterless constructor has already made here, so + /// that the moment a child could still take it is the few instructions in between. Only the + /// kept connection: an exchange on a connection of its own ends by shutting down its sending + /// half, which reaches the owner whoever else holds the handle. + /// + /// + static void KeepFromChildren(TcpClient client) { - kept?.Dispose(); - kept = null; + try + { + SetHandleInformation(client.Client.Handle, handleFlagInherit, 0); + } + catch (Exception exception) + // .NET Framework's shape on another runtime, with no kernel32 to ask + when (exception is DllNotFoundException or EntryPointNotFoundException) + { + } } + const int handleFlagInherit = 1; + + [DllImport("kernel32.dll")] + static extern bool SetHandleInformation(IntPtr handle, int mask, int flags); +#endif + /// /// True when a reply arrived and parsed, whatever it says. Callers that need the body, such as /// the tray listing pending snapshots, use this rather than . @@ -487,14 +715,44 @@ static void DropKept() /// and a listing answered from what was found ten minutes ago is a listing that misses one. /// The library's own sends pass true. /// + /// + /// A listing goes down a connection that is kept, where the owner keeps one and no other + /// listing is using it: see . Everything else is a connection of its own. + /// /// public static bool TrySend( ViewerMessage message, [NotNullWhen(true)] out ViewerResponse? response, int? port = null, TimeSpan? wait = null, - bool skipIfUnowned = false) => - Exchange(message, out response, out _, port, wait, skipIfUnowned); + bool skipIfUnowned = false) + { + if (!Lists(message)) + { + return Exchange(message, out response, out _, port, wait, skipIfUnowned); + } + + var endpointPort = port ?? Port; + switch (SendKept(listing, message, endpointPort, wait ?? timeout, out response)) + { + case KeptSend.Answered: + return response is not null; + case KeptSend.Unanswered: + return false; + } + + if (!Exchange(message, out response, out var keeps, port, wait, skipIfUnowned)) + { + return false; + } + + if (keeps) + { + KeepConnection(listing, endpointPort); + } + + return true; + } /// /// The ordinary exchange: a connection of its own, the request ended by closing the sending @@ -521,7 +779,7 @@ static bool Exchange( if (NothingListening(endpointPort)) { - Found(endpointPort, false); + FoundUnlisted(endpointPort); return false; } @@ -560,9 +818,11 @@ static bool Exchange( catch (Exception exception) when (Ignorable(exception)) { - // Only a connect that failed says the port is unowned. A connection that was accepted - // and then torn down is an owner behaving badly, which is not what the memory records - if (!connected) + // Only a connect that was refused says the port is unowned. A connection that was + // accepted and then torn down is an owner behaving badly, and a connect the machine + // could not make is no answer about the port, neither of which the memory records + if (!connected && + NobodyThere(exception)) { Found(endpointPort, false); } @@ -621,7 +881,7 @@ public static async Task SendAsync( if (NothingListening(endpointPort)) { - Found(endpointPort, false); + FoundUnlisted(endpointPort); return SendOutcome.NoOwner; } @@ -717,9 +977,11 @@ exception is not OperationCanceledException && catch (Exception exception) when (exception is not OperationCanceledException && Ignorable(exception)) { - // As on the synchronous overload: a connect that failed is an unowned port, and an - // accepted connection that fell over afterwards is not - if (!connected) + // As on the synchronous overload: a connect that was refused is an unowned port, and + // neither an accepted connection that fell over afterwards nor a connect the machine + // could not make is + if (!connected && + NobodyThere(exception)) { Found(endpointPort, false); } diff --git a/src/DiffEngine/Protocol/ViewerServer.cs b/src/DiffEngine/Protocol/ViewerServer.cs index 14cfc07c..ea5f2e42 100644 --- a/src/DiffEngine/Protocol/ViewerServer.cs +++ b/src/DiffEngine/Protocol/ViewerServer.cs @@ -190,7 +190,8 @@ async Task Accept(Cancel cancel) /// 16,000 ports to give out and keeps each for two minutes, a passing inline verification /// settles once, and so a large enough green run with a tray answering used up the machine's /// ports on telling the tray nothing. A client that has many of them to send keeps one - /// connection instead: see . + /// connection instead: see . And a second + /// for its listings, which a window showing this owner's queue sends five times a second. /// /// /// No request starts with this line, since every one starts with its version, so an owner diff --git a/src/DiffEngine/TextDiff/LineAnchors.cs b/src/DiffEngine/TextDiff/LineAnchors.cs new file mode 100644 index 00000000..8d54c9f6 --- /dev/null +++ b/src/DiffEngine/TextDiff/LineAnchors.cs @@ -0,0 +1,215 @@ +using System.Buffers; + +/// +/// The elements two sequences can be lined up by when searching for a diff has cost too much: +/// those that occur once in each, and of those the longest run that is in the same order in both. +/// +/// An element that is on each side once can only be matched with itself, so nothing has to be +/// searched to pair them up, and the longest run of the pairs still in order is found by patience +/// sorting, in the time of a sort. Each pair of the run is a point both sequences pass through, and +/// what lies between one point and the next is a diff of its own, a small one. +/// +/// +/// This is what falls back on, never what it starts with. A run of unique +/// elements in order is a diff, but not always the smallest: elements that repeat can line up +/// better some other way. So it is only asked for by a search that has already given up on the +/// smallest. +/// +/// +static class LineAnchors +{ + /// + /// The run, as positions in and the positions in + /// they pair with, both ascending. The arrays are rented, and the caller's to return, when + /// the count that comes back is more than zero. + /// + /// One sequence. + /// The other. + /// + /// How long a run is worth having. A shorter one comes back as none, and with fewer pairs + /// than this there is none to look for. + /// + /// Where each element of the run is in . + /// Where each is in . + public static int Find(ReadOnlySpan a, ReadOnlySpan b, int needed, out int[]? xs, out int[]? ys) + { + xs = null; + ys = null; + var most = Math.Min(a.Length, b.Length); + if (most == 0 || + most < needed) + { + return 0; + } + + var pool = ArrayPool.Shared; + var pairX = pool.Rent(most); + var pairY = pool.Rent(most); + try + { + var pairs = Pairs(a, b, pairX, pairY); + if (pairs == 0 || + pairs < needed) + { + return 0; + } + + // In the order of one side, the run is the longest increasing one of the other's + Array.Sort(pairX, pairY, 0, pairs); + return Longest(pairX, pairY, pairs, needed, out xs, out ys); + } + finally + { + pool.Return(pairX); + pool.Return(pairY); + } + } + + /// + /// Every element that is in each sequence exactly once, as where it is in each. Found by + /// sorting each side's elements with their positions and walking the two together, so nothing + /// is assumed about what an element is beyond that two of them can be compared. + /// + static int Pairs(ReadOnlySpan a, ReadOnlySpan b, int[] pairX, int[] pairY) + { + var pool = ArrayPool.Shared; + var valuesA = pool.Rent(a.Length); + var placesA = pool.Rent(a.Length); + var valuesB = pool.Rent(b.Length); + var placesB = pool.Rent(b.Length); + try + { + Sorted(a, valuesA, placesA); + Sorted(b, valuesB, placesB); + var pairs = 0; + var indexA = 0; + var indexB = 0; + while (indexA < a.Length && + indexB < b.Length) + { + var value = valuesA[indexA]; + if (value < valuesB[indexB]) + { + indexA++; + continue; + } + + if (value > valuesB[indexB]) + { + indexB++; + continue; + } + + var endA = indexA + 1; + while (endA < a.Length && + valuesA[endA] == value) + { + endA++; + } + + var endB = indexB + 1; + while (endB < b.Length && + valuesB[endB] == value) + { + endB++; + } + + if (endA - indexA == 1 && + endB - indexB == 1) + { + pairX[pairs] = placesA[indexA]; + pairY[pairs] = placesB[indexB]; + pairs++; + } + + indexA = endA; + indexB = endB; + } + + return pairs; + } + finally + { + pool.Return(valuesA); + pool.Return(placesA); + pool.Return(valuesB); + pool.Return(placesB); + } + } + + static void Sorted(ReadOnlySpan sequence, int[] values, int[] places) + { + sequence.CopyTo(values); + for (var index = 0; index < sequence.Length; index++) + { + places[index] = index; + } + + Array.Sort(values, places, 0, sequence.Length); + } + + /// + /// Patience sorting over the pairs' second halves. tops holds, for each length of run + /// so far, the pair that ends the one ending lowest, and each pair remembers the pair before + /// it in the run it extended, so the longest can be read back from its end. + /// + static int Longest(int[] pairX, int[] pairY, int pairs, int needed, out int[]? xs, out int[]? ys) + { + xs = null; + ys = null; + var pool = ArrayPool.Shared; + var tops = pool.Rent(pairs); + var before = pool.Rent(pairs); + try + { + var length = 0; + for (var pair = 0; pair < pairs; pair++) + { + var y = pairY[pair]; + var low = 0; + var high = length; + while (low < high) + { + var middle = (low + high) / 2; + if (pairY[tops[middle]] < y) + { + low = middle + 1; + } + else + { + high = middle; + } + } + + before[pair] = low == 0 ? -1 : tops[low - 1]; + tops[low] = pair; + if (low == length) + { + length++; + } + } + + if (length < needed) + { + return 0; + } + + xs = pool.Rent(length); + ys = pool.Rent(length); + var at = tops[length - 1]; + for (var index = length - 1; index >= 0; index--) + { + xs[index] = pairX[at]; + ys[index] = pairY[at]; + at = before[at]; + } + + return length; + } + finally + { + pool.Return(tops); + pool.Return(before); + } + } +} diff --git a/src/DiffEngine/TextDiff/MyersDiff.cs b/src/DiffEngine/TextDiff/MyersDiff.cs index 5d28ea46..81ab7cb9 100644 --- a/src/DiffEngine/TextDiff/MyersDiff.cs +++ b/src/DiffEngine/TextDiff/MyersDiff.cs @@ -19,6 +19,14 @@ /// fewer elements unchanged than it could have. Nothing the search used to do in a tenth of a /// second gets that far, so what was quick is exactly as it was. /// +/// +/// A search that settles first asks whether the two sequences can be lined up without searching, +/// by the elements each has once (). The same elements in another order +/// are edits from end to end, so a search gets nowhere in them however deep it goes, and split +/// wherever it stopped they came out as nearly all changed. The longest run of them still in +/// order is found in the time of a sort, and the sequences are split at every element of it at +/// once. +/// /// static class MyersDiff { @@ -47,6 +55,14 @@ static class MyersDiff /// internal const int MinimumDepth = 256; + /// + /// A run of elements in order, each on both sides once, is taken as how the sequences line up + /// once it is this long, and longer than whatever the search that settled had to go on. A + /// shorter one is a few lines that happen to be unique, and where they are says little about + /// the lines around them. + /// + internal const int AnchorRun = 16; + public static void Diff(ReadOnlySpan a, ReadOnlySpan b, Span changedA, Span changedB) => Diff(a, b, changedA, changedB, Budget, MinimumDepth); @@ -56,17 +72,19 @@ public static void Diff(ReadOnlySpan a, ReadOnlySpan b, Span cha /// Set for each element of with no match. /// /// The searching to do before settling. Given by the tests, along with - /// , which is the only way a search that settles can be had on - /// sequences small enough to check by other means. + /// and , which is the only way a + /// search that settles can be had on sequences small enough to check by other means. /// /// The depth a search is allowed with nothing left of the budget. + /// The shortest run of unique elements in order to split by. internal static void Diff( ReadOnlySpan a, ReadOnlySpan b, Span changedA, Span changedB, long budget, - int minimumDepth) + int minimumDepth, + int anchorRun = AnchorRun) { var length = VectorLength(a.Length, b.Length); int[]? rentedForward = null; @@ -79,7 +97,7 @@ internal static void Diff( : (rentedReverse = ArrayPool.Shared.Rent(length)).AsSpan(0, length); try { - Recurse(a, b, changedA, changedB, forward, reverse, ref budget, Math.Max(1, minimumDepth)); + Recurse(a, b, changedA, changedB, forward, reverse, ref budget, new(Math.Max(1, minimumDepth), Math.Max(1, anchorRun)), long.MaxValue); } finally { @@ -104,6 +122,24 @@ static int VectorLength(int n, int m) => /// is no longer follows from where the split is: one a search settled for can leave almost /// everything on either side of it. /// + /// One sequence, or the part of it left to diff. + /// The other. + /// The flags of . + /// The flags of . + /// The forward search's diagonals, shared by every search of the diff. + /// The reverse search's. + /// What is left of the searching the diff may do. + /// What a search with nothing left to spend may still do. + /// + /// How many elements, between the two sides, a part may have and still be looked at for + /// anchors when a search of it settles. Finding them costs a sort of the part, and a diff + /// that settles makes a split every few hundred elements, so looking after every one would be + /// the length squared again. Every part until one has been looked at; then nothing inside a + /// part that had no run worth splitting by, where a smaller piece of the same is unlikely to + /// have one either, and inside a part that was split by one only a part of half its size or + /// less. A part is inside at most as many such halvings as its length has, and the parts at + /// each are apart from one another, so all of it together is a few sorts of the whole. + /// static void Recurse( ReadOnlySpan a, ReadOnlySpan b, @@ -112,7 +148,8 @@ static void Recurse( Span forward, Span reverse, ref long budget, - int minimumDepth) + Limits limits, + long anchorBelow) { while (true) { @@ -158,9 +195,21 @@ static void Recurse( // whole of the grid when that is within reach, which is a search that cannot settle. var deepest = (a.Length + b.Length + 1) / 2; var affordable = budget <= 0 ? 0 : (long) Math.Sqrt(budget); - var limit = (int) Math.Min(deepest, Math.Max(minimumDepth, affordable)); - var found = TryMiddleSnake(a, b, forward, reverse, limit, out var x, out var y, out var depth); + var limit = (int) Math.Min(deepest, Math.Max(limits.Depth, affordable)); + var found = TryMiddleSnake(a, b, forward, reverse, limit, out var x, out var y, out var depth, out var settled, out var passed); budget -= (long) depth * depth; + if (settled && + (long) a.Length + b.Length <= anchorBelow) + { + var size = (long) a.Length + b.Length; + if (TryAnchors(a, b, changedA, changedB, forward, reverse, ref budget, limits, Math.Max(limits.AnchorRun, passed + 1), size / 2)) + { + return; + } + + anchorBelow = 0; + } + if (!found) { changedA.Fill(true); @@ -170,7 +219,7 @@ static void Recurse( if (x + y <= a.Length - x + b.Length - y) { - Recurse(a[..x], b[..y], changedA[..x], changedB[..y], forward, reverse, ref budget, minimumDepth); + Recurse(a[..x], b[..y], changedA[..x], changedB[..y], forward, reverse, ref budget, limits, anchorBelow); a = a[x..]; b = b[y..]; changedA = changedA[x..]; @@ -178,7 +227,7 @@ static void Recurse( } else { - Recurse(a[x..], b[y..], changedA[x..], changedB[y..], forward, reverse, ref budget, minimumDepth); + Recurse(a[x..], b[y..], changedA[x..], changedB[y..], forward, reverse, ref budget, limits, anchorBelow); a = a[..x]; b = b[..y]; changedA = changedA[..x]; @@ -187,6 +236,72 @@ static void Recurse( } } + /// + /// What a search may do once the budget has gone, and what a settled one may split by. + /// + /// See . + /// See . + readonly record struct Limits(int Depth, int AnchorRun); + + /// + /// Splits two sequences at every element of the longest run of elements still in order that + /// each has once, and diffs what lies between one and the next. False, having done nothing, + /// when there is no such run of elements. + /// + /// The elements of the run are the same on both sides and stay unflagged, which is what + /// unchanged is. Each part between two of them is a diff of its own, searched with whatever + /// budget is left, and all the parts together are at least as much unchanged as the run is + /// long. + /// + /// + static bool TryAnchors( + ReadOnlySpan a, + ReadOnlySpan b, + Span changedA, + Span changedB, + Span forward, + Span reverse, + ref long budget, + Limits limits, + int needed, + long anchorBelow) + { + var count = LineAnchors.Find(a, b, needed, out var xs, out var ys); + if (count == 0) + { + return false; + } + + try + { + var fromX = 0; + var fromY = 0; + for (var index = 0; index < count; index++) + { + var x = xs![index]; + var y = ys![index]; + // Nothing between two that follow straight on from one another, which in a run + // that was found whole is most of them + if (x > fromX || + y > fromY) + { + Recurse(a[fromX..x], b[fromY..y], changedA[fromX..x], changedB[fromY..y], forward, reverse, ref budget, limits, anchorBelow); + } + + fromX = x + 1; + fromY = y + 1; + } + + Recurse(a[fromX..], b[fromY..], changedA[fromX..], changedB[fromY..], forward, reverse, ref budget, limits, anchorBelow); + return true; + } + finally + { + ArrayPool.Shared.Return(xs!); + ArrayPool.Shared.Return(ys!); + } + } + /// /// Searches forward from the start and backward from the end, one edit at a time, until the /// two paths overlap. The point where they meet lies on an optimal path, and splitting there @@ -203,6 +318,12 @@ static void Recurse( /// both sequences changed. is how many edits deep it went either way, /// which is what it is charged for. /// + /// + /// says the split is one it settled for, and + /// what that split has going for it: the elements the search passed + /// as the same on its way there, or the run that follows from where the sequences were found + /// to line up. A run of anchors has to be longer than that to be split by instead. + /// /// static bool TryMiddleSnake( ReadOnlySpan a, @@ -212,8 +333,12 @@ static bool TryMiddleSnake( int limit, out int splitX, out int splitY, - out int depth) + out int depth, + out bool settled, + out int passed) { + settled = false; + passed = 0; var n = a.Length; var m = b.Length; var maxD = (n + m + 1) / 2; @@ -344,11 +469,14 @@ static bool TryMiddleSnake( if (d >= limit) { depth = d + 1; + settled = true; var furthest = TryFurthest(n, m, d, offset, forward, reverse, out splitX, out splitY, out var matched); - if (TryDisplaced(a, b, depth, matched, out var displacedX, out var displacedY)) + passed = matched; + if (TryDisplaced(a, b, depth, matched, out var displacedX, out var displacedY, out var run)) { splitX = displacedX; splitY = displacedY; + passed = run; return true; } @@ -394,10 +522,11 @@ static bool TryMiddleSnake( /// scattered along a diagonal it was already following. /// /// - static bool TryDisplaced(ReadOnlySpan a, ReadOnlySpan b, int depth, int matched, out int splitX, out int splitY) + static bool TryDisplaced(ReadOnlySpan a, ReadOnlySpan b, int depth, int matched, out int splitX, out int splitY, out int run) { splitX = 0; splitY = 0; + run = 0; // No further than the search looked, which was its depth squared, so a scan never costs // more than the search it follows did var window = (int) Math.Min(int.MaxValue, (long) depth * depth); @@ -412,12 +541,14 @@ static bool TryDisplaced(ReadOnlySpan a, ReadOnlySpan b, int depth, in (!fromB || inA <= inB)) { splitX = inA; + run = runA; return true; } if (fromB) { splitY = inB; + run = runB; return true; } diff --git a/src/DiffEngine/Viewer/ViewerLaunchGate.cs b/src/DiffEngine/Viewer/ViewerLaunchGate.cs index a1a3735a..4edbbbac 100644 --- a/src/DiffEngine/Viewer/ViewerLaunchGate.cs +++ b/src/DiffEngine/Viewer/ViewerLaunchGate.cs @@ -64,7 +64,7 @@ enum ViewerLaunchOutcome /// already up is not a new instance and spends nothing, which is why the caller cannot ask: it /// would charge all twenty of the callers above for the one window between them. Asked after the /// ownership probe, so the nineteen that find an owner still forward their work when the cap is -/// long since reached. +/// long since reached. And given back when the launch fails, since no window came of it. /// /// static class ViewerLaunchGate @@ -100,13 +100,23 @@ static class ViewerLaunchGate /// Supplied by the tests that are about the gate rather than about the cap, since the count it /// reads is shared with every other launch the process has made. /// + /// + /// Returns the slot spent, for a launch that then failed: the + /// cap is on windows, and that one opened none. Kept, five viewers that could not be run - a + /// copy too old for its arguments, one with no runtime - used up the cap, and every pair + /// after them was told too many diff tools were running when none was. Defaults to + /// where the slot was asked of it, and to nothing where a test + /// supplied the asking. + /// public static ViewerLaunchOutcome Launch( Func retry, Func launch, Func? isOwned = null, - Func? canLaunch = null) + Func? canLaunch = null, + Action? giveBack = null) { isOwned ??= () => ViewerClient.IsOwned(); + giveBack ??= canLaunch is null ? MaxInstance.GiveBack : () => { }; canLaunch ??= () => !MaxInstance.Reached(); bool owned; gate.Wait(); @@ -126,6 +136,7 @@ public static ViewerLaunchOutcome Launch( if (viewer is null || !WaitForBind(viewer, isOwned)) { + giveBack(); return ViewerLaunchOutcome.Failed; } } @@ -159,9 +170,11 @@ public static async Task LaunchAsync( Func> launch, Cancel cancel, Func? isOwned = null, - Func? canLaunch = null) + Func? canLaunch = null, + Action? giveBack = null) { isOwned ??= () => ViewerClient.IsOwned(); + giveBack ??= canLaunch is null ? MaxInstance.GiveBack : () => { }; canLaunch ??= () => !MaxInstance.Reached(); bool owned; await gate.WaitAsync(cancel).ConfigureAwait(false); @@ -179,6 +192,7 @@ public static async Task LaunchAsync( if (viewer is null || !await WaitForBindAsync(viewer, isOwned, cancel).ConfigureAwait(false)) { + giveBack(); return ViewerLaunchOutcome.Failed; } } diff --git a/src/DiffEngineTray.Tests/FileLockUtils.cs b/src/DiffEngineTray.Tests/FileLockUtils.cs index 1f396c06..ff0e241a 100644 --- a/src/DiffEngineTray.Tests/FileLockUtils.cs +++ b/src/DiffEngineTray.Tests/FileLockUtils.cs @@ -1,8 +1,21 @@ static class FileLockUtils { - public static Process StartFileLockProcess(string path) + /// The file the process holds open, with nothing else let in. + /// + /// A received file to name on the command line, for a process standing in for a diff tool + /// showing that pair: the tray tracks a process as a move's tool only when it was started + /// with the move's received file. + /// + public static Process StartFileLockProcess(string path, string? shows = null) { var script = $"$f = [System.IO.File]::Open('{path.Replace("'", "''")}', 'Open', 'ReadWrite', 'None'); [Console]::WriteLine('locked'); Start-Sleep -Seconds 60"; + if (shows is not null) + { + // A comment to the script, and the last thing inside the quotes the command is passed + // in, so the path ends at a quote as it does where a diff tool is handed one + script = $"{script} # {shows}"; + } + var process = new Process { StartInfo = new() diff --git a/src/DiffEngineTray.Tests/KeyRegisterTests.cs b/src/DiffEngineTray.Tests/KeyRegisterTests.cs index c0a3debb..2434ffcd 100644 --- a/src/DiffEngineTray.Tests/KeyRegisterTests.cs +++ b/src/DiffEngineTray.Tests/KeyRegisterTests.cs @@ -8,34 +8,32 @@ /// The filter is called directly here and never through a message loop, so nothing thrown can /// reach WinForms' own handling of it. /// +/// +/// Nothing is registered with the desktop. These used to take Ctrl+Alt+Shift+F24 for the length +/// of each test, a key held for every process on the machine: they failed wherever something +/// else had it, and beside themselves whenever two runs of them overlapped. What they are about +/// is what happens to a press once a key is bound, which the desktop has no part in. +/// /// -[NotInParallel] public class KeyRegisterTests { const int id = 9731; - - // Every modifier and a key no keyboard has, since registering is for the whole desktop for - // as long as the test takes const KeyModifiers modifiers = KeyModifiers.Control | KeyModifiers.Alt | KeyModifiers.Shift; [Test] public async Task AnActionThatThrowsIsReportedAndTheKeyIsStillHandled() { - // No awaiting until the register is disposed: a hot key registered with no window belongs - // to the thread that registered it, which is the one that has to take it back + var desktop = new Desktop(); bool bound; - var handled = false; - using (var register = new KeyRegister(IntPtr.Zero)) + bool handled; + using (var register = desktop.Register()) { bound = register.TryAddBinding( id, modifiers, Keys.F24, () => throw new InvalidOperationException("TheHotKeyFailure")); - if (bound) - { - handled = register.PreFilterMessage(ref HotKeyPressed); - } + handled = register.PreFilterMessage(ref HotKeyPressed); } await Assert.That(bound).IsTrue(); @@ -46,21 +44,91 @@ public async Task AnActionThatThrowsIsReportedAndTheKeyIsStillHandled() [Test] public async Task AnActionRuns() { + var desktop = new Desktop(); bool bound; - var handled = false; + bool handled; var ran = false; - using (var register = new KeyRegister(IntPtr.Zero)) + using (var register = desktop.Register()) { bound = register.TryAddBinding(id, modifiers, Keys.F24, () => ran = true); - if (bound) - { - handled = register.PreFilterMessage(ref HotKeyPressed); - } + handled = register.PreFilterMessage(ref HotKeyPressed); } await Assert.That(bound).IsTrue(); await Assert.That(handled).IsTrue(); await Assert.That(ran).IsTrue(); + await Assert.That(desktop.Asked).IsEquivalentTo([(id, modifiers, Keys.F24)]); + } + + /// + /// A combination something else holds is refused by the desktop, and the press that then + /// never comes must not find an action waiting for it. + /// + [Test] + public async Task AKeyTheDesktopRefusesIsNotBound() + { + var desktop = new Desktop + { + Refuses = true + }; + bool bound; + bool handled; + var ran = false; + using (var register = desktop.Register()) + { + bound = register.TryAddBinding(id, modifiers, Keys.F24, () => ran = true); + handled = register.PreFilterMessage(ref HotKeyPressed); + } + + await Assert.That(bound).IsFalse(); + await Assert.That(handled).IsFalse(); + await Assert.That(ran).IsFalse(); + } + + [Test] + public async Task EveryKeyBoundIsGivenBackOnDispose() + { + var desktop = new Desktop(); + using (var register = desktop.Register()) + { + register.TryAddBinding(id, modifiers, Keys.F24, () => + { + }); + register.TryAddBinding(id + 1, modifiers, Keys.F23, () => + { + }); + } + + await Assert.That(desktop.Held).IsEmpty(); + } + + /// + /// What a register asks of the desktop, answered here: which ids hold a key, and whether the + /// next one asked for is refused. + /// + class Desktop + { + public bool Refuses { get; init; } + + public HashSet Held { get; } = []; + + public List<(int id, KeyModifiers modifiers, Keys key)> Asked { get; } = []; + + public KeyRegister Register() => + new( + IntPtr.Zero, + (_, id, modifiers, key) => + { + Asked.Add((id, modifiers, key)); + if (Refuses) + { + return false; + } + + Held.Add(id); + return true; + }, + (_, id) => Held.Remove(id)); } // WM_HOTKEY, with the binding's id where the message carries it diff --git a/src/DiffEngineTray.Tests/MenuBuilderTest.cs b/src/DiffEngineTray.Tests/MenuBuilderTest.cs index 9498db5a..b78d8469 100644 --- a/src/DiffEngineTray.Tests/MenuBuilderTest.cs +++ b/src/DiffEngineTray.Tests/MenuBuilderTest.cs @@ -99,6 +99,75 @@ public async Task SnapshotThatWasNotWrittenCarriesItsReason() menu.Close(); } + /// + /// A delete "Accept all" passes over, because a move was accepted onto its file. It was listed + /// like every other delete, under a button that had just been pressed without removing it, and + /// only the log said why. Marked and given its reason the way a snapshot that was not written + /// is. + /// + /// The items are built without the menu being opened, so nothing is put on the desktop. + /// + /// + [Test] + public async Task ADeleteAcceptAllWouldKeepCarriesItsReason() + { + var directory = Path.Combine(Path.GetTempPath(), $"MenuBuilderTest_{Guid.NewGuid():N}"); + var staged = Path.Combine(directory, "staged"); + Directory.CreateDirectory(staged); + try + { + var received = Path.Combine(staged, "Sample.Test.received.txt"); + var verified = Path.Combine(directory, "Sample.Test.verified.txt"); + var plain = Path.Combine(directory, "Plain.verified.txt"); + await File.WriteAllTextAsync(received, "received"); + await File.WriteAllTextAsync(verified, "verified"); + await File.WriteAllTextAsync(plain, "plain"); + await using var tracker = new RecordingTracker(); + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + tracker.AddDelete(verified); + tracker.AddDelete(plain); + await tracker.AcceptAll(); + // The plain delete went with the sweep, and is tracked again for something to compare + await File.WriteAllTextAsync(plain, "plain"); + tracker.AddDelete(plain); + + var items = MenuBuilder.BuildTrackingMenuItems(tracker).ToList(); + try + { + var deletes = items + .OfType() + .ToList(); + var held = deletes.Single(_ => _.Text!.StartsWith("Sample.Test.verified.txt")); + await Assert.That(held.Text).IsEqualTo("Sample.Test.verified.txt !"); + await Assert.That(held.ToolTipText).IsEqualTo(Tracker.WroteItsFile); + var label = held.DropDownItems + .OfType() + .Single(_ => !_.Enabled); + await Assert.That(label.ToolTipText).IsEqualTo(Tracker.WroteItsFile); + await Assert.That(label.Text!.Length).IsLessThanOrEqualTo(60); + await Assert.That(label.Text).StartsWith("Kept by 'Accept all': a move was accepted onto this file"); + + var other = deletes.Single(_ => _.Text!.StartsWith("Plain.verified.txt")); + await Assert.That(other.Text).IsEqualTo("Plain.verified.txt"); + await Assert.That(other.DropDownItems + .OfType() + .Where(_ => !_.Enabled)) + .IsEmpty(); + } + finally + { + foreach (var item in items) + { + item.Dispose(); + } + } + } + finally + { + Directory.Delete(directory, true); + } + } + /// /// The same item with its drop down open, which is where the reason lives. The menu images /// elsewhere in this file stop at the top level and show only the marker, so nothing here saw diff --git a/src/DiffEngineTray.Tests/RecordingTracker.cs b/src/DiffEngineTray.Tests/RecordingTracker.cs index 92db2cf1..c01da7d4 100644 --- a/src/DiffEngineTray.Tests/RecordingTracker.cs +++ b/src/DiffEngineTray.Tests/RecordingTracker.cs @@ -1,4 +1,4 @@ -class RecordingTracker(LockedFilesResolver? lockedFilesResolver = null, Action? acceptFailed = null, Action? inlineFailed = null, IInlineHost? inline = null) : +class RecordingTracker(LockedFilesResolver? lockedFilesResolver = null, Action? acceptFailed = null, Action? inlineFailed = null, IInlineHost? inline = null, Action? scanFailing = null) : Tracker( () => { @@ -9,7 +9,8 @@ class RecordingTracker(LockedFilesResolver? lockedFilesResolver = null, Action + /// The version marker says a tray is running, and a clean exit removes it. A logoff reached + /// none of that, so the marker outlived the tray: only an owning tray was listening for the + /// session ending at all, and what it did then was stage its queue and nothing else. + /// + /// The removal is handed in, as Program hands in the real one, so nothing here touches the + /// marker of a tray running on this machine. + /// + /// + [Test] + public async Task ATrayThatDoesNotOwnTheQueueRemovesItsMarkerAsTheSessionEnds() + { + var removed = 0; + using (var window = new SessionEndWindow(Program.SessionEnding(null, () => removed++))) + { + SendMessage(window.Handle, queryEndSession, IntPtr.Zero, logoff); + SendMessage(window.Handle, endSession, new(1), logoff); + } + + await Assert.That(removed).IsEqualTo(1); + } + + [Test] + public async Task AnOwningTrayStagesItsQueueAndThenRemovesItsMarker() + { + var directory = Path.Combine(Path.GetTempPath(), $"tray-session-end-{Guid.NewGuid():N}"); + Directory.CreateDirectory(directory); + var host = OwnedInlineHost.TryOwn(_ => { }, new FakeLauncher(), 0) ?? + throw new("Could not bind an ephemeral port."); + try + { + await File.WriteAllTextAsync(Path.Combine(directory, "Sample.csproj"), ""); + var source = Path.Combine(directory, "SampleTests.cs"); + await File.WriteAllTextAsync(source, "// sample"); + host.Start(); + await Assert.That(Send(host, source).Ok).IsTrue(); + var staging = Path.Combine(directory, "obj", InlineStaging.DirectoryName); + bool? stagedWhenRemoved = null; + + using (var window = new SessionEndWindow( + Program.SessionEnding( + host, + () => stagedWhenRemoved = Directory.Exists(staging) && + Directory.GetFiles(staging).Any(_ => _.EndsWith(".inlinepatch"))))) + { + SendMessage(window.Handle, queryEndSession, IntPtr.Zero, logoff); + SendMessage(window.Handle, endSession, new(1), logoff); + } + + await Assert.That(stagedWhenRemoved).IsTrue(); + } + finally + { + await host.DisposeAsync(); + Directory.Delete(directory, recursive: true); + } + } + static void EndSession(OwnedInlineHost host) { using var window = new SessionEndWindow(host.SessionEnding); diff --git a/src/DiffEngineTray.Tests/StubInlineHost.cs b/src/DiffEngineTray.Tests/StubInlineHost.cs index cc14a9e4..c98214e8 100644 --- a/src/DiffEngineTray.Tests/StubInlineHost.cs +++ b/src/DiffEngineTray.Tests/StubInlineHost.cs @@ -93,12 +93,20 @@ public bool AcceptAll(out string? message, out bool refused) return AcceptAllSucceeds; } + /// + /// What a bulk discard reports. False is a queue that could not be reached, or one that would + /// not discard, with as what it said. + /// + public bool DiscardAllSucceeds { get; init; } = true; + + public string? DiscardAllMessage { get; init; } + public bool DiscardAll(out string? message) { - message = null; + message = DiscardAllMessage; DiscardStarted.Set(); DiscardBlock?.Wait(TimeSpan.FromSeconds(10)); - return true; + return DiscardAllSucceeds; } public void Focus(PendingSnapshot snapshot) diff --git a/src/DiffEngineTray.Tests/TrackerClearTest.cs b/src/DiffEngineTray.Tests/TrackerClearTest.cs index 17cd69ee..13a743b1 100644 --- a/src/DiffEngineTray.Tests/TrackerClearTest.cs +++ b/src/DiffEngineTray.Tests/TrackerClearTest.cs @@ -11,6 +11,93 @@ public async Task Simple() await tracker.AssertEmpty(); } + /// + /// "Discard (n)" is a menu click or a hot key, so the thread that asks is the one drawing + /// everything. Discarding a move ends its diff tool and waits up to half a second for it to + /// go, and the moves were discarded before the call returned: only the queue's half had been + /// moved to a worker. + /// + /// Asked from a thread of its own, which is never the worker's. Asked from the pool, the + /// thread that asked can be the one that picks the work up once the test awaits. + /// + /// + [Test] + public async Task TheFilesAreNotDiscardedOnTheThreadThatAsked() + { + await using var tracker = new WatchedTracker(); + tracker.AddDelete(file1); + tracker.AddMove(file2, file2, "theExe", "theArguments", true, null); + + var asked = new TaskCompletionSource<(int thread, Task clearing)>(TaskCreationOptions.RunContinuationsAsynchronously); + var thread = new Thread(() => asked.SetResult((Environment.CurrentManagedThreadId, tracker.Clear()))) + { + IsBackground = true + }; + thread.Start(); + var (caller, clearing) = await asked.Task; + await clearing; + + await Assert.That(tracker.DiscardedOn).IsNotNull(); + await Assert.That(tracker.DiscardedOn).IsNotEqualTo(caller); + await tracker.AssertEmpty(); + } + + /// + /// A queue that would not discard, or could not be reached, keeps its snapshots, and the menu + /// goes on listing them under a button that had just been pressed to be rid of them. The log + /// was the only place that said why. + /// + [Test] + public async Task ABulkDiscardTheQueueDidNotCarryOutIsSaid() + { + var warnings = new ConcurrentQueue(); + await using var tracker = new RecordingTracker( + inlineFailed: warnings.Enqueue, + inline: new StubInlineHost(new PendingSnapshot(@"c:\repo\sample.cs|12", "Sample.cs:12", null)) + { + DiscardAllSucceeds = false, + DiscardAllMessage = "The snapshot viewer did not answer." + }); + tracker.AddDelete(file1); + + await tracker.Clear(); + + await Assert.That(warnings).IsEquivalentTo( + ["Could not discard the pending snapshots. The snapshot viewer did not answer."]); + // The files went all the same, and the snapshots are still what the menu shows + await Assert.That(tracker.Deletes).IsEmpty(); + await Assert.That(tracker.Snapshots).HasSingleItem(); + } + + [Test] + public async Task ABulkDiscardThatWorkedSaysNothing() + { + var warnings = new ConcurrentQueue(); + await using var tracker = new RecordingTracker( + inlineFailed: warnings.Enqueue, + inline: new StubInlineHost(new PendingSnapshot(@"c:\repo\sample.cs|12", "Sample.cs:12", null))); + + await tracker.Clear(); + + await Assert.That(warnings).IsEmpty(); + await Assert.That(tracker.Snapshots).IsEmpty(); + } + + /// + /// A tracker that says which thread its files were discarded on. + /// + class WatchedTracker : + RecordingTracker + { + public int? DiscardedOn { get; private set; } + + protected override int DiscardFiles() + { + DiscardedOn = Environment.CurrentManagedThreadId; + return base.DiscardFiles(); + } + } + public void Dispose() { File.Delete(file1); diff --git a/src/DiffEngineTray.Tests/TrackerLockedMoveTest.cs b/src/DiffEngineTray.Tests/TrackerLockedMoveTest.cs index 6cf3fa87..7ffbdcdb 100644 --- a/src/DiffEngineTray.Tests/TrackerLockedMoveTest.cs +++ b/src/DiffEngineTray.Tests/TrackerLockedMoveTest.cs @@ -115,7 +115,7 @@ public async Task Ignore_DropsTheKilledProcessFromThePendingMove() (_, _) => LockedFilesResponse.Ignore); var lockProcess = FileLockUtils.StartFileLockProcess(target1); // Stands in for the diff tool: a killable process the move is tracking - var toolProcess = FileLockUtils.StartFileLockProcess(temp2); + var toolProcess = FileLockUtils.StartFileLockProcess(temp2, shows: temp1); try { // Named as the tool, since a process running something else is not tracked diff --git a/src/DiffEngineTray.Tests/TrackerMoveOntoDeleteTest.cs b/src/DiffEngineTray.Tests/TrackerMoveOntoDeleteTest.cs index b7054513..bcc9b7ad 100644 --- a/src/DiffEngineTray.Tests/TrackerMoveOntoDeleteTest.cs +++ b/src/DiffEngineTray.Tests/TrackerMoveOntoDeleteTest.cs @@ -76,6 +76,143 @@ public async Task AcceptAllHoldsADeleteOfTheFileItsMoveJustWrote() await Assert.That(tracker.Deletes.Select(_ => _.File)).IsEquivalentTo([verified]); } + /// + /// The sweep that held the delete forgot why as it ended. The delete sat in the menu looking + /// like any other, and pressing "Accept all" again, which is what a delete still listed after + /// an accept-all invites, deleted the snapshot the first press had just accepted. + /// + [Test] + public async Task ASecondAcceptAllStillHoldsTheDelete() + { + var warnings = new ConcurrentQueue(); + await using var tracker = new RecordingTracker(inlineFailed: warnings.Enqueue); + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + var delete = tracker.AddDelete(verified); + await tracker.AcceptAll(); + + await tracker.AcceptAll(); + + await Assert.That(await File.ReadAllTextAsync(verified)).IsEqualTo("received"); + await Assert.That(tracker.Deletes.Select(_ => _.File)).IsEquivalentTo([verified]); + await Assert.That(tracker.HeldReason(delete)).IsEqualTo(Tracker.WroteItsFile); + // Said each time, since each press was an accept-all that left something undone + await Assert.That(warnings).IsEquivalentTo( + [ + Tracker.DeletesKept([delete]), + Tracker.DeletesKept([delete]) + ]); + } + + /// + /// The same loss with no sweep in the first half of it: the move accepted from its own menu + /// item, and then "Accept all" for what was left. + /// + [Test] + public async Task AMoveAcceptedOnItsOwnHoldsTheDeleteFromALaterAcceptAll() + { + await using var tracker = new RecordingTracker(); + var move = tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + tracker.AddDelete(verified); + tracker.Accept(move); + + await tracker.AcceptAll(); + + await Assert.That(await File.ReadAllTextAsync(verified)).IsEqualTo("received"); + await Assert.That(tracker.Deletes).HasSingleItem(); + } + + [Test] + public async Task ASecondWireSweepStillHoldsTheDelete() + { + await using var tracker = new RecordingTracker(); + ITrackedFiles tracked = tracker; + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + tracker.AddDelete(verified); + tracked.AcceptAllTracked(holdDeletes: false); + + var (accepted, kept) = tracked.AcceptAllTracked(holdDeletes: false); + + await Assert.That(accepted).IsEqualTo(0); + await Assert.That(kept).IsEqualTo(1); + await Assert.That(await File.ReadAllTextAsync(verified)).IsEqualTo("received"); + } + + /// + /// Held from a sweep, and no more than that. Whoever knows the file is redundant accepts the + /// delete itself, from its own item or from the header over the deletes, or over the wire by + /// its key. + /// + [Test] + public async Task AHeldDeleteIsCarriedOutWhenAcceptedOnItsOwn() + { + await using var tracker = new RecordingTracker(); + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + var delete = tracker.AddDelete(verified); + await tracker.AcceptAll(); + + tracker.Accept(delete); + + await Assert.That(File.Exists(verified)).IsFalse(); + await tracker.AssertEmpty(); + } + + /// + /// A test run that raises the delete again has looked at the file the move wrote and still + /// says nothing produces it. That is the later statement, and the next sweep carries it out. + /// + [Test] + public async Task ADeleteRaisedAgainAfterTheWriteIsCarriedOut() + { + await using var tracker = new RecordingTracker(); + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + var delete = tracker.AddDelete(verified); + await tracker.AcceptAll(); + + tracker.AddDelete(verified); + await Assert.That(tracker.HeldReason(delete)).IsNull(); + await tracker.AcceptAll(); + + await Assert.That(File.Exists(verified)).IsFalse(); + await tracker.AssertEmpty(); + } + + /// + /// A delete waiting on a move that is still pending is held for as long as the move is, and + /// no longer: discarded, the move is never going to write the file. + /// + [Test] + public async Task ADeleteWaitingOnAMoveIsCarriedOutOnceTheMoveIsDiscarded() + { + await using var tracker = new RecordingTracker(); + var move = tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + var delete = tracker.AddDelete(verified); + await Assert.That(tracker.HeldReason(delete)).IsEqualTo(Tracker.AwaitsItsFile); + + tracker.Discard(move); + await Assert.That(tracker.HeldReason(delete)).IsNull(); + await tracker.AcceptAll(); + + await Assert.That(File.Exists(verified)).IsFalse(); + await tracker.AssertEmpty(); + } + + /// + /// Where the reason is read by somebody who was not looking when the balloon went by: beside + /// the delete in the debug view, as it is in the menu. + /// + [Test] + public async Task TheDebugViewSaysWhyADeleteIsHeld() + { + await using var tracker = new RecordingTracker(); + tracker.AddMove(received, verified, "theExe", "theArguments", true, null); + tracker.AddDelete(verified); + await Assert.That(DebugReport.Build(tracker, DateTime.Now)).DoesNotContain(Tracker.WroteItsFile); + + await tracker.AcceptAll(); + + await Assert.That(DebugReport.Build(tracker, DateTime.Now)).Contains(Tracker.WroteItsFile); + } + /// /// The sweep a viewer displaying the tray's queue asks for, which has the same two halves in /// the same order. diff --git a/src/DiffEngineTray.Tests/TrackerMoveTest.cs b/src/DiffEngineTray.Tests/TrackerMoveTest.cs index 69e8b4cd..e85a4d86 100644 --- a/src/DiffEngineTray.Tests/TrackerMoveTest.cs +++ b/src/DiffEngineTray.Tests/TrackerMoveTest.cs @@ -25,12 +25,19 @@ public async Task AddSame() { await using var tracker = new RecordingTracker(); tracker.AddMove(file1, file1, "theExe", "theArguments", true, null); - using var process = Process.GetCurrentProcess(); - var processId = process.Id; - var tracked = tracker.AddMove(file1, file1, Environment.ProcessPath, "theArguments", false, processId); - await Assert.That(tracker.Moves).HasSingleItem(); - await Assert.That(tracked.Process!.Id).IsEqualTo(process.Id); - await Assert.That(tracker.TrackingAny).IsTrue(); + // Stands in for the diff tool showing the pair, started with the received file as one is + var process = FileLockUtils.StartFileLockProcess(file3, shows: file1); + try + { + var tracked = tracker.AddMove(file1, file1, process.MainModule!.FileName, "theArguments", false, process.Id); + await Assert.That(tracker.Moves).HasSingleItem(); + await Assert.That(tracked.Process!.Id).IsEqualTo(process.Id); + await Assert.That(tracker.TrackingAny).IsTrue(); + } + finally + { + FileLockUtils.Cleanup(process); + } } [Test] @@ -121,7 +128,7 @@ public async Task OpenDiffToolFromAMenuBuiltBeforeTheMoveWasUpdated() var toolLock = file3; // Stands in for an auto refresh diff tool that is still open, which is what DiffRunner // resends the id of - var tool = FileLockUtils.StartFileLockProcess(toolLock); + var tool = FileLockUtils.StartFileLockProcess(toolLock, shows: temp); try { // Nothing at this path exists, so if the launcher gets past the process it starts diff --git a/src/DiffEngineTray.Tests/TrackerProcessImageTest.cs b/src/DiffEngineTray.Tests/TrackerProcessImageTest.cs index a8f1b9ae..0ddff291 100644 --- a/src/DiffEngineTray.Tests/TrackerProcessImageTest.cs +++ b/src/DiffEngineTray.Tests/TrackerProcessImageTest.cs @@ -89,6 +89,161 @@ public async Task AReRunNamingAProcessRunningSomethingElseLeavesTheMoveWithNone( await Assert.That(stranger.WaitForExit(1000)).IsFalse(); } + /// + /// The image says which program a process is, and not which window. A stale id that Windows + /// handed to another copy of the same tool - the one open on the next snapshot along, or one + /// started by hand for a merge - passed on the image alone, was tracked as this pair's, and + /// was ended when the pair was accepted. The tool for a pair was started with its received + /// file, and this one was not. + /// + [Test] + public async Task AnotherCopyOfTheToolIsNotTrackedAndNotEnded() + { + await using var tracker = new RecordingTracker(); + var copy = StartAnotherCopy(shows: Path.Combine(directory, "received", "another.trackerprocessimage")); + try + { + var tracked = tracker.AddMove(temp, target, Image, "theArguments", true, copy.Id); + + await Assert.That(tracked.Process).IsNull(); + await Assert.That(tracked.IsOpen).IsFalse(); + + tracker.Accept(tracked); + + await Assert.That(tracker.Moves).IsEmpty(); + await Assert.That(copy.WaitForExit(1000)).IsFalse(); + } + finally + { + FileLockUtils.Cleanup(copy); + } + } + + /// + /// Open on a file whose path only starts as the received file's does. + /// + [Test] + public async Task ACopyOfTheToolShowingALongerPathIsNotTracked() + { + await using var tracker = new RecordingTracker(); + var copy = StartAnotherCopy(shows: $"{temp}.bak"); + try + { + var tracked = tracker.AddMove(temp, target, Image, "theArguments", true, copy.Id); + + await Assert.That(tracked.Process).IsNull(); + } + finally + { + FileLockUtils.Cleanup(copy); + } + } + + [Test] + public async Task AReRunNamingAnotherCopyOfTheToolLeavesTheMoveWithNone() + { + await using var tracker = new RecordingTracker(); + tracker.AddMove(temp, target, Image, "theArguments", true, stranger.Id); + var copy = StartAnotherCopy(shows: null); + try + { + var tracked = tracker.AddMove(temp, target, Image, "theArguments", true, copy.Id); + + await Assert.That(tracked.Process).IsNull(); + tracker.Accept(tracked); + await Assert.That(copy.WaitForExit(1000)).IsFalse(); + } + finally + { + FileLockUtils.Cleanup(copy); + } + } + + /// + /// What the comparison reads, asked of a process whose command line is known. + /// + [Test] + public async Task TheCommandLineOfAProcessIsRead() + { + var commandLine = ProcessEx.CommandLine(stranger); + + await Assert.That(commandLine).IsNotNull(); + await Assert.That(commandLine!).Contains("-NoProfile"); + await Assert.That(commandLine).Contains(temp); + await Assert.That(ProcessEx.WasStartedFor(commandLine, temp)).IsTrue(); + } + + /// + /// A 32 bit process asked from a 64 bit one, which is most diff tools: they install under + /// Program Files (x86). The command line of one is in another place in its memory, which is + /// why it is asked for rather than read out of there. + /// + [Test] + public async Task TheCommandLineOfA32BitProcessIsRead() + { + var powershell = Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.SystemX86), + @"WindowsPowerShell\v1.0\powershell.exe"); + // Nothing to ask where there is no 32 bit side, or no 32 bit PowerShell on it + if (!Environment.Is64BitProcess || + !File.Exists(powershell)) + { + return; + } + + using var process = new Process + { + StartInfo = new() + { + FileName = powershell, + Arguments = $"-NoProfile -Command \"[Console]::WriteLine('ready'); Start-Sleep -Seconds 60 # {temp}\"", + UseShellExecute = false, + CreateNoWindow = true, + RedirectStandardOutput = true + } + }; + process.Start(); + try + { + await process.StandardOutput.ReadLineAsync(); + + var commandLine = ProcessEx.CommandLine(process); + + await Assert.That(ProcessEx.WasStartedFor(commandLine, temp)).IsTrue(); + } + finally + { + process.Kill(); + process.WaitForExit(5000); + } + } + + [Test] + public async Task ACommandLineNamesAPathOnlyAsTheWholeOfAnArgument() + { + const string path = @"C:\temp\Sample.received.txt"; + + await Assert.That(ProcessEx.WasStartedFor($"\"tool.exe\" \"C:\\code\\Sample.verified.txt\" \"{path}\"", path)).IsTrue(); + await Assert.That(ProcessEx.WasStartedFor($"\"tool.exe\" /left:\"{path}\" /right:\"C:\\code\\Sample.verified.txt\"", path)).IsTrue(); + await Assert.That(ProcessEx.WasStartedFor($"tool.exe {path} other", path)).IsTrue(); + await Assert.That(ProcessEx.WasStartedFor($"tool.exe {path.ToUpperInvariant()}", path)).IsTrue(); + // Found first as the start of a longer path, and then as itself + await Assert.That(ProcessEx.WasStartedFor($"tool.exe \"{path}.bak\" \"{path}\"", path)).IsTrue(); + await Assert.That(ProcessEx.WasStartedFor($"tool.exe \"{path}.bak\"", path)).IsFalse(); + await Assert.That(ProcessEx.WasStartedFor("tool.exe \"C:\\temp\\Other.received.txt\"", path)).IsFalse(); + await Assert.That(ProcessEx.WasStartedFor(null, path)).IsFalse(); + } + + /// + /// A second process running the image the first does, started with another file or with none. + /// + Process StartAnotherCopy(string? shows) + { + var locked = Path.Combine(directory, $"locked-{Guid.NewGuid():N}.txt"); + File.WriteAllText(locked, ""); + return FileLockUtils.StartFileLockProcess(locked, shows); + } + [Test] public async Task AMoveNamingNoToolIsNotGivenAProcess() { @@ -113,7 +268,8 @@ public TrackerProcessImageTest() File.WriteAllText(target, "verified"); var locked = Path.Combine(directory, "locked.txt"); File.WriteAllText(locked, ""); - stranger = FileLockUtils.StartFileLockProcess(locked); + // Started with the received file on its command line, as a diff tool showing the pair is + stranger = FileLockUtils.StartFileLockProcess(locked, shows: temp); } public void Dispose() diff --git a/src/DiffEngineTray.Tests/TrackerProcessReleaseTest.cs b/src/DiffEngineTray.Tests/TrackerProcessReleaseTest.cs index 93d5417d..3faa8be9 100644 --- a/src/DiffEngineTray.Tests/TrackerProcessReleaseTest.cs +++ b/src/DiffEngineTray.Tests/TrackerProcessReleaseTest.cs @@ -112,7 +112,7 @@ public TrackerProcessReleaseTest() File.WriteAllText(target, "verified"); var locked = Path.Combine(directory, "locked.txt"); File.WriteAllText(locked, ""); - tool = FileLockUtils.StartFileLockProcess(locked); + tool = FileLockUtils.StartFileLockProcess(locked, shows: temp); } public void Dispose() diff --git a/src/DiffEngineTray.Tests/TrackerScanTest.cs b/src/DiffEngineTray.Tests/TrackerScanTest.cs index fdf8fb86..97f30f7f 100644 --- a/src/DiffEngineTray.Tests/TrackerScanTest.cs +++ b/src/DiffEngineTray.Tests/TrackerScanTest.cs @@ -41,6 +41,31 @@ public async Task AMoveThatCameToEqualItsTargetIsDropped() await Assert.That(tracker.Moves).IsEmpty(); } + /// + /// A pair found different is remembered, so the next scan does not read both files through + /// again. Nothing forgot it when the move left, so the tray kept one for every received file + /// it had ever compared. + /// + [Test] + public async Task WhatAMoveWasFoundToBeIsForgottenOnceItHasLeft() + { + await using var tracker = new RecordingTracker(); + File.WriteAllText(temp, "received"); + File.WriteAllText(target, "verified"); + var scanned = tracker.AddMove(temp, target, "theExe", "theArguments", true, null); + await tracker.HandleScanMove(new(temp, scanned)); + await Assert.That(tracker.KnownDiffering).IsEqualTo(1); + + // Still tracked, so still worth knowing + await tracker.ScanFiles(Cancel.None); + await Assert.That(tracker.KnownDiffering).IsEqualTo(1); + + tracker.Discard(scanned); + await tracker.ScanFiles(Cancel.None); + + await Assert.That(tracker.KnownDiffering).IsEqualTo(0); + } + /// /// A file that may not be read is no more a failure of the scan than one that is locked: the /// pair cannot be compared this round. Only the locked one was caught, so this one failed the @@ -108,6 +133,114 @@ public async Task AScanThatFailsIsFollowedByTheNextWithNobodyAsked() await Assert.That(ModuleInitializer.IssuesAsked.Where(_ => _.Contains("Failed to scan files"))).IsEmpty(); } + /// + /// Logged and nothing more, a tray whose every scan failed looked like one with nothing wrong + /// while its menu and icon stopped following the files. A run of failures is said once, in a + /// balloon, and not again for each scan in it: that would be one every two seconds. + /// + /// The scans here are run by the test, beside the tracker's own two second timer over the same + /// failing queue, whose scans count towards the same run. So what is asserted holds however + /// many of those land in between. + /// + /// + [Test] + public async Task AScanThatKeepsFailingIsSaidOnce() + { + var listed = 0; + var told = new ConcurrentQueue(); + var host = new StubInlineHost + { + Listed = () => + { + // The first is the tracker starting, and the rest are scans + if (Interlocked.Increment(ref listed) > 1) + { + throw new InvalidOperationException("TheScanFailure"); + } + } + }; + await using var tracker = new RecordingTracker(inline: host, scanFailing: told.Enqueue); + + for (var scan = 0; scan < Tracker.ScanFailuresBeforeTelling * 4; scan++) + { + await tracker.Scan(Cancel.None); + } + + await Assert.That(told).HasSingleItem(); + await Assert.That(told.Single()).Contains("TheScanFailure"); + await Assert.That(ModuleInitializer.IssuesAsked.Where(_ => _.Contains("Failed to scan files"))).IsEmpty(); + } + + /// + /// A scan that works ends the run, so the next run of failures is news again. + /// + [Test] + public async Task AScanThatFailsAgainAfterWorkingIsSaidAgain() + { + var failing = false; + var told = new ConcurrentQueue(); + var host = new StubInlineHost + { + Listed = () => + { + if (Volatile.Read(ref failing)) + { + throw new InvalidOperationException("TheScanFailure"); + } + } + }; + await using var tracker = new RecordingTracker(inline: host, scanFailing: told.Enqueue); + + async Task ScanThrice() + { + for (var scan = 0; scan < Tracker.ScanFailuresBeforeTelling; scan++) + { + await tracker.Scan(Cancel.None); + } + } + + Volatile.Write(ref failing, true); + await ScanThrice(); + await Assert.That(told.Count).IsEqualTo(1); + + Volatile.Write(ref failing, false); + await tracker.Scan(Cancel.None); + + Volatile.Write(ref failing, true); + await ScanThrice(); + await Assert.That(told.Count).IsEqualTo(2); + } + + /// + /// One scan failing is a file that went between two lines of it, and nobody is told. + /// + [Test] + public async Task ASingleFailedScanIsNotSaid() + { + var failing = true; + var told = new ConcurrentQueue(); + var listed = 0; + var host = new StubInlineHost + { + Listed = () => + { + // The tracker starting, which is not a scan, and then the one scan that fails + if (Interlocked.Increment(ref listed) > 1 && + Volatile.Read(ref failing)) + { + throw new InvalidOperationException("TheScanFailure"); + } + } + }; + await using var tracker = new RecordingTracker(inline: host, scanFailing: told.Enqueue); + + await tracker.Scan(Cancel.None); + Volatile.Write(ref failing, false); + await tracker.Scan(Cancel.None); + + await Assert.That(told).IsEmpty(); + } + public TrackerScanTest() { directory = Path.Combine(Path.GetTempPath(), "DiffEngineTray.Tests", Guid.NewGuid().ToString("N")); diff --git a/src/DiffEngineTray.Tests/TrackerTrackedFilesTest.cs b/src/DiffEngineTray.Tests/TrackerTrackedFilesTest.cs index cc1280ef..843b694b 100644 --- a/src/DiffEngineTray.Tests/TrackerTrackedFilesTest.cs +++ b/src/DiffEngineTray.Tests/TrackerTrackedFilesTest.cs @@ -288,7 +288,7 @@ public async Task AMoveArrivingAgainOverTheViewerPortKeepsAnotherToolAndItsProce await File.WriteAllTextAsync(temp, "content"); // Stands in for the diff tool the sender started for the pair. A process of its own, // since the tray kills the tool of a move it may kill - var tool = FileLockUtils.StartFileLockProcess(file); + var tool = FileLockUtils.StartFileLockProcess(file, shows: temp); try { // Named as the tool, since a process running something else is not tracked diff --git a/src/DiffEngineTray.Tests/TrackerVersionTest.cs b/src/DiffEngineTray.Tests/TrackerVersionTest.cs new file mode 100644 index 00000000..f9981b84 --- /dev/null +++ b/src/DiffEngineTray.Tests/TrackerVersionTest.cs @@ -0,0 +1,96 @@ +/// +/// is how a queue owner tells a displaying viewer that nothing +/// it listed has changed, so it has to move whenever a listing could have been built from +/// something other than what is tracked now. +/// +/// An accept takes its move out of the tracker for as long as the move takes, which is seconds +/// when a file is locked, and puts the same object back when it could not be carried out. The +/// owner takes its tag and then builds its listing, so a listing built in that gap goes out +/// without the move under a tag taken with it. Compared by which objects are tracked, the tracker +/// then looked exactly as it had at the tag, and the viewer was told "unchanged" about a listing +/// that was missing a pending file, for as long as nothing else changed. +/// +/// +public class TrackerVersionTest : + IDisposable +{ + [Test] + public async Task AMoveTakenOutAndPutBackIsAChange() + { + ITrackedFiles tracked = null!; + int? listedWhileAccepting = null; + // Told of the failure while the move is still out, which is where a listing built + // between the two halves of an accept stands + await using var tracker = new RecordingTracker( + acceptFailed: _ => listedWhileAccepting = tracked.Moves().Count); + tracked = tracker; + var move = tracker.AddMove(temp, target, "theExe", "theArguments", false, null); + // Read-only, so the accept gives up at once rather than after its retries + File.SetAttributes(target, FileAttributes.ReadOnly); + var before = tracked.Version(); + + tracker.Accept(move); + + await Assert.That(listedWhileAccepting).IsEqualTo(0); + await Assert.That(tracker.FindMove(temp)).IsSameReferenceAs(move); + await Assert.That(tracked.Version()).IsNotEqualTo(before); + } + + /// + /// The same gap for a delete, which is untracked while its file is deleted and tracked again + /// when that fails. + /// + [Test] + public async Task ADeleteTakenOutAndPutBackIsAChange() + { + await using var tracker = new RecordingTracker(); + ITrackedFiles tracked = tracker; + var delete = tracker.AddDelete(target); + var before = tracked.Version(); + + await using (File.Open(target, FileMode.Open, FileAccess.Read, FileShare.None)) + { + tracker.Accept(delete); + } + + await Assert.That(tracker.Deletes).HasSingleItem(); + await Assert.That(tracked.Version()).IsNotEqualTo(before); + } + + [Test] + public async Task NothingHappeningIsNoChange() + { + await using var tracker = new RecordingTracker(); + ITrackedFiles tracked = tracker; + tracker.AddMove(temp, target, "theExe", "theArguments", false, null); + tracker.AddDelete(other); + + var before = tracked.Version(); + + await Assert.That(tracked.Version()).IsEqualTo(before); + } + + public TrackerVersionTest() + { + directory = Path.Combine(Path.GetTempPath(), "DiffEngineTray.Tests", Guid.NewGuid().ToString("N")); + var received = Path.Combine(directory, "received"); + Directory.CreateDirectory(received); + temp = Path.Combine(received, "file.received.txt"); + target = Path.Combine(directory, "file.verified.txt"); + other = Path.Combine(directory, "other.verified.txt"); + File.WriteAllText(temp, "received"); + File.WriteAllText(target, "verified"); + File.WriteAllText(other, "other"); + } + + public void Dispose() + { + File.SetAttributes(target, FileAttributes.Normal); + Directory.Delete(directory, true); + } + + readonly string directory; + readonly string temp; + readonly string target; + readonly string other; +} diff --git a/src/DiffEngineTray/DebugReport.cs b/src/DiffEngineTray/DebugReport.cs index 2e5cec88..0107d67c 100644 --- a/src/DiffEngineTray/DebugReport.cs +++ b/src/DiffEngineTray/DebugReport.cs @@ -48,6 +48,11 @@ public static string Build(Tracker tracker, DateTime now) AppendEntry(builder, index, delete.Name); AppendField(builder, "File", WithExistence(delete.File)); AppendField(builder, "Group", delete.Group); + // Only where there is one: why "Accept all" would leave this delete pending + if (tracker.HeldReason(delete) is { } held) + { + AppendField(builder, "Held", held); + } } AppendHeading(builder, "Moves", moves.Count); diff --git a/src/DiffEngineTray/HotKey/KeyRegister.cs b/src/DiffEngineTray/HotKey/KeyRegister.cs index 39cc8072..148e48a6 100644 --- a/src/DiffEngineTray/HotKey/KeyRegister.cs +++ b/src/DiffEngineTray/HotKey/KeyRegister.cs @@ -11,10 +11,28 @@ public class KeyRegister : IntPtr handle; Dictionary bindings = []; + Func register; + Func unregister; - public KeyRegister(IntPtr handle) + public KeyRegister(IntPtr handle) : + this(handle, RegisterHotKey, UnregisterHotKey) + { + } + + /// + /// With something other than the desktop to register with, for the tests that are about what + /// happens once a key is pressed. A hot key is taken from the whole desktop, so a test that + /// registers one for real fails wherever something else already holds it, and takes it from + /// whatever would have wanted it while the test runs. + /// + internal KeyRegister( + IntPtr handle, + Func register, + Func unregister) { this.handle = handle; + this.register = register; + this.unregister = unregister; Application.AddMessageFilter(this); } @@ -50,9 +68,9 @@ public bool TryAddBinding(int id, bool shift, bool control, bool alt, string key public bool TryAddBinding(int id, KeyModifiers modifiers, Keys keys, Action action) { - UnregisterHotKey(handle, id); + unregister(handle, id); - if (!RegisterHotKey(handle, id, modifiers, keys)) + if (!register(handle, id, modifiers, keys)) { return false; } @@ -64,7 +82,7 @@ public bool TryAddBinding(int id, KeyModifiers modifiers, Keys keys, Action acti public void ClearBinding(int id) { bindings.Remove(id); - UnregisterHotKey(handle, id); + unregister(handle, id); } public bool PreFilterMessage(ref Message message) @@ -107,7 +125,7 @@ public void Dispose() foreach (var id in bindings.Keys) { - UnregisterHotKey(handle, id); + unregister(handle, id); } } } \ No newline at end of file diff --git a/src/DiffEngineTray/MenuBuilder.cs b/src/DiffEngineTray/MenuBuilder.cs index 5ccf82d8..6b55b5ce 100644 --- a/src/DiffEngineTray/MenuBuilder.cs +++ b/src/DiffEngineTray/MenuBuilder.cs @@ -57,7 +57,7 @@ static List TrackingMenuItems(ToolStripItemCollection items, Hash .Where(_ => !fixedItems.Contains(_)) .ToList(); - static IEnumerable BuildTrackingMenuItems(Tracker tracker) + internal static IEnumerable BuildTrackingMenuItems(Tracker tracker) { // Read everything first and decide from the counts. TrackingAny is backed by the scan // cache, which drives the icon, and the snapshot half of it can be up to one scan behind @@ -163,7 +163,7 @@ static IEnumerable BuildMovesAndDeletes( Images.Delete); foreach (var delete in deletes) { - yield return BuildDelete(delete, () => tracker.Accept(delete)); + yield return BuildDelete(delete, tracker.HeldReason(delete), () => tracker.Accept(delete)); } } @@ -255,15 +255,34 @@ static string ForMenu(string name, string status) status = status[prefix.Length..]; } - return status.Length <= 60 ? status : $"{status[..59].TrimEnd()}…"; + return Shortened(status); } - static ToolStripDropDownButton BuildDelete(TrackedDelete delete, Action accept) + static string Shortened(string text) => + text.Length <= 60 ? text : $"{text[..59].TrimEnd()}…"; + + /// + /// Why "Accept all" leaves this delete pending, when it does. Marked and said the way a + /// snapshot that was not written is: a delete an accept-all had just passed over looked like + /// every other, and pressing "Accept all" again was the natural thing to try. + /// + static ToolStripDropDownButton BuildDelete(TrackedDelete delete, string? held, Action accept) { - var menu = new ToolStripDropDownButton($"{delete.Name}") + var marker = held == null ? "" : " !"; + var menu = new ToolStripDropDownButton($"{delete.Name}{marker}") { DropDownDirection = ToolStripDropDownDirection.Left }; + if (held != null) + { + menu.ToolTipText = held; + menu.DropDownItems.Add(new ToolStripMenuItem(Shortened(held)) + { + Enabled = false, + ToolTipText = held + }); + } + menu.DropDownItems.Add(new MenuButton("Accept delete", accept)); menu.DropDownItems.Add(BuildShowInExplorer(delete.File)); return menu; diff --git a/src/DiffEngineTray/ProcessEx.cs b/src/DiffEngineTray/ProcessEx.cs index 19a8ebe5..564898de 100644 --- a/src/DiffEngineTray/ProcessEx.cs +++ b/src/DiffEngineTray/ProcessEx.cs @@ -75,13 +75,19 @@ public static bool TryGet(int id, [NotNullWhen(true)] out Process? process) /// ended. /// An image that cannot be read. One this account cannot open was never held by /// either, and one that has exited has nothing left to end. + /// Another copy of the tool, which is running the right image for another pair or for + /// somebody's own use: see . /// /// /// A launcher that is itself an executable - a shim that starts the real tool and waits - is /// the image the sender started and the id it sent, so it matches as it always did. /// /// - public static bool TryGetTool(int id, string? exe, [NotNullWhen(true)] out Process? process) + /// + /// The received file of the move, which the tool showing the pair was started with: see + /// . + /// + public static bool TryGetTool(int id, string? exe, string temp, [NotNullWhen(true)] out Process? process) { if (!TryGet(id, out process)) { @@ -89,21 +95,141 @@ public static bool TryGetTool(int id, string? exe, [NotNullWhen(true)] out Proce } var image = ImagePath(process); - if (IsSameExecutable(image, exe)) + if (!IsSameExecutable(image, exe)) + { + Log.Warning( + "Process {Id} is not tracked as the diff tool: it is running `{Image}` and the move names `{Exe}`", + id, + image ?? "an image that could not be read", + exe ?? "no tool"); + process.Dispose(); + process = null; + return false; + } + + if (WasStartedFor(CommandLine(process), temp)) { return true; } Log.Warning( - "Process {Id} is not tracked as the diff tool: it is running `{Image}` and the move names `{Exe}`", + "Process {Id} is not tracked as the diff tool: it is running `{Image}` and was not started with `{Temp}`", id, - image ?? "an image that could not be read", - exe ?? "no tool"); + image, + temp); process.Dispose(); process = null; return false; } + /// + /// Whether a process running the tool is the one showing this pair, by whether it was started + /// with the received file on its command line. + /// + /// The image says which program a process is and not which window. A stale id handed to + /// another copy of the same tool - the one open on the next snapshot along, or one somebody + /// started by hand for a merge - passed on the image, was tracked as this pair's, and was + /// ended when the pair was accepted. Every tool DiffEngine starts is given the two paths as + /// arguments, the received file among them exactly as the move carries it, and + /// ProcessCleanup.StillRunning asks the same of the command line before a newer library + /// sends an id at all. This is that question asked again here, for a library that does not. + /// + /// + /// A process's start time was the other candidate and settles nothing: the id is checked as + /// the move arrives, when every process there is to find was started before it, the one that + /// took over a closed tool's id included. + /// + /// + /// A command line that cannot be read is not known to be the tool's, and is treated as every + /// other such case is. + /// + /// + internal static bool WasStartedFor(string? commandLine, string temp) + { + if (commandLine is null || + temp.Length == 0) + { + return false; + } + + var from = 0; + while (true) + { + var found = commandLine.IndexOf(temp, from, StringComparison.OrdinalIgnoreCase); + if (found < 0) + { + return false; + } + + // The whole of an argument's path, and not the start of a longer one: the tool open + // on `Sample.received.txt.bak` was not started for `Sample.received.txt` + var end = found + temp.Length; + if (end == commandLine.Length || + commandLine[end] == '"' || + char.IsWhiteSpace(commandLine[end])) + { + return true; + } + + from = found + 1; + } + } + + [DllImport("ntdll.dll", ExactSpelling = true)] + static extern int NtQueryInformationProcess(SafeProcessHandle process, int informationClass, IntPtr information, int length, out int returned); + + // ProcessCommandLineInformation. Answers with a UNICODE_STRING and the characters after it, in + // one buffer, and needs no more than the right to query the process: nothing is read out of + // the other process's memory, so it is the same call whatever the bitness of either side + const int processCommandLine = 60; + + /// + /// The command line a process was started with, asked of the handle that is held, or null + /// when it will not say. + /// + internal static string? CommandLine(Process process) + { + try + { + var handle = process.SafeHandle; + // Fails for want of room, and says how much it wants + NtQueryInformationProcess(handle, processCommandLine, IntPtr.Zero, 0, out var length); + if (length <= 0) + { + return null; + } + + var buffer = Marshal.AllocHGlobal(length); + try + { + if (NtQueryInformationProcess(handle, processCommandLine, buffer, length, out _) != 0) + { + return null; + } + + // Length in bytes, then MaximumLength, then the pointer at its natural alignment + var bytes = (ushort) Marshal.ReadInt16(buffer); + var text = Marshal.ReadIntPtr(buffer, IntPtr.Size); + if (text == IntPtr.Zero) + { + return null; + } + + return Marshal.PtrToStringUni(text, bytes / 2); + } + finally + { + Marshal.FreeHGlobal(buffer); + } + } + catch (Exception exception) + when (exception is Win32Exception or InvalidOperationException or EntryPointNotFoundException) + { + // Exited, or no longer this account's to ask + return null; + } + } + /// /// By file name and not by path. One executable has several paths - through a junction, a /// substituted drive, a short name, another case - and the one the sender resolved need not be diff --git a/src/DiffEngineTray/Program.cs b/src/DiffEngineTray/Program.cs index 5043ce50..9ec0c2e4 100644 --- a/src/DiffEngineTray/Program.cs +++ b/src/DiffEngineTray/Program.cs @@ -94,7 +94,8 @@ void Warn(string message) => acceptFailed: move => Warn( $"Could not accept '{move.Name}': the file move keeps failing. The move is still pending, so accept can be retried."), inlineFailed: Warn, - inline: owned); + inline: owned, + scanFailing: Warn); // Owning the queue means knowing the moment it changes, rather than finding out on the // next two second scan. Wired before serving starts, so the first patch to arrive counts. @@ -105,10 +106,9 @@ void Warn(string message) => owned.Start(); } - // A logoff or a shutdown does not come back through Application.Run(), so the staging the - // unwind below performs never happens for one. Only for a queue held here: a viewer that - // owns it stages its own - using var sessionEnd = owned is null ? null : new SessionEndWindow(owned.SessionEnding); + // A logoff or a shutdown does not come back through Application.Run(), so nothing the + // unwind below performs happens for one + using var sessionEnd = new SessionEndWindow(SessionEnding(owned, TrayVersionFile.Delete)); // Not a using. Anything throwing between here and the await below would dispose a task // that is still running, and Task.Dispose throws for one that has not completed - which @@ -157,6 +157,34 @@ void Warn(string message) => await task; } + /// + /// What a clean exit does on its way out, for a session that ends instead: the queue staged, + /// when it is held here, and the version marker removed. + /// + /// The queue only where this tray owns it, since a viewer that owns it stages its own. The + /// marker whoever owns the queue: it says a tray of this version is running, to whatever + /// reads it to decide what it may send, and left behind by a logoff it went on saying so + /// until a tray was next started, which after a logoff need not be this version or any. + /// + /// + /// The queue first, being the one that cannot be had again, and the marker whether or not + /// that worked. + /// + /// + /// , except in a test. + internal static Action SessionEnding(OwnedInlineHost? owned, Action removeMarker) => + () => + { + try + { + owned?.SessionEnding(); + } + finally + { + removeMarker(); + } + }; + [UnsafeAccessor(UnsafeAccessorKind.Method, Name = "ShowContextMenu")] static extern void ShowContextMenu(NotifyIcon icon); diff --git a/src/DiffEngineTray/TrackedDelete.cs b/src/DiffEngineTray/TrackedDelete.cs index ac490531..eafd0322 100644 --- a/src/DiffEngineTray/TrackedDelete.cs +++ b/src/DiffEngineTray/TrackedDelete.cs @@ -10,4 +10,11 @@ public TrackedDelete(string file, string? group) public string Name { get; } public string File { get; } public string? Group { get; } -} \ No newline at end of file + + /// + /// Set once a move has written this file while the delete was pending, and from then on no + /// accept-all carries the delete out: see . Cleared when a + /// test run raises the delete again, which is a statement made after the write. + /// + public bool Written { get; set; } +} diff --git a/src/DiffEngineTray/Tracker.cs b/src/DiffEngineTray/Tracker.cs index 814841bf..595b451c 100644 --- a/src/DiffEngineTray/Tracker.cs +++ b/src/DiffEngineTray/Tracker.cs @@ -17,21 +17,16 @@ class Tracker : AsyncTimer timer; int lastScanCount; - public Tracker(Action active, Action inactive, LockedFilesResolver? lockedFilesResolver = null, Action? acceptFailed = null, Action? inlineFailed = null, IInlineHost? inline = null) + public Tracker(Action active, Action inactive, LockedFilesResolver? lockedFilesResolver = null, Action? acceptFailed = null, Action? inlineFailed = null, IInlineHost? inline = null, Action? scanFailing = null) { this.active = active; this.inactive = inactive; this.lockedFilesResolver = lockedFilesResolver; this.acceptFailed = acceptFailed; this.inlineFailed = inlineFailed; + this.scanFailing = scanFailing; this.inline = inline ?? new RemoteInlineHost(); - timer = new( - ScanFiles, - TimeSpan.FromSeconds(2), - // Logged and no more. This is the timer's own thread, and the handler everything else - // uses follows the log with a modal box: no scan ran again until somebody answered it, - // and a scan is nothing anybody asked for, so the box arrived out of nowhere - exception => Log.Error(exception, "Failed to scan files")); + timer = new(Scan, TimeSpan.FromSeconds(2)); // Seeded rather than left empty until the first scan two seconds later. The menu reads // this cache now, so without it a tray that has just started shows none of what a viewer @@ -39,7 +34,57 @@ public Tracker(Action active, Action inactive, LockedFilesResolver? lockedFilesR Refresh(); } - Task ScanFiles(Cancel cancel) + /// + /// How many scans have to fail one after the other before anybody is told. One that fails + /// alone is a file that went between two lines of it, and is put right by the next. + /// + internal const int ScanFailuresBeforeTelling = 3; + + Action? scanFailing; + int failedScans; + + /// + /// One scan, and what is made of it failing. + /// + /// Logged, every time. This is the timer's own thread, and the handler everything else uses + /// follows the log with a modal box: no scan ran again until somebody answered it, and a scan + /// is nothing anybody asked for, so the box arrived out of nowhere. + /// + /// + /// The log alone left a tray whose every scan failed looking like one with nothing wrong: the + /// icon and the menu stopped following the files, and nothing said so. So a run of failures + /// is said once, in a balloon, when it has gone on long enough not to be a passing one. Once + /// for the run, rather than for each scan in it, which would be a balloon every two seconds + /// for as long as the cause stood. A scan that works ends the run, and the next run is told + /// afresh. + /// + /// + internal async Task Scan(Cancel cancel) + { + try + { + await ScanFiles(cancel); + Interlocked.Exchange(ref failedScans, 0); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception exception) + { + Log.Error(exception, "Failed to scan files"); + // Counted across threads, since a test runs scans beside the timer's + if (Interlocked.Increment(ref failedScans) == ScanFailuresBeforeTelling) + { + scanFailing?.Invoke(ScanFailingMessage(exception)); + } + } + } + + internal static string ScanFailingMessage(Exception exception) => + $"The pending files could not be checked, {ScanFailuresBeforeTelling} times running, so the menu and the icon may be behind what is on disk. {exception.Message} Every failure is in the log: 'Open logs' in the menu."; + + internal Task ScanFiles(Cancel cancel) { foreach (var delete in deletes.ToList() .Where(delete => !File.Exists(delete.Value.File))) @@ -47,6 +92,19 @@ Task ScanFiles(Cancel cancel) deletes.TryRemove(delete.Key, out _); } + // What a pair was last found to be is worth keeping only while the pair is tracked. Here, + // once a scan, rather than wherever a move leaves: there are a score of such places, and + // a scan that is still comparing a move as it leaves writes its entry after any of them + // had run. Left alone, a tray that stays up for weeks kept an entry for every received + // file it had ever found different. + foreach (var temp in differing.Keys) + { + if (!moves.ContainsKey(temp)) + { + differing.TryRemove(temp, out _); + } + } + // A passing re-run sends a settle message, and whoever owns the queue drops the entry // then, so there is nothing to expire here. Just refresh the listing that drives the icon. snapshots = inline.List(); @@ -124,6 +182,11 @@ void RemoveAndKill() readonly ConcurrentDictionary differing = new(StringComparer.OrdinalIgnoreCase); + /// + /// How many pairs are remembered as found different, for the tests. + /// + internal int KnownDiffering => differing.Count; + static (long, DateTime, long, DateTime)? Stamp(TrackedMove move) { try @@ -196,7 +259,7 @@ public TrackedMove AddMove( Process? process = null; if (processId != null) { - ProcessEx.TryGetTool(processId.Value, exe, out process); + ProcessEx.TryGetTool(processId.Value, exe, temp, out process); } var move = BuildTrackedMove(temp, exe, arguments, canKill, target, process); @@ -227,7 +290,7 @@ public TrackedMove AddMove( existing.Process = null; // Against the tool the pair was tracked with when this move names none, as // Retarget keeps that one - ProcessEx.TryGetTool(processId.Value, exe ?? existing.Exe, out process); + ProcessEx.TryGetTool(processId.Value, exe ?? existing.Exe, temp, out process); } var move = exe == null @@ -469,7 +532,14 @@ Task AcceptSnapshotsThenDeletes(List pending, HashSet wri { if (!SweepSnapshots(out var failure)) { - AcceptDeletes(pending, written); + // Said, as the deletes held for a snapshot are below. Only the log knew, so + // the menu went on listing a delete "Accept all" had just been pressed over + // with nothing to say it had been left on purpose + var kept = AcceptDeletes(pending, written); + if (kept.Count > 0) + { + failure = failure is null ? DeletesKept(kept) : $"{failure} {DeletesKept(kept)}"; + } } else if (pending.Any(_ => deletes.ContainsKey(_.File))) { @@ -590,9 +660,63 @@ public TrackedDelete AddDelete(string file) => updateValueFactory: (_, existing) => { Log.Information("DeleteUpdated. File:{file}", file); + // Raised again, so by a run that looked at the file as it is now. Whatever a move + // wrote there since the delete was first raised, this is the later statement + existing.Written = false; return existing; }); + /// + /// Why an accept-all leaves a delete pending, where it would: null when it would carry it + /// out. For the menu and the debug view, which say it beside the delete, and for the sweeps, + /// which act on it. + /// + public string? HeldReason(TrackedDelete delete) + { + if (delete.Written) + { + return WroteItsFile; + } + + if (moves.Values.Any(_ => string.Equals(_.Target, delete.File, StringComparison.OrdinalIgnoreCase))) + { + return AwaitsItsFile; + } + + return null; + } + + /// + /// A move was accepted onto the file while its delete was pending. + /// + /// withdraws the delete a move finds waiting for its target, so the two + /// are only pending together when the delete arrived second, and which of them is the stale + /// one is then not knowable from here. The move is the snapshot arriving and the delete is the + /// last copy leaving, so the move goes ahead and the delete waits to be accepted on its own. + /// + /// + /// Remembered on the delete, rather than only for the sweep that wrote the file. The sweep + /// held the delete and forgot why as it ended, so the delete sat in the menu looking like any + /// other, and a second "Accept all" deleted the snapshot the first had just accepted. The same + /// went for a move accepted on its own and an accept-all after it. + /// + /// + public const string WroteItsFile = "Kept by 'Accept all': a move was accepted onto this file after the delete was raised, so deleting it would remove what was just accepted. Accept the delete on its own to delete the file anyway, or run the tests again."; + + /// + /// A move still pending is going to write the file. Not remembered: it is true for as long as + /// the move is there, and stops being true if the move is discarded. + /// + public const string AwaitsItsFile = "Kept by 'Accept all': a pending move is still to be accepted onto this file. Accept the delete on its own to delete the file anyway."; + + void MarkWritten(string target) + { + if (deletes.TryGetValue(target, out var delete)) + { + delete.Written = true; + } + } + /// /// Through , which is what the wire path has always /// used: it catches, re-tracks so the delete can be retried, and reports why. @@ -687,8 +811,17 @@ void Restore(TrackedMove removed) { Release(removed); } + + Interlocked.Increment(ref restores); } + /// + /// How many times something taken out to be accepted has been put back, which is the one + /// change to what is tracked that cannot see by looking: + /// the same object in the same place as before it left. + /// + long restores; + /// /// Lets go of the process a move was tracked with, without ending it, once the move has left /// for good. @@ -750,6 +883,7 @@ bool InnerMove(TrackedMove move, AcceptBatch batch) if (FileEx.SafeMove(move.Temp, move.Target)) { batch.Written.Add(move.Target); + MarkWritten(move.Target); DeleteTempDirectory(move); return true; } @@ -938,20 +1072,22 @@ static void KillProcesses(TrackedMove move) /// skipped came back on the next scan two seconds later. /// /// - /// The tracked files here, on the calling thread, and the snapshots on a worker, for the - /// reason gives: a queue a viewer owns is asked over a - /// socket, and one slow to answer held the thread drawing everything for as long as that - /// took. The menu and the hot key discard the task; tests await it. + /// All of it on a worker, as an accept-all's second half is. The snapshots for the reason + /// gives: a queue a viewer owns is asked over a socket, + /// and one slow to answer held the thread drawing everything for as long as that took. The + /// tracked files because discarding a move ends its diff tool and waits up to half a second + /// for each to go, which for a screen full of them was seconds of a tray that drew nothing. + /// The files first, so a queue that is slow to answer holds up nothing but itself. The menu + /// and the hot key discard the task; tests await it. /// /// - public Task Clear() - { - ((ITrackedFiles) this).DiscardAll(); - - return Task.Run(() => + public Task Clear() => + Task.Run(() => { try { + DiscardFiles(); + // Only forget the cached snapshots when the owner actually discarded them. It used // to be cleared regardless, so a discard the owner never received still emptied the // menu - and everything came back on the next scan two seconds later @@ -961,7 +1097,11 @@ public Task Clear() } else { - Log.Error("{Message}", message ?? "Could not discard the pending snapshots."); + // Said, and not only logged: the snapshots are still in the menu, under the + // button that was just pressed to be rid of them + var failure = CouldNotDiscard(message); + Log.Error("{Message}", failure); + inlineFailed?.Invoke(failure); } // Nothing waits for the next scan to say so: the files went above, whatever the @@ -970,9 +1110,48 @@ public Task Clear() } catch (Exception exception) { - ExceptionHandler.Handle("Failed to discard the pending snapshots", exception); + ExceptionHandler.Handle("Failed to discard everything pending", exception); } }); + + /// + /// What a user is told when "Discard (n)" could not discard the snapshots, which stay pending. + /// + internal static string CouldNotDiscard(string? message) + { + if (message is { Length: > 0 }) + { + return $"Could not discard the pending snapshots. {message}"; + } + + return "Could not discard the pending snapshots."; + } + + /// + /// Every tracked move and delete discarded, for and for the wire. Virtual + /// so a test can see which thread it was asked of. + /// + protected virtual int DiscardFiles() + { + var count = 0; + foreach (var delete in deletes.Values.ToList()) + { + if (deletes.TryRemove(delete.File, out _)) + { + count++; + } + } + + foreach (var move in moves.Values.ToList()) + { + if (moves.TryRemove(move.Temp, out var removed)) + { + InnerDiscard(removed); + count++; + } + } + + return count; } /// @@ -1013,8 +1192,12 @@ public Task AcceptAll() return AcceptSnapshotsThenDeletes(pending, written); } - void AcceptDeletes(List pending, HashSet written) + /// + /// The deletes left pending because of a move onto their file, for the caller to say so. + /// + List AcceptDeletes(List pending, HashSet written) { + var kept = new List(); // One at a time, and no Clear afterwards: a delete that fails re-tracks itself, and // clearing would throw that away. Unguarded, the first bad one also took the rest of the // sweep with it, so "Accept all" stopped at the first read-only file @@ -1029,23 +1212,35 @@ void AcceptDeletes(List pending, HashSet written) if (WrittenOrAwaited(delete, written)) { Log.Information("Kept the pending delete of `{Name}`: a move wrote that file, or is still pending onto it", delete.Name); + kept.Add(delete); continue; } Accept(delete); } + + return kept; + } + + /// + /// What a user is told about deletes an accept-all kept because of a move onto their file. + /// The menu says the same beside each of them, for whoever missed the balloon. + /// + internal static string DeletesKept(IReadOnlyList kept) + { + var which = kept.Count == 1 + ? $"The pending delete of '{kept[0].Name}' was kept" + : $"{kept.Count} pending deletes were kept"; + return $"{which}, since a move was accepted onto the same file, or is still to be, and deleting it would remove what the move put there. Accept a delete on its own to delete its file anyway."; } /// /// Whether a sweep must leave this delete pending, because the file it would remove is one a /// move in the same sweep has just written or one a move still pending is going to write. /// - /// withdraws the delete a move finds waiting for its target, so the two - /// are only pending together when the delete arrived second, and which of them is the stale - /// one is then not knowable from here. The move is the snapshot arriving and the delete is the - /// last copy leaving, so the move goes ahead and the delete waits to be accepted on its own: - /// carried out, it removed the received file a moment after that file had been moved into - /// place, and neither was left. + /// Carried out, it removed the received file a moment after that file had been moved into + /// place, and neither was left: see , which is also why a delete + /// whose file an earlier accept wrote is held by every sweep after it. /// /// /// By what was written rather than by what was swept. A move whose received file has gone is @@ -1054,7 +1249,7 @@ void AcceptDeletes(List pending, HashSet written) /// bool WrittenOrAwaited(TrackedDelete delete, HashSet written) => written.Contains(delete.File) || - moves.Values.Any(_ => string.Equals(_.Target, delete.File, StringComparison.OrdinalIgnoreCase)); + HeldReason(delete) is not null; public ICollection Deletes => deletes.Values; @@ -1087,6 +1282,7 @@ IReadOnlyList ITrackedFiles.Deletes() => readonly Lock versionGate = new(); readonly List versioned = []; + long versionedRestores; long version; /// @@ -1103,16 +1299,29 @@ IReadOnlyList ITrackedFiles.Deletes() => /// Rather than a count of changes, kept wherever the dictionaries are written: there are a /// score of such places, and one missed is a viewer that goes on showing a file that left. /// + /// + /// Except for the one change that leaves the same objects behind it. An accept takes its move + /// out for as long as the move takes, seconds when a file is locked, and puts the same object + /// back when it could not be carried out. The owner takes its tag and then builds its listing, + /// so a listing built in that gap goes out without the move, under a tag taken while it was + /// there, and once the move was back nothing here looked different from when the tag was + /// taken: the viewer was told "unchanged" about a listing missing a pending file. Those are + /// two places, and the delete that could not be deleted, and they are + /// counted. + /// /// long ITrackedFiles.Version() { lock (versionGate) { - if (Unchanged()) + var restored = Interlocked.Read(ref restores); + if (restored == versionedRestores && + Unchanged()) { return version; } + versionedRestores = restored; versioned.Clear(); foreach (var move in moves) { @@ -1299,28 +1508,8 @@ bool ITrackedFiles.Untrack(string key) return (accepted, kept); } - int ITrackedFiles.DiscardAll() - { - var count = 0; - foreach (var delete in deletes.Values.ToList()) - { - if (deletes.TryRemove(delete.File, out _)) - { - count++; - } - } - - foreach (var move in moves.Values.ToList()) - { - if (moves.TryRemove(move.Temp, out var removed)) - { - InnerDiscard(removed); - count++; - } - } - - return count; - } + int ITrackedFiles.DiscardAll() => + DiscardFiles(); (bool ok, string? message) AcceptTracked(TrackedDelete delete) { @@ -1337,6 +1526,7 @@ int ITrackedFiles.DiscardAll() { // Re-tracked so it can be retried, and refused so the caller shows why. deletes.TryAdd(removed.File, removed); + Interlocked.Increment(ref restores); return (false, $"Could not delete {removed.Name}. {exception.Message}"); } diff --git a/src/DiffEngineViewer.Benchmarks/FrameBenchmarks.cs b/src/DiffEngineViewer.Benchmarks/FrameBenchmarks.cs index a9bab873..0031b4d7 100644 --- a/src/DiffEngineViewer.Benchmarks/FrameBenchmarks.cs +++ b/src/DiffEngineViewer.Benchmarks/FrameBenchmarks.cs @@ -36,11 +36,18 @@ public void Setup() readonly ScreenCache screens = new(); - // One screen from one state: what every frame paid, and what a change to the state still does. + // One screen from one state: what every frame paid, and what a change to the state that leaves + // the queue the list it was still does. A scroll and a drag are that, every frame they last. [Benchmark(Baseline = true)] public object BuildScreen() => ScreenBuilder.Build(state); + // And the screen of a queue that is another list, which is what an arrival, an accept or a + // settle leaves: the labels are worked out again, over every entry. + [Benchmark] + public object BuildScreenOfAChangedQueue() => + ScreenBuilder.Build(state with { Queue = [..state.Queue] }); + // What the loop asks for each frame now: the screen of the state, which is the one in hand // whenever the state is the one the last frame had. [Benchmark] diff --git a/src/DiffEngineViewer.Benchmarks/QueueChangeBenchmarks.cs b/src/DiffEngineViewer.Benchmarks/QueueChangeBenchmarks.cs new file mode 100644 index 00000000..c4c5a012 --- /dev/null +++ b/src/DiffEngineViewer.Benchmarks/QueueChangeBenchmarks.cs @@ -0,0 +1,80 @@ +using BenchmarkDotNet.Attributes; + +// What one change to a long queue costs the session: a snapshot arriving, a test failing again +// with something else, a test that started passing, and one accept. Each is done under the lock +// the render loop takes, on the listener thread a test process is waiting on, and each is about +// one entry of the queue however many there are. The applier here answers at once and touches +// nothing, so everything measured is the session's own work. +[MemoryDiagnoser] +public class QueueChangeBenchmarks +{ + [Params(200, 2000)] + public int Entries; + + SessionState state = null!; + InlinePatch arriving = null!; + InlinePatch again = null!; + string settling = ""; + + static readonly ViewerActions applied = new( + static _ => InlineApplyResult.Applied, + static (_, _) => + { + }, + static _ => + { + }); + + [GlobalSetup] + public void Setup() + { + // Nothing here is read: an inline entry is built from its patch, and the directory only has + // to be somewhere with no solution above it. + var directory = Path.Combine(Path.GetTempPath(), "deview-benchmarks", "change"); + var built = SessionState.Start(ViewerMode.Inline, 160, 50); + for (var index = 0; index < Entries; index++) + { + built = ViewerSession.EnqueueInline(built, Patch(directory, index, $"new {index}")); + } + + // On an entry in the middle, as a reader part way through a queue is + state = ViewerSession.Apply(built, Command.Select(Entries / 2)); + arriving = Patch(directory, Entries, "new"); + again = Patch(directory, Entries / 4, "something else this time"); + settling = InlineKey.For(again.SourceFile, again.LineHint); + } + + static InlinePatch Patch(string directory, int index, string content) => + new(Path.Combine(directory, $"Sample{index % 40}Tests.cs"), 10 + index, "\"old\"", content) + { + TestName = $"Test{index}", + MemberName = $"Test{index}", + Framework = "net10.0" + }; + + [Benchmark] + public object ASnapshotArrives() => + Changed(ViewerSession.EnqueueInline(state, arriving), Entries + 1); + + [Benchmark] + public object ATestFailsAgainWithSomethingElse() => + Changed(ViewerSession.EnqueueInline(state, again), Entries); + + [Benchmark] + public object ATestStartsPassing() => + Changed(ViewerSession.Settle(state, settling, "net10.0", again.MemberName), Entries - 1); + + [Benchmark] + public object OneAccept() => + Changed(ViewerSession.Apply(state, CommandKind.Accept, applied), Entries - 1); + + SessionState Changed(SessionState changed, int count) + { + if (changed.Queue.Count != count) + { + throw new($"{changed.Queue.Count} entries where {count} were expected: {changed.Message}"); + } + + return changed; + } +} diff --git a/src/DiffEngineViewer.Benchmarks/ScreenPayloadBenchmarks.cs b/src/DiffEngineViewer.Benchmarks/ScreenPayloadBenchmarks.cs index e3b4de9e..613f3dc9 100644 --- a/src/DiffEngineViewer.Benchmarks/ScreenPayloadBenchmarks.cs +++ b/src/DiffEngineViewer.Benchmarks/ScreenPayloadBenchmarks.cs @@ -7,8 +7,9 @@ // the row's bytes again from its start, so such a row cost by the square of its length. // // A window the size of a 4K display: 426 cells across and 130 down at the 9 pixel cell. Lines of -// 300 characters, since a row is encoded as far as the window is wide rather than the pane, so a -// long line is where a row's length shows. +// 300 characters, which is past the end of a pane whatever the characters are: a row is encoded as +// far as a pane can show it, 214 cells here, where it was once encoded as far as the window is +// wide in characters, which for two cell characters was all 300 of them. [MemoryDiagnoser] public class ScreenPayloadBenchmarks { diff --git a/src/DiffEngineViewer.Linux/runtimes/linux-arm64/native/libdiffengine_viewer.so b/src/DiffEngineViewer.Linux/runtimes/linux-arm64/native/libdiffengine_viewer.so index 9a874716..59afe83e 100644 Binary files a/src/DiffEngineViewer.Linux/runtimes/linux-arm64/native/libdiffengine_viewer.so and b/src/DiffEngineViewer.Linux/runtimes/linux-arm64/native/libdiffengine_viewer.so differ diff --git a/src/DiffEngineViewer.Linux/runtimes/linux-x64/native/libdiffengine_viewer.so b/src/DiffEngineViewer.Linux/runtimes/linux-x64/native/libdiffengine_viewer.so index 072ba222..7f232f08 100644 Binary files a/src/DiffEngineViewer.Linux/runtimes/linux-x64/native/libdiffengine_viewer.so and b/src/DiffEngineViewer.Linux/runtimes/linux-x64/native/libdiffengine_viewer.so differ diff --git a/src/DiffEngineViewer.Mac/runtimes/osx-arm64/native/libdiffengine_viewer.dylib b/src/DiffEngineViewer.Mac/runtimes/osx-arm64/native/libdiffengine_viewer.dylib index c80b369d..9378285c 100644 Binary files a/src/DiffEngineViewer.Mac/runtimes/osx-arm64/native/libdiffengine_viewer.dylib and b/src/DiffEngineViewer.Mac/runtimes/osx-arm64/native/libdiffengine_viewer.dylib differ diff --git a/src/DiffEngineViewer.Mac/runtimes/osx-x64/native/libdiffengine_viewer.dylib b/src/DiffEngineViewer.Mac/runtimes/osx-x64/native/libdiffengine_viewer.dylib index c80b369d..9378285c 100644 Binary files a/src/DiffEngineViewer.Mac/runtimes/osx-x64/native/libdiffengine_viewer.dylib and b/src/DiffEngineViewer.Mac/runtimes/osx-x64/native/libdiffengine_viewer.dylib differ diff --git a/src/DiffEngineViewer.Tests/CellGridTests.cs b/src/DiffEngineViewer.Tests/CellGridTests.cs index c37ebdb0..80b4d204 100644 --- a/src/DiffEngineViewer.Tests/CellGridTests.cs +++ b/src/DiffEngineViewer.Tests/CellGridTests.cs @@ -204,6 +204,34 @@ public async Task SnapsToWholeCharacters(int cell, int snapped, int index) await Assert.That(CellGrid.Index(text, cell)).IsEqualTo(index); } + /// + /// The segments of the start of a row, found in one walk, are the segments of the row cut + /// where its first cells end: every kind of character, cut at every cell. + /// + [Test] + public async Task TheSegmentsOfTheStartOfARowAreThoseOfTheRowCutThere() + { + string[] pieces = ["a", "bc", " ", "中", "ｆ", "é", "é̂", "\U0001D400", "\U0001F600", "\U0001F468‍\U0001F469‍\U0001F467", "─", "\uD800", "́", "→"]; + var random = new Random(31); + for (var iteration = 0; iteration < 300; iteration++) + { + var builder = new StringBuilder(); + for (var count = random.Next(0, 12); count > 0; count--) + { + builder.Append(pieces[random.Next(pieces.Length)]); + } + + var text = builder.ToString(); + for (var cells = -1; cells <= CellGrid.Cells(text) + 1; cells++) + { + var start = CellGrid.Segments(text, cells, out var end); + + await Assert.That(end).IsEqualTo(CellGrid.Index(text, cells)); + await Assert.That(start.SequenceEqual(CellGrid.Segments(text[..end]))).IsTrue(); + } + } + } + [Test] public async Task NeverSplitsACharacterFromItsMarks() { diff --git a/src/DiffEngineViewer.Tests/DiscardBatchTests.cs b/src/DiffEngineViewer.Tests/DiscardBatchTests.cs new file mode 100644 index 00000000..95283792 --- /dev/null +++ b/src/DiffEngineViewer.Tests/DiscardBatchTests.cs @@ -0,0 +1,197 @@ +/// +/// A bulk discard throws a received file away for every pending pair, and used to do all of it in +/// one transition: under the lock every arrival waits on, and for a window's own discard on the +/// thread that draws. It is the batch an accept-all is, over the moves: the snapshots and the +/// pending deletes go as it begins, since neither touches a file, and each received file is +/// thrown away in a step of its own, outside the lock. +/// +public class DiscardBatchTests +{ + [Test] + public async Task BeginningTouchesNoFile() + { + var deleted = new List(); + var state = ViewerSession.BeginDiscardAll(Mixed()); + + // The snapshot and the pending delete are gone, and the two moves are still listed + await Assert.That(state.Queue.Select(_ => _.Kind)).IsEquivalentTo([QueueEntryKind.Move, QueueEntryKind.Move]); + await Assert.That(state.Batch!.Discarding).IsTrue(); + await Assert.That(state.Batch.Current).IsNull(); + await Assert.That(deleted).IsEmpty(); + + var seen = new List(); + while ((state = ViewerSession.ClaimNext(state)).Batch?.Current is not null) + { + seen.Add($"{ScreenBuilder.Build(state).Status}, {state.Queue.Count} pending, {deleted.Count} thrown away"); + state = ViewerSession.ApplyClaimed(state, Deleting(deleted.Add))(state); + } + + await Assert.That(string.Join("\n", seen)).IsEqualTo( + """ + Discarding 1 of 2, 2 pending, 0 thrown away + Discarding 2 of 2, 1 pending, 1 thrown away + """); + await Assert.That(deleted.Count).IsEqualTo(2); + await Assert.That(state.Batch).IsNull(); + await Assert.That(state.Message).IsEqualTo("Discarded 1, plus 3 files"); + await Assert.That(state.Exit).IsTrue(); + } + + /// + /// A step at a time and in one call come to the same thing, which is what a discard-all was + /// before it was a batch: the tests and the wire's single discards still drive it whole. + /// + [Test] + public async Task InOneCallItSaysWhatTheStepsSay() + { + var deleted = new List(); + + var state = ViewerSession.Apply(Mixed(), CommandKind.DiscardAll, Deleting(deleted.Add)); + + await Assert.That(deleted.Count).IsEqualTo(2); + await Assert.That(state.Queue).IsEmpty(); + await Assert.That(state.Message).IsEqualTo("Discarded 1, plus 3 files"); + } + + /// + /// A received file that could not be thrown away stays pending, saying why, and is counted. + /// + [Test] + public async Task AFileThatWouldNotGoIsKept() + { + var actions = Deleting(_ => throw new IOException("the file is held")); + + var state = ViewerSession.Apply(Mixed(), CommandKind.DiscardAll, actions); + + await Assert.That(state.Queue.Count).IsEqualTo(2); + await Assert.That(state.Queue.All(_ => _.Status == "the file is held")).IsTrue(); + await Assert.That(state.Message).IsEqualTo("Discarded 1, plus 1 files (2 kept)"); + await Assert.That(state.Batch).IsNull(); + } + + /// + /// A header's discard is the same batch over its own members, and leaves everything else. + /// + [Test] + public async Task AGroupsDiscardIsTheBatchOverItsMembers() + { + var state = Fixtures.Inline( + Fixtures.Patch(Fixtures.SolutionFile("SolutionA", "Tests", "ATests.cs"), 10), + Fixtures.Patch(Fixtures.SolutionFile("SolutionB", "Tests", "BTests.cs"), 20)); + state = ViewerSession.EnqueueTracked(state, Fixtures.Move("One.Test (txt)", "SolutionA")); + state = ViewerSession.EnqueueTracked(state, Fixtures.Move("Two.Test (txt)", "SolutionB")); + var header = QueueProjection.Rows(state).ToList().FindIndex(_ => _.GroupName == "SolutionA"); + var deleted = new List(); + + var begun = ViewerSession.BeginDiscardGroup(ViewerSession.OpenMenu(state, header)); + + await Assert.That(begun.Menu).IsNull(); + await Assert.That(begun.Batch!.Remaining.Count).IsEqualTo(1); + await Assert.That(begun.Queue.Count).IsEqualTo(3); + var done = begun; + while ((done = ViewerSession.ClaimNext(done)).Batch?.Current is not null) + { + done = ViewerSession.ApplyClaimed(done, Deleting(deleted.Add))(done); + } + + await Assert.That(deleted.Count).IsEqualTo(1); + await Assert.That(string.Join(",", done.Queue.Select(_ => _.Solution))).IsEqualTo("SolutionB,SolutionB"); + await Assert.That(done.Message).IsEqualTo("Discarded 1, plus 1 files"); + } + + /// + /// What the batch is for. A discard-all asked over the socket throws each file away with the + /// session's lock free, so the render loop and an arriving snapshot get in between two of + /// them: seen by another thread taking the lock from inside a delete, which waited for the + /// whole discard when the discard was one mutation. + /// + [Test] + public async Task AFileIsThrownAwayOutsideTheLock() + { + var host = new SessionHost(Mixed()); + var took = new List(); + var actions = Deleting(_ => + { + var taking = new Thread(() => host.Mutate(static _ => _)) + { + IsBackground = true + }; + taking.Start(); + // Held, it is never let go while this delete waits, so this is only ever the whole + // wait when the test is failing + took.Add(taking.Join(TimeSpan.FromSeconds(20))); + }); + IQueueOwner owner = new MessageHandler(host, actions, _ => { }); + + var message = owner.DiscardAll(); + + await Assert.That(took.Count).IsEqualTo(2); + await Assert.That(took.All(_ => _)).IsTrue(); + await Assert.That(message).IsEqualTo("Discarded 1, plus 3 files"); + await Assert.That(host.State.Queue).IsEmpty(); + } + + /// + /// A window's own discard-all is only begun by the frame that asked for it, as its accept-all + /// is, and the buttons that change the queue wait for it. + /// + [Test] + public async Task AWindowOnlyBeginsItsDiscard() + { + var state = ViewerProgram.Apply(Mixed(), Input(CommandKind.DiscardAll), null, new Window()); + + await Assert.That(state.Batch!.Discarding).IsTrue(); + await Assert.That(state.Queue.Count).IsEqualTo(2); + var screen = ScreenBuilder.Build(state); + await Assert.That(screen.Buttons.Where(_ => _.Command is CommandKind.Accept or CommandKind.Discard or CommandKind.AcceptAll).Any(_ => _.Enabled)).IsFalse(); + // Not something a listing says: the wire's progress is an accept's, in so many words + await Assert.That(state.ListedProgress).IsNull(); + } + + // A snapshot, two pending moves and a pending delete + static SessionState Mixed() + { + var state = Fixtures.Inline(Fixtures.Patch()); + state = ViewerSession.EnqueueTracked(state, Fixtures.Move("One.Test (txt)")); + state = ViewerSession.EnqueueTracked(state, Fixtures.Move("Two.Test (txt)")); + return ViewerSession.EnqueueTracked(state, Fixtures.Delete()); + } + + static ViewerActions Deleting(Action delete) => + Fixtures.Applied with + { + MoveFile = static (_, _) => throw new("A discard moves nothing."), + DeleteFile = delete + }; + + static ViewerInput Input(CommandKind key) => + new(key, -1, -1, 0, false, Fixtures.Columns, Fixtures.Rows); + + sealed class Window : IViewerWindow + { + public bool Present(Screen screen) => + true; + + public ViewerInput Poll() => + default; + + public void SetHidden(bool hidden) + { + } + + public void Focus() + { + } + + public void SetClipboard(string text) + { + } + + public bool Capture(Screen screen, int width, int height, string pngPath) => + false; + + public void Dispose() + { + } + } +} diff --git a/src/DiffEngineViewer.Tests/PixelTests.ImageTooLargeForATexture.Linux.verified.png b/src/DiffEngineViewer.Tests/PixelTests.ImageTooLargeForATexture.Linux.verified.png index 86935c0a..cc95c9a7 100644 Binary files a/src/DiffEngineViewer.Tests/PixelTests.ImageTooLargeForATexture.Linux.verified.png and b/src/DiffEngineViewer.Tests/PixelTests.ImageTooLargeForATexture.Linux.verified.png differ diff --git a/src/DiffEngineViewer.Tests/PixelTests.cs b/src/DiffEngineViewer.Tests/PixelTests.cs index b933809e..4ed645ab 100644 --- a/src/DiffEngineViewer.Tests/PixelTests.cs +++ b/src/DiffEngineViewer.Tests/PixelTests.cs @@ -432,13 +432,16 @@ public Task ImagesReduced() } /// - /// A picture longer on one side than a texture can be, beside one that is not. The rows say - /// what it is, as they do for a format the head has no decoder for, and nothing is drawn under - /// them: it was a black box the shape of the picture, which is what GL makes of a texture it - /// was handed and would not take. + /// A picture longer on one side than a texture can be, beside one that is not. It is brought + /// down to the largest size of its own shape that a texture takes as it is read, and drawn + /// from that, fitted as any other. It was first a black box the shape of the picture, which + /// is what GL makes of a texture it was handed and would not take, and then nothing at all + /// under rows that said what it was. /// - /// Linux only, and only where the limit is what it is under Mesa's software rasteriser, which - /// is what these baselines are pinned to: 16384 pixels, one fewer than this picture is wide. + /// Linux only. It is past the limit where the limit is what it is under Mesa's software + /// rasteriser, which is what these baselines are pinned to: 16384 pixels, one fewer than this + /// picture is wide. Where a texture can be larger the picture is drawn as it is, and the + /// scene is of the same thing by the other way. /// /// [Test] @@ -551,6 +554,154 @@ public async Task PresentWaitsForTheNextFrame() await Assert.That(elapsed).IsGreaterThan(TimeSpan.FromMilliseconds(750)); } + /// + /// A window nothing is happening to is left alone: the Linux head builds no frame for it and + /// draws nothing into it, where it used to draw the frame already there sixty times a second, + /// which under a software rasteriser was more than half a core for a viewer left open. Nothing + /// a capture does can tell the two apart, so this is asked of the window itself, shown for + /// as long as the test takes: the same screen is presented a second at a time until a second + /// goes by in which nothing was drawn. + /// + /// A second, and not the first one. The frame after a window is shown is drawn, and the head + /// goes on building frames for a second after anything changes before it leaves a window + /// alone, so the first second draws and a later one does not. Which later one is not asserted: + /// the window system may ask for the window again, and a runner may be slow. A head that has + /// gone back to drawing every frame never has such a second. + /// + /// + [Test] + [PixelTest] + [NotInParallel(nameof(PixelTests), Order = 21)] + [SkipOnMac("A capture host never creates the macOS window, and the counts are asked of OpenGL, which that head does not draw with.")] + public async Task AWindowLeftAloneIsNotDrawn() + { + var screen = ScreenBuilder.Build(ViewerSession.Resize(Fixtures.File(), columns, rows)); + var (first, rested) = await OnShimThread( + () => + { + window!.SetHidden(false); + try + { + var first = Drawn(60, screen); + var watch = Stopwatch.StartNew(); + var rested = false; + while (!rested && + watch.Elapsed < TimeSpan.FromSeconds(30)) + { + rested = Drawn(60, screen) == 0; + } + + return (first, rested); + } + finally + { + window.SetHidden(true); + } + }); + + // Or the count is of nothing, and a second with none drawn says nothing either + await Assert.That(first).IsGreaterThan(0); + await Assert.That(rested).IsTrue(); + } + + /// + /// A hidden window is handed another screen by every arrival in its queue, and the Linux head + /// built and drew each of them into a window nobody could see. Hidden, it now draws none, + /// however the screen changes, and the first present after it is shown draws the screen it is + /// handed. That what it draws then is the right screen is not something a count can say: it + /// was photographed off the X server when this was changed. + /// + [Test] + [PixelTest] + [NotInParallel(nameof(PixelTests), Order = 22)] + [SkipOnMac("A capture host never creates the macOS window, and the counts are asked of OpenGL, which that head does not draw with.")] + public async Task AHiddenWindowIsNotDrawn() + { + var screen = ScreenBuilder.Build(ViewerSession.Resize(Fixtures.File(), columns, rows)); + // The same frame twice, told apart by its status line, so that every present is of a + // screen that is not the one before it + Screen[] screens = + [ + screen with {Status = "tick"}, + screen with {Status = "tock"} + ]; + var (hidden, shown) = await OnShimThread( + () => + { + // A window's first frame is built wherever the window is, and this test may be + // the first to present anything + window!.Present(screen); + var hidden = Drawn(60, screens); + window.SetHidden(false); + try + { + return (hidden, Drawn(1, screens[0])); + } + finally + { + window.SetHidden(true); + } + }); + + await Assert.That(hidden).IsEqualTo(0); + await Assert.That(shown).IsEqualTo(1); + } + + static uint drawnQuery; + + /// + /// How many of so many presents, of the screens given in turn, put anything on the screen. + /// Asked of OpenGL, as NativeHead in the benchmarks asks it: the primitives generated + /// between the start of a present and its end are the triangles the shim submitted, whichever + /// rasteriser then filled them, and a present that generated none drew nothing. On the shim's + /// thread, which is the one the GL context belongs to. + /// + static int Drawn(int presents, params Screen[] screens) + { + if (drawnQuery == 0) + { + Gl.GenQueries(1, out drawnQuery); + } + + var drawn = 0; + for (var present = 0; present < presents; present++) + { + Gl.BeginQuery(Gl.PrimitivesGenerated, drawnQuery); + window!.Present(screens[present % screens.Length]); + Gl.EndQuery(Gl.PrimitivesGenerated); + Gl.GetQueryObject(drawnQuery, Gl.QueryResult, out var primitives); + if (primitives > 0) + { + drawn++; + } + } + + return drawn; + } + + /// + /// The four OpenGL calls a query takes, from the library the shim's own context came from. + /// + static class Gl + { + const string library = "libGL.so.1"; + + public const uint PrimitivesGenerated = 0x8C87; + public const uint QueryResult = 0x8866; + + [DllImport(library, EntryPoint = "glGenQueries")] + public static extern void GenQueries(int count, out uint id); + + [DllImport(library, EntryPoint = "glBeginQuery")] + public static extern void BeginQuery(uint target, uint id); + + [DllImport(library, EntryPoint = "glEndQuery")] + public static extern void EndQuery(uint target); + + [DllImport(library, EntryPoint = "glGetQueryObjectuiv")] + public static extern void GetQueryObject(uint id, uint name, out uint value); + } + static async Task Capture(SessionState state, int gridRows = rows) { var screen = ScreenBuilder.Build(ViewerSession.Resize(state, columns, gridRows)); diff --git a/src/DiffEngineViewer.Tests/QueueRebuildTests.cs b/src/DiffEngineViewer.Tests/QueueRebuildTests.cs new file mode 100644 index 00000000..2f771891 --- /dev/null +++ b/src/DiffEngineViewer.Tests/QueueRebuildTests.cs @@ -0,0 +1,193 @@ +/// +/// A change to the queue that is about one entry leaves every other entry where it is, as the +/// entry it was, rather than rebuilding the list from the whole queue. What it leaves has to be +/// what the rebuild would have: the same entries saying the same things, in the same order. +/// +public class QueueRebuildTests +{ + /// + /// Random queues changed at random - arrivals at new call sites and old ones, from the same + /// framework and another, call sites that moved, settles, accepts that land and that fail, + /// discards, and files arriving between them - each checked against the list rebuilt whole + /// from the same queue. + /// + [Test] + public void EveryChangeLeavesWhatAWholeRebuildWould() + { + var random = new Random(41); + for (var iteration = 0; iteration < 300; iteration++) + { + var state = SessionState.Start(ViewerMode.Inline, Fixtures.Columns, Fixtures.Rows); + var patches = new List(); + for (var step = 0; step < 60; step++) + { + var before = state; + var pending = Pending(state); + InlineQueue expected; + string what; + switch (random.Next(8)) + { + case 0 when patches.Count > 0: + { + // The same call site again: the same content, other content, or another + // framework's + var sent = patches[random.Next(patches.Count)]; + var again = Copy(sent, sent.LineHint, $"content {random.Next(3)}", random.Next(3) == 0 ? "net9.0" : "net10.0"); + what = $"again {again.SourceFile}:{again.LineHint}"; + expected = pending.Enqueue(again); + state = ViewerSession.EnqueueInline(state, again); + break; + } + case 1 when patches.Count > 0: + { + // The same call site, reported at another line + var index = random.Next(patches.Count); + var moved = Copy(patches[index], patches[index].LineHint + 1000, patches[index].NewContent, "net10.0"); + patches[index] = moved; + what = $"moved {moved.SourceFile}:{moved.LineHint}"; + expected = pending.Enqueue(moved); + state = ViewerSession.EnqueueInline(state, moved); + break; + } + case 2 when patches.Count > 0: + { + var settling = patches[random.Next(patches.Count)]; + var key = InlineKey.For(settling.SourceFile, settling.LineHint); + var origin = random.Next(2) == 0 ? "net10.0" : null; + what = $"settle {key} {origin}"; + expected = pending.Settle(key, origin, settling.MemberName); + state = ViewerSession.Settle(state, key, origin, settling.MemberName); + break; + } + case 3 when state.Current is { Kind: QueueEntryKind.Inline } current: + { + var result = random.Next(3) == 0 + ? InlineApplyResult.Failed("the file is held") + : InlineApplyResult.Applied; + what = $"accept {current.Key} {result.Status}"; + expected = current.Conflicted + ? pending.Accept(current.Key, current.Variants[current.SelectedVariant].Origins[0], _ => result, out _) + : pending.Accept(current.Key, _ => result, out _); + state = ViewerSession.Apply(state, CommandKind.Accept, Fixtures.Applying(result)); + break; + } + case 4 when state.Current is { Kind: QueueEntryKind.Inline } current: + what = $"discard {current.Key}"; + expected = pending.Discard(current.Key, out _); + state = ViewerSession.Apply(state, CommandKind.Discard, Fixtures.Applied); + break; + case 5: + // A file, which goes in where it arrived and is put after its solution's + // snapshots by the next rebuild + state = ViewerSession.EnqueueTracked( + state, + Fixtures.Move($"Sample{random.Next(4)}.Test (txt)", Solution(random))); + continue; + case 6 when state.Queue.Count > 0: + state = ViewerSession.Apply(state, Command.Select(random.Next(state.Queue.Count))); + continue; + default: + { + var solution = Solution(random); + var file = $"Tests{random.Next(3)}.cs"; + var arriving = Fixtures.Patch( + solution is null ? file : Fixtures.SolutionFile(solution, "Tests", file), + 10 + step + iteration * 100, + $"\"old{step}\"", + $"content {random.Next(3)}", + testName: random.Next(2) == 0 ? $"Test{random.Next(3)}" : null, + framework: "net10.0"); + arriving.MemberName = $"Member{random.Next(6)}"; + patches.Add(arriving); + what = $"arrive {arriving.SourceFile}:{arriving.LineHint}"; + expected = pending.Enqueue(arriving); + state = ViewerSession.EnqueueInline(state, arriving); + break; + } + } + + var whole = ViewerSession.RebuildWhole(before, expected); + if (Describe(whole) != Describe(state.Queue)) + { + Assert.Fail( + $""" + iteration {iteration} step {step}: {what} + before: {Describe(before.Queue)} + whole: {Describe(whole)} + left: {Describe(state.Queue)} + """); + } + + // An entry the change did not touch is the entry it was, never one built again + foreach (var entry in state.Queue) + { + var was = before.Queue.FirstOrDefault(_ => _.Key == entry.Key); + if (was is not null && + ReferenceEquals(was.Variants, entry.Variants) && + was.Status == entry.Status && + !ReferenceEquals(was, entry)) + { + Assert.Fail($"iteration {iteration} step {step}: {what} built {entry.Key} again"); + } + } + } + } + } + + /// + /// What the short way is for, seen by what a test failing again allocates in a long queue: the + /// list is copied, and nothing is made for the entries the arrival was not about. + /// + [Test] + public async Task ARerunInALongQueueIsAboutOneEntry() + { + var state = SessionState.Start(ViewerMode.Inline, Fixtures.Columns, Fixtures.Rows); + for (var index = 0; index < 2000; index++) + { + state = ViewerSession.EnqueueInline( + state, + Fixtures.Patch($"Sample{index % 40}Tests.cs", 10 + index, "\"old\"", $"new {index}", testName: $"Test{index}")); + } + + var arriving = Fixtures.Patch("Sample20Tests.cs", 510, "\"old\"", "something else", testName: "Test500"); + var before = GC.GetAllocatedBytesForCurrentThread(); + var arrived = ViewerSession.EnqueueInline(state, arriving); + var allocated = GC.GetAllocatedBytesForCurrentThread() - before; + + await Assert.That(arrived.Queue.Count).IsEqualTo(2000); + await Assert.That(arrived.Queue.Single(_ => _.TestName == "Test500").LeftText).IsEqualTo("something else"); + // Rebuilt whole it was nearly two megabytes. What is left is the queue the session asks + // InlineQueue of, an item an entry, and the list + await Assert.That(allocated).IsLessThan(700_000); + } + + static InlinePatch Copy(InlinePatch patch, int line, string content, string framework) => + new(patch.SourceFile, line, patch.OriginalExpression, content) + { + TestName = patch.TestName, + MemberName = patch.MemberName, + Framework = framework + }; + + static string? Solution(Random random) => + random.Next(3) switch + { + 0 => "SolutionA", + 1 => "SolutionB", + _ => null + }; + + static InlineQueue Pending(SessionState state) => + InlineQueue.From( + state.Queue + .Where(_ => _.Kind == QueueEntryKind.Inline) + .Select(_ => new PendingInline(_.Variants, _.Status))); + + // Everything a rebuild decides about an entry: where it is, what it holds and from which + // frameworks, which of those is on screen, and what the last attempt said + static string Describe(IReadOnlyList queue) => + string.Join( + " | ", + queue.Select(_ => + $"{_.Kind} {_.Key} [{string.Join(";", _.Variants.Select(variant => $"{variant.Patch.NewContent}@{variant.Patch.LineHint}<{string.Join(",", variant.Origins)}>"))}] v{_.SelectedVariant} {_.Status} '{_.LeftText}'")); +} diff --git a/src/DiffEngineViewer.Tests/QueueSliceTests.cs b/src/DiffEngineViewer.Tests/QueueSliceTests.cs new file mode 100644 index 00000000..947bafa7 --- /dev/null +++ b/src/DiffEngineViewer.Tests/QueueSliceTests.cs @@ -0,0 +1,126 @@ +/// +/// The queue column is sliced to the rows that fit before any of them is described. Every frame +/// of a scroll or a drag is another state and so another screen, and describing every row of the +/// queue to draw the forty that fit was half a millisecond and a megabyte a frame at 2,000 +/// entries. A row has to say what it said when every row was described. +/// +public class QueueSliceTests +{ + /// + /// Whichever entry is selected and whatever is folded, the rows on screen are the same rows of + /// the whole list, labels grown by a collision elsewhere in the queue included. + /// + [Test] + public async Task TheRowsOnScreenAreRowsOfTheWholeList() + { + var random = new Random(21); + for (var iteration = 0; iteration < 40; iteration++) + { + // Ten rows of window is two of queue, so nearly every queue here is longer than it + var state = ViewerSession.Resize(Mixed(random), Fixtures.Columns, 10 + random.Next(6)); + await EverySelection(state); + foreach (var header in QueueProjection.Rows(state).Where(_ => _.GroupKey is not null).ToList()) + { + await EverySelection(ViewerSession.ToggleGroup(state, header.GroupKey!)); + } + } + } + + static async Task EverySelection(SessionState state) + { + var body = ScreenBuilder.BodyRows(state); + for (var selected = 0; selected < state.Queue.Count; selected++) + { + var selecting = state with { Selected = selected }; + var rows = QueueProjection.Rows(selecting); + var visible = QueueProjection.Visible(selecting, body, out var top); + await Assert.That(visible.Count).IsEqualTo(Math.Min(body, rows.Count)); + await Assert.That(visible.SequenceEqual(rows.Skip(top).Take(body))).IsTrue(); + } + } + + /// + /// A label is grown to tell an entry from one it would otherwise read the same as, and the + /// other one need not be on screen. So the labels are decided over the whole queue, whichever + /// rows are described. + /// + [Test] + public async Task ALabelIsGrownByAnEntryThatIsNotOnScreen() + { + var patches = new List + { + Fixtures.Patch(Fixtures.SolutionFile("SolutionA", "First", "Tests.cs"), 10, testName: "Same name") + }; + for (var index = 0; index < 30; index++) + { + patches.Add(Fixtures.Patch(Fixtures.SolutionFile("SolutionA", "First", $"Other{index}.cs"), 10, testName: $"Test {index}")); + } + + patches.Add(Fixtures.Patch(Fixtures.SolutionFile("SolutionA", "Second", "Tests.cs"), 10, testName: "Same name")); + var state = Fixtures.Inline([.. patches]); + + var visible = QueueProjection.Visible(state, ScreenBuilder.BodyRows(state), out var top); + + await Assert.That(top).IsEqualTo(0); + await Assert.That(visible.Count).IsLessThan(state.Queue.Count); + await Assert.That(visible[0].Label).IsEqualTo("First/Same name"); + } + + /// + /// What the slicing is for, seen by what a screen allocates: a scroll leaves the queue the + /// list it was, and the screen built after one describes the rows that fit and no others. + /// + [Test] + public async Task AScrollDescribesOnlyTheRowsThatFit() + { + var state = SessionState.Start(ViewerMode.Inline, Fixtures.Columns, Fixtures.Rows); + for (var index = 0; index < 2000; index++) + { + state = ViewerSession.EnqueueInline( + state, + Fixtures.Patch($"Sample{index % 40}Tests.cs", 10 + index, "\"old\"", $"new {index}\nand more\nand more", testName: $"Test{index}")); + } + + // The first screen of a queue works out its labels, which is once a queue + ScreenBuilder.Build(state); + var scrolled = ViewerSession.Apply(state, CommandKind.ScrollDown); + await Assert.That(scrolled).IsNotSameReferenceAs(state); + + var before = GC.GetAllocatedBytesForCurrentThread(); + var screen = ScreenBuilder.Build(scrolled); + var allocated = GC.GetAllocatedBytesForCurrentThread() - before; + + await Assert.That(screen.Queue.Count).IsEqualTo(ScreenBuilder.BodyRows(state)); + // Every row described was over a megabyte. What is left is the walk, a few bytes an entry + await Assert.That(allocated).IsLessThan(150_000); + } + + /// + /// Snapshots across two solutions and outside any, some sharing a test and some sharing only + /// a test's name across projects, so there are headers of both kinds and labels that collide. + /// + static SessionState Mixed(Random random) + { + string[] solutions = ["SolutionA", "SolutionB"]; + string[] projects = ["First", "Second"]; + var patches = new List(); + for (var index = random.Next(6, 24); index > 0; index--) + { + var file = $"Tests{random.Next(3)}.cs"; + var source = random.Next(4) == 0 + ? file + : Fixtures.SolutionFile(solutions[random.Next(2)], projects[random.Next(2)], file); + patches.Add( + Fixtures.Patch( + source, + 10 + patches.Count, + $"\"old{patches.Count}\"", + $"new {patches.Count}", + testName: random.Next(3) == 0 ? null : $"Test{random.Next(4)}")); + } + + var state = Fixtures.Inline([.. patches]); + state = ViewerSession.EnqueueTracked(state, Fixtures.Move(solution: "SolutionA")); + return ViewerSession.EnqueueTracked(state, Fixtures.Delete(solution: "SolutionB")); + } +} diff --git a/src/DiffEngineViewer.Tests/ScreenPayloadTests.cs b/src/DiffEngineViewer.Tests/ScreenPayloadTests.cs index b4993a6e..4a448f8f 100644 --- a/src/DiffEngineViewer.Tests/ScreenPayloadTests.cs +++ b/src/DiffEngineViewer.Tests/ScreenPayloadTests.cs @@ -31,7 +31,7 @@ public async Task ASegmentIsTheBytesOfItsOwnCharacters() var checkedSegments = 0; for (var index = 0; index < screen.Left.Rows.Count; index++) { - var text = RowText.Clip(RowText.Flatten(screen.Left.Rows[index].Text), screen.Columns); + var text = RowText.Shown(screen.Left.Rows[index].Text, screen.PaneCells); var row = payload.Rows[index]; var segments = CellGrid.Segments(text); await Assert.That(row.SegmentCount).IsEqualTo(segments.Count); @@ -55,6 +55,35 @@ public async Task ASegmentIsTheBytesOfItsOwnCharacters() await Assert.That(checkedSegments).IsGreaterThan(15); } + /// + /// A row is encoded as far as a pane can show it and no further. Neither pane is wider than + /// half the window, and a row of two cell characters cut at the window's width in characters + /// was encoded four times that far, a segment a character. + /// + [Test] + public async Task ARowIsEncodedAsFarAsAPaneCanShowIt() + { + var wide = new string('漢', 300); + var narrow = new string('a', 300); + var state = ViewerSession.Resize(Fixtures.File($"{wide}\n{narrow}\n\t{narrow}\nshort", "other"), 200, 30); + var screen = ScreenBuilder.Build(state); + var payload = new ScreenPayload(); + + payload.Build(screen); + + await Assert.That(screen.PaneCells).IsEqualTo(101); + // Fifty one characters are the first to reach a hundred and one cells, at two cells each + await Assert.That(Text(payload, 0)).IsEqualTo(new string('漢', 51)); + await Assert.That(payload.Rows[0].SegmentCount).IsEqualTo(51); + await Assert.That(Text(payload, 1)).IsEqualTo(new string('a', 101)); + // A tab is the four cells it is drawn as + await Assert.That(Text(payload, 2)).IsEqualTo(" " + new string('a', 97)); + await Assert.That(Text(payload, 3)).IsEqualTo("short"); + } + + static string Text(ScreenPayload payload, int row) => + Encoding.UTF8.GetString(payload.Strings.Slice(payload.Rows[row].TextOffset, payload.Rows[row].TextLength)); + /// /// The loop hands over the screen it handed over last frame for as long as nothing happens, /// and encoding that one again was most of what an idle frame cost these heads. Seen by what diff --git a/src/DiffEngineViewer.Tests/TrackedWatchTests.cs b/src/DiffEngineViewer.Tests/TrackedWatchTests.cs index b7b0175b..499c159b 100644 --- a/src/DiffEngineViewer.Tests/TrackedWatchTests.cs +++ b/src/DiffEngineViewer.Tests/TrackedWatchTests.cs @@ -331,6 +331,52 @@ public async Task AFileWrittenAgainWithWhatItHeldIsNotDiffedAgain() await Assert.That(ReferenceEquals(host.State, settled)).IsTrue(); } + /// + /// And nothing about the window is another thing for it. A run that fails the same way writes + /// its received file again every time, and the pass that took the new stamp used to close a + /// context menu the reader had open, as an entry that changed does. The menu, what it was + /// opened over and the status line are all what they were, whether the file written again is + /// the one on screen or another. + /// + [Test] + [Arguments(0)] + [Arguments(1)] + public async Task AFileWrittenAgainWithWhatItHeldLeavesAnOpenMenuOpen(int written) + { + var host = OwnedAll(2); + host.Mutate(_ => ViewerSession.OpenMenu(_, 0) with { Message = "Accepted Sample9.Test" }); + var menu = host.State.Menu; + await Assert.That(menu).IsNotNull(); + var before = host.State.Queue[written]; + File.SetLastWriteTimeUtc(before.LeftFile!, DateTime.UtcNow.AddMinutes(1)); + + new TrackedWatch(host).Pump(); + + var after = host.State.Queue[written]; + await Assert.That(after.LeftStamp).IsNotEqualTo(before.LeftStamp); + await Assert.That(host.State.Menu).IsSameReferenceAs(menu); + await Assert.That(host.State.Selected).IsEqualTo(0); + await Assert.That(host.State.Message).IsEqualTo("Accepted Sample9.Test"); + } + + /// + /// A file written again with something else is an entry that changed, and that still closes + /// the menu: what it offered was offered of what the entry showed. + /// + [Test] + public async Task AFileWrittenAgainWithSomethingElseStillClosesAnOpenMenu() + { + var host = OwnedAll(2); + host.Mutate(_ => ViewerSession.OpenMenu(_, 0)); + await Assert.That(host.State.Menu).IsNotNull(); + await File.WriteAllTextAsync(host.State.Queue[1].LeftFile!, "what a later run received instead"); + + new TrackedWatch(host).Pump(); + + await Assert.That(host.State.Queue[1].LeftText).IsEqualTo("what a later run received instead"); + await Assert.That(host.State.Menu).IsNull(); + } + /// /// The same over the socket, which is how the run itself says the pair is pending again. /// diff --git a/src/DiffEngineViewer.Windows.Tests/AmpersandTests.cs b/src/DiffEngineViewer.Windows.Tests/AmpersandTests.cs index 7819f7c1..f8d72e3f 100644 --- a/src/DiffEngineViewer.Windows.Tests/AmpersandTests.cs +++ b/src/DiffEngineViewer.Windows.Tests/AmpersandTests.cs @@ -31,4 +31,32 @@ public async Task The_status_line_does_not_read_one_as_a_mnemonic() await Assert.That(status.UseMnemonic).IsFalse(); } + + /// + /// Nor a footer button, whose label is the model's as the status is. One that did made the + /// letter after the ampersand an Alt chord that pressed it. + /// + [Test] + public async Task A_footer_button_does_not_read_one_as_a_mnemonic() + { + using var form = new ViewerForm("title", 800, 600); + var screen = ScreenBuilder.Build(Fixtures.File()); + + form.Apply( + screen with + { + Buttons = [new("Accept R&D", true, CommandKind.Accept), .. screen.Buttons] + }); + + var buttons = form.Controls + .Find("status", true) + .Single() + .Parent! + .Controls + .OfType() + .ToList(); + await Assert.That(buttons.Count).IsGreaterThan(1); + await Assert.That(buttons[0].Text).IsEqualTo("Accept R&D"); + await Assert.That(buttons.All(_ => !_.UseMnemonic)).IsTrue(); + } } diff --git a/src/DiffEngineViewer.Windows.Tests/EnlargedPictureTests.cs b/src/DiffEngineViewer.Windows.Tests/EnlargedPictureTests.cs index f44dc949..cd63ae81 100644 --- a/src/DiffEngineViewer.Windows.Tests/EnlargedPictureTests.cs +++ b/src/DiffEngineViewer.Windows.Tests/EnlargedPictureTests.cs @@ -357,7 +357,7 @@ static string Write(string name, bool across) /// sealed class Host : IDisposable { - readonly Form form = new() + readonly Form form = new ParkedForm { StartPosition = FormStartPosition.Manual, Location = new(-4000, -2000), diff --git a/src/DiffEngineViewer.Windows.Tests/FormsHeadTests.cs b/src/DiffEngineViewer.Windows.Tests/FormsHeadTests.cs index b7b2bd45..2ea37526 100644 --- a/src/DiffEngineViewer.Windows.Tests/FormsHeadTests.cs +++ b/src/DiffEngineViewer.Windows.Tests/FormsHeadTests.cs @@ -456,6 +456,176 @@ public async Task AFooterButtonIsSizedToItsCurrentLabel() await Assert.That(after.Width).IsGreaterThanOrEqualTo(button.MinimumSize.Width); } + /// + /// A document's ten buttons in a window too narrow for one row of them: every one of them is + /// inside the footer, on more rows than one. In one row, those past the window's edge could + /// not be reached at all. + /// + [Test] + public async Task ButtonsThatDoNotFitOneRowWrap() + { + using var host = new FormHost(Fixtures.Document()); + host.Settle(); + var footer = Field(host.Form, "footer"); + var oneRow = footer.Height; + + host.Form.ClientSize = new(560, 700); + host.Settle(); + + var buttons = FooterButtons(host.Form); + await Assert.That(buttons.Count).IsEqualTo(10); + await Assert.That(buttons.Select(_ => _.Top).Distinct().Count()).IsEqualTo(2); + await Assert.That(buttons.All(_ => footer.ClientRectangle.Contains(_.Bounds))).IsTrue(); + await Assert.That(footer.Height).IsGreaterThan(oneRow); + } + + /// + /// What the footer takes comes off the canvas, which says so: the model is asked for the + /// rows the canvas has left, so none is drawn behind a footer that grew. + /// + [Test] + public async Task ATallerFooterIsRowsTheBodyIsNotAskedFor() + { + using var host = new FormHost(Fixtures.File(Lines(300, 3), Lines(300))); + host.Settle(); + var footer = Field(host.Form, "footer"); + var before = (Rows: host.State.Rows, Footer: footer.Height, Canvas: host.Canvas.Height); + + // A file's five buttons are on two rows at this width + host.Form.ClientSize = new(300, host.Form.ClientSize.Height); + host.Settle(); + + var grew = footer.Height - before.Footer; + var cell = host.Canvas.CellSize(); + await Assert.That(grew).IsGreaterThan(cell.Height); + await Assert.That(host.Canvas.Height).IsEqualTo(before.Canvas - grew); + await Assert.That(host.State.Rows).IsEqualTo(host.Canvas.BodyCapacity + ScreenBuilder.Chrome); + await Assert.That(host.State.Rows).IsLessThan(before.Rows); + // The last row the model sliced ends inside the canvas + var rows = ScreenBuilder.Build(host.State).Left.Rows.Count; + await Assert.That(host.Canvas.BodyTop() + rows * cell.Height).IsLessThanOrEqualTo(host.Canvas.Height); + } + + /// + /// A status with no room beside the buttons goes under them, and the label is as tall as the + /// lines it wraps to: nothing of it is cut. Then one that fits beside them is beside them + /// again, in a footer the height it always was. + /// + [Test] + public async Task AStatusWithNoRoomBesideTheButtonsHasLinesOfItsOwn() + { + using var form = new ViewerForm("title", 700, 600); + var screen = ScreenBuilder.Build(ViewerSession.Resize(Fixtures.Document(), columns, rows)); + var footer = Field(form, "footer"); + var status = Field