Skip to content

Replace the alternates workaround with the upstream fix - #1011

Open
Johannes Schindelin (dscho) wants to merge 29 commits into
microsoft:vfs-2.56.0from
dscho:replace-alternates-workaround
Open

Johannes Schindelin (dscho) wants to merge 29 commits into
microsoft:vfs-2.56.0from
dscho:replace-alternates-workaround

Conversation

@dscho

Copy link
Copy Markdown
Member

This replaces our alternates workaround with Patrick Steinhardt's upstream v3 fix, backported from seen. Keeping this in one PR against vfs-2.56.0 lets the VFSForGit and Scalar functional workflows expose integration problems before the next VFS rebase.

The cancellation is deliberately a fixup!, allowing the next merging rebase to autosquash away the original workaround rather than retain it alongside a revert. The backport also includes the necessary setup-split/alternates-at-creation and files-backend-alternates prerequisites missing from this base.

The complete C/Git/Scalar build (NO_RUST=1), unit suite, and 18 focused scripts (447 tests) passed. The final alternates regression passes with both SHA-1 and SHA-256. The VFS compatibility changes are included alongside the prerequisite API changes, and those intermediate states build too.

A broader Scalar rerun hit this host's inotify limit (inotify_init1(): Too many open files). The existing PR workflows should provide clean VFSForGit and Scalar functional coverage; local Perforce/Subversion/CVS coverage is not required for this preparation.

This reverts commit 78c3276.

Make way for Patrick Steinhardt's replacement from upstream/seen rather
than carry duplicate logic into future rebases.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The function `init_db()` is responsible for creating the on-disk
directory structure required for a Git repository. It is used by both
git-init(1) and git-clone(1), and because their expected behaviour is
different we support a couple of flags:

  - The `QUIET` flag controls whether the command is quiet or not. For
    git-init(1) this is user-controllable, whereas for git-clone(1)
    we're always quiet.

  - The `EXIST_OK` flag controls whether a preexisting repository is
    okay or not. For git-init(1) it is, for git-clone(1) it's not.

  - The `SKIP_REFDB` flag controls whether the reference database should
    already be created or not. For git-init(1) we do, but for
    git-clone(1) we don't because it does not yet know about the default
    branch and about the remote object hash.

Furthermore, we're about to add another divergence in behaviour, where
we have to also skip creation of the object database in git-clone(1).
This is becoming quite cumbersome though.

Instead of introducing another flag, start to split up concerns of the
function so that we never create the reference or object database. This
becomes the responsibility of the caller, which is thus free to defer
their creation to a later point in time. This lets us get rid of most of
the divergent behaviour:

  - We don't need the `SKIP_REFDB` and a potential `SKIP_ODB` flags
    anymore.

  - We don't need the `QUIET` flag anymore, as nothing prints output
    except for the final status message that tells the user that the
    repository has been (re)initialized. But as this message is specific
    to git-init(1), we can easily move it there.

The only piece of information we still have to convey is whether or not
reinitialization of a preexisting repository is okay. This is handled by
a new `reinit_ok` pointer that, if non-`NULL`, indicates that it is okay
to reinitialize the repository. Furthermore, the pointer will be written
to to indicate whether the repository was reinitialized or not, which we
need in git-init(1) to print the correct initialization message.

With these refactorings, `init_db()` is named quite misleadingly though,
as we don't create any of the reference or object databases anymore.
Rename it to `create_repository()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a repository we defer initialization of the reference
database. This is because we don't yet know all details required for us
to initialize the refdb in the first place. Most importantly, what we
are missing is information about the object hash.

We don't do the same thing for the object database yet, but here we
essentially have the same problem. While the "files" database does not
need any information about the object format at creation time, alternate
backends are likely to require that information so that they can
properly set up their data structures.

Besides this forward-looking future proofing though, we also have a
second use case for deferring initialization of the object database,
namely alternates. When initializing the object database we do not yet
know whether we'll need alternates or not because this depends on the
repository we're about to clone from. If it is a local repository and
the user has passed "--reference{,-if-able}", then we will end up
writing alternates into the object database.

The ugly part though is that we cannot determine where the repository is
getting cloned from before it has been initialized. While we of course
already have access to the user-provided URI, that URI can be very well
rewritten via "url.<base>.insteadOf". We can of course read the global-
and system-level configuration to resolve it. But we explicitly resolve
the URI a second time after we have initialized the repository because
it can happen that we copy a ".git/config" over from our templates, and
that file may cause us to rewrite the path.

In a subsequent commit though we'll start to write alternates as part of
the repository initialization, so we'll need to have the URI properly
resolved before we can initialize the object database. This is ugly, but
as mentioned above it makes sense for us to defer its initialization
anyway so that we also know about the object hash already.

Defer creation of the object database until after we have resolved the
URI.

Note that this also requires us to defer the call to `setup_reference()`
until after we have created the object database. While you might think
that this function has something to do with references ("refs/*"), it is
in fact responsible for setting up alternates. Consequently, we can only
call it after we have created the object database already.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit, `setup_reference()` will start to call
`copy_alternates()`. Prepare for this by moving the function further
down so that we can avoid adding a declaration.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can pass "--reference{,-if-able}" to git-clone(1) to instruct it
to set up alternates for the newly created repository. This allows it to
reuse objects from the source repository so that in the best case we
don't have to clone all objects over.

Those options are handled by the confusingly named `setup_reference()`
function -- without the above context, one might rightfully believe that
it was about refs, not about alternates. The function itself is rather
simple: we loop through all provided alternate paths and then, if such
an alternate is valid, we write it to the object database.

In subsequent commits we're about to consolidate the complete setup of
alternates into this function, and furthermore we'll refactor the setup
of the object database to handle doing this for us instead of writing
the alternates into it one by one.

Prepare for this refactoring by collecting the alternates into a strvec.
Rename the function to `collect_alternates()` to clarify its scope, as
it does not set up the references itself anymore.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When cloning a local repository with "--shared" we add that repository
to the new repository's alternates. This is done in `clone_local()`,
which is responsible for performing local clones.

Move the logic into `collect_alternates()` to unify our setup of
alternates. Furthermore, this will allow us to set up alternates right
at creation time of the object database.

Note that the logic for cloning a local repository with "--no-shared" is
not yet part of `collect_alternates()`. This will be handled in the next
commit, but means that at this step, we may compute `commondir` without
it being used. It will become used in the next step though.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar as in the preceding commit, move the setup of alternates for
local clones with "--no-shared" into `collect_alternates()`. With this
step, the complete setup of alternates is now handled by that function.

Note that besides moving stuff around, it also fixes a bug: previously,
we did not know to resolve the referenced repository's common directory.
Consequently, when referencing a worktree we failed to resolve
alternates. But as `collect_alternates()` already knows to resolve the
commondir for "--local" we can simply reuse this resolved path for our
purpose.

Add two tests, the first one of which exercises this bug to avoid future
regressions. The second test ensures that we properly handle relative
alternates for a referenced worktree.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the ability to write alternates when creating the object database.
This change allows us to remove the `write_alternates()` callback in a
subsequent patch.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating a repository with alternates we first initialize the
object database and then write alternates to it in a separate step. This
is unfortunate due to a couple of reasons:

  - It requires us to have a `write_alternates()` callback, which is
    unfortunate as we never even write alternates to an object database
    after it has been created.

  - We're about to make alternates an implementation detail of the
    object database's backend in a future patch series, so having this
    callback is suboptimal there.

  - The backend has more flexibility with how exactly alternates are
    configured when it itself is in full control over their setup at the
    time where it creates the object database itself.

We have thus introduced the ability to write alternates right at
creation time in the preceding commits, and we have unified setup of
alternates into a single location. All that's left to do for us now is
to wire up alternates as an option for the database creation.

Do so.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are no users of `odb_source_write_alternates()` in our tree
anymore. Remove that function and its supporting infrastructure.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users can ask git-commit-graph(1) to write a commit graph specifically
for a set of packfiles via the "--stdin-packs" option. Those users are
expected to pass in relative paths, and those eventually get resolved in
`fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
specific object database source, as the subsystem now needs to assume
where a specific packfile is located relative to the source itself.

Refactor the logic to instead require the caller to pass in resolved
packfiles to untangle that dependency. This also makes the next change
easier to implement, where we'll get rid of passing the source to the
commit-graph subsystem.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To read or write a commit graph we require access to the repository that
the graph should be read from or written for as well as the object
directory to derive its location. Instead of passing in these two bits
of information explicitly though, we pass in a `struct odb_source`,
which carries with it both of these pieces of information.

But ultimately, this is somewhat flawed as we now assume that the source
even has an on-disk representation, and furthermore we assume that the
commit graphs would live in ".git/objects/info". This is true for the
"files" backend, but it's not necessarily true for any other backend
that we may eventually want to introduce. So eventually, we'll want to
evolve the commit-graph subsystem to become agnostic of the backend's
layout and let the backend itself decide where to read a commit graph
from or where to write it to.

We're not there yet to do that switch, but the current design is already
causing issues for the intent of this patch series where we want to move
alternates into the "files" backend.

Convert the subsystem to take a repository plus an object directory path
instead. This unblocks moving around alternates, and it's also a step
in the right direction for moving commit graphs into the source in a
later series.

Note that we previously compared the `struct odb_source` pointers of two
commit graphs to figure out whether they were located in the same object
directory, whereas we now have to compare their paths. Callers may pass
in those paths in different forms though, for example relative to the
current working directory or as absolute paths. Convert the object
directory into an absolute path both when loading and when writing
commit graphs so that the comparisons become more robust.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "files" object database source consists of two separate backends for
storing loose and packed objects. These are managed as somewhat separate
entities even though they derive from the same object directory, whether
it's the primary object directory or that one from an alternate.

In a subsequent commit we'll move the handling of alternates into the
"files" backend completely so that it becomes another implementation
detail thereof. As part of this, we'll want to keep track of both of
these sub-sources as a single entity derived from their respective
object directory.

Prepare for this change by introducing a new `struct odb_files_dir` that
encapsulates them. For now, every "files" source has exactly one such
directory. In a subsequent commit though, we'll make it a linked list of
directories so that we can manage multiple such directories in a single
"files" source.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The function `odb_for_each_alternate()` iterates through all alternates
of an object database. This is becoming an implementation detail of the
"files" backend though, where each alternate will be represented by one
`struct odb_files_dir`.

Adapt `odb_for_each_alternate()` to already iterate through these
structs instead of iterating through sources.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar as in the preceding commit, `odb_find_source()` can be used to
yield a single source by its object directory. This function is also
specific to the "files" backend once alternates are an implementation
detail thereof.

Refactor it to be specific to the "files" backend and return an `struct
odb_files_dir` to prepare for moving alternates into the "files" source.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit we'll migrate handling of alternates into the
"files" backend. As part of that, the backend will need to grow the
ability to handle multiple object directories for alternates and for
transactions.

Introduce a `struct odb_files_dir::next` pointer so that we can have
multiple directories. Adapt the backend to loop through this list as
necessary. In general:

  - For reading paths we will loop through all object directories until
    we have found the object.

  - For writing paths we will typically write to the first of our object
    directories.

This mechanism isn't used yet as we still only ever have a single object
directory at the current point in time. But that will change over
subsequent commits.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The functions `odb_set_temporary_primary_source()` and
`odb_restore_primary_source()` can be used to adapt the primary object
database source. Nowadays though we only have a single user of this
subsystem left, which is the "tmp-objdir" subsystem.

Despite that, this functionality is also becoming less useful overall as
alternates are becoming an implementation detail of the "files" backend.
And with that change, there will only ever be a single source attached
to the object database anyway.

Move the logic into the "tmp-objdir" subsystem accordingly.

Assisted-by: GPT-6.1 Sol
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When creating a quarantine directory via the "tmp-objdir" subsystem we
create a new "files" backend that new objects part of the transaction
can be written to. In a future commit though we'll move handling of
alternates into the "files" backend, and as part of that it will no
longer be possible for us to have multiple sources attached to a single
object database.

In a preceding commit, we have prepared the "files" backend to be able
to handle multiple object directories. We don't use that mechanism for
alternates yet, but will start doing so in a subsequent commit. But with
that infrastructure ready we can already migrate tmp-objdirs over to use
this new mechanism.

Adapt the subsystem so we create a `struct odb_files_dir` instead of a
new "files" source.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When creating transactions via the "tmp-objdir" subsystem callers are
expected to first call `tmp_objdir_create()` to create the quarantine
directory and then `tmp_objdir_replace_primary_odb()` to activate that
quarantine directory as the primary object database source so that all
newly written objects are written into it.

This dance is performed by all users of temporary object directories, so
that makes the setup unnecessarily involved. Merge the logic to replace
the primary object database source into `tmp_objdir_create()` to
simplify the dance.

Note that with the quarantine directory being installed by
`tmp_objdir_create()`, users of this function no longer get a pointer to
the underlying quarantine directory. So instead of repreparing only that
directory, we now reprepare the whole source instead, and as we know
that the source has wired up the quarantine directory already, this will
cause us to reprepare the quarantine directory, too.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `struct odb_source::will_destroy` flag tracks whether a source is
part of a transaction that we know we'll destruct anyway. If so, the
backend can optimize for that particular case, for example by not
flushing any data to disk.

While the intent is sensible, it assumes that transactions are backed by
a separate source that's being linked into the object database. But that
may or may not be true, as backends may have significantly better ways
to achieve the same. So the assumption doesn't make much sense in the
first place, as we're now tracking backend-specific details on the
generic `struct odb_source` level.

Move the field from the generic source into the "loose" source, as this
is the only source that ever makes use of it anyway.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the next commit we'll add a new callsite that wants to read
alternates without having a proper object database source for a given
alternate available to it. Prepare for this by extracting the logic into
a separate function that only requires an object directory path as
input.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.

But while well-intentioned, this design led to a bunch of conceptual
problems:

  - We're now trying to read objects in source order, whereas we
    previously tried to read objects via packfiles before trying to read
    them via loose objects. This led to a performance regression when
    using alternates or when using a quarantine directory.

  - Some data structures are supposed to only ever exist once, like for
    example bitmaps and commit graphs. At the same time, those data
    structures also span across the union of all objects, so they may
    cross sources.

  - It is unclear how we can extend GIT_OBJECT_DIRECTORY or
    GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
    backwards-compatible way. In general, introducing an object storage
    extension into the current status quo where alternates may have to
    be extended to become generic was proving to be painful.

  - Some mechanisms of alternates assume way too much about how exactly
    their backends work. Alternate refs for example assume that the
    alternate is backed by a filesystem path, and that this filesystem
    path may also allow us to read references. This is not a given
    though, as backends may not even have local data at all.

In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.

Correct course by moving alternates into the "files" source itself so
that it becomes an implementation detail thereof so that we can avoid
all of these shortcomings. While it's unfortunate that we cannot easily
mix and match sources now, that ability doesn't go away. It's still very
much feasible to introduce a new backend that allows for exactly that
use case, and such a backend may also be a lot more flexible as we can
now add new logic to determine which objects should be stored where. So
the original motivation for having per-source backends can still be
realized with the new architecture.

Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
"files" backend. This may be surprising at first, but object directories
are very much a concept of that backend, too. So these variables would
have bad interactions with other backends, and they create a bit of a
mismatch with the eventual object storage extension that we plan to
introduce.

Note that this commit is way larger than I'd like it to be. I'm sorry, I
couldn't find a way to split it up further. That being said, most of the
changes are straight-forward conversions that go from iterating over
sources to iterating over object directories. The more involved changes
are moving the infrastructure to track individual sources from "odb.c"
and moving them into "odb/source-files.c".

This also serves to show a bit of a who's-who of commands that don't
work properly with pluggable object databases. Almost all of these
commands are related to housekeeping though -- some of them will be
converted eventually, like for example commit graphs or MIDXs. But many
of them will stay incompatible going forward as they are simply too
specific to the "files" backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that alternates are an implementation detail of the "files" backend
there is no reason anymore to expose them anywhere else. The callback to
read alternates is thus no longer required. Remove it.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the move towards pluggable object databases we reorganized how we
access alternates. Before the introduction of pluggability, we were
reading objects by first scanning through packfiles from all alternates
before scanning through all loose object directories. With the change
though, we are now iterating through the sources one by one, checking
each of them for whether they contain the object either as a packed or
loose object.

Checking whether an object exists in packfiles is comparatively cheap
because the list of packs is cached, and the pack indices are likely to
be in the page cache. In contrast to that, checking whether an object
exists in loose format always incurs one stat(3p) syscall. So scanning
for objects one alternate at a time led to a performance regression in
in the case where most of the objects are contained in a packfile of an
alternate, as reported in [1].

We could fix that issue by introducing another flag that tells the
backends to only search the optimized storage backend. But that is
adding more complexity to the object database layers that is ultimately
only really required because of the way the "files" backend stores its
data. So for an alternate backend, such new infrastructure would not
really be a good fit.

But since then we have moved handling of alternates into the "files"
backend itself. One of the consequences of this is that this backend is
now having full control over all of its sub-sources, and that also makes
it way easier for us to optimize for this access pattern again.

Do so by first trying to read objects from packfiles located in any of
its object directories before trying to read loose objects.

The following benchmark demonstrates that this significantly improves
performance in some cases. The benchmark uses the Git repository itself
and sets up its object directory as an alternate, while the primary
object directory is simply an empty object directory. It then reads all
commits enumerated via `git rev-list HEAD` with git-cat-file(1):

  Benchmark 1: git cat-file --batch-check --unordered (rev = HEAD~)
    Time (mean ± σ):     179.4 ms ±   2.6 ms    [User: 114.6 ms, System: 61.4 ms]
    Range (min … max):   177.1 ms … 187.7 ms    16 runs

  Benchmark 2: git cat-file --batch-check --unordered (rev = HEAD)
    Time (mean ± σ):      95.0 ms ±   1.5 ms    [User: 79.8 ms, System: 15.0 ms]
    Range (min … max):    93.0 ms …  97.9 ms    30 runs

  Summary
    git cat-file --batch-check --unordered (rev = HEAD) ran
      1.89 ± 0.04 times faster than git cat-file --batch-check --unordered (rev = HEAD~)

[1]: <CAFXJcxvpKHoVDwE5mBOd=w-A5vPdUmehqr8SHLUD7qv1qB00rA@mail.gmail.com>

Reported-by: Wolfgang Kritzinger <wkritzinger@atlassian.com>
Helped-by: Tyrie Vella <tyrielv@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch is essentially the equivalent to the preceding patch, but for
streaming objects instead of just reading them directly.

Exercising performance is a bit more complicated here as streaming only
really kicks in for large objects. Furthermore, I expect that the impact
is not as big as with the last commit, either, because for large objects
the time to stat the file will probably be dwarfed by the time it takes
to read the object itself.

But even though this is probably not as impactful, it makes sense anyway
to do this change to align behaviour across both of these functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When looking up an object fails we perform a second read in all sources
with the `OBJECT_INFO_SECOND_READ` flag set. This flag tells sources
that the first lookup has failed and that they should reload any on-disk
state that may have changed in the meantime so that the object may be
surfaced after all. The "packed" source uses this to reprepare its
packfiles, whereas the "loose" source short-circuits the second read
because there are no caches to flush.

This logic lives in the generic object database layer even though it is
highly specific to how the "files" backend handles its reads. But now
that alternates are an implementation detail of that source, we can
easily move that logic into the backend, as well.

Move the logic into the "files" source and drop the flag.

Note that the "packed" source also used this flag to figure out whether
it should try to surface an object via a packfile that should have the
object as determined by a multi-pack index, but where the packfile has
vanished. The multi-pack index is stale in that case, and that scenario
should really only happen when the process is racing with a concurrent
maintenance task.

This code is adapted to instead use `OBJECT_INFO_QUICK`, which means
that the fallback may trigger on the first read already, but not in a
case where callers know to handle false negatives already. In the worst
case this could cause another scan through all packfiles covered by the
multi-pack index. But that scan should be fast enough anyway, and it
only triggers when we hit the race. So the effect thereof should be
limited.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before moving alternates into the "files" backend we had to handle
alternates in `do_oid_object_info_extended()`. This made the retry logic
somewhat complex, as we had to juggle reloading of the sources, the
second read and fetching missing objects. And to handle the potential
retries we loop around that logic multiple times.

Nowadays though, the reload and retry logic are located in the "files"
backend itself. Consequently, using a loop is a bit on the heavy side
and makes the code harder to reason about.

Simplify the logic by removing the loop. Instead, we explicitly re-read
exactly once after having fetched objects from a promisor remote.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we failed reading an object in a repository with a promisor remote
we try to fetch it from the promisor and then re-read it. This logic
kicks in not only when the object couldn't be found, but also in the
case where it could be found but the object is corrupt. It feels dubious
though to just paper over corruption and proceed as if nothing has
happened, happily fetching data from the promisor.

Adapt the code so that we only fetch objects in case they are missing.

Suggested-by: Justin Tobler <jltobler@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `local` flag indicates whether a given `struct odb_source` is a
local source or not, where "local" is defined as being the object
directory that is directly attached to the current repository. But now
that alternates are handled in the "files" backend, this property is not
something that even makes sense anymore to have on the generic source.

In fact, we already have a partial replacement for this field in `struct
odb_files_dir`. The only other site that requires it is the packed
object database source, because it propagates the bit into the packfiles
it is opening.

Drop the flag from `struct odb_source` and instead move it into the
packed source.

Note that we stop considering the `local` bit when iterating through all
loose objects with the `ODB_FOR_EACH_OBJECT_LOCAL_ONLY` flag. This is
fine though, as we already handle the flag in the "files" backend
itself, where we skip over any object directory that is not marked as
local.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants