Repository navigation
gvfs-helper: add config to incrementally replace cache servers - #836
Merged
Derrick Stolee (derrickstolee) merged 8 commits intoJan 22, 2026
Conversation
Before modifying the config documentation more, fill in these blanks. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Derrick Stolee (derrickstolee)
force-pushed
the
override-cache-server-by-verb
branch
2 times, most recently
from
January 18, 2026 02:16
3c79ddd to
c83a879
Compare
In anticipation of tests for multiple cache-servers, update the existing logic that sets up and tears down cache-servers to allow multiple instances on different ports. Signed-off-by: Derrick Stolee <stolee@gmail.com>
This extension of the gvfs.cache-server config now allows a new key, gvfs.prefetch.cache-server, to override the cache-server URL for only the prefetch endpoint. The purpose of this config is to allow for incremental testing and deployment of new cache-server infrastructure. Hypothetically, we could have special-purpose cache-servers that are glorified bundle servers and other servers that focus on the object and size endpoints. More realistically, this will allow us to test cache servers that have only the prefetch endpoint ready to go. This allows some incremental rollout that is more controlled than a flag day replacing the entire infrastructure. Signed-off-by: Derrick Stolee <stolee@gmail.com>
This extension of the gvfs.cache-server config now allows a new key, gvfs.get.cache-server, to override the cache-server URL for only the prefetch endpoint. The purpose of this config is to allow for incremental testing and deployment of new cache-server infrastructure. Signed-off-by: Derrick Stolee <stolee@gmail.com>
This extension of the gvfs.cache-server config now allows a new key, gvfs.post.cache-server, to overrid the cache-server URL for only the batched objects endpoint. The purpose of this config is to allow for incremental testing and deployment of new cache-server infrastructure. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Test that all three verb-specific cache-server configs can be used simultaneously, each directing requests to a different server. This verifies that prefetch, get, and post verbs each respect their own override and don't interfere with each other.
The t5799-gvfs-helper.sh script is long and takes forever. This slows down PR merges and the local development inner loop is a pain. Before distributing the tests into a set of new test scripts by topic, extract important helper methods that can be imported by the new scripts. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Derrick Stolee (derrickstolee)
force-pushed
the
override-cache-server-by-verb
branch
from
January 18, 2026 02:30
c83a879 to
7222e1a
Compare
Move the tests from t5799-gvfs-helper.sh into multiple scripts that can run in parallel. To ensure that the ports do not overlap, add a large multiplier on the instance when needing multiple ports within the same test (currently limited to the verb-specific cache servers). Signed-off-by: Derrick Stolee <stolee@gmail.com>
Derrick Stolee (derrickstolee)
force-pushed
the
override-cache-server-by-verb
branch
from
January 18, 2026 02:35
7222e1a to
77a05f6
Compare
Derrick Stolee (derrickstolee)
marked this pull request as ready for review
January 18, 2026 03:41
Tyrie Vella (tyrielv)
approved these changes
Jan 21, 2026
Matthew John Cheetham (mjcheetham)
approved these changes
Jan 22, 2026
Matthew John Cheetham (mjcheetham)
left a comment
Matthew John Cheetham (mjcheetham)
left a comment
Member
There was a problem hiding this comment.
This looks good!
Derrick Stolee (derrickstolee)
merged commit Jan 22, 2026
9db3c15
into
microsoft:vfs-2.52.0
122 of 123 checks passed
1 task done
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jan 27, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jan 28, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jan 31, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Feb 3, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Feb 11, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Feb 11, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 12, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 17, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 17, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Apr 20, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 12, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Jun 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 12, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Jun 12, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 17, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Jun 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Jun 17, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
Derrick Stolee (derrickstolee)
added a commit
to microsoft/VFSForGit
that referenced
this pull request
Sep 4, 2026
## Why This Matters The microsoft/git GVFS helper can route protocol endpoints to dedicated cache servers, but VFS for Git previously sent every protocol request to one global cache URL. Supporting the same endpoint-specific preferences lets cache infrastructure migrate incrementally without making a dedicated endpoint a hard dependency. The `/gvfs/sizes` endpoint is unique to VFS for Git, so this change adds the corresponding `gvfs.sizes.cache-server` setting alongside the prefetch, GET, and POST settings used by Scalar. ## Commit Narrative 1. **Route GVFS endpoints to dedicated cache servers**: load, persist, and validate endpoint overrides; add matching `gvfs clone` options; preserve overrides during mount-time cache resolution. 2. **Fall back safely from dedicated cache endpoints**: retry failed endpoint requests through the global cache, preserve circuit-breaker budgets, distinguish network body failures from local processing errors, propagate cancellation unchanged, and emit authority-only telemetry. 3. **Explain endpoint-specific cache routing**: document configuration, precedence, fallback order, and troubleshooting. 4. **Cover prefetch failure telemetry redaction**: exercise terminal prefetch failures at the production entry point and ensure diagnostics retain only URI authority. ## Routing and Fallback Behavior - `gvfs.prefetch.cache-server` routes `/gvfs/prefetch`. - `gvfs.get.cache-server` routes object GET requests. - `gvfs.post.cache-server` routes object POST requests. - `gvfs.sizes.cache-server` routes `/gvfs/sizes`. - If an endpoint-specific route is absent or fails, the request uses `gvfs.cache-server`. - Sizes requests can additionally fall back from the global cache to origin. - Existing repositories without endpoint overrides retain their previous behavior. Endpoint-to-global transitions do not consume the process-wide circuit-breaker budget. Response-body transport failures can trigger fallback, while local processing failures remain on the current route. `OperationCanceledException` propagates without retry or fallback. Fallback and terminal prefetch diagnostics include only URI authority, excluding credentials, paths, queries, and fragments. ## Clone and Configuration Surface `gvfs clone` accepts `--prefetch-cache-server-url`, `--get-cache-server-url`, `--post-cache-server-url`, and `--sizes-cache-server-url`. Values are stored in local Git config and malformed absolute URLs are rejected. `gvfs cache-server --get` and `--set` continue to operate only on the global cache setting. Endpoint-specific values can be inspected or changed with `git config --local`. ## Related microsoft/git Changes - [microsoft/git#836: gvfs-helper: add config to incrementally replace cache servers](microsoft/git#836) - [microsoft/git#849: scalar: add endpoint cache-server clone options](microsoft/git#849) ## Tests Coverage includes configuration precedence and persistence, mount-time resolution, clone option parsing and URL validation, endpoint/global/origin routing, HTTP and transport failures, response-body failures, local handler failures, cancellation propagation, telemetry redaction, terminal endpoint reporting, and circuit-breaker accounting.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Sep 22, 2026
I'm exploring ideas around a new cache-server infrastructure. One of the
trickiest parts of deploying new infrastructure is the need to have all
endpoints ready to go at launch without incremental learning.
This change adds new `gvfs.<verb>.cache-server` config keys that allow
for verb-by-verb overrides of the typical `gvfs.cache-server` config
key. These are loaded on a per-verb basis and then reset after the
request. Further, if there is a failure then the request is retried with
the base cache-server URL.
This would allow us to, for example, deploy a service that serves only
the `gvfs/prefetch` endpoint and see if that is improving on latency and
throughput expectations before moving on to the GET and POST verbs for
single and batched object downloads.
As I was adding tests, I realized that we should split this test script
into distinct parts so we can have a faster inner loop when testing
specific areas. I know that this script is frequently the longest script
running in our PR and CI builds, so the parallel split should help
significantly.
Use commit-by-commit review. I tried to keep the last two commits as
obviously "copy-and-paste only" except for a small change to the port
calculation to avoid overlap when using multiple ports in parallel
tests.
* [X] This change only applies to interactions with Azure DevOps and the
GVFS Protocol.
Johannes Schindelin (dscho)
pushed a commit
that referenced
this pull request
Sep 22, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. Signed-off-by: Derrick Stolee <stolee@gmail.com>
Johannes Schindelin (dscho)
added a commit
that referenced
this pull request
Sep 22, 2026
In #836, we implemented custom gvfs.<verb>.cache-server config options that allow existing enlistments to point specific GVFS Protocol verbs at different cache servers. However, that did not help at clone time. This change adds --<verb>-cache-server-url options to 'scalar clone' to allow initial clones to take advantage of different cache servers. The intended use here is to take advantage of new cache server infrastructure as each protocol endpoint is deployed independently. * [X] This change only applies to interactions with Azure DevOps and the GVFS Protocol.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm exploring ideas around a new cache-server infrastructure. One of the trickiest parts of deploying new infrastructure is the need to have all endpoints ready to go at launch without incremental learning.
This change adds new
gvfs.<verb>.cache-serverconfig keys that allow for verb-by-verb overrides of the typicalgvfs.cache-serverconfig key. These are loaded on a per-verb basis and then reset after the request. Further, if there is a failure then the request is retried with the base cache-server URL.This would allow us to, for example, deploy a service that serves only the
gvfs/prefetchendpoint and see if that is improving on latency and throughput expectations before moving on to the GET and POST verbs for single and batched object downloads.As I was adding tests, I realized that we should split this test script into distinct parts so we can have a faster inner loop when testing specific areas. I know that this script is frequently the longest script running in our PR and CI builds, so the parallel split should help significantly.
Use commit-by-commit review. I tried to keep the last two commits as obviously "copy-and-paste only" except for a small change to the port calculation to avoid overlap when using multiple ports in parallel tests.
GVFS Protocol.