From 145c6af9f55ad81b8fd854bc0086ec9c0fb8f249 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 5 Nov 2026 22:41:38 +0000 Subject: [PATCH 01/12] Clarify Azure billing transitions for volume-based Enterprise subscriptions (#64282) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bot-digital-customer-success <121182491+bot-digital-customer-success@users.noreply.github.com> Co-authored-by: tecrogue Co-authored-by: Daniel Klemm Co-authored-by: Siara <108543037+SiaraMist@users.noreply.github.com> --- content/billing/concepts/azure-subscriptions.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/billing/concepts/azure-subscriptions.md b/content/billing/concepts/azure-subscriptions.md index 41cf10bcafbc..4e15fca532f7 100644 --- a/content/billing/concepts/azure-subscriptions.md +++ b/content/billing/concepts/azure-subscriptions.md @@ -16,6 +16,10 @@ category: You can pay for {% data variables.product.github %} use through an Azure subscription by connecting the subscription to {% data variables.product.github %}. See [AUTOTITLE](/billing/how-tos/set-up-payment/connect-azure-sub). +If you pay for {% data variables.product.prodname_copilot %} with a credit card or PayPal, you cannot switch your payment method to an Azure subscription. Newly created enterprise accounts paying through Azure must contact {% data variables.contact.github_support %} to enable {% data variables.product.prodname_copilot_short %}. + +If your existing {% data variables.product.prodname_enterprise %} subscription is invoiced and volume-based, connecting an Azure subscription routes metered usage to Azure but does not move existing license charges. To have these charges billed through Azure, work with your account manager to transition your enterprise to fully metered billing. Ask your account manager to confirm when the transition can take effect relative to your renewal and how it will affect your billing. + {% data variables.product.github %} installs a Subscription Permission Validation app (SPV app) on the Azure tenant, which it uses to get a list of available subscriptions from active directory. Installing the SPV app requires tenant-wide admin consent. You must sign into an Azure account that can provide tenant-wide admin consent, or work with an Azure administrator to configure the admin consent workflow. * [Grant tenant-wide admin consent to an application](https://learn.microsoft.com/azure/active-directory/manage-apps/grant-admin-consent) in Microsoft Docs From 453f91c066afa54fb58a833ff35a9a52550fbc8e Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 5 Nov 2026 22:41:52 +0000 Subject: [PATCH 02/12] Clarify pull request preservation during sensitive-data cleanup (#64509) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bot-digital-customer-success <121182491+bot-digital-customer-success@users.noreply.github.com> Co-authored-by: Daniel Klemm Co-authored-by: Anne-Marie <102995847+am-stead@users.noreply.github.com> Co-authored-by: Siara <108543037+SiaraMist@users.noreply.github.com> --- .../removing-sensitive-data-from-a-repository.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/content/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository.md b/content/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository.md index 8776b7585cf8..1e0ae39cd8aa 100644 --- a/content/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository.md +++ b/content/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository.md @@ -149,7 +149,10 @@ After using `git-filter-repo` to remove the sensitive data and pushing your chan {% ifversion fpt or ghec %}If you have successfully cleaned up all references other than PRs, and no forks have references to the sensitive data, Support will then:{% endif %} {% ifversion ghes %}If you have successfully cleaned up all references other than PRs, and no forks have references to the sensitive data, your site administrator will then:{% endif %} - * Dereference or delete any affected PRs on {% data variables.product.github %}. + * Dereference or delete any affected pull requests on {% data variables.product.github %}. Deleting an entire pull request removes its conversation, reviews, and diffs. + + {% ifversion fpt or ghec %}{% data variables.contact.github_support %} may offer to remove only a pull request's internal references ("dereferencing"). This can preserve the pull request body and comment history while making affected historical diffs inaccessible. Dereferencing does not remove sensitive data quoted in retained bodies or comments.{% endif %} + * Run a garbage collection on the server to expunge the sensitive data from storage. * Remove cached views. * If LFS Objects are involved, delete and/or purge the orphaned LFS objects. From 8938d999bff1d1570846a50203f121020fa6f8cd Mon Sep 17 00:00:00 2001 From: Eric Hollenberry Date: Thu, 5 Nov 2026 22:49:42 +0000 Subject: [PATCH 03/12] Document BYOK and enterprise custom model fields in Copilot usage metrics (#64387) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Siara <108543037+SiaraMist@users.noreply.github.com> Copilot-Session: 135ac0e8-9fc2-4f55-8330-8f5b4f0fe5ae --- .../concepts/models/bring-your-own-key.md | 9 +++ .../copilot-usage-metrics.md | 48 +++++++++++-- .../copilot-usage-metrics/example-schema.md | 71 +++++++++++++++++-- 3 files changed, 119 insertions(+), 9 deletions(-) diff --git a/content/copilot/concepts/models/bring-your-own-key.md b/content/copilot/concepts/models/bring-your-own-key.md index 5f1f5396b11a..35d5a48d987f 100644 --- a/content/copilot/concepts/models/bring-your-own-key.md +++ b/content/copilot/concepts/models/bring-your-own-key.md @@ -45,3 +45,12 @@ Custom models are available in {% data variables.copilot.copilot_byok_supported_ Users access these models through {% data variables.product.github %}'s {% data variables.product.prodname_copilot_short %} API without configuring the provider's credentials themselves. Users must have a {% data variables.product.prodname_copilot_short %} license and internet access to use the custom models. For more information, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/enable-custom-models). + +## Choosing between the two + +For most organizations, we recommend enterprise custom models as the default, with BYOK configured by users allowed where it's needed. + +* **Enterprise custom models** keep provider keys under central control, so users never handle credentials. Requests are routed through {% data variables.product.github %}, so usage is fully reported in {% data variables.product.prodname_copilot_short %} usage metrics. +* **BYOK configured by users** fits local or on-device models, air-gapped environments, and experimentation with new providers. Its usage is reported from client telemetry in supported clients only. You can disable it in IDEs with a policy. + +To see how each kind is used, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#byok-metrics-fields). diff --git a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md index 9afa23afd495..bb6e9a437dc6 100644 --- a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md +++ b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md @@ -102,7 +102,7 @@ These fields identify the scope and date of each record. The exact set present d ### Per-user report fields -Per-user reports contain one record per user for the reporting period. The 28-day per-user report also includes `report_start_day` and `report_end_day` to mark the reporting window. +Per-user reports contain one record per user per day. The 28-day per-user report contains a record for each day in the window, and each record also includes `report_start_day` and `report_end_day` to mark the reporting window. | Field | Type | Nullable | Description | |:--|:--|:--|:--| @@ -120,6 +120,7 @@ Per-user reports contain one record per user for the reporting period. The 28-da | `used_chat` | `boolean` | No | Whether the user used IDE chat that day. | | `used_cli` | `boolean` | No | Whether the user used {% data variables.copilot.copilot_cli_short %} that day. | | `used_copilot_app` | `boolean` | No | Whether the user used the {% data variables.copilot.github_copilot_app_short %} that day. | +| `used_local_byok` | `boolean` | Yes | Whether the user made at least one bring your own key (BYOK) request in {% data variables.copilot.copilot_cli_short %} or the {% data variables.copilot.github_copilot_app_short %} that day. `false` when no BYOK request was observed. Null when the signal is unavailable. This is a per-day value, including in each day of 28-day reports. See [BYOK metrics fields](#byok-metrics-fields). | | `used_copilot_coding_agent` | `boolean` | No | Whether the user used {% data variables.copilot.copilot_cloud_agent %} (previously Copilot coding agent) that day. | | `used_copilot_cloud_agent` | `boolean` | No | Whether the user used {% data variables.copilot.copilot_cloud_agent %} that day. Carries the same value as `used_copilot_coding_agent`; both names are retained for backward compatibility. | | `used_copilot_code_review_active` | `boolean` | Yes | Whether the user actively engaged with {% data variables.copilot.copilot_code-review_short %} that day. A user is considered active if they manually requested a {% data variables.product.prodname_copilot_short %} review, or applied a {% data variables.product.prodname_copilot_short %} review suggestion. Null when there is no {% data variables.copilot.copilot_code-review_short %} signal for the user that day. | @@ -138,6 +139,7 @@ Per-user reports contain one record per user for the reporting period. The 28-da | `totals_by_plugin` | `array` | Yes | Top plugins by recorded interaction count across supported clients. See [Customization fields](#customization-fields-api-only). | | `totals_by_cli` | `object` | Yes | CLI-specific metrics for the user. Omitted when the user had no {% data variables.copilot.copilot_cli_short %} usage that day. See [{% data variables.copilot.copilot_cli_short %} metrics fields](#copilot-cli-metrics-fields). | | `totals_by_copilot_app` | `object` | Yes | The {% data variables.copilot.github_copilot_app_short %} metrics for the user. Omitted when the user did not use the {% data variables.copilot.github_copilot_app_short %} that day. See [Metrics fields for the {% data variables.copilot.github_copilot_app_short %}](#copilot-app-metrics-fields). | +| `totals_by_local_byok` | `array` | Yes | BYOK usage for the user, with one entry for each BYOK kind and surface that had activity that day. This usage is a subset of `totals_by_cli` and `totals_by_copilot_app`. Omitted or null when the user had no BYOK requests that day. See [BYOK metrics fields](#byok-metrics-fields). | | `totals_by_3rd_party_agent` | `array` | Yes | Per-agent usage metrics for recognized {% data variables.copilot.agent_apps %}. Omitted when the user had no recognized {% data variables.copilot.agent_app %} activity during the reporting period. See [{% data variables.copilot.agent_apps_caps %} metrics fields](#agent-apps-metrics-fields). | | `totals_by_client` | `array` | No | Per-client breakdown of the user's successful model requests and token usage, counted server-side. The array is always present and is empty when there is no usage to report. See [Client metrics fields](#client-metrics-fields). | | `totals_by_vscode_agent` | `object` | Yes | Dedicated {% data variables.product.prodname_vscode_shortname %} Agents-window metrics for the user. Omitted when both required counts are not available. See [{% data variables.product.prodname_vscode_shortname %} Agents-window metrics fields](#vs-code-agents-window-metrics-fields). | @@ -173,6 +175,9 @@ Active-user counts: | `monthly_passive_copilot_code_review_users` | `integer` | No | Number of unique users who had {% data variables.copilot.copilot_code-review_short %} automatically assigned to review their pull request during a trailing 28-day window, with no active engagement. | | `daily_active_cli_users` | `integer` | Yes | Number of unique users who used {% data variables.copilot.copilot_cli_short %} on a given day. This count is **independent** of IDE active-user counts and is **not** included in IDE-based active-user definitions. Omitted for enterprises or organizations with no CLI usage that day. | | `daily_active_copilot_app_users` | `integer` | Yes | Number of unique users who used the {% data variables.copilot.github_copilot_app_short %} on a given day. Null when the enterprise or organization has no activity in the {% data variables.copilot.github_copilot_app_short %} that day. | +| `daily_active_local_byok_users` | `integer` | Yes | Number of unique users who made BYOK requests in {% data variables.copilot.copilot_cli_short %} or the {% data variables.copilot.github_copilot_app_short %} on a given day. See [BYOK metrics fields](#byok-metrics-fields). | +| `weekly_active_local_byok_users` | `integer` | Yes | Number of unique users who made BYOK requests from Sunday through the report day, inclusive. This calendar-to-date count is not a sum of daily counts. | +| `monthly_active_local_byok_users` | `integer` | Yes | Number of unique users who made BYOK requests from the first day of the calendar month through the report day, inclusive. This calendar-to-date count is not a sum of daily counts. | | `daily_active_vscode_agent_users` | `integer` | Yes | Number of unique users who used the dedicated {% data variables.product.prodname_vscode_shortname %} Agents window on a given day. Omitted when no dedicated Agents-window data is available for the enterprise or organization. | Activity totals and breakdowns: @@ -203,6 +208,7 @@ Activity totals and breakdowns: | `totals_by_model_feature` | `array` | No | Aggregated model-and-feature activity breakdown. See [Activity breakdown objects](#activity-breakdown-objects). | | `totals_by_cli` | `object` | Yes | Aggregated {% data variables.copilot.copilot_cli_short %} metrics for the day. Omitted when there is no CLI usage that day. Unlike the per-user form, it does not include `last_known_cli_version`. See [{% data variables.copilot.copilot_cli_short %} metrics fields](#copilot-cli-metrics-fields). | | `totals_by_copilot_app` | `object` | Yes | Aggregated daily metrics for the {% data variables.copilot.github_copilot_app_short %}. Null when the enterprise or organization has no activity in the {% data variables.copilot.github_copilot_app_short %} that day. See [Metrics fields for the {% data variables.copilot.github_copilot_app_short %}](#copilot-app-metrics-fields). | +| `totals_by_local_byok` | `array` | Yes | Aggregated BYOK usage for the day, with one entry for each BYOK kind and surface that had activity. This usage is a subset of `totals_by_cli` and `totals_by_copilot_app`. Omitted or null when the enterprise or organization had no BYOK requests that day. See [BYOK metrics fields](#byok-metrics-fields). | | `totals_by_3rd_party_agent` | `array` | Yes | Aggregated per-agent usage metrics for recognized {% data variables.copilot.agent_apps %}. Omitted when the enterprise or organization had no recognized {% data variables.copilot.agent_app %} activity that day. See [{% data variables.copilot.agent_apps_caps %} metrics fields](#agent-apps-metrics-fields). | | `totals_by_client` | `array` | No | Aggregated per-client breakdown of successful model requests and token usage for the day, counted server-side. The array is always present and is empty when there is no usage to report. In 28-day reports, this field appears within each `day_totals` entry. See [Client metrics fields](#client-metrics-fields). | | `totals_by_vscode_agent` | `object` | Yes | Aggregated dedicated {% data variables.product.prodname_vscode_shortname %} Agents-window metrics for the day. Omitted when both required counts are not available. See [{% data variables.product.prodname_vscode_shortname %} Agents-window metrics fields](#vs-code-agents-window-metrics-fields). | @@ -387,6 +393,39 @@ The `totals_by_copilot_app` object contains the following nested fields when usa | `totals_by_copilot_app.token_usage.prompt_tokens_sum` | `integer` | No | Total prompt tokens sent across all requests in the {% data variables.copilot.github_copilot_app_short %} on this day. | | `totals_by_copilot_app.token_usage.avg_tokens_per_request` | `number` | Yes | Average of output and prompt tokens per request in the {% data variables.copilot.github_copilot_app_short %}, computed as `(output_tokens_sum + prompt_tokens_sum) ÷ request_count`. Null when there were no requests that day. | +### BYOK metrics fields + +The bring your own key (BYOK) fields break out usage of user-configured provider keys and local models, such as Ollama, Foundry Local, LM Studio, and llama-server, in {% data variables.copilot.copilot_cli_short %} and the {% data variables.copilot.github_copilot_app_short %}. The fields appear in enterprise, organization, enterprise-user, and organization-user 1-day reports, and in each day of the corresponding 28-day reports. The top-level BYOK fields (`used_local_byok`, `totals_by_local_byok`, and the active local BYOK user counts) are nullable. + +The `totals_by_local_byok` array is a subset of `totals_by_cli` (surface `copilot_cli`) and `totals_by_copilot_app` (surface `copilot_app`), not additional usage. For each surface, the summed `request_count`, `prompt_count`, and token sums are less than or equal to the values in the parent object. BYOK requests have no {% data variables.product.prodname_copilot_short %} usage charges or premium request cost. Your model provider may still charge for hosted-model usage. Values are a lower bound because client telemetry can be disabled, and token sums can undercount because some local providers don't return token usage. + +> [!NOTE] +> Local BYOK and enterprise custom models are reported separately. +> +> * **Local BYOK** is configured by a developer in their client, and requests go directly to the provider. It's reported from client telemetry in the `*_local_byok` fields. {% data variables.copilot.copilot_cli_short %} and the {% data variables.copilot.github_copilot_app_short %} are covered today. IDEs aren't covered yet. +> * **Enterprise custom models** are configured by an enterprise or organization owner in {% data variables.product.github %} settings, and requests are routed through {% data variables.product.github %}. They're reported server-side for every client through `is_custom_enterprise_model` in `totals_by_client[].totals_by_model[]`. See [Client metrics fields](#client-metrics-fields). + +Each entry in the `totals_by_local_byok` array contains the following fields. + +| Field | Type | Nullable | Description | +|:--|:--|:--|:--| +| `byok_kind` | `string` | No | Kind of BYOK usage: `local_managed` (a {% data variables.product.prodname_copilot_short %}-managed local model), `local_user` (a user-run local model server, such as Ollama or LM Studio), `local_auto` (a local model selected through Auto), or `remote_user` (the user's own key for a hosted provider). | +| `surface` | `string` | No | Client surface: `copilot_cli` or `copilot_app`. Treat this as an open set, because more values can be added. | +| `session_count` | `integer` | No | Number of distinct sessions with this BYOK kind on this surface. A session can switch kinds, so don't sum session counts across entries. | +| `request_count` | `integer` | No | Number of model requests with this BYOK kind on this surface. | +| `prompt_count` | `integer` | Yes | Number of reported user prompts. A value of `0` can mean that the client did not report prompts. | +| `token_usage.output_tokens_sum` | `integer` | No | Total output tokens across those requests. | +| `token_usage.prompt_tokens_sum` | `integer` | No | Total prompt tokens across those requests. | +| `token_usage.avg_tokens_per_request` | `number` | Yes | Average of output and prompt tokens per request, computed as `(output_tokens_sum + prompt_tokens_sum) ÷ request_count`. When combining entries, recompute this value from the summed totals instead of averaging the averages. | + +#### Comparing BYOK and {% data variables.product.prodname_copilot_short %}-hosted usage + +Per-user reports put BYOK usage next to the activity fields you already use. You can group users by how much of their usage is local BYOK (`totals_by_local_byok`) or enterprise custom models (`is_custom_enterprise_model` in `totals_by_client`). Then compare those groups on fields such as `loc_added_sum`, `ai_credits_used`, and `ai_adoption_phase`. + +Pull request fields are aggregated rather than per user. To compare pull request throughput or merge rate, join on `user_login` with pull request data from [AUTOTITLE](/rest/pulls/pulls). + +These comparisons are observational. Users can mix models within a day, and outcomes aren't attributed to a specific model. + ### {% data variables.product.prodname_vscode_shortname %} Agents-window metrics fields The dedicated {% data variables.product.prodname_vscode_shortname %} Agents-window fields appear in enterprise, organization, enterprise-user, and organization-user 1-day and 28-day reports. In aggregated 28-day reports, they appear within each applicable `day_totals` entry, and weekly and monthly counts are calculated relative to the `day` of each entry. In per-user 28-day reports, they appear at the top level of each user record. @@ -441,10 +480,11 @@ The `totals_by_client` array appears at the top level of each record in per-user | `totals_by_client[].token_usage.prompt_tokens_sum` | `integer` | No | Total number of prompt tokens sent across the client's requests. | | `totals_by_client[].token_usage.output_tokens_sum` | `integer` | No | Total number of output tokens generated across the client's requests. | | `totals_by_client[].token_usage.avg_tokens_per_request` | `number` | No | Average number of **output** and **prompt** tokens per request, computed as `(output_tokens_sum + prompt_tokens_sum) ÷ request_count`. | -| `totals_by_client[].totals_by_model` | `array` | No | Per-model breakdown of the client's requests and token usage. Entries are keyed on `model` and `model_selection_mode`, so the same model can appear more than once with different selection modes. | -| `totals_by_client[].totals_by_model[].model` | `string` | No | Model used for the requests. For `code_review`, the model is always reported as `unknown`. | +| `totals_by_client[].totals_by_model` | `array` | No | Per-model breakdown of the client's requests and token usage. Entries are keyed on `model`, `model_selection_mode`, and `is_custom_enterprise_model`, so the same model can appear more than once. For example, a model used both as an enterprise custom model and as a {% data variables.product.prodname_copilot_short %}-hosted model has two entries. | +| `totals_by_client[].totals_by_model[].model` | `string` | No | Model used for the requests. For `code_review`, the model is always reported as `unknown`. Enterprise custom model names that can't be matched to a known model are also reported as `unknown`. | | `totals_by_client[].totals_by_model[].model_selection_mode` | `string` | No | How the model was selected. Possible values are `user_selected` (a specific model was requested rather than Auto), `product_selected` (the product chose the model, used for all {% data variables.copilot.copilot_code-review_short %} and {% data variables.copilot.copilot_cloud_agent %} requests), `auto` (Auto chose the model), `other` (selection modes that are grouped together and not yet reported individually), and `unknown` (a selection mode sent by the client that couldn't be normalized). | -| `totals_by_client[].totals_by_model[].request_count` | `integer` | No | Number of successful model requests from the client that used this model and selection mode. | +| `totals_by_client[].totals_by_model[].is_custom_enterprise_model` | `boolean` | Yes | Whether the requests used an enterprise custom model. `true` for custom models that an enterprise or organization owner configured in {% data variables.product.github %} settings. `false` for {% data variables.product.prodname_copilot_short %}-hosted models, including when no custom model signal is available. Determined server-side. Omitted for report days before this field was introduced. See [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/enable-custom-models). | +| `totals_by_client[].totals_by_model[].request_count` | `integer` | No | Number of successful model requests from the client that used this model, selection mode, and custom model status. | | `totals_by_client[].totals_by_model[].token_usage.prompt_tokens_sum` | `integer` | No | Total number of prompt tokens sent across these requests. | | `totals_by_client[].totals_by_model[].token_usage.output_tokens_sum` | `integer` | No | Total number of output tokens generated across these requests. | | `totals_by_client[].totals_by_model[].token_usage.avg_tokens_per_request` | `number` | No | Average number of **output** and **prompt** tokens per request, computed as `(output_tokens_sum + prompt_tokens_sum) ÷ request_count`. | diff --git a/content/copilot/reference/copilot-usage-metrics/example-schema.md b/content/copilot/reference/copilot-usage-metrics/example-schema.md index b393edb74501..d38ac6456161 100644 --- a/content/copilot/reference/copilot-usage-metrics/example-schema.md +++ b/content/copilot/reference/copilot-usage-metrics/example-schema.md @@ -64,6 +64,7 @@ The following examples show schemas for user-, enterprise-, and repository-level "totals_by_model": [{ "model": "claude-opus-5.5", "model_selection_mode": "user_selected", + "is_custom_enterprise_model": false, "request_count": 2, "token_usage": { "avg_tokens_per_request": 1200.0, @@ -73,6 +74,7 @@ The following examples show schemas for user-, enterprise-, and repository-level }, { "model": "claude-sonnet-5", "model_selection_mode": "auto", + "is_custom_enterprise_model": false, "request_count": 1, "token_usage": { "avg_tokens_per_request": 900.0, @@ -82,22 +84,33 @@ The following examples show schemas for user-, enterprise-, and repository-level }] }, { "client": "copilot_cli", - "request_count": 1, + "request_count": 2, "session_count": 1, "token_usage": { - "avg_tokens_per_request": 600.0, - "output_tokens_sum": 100, - "prompt_tokens_sum": 500 + "avg_tokens_per_request": 900.0, + "output_tokens_sum": 300, + "prompt_tokens_sum": 1500 }, "totals_by_model": [{ "model": "gpt-5.6-sol", "model_selection_mode": "user_selected", + "is_custom_enterprise_model": false, "request_count": 1, "token_usage": { "avg_tokens_per_request": 600.0, "output_tokens_sum": 100, "prompt_tokens_sum": 500 } + }, { + "model": "gpt-5.6-sol", + "model_selection_mode": "user_selected", + "is_custom_enterprise_model": true, + "request_count": 1, + "token_usage": { + "avg_tokens_per_request": 1200.0, + "output_tokens_sum": 200, + "prompt_tokens_sum": 1000 + } }] }], "totals_by_copilot_app": { @@ -184,6 +197,18 @@ The following examples show schemas for user-, enterprise-, and repository-level "loc_suggested_to_delete_sum": 6 }], "totals_by_language_model": [], + "totals_by_local_byok": [{ + "byok_kind": "local_managed", + "prompt_count": 2, + "request_count": 2, + "session_count": 2, + "surface": "copilot_cli", + "token_usage": { + "avg_tokens_per_request": 4200.0, + "output_tokens_sum": 4800, + "prompt_tokens_sum": 3600 + } + }], "totals_by_mcp": [{ "interaction_count": 8, "mcp": "github-mcp-server" @@ -225,6 +250,7 @@ The following examples show schemas for user-, enterprise-, and repository-level "used_copilot_code_review_active": null, "used_copilot_code_review_passive": null, "used_copilot_coding_agent": false, + "used_local_byok": true, "used_vscode_agent": true, "user_id": 1, "user_login": "login1", @@ -242,7 +268,7 @@ Customizations with unavailable names or names that look like file paths remain For coverage and naming rules, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#customization-fields-api-only). -The `totals_by_client` array appears at the top level of each user record in both 1-day and 28-day per-user reports, for both enterprise and organization scopes. It counts successful model requests server-side, so it can include clients, such as OpenCode, that don't appear in `totals_by_ide`. +The `totals_by_client` array appears at the top level of each user record in both 1-day and 28-day per-user reports, for both enterprise and organization scopes. It counts successful model requests server-side, so it can include clients, such as OpenCode, that don't appear in `totals_by_ide`. Each `totals_by_model` entry includes `is_custom_enterprise_model`. In this example, `gpt-5.6-sol` appears twice for `copilot_cli` because the user ran it both as a {% data variables.product.prodname_copilot_short %}-hosted model and as an enterprise custom model. Local BYOK requests don't go through {% data variables.product.github %}, so they appear in `totals_by_local_byok` rather than `totals_by_client`. In this example, `used_copilot_code_review_active` and `used_copilot_code_review_passive` are `null` because there was no {% data variables.copilot.copilot_code-review_short %} signal for the user that day. `used_copilot_cloud_agent` and `used_copilot_coding_agent` always carry the same value. @@ -257,6 +283,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "daily_active_copilot_app_users" : 1, "daily_active_copilot_cloud_agent_users" : 1, "daily_active_copilot_code_review_users" : 1, + "daily_active_local_byok_users" : 2, "daily_active_users" : 2, "daily_active_vscode_agent_users" : 2, "daily_passive_copilot_code_review_users" : 0, @@ -275,6 +302,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "monthly_active_chat_users" : 0, "monthly_active_copilot_cloud_agent_users" : 1, "monthly_active_copilot_code_review_users" : 1, + "monthly_active_local_byok_users" : 2, "monthly_active_users" : 2, "monthly_active_vscode_agent_users" : 2, "monthly_passive_copilot_code_review_users" : 0, @@ -362,6 +390,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "totals_by_model" : [ { "model" : "claude-opus-5.5", "model_selection_mode" : "user_selected", + "is_custom_enterprise_model" : false, "request_count" : 2, "token_usage" : { "avg_tokens_per_request" : 1200.0, @@ -371,6 +400,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review }, { "model" : "claude-sonnet-5", "model_selection_mode" : "auto", + "is_custom_enterprise_model" : false, "request_count" : 1, "token_usage" : { "avg_tokens_per_request" : 900.0, @@ -390,6 +420,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "totals_by_model" : [ { "model" : "gpt-5.6-sol", "model_selection_mode" : "user_selected", + "is_custom_enterprise_model" : false, "request_count" : 1, "token_usage" : { "avg_tokens_per_request" : 600.0, @@ -464,6 +495,29 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "loc_suggested_to_delete_sum" : 6 } ], "totals_by_language_model" : [ ], + "totals_by_local_byok" : [ { + "byok_kind" : "local_managed", + "prompt_count" : 2, + "request_count" : 2, + "session_count" : 2, + "surface" : "copilot_cli", + "token_usage" : { + "avg_tokens_per_request" : 4900.0, + "output_tokens_sum" : 5500, + "prompt_tokens_sum" : 4300 + } + }, { + "byok_kind" : "remote_user", + "prompt_count" : 1, + "request_count" : 1, + "session_count" : 1, + "surface" : "copilot_cli", + "token_usage" : { + "avg_tokens_per_request" : 2500.0, + "output_tokens_sum" : 1500, + "prompt_tokens_sum" : 1000 + } + } ], "totals_by_mcp" : [ { "interaction_count" : 12, "mcp" : "github-mcp-server" @@ -494,6 +548,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "user_initiated_interaction_count" : 1, "weekly_active_copilot_cloud_agent_users" : 1, "weekly_active_copilot_code_review_users" : 1, + "weekly_active_local_byok_users" : 2, "weekly_active_users" : 2, "weekly_active_vscode_agent_users" : 2, "weekly_passive_copilot_code_review_users" : 0 @@ -532,6 +587,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "code_generation_activity_count" : 3, "daily_active_copilot_cloud_agent_users" : 0, "daily_active_copilot_code_review_users" : 0, + "daily_active_local_byok_users" : 0, "daily_active_users" : 2, "daily_passive_copilot_code_review_users" : 0, "day" : "2025-10-01", @@ -549,6 +605,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "monthly_active_chat_users" : 0, "monthly_active_copilot_cloud_agent_users" : 0, "monthly_active_copilot_code_review_users" : 0, + "monthly_active_local_byok_users" : 0, "monthly_active_users" : 2, "monthly_passive_copilot_code_review_users" : 0, "pull_requests" : { @@ -611,6 +668,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "totals_by_model" : [ { "model" : "gpt-5.6-sol", "model_selection_mode" : "user_selected", + "is_custom_enterprise_model" : false, "request_count" : 2, "token_usage" : { "avg_tokens_per_request" : 1000.0, @@ -630,6 +688,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "totals_by_model" : [ { "model" : "unknown", "model_selection_mode" : "product_selected", + "is_custom_enterprise_model" : false, "request_count" : 2, "token_usage" : { "avg_tokens_per_request" : 800.0, @@ -688,6 +747,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "loc_suggested_to_delete_sum" : 3 } ], "totals_by_language_model" : [ ], + "totals_by_local_byok" : null, "totals_by_mcp" : [ ], "totals_by_model_feature" : [ ], "totals_by_plugin" : [ ], @@ -696,6 +756,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "user_initiated_interaction_count" : 1, "weekly_active_copilot_cloud_agent_users" : 0, "weekly_active_copilot_code_review_users" : 0, + "weekly_active_local_byok_users" : 0, "weekly_active_users" : 2, "weekly_passive_copilot_code_review_users" : 0 } ], From 35216638332a8b774db0e8717adf8d815e680a2b Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 5 Nov 2026 22:52:11 +0000 Subject: [PATCH 04/12] Update documentation for recovering history after force push (#64254) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: bot-digital-customer-success <121182491+bot-digital-customer-success@users.noreply.github.com> Co-authored-by: chiph <27255831+chiph@users.noreply.github.com> Co-authored-by: Daniel Klemm Co-authored-by: Siara <108543037+SiaraMist@users.noreply.github.com> --- .../commit-changes/troubleshooting-commits.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/content/pull-requests/how-tos/commit-changes/troubleshooting-commits.md b/content/pull-requests/how-tos/commit-changes/troubleshooting-commits.md index 3b7432df44b4..2f651f663522 100644 --- a/content/pull-requests/how-tos/commit-changes/troubleshooting-commits.md +++ b/content/pull-requests/how-tos/commit-changes/troubleshooting-commits.md @@ -70,7 +70,17 @@ Use `git fetch upstream` for a fork's upstream repository, or `git fetch origin` ### The branch that contained the commit was deleted -If the branch was deleted or force pushed, ask a collaborator who still has the commit locally to push it to a new branch. +If a force push overwrote branch history, you may be able to recover it on {% data variables.product.github %}, including in a private repository. You need write access to create a recovery branch. This workflow only works if the previous commit still opens on {% data variables.product.github %}. + +1. Open the repository's activity view. See [AUTOTITLE](/repositories/viewing-activity-and-data-for-your-repository/using-the-activity-view-to-see-changes-to-a-repository). +1. Use the branch dropdown to select the affected branch. In the **All activity** dropdown, select **Force pushes**. +1. Find the force-push event that overwrote the history. If available, you can select **Compare changes** from the event's {% octicon "kebab-horizontal" aria-label="The horizontal kebab icon" %} menu to view the comparison. +1. If you choose to view the comparison, copy the base commit SHA, the first SHA in the comparison URL. Otherwise, use the event's **before** commit SHA, which was the branch tip before the force push, not its **after** SHA. +1. Open `https://{% data variables.product.product_url %}/OWNER/REPO/tree/BEFORE_SHA`, replacing `OWNER` and `REPO` with the repository owner and name, and `BEFORE_SHA` with the previous commit SHA.{% ifversion ghes %} Replace `HOSTNAME` with your instance's hostname.{% endif %} +1. If the commit opens, select the branch dropdown above the file list. Type a new recovery-branch name, such as `recover-history`, then select **Create branch**. +1. On the new branch, check the files and commit history to verify that the expected history is present. + +If you cannot find or open the SHA, or if the branch was deleted, ask a collaborator who still has the commit in a local clone to push it to a new branch. ```shell git branch recover-B B From 6abed3d818a57eeb80b1a9275b96a5446cb16d23 Mon Sep 17 00:00:00 2001 From: Kranthi Kiran Date: Fri, 6 Nov 2026 06:47:01 +0000 Subject: [PATCH 05/12] Document that empty job outputs aren't set and that jobs not re-run keep their outputs (#64506) Co-authored-by: Kranthi A Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: mc <42146119+mchammer01@users.noreply.github.com> Copilot-Session: b729a193-1aea-40ef-97c7-b4e495668513 Copilot-Session: 14a18908-da88-49b0-9935-ee0ca4a72392 --- .../manage-workflow-runs/re-run-workflows-and-jobs.md | 2 ++ .../actions/jobs/section-defining-outputs-for-jobs.md | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/content/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs.md b/content/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs.md index 6b8e35527446..84d4386dda25 100644 --- a/content/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs.md +++ b/content/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs.md @@ -21,6 +21,8 @@ Re-runs use the privileges of the actor who initially triggered the workflow, no A workflow run can be re-run a maximum of 50 times. This limit includes both full re-runs and re-runs of a subset of jobs. +When you re-run failed jobs or a specific job, {% data variables.product.github %} also re-runs the jobs that depend on them. Jobs that aren't included keep their outputs from the previous attempt, which the newly run jobs can use. + You can't re-run a workflow run, or any of its jobs, while the workflow is disabled. You need to enable the workflow first. See [AUTOTITLE](/actions/how-tos/manage-workflow-runs/disable-and-enable-workflows). ## Re-running all the jobs in a workflow diff --git a/data/reusables/actions/jobs/section-defining-outputs-for-jobs.md b/data/reusables/actions/jobs/section-defining-outputs-for-jobs.md index 918dcd08f2f1..567afb0f4a5e 100644 --- a/data/reusables/actions/jobs/section-defining-outputs-for-jobs.md +++ b/data/reusables/actions/jobs/section-defining-outputs-for-jobs.md @@ -4,7 +4,9 @@ You can use `jobs..outputs` to create a `map` of outputs for a job. Job Job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to {% data variables.product.prodname_actions %}. -If an output is skipped because it may contain a secret, you will see the following warning message: "Skip output `{output.Key}` since it may contain secret." For more information on how to handle secrets, please refer to the [Example: Masking and passing a secret between jobs or workflows](/actions/reference/workflows-and-actions/workflow-commands#example-masking-and-passing-a-secret-between-jobs-or-workflows). +If an output is skipped because it may contain a secret, you will see this warning message, where OUTPUT-NAME is the name of the output: `Skip output 'OUTPUT-NAME' since it may contain secret`. For more information on how to handle secrets, see [AUTOTITLE](/actions/reference/workflows-and-actions/workflow-commands#example-masking-and-passing-a-secret-between-jobs-or-workflows). + +If an output evaluates to an empty string, the output is not set. If step debug logging is enabled, the job log shows this debug message, where OUTPUT-NAME is the name of the output: `Skip output 'OUTPUT-NAME' since it's empty`. For more information, see [AUTOTITLE](/actions/how-tos/monitor-workflows/enable-debug-logging#enabling-step-debug-logging). To use job outputs in a dependent job, you can use the `needs` context. For more information, see [AUTOTITLE](/actions/reference/workflows-and-actions/contexts#needs-context). @@ -76,4 +78,4 @@ jobs: {% endraw %} > [!WARNING] -> Actions does not guarantee the order that matrix jobs will run in. Ensure that the output name is unique, otherwise the last matrix job that runs will override the output value. +> {% data variables.product.prodname_actions %} does not guarantee the order that matrix jobs will run in. Ensure that the output name is unique, otherwise the last matrix job that runs and sets the output will override the output value. An output with an empty value is not set, so it does not override a value set by another matrix job. From 66cf7e967d7659198dad771fa47a80717cceb4a9 Mon Sep 17 00:00:00 2001 From: bonsohi <31749534+bonsohi@users.noreply.github.com> Date: Fri, 6 Nov 2026 08:15:43 +0000 Subject: [PATCH 06/12] Revise prerequisites for GHES backup restoration process (#64484) Co-authored-by: docs-bot <77750099+docs-bot@users.noreply.github.com> Co-authored-by: mc <42146119+mchammer01@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- .../configuring-backups-on-your-instance.md | 2 +- .../restoring-from-a-backup.md | 16 +++++++--------- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/content/admin/backing-up-and-restoring-your-instance/configuring-backups-on-your-instance.md b/content/admin/backing-up-and-restoring-your-instance/configuring-backups-on-your-instance.md index 6b0cf750ffb1..79bb6cf907a8 100644 --- a/content/admin/backing-up-and-restoring-your-instance/configuring-backups-on-your-instance.md +++ b/content/admin/backing-up-and-restoring-your-instance/configuring-backups-on-your-instance.md @@ -159,7 +159,7 @@ Network settings are excluded from the backup snapshot. After restoration, you m ### Prerequisites 1. Ensure maintenance mode is enabled on the primary instance and all active processes have completed. For more information, see [AUTOTITLE](/admin/administering-your-instance/configuring-maintenance-mode/enabling-and-scheduling-maintenance-mode). -1. Stop replication on all replica nodes in a high-availability configuration. For more information, see [AUTOTITLE](/admin/monitoring-and-managing-your-instance/configuring-high-availability/about-high-availability-configuration#ghe-repl-stop). +1. Tear down replication on all replica nodes in a high-availability configuration. For more information, see [AUTOTITLE](/admin/monitoring-and-managing-your-instance/configuring-high-availability/removing-a-high-availability-replica). 1. Provision a new {% data variables.product.prodname_ghe_server %} instance to use as a target for the restoration of your backup. For more information, see [AUTOTITLE](/admin/installing-your-enterprise-server/setting-up-a-github-enterprise-server-instance). 1. If {% data variables.location.product_location %} has {% data variables.product.prodname_actions %} enabled, you must configure the external storage provider for {% data variables.product.prodname_actions %} on the replacement instance. For more information, see [AUTOTITLE](/admin/managing-github-actions-for-your-enterprise/advanced-configuration-and-troubleshooting/backing-up-and-restoring-github-enterprise-server-with-github-actions-enabled). diff --git a/content/admin/backing-up-and-restoring-your-instance/restoring-from-a-backup.md b/content/admin/backing-up-and-restoring-your-instance/restoring-from-a-backup.md index 841eafcf1c41..0fb8beca092e 100644 --- a/content/admin/backing-up-and-restoring-your-instance/restoring-from-a-backup.md +++ b/content/admin/backing-up-and-restoring-your-instance/restoring-from-a-backup.md @@ -30,10 +30,11 @@ You also can’t restore from a newer version to an older one. For example, tryi Before restoring a backup: -1. **Enable maintenance mode** on the target instance. See [AUTOTITLE](/admin/administering-your-instance/configuring-maintenance-mode/enabling-and-scheduling-maintenance-mode). -1. **Verify access** to the backup storage containing the snapshot. +1. **Provision a new {% data variables.product.prodname_ghe_server %} instance** to use as a target for the restoration of your backup. For more information, see [AUTOTITLE](/admin/installing-your-enterprise-server/setting-up-a-github-enterprise-server-instance). 1. **Tear down replication**. If using High Availability (HA), run `ghe-repl-teardown` on the target instance. Stopping replication with `ghe-repl-stop` is not sufficient, because the target remains configured as part of a replication pair and the restore fails. See [AUTOTITLE](/admin/monitoring-and-managing-your-instance/configuring-high-availability/removing-a-high-availability-replica). -1. **Prepare for {% data variables.product.prodname_actions %}** — if enabled, ensure the target instance is configured with the correct external storage. See [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/restoring-with-github-actions-enabled) for details. +1. **Prepare for {% data variables.product.prodname_actions %}**: if enabled, ensure the target instance is configured with the correct external storage. See [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/restoring-with-github-actions-enabled) for details. +1. **Enable maintenance mode** on the target instance. See [AUTOTITLE](/admin/administering-your-instance/configuring-maintenance-mode/enabling-and-scheduling-maintenance-mode). +1. **Verify access** to the backup storage containing the snapshot after attaching and mounting the backup disk volume to the target instance. See [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/configuring-the-backup-service#reusing-a-previously-initialized-disk). ## Starting the restore operation @@ -41,7 +42,6 @@ To restore from a snapshot: 1. SSH into the target instance as the `admin` user. 1. Run one of the following commands: - * Restore the latest snapshot: ```shell @@ -57,19 +57,17 @@ To restore from a snapshot: * (Optional) Overwrite appliance configuration, certificates, and license data from the backup. The `-c` option does not control database or repository restoration: ```shell - ghe-restore -c # Latest snapshot - ghe-restore -s -c # Specific snapshot + ghe-restore -c # Latest snapshot + ghe-restore -s -c # Specific snapshot ``` 1. **Finalize in {% data variables.enterprise.management_console %}:** - * Review all configuration settings (network, auth, TLS, etc.). * Click **Save settings** to apply them and start services. * The instance is not fully operational until this step is complete. - 1. **Validate the restored instance** to ensure everything works as expected. + * (Optional) Configure an IP exception list on the restored instance to allow access to a specified list of IP addresses. For more information, see [AUTOTITLE](/admin/administering-your-instance/configuring-maintenance-mode/enabling-and-scheduling-maintenance-mode#validating-changes-in-maintenance-mode-using-the-ip-exception-list). 1. **If using HA**, complete the restore on a standalone instance first. Then reconfigure HA. - * If you run into sync issues (e.g., stale UUIDs in `ghe-repl-status`), run `ghe-repl-teardown`. * For help, contact {% data variables.contact.github_support %}. From aaec457c1986250d31d07e73c3113d1837291d14 Mon Sep 17 00:00:00 2001 From: Eric Hollenberry Date: Fri, 6 Nov 2026 11:14:36 +0000 Subject: [PATCH 07/12] Copilot code review outcomes in repos-1-day reports (#64529) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Sophie <29382425+sophietheking@users.noreply.github.com> --- .../copilot-usage-metrics.md | 43 +++++- .../copilot-usage-metrics/example-schema.md | 134 +++++++++++++++++- 2 files changed, 172 insertions(+), 5 deletions(-) diff --git a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md index bb6e9a437dc6..f02195591615 100644 --- a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md +++ b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md @@ -260,7 +260,9 @@ Teams with fewer than 5 seated {% data variables.product.prodname_copilot_short ### Repository-level fields (API only) -Repository-level reports show daily pull request creation, review, merge, and suggestion activity, plus pull request review stage durations, for repositories in an organization or enterprise, including pull requests created by {% data variables.copilot.copilot_cloud_agent %} and reviewed by {% data variables.copilot.copilot_code-review_short %}. Each row represents one repository that had pull request activity on the requested day; repositories with no activity are omitted. These reports focus on pull request lifecycle activity and can contain data even when IDE usage metrics are absent. +Repository-level reports cover repositories in an organization or enterprise. They show daily pull request creation, review, merge, and suggestion activity. They also show pull request review stage durations and {% data variables.copilot.copilot_code-review_short %} outcomes. Reports include pull requests created by {% data variables.copilot.copilot_cloud_agent %} and reviewed by {% data variables.copilot.copilot_code-review_short %}. + +Each row represents one repository that had pull request activity on the requested day; repositories with no activity are omitted. These reports focus on pull request lifecycle activity and can contain data even when IDE usage metrics are absent. These fields are returned by the daily repository-level report (`repos-1-day`), available via the REST API at the organization and enterprise scopes. For the endpoint URLs, response envelope, permissions, and download workflow, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics). For example rows, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/example-schema#repository-level-schema-example). @@ -275,6 +277,7 @@ These fields are returned by the daily repository-level report (`repos-1-day`), | `repo_visibility` | `string` | No | Repository visibility. Possible values are `PRIVATE`, `INTERNAL`, and `PUBLIC`. | | `pull_requests` | `object` | No | Daily pull request activity for the repository. See [Pull request activity fields](#pull-request-activity-fields). | | `pull_request_review_times` | `array` | No | Review stage durations for qualifying pull requests merged in the repository on this day, with one entry per author and reviewer type. Empty when no qualifying pull requests were merged. See [Pull request review time fields](#pull-request-review-time-fields). | +| `copilot_code_review` | `object` | No | Outcomes of {% data variables.copilot.copilot_code-review_short %} comments on pull requests merged in the repository on this day. Present on every row; counts are `0` and arrays are empty when there was no {% data variables.copilot.copilot_code-review_short %} activity. Not included in reports generated before this field was released. See [{% data variables.copilot.copilot_code-review_short %} outcome fields](#copilot-code-review-outcome-fields). | ### Activity breakdown objects @@ -553,6 +556,44 @@ Exclusions are not reported for individual pull requests. Because `pull_requests Medians and 90th percentiles are exact values calculated with linear interpolation and rounded to two decimal places. +### {% data variables.copilot.copilot_code-review_short %} outcome fields + +The `copilot_code_review` object appears only in repository-level reports. It shows whether developers acted on {% data variables.copilot.copilot_code-review_short %} comments, and whether people still requested changes after {% data variables.copilot.copilot_code-review_short %} reviewed a pull request. + +#### How comments are counted + +* Only comments that {% data variables.copilot.copilot_code-review_short %} posts on pull requests count. Comments from people or other review tools, and {% data variables.copilot.copilot_code-review_short %} reviews in the IDE, are not included. +* Comments are evaluated when their pull request merges. Each comment counts on the day its pull request merged, not the day {% data variables.copilot.copilot_code-review_short %} posted it. +* {% data variables.copilot.copilot_code-review_short %} usually does not evaluate comments on pull requests opened by bots, such as {% data variables.copilot.copilot_cloud_agent %}. +* A comment is addressed when the developer takes any of these actions: + * Applies the suggested change + * Asks {% data variables.copilot.copilot_code-review_short %} to make the fix + * Edits the code by hand + * Replies that the fix will come in a later pull request + +Comments that were ignored, or that {% data variables.copilot.copilot_code-review_short %} later found to be wrong, count as evaluated but not addressed. + +These counts differ from the suggestion counts in `pull_requests`. Suggestion counts include only comments with a ready-to-apply code suggestion. A suggestion counts as applied only when someone applies it. The `copilot_code_review` counts include every evaluated comment, regardless of whether or how the developer acted on it. + +#### Breakdown arrays + +Each `review_comments_by_*` array splits the evaluated and addressed comments by one attribute. A value gets an entry only if at least one comment with that value was evaluated that day. + +* Each comment counts once in the severity, review tier, and comment type arrays. A comment whose severity, review tier, or type was not recorded counts only in the totals, so these arrays can add up to less than the totals. +* A comment that drew on more than one customization counts once for each, so `review_comments_by_customization` can add up to more than the totals. + +To calculate the addressed rate, divide `total_comments_addressed` by `total_comments_evaluated`. + +| Field | Type | Nullable | Description | +|:--|:--|:--|:--| +| `copilot_code_review.total_comments_evaluated` | `integer` | No | Number of {% data variables.copilot.copilot_code-review_short %} comments on pull requests merged on this day that were checked to see whether the developer acted on them. | +| `copilot_code_review.total_comments_addressed` | `integer` | No | Number of evaluated comments that the developer acted on. | +| `copilot_code_review.review_comments_by_severity` | `array` | No | Evaluated and addressed comments by severity. Each entry includes `severity` (`high`, `medium`, or `low`), `total_comments_evaluated`, and `total_comments_addressed`. Always present but can be empty. | +| `copilot_code_review.review_comments_by_review_tier` | `array` | No | Evaluated and addressed comments by the tier of the review that posted them. Each entry includes `review_tier` (`lite`, `balanced`, or `max`), `total_comments_evaluated`, and `total_comments_addressed`. Always present but can be empty. | +| `copilot_code_review.review_comments_by_comment_type` | `array` | No | Evaluated and addressed comments by the kind of issue raised, for example `bug`, `security`, `performance`, `accessibility`, or `documentation`. `custom` means the comment is about one of the repository's own coding guidelines. `other` means the comment did not fit any of the predefined types. Each entry includes `comment_type`, `total_comments_evaluated`, and `total_comments_addressed`. New types can appear over time. Unlike `pull_requests.copilot_suggestions_by_comment_type`, this array is not limited to the top 10 types and has no `others` entry. Always present but can be empty. | +| `copilot_code_review.review_comments_by_customization` | `array` | No | Evaluated and addressed comments by what {% data variables.copilot.copilot_code-review_short %} drew on to write them. Each entry includes `customization` (`custom_instruction`, `skill`, `mcp_server`, or `none`), `total_comments_evaluated`, and `total_comments_addressed`. Always present but can be empty. | +| `copilot_code_review.total_merged_no_human_change_request_after_review` | `integer` | No | Number of pull requests merged on this day that {% data variables.copilot.copilot_code-review_short %} reviewed, where no person requested changes after {% data variables.copilot.copilot_code-review_short %}'s first review.

A change request after the first review excludes the pull request, even if the request was later dismissed or the pull request was approved. A change request made before {% data variables.copilot.copilot_code-review_short %}'s first review does not. Includes pull requests opened by bots, such as {% data variables.copilot.copilot_cloud_agent %}. Never higher than `pull_requests.total_merged_reviewed_by_copilot`. | + ### AI adoption phase fields {% data variables.product.prodname_copilot_short %} groups users into AI adoption phases based on their activity. Phase information appears in two places: the per-user `ai_adoption_phase` object, and the aggregated `totals_by_ai_adoption_phase` array. For the phase values, see [Breakdown dimension values](#breakdown-dimension-values). diff --git a/content/copilot/reference/copilot-usage-metrics/example-schema.md b/content/copilot/reference/copilot-usage-metrics/example-schema.md index d38ac6456161..bcc757316646 100644 --- a/content/copilot/reference/copilot-usage-metrics/example-schema.md +++ b/content/copilot/reference/copilot-usage-metrics/example-schema.md @@ -872,6 +872,8 @@ The following example shows a repository-level report from an NDJSON file downlo Both enterprise- and organization-scoped rows include `organization_id`, which identifies the organization that owns the repository. Enterprise-scoped rows also include `enterprise_id`. Organization-scoped rows include `enterprise_id` only when an enterprise owns the organization. +The `copilot_code_review` object is present on every row. Its counts are `0` and its arrays are empty when the repository had no {% data variables.copilot.copilot_code-review_short %} activity that day. For how comments are counted, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#copilot-code-review-outcome-fields). + The `pull_request_review_times` array is empty when the repository merged no qualifying pull requests that day. For which pull requests are counted, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#pull-request-review-time-fields). For the field reference, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#repository-level-fields-api-only). @@ -903,7 +905,16 @@ For the field reference, see [AUTOTITLE](/copilot/reference/copilot-usage-metric "median_minutes_to_merge_copilot_reviewed": 372.62, "copilot_suggestions_by_comment_type": [] }, - "pull_request_review_times": [] + "pull_request_review_times": [], + "copilot_code_review": { + "total_comments_evaluated": 0, + "total_comments_addressed": 0, + "review_comments_by_severity": [], + "review_comments_by_review_tier": [], + "review_comments_by_comment_type": [], + "review_comments_by_customization": [], + "total_merged_no_human_change_request_after_review": 1 + } }, { "day": "2026-07-14", @@ -935,7 +946,55 @@ For the field reference, see [AUTOTITLE](/copilot/reference/copilot-usage-metric } ] }, - "pull_request_review_times": [] + "pull_request_review_times": [], + "copilot_code_review": { + "total_comments_evaluated": 3, + "total_comments_addressed": 2, + "review_comments_by_severity": [ + { + "severity": "high", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + }, + { + "severity": "medium", + "total_comments_evaluated": 2, + "total_comments_addressed": 1 + } + ], + "review_comments_by_review_tier": [ + { + "review_tier": "balanced", + "total_comments_evaluated": 3, + "total_comments_addressed": 2 + } + ], + "review_comments_by_comment_type": [ + { + "comment_type": "bug", + "total_comments_evaluated": 2, + "total_comments_addressed": 1 + }, + { + "comment_type": "documentation", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + } + ], + "review_comments_by_customization": [ + { + "customization": "none", + "total_comments_evaluated": 2, + "total_comments_addressed": 1 + }, + { + "customization": "custom_instruction", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + } + ], + "total_merged_no_human_change_request_after_review": 1 + } }, { "day": "2026-07-14", @@ -973,7 +1032,65 @@ For the field reference, see [AUTOTITLE](/copilot/reference/copilot-usage-metric } ] }, - "pull_request_review_times": [] + "pull_request_review_times": [], + "copilot_code_review": { + "total_comments_evaluated": 4, + "total_comments_addressed": 3, + "review_comments_by_severity": [ + { + "severity": "high", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + }, + { + "severity": "medium", + "total_comments_evaluated": 2, + "total_comments_addressed": 2 + }, + { + "severity": "low", + "total_comments_evaluated": 1, + "total_comments_addressed": 0 + } + ], + "review_comments_by_review_tier": [ + { + "review_tier": "max", + "total_comments_evaluated": 4, + "total_comments_addressed": 3 + } + ], + "review_comments_by_comment_type": [ + { + "comment_type": "bug", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + }, + { + "comment_type": "custom", + "total_comments_evaluated": 2, + "total_comments_addressed": 1 + }, + { + "comment_type": "security", + "total_comments_evaluated": 1, + "total_comments_addressed": 1 + } + ], + "review_comments_by_customization": [ + { + "customization": "custom_instruction", + "total_comments_evaluated": 2, + "total_comments_addressed": 1 + }, + { + "customization": "none", + "total_comments_evaluated": 2, + "total_comments_addressed": 2 + } + ], + "total_merged_no_human_change_request_after_review": 1 + } }, { "day": "2026-07-14", @@ -1010,7 +1127,16 @@ For the field reference, see [AUTOTITLE](/copilot/reference/copilot-usage-metric "median_minutes_final_review_to_merge": 674.95, "p90_minutes_final_review_to_merge": 1213.66 } - ] + ], + "copilot_code_review": { + "total_comments_evaluated": 0, + "total_comments_addressed": 0, + "review_comments_by_severity": [], + "review_comments_by_review_tier": [], + "review_comments_by_comment_type": [], + "review_comments_by_customization": [], + "total_merged_no_human_change_request_after_review": 0 + } } ] ``` From 087ba79b6025d493406067a7088a83ece272dd99 Mon Sep 17 00:00:00 2001 From: Eric Hollenberry Date: Fri, 6 Nov 2026 13:16:09 +0000 Subject: [PATCH 08/12] Document client telemetry and minimum-version fields in Copilot usage metrics (#64568) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../copilot-usage-metrics/copilot-usage-metrics.md | 7 +++++++ .../copilot-usage-metrics/example-schema.md | 14 ++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md index f02195591615..1483a8861199 100644 --- a/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md +++ b/content/copilot/reference/copilot-usage-metrics/copilot-usage-metrics.md @@ -126,6 +126,11 @@ Per-user reports contain one record per user per day. The 28-day per-user report | `used_copilot_code_review_active` | `boolean` | Yes | Whether the user actively engaged with {% data variables.copilot.copilot_code-review_short %} that day. A user is considered active if they manually requested a {% data variables.product.prodname_copilot_short %} review, or applied a {% data variables.product.prodname_copilot_short %} review suggestion. Null when there is no {% data variables.copilot.copilot_code-review_short %} signal for the user that day. | | `used_copilot_code_review_passive` | `boolean` | Yes | Whether the user had {% data variables.product.prodname_copilot_short %} automatically assigned to review their pull request that day, without actively engaging with the review. Null when there is no {% data variables.copilot.copilot_code-review_short %} signal for the user that day. | | `used_vscode_agent` | `boolean` | Yes | Whether the user used the dedicated {% data variables.product.prodname_vscode_shortname %} Agents window during the reporting period. Omitted when no dedicated Agents-window data is available for the user. | +| `client_telemetry_present` | `boolean` | Yes | Whether qualifying client usage telemetry was received for the user that day. `false` means {% data variables.product.prodname_copilot_short %} recorded the user's activity server-side, but no qualifying client telemetry arrived. Null or omitted when unknown or not evaluated, including for users with only {% data variables.copilot.copilot_cloud_agent %} or {% data variables.copilot.copilot_code-review_short %} activity. `true` doesn't guarantee complete telemetry, and `false` doesn't indicate why telemetry is missing. In 28-day reports, describes the record's `day`. | +| `usage_metrics_requirements` | `object` | Yes | Checks the last-known version of each identifiable IDE and {% data variables.product.prodname_copilot_short %} extension against the supported minimum versions. Each component appears in exactly one of `met`, `unmet`, or `unknown`. Null or omitted when no components can be identified or the record predates this field. This field is independent of `client_telemetry_present`, so a user can meet every minimum and still have `client_telemetry_present: false`. In 28-day reports, describes the record's `day`. For minimum versions, see [AUTOTITLE](/copilot/concepts/billing-and-usage/copilot-usage-metrics/copilot-metrics#supported-ides). | +| `usage_metrics_requirements.met` | `array` | No | Sorted IDE and extension identifiers, such as `vscode` or `vscode/copilot-chat`, whose last-known version meets the minimum. Present, possibly empty, whenever `usage_metrics_requirements` is present. | +| `usage_metrics_requirements.unmet` | `array` | No | Sorted IDE and extension identifiers whose last-known version is below the minimum. Present, possibly empty, whenever `usage_metrics_requirements` is present. | +| `usage_metrics_requirements.unknown` | `array` | No | Sorted IDE and extension identifiers whose version is missing or unrecognized. Present, possibly empty, whenever `usage_metrics_requirements` is present. | | `ai_adoption_phase` | `object` | No | The user's AI adoption phase for the day. Always present; defaults to the "No Cohort" phase. See [AI adoption phase fields](#ai-adoption-phase-fields). | | `distinct_skill_use_count` | `integer` | Yes | Number of different skill identifiers with recorded activity for the user. See [Customization fields](#customization-fields-api-only). | | `distinct_custom_agent_use_count` | `integer` | Yes | Number of different custom agent identifiers with recorded activity for the user. See [Customization fields](#customization-fields-api-only). | @@ -158,6 +163,8 @@ Active-user counts: | Field | Type | Nullable | Description | |:--|:--|:--|:--| | `daily_active_users` | `integer` | No | Number of unique users who used {% data variables.product.prodname_copilot_short %} on a given day. | +| `missing_client_telemetry_user_count` | `integer` | Yes | Number of unique users on a given day whose `client_telemetry_present` is `false`. Users with a null value aren't counted. Enterprise counts are computed independently, not summed from organizations. This is a daily count, so don't sum it across days. `0` is a measured zero. Null or omitted when unavailable, for example for days before this field existed. In 28-day reports, this field appears within each `day_totals` entry. | +| `below_usage_metrics_minimum_user_count` | `integer` | Yes | Number of unique users on a given day with at least one entry in `usage_metrics_requirements.unmet`. Users with only `unknown` entries or no assessment aren't counted. Can overlap with `missing_client_telemetry_user_count`. Enterprise counts are computed independently, not summed from organizations. This is a daily count, so don't sum it across days. `0` is a measured zero. Null or omitted when unavailable, for example for days before this field existed. In 28-day reports, this field appears within each `day_totals` entry. | | `weekly_active_users` | `integer` | No | Number of unique users who used {% data variables.product.prodname_copilot_short %} during a trailing seven-day window. | | `weekly_active_vscode_agent_users` | `integer` | Yes | Number of unique users who used the dedicated {% data variables.product.prodname_vscode_shortname %} Agents window from Sunday through the report day, inclusive. This calendar-to-date count is not a trailing seven-day window. | | `monthly_active_users` | `integer` | No | Number of unique users who used {% data variables.product.prodname_copilot_short %} during a trailing 28-day window. | diff --git a/content/copilot/reference/copilot-usage-metrics/example-schema.md b/content/copilot/reference/copilot-usage-metrics/example-schema.md index bcc757316646..face41037c2e 100644 --- a/content/copilot/reference/copilot-usage-metrics/example-schema.md +++ b/content/copilot/reference/copilot-usage-metrics/example-schema.md @@ -25,6 +25,7 @@ The following examples show schemas for user-, enterprise-, and repository-level "version": "v1" }, "ai_credits_used": 12.5, + "client_telemetry_present": true, "code_acceptance_activity_count": 3, "code_generation_activity_count": 3, "day": "2026-10-19", @@ -242,6 +243,11 @@ The following examples show schemas for user-, enterprise-, and repository-level "interaction_count": 1, "slash_cmd": "/release-check" }], + "usage_metrics_requirements": { + "met": [], + "unmet": ["vscode"], + "unknown": [] + }, "used_agent": false, "used_chat": false, "used_cli": true, @@ -272,11 +278,14 @@ The `totals_by_client` array appears at the top level of each user record in bot In this example, `used_copilot_code_review_active` and `used_copilot_code_review_passive` are `null` because there was no {% data variables.copilot.copilot_code-review_short %} signal for the user that day. `used_copilot_cloud_agent` and `used_copilot_coding_agent` always carry the same value. +`client_telemetry_present` and `usage_metrics_requirements` appear at the top level of each user record in 1-day and 28-day per-user reports, for both enterprise and organization scopes. In 28-day reports, they describe the record's `day`. In this example, `usage_metrics_requirements.unmet` lists `vscode` because the user's last-known {% data variables.product.prodname_vscode_shortname %} version is below the supported minimum. `client_telemetry_present` is null or omitted when telemetry status is unknown or wasn't evaluated, such as for users with only {% data variables.copilot.copilot_cloud_agent %} or {% data variables.copilot.copilot_code-review_short %} activity. `usage_metrics_requirements` is null or omitted when no IDE or extension can be identified. + ## Enterprise-level schema example ```json copy [ { "day_totals" : [ { + "below_usage_metrics_minimum_user_count" : 1, "code_acceptance_activity_count" : 4, "code_generation_activity_count" : 4, "daily_active_cli_users" : 2, @@ -298,6 +307,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "loc_deleted_sum" : 6, "loc_suggested_to_add_sum" : 59, "loc_suggested_to_delete_sum" : 6, + "missing_client_telemetry_user_count" : 0, "monthly_active_agent_users" : 0, "monthly_active_chat_users" : 0, "monthly_active_copilot_cloud_agent_users" : 1, @@ -583,6 +593,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "report_start_day" : "2025-09-04" }, { "day_totals" : [ { + "below_usage_metrics_minimum_user_count" : 0, "code_acceptance_activity_count" : 2, "code_generation_activity_count" : 3, "daily_active_copilot_cloud_agent_users" : 0, @@ -601,6 +612,7 @@ In this example, `used_copilot_code_review_active` and `used_copilot_code_review "loc_deleted_sum" : 3, "loc_suggested_to_add_sum" : 52, "loc_suggested_to_delete_sum" : 3, + "missing_client_telemetry_user_count" : 1, "monthly_active_agent_users" : 0, "monthly_active_chat_users" : 0, "monthly_active_copilot_cloud_agent_users" : 0, @@ -799,6 +811,8 @@ The customization fields shown in `day_totals` apply to enterprise and organizat Similarly, `totals_by_client` appears within each `day_totals` entry in enterprise and organization aggregated 28-day reports, and directly on the record in aggregated 1-day reports. In aggregated reports, each `totals_by_client` entry's `session_count` is de-duplicated across all users rather than summed from per-user counts. +`missing_client_telemetry_user_count` and `below_usage_metrics_minimum_user_count` appear within each `day_totals` entry in enterprise and organization aggregated 28-day reports, and directly on the record in aggregated 1-day reports. They are daily distinct-user counts, so don't sum them across days. A value of `0` is a measured zero, while null or omitted means the count is unavailable, for example for days before the fields existed. + The following user-teams report examples are returned by the `user-teams-1-day` endpoints and are intended to be joined with the per-user usage report on `user_id`, `day`, and the relevant entity id (`organization_id` for the organization scope, `enterprise_id` for the enterprise scope) to derive team-level metrics. For more guidance, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/team-level-metrics). > [!NOTE] From 0f59cab2fd59559bc8192a5d856e323680c5c390 Mon Sep 17 00:00:00 2001 From: "Hector A." Date: Fri, 6 Nov 2026 13:27:34 +0000 Subject: [PATCH 09/12] Update GHES deprecation runbook and tooling with 3.18 findings (#64566) --- src/ghes-releases/lib/deprecation-steps.md | 55 ++++++++++++++----- .../create-docs-ghes-version-repo.sh | 13 ++++- 2 files changed, 54 insertions(+), 14 deletions(-) diff --git a/src/ghes-releases/lib/deprecation-steps.md b/src/ghes-releases/lib/deprecation-steps.md index d4bb172ef659..bed9bb0b487e 100644 --- a/src/ghes-releases/lib/deprecation-steps.md +++ b/src/ghes-releases/lib/deprecation-steps.md @@ -31,7 +31,7 @@ The release controller's Slack chatops are the authoritative source for both the * `.rc get-cycle {{ release-number }}` returns the release cycle for `{{ release-number }}`, including the deprecation date. * `.rc release-dates` returns the current DRI for each release. Run it in either `#ghes-releases` or `#ghes-release-ops`. It may stop listing a release once its deprecation completes, so run it a few days before the deprecation date if you can. -Draft both chatops and a follow-up confirmation message for the human, who runs the chatops in Slack and posts the confirmation. Cross-check the date returned by `.rc get-cycle` against the one in this issue's title and in `src/ghes-releases/lib/enterprise-dates.json`. +Draft both chatops and a follow-up confirmation message for the human, who runs the chatops in Slack and posts the confirmation. Cross-check the date returned by `.rc get-cycle` against the one in this issue's title and in `src/ghes-releases/lib/enterprise-dates.json`. You might not have read access to `github/enterprise-releases`. If you can't read the release date list, ask the human to cross-check it. 🛑 **HUMAN**: Run `.rc get-cycle {{ release-number }}` and `.rc release-dates` in `#ghes-releases` (or `#ghes-release-ops`). Post a short message in `#ghes-releases` `@`-mentioning the DRI returned by `.rc release-dates` asking them to confirm the deprecation date, and add a comment on this issue linking to that Slack message for posterity. If `.rc release-dates` doesn't return a DRI for `{{ release-number }}`, ask in `#docs-content-enterprise` or `#ghes-releases` instead. @@ -50,12 +50,14 @@ This clones every language into `./translations/` inside your `github/docs ## Step 3: Create the archive repository -Each deprecated version's docs live in their own repository, for example `github/docs-ghes-3.11`. Create the new one: +Each deprecated version's docs live in their own repository, for example `github/docs-ghes-3.11`. Creating it requires the human to be a maintainer of `@github/docs-engineering`, and the script exits early if they aren't. The script clones the new repository into `~/Documents/gh/github`; set `DOCS_GHES_CLONE_ROOT` to use another directory. Create the new one: ```shell npm run deprecate-ghes -- create-repo --version {{ release-number }} ``` +The repository's first GitHub Pages build can show as errored, because the initial push cancels it. The next build succeeds, so you can ignore it. + 🛑 **HUMAN**: On the new repository's home page, click the gear next to "About" and clear the "Releases", "Packages", and "Deployments" checkboxes, then save. No public API covers these toggles. You can continue once the human has said they completed that step. @@ -76,7 +78,7 @@ If the date in the [release date list](https://github.com/github/enterprise-rele Update your translation clones to the latest `main`. -Hide the search components so they don't get scraped into the static archive: in `src/frame/components/page-header/Header.tsx`, temporarily set the `SubdomainNavBar.Search` component's `className` to `"visually-hidden"`. Also wrap the `SearchOverlay` in `src/search/components/input/SearchOverlayContainer.tsx` in a `
`. +Hide the search components so they don't get scraped into the static archive: in `src/frame/components/page-header/Header.tsx`, temporarily set the `SubdomainNavBar.Search` component's `className` to `"visually-hidden"`. Also wrap the `SearchOverlay` in `src/search/components/input/SearchOverlayContainer.tsx` in a `
`, and, in `src/landings/components/HomePageHero.tsx`, temporarily remove the homepage hero search `