Skip to content

Commit 5363435

Browse files
authored
Merge pull request #45934 from github/repo-sync
Repo sync
2 parents 2cb18fc + 6fb9ff1 commit 5363435

193 files changed

Lines changed: 245325 additions & 9976 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/instructions/content.instructions.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,13 @@ Articles with `contentType: rai` in their frontmatter are **application or platf
8888
* **Reusables**: RAI articles must only reference reusables from `data/reusables/rai/`. Place new RAI reusables there too.
8989
* **Frontmatter**: New application cards use `contentType: rai`. The older `type: rai` is for legacy transparency notes not yet migrated.
9090

91-
## Parenthetical dashes
91+
## Punctuation
92+
93+
### Semicolons
94+
95+
Avoid semicolons except where necessary. Generally they can be avoided and shorter sentences used instead. As a general rule, try to avoid very long sentences.
96+
97+
### Parenthetical dashes
9298

9399
Where a sentence of normal body text contains a parenthetical dash, the dash should always be an em dash without spaces at either side. This rule does not apply to text within code blocks.
94100

‎content/admin/backing-up-and-restoring-your-instance/configuring-backups-on-your-instance.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,14 @@ Backup snapshots created by {% data variables.product.prodname_enterprise_backup
116116

117117
For more information on advanced usage, see the [{% data variables.product.prodname_enterprise_backup_utilities %} README](https://github.com/github/backup-utils#readme) in the {% data variables.product.prodname_enterprise_backup_utilities %} project documentation.
118118

119+
{% ifversion ghes > 3.19 %}
120+
121+
<!-- The linked article is versioned `ghes: '>=3.20'`, so this paragraph must stay gated to matching versions. Without the gate, the link is unresolvable in 3.19 and earlier and rendering fails. -->
122+
123+
By default, {% data variables.product.prodname_enterprise_backup_utilities %} backs up search index data by copying files directly from disk. You can optionally configure {% data variables.location.product_location %} to use Elasticsearch's native, incremental snapshot functionality with a customer-managed cloud storage provider instead. For more information, see [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/configuring-elasticsearch-snapshots).
124+
125+
{% endif %}
126+
119127
## Upgrading {% data variables.product.prodname_enterprise_backup_utilities %}
120128

121129
When upgrading {% data variables.product.prodname_enterprise_backup_utilities %}, you must choose a version that will work with your current version of {% data variables.product.prodname_ghe_server %}. Your installation of {% data variables.product.prodname_enterprise_backup_utilities %} must be at least the same version as {% data variables.location.product_location %}, and cannot be more than two versions ahead. For more information, see [{% data variables.product.prodname_ghe_server %} version requirements](https://github.com/github/backup-utils/blob/master/docs/requirements.md#github-enterprise-server-version-requirements) in the {% data variables.product.prodname_enterprise_backup_utilities %} project documentation.
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
---
2+
title: Configuring Elasticsearch snapshots
3+
shortTitle: Elasticsearch snapshots
4+
intro: 'You can configure {% data variables.location.product_location %} to use a customer-managed cloud storage provider for native Elasticsearch snapshots and restores, instead of copying search index data directly from disk.'
5+
permissions: Site administrators can configure Elasticsearch snapshot storage.
6+
versions:
7+
ghes: '>=3.20'
8+
contentType: how-tos
9+
category:
10+
- Back up and upgrade your instance
11+
---
12+
13+
<!-- TODO(docs review): confirm the minimum GHES version for this feature before merging. -->
14+
15+
## About Elasticsearch snapshots
16+
17+
By default, {% data variables.product.prodname_enterprise_backup_utilities %} and {% data variables.product.prodname_enterprise_backup_service %} back up search index data by copying files directly from disk. This approach doesn't follow Elasticsearch's own recommendations, can consume a large amount of backup appliance storage, and carries some risk of corrupting the backed-up indices.
18+
19+
As an opt-in alternative, {% data variables.location.product_location %} supports Elasticsearch's built-in, incremental snapshot and restore functionality. When you configure a supported cloud storage provider and enable the snapshot backup strategy:
20+
21+
* `ghe-backup` creates an incremental snapshot of your search index data in your configured cloud storage account. Incremental snapshots are faster than a full copy, especially on subsequent backups.
22+
* `ghe-restore` restores search index data from your cloud storage account, if a snapshot exists that matches the target instance's {% data variables.product.prodname_ghe_server %} version.
23+
* Snapshots are isolated per {% data variables.product.prodname_ghe_server %} patch version, which allows for safer rollbacks between versions.
24+
25+
This feature is opt-in. If you don't configure a snapshot repository, {% data variables.location.product_location %} continues to back up search index data using the existing disk-based method.
26+
27+
## Prerequisites
28+
29+
* A supported cloud storage provider: Azure Blob storage, Amazon S3, or Google Cloud Storage (or a service that's compatible with one of these APIs).
30+
* The storage container or bucket you plan to use as the snapshot repository. **You must create this container or bucket yourself.** {% data variables.product.prodname_ghe_server %} does not create it for you, and repository registration will fail if the container or bucket doesn't already exist.
31+
32+
## Configuring a snapshot storage provider
33+
34+
Configure the following settings using `ghe-config` over SSH. For more information, see [AUTOTITLE](/admin/administering-your-instance/administering-your-instance-from-the-command-line/accessing-the-administrative-shell-ssh).
35+
36+
You'll apply the configuration to register the snapshot repository with Elasticsearch. The repository is named `search_index_snapshots-VERSION`, where `VERSION` is the current {% data variables.product.prodname_ghe_server %} release version.
37+
38+
### Azure Blob storage
39+
40+
1. Set the following secrets:
41+
42+
```shell
43+
ghe-config secrets.elasticsearch.snapshot-provider azure
44+
ghe-config secrets.elasticsearch.snapshot.azure.container YOUR-CONTAINER
45+
ghe-config secrets.elasticsearch.snapshot.azure.account-name YOUR-STORAGE-ACCOUNT
46+
ghe-config secrets.elasticsearch.snapshot.azure.account-key YOUR-ACCOUNT-KEY
47+
```
48+
49+
Optionally, set a custom blob endpoint suffix (defaults to `core.windows.net`):
50+
51+
```shell
52+
ghe-config secrets.elasticsearch.snapshot.azure.endpoint-suffix YOUR-ENDPOINT-SUFFIX
53+
```
54+
55+
1. Run configuration apply.
56+
57+
```shell
58+
ghe-config-apply
59+
```
60+
61+
### Amazon S3
62+
63+
1. Set the following secrets:
64+
65+
```shell
66+
ghe-config secrets.elasticsearch.snapshot-provider s3
67+
ghe-config secrets.elasticsearch.snapshot.bucket-name YOUR-BUCKET-NAME
68+
ghe-config secrets.elasticsearch.snapshot.service-url YOUR-SERVICE-URL
69+
ghe-config secrets.elasticsearch.snapshot.region YOUR-AWS-REGION
70+
ghe-config secrets.elasticsearch.snapshot.access-key-id YOUR-ACCESS-KEY-ID
71+
ghe-config secrets.elasticsearch.snapshot.access-secret YOUR-ACCESS-SECRET
72+
```
73+
74+
For the service URL, use your S3 endpoint, for example `https://s3.us-east-1.amazonaws.com`.
75+
76+
1. Run configuration apply.
77+
78+
```shell
79+
ghe-config-apply
80+
```
81+
82+
### Google Cloud Storage
83+
84+
1. Create a JSON service account key with access to your bucket, then encode it as base64.
85+
86+
```shell
87+
base64 -w0 service-account.json
88+
```
89+
90+
1. Set the following secrets:
91+
92+
```shell
93+
ghe-config secrets.elasticsearch.snapshot-provider gcs
94+
ghe-config secrets.elasticsearch.snapshot.gcs.bucket YOUR-BUCKET-NAME
95+
ghe-config secrets.elasticsearch.snapshot.gcs.credentials YOUR-BASE64-ENCODED-CREDENTIALS
96+
```
97+
98+
1. Run configuration apply.
99+
100+
```shell
101+
ghe-config-apply
102+
```
103+
104+
### Verifying registration
105+
106+
After configuration apply completes, confirm the snapshot repository was registered.
107+
108+
```shell
109+
curl -k "http://127.0.0.1:9200/_snapshot/search_index_snapshots-$(ghe-version -v)/_status"
110+
```
111+
112+
If a required secret is missing, configuration apply fails while registering the repository. Update the secrets and rerun `ghe-config-apply` to retry.
113+
114+
## Enabling snapshots during backup and restore
115+
116+
Configuring a storage provider registers the snapshot repository, but you must also enable the snapshot backup strategy so `ghe-backup` and `ghe-restore` use it automatically.
117+
118+
* **{% data variables.product.prodname_enterprise_backup_service %}:** Set the strategy using `ghe-config` on the appliance.
119+
120+
```shell
121+
ghe-config backup.es-backup-strategy snapshot
122+
```
123+
124+
* **{% data variables.product.prodname_enterprise_backup_utilities %}:** Set the strategy in your `backup.config` file on the backup host.
125+
126+
```shell
127+
GHE_ES_BACKUP_STRATEGY=snapshot
128+
```
129+
130+
The default value is `rsync`, which preserves the existing disk-based backup behavior. {% data variables.product.prodname_enterprise_backup_utilities %} stores only snapshot metadata locally; snapshot contents remain in your configured cloud storage.
131+
132+
If you don't set a backup strategy, the snapshot repository can still be managed manually, but `ghe-backup` and `ghe-restore` will continue to use the disk-based method.
133+
134+
## Managing snapshots manually
135+
136+
You can manage Elasticsearch snapshots directly using the following commands. Run any command with the `-h` flag for usage information.
137+
138+
| Command | Description |
139+
| --- | --- |
140+
| `ghe-es-create-snapshot` | Creates a new snapshot in the configured repository. |
141+
| `ghe-es-list-snapshots` | Lists snapshot repositories, or snapshots within a repository. |
142+
| `ghe-es-restore-snapshot` | Restores search indices from a snapshot. |
143+
144+
> [!TIP]
145+
> For routine backups and restores, use {% data variables.product.prodname_enterprise_backup_utilities %} or {% data variables.product.prodname_enterprise_backup_service %} instead of these commands directly. The backup tooling coordinates Elasticsearch snapshots with the rest of your instance's data to ensure a consistent backup or restore. Use the manual commands only when you need to manage search index snapshots independently of a full instance backup or restore.
146+
147+
### Restoring a snapshot from an earlier version
148+
149+
You can restore a snapshot created by an earlier {% data variables.product.prodname_ghe_server %} version to a later version. You cannot restore a snapshot from a later version to an earlier version.
150+
151+
1. List available snapshots for the earlier version.
152+
153+
```shell
154+
ghe-es-list-snapshots -v 3.14.0
155+
```
156+
157+
1. Restore the snapshot you want.
158+
159+
```shell
160+
ghe-es-restore-snapshot -v 3.14.0 -s SNAPSHOT-NAME
161+
```
162+
163+
Elasticsearch automatically upgrades the restored indices to the current version's format.
164+
165+
## Managing snapshot storage
166+
167+
Elasticsearch snapshots are incremental, but your cloud storage account can still accumulate old snapshots over time. To avoid unbounded storage growth, configure a lifecycle policy with your storage provider to automatically delete snapshots older than your retention requirements. For more information, see your provider's documentation:
168+
169+
* [Amazon S3 Lifecycle rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html)
170+
* [Azure Blob storage lifecycle management](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview)
171+
* [Google Cloud Storage Object Lifecycle Management](https://cloud.google.com/storage/docs/lifecycle)

‎content/admin/backing-up-and-restoring-your-instance/configuring-the-backup-service.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,13 @@ Once the service is configured, you can define a backup schedule.
179179
{% endif %}
180180

181181
The first run will be a full backup. Future runs will be incremental. If a new backup attempt starts while a previous one is still running, it may be skipped or fail. In that case, adjust the schedule to avoid overlap.
182+
183+
{% ifversion ghes > 3.19 %}
184+
185+
<!-- The linked article is versioned `ghes: '>=3.20'`, so this whole section must stay gated to matching versions. Without the gate, the link is unresolvable in 3.19 and earlier and rendering fails. -->
186+
187+
## Using Elasticsearch snapshots for search index data
188+
189+
By default, search index data is backed up by copying files directly from disk. You can optionally configure {% data variables.location.product_location %} to use Elasticsearch's native, incremental snapshot functionality with a customer-managed cloud storage provider instead. For more information, see [AUTOTITLE](/admin/backing-up-and-restoring-your-instance/configuring-elasticsearch-snapshots).
190+
191+
{% endif %}

‎content/admin/backing-up-and-restoring-your-instance/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ children:
88
- /about-the-backup-service-for-github-enterprise-server
99
- /understanding-the-backup-service
1010
- /configuring-the-backup-service
11+
- /configuring-elasticsearch-snapshots
1112
- /creating-and-monitoring-backups
1213
- /restoring-from-a-backup
1314
- /configuring-remote-archives-for-backups

‎content/authentication/authenticating-with-single-sign-on/authorizing-a-personal-access-token-for-use-with-single-sign-on.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@ redirect_from:
77
- /github/authenticating-to-github/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
88
- /github/authenticating-to-github/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
99
- /authentication/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on
10-
- /authentication/authenticating-with-single-sign-on/authorizing-credentials-for-single-sign-on-with-a-github-app
1110
versions:
1211
ghec: '*'
1312
shortTitle: '{% data variables.product.pat_generic_caps %} with SSO'
@@ -20,6 +19,10 @@ You must authorize your {% data variables.product.pat_v1 %} after creation befor
2019

2120
{% data reusables.saml.authorized-creds-info %}
2221

22+
Enterprise administrators can use a {% data variables.product.prodname_github_app %} to authorize credentials for multiple organizations. See [AUTOTITLE](/authentication/authenticating-with-single-sign-on/authorizing-credentials-for-single-sign-on-with-a-github-app).
23+
24+
## Authorizing a {% data variables.product.pat_v1 %}
25+
2326
{% data reusables.user-settings.access_settings %}
2427
{% data reusables.user-settings.developer_settings %}
2528
{% data reusables.user-settings.personal_access_tokens %}

0 commit comments

Comments
 (0)