Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,40 @@ $ ./jmsctl.sh install

## 管理命令

新安装默认使用 `postgres:18.6-bookworm`。统一的 `compose/postgresql.yml` 显式设置
`PGDATA=/var/lib/postgresql/data`,继续把 `${VOLUME_DIR}/postgresql/data`
挂载到该目录,以兼容 JDMC 的主备重建、回切和启动写入隔离。同时显式挂载
`${VOLUME_DIR}/postgresql:/var/lib/postgresql`,避免 PostgreSQL 18 镜像自动创建
额外匿名卷,导致 JDMC 的 Compose 挂载一致性校验失败。
主 Compose 的镜像固定为 `postgres:18.6-bookworm`,不需要数据库镜像配置项。
旧版本按实际数据版本选择镜像,仅在迁移前自动生成 `${CONFIG_DIR}/postgresql/image-<tag>.yml`
运行时镜像覆盖文件,使用相同的 `PGDATA`;迁移完成后使用主 Compose 中的 18 镜像。
运行时文件会保留,避免影响 JDMC 已记录的 Compose 绑定。JDMC 升级已有主备时继续使用
已绑定的旧 Compose 文件,在协调迁移切换时更新数据库配置和绑定,不改写旧文件。
单机生成数据库 Compose 命令时会补齐父目录的可穿越权限,避免旧目录为 `700` 时容器
无法访问;数据目录 `postgresql/data` 的权限保持不变。

从 JumpServer 5.0 等旧应用版本升级时,内置 PostgreSQL 10–17 会在同一次维护中
迁移到 `postgres:18.6-bookworm`。安装器先加载旧镜像和 18 镜像,停止应用写入并禁用
自动重启,使用 18 客户端导出全部数据库和角色,在隔离目录恢复并校验后再切换数据目录。
迁移前仍按 `PG_VERSION` 选择旧镜像;不能让 18 直接打开旧数据。
迁移完成后更新业务表结构并恢复原先运行的应用。外置数据库不会被升级。

旧目录、逻辑备份和进度保存在 `${VOLUME_DIR}/postgresql/data.upgrade18/`;
`original/` 是原数据,`cluster.sql` 包含全库及角色凭据,目录和文件必须保持私有。
迁移备份是必需步骤,不受 `SKIP_BACKUP_DB` 影响。中断时应用保持停止,重试同一升级
继续处理;完成迁移前普通启动会被拒绝。恢复业务后新产生的数据不会写回旧目录,
不能直接把旧目录切回去。空间预检按旧数据三倍加 1 GiB 估算;大对象或高压缩数据可能需要更多空间。
自动迁移要求可验证的管理员账号、标准数据路径和配置;自定义表空间、外置配置或 SSL
文件、逻辑复制、非 libc 模板区域设置等定制部署在预检时中止,需要单独迁移。原镜像及旧数据不会自动清理。

JDMC 主备由两台协调维护:停止业务和选举,迁移主库、重新建立 18 备库,校验流复制、
同步提交和启动写入隔离,再更新应用并退出维护。两台 JDMC 必须先具备此迁移能力;
新的全量升级包声明 `maintenance-pg18-v1`,旧 JDMC 会拒绝该包,避免跳过数据库迁移。
`pg_rewind` 不承担数据库大版本迁移。
参考 [PostgreSQL 升级说明](https://www.postgresql.org/docs/18/upgrading.html)
和 [主备版本要求](https://www.postgresql.org/docs/18/warm-standby.html)。

```
# 启动
$ ./jmsctl.sh start
Expand Down
6 changes: 5 additions & 1 deletion compose/postgresql.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
services:
postgresql:
image: postgres:16.15-bookworm
image: postgres:18.6-bookworm
container_name: jms_postgresql
restart: always
environment:
TZ: ${TZ:-Asia/Shanghai}
POSTGRES_DB: $DB_NAME
POSTGRES_USER: $DB_USER
POSTGRES_PASSWORD: $DB_PASSWORD
# Keep the data path used by existing installations and JDMC on every major.
PGDATA: /var/lib/postgresql/data
volumes:
# PostgreSQL 18 declares this parent volume; make it explicit for JDMC.
- ${VOLUME_DIR}/postgresql:/var/lib/postgresql
- ${VOLUME_DIR}/postgresql/data:/var/lib/postgresql/data
healthcheck:
test: "pg_isready -q -U $$POSTGRES_USER"
Expand Down
2 changes: 2 additions & 0 deletions config-example.txt
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ DB_HOST=postgresql
# For a new built-in DB_ENGINE=mysql installation, choose mysql:8.0 or
# mariadb:10.6. Existing mysql/data or mariadb/data takes precedence.
DB_IMAGE=mariadb:10.6
# New PostgreSQL installations use 18.6-bookworm. Upgrades select the old image
# from the existing data version until migration to 18 has completed.
DB_PORT=5432
DB_USER=postgres
DB_PASSWORD=
Expand Down
9 changes: 9 additions & 0 deletions scripts/3_load_images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,15 @@ function load_image_files() {
function main() {
# Panda belongs to remote publishers, even if packaging flags are inherited.
local INCLUDE_PANDA_IMAGE=0
if [[ "$(get_config HA_MODE)" == 1 && -n "${JDMC_HA_MAINTENANCE_ID:-}" &&
"${JDMC_HA_PG18_COORDINATED:-0}" != 1 && "${POSTGRESQL_MAJOR_UPGRADE:-0}" != 1 && "$(get_config DB_ENGINE)" == postgresql ]]; then
local pg_major
pg_major=$(get_internal_postgresql_data_version) || return 1
if [[ -n "${pg_major}" && "${pg_major}" -lt 18 ]]; then
log_error 'This installer requires coordinated PostgreSQL 18 migration. Update JDMC on both nodes before applying this package.'
return 1
fi
fi
if [[ -d "${IMAGE_DIR}" && $(find "${IMAGE_DIR}" -type f -name "*.zst" -print -quit 2>/dev/null) ]]; then
load_image_files || return 1
else
Expand Down
33 changes: 30 additions & 3 deletions scripts/7_upgrade.sh
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,9 @@ function migrate_internal_mysql_business_account() {

function remove_obsolete_containers() {
local container existing_containers
local containers=("jms_guacamole" "jms_lina" "jms_luna" "jms_nginx" "jms_xpack" "jms_lb" "jms_omnidb" "jms_kael" "jms_magnus" "jms_video" "jms_lion" "jms_facelive" "jms_panda")
# Kael and Magnus are current services. Keep their containers until the normal
# stop/recreate flow so PostgreSQL migration can record their restart intent.
local containers=("jms_guacamole" "jms_lina" "jms_luna" "jms_nginx" "jms_xpack" "jms_lb" "jms_omnidb" "jms_video" "jms_lion" "jms_facelive" "jms_panda")

# XRDP is still supported when explicitly enabled in the enterprise edition.
if [[ "$(get_config_or_env USE_XPACK)" != "1" || "$(get_config_or_env XRDP_ENABLED)" != "1" ]]; then
Expand Down Expand Up @@ -455,6 +457,7 @@ function cleanup_failed_upgrade() {
local status=$? safe_to_restore=1

if [[ "${status}" != "0" && "${UPGRADE_COMMITTED}" != "1" ]]; then
fence_internal_postgresql_upgrade || log_error "Could not stop all application containers after the interrupted PostgreSQL migration"
if [[ "${UPGRADE_CONFIG_CHANGED}" == "1" && \
"${UPGRADE_DATABASE_MIGRATION_STARTED}" == "0" && \
-f "${backup_config_file:-}" ]]; then
Expand Down Expand Up @@ -484,13 +487,21 @@ function cleanup_failed_upgrade() {
}

function backup_db() {
local db_image backup_failed=0 confirm="y"
local db_image migration_backup backup_failed=0 confirm="y"

if [[ "${SKIP_BACKUP_DB:-0}" == "1" ]]; then
echo "SKIP_BACKUP_DB=${SKIP_BACKUP_DB}, $(gettext 'Skip database backup')"
return 0
fi

migration_backup="$(get_config VOLUME_DIR)/postgresql/data.upgrade18/cluster.sql"
if postgresql_upgrade_pending && [[ -s "${migration_backup}" ]]; then
# A retry must not start the temporary Core container while migration is
# fenced. The atomic full-cluster export still contains the original data.
log_success "Resuming PostgreSQL migration; retained full-cluster backup: ${migration_backup}"
return 0
fi

if ! db_image=$(get_db_images) || [[ -z "${db_image}" ]]; then
log_warn 'Could not determine the database backup image.'
backup_failed=1
Expand Down Expand Up @@ -536,6 +547,10 @@ function db_migrations() {
fi
if ! perform_db_migrations; then
log_error "$(gettext 'Failed to change the table structure')!"
if [[ "${PG18_UPGRADE_ACTIVE:-0}" == 1 ]]; then
log_error 'PostgreSQL migrated, but application schema migration failed; keep business stopped and retry the same upgrade'
exit 1
fi
confirm="n"
read_from_input confirm "$(gettext 'Failed to change the table structure. Continue to upgrade')?" "y/n" "${confirm}"
if [[ "${confirm}" != "y" ]]; then
Expand Down Expand Up @@ -660,7 +675,7 @@ function main() {
# Database backup starts a Core container using the target version.
# Load the offline images before that container is created.
echo_yellow "\n3. $(gettext 'Loading Docker Image')"
if ! ALLOW_MISSING_DB_IMAGE=1 bash "${BASE_DIR}/3_load_images.sh"; then
if ! POSTGRESQL_MAJOR_UPGRADE=1 ALLOW_MISSING_DB_IMAGE=1 bash "${BASE_DIR}/3_load_images.sh"; then
log_error "$(gettext 'Failed to load Docker images')"
exit 1
fi
Expand All @@ -681,6 +696,14 @@ function main() {
echo "$(gettext 'Changing database schema may take a while, please wait patiently')"
write_upgrade_recovery_state recovery_required || exit 1
UPGRADE_DATABASE_MIGRATION_STARTED=1
upgrade_internal_postgresql || {
log_error "PostgreSQL migration did not complete; business remains stopped. Retry the same upgrade."
exit 1
}
# Remove the former image setting only after cutover; an old HA Compose
# binding may still require it while its database is being migrated.
remove_config POSTGRESQL_IMAGE || exit 1
gen_safe_config >/dev/null || exit 1
db_migrations

echo_yellow "\n7. $(gettext 'Upgrade Docker')"
Expand Down Expand Up @@ -708,6 +731,10 @@ function main() {
log_error "Failed to update CURRENT_VERSION"
exit 1
fi
complete_internal_postgresql_upgrade || {
log_error "PostgreSQL migrated, but application recovery is incomplete; retry the same upgrade"
exit 1
}
UPGRADE_COMMITTED=1
trap - EXIT
installation_log "upgrade"
Expand Down
14 changes: 11 additions & 3 deletions scripts/gists/image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,15 @@ function report_image_failure() {
function reuse_local_db_image() {
local image=$1

# New offline packages only bundle PostgreSQL. Existing MySQL/MariaDB
# installations retain their database image, including legacy tags.
# New offline packages bundle the default PostgreSQL image. Existing
# databases may reuse their local image when no matching archive is bundled.
case "${image}" in
mysql:8.0|mariadb:10.6) ;;
postgres:*) is_external_db_image "${image}" || return 1 ;;
postgres:*)
if ! is_external_db_image "${image}"; then
[[ -n "$(get_internal_postgresql_data_version)" && ( "${image}" == "$(get_db_images)" || ( "${POSTGRESQL_MAJOR_UPGRADE:-0}" == 1 && "${image}" == postgres:18.6-bookworm ) ) ]] || return 1
fi
;;
*) return 1 ;;
esac
if docker image inspect "${image}" &>/dev/null; then
Expand Down Expand Up @@ -97,6 +101,10 @@ function get_image_mappings() {
if [[ -n "${db_image}" ]] && ! should_skip_external_db_image "${db_image}"; then
emit_image_mapping "${db_image}" "${db_image}"
fi
if [[ "${POSTGRESQL_MAJOR_UPGRADE:-0}" == 1 && "$(get_config DB_ENGINE)" == postgresql &&
( "$(get_config DB_HOST)" == postgresql || "$(get_config HA_MODE)" == 1 ) && "${db_image}" != postgres:18.6-bookworm ]]; then
emit_image_mapping postgres:18.6-bookworm postgres:18.6-bookworm
fi

enabled_services=${OFFLINE_IMAGE_SERVICES:-$(get_enabled_services)}
enabled_services=${enabled_services//,/ }
Expand Down
130 changes: 130 additions & 0 deletions scripts/gists/postgresql-upgrade-standalone.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
#!/usr/bin/env bash
# PG18_* variables are inputs to the separately sourced migration library.
# shellcheck disable=SC2034

postgresql_upgrade_is_internal() {
[[ "$(get_config DB_ENGINE)" == postgresql && "$(get_config DB_HOST)" == postgresql && "$(get_config HA_MODE)" != 1 ]]
}

postgresql_upgrade_pending() {
local directory phase
postgresql_upgrade_is_internal || return 1
directory="$(get_config VOLUME_DIR)/postgresql/data.upgrade18"
[[ -e "${directory}/phase" ]] || return 1
phase=$(cat "${directory}/phase") || return 0
[[ "${phase}" != complete ]]
}

postgresql_upgrade_start_guard() {
if [[ "${PG18_UPGRADE_ACTIVE:-0}" != 1 ]] && postgresql_upgrade_pending; then
log_error 'PostgreSQL migration is pending; retry the same upgrade before starting services'
return 1
fi
}

upgrade_internal_postgresql() {
local version container policy running retries service cmd
local -a names=() services=()
postgresql_upgrade_is_internal || return 0
PG18_DATA="$(get_config VOLUME_DIR)/postgresql/data"
version=$(cat "${PG18_DATA}/PG_VERSION" 2>/dev/null || true)
if [[ ! -e "${PG18_DATA}.upgrade18/phase" ]]; then
[[ -n "${version}" && "${version}" -lt 18 ]] || return 0
elif [[ "$(cat "${PG18_DATA}.upgrade18/phase")" == complete ]]; then
return 0
fi
. "${SCRIPT_DIR}/gists/postgresql-upgrade.sh"
PG18_CONTAINER=jms_postgresql PG18_USER=$(get_config DB_USER) PG18_PASSWORD=$(get_config DB_PASSWORD) PG18_PORT=$(get_config DB_PORT 5432)
pg18_paths || return 1
exec 201>"${PG18_WORK}/upgrade.lock"
flock -n 201 || { log_error "Another PostgreSQL migration is running"; return 1; }
export PG18_UPGRADE_ACTIVE=1
if [[ ! -f "${PG18_WORK}/phase" ]]; then
docker start "${PG18_CONTAINER}" >/dev/null && wait_container_healthy "${PG18_CONTAINER}" || return 1
pg18_preflight || return 1
fi
# Save restart intent before stopping anything. A host restart during the
# migration must not restart an old application against a partially upgraded DB.
if [[ ! -f "${PG18_WORK}/writers" ]]; then
for service in $(get_enabled_services); do
container="jms_${service}"
[[ "${service}" != ansible-executor ]] || container=jms_ansible_executor
names+=("${container}")
done
: >"${PG18_WORK}/writers.partial"
for container in "${names[@]}"; do
if docker inspect "${container}" >/dev/null 2>&1; then
docker inspect -f '{{.Name}} {{.State.Running}} {{.HostConfig.RestartPolicy.Name}} {{.HostConfig.RestartPolicy.MaximumRetryCount}} {{index .Config.Labels "com.docker.compose.service"}}' "${container}" >>"${PG18_WORK}/writers.partial" || return 1
fi
done
chmod 600 "${PG18_WORK}/writers.partial" && sync "${PG18_WORK}/writers.partial" &&
mv "${PG18_WORK}/writers.partial" "${PG18_WORK}/writers" || return 1
fi
[[ -f "${PG18_WORK}/phase" ]] || pg18_save phase reserved || return 1
while read -r container running policy retries service; do
[[ "${container}" == /jms_* && "${service}" =~ ^[A-Za-z0-9_.-]+$ ]] || return 1
if docker inspect "${container}" >/dev/null 2>&1; then
docker update --restart=no "${container}" >/dev/null && docker stop "${container}" >/dev/null || return 1
[[ "$(docker inspect -f '{{.State.Running}}' "${container}")" == false ]] || return 1
fi
done <"${PG18_WORK}/writers"
# The data container's old image remains valid until the staged cluster is ready.
if [[ ! -f "${PG18_WORK}/phase" || "$(cat "${PG18_WORK}/phase")" == preparing || "$(cat "${PG18_WORK}/phase")" == reserved ]]; then
docker start "${PG18_CONTAINER}" >/dev/null && wait_container_healthy "${PG18_CONTAINER}" || return 1
fi
pg18_prepare && pg18_cutover || return 1
gen_safe_config >/dev/null || return 1
cmd=$(get_db_compose_cmd db) || return 1
${cmd} up -d --no-deps --force-recreate postgresql || return 1
wait_container_healthy "${PG18_CONTAINER}" || return 1
[[ "$(pg18_sql "select current_setting('server_version_num')::int/10000;")" == 18 ]] || return 1
pg18_save phase switched
}

complete_internal_postgresql_upgrade() {
local container running policy retries service cmd state deadline timeout
local -a services=()
[[ "${PG18_UPGRADE_ACTIVE:-0}" == 1 && -f "${PG18_WORK:-}/writers" ]] || return 0
# Called only after application schema and JDMC upgrades have succeeded.
while read -r container running policy retries service; do
[[ "${running}" != true ]] || services+=("${service}")
done <"${PG18_WORK}/writers"
if (( ${#services[@]} )); then
cmd=$(get_docker_compose_cmd_line) || return 1
${cmd} up -d "${services[@]}" || return 1
fi
timeout=$(get_config CONTAINER_HEALTH_TIMEOUT 300)
[[ "${timeout}" =~ ^[0-9]+$ ]] || timeout=300
deadline=$((SECONDS + timeout))
while read -r container running policy retries service; do
if [[ "${running}" == true ]]; then
while :; do
state=$(docker inspect -f '{{if not .State.Running}}{{.State.Status}}{{else if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' "${container}") || return 1
[[ "${state}" != healthy && "${state}" != running ]] || break
if [[ "${state}" == exited || "${state}" == dead ]] || (( SECONDS >= deadline )); then
log_error "Failed to recover ${container}: ${state}"
return 1
fi
sleep 2
done
fi
[[ "${policy}" =~ ^(no|always|unless-stopped|on-failure)$ && "${retries}" =~ ^[0-9]+$ ]] || return 1
if [[ "${policy}" == on-failure && "${retries}" != 0 ]]; then policy+="${policy:+:}${retries}"; fi
if docker inspect "${container}" >/dev/null 2>&1; then docker update --restart="${policy}" "${container}" >/dev/null || return 1; fi
done <"${PG18_WORK}/writers"
pg18_save phase complete || return 1
log_success "PostgreSQL is now ${PG18_IMAGE}; original data and migration backup retained in ${PG18_WORK}"
}

# EXIT cleanup must re-fence even a partly started application on recovery failure.
fence_internal_postgresql_upgrade() {
local service container
[[ "${PG18_UPGRADE_ACTIVE:-0}" == 1 && -f "${PG18_WORK:-}/phase" ]] || return 0
for service in $(get_enabled_services); do
container="jms_${service}"
[[ "${service}" != ansible-executor ]] || container=jms_ansible_executor
if docker inspect "${container}" >/dev/null 2>&1; then
docker update --restart=no "${container}" >/dev/null && docker stop "${container}" >/dev/null || return 1
fi
done
}
Loading
Loading