diff --git a/docs/cn/guides/20-self-hosted/04-references/node-config/query-config.md b/docs/cn/guides/20-self-hosted/04-references/node-config/query-config.md index c5af8a077c..4b42a2e75a 100644 --- a/docs/cn/guides/20-self-hosted/04-references/node-config/query-config.md +++ b/docs/cn/guides/20-self-hosted/04-references/node-config/query-config.md @@ -6,7 +6,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; import LanguageDocs from '@site/src/components/LanguageDocs'; import DetailsWrap from '@site/src/components/DetailsWrap'; - + 本页介绍 [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) 配置文件中可用的 Query 节点配置。 @@ -57,6 +57,33 @@ import DetailsWrap from '@site/src/components/DetailsWrap'; SQL 配置与信任边界见[外部访问集成](/guides/security/external-access-integration)。 + + +### 脚本 UDF 资源 + +以下节点配置属于 `[query]`，不是 `[query.settings]`。查询专属的 Python 标量 Worker 和自动 Worker 容量选择在 v1.2.955 中引入。从 v1.2.957 开始，本地 Python UDF 和 UDAF 使用 [CREATE SCALAR FUNCTION](/sql/sql-commands/ddl/udf/ddl-create-function#python) 中描述的强制 OS 沙箱。 + +| 参数 | 默认值 | 说明 | +|-----------|---------|-------------| +| `enable_udf_python_script` | `true` | 允许本地 Python 脚本，包括聚合函数。 | +| `enable_udf_sandbox` | `false` | 将 Python 标量函数路由到外部沙箱服务。此模式下拒绝 Python UDAF。此设置不会禁用本地 OS 沙箱。 | +| `python_udf_max_workers` | `0` | 节点 Worker 容量。`0` 使用 `query.num_cpus`；若 `num_cpus = 0`，则使用可用 CPU 并行度。 | +| `python_udf_max_workers_per_tenant` | `0` | 此节点上的租户容量。`0` 继承节点容量。 | +| `python_udf_max_workers_per_query` | `0` | 单次查询的驻留 Worker 上限。`0` 使用 `M + P - 1`，其中 `M` 为本地运行时数量，`P` 为有效并行度。 | +| `python_udf_max_queued_queries` | `128` | 最大准入等待数量，必须为正数。 | +| `python_udf_queue_timeout_secs` | `60` | 准入等待超时，单位为秒，必须为正数。 | +| `python_udf_execution_timeout_secs` | `300` | 每个 Worker 请求的超时，包括首次请求的解释器启动时间，必须为正数。 | +| `python_udf_worker_idle_timeout_secs` | `300` | 额外 Worker 的空闲超时。每个运行时保留一个最小 Worker；不会使忙碌 Worker 过期。必须为正数。 | +| `python_udf_pip_only_binary` | `true` | 只安装 wheel。`false` 允许源码分发包在 Query 节点上、Worker 沙箱之外执行构建脚本。 | +| `udf_script_memory_limit_mb` | `1024` | 每个 JavaScript 运行时的堆限制和每个 WASM 实例的线性内存限制，单位为 MiB。`0` 表示无限制。不限制 Python Worker 内存。 | +| `udf_script_execution_timeout_secs` | `300` | JavaScript 操作截止时间，包括模块初始化和转换。`0` 表示无限制。不对 WASM 强制执行。 | + +Python 容量是单节点限制，不是整个计算集群的限制。非零配置的 Worker 限制必须满足 `query <= tenant <= node`。有效并行度遵循 SQL `max_threads`，并受节点和租户容量约束。每个已注册运行时准入一个最小 Worker；额外 Worker 按需增长。若查询预算无法容纳最小运行时数量，则会失败，而不是无限等待。 + +Python 请求超时不是整个 SQL 的超时。普通查询超时仍控制总执行时间。取消或失败/超时的 Worker 请求会使该运行时失效；批次不会自动重试。查询完成后会终止其 Worker。Worker 永远不会被后续查询共享。 + +进程数量和请求超时不提供 OS 强制执行的 Python 内存或 CPU 预算。Worker 内存不计入查询分配器。对于内存密集型包，请显式设置较小的 Worker 容量，并使用独立的 OS 资源控制。JavaScript 和 WASM 在进程内运行；WASM 没有执行超时，且这两个引擎都不会在运行中的调用内响应取消。 + ### 本地 Python UDF 外部访问 diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md index 1ac7bc8138..43ea8a87ef 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md @@ -4,7 +4,7 @@ sidebar_position: 1 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + 创建用户自定义聚合函数（UDAF），在 Databend 内置的 JavaScript 或 Python 运行时中执行。 @@ -36,7 +36,7 @@ $$ | `STATE { }` | 定义聚合过程中需要在部分聚合和最终聚合步骤间持久化的状态结构（例如 `STATE { sum DOUBLE, count DOUBLE }`）。 | | `` | 聚合返回的数据类型。 | | `LANGUAGE` | 运行脚本的语言，支持 `javascript`、`python`。 | -| `IMPORTS` / `PACKAGES` | 可选列表，用于加载额外文件（imports）或 PyPI 包（仅限 Python）。 | +| `IMPORTS` / `PACKAGES` | 语句语法接受这些子句，但本地 Python 聚合函数不会安装包或导入文件。不要依赖这些子句提供 Python UDAF 依赖。 | | `` | 脚本主体，必须实现并暴露 `create_state`、`accumulate`、`merge` 和 `finish` 入口函数。 | | `DESC` | 可选描述。 | @@ -47,6 +47,17 @@ $$ - `merge(state1, state2)` – 合并两个部分聚合状态。 - `finish(state)` – 生成最终结果（返回 `None` 表示 SQL `NULL`）。 +## 运行时限制 + +从 v1.2.957 开始，Python UDAF 与 Python 标量 UDF 使用相同的查询专属 Worker 池、OS 沙箱和配额。聚合状态保留在 Query 进程中，并随 Worker 请求传递。Worker 不会跨查询复用。 + +- 在创建、绑定和执行时，`enable_udf_python_script` 都必须允许 Python。 +- `enable_udf_sandbox = true` 会为 Python 标量函数选择外部服务。该服务没有聚合 API，因此此模式下会拒绝 Python UDAF。此设置不能绕过强制使用的本地 OS 沙箱。 +- Linux 部署和依赖安装边界请参阅 [CREATE SCALAR FUNCTION](ddl-create-function.md#python)。macOS 仅用于本地开发，不提供 OS 沙箱。 +- Python Worker 数量和请求超时不会强制设置 Worker 内存上限。请参阅[脚本 UDF 资源](/guides/self-hosted/references/node-config/query-config#script-udf-resources)。 +- JavaScript UDAF 使用配置的进程内堆限制和操作截止时间，覆盖 `create_state`、`accumulate`、`merge` 和 `finish`。 +- 不支持 WASM 聚合函数。 + ## 访问控制要求 | 权限 | 对象类型 | 描述 | diff --git a/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md b/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md index 51e4b4c571..ff113f9e75 100644 --- a/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md +++ b/docs/cn/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md @@ -4,7 +4,7 @@ sidebar_position: 0 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + 创建标量用户自定义函数（Scalar UDF）。同一条 `CREATE FUNCTION` 语句支持以下三种实现方式： @@ -106,6 +106,23 @@ SELECT calculate_bmi(70.0, 1.75) AS bmi; Python 运行时需要企业版。你可以用 `PACKAGES` 安装 PyPI 包，用 `IMPORTS` 引入 Stage 文件（例如模型、依赖压缩包）。 +### Worker 沙箱与依赖 + +从 v1.2.955 开始，本地 Python 标量 UDF 在查询专属的 Worker 进程中运行，不再共享 Query 进程的 Python 解释器。每个本地运行时按需创建 Worker，并在一次 SQL 执行内复用，不跨查询复用。并发调用遵循 SQL `max_threads`，同时受节点和租户容量约束。额外的空闲 Worker 默认在 300 秒后回收；每个运行时保留一个最低 Worker。这些进程边界与 v1.2.957 引入的 OS 沙箱不同。 + +从 v1.2.957 开始，本地 Python Worker 必须先进入 OS 沙箱，再执行用户代码。Linux 要求内核版本不低于 5.13，活动 LSM 列表中启用 Landlock，支持 seccomp，并使用 x86_64 或 aarch64。容器的 seccomp 配置必须允许 `landlock_*` 系统调用。沙箱设置失败时，执行会报错 `Python UDF sandbox is unavailable, refusing to run user code`。本地沙箱没有绕过开关。 + +- Worker 只接收允许列表中的环境变量，不会继承 Query 节点的完整环境。每个 Worker 都有私有可写目录，用于临时文件、依赖和 Stage 导入。 +- Python 安装目录和所需系统库只读。禁止读取 Query 配置/数据文件和其他进程的环境文件。路径元数据查询仍可能暴露路径是否存在。 +- 用户代码不能打开原生网络 Socket、启动子程序/子进程，或向 Query 进程发送信号。通过 integration 访问 HTTP(S) 的方法见[受控 HTTP(S) 访问](#受控-https-访问)。 +- macOS Worker 不使用 OS 沙箱，并会记录警告。macOS 仅适用于本地开发，不适合执行不可信 UDF。其他非 Linux 平台拒绝本地 Python 执行。 + +依赖安装在 Query 节点上执行，不在 Worker 沙箱中。依赖必须使用包名称及可选的 extras/版本约束，例如 `numpy>=1.26`。pip 选项、URL、直接引用、本地路径和环境标记会被拒绝。运维配置的 pip 镜像/代理和 CA 设置仍通过允许列表获得支持。 + +`python_udf_pip_only_binary = true` 是默认值，只安装 wheel 包。将它设为 `false` 会允许源码分发包在 Query 节点上执行构建脚本。不要认为 Worker 沙箱能够防护依赖安装代码。 + +本地 Python 标量和聚合函数共用 Worker 准入、排队和请求超时控制。Worker 属于一次查询执行，不会跨查询复用。Python Worker 内存不计入查询分配器，进程数量限制也不能约束 Worker 总内存。请使用 OS 资源控制实现内存隔离。配置方法和超时范围请参阅[脚本 UDF 资源](/guides/self-hosted/references/node-config/query-config#script-udf-resources)。 + ### 受控 HTTP(S) 访问 @@ -170,6 +187,8 @@ SELECT numpy_sqrt(9.0) AS sqrt_val; ## JavaScript +从 v1.2.957 开始，每个进程内 JavaScript 运行时的默认堆限制为 1024 MiB，操作截止时间为 300 秒。该期限覆盖参数/结果转换、异步延续和模块顶层代码。这些限制不会为 Python Worker 设置内存上限。请参阅[脚本 UDF 资源](/guides/self-hosted/references/node-config/query-config#script-udf-resources)。 + ### 数据类型映射（JavaScript） | Databend 类型 | JavaScript 类型 | @@ -215,6 +234,8 @@ $$; WASM UDF 支持使用 Rust 实现标量函数，并在 Databend 中直接运行编译后的模块。 +从 v1.2.957 开始，每个 WASM 实例的默认线性内存限制为 1024 MiB。`udf_script_execution_timeout_secs` 仅适用于 JavaScript；WASM 尚无执行超时。这两个进程内引擎都不会在运行中的调用内响应查询取消。不要认为这些控制可以中断 WASM 无限循环。 + WASM 模块必须使用 Arrow UDF ABI 3.0，可通过导出的 `ARROWUDF_VERSION_3_0` 标识确认。ABI 3.0 从 `arrow-udf` 0.3.0 开始引入，并使用 `int32` 等 Arrow 类型名。更早的模块使用 `int4` 等旧类型名，与当前的类型化函数查找不兼容；高于 3 的 ABI 版本也不受当前运行时支持。 以下示例将构建并注册一个计算最大公约数的函数。 diff --git a/docs/en/guides/20-self-hosted/04-references/node-config/query-config.md b/docs/en/guides/20-self-hosted/04-references/node-config/query-config.md index 96a05845bd..fe1c069c48 100644 --- a/docs/en/guides/20-self-hosted/04-references/node-config/query-config.md +++ b/docs/en/guides/20-self-hosted/04-references/node-config/query-config.md @@ -6,7 +6,7 @@ import FunctionDescription from '@site/src/components/FunctionDescription'; import LanguageDocs from '@site/src/components/LanguageDocs'; import DetailsWrap from '@site/src/components/DetailsWrap'; - + This page describes the Query node configurations available in the [databend-query.toml](https://github.com/databendlabs/databend/blob/main/scripts/distribution/configs/databend-query.toml) configuration file. @@ -55,6 +55,31 @@ For a UDF with explicit integrations, a nonempty config list remains an operator For SQL setup and the trust boundary, see [External Access Integrations](/guides/security/external-access-integration). +### Script UDF Resources + +These node configuration options belong in `[query]`, not `[query.settings]`. Query-owned scalar Python workers and automatic worker-capacity selection were introduced in v1.2.955. Since v1.2.957, local Python UDFs and UDAFs use the mandatory OS sandbox described in [CREATE SCALAR FUNCTION](/sql/sql-commands/ddl/udf/ddl-create-function#python). + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `enable_udf_python_script` | `true` | Allows local Python scripts, including aggregates. | +| `enable_udf_sandbox` | `false` | Routes scalar Python functions to an external sandbox service. Python UDAFs are rejected in that mode. This does not disable the local OS sandbox. | +| `python_udf_max_workers` | `0` | Node worker capacity. `0` uses `query.num_cpus`, or available CPU parallelism if `num_cpus = 0`. | +| `python_udf_max_workers_per_tenant` | `0` | Tenant capacity on this node. `0` inherits the node capacity. | +| `python_udf_max_workers_per_query` | `0` | Maximum resident workers for a query. `0` uses `M + P - 1`, where `M` is its local runtime count and `P` is effective parallelism. | +| `python_udf_max_queued_queries` | `128` | Maximum admission waiters. Must be positive. | +| `python_udf_queue_timeout_secs` | `60` | Admission wait timeout in seconds. Must be positive. | +| `python_udf_execution_timeout_secs` | `300` | Timeout of each worker request, including interpreter startup on the first request. Must be positive. | +| `python_udf_worker_idle_timeout_secs` | `300` | Idle timeout for extra workers. Keeps one minimum worker per runtime; it does not expire busy workers. Must be positive. | +| `python_udf_pip_only_binary` | `true` | Installs wheels only. `false` permits source-distribution build scripts on the query node, outside the worker sandbox. | +| `udf_script_memory_limit_mb` | `1024` | JavaScript heap per runtime and WASM linear memory per instance, in MiB. `0` means unlimited. Does not limit Python worker memory. | +| `udf_script_execution_timeout_secs` | `300` | JavaScript operation deadline, including module initialization and conversion. `0` means unlimited. Not enforced for WASM. | + +Python capacities are local-node limits, not warehouse-wide limits. Nonzero configured worker limits must satisfy `query <= tenant <= node`. Effective parallelism follows SQL `max_threads`, bounded by node and tenant capacity. One minimum worker is admitted per registered runtime; additional workers grow on demand. A query budget that cannot fit its minimum runtimes fails instead of waiting forever. + +The Python request timeout is not a total SQL timeout. The normal query timeout still controls total execution. Cancellation or a failed/timed-out worker request poisons that runtime; batches are not retried automatically. Query completion terminates its workers. Workers are never shared with a later query. + +Process counts and request timeouts do not provide OS-enforced Python memory or CPU budgets. Worker memory is outside the query allocator. Use explicit smaller worker capacities for memory-heavy packages and separate OS resource controls. JavaScript and WASM run in-process; WASM has no execution timeout, and neither engine observes cancellation during a running call. + ### Local Python UDF External Access diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md b/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md index 87a8401639..ae1db47a32 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-aggregate-function.md @@ -4,7 +4,7 @@ sidebar_position: 1 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + Creates a user-defined aggregate function (UDAF) that runs inside Databend's JavaScript or Python runtime. @@ -36,7 +36,7 @@ $$ | `STATE { }` | Struct definition that Databend stores between partial/final aggregation steps (for example `STATE { sum DOUBLE, count DOUBLE }`). | | `` | Data type returned by the aggregate (`DOUBLE`, `INT`, etc.). | | `LANGUAGE` | Runtime used to execute the script. Supported values: `javascript`, `python`. | -| `IMPORTS` / `PACKAGES` | Optional lists for shipping extra files (imports) or PyPI packages (Python only). | +| `IMPORTS` / `PACKAGES` | Accepted by the statement syntax, but local Python aggregates do not install packages or imports. Do not rely on these clauses for Python UDAF dependencies. | | `` | Script body that must expose `create_state`, `accumulate`, `merge`, and `finish` entry points. | | `DESC` | Optional description. | @@ -47,6 +47,17 @@ The script must implement these functions: - `merge(state1, state2)` – merge two partial states. - `finish(state)` – produce the final result (return `None` for SQL `NULL`). +## Runtime Restrictions + +Since v1.2.957, Python UDAFs run in the same query-owned worker pool, OS sandbox, and quotas as scalar Python UDFs. Aggregate state remains in the query process and travels with worker requests. Workers are not reused across queries. + +- `enable_udf_python_script` must allow Python at creation, binding, and execution. +- `enable_udf_sandbox = true` selects an external service for scalar Python functions. That service has no aggregate API, so Python UDAFs are rejected in that mode. This setting is not a bypass for the mandatory local OS sandbox. +- Linux deployment and dependency-installation boundaries are described in [CREATE SCALAR FUNCTION](ddl-create-function.md#python). macOS is for local development only and does not provide the OS sandbox. +- Python worker count and request timeouts do not enforce a worker memory limit. See [Script UDF Resources](/guides/self-hosted/references/node-config/query-config#script-udf-resources). +- JavaScript UDAFs use the configured in-process heap and operation deadline, including `create_state`, `accumulate`, `merge`, and `finish`. +- WASM aggregate functions are not supported. + ## Access control requirements | Privilege | Object Type | Description | diff --git a/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md b/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md index 8218e9a605..cf9e26ced8 100644 --- a/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md +++ b/docs/en/sql-reference/10-sql-commands/00-ddl/10-udf/ddl-create-function.md @@ -4,7 +4,7 @@ sidebar_position: 0 --- import FunctionDescription from '@site/src/components/FunctionDescription'; - + Creates a scalar user-defined function (Scalar UDF). The same `CREATE FUNCTION` statement supports three implementation styles: @@ -105,6 +105,23 @@ SELECT calculate_bmi(70.0, 1.75) AS bmi; Python runtime requires Databend Enterprise. You can install PyPI packages via `PACKAGES` and import stage files via `IMPORTS`. +### Worker Sandbox and Dependencies + +Since v1.2.955, local Python scalar UDFs run in query-owned worker processes instead of sharing the Query process's Python interpreter. Each local runtime creates workers on demand and reuses them within one SQL execution, never across queries. Concurrent calls follow SQL `max_threads`, bounded by node and tenant capacity. Extra idle workers retire after 300 seconds by default; one minimum worker per runtime is retained. These process boundaries are separate from the OS sandbox added in v1.2.957. + +Since v1.2.957, local Python workers must enter an OS sandbox before user code runs. Linux requires kernel 5.13 or later, Landlock enabled in the active LSM list, seccomp, and x86_64 or aarch64. Container seccomp profiles must permit the `landlock_*` syscalls. If sandbox setup fails, execution fails with `Python UDF sandbox is unavailable, refusing to run user code`. There is no local sandbox bypass switch. + +- Workers receive an allowlisted environment, not the query node's full environment. Each worker has a private writable directory for temporary files, dependencies, and stage imports. +- Python installations and required system libraries are read-only. Access to query configuration/data files and other processes' environment files is denied. Path metadata lookup can still reveal that a path exists. +- User code cannot open native network sockets, start child programs/processes, or signal the query process. For integration-bound HTTP(S), see [Controlled HTTP(S) Access](#controlled-https-access). +- macOS workers run without the OS sandbox and log a warning. Use macOS only for local development, not for untrusted UDFs. Other non-Linux platforms refuse local Python execution. + +Dependency installation runs on the query node, outside the worker sandbox. Requirements must use package names with optional extras/version specifiers, such as `numpy>=1.26`. pip options, URLs, direct references, local paths, and environment markers are rejected. Operator-configured pip mirrors/proxies and CA settings remain supported through an allowlist. + +`python_udf_pip_only_binary = true` is the default. Only wheel packages are installed. Setting it to `false` permits source distributions to execute build scripts on the query node. Do not treat the worker sandbox as protection from dependency-installation code. + +Local Python scalar and aggregate functions share worker admission, queue, and request-timeout controls. Workers belong to one query execution and are not reused across queries. Python worker memory is not charged to the query allocator, and process limits do not bound total worker memory. Use OS resource controls for memory containment. See [Script UDF Resources](/guides/self-hosted/references/node-config/query-config#script-udf-resources) for configuration and timeout scope. + ### Controlled HTTP(S) Access @@ -169,6 +186,8 @@ SELECT numpy_sqrt(9.0) AS sqrt_val; ## JavaScript +Since v1.2.957, each in-process JavaScript runtime has a default heap limit of 1024 MiB and an operation deadline of 300 seconds. The deadline covers argument/result conversion, async continuations, and top-level module code. These limits do not impose a Python worker memory limit. See [Script UDF Resources](/guides/self-hosted/references/node-config/query-config#script-udf-resources). + ### Data type mappings (JavaScript) | Databend Type | JavaScript Type | @@ -214,6 +233,8 @@ $$; WASM UDFs let you implement scalar functions in Rust and run the compiled module directly in Databend. +Since v1.2.957, each WASM instance has a default linear-memory limit of 1024 MiB. `udf_script_execution_timeout_secs` applies only to JavaScript; WASM does not yet have an execution timeout. Neither in-process engine observes query cancellation during a running call. Do not assume these controls can interrupt an infinite WASM loop. + WASM modules must use Arrow UDF ABI 3.0, identified by the `ARROWUDF_VERSION_3_0` export. ABI 3.0 was introduced in `arrow-udf` 0.3.0 and uses Arrow type names such as `int32`. Earlier modules use legacy type names such as `int4` and are not compatible with the current typed function lookup. Modules that introduce an ABI version above 3 are not supported by the current runtime. The following example builds and registers a greatest common divisor function.