[sundy-li]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

• Add controlled HTTP(S) access for local Python scalar/aggregate Script UDFs through existing tenant-scoped NETWORK RULE / EXTERNAL ACCESS INTEGRATION objects and USAGE checks. Preserve enable_udf_sandbox routing and reject unsupported external-access bindings at CREATE/ALTER and invocation.
• Operator opt-in, default deny: query.enable_python_udf_external_access defaults to false, independently of remote UDF transport flags. Missing query config fails closed. After opt-in, the nonempty udf_server_allow_list remains a hostname-only ceiling; every tenant HOST_PORT rule must additionally approve the destination and port.
• Validate all actual DNS answers before any TCP connect and connect only to the checked socket addresses. Non-public addresses (including loopback, metadata/link-local, unspecified, private/shared/reserved, multicast and IPv4-mapped IPv6) are rejected unless the operator explicitly lists the exact IP literal. Hostname entries cannot grant IP exceptions; mixed public/non-public DNS answers fail closed. Deprecated IPv4-compatible/site-local and well-known IPv6 translation ranges require explicit IP exceptions too. IP exceptions retain the ceiling's host-only/all-ports semantics; tenant HOST_PORT authorization still applies.
• A private per-worker HTTP/CONNECT transport removes hop headers/proxy credentials and authorizes independently of the worker's Host header. Workers always retain kernel direct-network denial; no unrestricted-network config/CLI/SQL setting is exposed. Support synchronous proxy-aware urllib/requests, not async/native TCP/UDP clients. CONNECT is an opaque stream to an approved endpoint, not TLS/HTTP payload or SNI enforcement; approved endpoints must not be untrusted relays. Host-side package installation remains separate.
• Bind with EXTERNAL_ACCESS_INTEGRATIONS = (events_access) on Python CREATE/ALTER, including aggregate definitions. Binding requires USAGE on every integration. Metadata reader version 192 and a distinct distributed-plan variant reject old readers/executors. Serialization selects that variant even if callers bypass UDFType::script().
• Active/idle HTTP streams and CONNECT tunnels recheck policy every second with five-second check deadlines, after asynchronous meta-watch propagation. Disable/drop integration or change/drop rules to revoke existing egress; revoking USAGE only prevents future bindings/ALTERs. Five-minute connection lifetime; five-second worker-to-proxy request-header and DNS/connect deadlines; upstream response headers use the overall lifetime/client cancellation, not a separate header deadline. Plaintext request/response bodies are limited to 16 MiB, with at most 32 host streams per worker.
• Single Databend PR: supersedes closed feat(query): add admin Python UDF network exception #20874 and targets main directly. Landing remains blocked on feat(python-worker): add sandboxed HTTP proxy transport datafuse-extras/arrow-udf#5: merge that dependency and repin to its merged commit before merging this PR. Current pin is ccb8b79e075836acd1ce833aaf1c2e87b2c69b99; the dependency is still Draft and its macOS worker tests currently fail. Do not merge feat(query): add admin Python UDF network exception #20874. Nullable refactor(expression): unify nullable construction and validity #20905 remains independent.
• Review follow-up tests cover default-off/CLI/TOML opt-in, missing config, CREATE/ALTER and invocation route checks, post-DNS IP rejection/operator exceptions, direct-constructor serialization, and tenant rule denial below an allowed hostname ceiling. Proxy integration tests use the production public transport factory from tests/it/, without compiling production code via include! or environment-specific probe filtering. SQL regression polling has bounded deadlines rather than fixed propagation sleeps, and redirect/direct denial use different ports on the same operator-approved host.
• Local follow-up validation passed: operator CLI/TOML/default-off config; 39 worker unit tests, production-factory proxy/live-revocation tests and the real worker process suite; all 195 SQL unit tests and 43 semantic/type-check tests; 6 external-access DDL tests and 7 remote UDF transport tests. Strict affected-crate clippy with Python enabled, Script-only and feature-disabled clippy, Python-enabled query binary build, full Taplo/Rustfmt, shell syntax/shfmt and bounded-polling output checks passed. An isolated query binary passed actual SQL suites 08_0006_network_policy and 08_0007_external_access with exact .result output; separate fresh-node runs verified that omitting the opt-in rejects CREATE and that an enabled node with an empty ceiling rejects a tenant-approved loopback target. Existing query/meta PIDs were preserved. Prior runs also exercised urllib/requests HTTPS and two-node execution; those earlier results do not substitute for validating this review follow-up. Full Dev CI remains held during review. Full-workspace tests, platform runtime coverage and performance benchmarks have not been run.

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

AI assistance

• AI usage: An AI coding agent drafted the implementation, review fixes, regression tests and documentation, and ran local validation; the responsible human reviewed the complete updated diff and confirmed submission.
• Responsible human: @sundy-li
• The responsible human has read every line of this diff and can explain each change

---

This change is

[github-actions]

⏱ CI timing · Dev

run 41046786248 · 46.0 min (main p50 63.3 min, −17.3 min)

stage	this run	main p50	Δ
total	46.0 min	63.3 min	−17.3 min
build (aarch64, ci profile)	34.4 min	23.4 min	+11.0 min ⚠️
├ deps + crates (parallel)	17.7 min	—	—
├ databend-query codegen	2.8 min	—	—
├ databend-binaries link	11.8 min	—	—
└ sccache hit rate	95.8 %	—	—
test_unit	20.3 min	17.6 min	+2.7 min
mac_check	—	13.1 min	—
tests after build (49 jobs)	11.0 min	30.4 min	−19.4 min
├ queue p50 / max	2.2 min / 4.4 min	4.9 min / 26.9 min	−2.7 min
├ sqllogic standalone (8 jobs, max)	4.4 min	4.3 min	+0.1 min
├ sqllogic cluster (8 jobs, max)	8.3 min	7.2 min	+1.1 min
├ sqllogic stage (2 jobs, max)	5.5 min	5.0 min	+0.5 min
├ sqllogic other (10 jobs, max)	5.8 min	5.7 min	+0.1 min
├ stateless (2 jobs, max)	6.7 min	6.7 min	+0.0 min
├ stateful (4 jobs, max)	4.8 min	4.8 min	−0.0 min
├ compat (6 jobs, max)	6.0 min	5.7 min	+0.4 min
├ meta / logs / lineage / udf (5 jobs, max)	4.7 min	4.4 min	+0.3 min
├ ee (2 jobs, max)	2.0 min	1.3 min	+0.7 min
├ private_tasks (1 jobs, max)	8.9 min	8.8 min	+0.0 min
└ fuzz (1 jobs, max)	1.4 min	1.1 min	+0.3 min

_{Minutes unless stated; durations exclude queue time unless stated; "max" = longest job in the group. Baseline = median of the last 10 successful runs (Dev: Main on main; Cloud: recent Cloud runs). Build breakdown has no baseline. Updated on each push.}