[sundy-li]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

• Converge nullable construction and validity handling around Column::wrap_nullable, with a checked typed NullableColumn::new. Make the inner column and bitmap private, remove new_column / new_unchecked, and migrate consumers to accessors, destructure and merge_validity. Wrapping an already-nullable column intersects masks without adding a layer; Null columns stay Null. Reject mismatched lengths and invalid Nullable/Null inner types at construction boundaries, including dynamic/generic columns and Arrow conversion.
• Fix scalar-returning nullable adapters losing mixed row masks. Typed/dynamic passthrough and combine adapters, TRY/error-to-null wrappers and context validity now preserve per-row nullness, expanding a scalar into a column only when the mask is mixed. Update nullable tuple-field extraction and cast paths to compose validity through the same entry point. The original three regexp golden differences affected only payload bytes in invalid rows, not visible rows or validity. After merging current main, regexp.txt is byte-identical to main and is no longer part of this PR's diff; the regexp golden and partial-evaluation regression tests pass without regeneration.
• Migrate expression kernels, Arrow/legacy conversions, formats, joins, sorting, indexes, statistics, virtual columns and storage consumers as required by the private fields and removed constructors. Legacy nested nullable columns normalize by intersecting validity. No persisted schema/version change, configuration toggle or Python UDF networking change is included. This is independent of feat(query): control Python UDF egress with external integrations #20898.
• Regression coverage in expression/tests/it/nullable.rs and types.rs exercises empty/all-valid/all-null/mixed masks, outer context, scalar/column results, output validity intersection, inactive-row errors, TRY wrappers, repeated wrapping, Null columns, legacy normalization and invalid constructors/builders. Add 02_0013_nullable_construction.test for casts, nullable tuple fields and outer joins, for example:

  SELECT id, to_nullable(to_nullable(a)), try_to_int64(b), get(1)(tuple(a, b))
  FROM nullable_construction_regression ORDER BY id;

• Conflict refresh: merge main at 1bb457513f685b40aea560b557a59893a493e1b8 without rewriting branch history. Preserve main's encoded-dictionary/NDV behavior and existing nullable error-to-null domains, retain this PR's scalar mixed-validity handling, and adapt newly added decimal/encoded/vector-index callers to the private Nullable API. No toolchain updates or Python UDF networking branch changes are included.
• Validation after the main merge: all 165 expression integration tests, 107 expression unit tests, 257 functions integration tests, and 28 formats tests passed (one formats test ignored), plus the 3 targeted regexp tests, 7 Parquet encoding tests, 2 vector-bundle tests and 15 HTTP serialization tests. Strict affected-crate clippy, Rustfmt and diff checks passed; no golden files were regenerated. Full Dev CI remains held during review.
• Prior branch validation: refreshed expression/functions/formats suites (566 passing tests, one ignored), cargo fmt --all -- --check, and cargo clippy -p databend-common-expression -p databend-common-functions -p databend-common-formats -p databend-common-pipeline-transforms -p databend-storages-common-index -p databend-common-storages-fuse -p databend-query --tests -- -D warnings passed. Refreshed the pipeline integration suite (7 tests), Parquet encoding tests (7 tests) and HTTP serialization tests (15 tests); rebuilt the query binary and executed 51 SQL logic records from the new construction regression plus existing nullable/tuple/try_cast files against an isolated local query node using a temporary HTTP record-checking script. The official sqllogictest runner was not used for this local SQL check. Full-workspace tests and performance benchmarks were not run; no performance claim is made.

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

AI assistance

• AI usage: An AI coding agent drafted the nullable construction/validity refactor, regression tests and PR summary, and ran local validation; the responsible human reviewed the complete diff and confirmed submission.
• Responsible human: @sundy-li
• The responsible human has read every line of this diff and can explain each change

---

This change is