Year and Month transforms depend on the input array's timezone tag; Iceberg computes them in UTC

[andygrove]

Apache Iceberg Rust version

main @ 3d84c81353b1b23b6e4ae8eea8f8a021cc6927a7

Describe the bug

Year::transform and Month::transform in crates/iceberg/src/transform/temporal.rs extract the calendar field with Arrow's date_part:

impl TransformFunction for Year {
    fn transform(&self, input: ArrayRef) -> Result<ArrayRef> {
        let array = date_part(&input, DatePart::Year)?;
        ...unary(|v| v - UNIX_EPOCH_YEAR)
    }
}

date_part is timezone-aware: for a Timestamp(Microsecond, Some(tz)) array it converts each value into tz before reading the year. Iceberg computes all four temporal transforms in UTC — DateTimeUtil.convertMicros builds an OffsetDateTime at ZoneOffset.UTC regardless of anything else. So for an array carrying a non-UTC tag the two disagree.

Day and Hour are not affected — they do floor division on the raw epoch value.

This is also internally inconsistent. transform_literal on the same two structs does the UTC thing:

(PrimitiveType::Timestamptz, PrimitiveLiteral::Long(v)) => Self::timestamp_to_year_micros(*v)?
// -> DateTime::from_timestamp_micros(v).year() - UNIX_EPOCH_YEAR

DateTime::from_timestamp_micros is UTC, so transform and transform_literal return different values for the same instant when the array is tagged non-UTC. Predicate projection and partition-value computation would then disagree with each other.

In practice iceberg-rust's own callers pass arrays built from an Iceberg schema, where Timestamptz is tagged +00:00, so this is latent rather than actively broken inside the crate. It is reachable for any external caller handing the transform an Arrow array it built itself.

To Reproduce

let tagged: ArrayRef = Arc::new(
    TimestampMicrosecondArray::from(vec![-1i64]).with_timezone("Asia/Kathmandu"),
);
let years = create_transform_function(&Transform::Year).unwrap().transform(tagged).unwrap();
// yields 0; Iceberg Java's DateTimeUtil.microsToYears(-1) is -1

-1 micros is 1969-12-31T23:59:59.999999Z, which is 1970-01-01T05:44:59.999999 in Kathmandu, so date_part reads year 1970 and the transform returns 0 where Iceberg returns -1. Any tag with a non-zero offset produces a similar disagreement for values near a year or month boundary.

Expected behavior

Year and Month computed in UTC for every input, matching DateTimeUtil and matching their own transform_literal. Computing from the epoch value directly — the way Day and Hour already do — gives that. Normalising the array to UTC before calling date_part would also work but keeps a dependency on the tag being present and correct.

Two smaller things in the same area, if they are worth folding in:

• Year::transform / Month::transform inherit chrono's date range through date_part, which stops at about year 262143, while Java's LocalDate covers the whole i32 epoch-day domain (i32::MAX days is +5881580-07-11). Epoch days beyond that are representable in an Arrow Date32 but not handled here.
• Year::transform has no Date32 arm distinct from the timestamp ones; it relies on date_part accepting both. Fine today, just worth noting if the kernel is rewritten.

Willingness to contribute

I would be willing to contribute a fix for this bug with guidance from the Iceberg community.

---

How this was found. Apache DataFusion Comet is adding native kernels for Iceberg's Spark system functions (apache/datafusion-comet#5638). We cross-check those kernels against create_transform_function over boundary inputs so a future iceberg-rust bump cannot silently desynchronise a partitioned write. The timezone-tag dependency is the reason years and months could not be delegated to iceberg-rust while bucket, days, and hours could.

This report was drafted with LLM assistance (Claude Code); the behaviour was verified against Iceberg Java on a JVM.