Skip to content

fix(Record): wildcard for missing partition keys in where()#2

Merged
gedera merged 1 commit intomainfrom
fix/where-partial-partition-keys
Apr 16, 2026
Merged

fix(Record): wildcard for missing partition keys in where()#2
gedera merged 1 commit intomainfrom
fix/where-partial-partition-keys

Conversation

@gedera
Copy link
Copy Markdown
Contributor

@gedera gedera commented Apr 16, 2026

Summary

  • Fix where() generating invalid paths with empty values (key=) when not all partition_keys are specified
  • Now uses wildcards (key=*) for missing partition keys, matching behavior of destroy_partitions
  • Added tests for partial partition key scenarios

Changes

  • lib/data_drain/record.rb:134 - Use wildcard when value is nil or empty
  • spec/data_drain/record_spec.rb - 2 new test cases

Closes #1

Copy link
Copy Markdown
Contributor Author

@gedera gedera left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: fix(Record): wildcard for missing partition keys in where()

Verdict: LGTM

Overview

Fix correcto para issue #1. build_query_path generaba paths con valores vacíos (key=) cuando where() recibía partition keys parciales. Ahora usa wildcards (key=*), consistente con destroy_partitions en local adapter (línea 51).

Correctness

  • Lógica val.nil? || val.to_s.empty? cubre ambos casos (key ausente → nil, key presente pero vacía).
  • Wildcard * funciona tanto para Dir.glob (local) como para read_parquet de DuckDB con el sufijo /**/*.parquet que agrega build_path.
  • Consistente con patrón existente en Storage::Local#destroy_partitions.

Un detalle menor (no bloqueante)

val = partitions[k.to_sym] || partitions[k.to_s]

El || trata valores falsy (false, 0) como ausentes. Si partitions[:month] fuera 0, caería al string key lookup y potencialmente generaría wildcard. Para partition keys reales (strings, integers positivos) no es problema práctico — pero destroy_partitions en local adapter usa partitions[key] directo sin doble lookup, así que hay una asimetría menor en el manejo de keys.

Considerar para futuro:

val = partitions.key?(k.to_sym) ? partitions[k.to_sym] : partitions[k.to_s]

Tests

  • 2 nuevos specs cubren partial keys y all-missing. Adecuado.
  • 23 specs pasan, 0 failures.
  • RuboCop clean.

Risk

Bajo. Cambio backward-compatible — paths que antes tenían valores siguen igual, solo cambia el caso de valores ausentes (que antes generaba paths rotos).

When where() is called without specifying all partition_keys, the path
now uses 'key=*' wildcards instead of empty values like 'key='.

Closes #1
@gedera gedera force-pushed the fix/where-partial-partition-keys branch from 2cbd1e2 to 6523103 Compare April 16, 2026 15:20
@gedera gedera merged commit eca85bf into main Apr 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

where() genera path con valores vacíos cuando no se usan todas las partition_keys

1 participant