Skip to content

fix(rest-catalog): serialize purgeRequested as lowercase boolean string - #3837

Open
swjtu-zhanglei wants to merge 1 commit into
apache:mainfrom
swjtu-zhanglei:main-fix
Open

fix(rest-catalog): serialize purgeRequested as lowercase boolean string#3837
swjtu-zhanglei wants to merge 1 commit into
apache:mainfrom
swjtu-zhanglei:main-fix

Conversation

@swjtu-zhanglei

Copy link
Copy Markdown
  • Python's requests library serializes bool True as "True" (capitalized) in query parameters. Per OpenAPI 3.0 spec (JSON Schema Wright Draft 00, RFC 7159 Section 3), boolean query parameters must be serialized as lowercase "true"/"false". Servers that strictly validate boolean values (e.g., Aliyun OSS Tables) reject "True" with 400 Bad Request.

  • Use explicit "true"/"false" string literals for spec-compliant wire format.

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

* Python's requests library serializes bool True as "True" (capitalized)
in query parameters. Per OpenAPI 3.0 spec (JSON Schema Wright Draft 00,
RFC 7159 Section 3), boolean query parameters must be serialized as
lowercase "true"/"false". Servers that strictly validate boolean values
(e.g., Aliyun OSS Tables) reject "True" with 400 Bad Request.

* Use explicit "true"/"false" string literals for spec-compliant wire format.

@rambleraptor rambleraptor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused by this. Doesn't requests do this for us? We aren't doing this anywhere else. Why is this the only place we're seeing this issue?

@swjtu-zhanglei

Copy link
Copy Markdown
Author

I'm a little confused by this. Doesn't requests do this for us? We aren't doing this anywhere else. Why is this the only place we're seeing this issue?

Thanks for the question. I traced through the full code path — requests does not handle boolean serialization for us. Here's the detailed
evidence.

The full call chain with source references

Step 1: requests/models.py:473 — entry point
enc_params = self._encode_params(params)
Step 2: requests/models.py:107-134 — _encode_params
@staticmethod
def _encode_params(data):
...
elif hasattr(data, "iter"):
result = []
for k, vs in to_key_val_list(data):
if isinstance(vs, basestring) or not hasattr(vs, "iter"):
vs = [vs] # True → [True]
for v in vs:
if v is not None:
result.append(
(
k.encode("utf-8") if isinstance(k, str) else k,
v.encode("utf-8") if isinstance(v, str) else v, # ← bool is NOT str, kept as-is
)
)
return urlencode(result, doseq=True) # ← passes raw bool to stdlib
Key point at line 129: v.encode("utf-8") if isinstance(v, str) else v — a Python bool is not a str, so it is not encoded/converted here. The
raw True object is passed through to urlencode.

Step 3: urllib/parse.py:980 — stdlib urlencode(query, doseq=True)

With doseq=True, the code enters the branch at line 1029. For each (k, v) pair:
# urllib/parse.py:1036-1048
if isinstance(v, bytes): # True is not bytes → skip
v = quote_via(v, safe)
l.append(k + '=' + v)
elif isinstance(v, str): # True is not str → skip
v = quote_via(v, safe, encoding, errors)
l.append(k + '=' + v)
else:
try:
itr = iter(v) # iter(True) → TypeError!
except TypeError:
v = quote_via(str(v), safe, encoding, errors) # ← str(True) = "True" (capital T)
l.append(k + '=' + v)
The smoking gun is line ~1047: str(v) where v is Python True → produces "True" (capitalized). There is no boolean-aware branch anywhere in this
path.

Why this is the only place we see the issue

A grep for params={ in pyiceberg/catalog/rest/init.py shows drop_table is the only call site that passes a Python bool as a query param
value. All other params (snapshotId, branch, tag, etc.) are strings, which hit the isinstance(v, str) branch and are handled correctly.

Other Iceberg implementations get this right

iceberg-go (catalog/rest/rest.go:1838,1864):
v.Set("purgeRequested", "false")
v.Set("purgeRequested", "true")
iceberg-rust (crates/catalog/rest/src/catalog.rs:738):
request_builder = request_builder.query(&[("purgeRequested", "true")]);
Both use explicit lowercase string literals — no implicit bool-to-string conversion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants