Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ All notable changes are recorded here. Versions follow
The 0.1.0 entry describes what the library is rather than what changed, since
there was nothing to have changed from. Everything above it is ordinary.

## 0.22.3 — 2026-10-20

### Fixed

- **A reader that passes no `s3_options` opens a published table again.**
0.22.1's manifest read handed pyiceberg no properties when a reader passed
no options, rather than resolving them from the environment as litelink
does, so a non-AWS endpoint (`AWS_ENDPOINT_URL`) never reached it and the
open asked AWS itself (`ACCESS_DENIED`). Readers passing explicit options
were unaffected.

## 0.22.2 — 2026-10-20

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "streamcast"
version = "0.22.2"
version = "0.22.3"
description = "A JSON WebSocket pubsub framework for structured data. Every message lands in an Iceberg table: no ETL, no second copy."
readme = "README.md"
authors = [
Expand Down
6 changes: 5 additions & 1 deletion src/streamcast/_published.py
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,11 @@ def _counted_from_manifests(
column, or a delete file whose rows a count would include — and the
caller asks DuckDB as before.
"""
properties = {} if s3_options is None else s3_options.catalog_properties()
# Resolved, as litelink resolves its own: what the caller did not set
# comes from the environment, the endpoint included. Unresolved, a reader
# that passes no options gave pyiceberg none, and a non-AWS endpoint in
# the environment never reached it (0.22.1).
properties = (s3_options or S3Options()).resolved().catalog_properties()
table = StaticTable.from_metadata(metadata, properties=properties)
field = table.schema().find_field(COLUMN)
low: int | None = None
Expand Down
27 changes: 27 additions & 0 deletions tests/test_published.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,33 @@ def test_an_s3_table_reads_with_the_readers_own_credentials(tmp_path, s3, bucket
table.close()


@pytest.mark.replication
def test_an_s3_table_opened_without_options_reads_with_the_environments(
tmp_path, s3, bucket
):
"""A reader that passes no `s3_options` reads with what the environment
resolves (the `s3` fixture puts its endpoint and key there), the extent
read from the manifests included: 0.22.1 handed pyiceberg no properties,
so a non-AWS endpoint in the environment never reached it and the open
asked AWS itself (ACCESS_DENIED)."""
log = litelink.new(
tmp_path, "trades", schema=SCHEMA, published=bucket, s3_options=s3
)
with log:
log.extend([{"i": i} for i in range(4)])
while log.seal(flush=True) is not None:
pass

log.publish(flush=True)

table = _published.Table.open(bucket, "trades", None)
try:
assert table.extent == (1, 5)
assert table.record_count == 4
finally:
table.close()


def publish(log: litelink.WriteHandle, start: int, count: int) -> None:
log.extend([{"i": i} for i in range(start, start + count)])
while log.seal(flush=True) is not None:
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading