Skip to content

Commit 0e409a0

Browse files
owenpearsonclaude
andcommitted
test: derive the REST client, request and fallback unit specs
Covers fallback.md, rest_client.md, request.md, request_endpoint.md, stats.md, logging.md and channels_collection.md. Five tests record where the SDK departs from a specification and carry the deviation mark: the four covering log_level and log_handler, which Options stores and nothing reads, and RSC19b, where caller-supplied headers override the configured Authorization. Six more cannot be observed through the injected transport at all and are skipped outright, the connectivity check among them. deviations.md carries the reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent d7aaa69 commit 0e409a0

8 files changed

Lines changed: 2484 additions & 0 deletions

File tree

‎test/uts/deviations.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,85 @@ site. The entries below cover both kinds and say which applies.
3737
The three sections that follow this one record SDK behaviour rather than specification
3838
faults.
3939

40+
### `/time` is stubbed as an object rather than an array
41+
42+
`fallback.md` (~14 setups), `rest_client.md` (6), `request_endpoint.md` (4),
43+
`token_renewal.md` and `authorize.md` stub `/time` as `{"time": N}`. The endpoint
44+
returns a one-element array, which is what `time.md` itself uses and what RSC16
45+
describes. Any SDK that indexes the array raises.
46+
47+
### Fallback tests contradict each other and the retry rules
48+
49+
- `RSC15f/successful-fallback-cached-0` queues a response for one named fallback and
50+
asserts it is chosen, which contradicts `RSC15a/fallback-random-order-0` in the
51+
same file.
52+
- `RSC19d/response-status-code-0` and `response-success-indicator-1` queue a single
53+
500, but RSC15l3 requires a 5xx to be retried against every fallback host, so the
54+
first attempt consumes it.
55+
- `RSC15l4`'s CloudFront body `{"error": "Forbidden"}` is not a valid Ably error
56+
shape; real CloudFront returns HTML.
57+
- Error bodies throughout omit `message` and `statusCode` while tests assert
58+
`error.statusCode`. An SDK that reads the status from the payload cannot satisfy both.
59+
60+
### Smaller faults
61+
62+
| Spec | Fault |
63+
|---|---|
64+
65+
| `channels_collection.md` | Header claims RSN3b and RSN3c; neither has a test |
66+
| `stats.md` | Fixture nests counts under `all`, which `Stats.from_dict` never reads |
67+
| `rest_client.md` | `RSC17` has two byte-identical tests; header lists RSC7 and RSC7b with no tests |
68+
| `rest_client.md` | `RSC18` requires constructor-time failure; RSA1/RSC18 only say "any attempt to use" |
69+
| `request.md` | `version` is written as an integer but lands in a header |
70+
| `fallback.md` | REC3a, REC3b and REC3 drive a Realtime client but sit in `rest/unit` |
71+
4072
## Failing Tests
4173

4274
The specification's assertion is preserved and gated behind `@deviation`. Removing
4375
the mark is the only change needed once the SDK behaviour lands.
4476

77+
### Unimplemented features
78+
79+
| Spec points | Missing | Tests |
80+
|---|---|---|
81+
82+
| RSC2, RSC3, RSC4, TO3b, TO3c, TO3c2 | `log_handler` as a client option, and any use of `log_level` — it is stored on `Options` and read by nothing | 4 |
83+
84+
### Requests
85+
86+
| Spec points | Behaviour |
87+
|---|---|
88+
| RSC19b | Caller-supplied headers override the configured `Authorization`, because `Http.make_request` applies `headers` after `auth_headers`. RSC19b says requests "unconditionally" use the configured mechanism |
89+
4590
## Adapted Tests
4691

4792
The test asserts what the SDK does, with the specification's expectation in a
4893
comment above. These run, so they guard against regression.
4994

95+
| Spec points | Specification | ably-python | Status |
96+
|---|---|---|---|
97+
98+
| RSC1b | Error code 40106 | A bare `ValueError` from `AblyRest.__init__` with an informative message, not an `AblyException`, so there is no code | Open bug |
99+
| RSC18 | The constructor rejects basic auth over HTTP | Construction succeeds; 40103 is raised from `make_request` when a request needing Basic Auth is attempted, and no request goes out. RSA1/RSC18 say only "any attempt to use" | Compliant; the UTS is stricter than its source |
100+
| REC1b1, REC1c1 | Code 40000, or a message containing "invalid" or "conflict" | 400/40106 with a specific message. The features spec mandates no code | Cosmetic |
101+
102+
| RSC19e | An error indicated idiomatically | `httpx.ConnectError` / `ReadTimeout` reach the caller unwrapped, because `AblyRest.request` carries no `@catch_all` unlike `time()` and `stats()`. The messages do name the failure | Borderline; defensible under RSC19e |
103+
| RSC15a | Six hosts tried | Three. `Options.__get_hosts` truncates to `http_max_retry_count`, which TO3l5 sanctions | Intentional |
104+
50105
## Mock Infrastructure Limitations
106+
107+
Tests that cannot be implemented as written. The first is caused by the SDK, not by
108+
the mock, but it lands here because the effect is the same: no test can observe the
109+
behaviour.
110+
111+
### The connectivity check bypasses the injected transport — 3 tests
112+
113+
`ConnectionManager.check_connection` is internal, synchronous, and calls
114+
`httpx.get` directly. `REC3a`, `REC3b` and `REC3` are skipped. These specs drive a
115+
Realtime client and belong under `realtime/unit` in any case.
116+
117+
### `fallbackHostsUseDefault` is not implemented — 3 tests
118+
119+
Optional per TO3k7, and `REC1b1` and `REC2a1` scope their checks to libraries that
120+
support it, so these are skipped as not applicable rather than recorded as
121+
deviations.
Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
"""Derived from uts/rest/unit/channels_collection.md in ably/specification.
2+
3+
Spec points: RSN1, RSN2, RSN3a, RSN4a, RSN4b
4+
5+
The collection is held in memory, so no request reaches the mock. A mock is still
6+
installed so that nothing in these tests can touch the network.
7+
"""
8+
9+
import uuid
10+
11+
from ably.rest.channel import Channel, Channels
12+
from test.uts.helpers.client import rest_client
13+
from test.uts.helpers.mock_http import MockHttpClient
14+
15+
16+
def random_id():
17+
return uuid.uuid4().hex[:8]
18+
19+
20+
def offline_client():
21+
mock_http = MockHttpClient(
22+
on_connection_attempt=lambda conn: conn.respond_with_success(),
23+
on_request=lambda request: request.respond_with(200, []),
24+
)
25+
return rest_client(mock_http)
26+
27+
28+
# UTS: rest/unit/RSN1/channels-collection-accessible-0
29+
async def test_rsn1_channels_collection_accessible():
30+
client = offline_client()
31+
32+
assert isinstance(client.channels, Channels)
33+
assert client.channels is not None
34+
35+
36+
# UTS: rest/unit/RSN2/check-channel-exists-0
37+
async def test_rsn2_check_channel_exists():
38+
channel_name = f'test-RSN2-{random_id()}'
39+
client = offline_client()
40+
41+
# NOTE: the spec spells this channels.exists(name); Python spells membership
42+
# as `in`, which Channels implements via __contains__.
43+
exists_before = channel_name in client.channels
44+
45+
client.channels.get(channel_name)
46+
47+
exists_after = channel_name in client.channels
48+
49+
other_channel_name = f'test-RSN2-other-{random_id()}'
50+
exists_other = other_channel_name in client.channels
51+
52+
assert exists_before is False
53+
assert exists_after is True
54+
assert exists_other is False
55+
56+
57+
# UTS: rest/unit/RSN2/iterate-channels-1
58+
async def test_rsn2_iterate_channels():
59+
channel_name_a = f'test-RSN2-a-{random_id()}'
60+
channel_name_b = f'test-RSN2-b-{random_id()}'
61+
channel_name_c = f'test-RSN2-c-{random_id()}'
62+
client = offline_client()
63+
64+
client.channels.get(channel_name_a)
65+
client.channels.get(channel_name_b)
66+
client.channels.get(channel_name_c)
67+
68+
channel_names = [channel.name for channel in client.channels]
69+
70+
assert channel_name_a in channel_names
71+
assert channel_name_b in channel_names
72+
assert channel_name_c in channel_names
73+
assert len(channel_names) == 3
74+
75+
76+
# UTS: rest/unit/RSN3a/get-creates-new-channel-0
77+
async def test_rsn3a_get_creates_new_channel():
78+
channel_name = f'test-RSN3a-{random_id()}'
79+
client = offline_client()
80+
81+
channel = client.channels.get(channel_name)
82+
83+
assert isinstance(channel, Channel)
84+
assert channel.name == channel_name
85+
assert (channel_name in client.channels) is True
86+
87+
88+
# UTS: rest/unit/RSN3a/get-returns-existing-channel-1
89+
async def test_rsn3a_get_returns_existing_channel():
90+
channel_name = f'test-RSN3a-existing-{random_id()}'
91+
client = offline_client()
92+
93+
channel1 = client.channels.get(channel_name)
94+
channel2 = client.channels.get(channel_name)
95+
96+
assert channel1 is channel2
97+
assert channel1.name == channel_name
98+
99+
100+
# UTS: rest/unit/RSN3a/subscript-creates-or-returns-2
101+
async def test_rsn3a_subscript_creates_or_returns():
102+
channel_name = f'test-RSN3a-subscript-{random_id()}'
103+
client = offline_client()
104+
105+
channel1 = client.channels[channel_name]
106+
channel2 = client.channels.get(channel_name)
107+
channel3 = client.channels[channel_name]
108+
109+
assert channel1 is channel2
110+
assert channel2 is channel3
111+
assert channel1.name == channel_name
112+
113+
114+
# UTS: rest/unit/RSN4a/release-removes-channel-0
115+
async def test_rsn4a_release_removes_channel():
116+
channel_name = f'test-RSN4a-{random_id()}'
117+
client = offline_client()
118+
119+
client.channels.get(channel_name)
120+
assert (channel_name in client.channels) is True
121+
122+
client.channels.release(channel_name)
123+
124+
assert (channel_name in client.channels) is False
125+
126+
127+
# UTS: rest/unit/RSN4b/release-nonexistent-noop-0
128+
async def test_rsn4b_release_nonexistent_noop():
129+
channel_name = f'test-RSN4b-{random_id()}'
130+
client = offline_client()
131+
132+
client.channels.release(channel_name)
133+
134+
assert (channel_name in client.channels) is False
135+
136+
137+
# UTS: rest/unit/RSN3a/get-after-release-new-instance-3
138+
async def test_rsn3a_get_after_release_new_instance():
139+
channel_name = f'test-RSN3a-release-{random_id()}'
140+
client = offline_client()
141+
142+
channel1 = client.channels.get(channel_name)
143+
144+
client.channels.release(channel_name)
145+
146+
channel2 = client.channels.get(channel_name)
147+
148+
assert channel1 is not channel2
149+
assert channel2.name == channel_name
150+
assert (channel_name in client.channels) is True

0 commit comments

Comments
 (0)