Skip to content

Commit bc6d2bb

Browse files
authored
feat(peering): treat authenticated peers as feeds (#1034)
<!--- Provide a general summary of your changes in the Title above --> ## Description <!--- Describe your changes in detail --> This PR :- - Adds \`limits.peer\` with separate subscription caps and message rate limits for authenticated peering peers. - Peer REQ, COUNT and NIP-77 filter handling use peer caps instead of client caps; a full-feed subscription no longer trips \`max_subscriptions\` or the client EWMA message bucket. - \`limits.peer.subscription.maxLimit\` defaults to 0 (no cap), so a peer can request a full feed. - Peer messages are counted in a separate rate-limit bucket; defaults remain high but finite. - Only a verified AUTH from an enabled \`peering.peers\` pubkey selects peer limits. Unauthenticated and unconfigured connections keep the existing client limits exactly as before. - Documents and validates the new limits. ## Related Issue <!--- This project only accepts pull requests related to open issues --> <!--- If suggesting a new feature or change, please discuss it in an issue first --> <!--- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!--- Please link to the issue here: --> Closes :- #934 ## Motivation and Context <!--- Why is this change required? What problem does it solve? --> ## How Has This Been Tested? <!--- Please describe in detail how you tested your changes. --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> ## Screenshots (if appropriate): ## Types of changes <!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Non-functional change (docs, style, minor refactor) - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Checklist: <!--- Go over all the following points, and put an `x` in all the boxes that apply. --> - [ ] My code follows the code style of this project. - [ ] My change requires a change to the documentation. - [ ] I have updated the documentation accordingly. - [ ] I have read the **CONTRIBUTING** document. - [ ] I have added tests to cover my code changes. - [ ] I added a changeset, or this is docs-only and I added an empty changeset. - [ ] All new and existing tests passed. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## Summary by CodeRabbit * **New Features** * Authenticated connections from configured peers now use separate subscription and message-rate limits; other connections retain client limits. * Peer defaults allow up to 100 subscriptions, 100 filters, and 10,000 filter values. Subscription IDs can be up to 256 characters. * Peer feeds have no result-limit cap by default and exclude deleted or expired events. Filter counts are governed by connection-specific limits, checked before duplicate filters are removed. * Feed reads wait for pending sends to complete, helping accommodate slower connections. * **Documentation** * Added guidance on peer feed limits and documented available settings and defaults. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents c358a6a + e8e8f22 commit bc6d2bb

23 files changed

Lines changed: 495 additions & 39 deletions
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
"nostream": minor
3+
---
4+
5+
feat(peering): treat authenticated peers as feeds, not clients
6+
7+
Adds `limits.peer`, a separate limit set for connections that have proven a
8+
configured `peering.peers` pubkey with NIP-42 AUTH. Peer subscriptions use
9+
`limits.peer.subscription` instead of `limits.client.subscription`, and peer
10+
messages use `limits.peer.message` in a separate rate-limit bucket, so a
11+
full-feed subscription cannot trip the client EWMA limiter or the client
12+
`max_subscriptions` cap.
13+
14+
Peers are eligible for full feeds by default: `maxLimit` is uncapped and the
15+
subscription caps are higher. An omitted peer REQ `limit` queries the full
16+
stored feed instead of the client 500-event default, and peer filter counts are
17+
not rejected by the protocol's former fixed ten-filter cap. Raw filter counts are
18+
still bounded before deduplication, and stored reads wait for the socket's
19+
send-flush callback so a slow peer cannot grow worker memory. Deleted and expired rows
20+
are excluded in SQL before the limit, so they cannot consume a page slot.
21+
22+
A high but finite peer message rate keeps the credential from becoming an
23+
unbounded egress path. Unauthenticated or unconfigured connections keep the
24+
existing client limits exactly as before. The new limits are validated with the
25+
rest of the settings document and documented in `CONFIGURATION.md` and
26+
`docs/PEERING.md`.

‎CONFIGURATION.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,13 @@ The settings below are listed in alphabetical order by name. Please keep this ta
195195
| limits.client.subscription.maxFilters | Maximum number of filters per subscription. Defaults to 10. Disabled when set to zero. |
196196
| limits.message.rateLimits[].period | Rate limiting period in milliseconds. For `sliding_window`: the time window. For `ewma`: the half-life of the decay function. |
197197
| limits.message.rateLimits[].rate | Maximum number of messages during period. |
198+
| limits.peer.message.rateLimits[].period | Rate limit period in milliseconds for authenticated peering peers. Separate from `limits.message`, so a peer feed never spends client message budget. |
199+
| limits.peer.message.rateLimits[].rate | Maximum peer messages during period. Defaults to 6000/min. |
200+
| limits.peer.subscription.maxFilters | Maximum number of filters per peer subscription. Defaults to 100. Disabled when set to zero. |
201+
| limits.peer.subscription.maxFilterValues | Maximum number of values allowed in each peer filter. Defaults to 10000. Disabled when set to zero. |
202+
| limits.peer.subscription.maxLimit | Maximum REQ `limit` for peers. Defaults to 0 (no cap), so a peer can request a full feed. |
203+
| limits.peer.subscription.maxSubscriptionIdLength | Maximum peer subscription id length. Defaults to 256. |
204+
| limits.peer.subscription.maxSubscriptions | Maximum number of subscriptions per authenticated peer. Defaults to 100. Disabled when set to zero. |
198205
| mirroring.static | Removed: the static mirroring worker no longer exists, so this setting is ignored and logs a warning at start-up. To sync events with another relay, use NIP-77 (Negentropy) from a client that speaks it, for example `strfry sync`, against a relay with `nip77.enabled` set. |
199206
| network.maxPayloadSize | Maximum number of bytes accepted per WebSocket frame |
200207
| network.remoteIpHeader | HTTP header from proxy containing IP address from client. |

‎docs/PEERING.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,34 @@ authenticated for the caller to use for a feed or negentropy session. This
8181
proves our identity to the peer; it does not verify the peer's identity in
8282
return.
8383

84+
## Feed treatment
85+
86+
Once a connection has proven a configured peer pubkey, its subscription and
87+
message limits come from `limits.peer`, not `limits.client` or `limits.message`:
88+
89+
- Peer subscription caps (`maxSubscriptions`, `maxFilters`, `maxFilterValues`,
90+
`maxSubscriptionIdLength`) are separate and larger by default.
91+
- `limits.peer.subscription.maxLimit: 0` leaves REQ `limit` uncapped and makes an
92+
omitted `limit` query the full stored feed instead of the client-sized
93+
500-event default. A positive value becomes the peer's default page size.
94+
- Peer filter counts are bounded by `limits.peer.subscription.maxFilters`, not
95+
the client cap, so a full-feed filter set is not rejected before the handler
96+
can apply peer limits. The raw count is checked before duplicate removal, so
97+
a large set of duplicate filters cannot stall the worker in `uniqWith`.
98+
- Stored reads wait for the WebSocket send callback to report the previous
99+
frame as flushed, so a slow peer cannot grow the worker's outbound memory.
100+
Soft-deleted and expired events are excluded in SQL before the limit, so dead
101+
rows cannot consume a page slot.
102+
- Peer messages are counted in a separate rate-limit bucket, so a feed never
103+
trips the client EWMA limiter. The default is a high but finite bound, not an
104+
exemption.
105+
- Only a verified AUTH from an enabled `peering.peers` pubkey selects these
106+
limits. An unauthenticated or unconfigured connection stays on the client
107+
limits exactly as before.
108+
109+
Scope enforcement (only the agreed kinds and time ranges) is part of the sync
110+
work; this phase grants feed-sized limits to trusted peers.
111+
84112
Only the public half of the keypair is ever visible on the wire. Inbound AUTH is
85113
verified by `AuthMessageHandler`: event id, signature, timestamp, challenge, and
86114
relay host. Outbound AUTH proves this relay's identity to the peer; it does not

‎resources/default-settings.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -409,6 +409,25 @@ limits:
409409
maxSubscriptionIdLength: 256
410410
maxLimit: 5000
411411
minPrefixLength: 4
412+
peer:
413+
# Configured peering peers get feed treatment, not client treatment. These
414+
# caps are separate from limits.client and limits.message, so a full-feed
415+
# subscription does not trip the client EWMA bucket or max_subscriptions.
416+
# A value of 0 disables that cap.
417+
subscription:
418+
maxSubscriptions: 100
419+
maxFilters: 100
420+
maxFilterValues: 10000
421+
maxSubscriptionIdLength: 256
422+
# 0 = no cap, so a peer can request a full feed.
423+
maxLimit: 0
424+
minPrefixLength: 4
425+
message:
426+
# period is milliseconds (60000 = one minute)
427+
rateLimits:
428+
- description: 6000 peer messages/min
429+
period: 60000
430+
rate: 6000
412431
message:
413432
# period is milliseconds (60000 = one minute)
414433
rateLimits:

‎src/@types/adapters.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,12 @@ export type IWebSocketAdapter = EventEmitter & {
2828
getPeerPubkey(): string | undefined
2929
/** Mark `pubkey` as the peer identity for this socket. Ignored if it is not authenticated. */
3030
setPeerPubkey(pubkey: string): void
31+
/**
32+
* Resolves once the previous send has flushed to the underlying socket, or
33+
* immediately when there is nothing pending. Used to pause a full-feed read
34+
* while the client is slow.
35+
*/
36+
waitForWritable(): Promise<void>
3137
drainAndClose(reason?: string): void
3238
/** NIP-77 sessions of this connection. */
3339
getNegentropySessions(): INegentropySessions

‎src/@types/repositories.ts‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,12 +40,25 @@ export interface IQueryResult<T> extends Pick<Promise<T>, keyof Promise<T> & Exp
4040
stream(options?: Record<string, any>): PassThrough & AsyncIterable<T>
4141
}
4242

43+
export interface EventQueryOptions {
44+
/**
45+
* Limit applied to filters that omit `limit`. Defaults to
46+
* DEFAULT_FILTER_LIMIT. `0` means no limit, used for peer full feeds.
47+
*/
48+
defaultLimit?: number
49+
/**
50+
* Exclude soft-deleted and expired rows in SQL, before the limit is applied.
51+
* Peer full feeds set this so dead rows cannot consume a page slot.
52+
*/
53+
excludeDeleted?: boolean
54+
}
55+
4356
export interface IEventRepository {
4457
create(event: Event): Promise<number>
4558
createMany(events: Event[]): Promise<number>
4659
upsert(event: Event): Promise<number>
4760
upsertMany(events: Event[]): Promise<number>
48-
findByFilters(filters: SubscriptionFilter[]): IQueryResult<DBEvent[]>
61+
findByFilters(filters: SubscriptionFilter[], options?: EventQueryOptions): IQueryResult<DBEvent[]>
4962
countByFilters(filters: SubscriptionFilter[], options?: { timeoutMs?: number }): Promise<number>
5063
findReconciliationItems(filter: SubscriptionFilter, limit?: number): IQueryResult<ReconciliationItem[]>
5164
deleteByPubkeyAndIds(pubkey: Pubkey, ids: EventId[]): Promise<number>

‎src/@types/settings.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,15 @@ export interface ClientLimits {
163163
subscription?: ClientSubscriptionLimits
164164
}
165165

166+
/**
167+
* Limits applied to configured peering peers instead of client limits. Peer
168+
* feeds need larger caps than end-user clients, while still being bounded.
169+
*/
170+
export interface PeerLimits {
171+
subscription?: ClientSubscriptionLimits
172+
message?: MessageLimits
173+
}
174+
166175
export interface MessageRateLimit extends RateLimit {
167176
type?: MessageType
168177
}
@@ -206,6 +215,7 @@ export interface Limits {
206215
admin?: AdminLimits
207216
connection?: ConnectionLimits
208217
client?: ClientLimits
218+
peer?: PeerLimits
209219
event?: EventLimits
210220
message?: MessageLimits
211221
}

‎src/adapters/web-socket-adapter.ts‎

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import { getRemoteAddress } from '../utils/http'
2828
import { createReadAuthorizationGuard } from '../utils/nip42'
2929
import { Nip42SessionManager } from '../utils/nip42-session'
3030
import { isConfiguredPeeringPeer } from '../utils/peering-auth'
31+
import { messageLimitsFor } from '../utils/peer-limits'
3132
import { IRateLimiter } from '../@types/utils'
3233
import { isEventMatchingFilter } from '../utils/event'
3334
import { isHidden } from '../utils/hidden-content-cache'
@@ -74,6 +75,7 @@ export class WebSocketAdapter extends EventEmitter implements IWebSocketAdapter
7475
public clientId: string
7576
private clientAddress: SocketAddress
7677
private alive: boolean
78+
private pendingSend: Promise<void> | undefined
7779
private subscriptions: Map<SubscriptionId, SubscriptionFilter[]>
7880
private readonly session: Nip42SessionManager
7981
/** Configured peering peer proven by AUTH on this socket, if any. */
@@ -192,7 +194,28 @@ export class WebSocketAdapter extends EventEmitter implements IWebSocketAdapter
192194
if (this.client.readyState !== WebSocket.OPEN) {
193195
return
194196
}
195-
this.client.send(JSON.stringify(message))
197+
198+
// `ws` does not emit `drain`; the send callback is its write-completion
199+
// signal, fired once the frame has been handed to the underlying socket.
200+
const pending = new Promise<void>((resolve) => {
201+
this.client.send(JSON.stringify(message), () => resolve())
202+
})
203+
this.pendingSend = pending
204+
void pending.then(() => {
205+
if (this.pendingSend === pending) {
206+
this.pendingSend = undefined
207+
}
208+
})
209+
}
210+
211+
public waitForWritable(): Promise<void> {
212+
if (this.client.readyState !== WebSocket.OPEN) {
213+
return Promise.resolve()
214+
}
215+
216+
// A full feed can outrun a slow peer; wait for the previous frame to flush
217+
// instead of letting the outbound buffer grow without bound.
218+
return this.pendingSend ?? Promise.resolve()
196219
}
197220

198221
public onHeartbeat(): void {
@@ -342,16 +365,20 @@ export class WebSocketAdapter extends EventEmitter implements IWebSocketAdapter
342365
* the wrong one to throttle.
343366
*/
344367
private async isRateLimited(client: string, type: string): Promise<false | number> {
345-
const { rateLimits, ipWhitelist = [] } = this.settings().limits?.message ?? {}
368+
const isPeer = this.getPeerPubkey() !== undefined
369+
const { rateLimits, ipWhitelist = [] } = messageLimitsFor(this.settings(), isPeer) ?? {}
346370

347371
if (!Array.isArray(rateLimits) || !rateLimits.length || ipWhitelist.includes(client)) {
348372
return false
349373
}
350374

351375
const rateLimiter = this.rateLimiter()
352376

377+
// Peer buckets are separate so a connection that spent client budget before
378+
// AUTH does not carry that score into feed treatment.
379+
const bucket = isPeer ? 'peer-message' : 'message'
353380
const hit = (period: number, rate: number) =>
354-
rateLimiter.hit(`${client}:message:${type}:${period}`, 1, { period, rate })
381+
rateLimiter.hit(`${client}:${bucket}:${type}:${period}`, 1, { period, rate })
355382

356383
for (const { rate, period } of rateLimits) {
357384
const result = await hit(period, rate)

‎src/handlers/count-message-handler.ts‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import { createLogger } from '../factories/logger-factory'
1111
import { countFilterValues } from '../utils/filter'
1212
import { createClosedMessage, createCountResultMessage } from '../utils/messages'
1313
import { isCountAuthorized } from '../utils/nip42'
14+
import { subscriptionLimitsFor } from '../utils/peer-limits'
1415

1516
const debug = createLogger('count-message-handler')
1617

@@ -29,9 +30,24 @@ export class CountMessageHandler implements IMessageHandler {
2930
return
3031
}
3132

32-
// Some clients send the same filter more than once.
33-
// We remove duplicates so we do less DB work.
34-
const filters = uniqWith(equals, message.slice(2)) as SubscriptionFilter[]
33+
// Some clients send the same filter more than once. Bound the raw list
34+
// before deduplicating it, since uniqWith() is O(n^2) and the schema no
35+
// longer caps filter counts so peer limits can be larger.
36+
const incomingFilters = message.slice(2) as SubscriptionFilter[]
37+
const maxFilters =
38+
subscriptionLimitsFor(this.settings(), this.webSocket.getPeerPubkey() !== undefined)?.maxFilters ?? 0
39+
if (maxFilters > 0 && incomingFilters.length > maxFilters) {
40+
this.webSocket.emit(
41+
WebSocketAdapterEvent.Message,
42+
createClosedMessage(
43+
queryId,
44+
`Too many filters: Number of filters per count query must be less than or equal to ${maxFilters}`,
45+
),
46+
)
47+
return
48+
}
49+
50+
const filters = uniqWith(equals, incomingFilters) as SubscriptionFilter[]
3551

3652
const reason = this.canCount(queryId, filters)
3753
if (reason) {
@@ -52,7 +68,10 @@ export class CountMessageHandler implements IMessageHandler {
5268
}
5369

5470
private canCount(queryId: SubscriptionId, filters: SubscriptionFilter[]): string | undefined {
55-
const subscriptionLimits = this.settings().limits?.client?.subscription
71+
const subscriptionLimits = subscriptionLimitsFor(
72+
this.settings(),
73+
this.webSocket.getPeerPubkey() !== undefined,
74+
)
5675
const maxFilters = subscriptionLimits?.maxFilters ?? 0
5776

5877
if (maxFilters > 0 && filters.length > maxFilters) {

‎src/handlers/neg-open-message-handler.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import { WebSocketAdapterEvent } from '../constants/adapter'
1212
import { countFilterValues } from '../utils/filter'
1313
import { createLogger } from '../factories/logger-factory'
1414
import { isReconciliationAuthorized } from '../utils/nip42'
15+
import { subscriptionLimitsFor } from '../utils/peer-limits'
1516

1617
const logger = createLogger('neg-open-message-handler')
1718

@@ -122,7 +123,10 @@ export class NegOpenMessageHandler implements IMessageHandler {
122123
initialMessage: Buffer,
123124
): string | undefined {
124125
const settings = this.settings()
125-
const subscriptionLimits = settings.limits?.client?.subscription
126+
const subscriptionLimits = subscriptionLimitsFor(
127+
settings,
128+
this.webSocket.getPeerPubkey() !== undefined,
129+
)
126130

127131
const maxFilterValues = subscriptionLimits?.maxFilterValues ?? 0
128132
if (maxFilterValues > 0 && countFilterValues(filter) > maxFilterValues) {

0 commit comments

Comments
 (0)