[Feature] Support Fluss as ActionStateStore backend#628
[Feature] Support Fluss as ActionStateStore backend#628beryllw wants to merge 3 commits intoapache:mainfrom
Conversation
7c1fc60 to
0aee165
Compare
|
Thanks for working on this. I’m also looking into Fluss as an ActionStateStore backend, so I’ve been comparing different designs here. One question on the PK table approach: with synchronous upsert in put() and no-op rebuildState(), it seems backend upsert/lookup may sit on the hot path. For new seqNums, get() will usually need to hit Fluss, while the Kafka-style design keeps get() local by rebuilding post-checkpoint state into an in-memory cache. Did you consider using a Log Table instead, plus local cache + recovery marker + changelog rebuild? My main concern is the latency tradeoff of putting PK-table lookup/upsert directly on the action execution path. |
xintongsong
left a comment
There was a problem hiding this comment.
LGTM, except for the code conflicts.
@Sxnan, as one of the contributor of the original Kafka action state store, could you also help check on this PR?
4d6865f to
3f8697b
Compare
Sxnan
left a comment
There was a problem hiding this comment.
Thanks for the PR! I left a few comments belows.
Linked issue: #627
Purpose of change
Uses Fluss log table + in-memory HashMap (similar to Kafka approach):
Table schema: (
state_key STRING, state_payload BYTES, agent_key STRING), distributed byagent_key.Tests
API
Documentation
doc-neededdoc-not-neededdoc-included