Repository navigation
feat(cardano): add Dijkstra sub transactions and ledger fields - #215
geofflittle wants to merge 7 commits into
Conversation
- Tx: sub transactions, deposits, balance intervals - Tx: guards and required top level guards - scripts: guard clause and guarding redeemer purpose - PoolRegistrationCert BLS key and Dijkstra PParams
- v1beta cardano.proto: the fields added to v1 - same message names, field names and numbers
- v1beta/cardano.proto: restored to main, the fields stay in v1
| uint64 max_endorser_block_references_size = 43; // The maximum endorser block references size. | ||
| uint64 max_endorser_block_txs_size = 44; // The maximum endorser block transactions size. | ||
| ExUnits max_endorser_block_execution_units = 45; // The maximum execution units per endorser block. | ||
| uint64 max_ref_script_size_per_endorser_block = 46; // The maximum reference script size per endorser block. |
There was a problem hiding this comment.
There was a problem hiding this comment.
The six Peras parameters and the two reference input fees are now PParams fields 47 to 54 and Genesis fields 58 to 65, so PParams has a field for each entry of the record you linked.
- v1/cardano.proto: PParams fields 47 to 54 from the Dijkstra ledger - slot interval is uint64, as the epoch interval parameters - bootstrap round Word32 is optional uint64, as the Word32 sizes
There was a problem hiding this comment.
Nice work.
However, this PR makes Tx recursive, which does not match the CDDL: a sub transaction could then have its own sub transactions, which the ledger does not allow.
I suggest a common message type with the fields shared by top-level and sub transactions - the comment on sub_transactions has a draft.
| bytes hash = 13; // Hash of the transaction that serves as main identifier | ||
| repeated GovernanceActionProposal proposals = 14; // List of governance actions proposed | ||
| repeated VoterVotes votes = 15; // List of voters and their corresponding votes cast in this transaction | ||
| repeated Tx sub_transactions = 16; // List of sub transactions, in body order |
There was a problem hiding this comment.
Tx also cannot carry several transaction body fields, and in Dijkstra sub transactions carry them too.
That gap predates this PR, so I opened an issue for it:
| repeated Tx sub_transactions = 16; // List of sub transactions, in body order | ||
| repeated DirectDeposit direct_deposits = 17; // List of deposits into reward accounts | ||
| repeated AccountBalanceInterval account_balance_intervals = 18; // List of account balance intervals | ||
| repeated AccountBalanceInterval starting_account_balance_intervals = 19; // List of starting account balance intervals | ||
| repeated StakeCredential guards = 20; // List of credentials in body key 14, the required signers from Alonzo and the guards from Dijkstra | ||
| repeated RequiredTopLevelGuard required_top_level_guards = 21; // List of credentials the top level transaction must hold in its guards |
There was a problem hiding this comment.
Reusing Tx for sub transactions lets a sub transaction carry fields that only a top-level transaction has: fee, collateral, successful, sub_transactions and starting_account_balance_intervals.
successful is the risky one, because it reads as false on every sub transaction unless the server copies the top-level flag.
A ledger sub transaction has none of these fields, and cardano-api keeps the shared fields in one type with the top-level-only ones apart:
- https://github.com/IntersectMBO/cardano-ledger/blob/a6f303ade80b2f4729403861546db828a41e6408/eras/dijkstra/impl/cddl/data/dijkstra.cddl#L829-L849
- Add Dijkstra fields to experimental TxBodyContent and draft SubTx type IntersectMBO/cardano-api#1361
I suggest moving the shared fields into their own message:
// The part of a transaction that top-level and sub transactions share.
message TxCommon {
repeated TxInput inputs = 1; // Resolved against the UTxO before the batch.
repeated TxOutput outputs = 2; // Referenced by this transaction's own hash and their index.
repeated Certificate certificates = 3;
repeated Withdrawal withdrawals = 4;
repeated Multiasset mint = 5;
repeated TxInput reference_inputs = 6; // Resolved against the UTxO before the batch.
WitnessSet witnesses = 7;
optional TxValidity validity = 8;
optional AuxData auxiliary = 9;
bytes hash = 10; // Hash of this transaction's own body.
repeated GovernanceActionProposal proposals = 11;
repeated VoterVotes votes = 12;
repeated StakeCredential guards = 13; // Required signers before Dijkstra, guards from Dijkstra.
repeated RequiredTopLevelGuard required_top_level_guards = 14;
repeated DirectDeposit direct_deposits = 15;
repeated AccountBalanceInterval account_balance_intervals = 16; // Checked against the balances this transaction sees.
}
message Tx {
TxCommon common = 1;
optional Collateral collateral = 2; // Covers every script of the batch.
BigInt fee = 3; // Pays for the whole batch.
bool successful = 4; // When false, only the collateral is collected and no sub transaction is applied.
repeated TxCommon sub_transactions = 5; // In body order; sub transactions do not nest.
repeated AccountBalanceInterval starting_account_balance_intervals = 6; // Checked against the balances before the batch.
}Fields marked optional are optional in the ledger, and the repeated ones are empty when absent.
TxValidity.start and TxValidity.ttl are plain uint64, so an unset bound reads as 0, and they should be optional too.
Clients and field masks would then address the shared fields through common, and TxPattern would match against common of the transaction and of each sub transaction.
The fields from #218 would also go in TxCommon.
There was a problem hiding this comment.
Done. The shared fields are in TxCommon, which Tx holds as common, and each sub transaction is a TxCommon. TxValidity fields are optional too.
| repeated bytes pool_owners = 7; // List of pool owner key hashes. | ||
| repeated Relay relays = 8; // List of relays. | ||
| PoolMetadata pool_metadata = 9; // Pool metadata. | ||
| PoolBlsKey bls_key = 10; // Pool BLS key. |
There was a problem hiding this comment.
The ledger has bls_key as optional, so I suggest declaring it optional PoolBlsKey bls_key = 10;, as TxInput.redeemer and TxOutput.datum are.
The wire format stays the same, because a message field already has presence.
| uint64 max_ref_script_size_per_block = 32; // The maximum reference script size per block. | ||
| uint64 max_ref_script_size_per_tx = 33; // The maximum reference script size per transaction. | ||
| uint64 ref_script_cost_stride = 34; // The reference script cost stride. | ||
| RationalNumber ref_script_cost_multiplier = 35; // The reference script cost multiplier. | ||
| RationalNumber max_pledge_leverage = 36; // The maximum pledge leverage. | ||
| RationalNumber min_pool_margin = 37; // The minimum pool margin. | ||
| uint64 leios_announcement_period_length = 38; // The Leios announcement period length in milliseconds. | ||
| uint64 leios_vote_period_length = 39; // The Leios vote period length in milliseconds. | ||
| uint64 leios_diffusion_period_length = 40; // The Leios diffusion period length in milliseconds. | ||
| uint32 leios_committee_size = 41; // The Leios committee size. | ||
| RationalNumber leios_quorum_stake_threshold = 42; // The Leios quorum stake threshold. | ||
| uint64 max_endorser_block_references_size = 43; // The maximum endorser block references size. | ||
| uint64 max_endorser_block_txs_size = 44; // The maximum endorser block transactions size. | ||
| ExUnits max_endorser_block_execution_units = 45; // The maximum execution units per endorser block. | ||
| uint64 max_ref_script_size_per_endorser_block = 46; // The maximum reference script size per endorser block. | ||
| uint64 peras_min_candidate_block_age = 47; // The Peras minimum candidate block age in slots. | ||
| RationalNumber peras_healing_factor = 48; // The Peras healing factor for the cooldown length. | ||
| uint32 peras_cert_boost = 49; // The Peras chain weight boost per certificate. | ||
| uint32 peras_target_committee_size = 50; // The Peras target committee size. | ||
| optional uint64 peras_bootstrap_round = 51; // The Peras round that bootstraps voting, if set. | ||
| RationalNumber peras_quorum_threshold_safety_margin = 52; // The Peras quorum threshold safety margin. | ||
| BigInt ref_inputs_cost_per_multi_asset_policy = 53; // The fee per multi-asset policy in reference inputs. | ||
| BigInt ref_inputs_cost_per_datum_byte = 54; // The fee per datum byte in reference inputs. |
There was a problem hiding this comment.
These new scalar fields have no presence, so an unset value reads as 0: max_ref_script_size_per_block, max_ref_script_size_per_tx, ref_script_cost_stride, leios_announcement_period_length, leios_vote_period_length, leios_diffusion_period_length, leios_committee_size, max_endorser_block_references_size, max_endorser_block_txs_size, max_ref_script_size_per_endorser_block, peras_min_candidate_block_age, peras_cert_boost and peras_target_committee_size.
PParams also carries partial updates in ParameterChangeAction and the parameters of earlier eras in EraSummary, where these fields are absent, not 0.
I suggest marking them optional, as peras_bootstrap_round already is.
The older scalar fields have the same problem, but that predates this PR.
There was a problem hiding this comment.
Done. Updates now use a separate PParamsUpdate.
- Leios and Peras block data, pool BLS keys, Dijkstra genesis - PParamsUpdate as the type of parameter change updates - SubTx, five missing tx body keys, presence on optional fields - ReadTx by a sub transaction hash
- cardano.proto: add TxCommon with the fields both tx levels share - Tx holds common, the top level only fields and TxCommon sub txs - drop SubTx and point the TxPattern and Asset docs at TxCommon
- cardano.proto: remove Block.endorser_block and its two messages - BlockBody.tx: document the certified endorser block transactions
| // Request to get a transaction by hash | ||
| message ReadTxRequest { | ||
| bytes hash = 1; // The hash of the transaction. | ||
| bytes hash = 1; // The hash of a transaction, or of a sub transaction to read the last transaction that includes it. |
There was a problem hiding this comment.
For a sub transaction hash, this returns the top-level transaction that includes it, because a sub transaction is never on chain by itself; the sub transaction is the entry of sub_transactions with that hash. "The last transaction" needs spelling out, because one sub transaction can be in several top-level transactions: when a batch fails phase 2, none of its sub transactions take effect, so their inputs stay unspent and the same sub transaction can be included again in a later batch.
I suggest saying that the response is the most recent such top-level transaction on the canonical chain, and that it can be unsuccessful. A successful inclusion is always the last one, because it spends the inputs, so the returned transaction's successful flag tells the client whether the sub transaction has taken effect.
| message Tx { | ||
| TxCommon common = 1; // Fields the transaction shares with its sub transactions | ||
| optional Collateral collateral = 2; // Collateral details in case of failed transaction | ||
| BigInt fee = 3; // Transaction fee in ADA | ||
| bool successful = 4; // Flag indicating whether the transaction was successful | ||
| repeated TxCommon sub_transactions = 5; // List of sub transactions, in body order | ||
| repeated AccountBalanceInterval starting_account_balance_intervals = 6; // List of starting account balance intervals | ||
| } |
There was a problem hiding this comment.
The structure looks good now, but the field comments still describe a single transaction. A client applying a block needs the batch rules (a batch is a top-level transaction and its sub transactions), so I suggest stating them on the fields:
TxCommon.inputsandreference_inputsresolve against the UTxO before the batch, and spending inputs are disjoint across the batch.TxCommon.outputsare referenced by thehashof theTxCommonthat holds them, also in a sub transaction.feeandcollateralcover the whole batch.- When
successfulis false, only collateral processing applies, including any collateral return; no ordinary effects of the top-level transaction or its sub transactions are applied. starting_account_balance_intervalsare checked against the balances before the batch. Each transaction'saccount_balance_intervalsare checked after earlier sub transactions, but before its own withdrawals, certificates and deposits. The top-level transaction's check follows all sub transactions.
| // Contains the transactions and the Leios and Peras certificates of a block. | ||
| message BlockBody { | ||
| repeated Tx tx = 1; // List of transactions. | ||
| repeated Tx tx = 1; // List of transactions. For a block with a Leios certificate, the transactions of the certified endorser block. |
There was a problem hiding this comment.
For a block with a Leios certificate, the transactions in tx come from the endorser block announced by the previous block. The ledger validates them at the announcing block's slot, against the ledger state that block left, not at this block's slot. A transaction here is therefore valid even if its TTL is earlier than this block's slot. I suggest saying this in the comment on tx, so that clients do not treat such transactions as expired.
What?
The fields that top level and sub transactions share move into a new
TxCommon, whichTxholds ascommonbeside its top level only fieldscollateral,fee,successful,sub_transactionsandstarting_account_balance_intervals. Each sub transaction is aTxCommon, so it has none of the fields a ledger sub transaction lacks and cannot hold sub transactions of its own.TxCommongainsdirect_deposits,account_balance_intervals,guardsandrequired_top_level_guards, and five older body keysTxhad no field for (auxiliary_data_hash,script_data_hash,network_id,treasury_value,treasury_donation).Txalso had no field for body key 14, the required signers before Dijkstra and the guards from Dijkstra, andguardsis that key.TxPatternmatches aTxand theTxCommonof each of its sub transactions, andReadTxRequest.hashalso accepts a sub transaction hash.BlockHeadergainsendorser_block_announcementandbody_contains_leios_certificate, andBlockBodygainsleios_certificateandperas_certificate.NativeScriptgainsscript_require_guard,RedeemerPurposegainsREDEEMER_PURPOSE_GUARDING, andPoolRegistrationCertandPoolStakeSharegain aPoolBlsKey.PParamsgains the Dijkstra parameters as fields 32 to 54 andGenesisas fields 43 to 65. A parameter change proposal is now aPParamsUpdate, where an absent field leaves the parameter unchanged.Why?
A client of a Dijkstra node needs every value the ledger produces, and these messages had no field for the values Dijkstra adds. Without
sub_transactionsa client misses the outputs sub transactions create and the inputs they spend, so its UTxO set is wrong.In
PParamsa plain scalar set to zero reads as unset, and an absentmax_pledge_leveragemeans no limit, so a proposal that sets a zero or removes the limit reads as no change.ParameterChangeAction.protocol_param_updateis now typedPParamsUpdate, so code generated from the unreleased v1 that reads it asPParamsmust change.Txmoves its shared fields undercommon, so the same code readstx.common.inputswhere it readtx.inputs.Testing done
buf lintreports the same 28 findings at this head as on main. Against main,buf breakingreports the presence change onTxValidity.startandttl, the type change ofprotocol_param_update, and 31 lines onTxfor fields 1 to 6 changing name and type and fields 7 to 15 deleted. Under the WIRE rules it reports the type change and 19 lines onTx. Against the Dijkstra CDDL at IntersectMBO/cardano-ledger@845f131, the fields ofTxwithcommonexpanded are thetransaction_bodykeys, and the fields ofTxCommonare thesub_transaction_bodykeys. With buf 1.74.0:Notes
BlockBody.peras_certificateis the raw certificate bytes, empty when the block has none, because the ledger encodes the certificate as abytesplaceholder and refuses any certificate until Peras activates. If [DRAFT] Use peras types from cardano-base in ledger IntersectMBO/cardano-ledger#6164 defines the certificate structure before this PR merges, field 3 becomes a message. Otherwise a message field is added beside the bytes once the ledger defines the type, sincebuf breakingflags a bytes to message change under WIRE and WIRE_JSON. There is no Leios committee query because the node serves none.body.txlists the transactions of the endorser block its certificate certifies, as a Dijkstra node serves the block over local chain sync.PoolStakeShare.bls_keyis absent when the pool had no BLS key in the stake snapshot, even if it has registered one since.v1only.TxValidity.startandttlare nowoptional, so a bound of 0 differs from an absent bound. The generated code changes and the wire format does not.ReadTxwith a sub transaction hash returns the last transaction that includes it. A sub transaction can appear in several transactions on chain, and only the last can have applied it, because the earlier ones failed phase 2 validation.