Skip to content

Latest commit

 

History

History
513 lines (414 loc) · 23.4 KB

File metadata and controls

513 lines (414 loc) · 23.4 KB

icp-js-plugin API

The scripting API of icp-js-plugin: the script's inputs, the host functions it may call, and the Candid coercion rules the typed calls follow.

Scripts run on QuickJS via rquickjs; it is a small ES2020-class engine without Node or Web APIs, so no require/import, no fetch, no timers — just the language plus the host functions documented here.

The script

The entry script comes from one of two places, and declaring both is an error:

  • A script field, whose value is the JavaScript source inline.
  • A file declared under the script key, whose contents are the JavaScript source.
sync:
  steps:
    - plugin: ./icp_js_plugin.wasm
      files:
        script: sync.js
        seed: [seed/users.json, seed/roles.json]
      dirs:
        assets: assets

Every declared file — the entry script included — is read by the host and handed to the script via the files object, keyed by path. Directories declared in dirs are preopened read-only and reachable with the filesystem functions below. Declaring files:/dirs: as a map instead of a plain list tags each entry with its key, which the script reads back through fileKeys / dirKeys.

A script runs to completion for a clean sync; throwing (or a runtime error) fails the step with the thrown message.

Promises have no event loop under them. async/await, .then and queueMicrotask queue jobs, which run after the script's last statement until the queue is empty — that is when the step ends. A rejection nothing handled fails the step, so a throw inside an async function is as fatal as one at the top level, and a promise that never settles leaves the rest of its function unrun. Canister calls are synchronous, so nothing suspends but what the script suspends itself.

Sync inputs (globals)

Name Type Description
canisterId string Textual principal of the target canister.
canister Principal The target canister as a Principal.
environment string Environment being synced (e.g. "production").
identityId string Textual principal of the signing identity.
identity Principal The signing identity as a Principal.
proxy Principal | null Proxy canister if --proxy was set, else null.
dirs string[] Declared directory paths (preopened read-only).
dirKeys object (key → string[]) Manifest key → the directory paths declared under it.
files object (path → string) Contents of every declared file, by path.
fileKeys object (key → string[]) Manifest key → the file paths declared under it.
fields object (name → string) Key-value fields declared in the step's fields.
canisterIds object (name → string) Every project canister's name → textual principal.

dirKeys and fileKeys cover only the entries declared under a map key; a plain-list dirs:/files: has none, and appears only in dirs/files. One key may name several paths, so each maps to an array:

// Contents of every file declared under the `seed` key.
let seeds = fileKeys.seed.map((path) => files[path]);

canisterIds is informational: it maps each named canister in the project (both subproject:local keys and bare local names for same-subproject siblings) to its textual principal for the environment being synced. Being listed does not grant permission to call a canister — that still requires declaring it in the step's canisters: list. Wrap a value in Principal.from(..) for a Principal.

Canister calls

Every call names its receiver: the global self is the canister being synced (canisterId), which is also what an omitted target means. A call may instead target any canister listed in the sync step's canisters:by name, spelled exactly as that list does. A principal is not a target: the host resolves names so that the permission it checks is the one the manifest granted, and passing one is an error that names the canister instead. Each call returns the raw Candid-encoded response bytes as a Uint8Array, or throws with the host's error message.

// Shorthands: the receiver first, then the method, then the argument list —
// which may be omitted where the method takes none.
let resp = callQuery(self, "get_count");
let resp = callUpdate(self, "set_count", candid`(7 : nat64)`);
let resp = callUpdate("ledger", "set_count", candid`(7 : nat64)`);

// General form. Only `method` is required.
let resp = canisterCall({
    method: "transfer",
    arg: candid`(record { to = ${dest}; amount = ${10} })`,
    query: false,   // default false → update; true → query
    direct: false,  // default false → route update through the proxy if configured
    cycles: 0,      // attached to a proxied update call only
    // target: `self` (or omitted) → the canister being synced. Otherwise the
    // name of a canister listed in the step's `canisters:`.
    target: "ledger",
});

arg takes a CandidArgs — what the candid tag yields — or a Uint8Array of bytes encoded some other way.

These are the raw calls: the script encodes the argument and decodes the response itself. Coerced calls do both against the callee's own interface instead.

Metadata sections

let did = canisterMetadata("candid:service"); // → Uint8Array, or null if absent
let text = decodeUtf8(did);

// General form. Only `name` is required.
let section = canisterMetadata({
    name: "candid:args",
    target: "ledger", // omitted → the canister being synced
    direct: false,    // default false → read through the proxy if configured
});

The section name is spelled as the wasm module's custom section does, minus the icp:public /icp:private prefix. null means the target provably has no section by that name — including having no module installed at all; a section the reader may not have is an error. A direct read is a certified read_state signed by the sync identity, which reaches a private section only if that identity controls the target; a proxied read reaches one private to the proxy's control.

Environment variables

canisterSetenv sets one of a canister's runtime environment variables, leaving its other variables — and the rest of its settings — as they are. It names its receiver first, as a call shorthand does.

canisterSetenv(self, "SEEDED_BY", environment);
canisterSetenv("ledger", "ADMIN", canisterIds.backend);

// Optional trailing options; `direct` is the only one.
canisterSetenv(self, "ADMIN", identityId, { direct: true });

The value is a string: the canister reads it back verbatim, so anything else is the script's to render (String(x), or x.toText() for a Principal). The update is controller-gated — with direct the sync identity must control the receiver, and by default the proxy configured via --proxy makes it, so that is what must control it. With no proxy configured the sync identity signs either way.

Set the variable on every sync rather than once. The management canister can only replace a canister's variables as a whole list, so the host reads them and writes them back with yours added — and a later icp deploy rewrites that list from the manifest, dropping what a plugin added. Deploy runs the sync phase afterwards, so a script that always sets it always restores it. For a variable that should not depend on the plugin running, declare it in the manifest's environment_variables setting instead.

Candid

An argument is written as Candid source with JavaScript values interpolated into it, using the candid template tag:

let args = candid`(record { to = ${dest}; amount = ${10} })`;
let args = candid`(${[1, 2, 3]}, "literal text")`; // → CandidArgs

The template is an argument list, parenthesized like the text new CandidArgs(text) takes. Interpolation is not textual: each ${…} is parsed as a placeholder and the JavaScript value is grafted onto the parsed value afterwards, so a value can never inject Candid syntax of its own — ${"1; b = 2"} is one text value, not a record field.

Each hole becomes the Candid value Candid's own syntax would give the same literal:

JavaScript Candid
null, undefined null
true / false bool
an integer, or a BigInt a width-undetermined number (int by default)
a fractional number float64
a string text
a Principal principal
a Uint8Array blob
a number class that class's type (see below)
an exact-encoding class that class's value (see below)
an array vec (its elements must share one type)
any other object record of its own enumerable properties

A hole inside a string literal interpolates text instead, the way JavaScript's own template literals do — candid`("hello ${name}")` is one text value.

What a hole cannot stand for is anything the parser resolves as it goes: a type annotation (${n} : nat64), a field name, or the contents of a principal or blob literal. Those are errors that name the offending ${…}; spell the literal out, or interpolate the whole value (${Principal.from(id)} rather than principal "${id}").

CandidArgs

An argument list, and what a call's arg takes. It carries the argument values, not only the bytes they encode to untyped, which is what lets a coerced call serialize them at the types the callee declares.

let args = candid`(1, "x")`;
args.length;         // → 2
args.toUint8Array(); // → Uint8Array of encoded bytes
args.toValues();     // → [1n, "x"], the arguments as JavaScript values
`${args}`;           // → '(1, "x")', the list as Candid text

new CandidArgs('(42 : nat64)');    // parse an argument list from Candid source
CandidArgs.decode(bytes);          // read back encoded bytes, untyped

A one-value candid`…` also stands for that one value wherever a value is wanted — a record field, or one argument of a coerced call.

An argument list can also be built from JavaScript values alone, with no source to write: candidEncode takes one value per argument and converts each exactly as a ${…} hole is converted, by the table above.

let args = candidEncode({ to: dest, amount: 10 }); // → CandidArgs, one record
let args = candidEncode(1, 'hi');                  // → CandidArgs, two arguments
candidEncode('(42)');                              // → ('(42)'), a text value —
                                                   //   source is CandidArgs' job
let text = candidDecode(args);                     // CandidArgs/Uint8Array → text

candidDecode reconstructs a structural view without type information, so record fields appear as their numeric hashes — it is meant for inspection, not round-tripping.

Number types

A bare JavaScript number is width-undetermined and encodes as int, which is wrong for a method that declares a fixed width. Since a hole cannot carry a type annotation, the number classes say which type is meant:

candid`(record { amount = ${new Nat64(10)}; rate = ${new Float32(0.5)} })`;

Nat, Int, Nat8, Nat16, Nat32, Nat64, Int8, Int16, Int32, Int64, Float32 and Float64 each map to the Candid type of the same name. Each takes a number, a BigInt or a string — so a nat64 past 2^53 stays exact, new Nat64("18446744073709551615") — and throws when the value does not fit. They are wrappers for encoding: toString() reads one back, and in a string a hole holding one interpolates its decimal.

Exact-encoding classes

The literal mapping covers what JavaScript syntax denotes. What is left over — a variant, a canister or function reference, an optional distinct from null, and a tuple of mixed types — has a class that says it exactly:

new Variant("ok");                       // variant { ok }
new Variant("member", { since: 1 });     // variant { member = record { since = 1 } }
new Opt(5);                              // opt 5
new Opt();                               // an absent optional
new Service(canister);                   // service "…"
new Func(canister, "transfer");          // func "…".transfer
new Tuple("x", new Nat32(7));            // record { "x"; 7 : nat32 }

Variant reads its tag the way a Candid field name is read: _123_ is the hash itself, anything else is hashed. Opt is for the two cases the coercion rules cannot spell — an optional holding null (new Opt(null)) and an optional of an optional. Service and Func take a Principal or its text. Tuple is a record of numbered fields, which an array cannot be because a vec holds one type.

Each carries the accessors its shape suggests (.tag, .hasValue, .canister, .method, .length) and a toString() that renders the value as Candid text.

Coerced calls

A script that knows what a method's arguments are need not write any Candid: the types the callee declares decide how each JavaScript value encodes, and how the response reads back. The interface comes from the receiver itself, out of its candid:service metadata section.

callTyped("ledger", "transfer", { to: dest, amount: 10 });

const balance = callTyped("ledger", "balance_of", identity); // → decoded JavaScript

The first argument is the receiver: a canister name as a call's target is, or self for the canister being synced. A target is otherwise always a name, and the canister being synced has none to give — so self is the global that names it.

callTyped(self, "increment");

The interface decides whether each call is a query or an update, and is read once per receiver per run rather than once per call. A method with one result returns it directly, one with none returns undefined, and one with several returns an array.

The general form takes the options a raw canisterCall does, plus an interface to use instead of the one read from the receiver:

canisterCallTyped({
    method: "transfer",
    args: [{ to: dest, amount: 10 }], // the argument list; omitted → no arguments
    target: "ledger",                  // omitted → the canister being synced
    interface: files["ledger.did"],    // a CandidInterface, or `.did` source
    direct: false,
    cycles: 0,
});

args is the argument list, so a method taking one vec gets args: [[1, 2, 3]]. A candid`…` may stand for the whole list instead of an array.

CandidInterface is the parsed .did on its own, for encoding and decoding without calling:

const iface = new CandidInterface(files["backend.did"]);
const iface = CandidInterface.fromCanister("ledger"); // read from its metadata
const iface = CandidInterface.fromCanister(self);

iface.methods();                      // → string[]
iface.signature("get");               // → "(nat64) -> (text) query"
iface.isQuery("get");                 // → boolean
iface.encode("transfer", {});      // → CandidArgs, at the declared types
iface.decodeResult("transfer", resp); // → decoded JavaScript
iface.decodeArgs("transfer", bytes);  // → an array, one per declared argument

What coerces to what

Every rule is there to make a JavaScript literal land on the type the method declares. A value the declared type cannot account for is an error naming the argument and the field it sits at — never a guess.

Declared type JavaScript
bool a boolean
text a string
nat, int, any fixed width an integral number, or a BigInt
float32, float64 a number
principal a Principal, or a textual principal
service a Principal, a textual principal, or a Service
func a Func — no literal denotes one
opt T null/undefined for absent, anything else for present
vec T an array; a Uint8Array where T is nat8
record an object keyed by field name; an array for a tuple record
variant { tag: value }, or "tag" for a tag that carries nothing
null null or undefined
reserved anything

An omitted record field is absent where the type says opt, null or reserved, and an error otherwise; an unknown field is an error rather than something dropped on the floor. A string is not a number — BigInt is how JavaScript writes an integer too wide for a number, so there is nothing a decimal string could add.

A value one of the exact-encoding or number classes holds passes through as it stands, as does a candid`…` — which supplies the whole argument list when it is a call's only argument, and one value where a single value is wanted.

Decoded results are the same mapping read backwards, so a response goes straight back into another call: a record is an object keyed by field name, a variant is a one-entry object, principal and service are Principals, func is a Func, blob is a Uint8Array, and an absent optional is null. nat, int, nat64 and int64 come back as BigInts, being wider than a number is exact for, which JSON.stringify refuses; the narrower widths and the floats are numbers. opt null and nested optionals are what this loses — candidDecode shows a response exactly.

Principals

Principal is the class of icp-js-core, minus selfAuthenticating (which would need a key the plugin has no way to reach), so code written against @icp-sdk/core carries over:

Principal.from("ryjl3-tyaaa-aaaaa-aaaba-cai"); // text, bytes or a Principal
Principal.fromText("ryjl3-tyaaa-aaaaa-aaaba-cai"); // throws if invalid
Principal.fromUint8Array(bytes);
Principal.fromHex("00000000000000020101"); // ryjl3-tyaaa-aaaaa-aaaba-cai
Principal.anonymous();          // 2vxsx-fae
Principal.managementCanister(); // aaaaa-aa
Principal.isPrincipal(value);

p.toText();       // → string
p.toUint8Array(); // → Uint8Array (raw bytes)
p.toHex();        // → hex string
p.isAnonymous();  // → boolean
p.toJSON();       // → { "__principal__": "<text>" }
`${p}`;           // string coercion → textual principal

p.compareTo(q);   // → "lt" | "eq" | "gt", byte-wise
p.ltEq(q);
p.gtEq(q);

new Principal(..) takes what Principal.from does; icp-js-core keeps its own constructor protected.

Encoding helpers

sha256(bytes);         // Uint8Array → 32-byte Uint8Array
encodeUtf8("hi");      // string → Uint8Array
decodeUtf8(bytes);     // Uint8Array → string (throws on invalid UTF-8)

The bytes a metadata section or a file comes back as are usually text, which is what the last two are for.

Randomness

randomBytes(32); // → Uint8Array of 32 cryptographically random bytes

The bytes come from the host's wasi:random, so they are unguessable — for a nonce, a salt, or a fresh subaccount. Math.random() is a clock-seeded PRNG: fine for jitter or a sample, not for anything that must not be predicted. One call returns at most 1 MiB.

Filesystem

Read-only access to the directories the step declared under dirs:, backed by WASI. Each is readable at the path the manifest declared it at, and nothing outside them is readable at all.

readFile("assets/data.json");        // → string (throws if not valid UTF-8)
readFileBytes("assets/logo.png");    // → Uint8Array
readDir("assets");                   // → DirEntries, an iterator of entry names

exists("assets/data.json");          // → boolean
isFile("assets/data.json");          // → boolean
isDir("assets");                     // → boolean
isSymlink("assets/current");         // → boolean, about the link itself
fileSize("assets/logo.png");         // → number of bytes

joinPath("assets", "img", "a.png");  // → "assets/img/a.png"

readDir yields entry names, without the directory they sit in — joinPath puts the two back together. The DirEntries it returns is an iterator: for … of walks it, [...readDir(dir)] or Array.from collects it, and it is consumed once. The names are read and sorted when readDir is called, so an unreadable directory fails there rather than partway through the loop, and a walk does the same work in the same order on every run.

for (const name of readDir("assets")) {
    const path = joinPath("assets", name);
    if (isFile(path)) print(path, fileSize(path));
}

The reads throw with the underlying error; the predicates answer false instead, so a path that may not be there — or may not be reachable — can be asked about. Since only the declared dirs: are readable, a failed read also says what the step declared, and a path naming a declared file says to read it from files: the host passes those contents inline rather than putting them on the filesystem.

joinPath separates the parts it is given with single slashes however they are punctuated, dropping empty ones. A part that starts at the root replaces what came before it, the way pushing onto a path does.

Writes are unavailable because the host preopens directories read-only; a script that has something to hand back makes a canister call with it.

Output

print(..) and console.log(..) write to stdout, shown as transient progress and discarded when the step ends. eprint(..), console.error(..), console.warn(..) and console.debug(..) write to stderr, which is also printed persistently after the step completes — use them for warnings and summaries the user should still see. All accept multiple arguments and join them with spaces.