Skip to content

fix(activation-service): stop leaking internal errors, drop unused KYC_PUBLIC_KEY - #1107

Closed
sameh-farouk wants to merge 2 commits into
fix/activation-service-migrate-tfchain-clientfrom
fix/activation-service-error-exposure
Closed

fix(activation-service): stop leaking internal errors, drop unused KYC_PUBLIC_KEY#1107
sameh-farouk wants to merge 2 commits into
fix/activation-service-migrate-tfchain-clientfrom
fix/activation-service-error-exposure

Conversation

@sameh-farouk

Copy link
Copy Markdown
Member

Two follow-ups from #1105. Based on #1105, not development — the readme text here describes ACTIVATION_AMOUNT as being read, which only becomes true with that PR. Retarget to development once it merges.

1. Internal error messages were returned to callers

The error middleware computed http-errors' expose flag and then ignored it. Both branches of the NODE_ENV check sent the message anyway, because lodash's omit(err, ['stack']) copies message through:

omit(httpError(500, 'Insufficient Balance Error: ...'), ['stack'])
// => {"message":"Insufficient Balance Error: ...","status":500,"statusCode":500,"expose":false}
//                                                                              ^^^^^^^^^^^^^^

And the development branch — the default, since NODE_ENV is set in neither the Dockerfile nor the chart — sent the whole error including the stack trace. So every deployment has been running in the more verbose mode.

Verified against the built image: a request that failed because the funding account was empty replied with Insufficient Balance Error: Failed to apply 'transfer' on section 'balances' with args '<address>,1000000.

Now the flag decides. 4xx keeps its message, since the caller sent something unusable and needs to know what; 5xx returns the generic status text and the real error goes to the log only:

400 -> {"message":"substrateAccountID is not a valid account id"}
500 -> {"message":"Internal Server Error"}        # log: "error happened handling the request"

NODE_ENV no longer affects what is returned, so a deployment cannot leak by omitting it — which is what every deployment currently does.

Behaviour change: response bodies are now consistently { message }. 4xx previously also carried status, statusCode and expose; non-http errors fell through to res.send(err.message) as text rather than JSON. The frontend catches errors and renders a fixed string without reading the body, so there is no UI impact.

2. KYC_PUBLIC_KEY was required but unread

It has been in bin/www's required env variables since the service was written, but nothing reads it: the signature verification was commented out years ago and its only caller, POST /activation/create-entity, was removed in #1103 as an unauthenticated route that signed a fee-paying extrinsic from the service wallet. Requiring an unused variable only gives the service a reason to refuse to start.

Removed from the required list, the chart and the documented .env. Re-add it alongside the verification code if KYC checks return.

While in the readme, two claims that no longer held: it documented the removed /activation/create-entity endpoint, and said activation "puts 500 tokens" on an account — true in 2021, since reduced by three orders of magnitude.

Verification

  • lint + unit: 27/27
  • live devnet: 400 exposes its message, 500 returns Internal Server Error while the underlying error still reaches the log, boot and SIGTERM unaffected

🤖 Generated with Claude Code

https://claude.ai/code/session_011YKJm3zuWdSepriT9KL9zy

@sameh-farouk
sameh-farouk requested a review from LeeSmet as a code owner July 26, 2026 14:51
…llers

The error middleware computed http-errors' `expose` flag and then ignored it.
Both branches of the NODE_ENV check sent the message regardless: lodash's
`omit(err, ['stack'])` copies `message` through, so the "production" path replied
with things like a decoded chain error next to `expose: false`, and the
development path — the default, since NODE_ENV is set in neither the Dockerfile
nor the chart — added the stack trace on top.

Concretely, a request that failed because the funding account was empty replied
with `Insufficient Balance Error: Failed to apply 'transfer' on section
'balances' with args '<address>,1000000.` Verified against the built image.

Now the flag decides: 4xx keeps its message, because the caller sent something
unusable and needs to know what; 5xx returns the generic status text and the real
error goes to the log only. Responses are consistently JSON, where a non-http
error used to fall through to `res.send(err.message)` as text.

NODE_ENV no longer changes what is returned, so a deployment cannot leak by
omitting it — which is what every deployment does today.
`KYC_PUBLIC_KEY` has been in bin/www's required env variables since the service
was written, but nothing reads it: the signature verification was commented out
years ago, and its only caller — `POST /activation/create-entity` — was removed
in #1103 as an unauthenticated route that signed a fee-paying extrinsic from the
service wallet. A variable that is required but unused only gives the service a
reason to refuse to start.

Removed from the required list, the chart, and the documented .env. Re-add it
alongside the verification code if KYC checks come back.

While in the readme, two claims that no longer hold: it documented the removed
`/activation/create-entity` endpoint, and said activation "puts 500 tokens" on an
account — the amount at the time, since reduced by three orders of magnitude.
@sameh-farouk
sameh-farouk force-pushed the fix/activation-service-error-exposure branch from 811e464 to 6c90479 Compare July 26, 2026 14:53
sameh-farouk added a commit that referenced this pull request Jul 26, 2026
The comment claimed lodash's omit drops the non-enumerable message, so the chain
error would be lost from the response. It does not — omit(httpError(500, 'msg'),
['stack']) returns the message alongside expose: false, which is the leak #1107
fixes. Stated the real reason for logging instead.
sameh-farouk added a commit that referenced this pull request Jul 26, 2026
* fix(activation-service): migrate to @threefold/tfchain_client

The in-repo `clients/tfchain-client-js` cannot be published: the npm credentials
for `tfgrid-api-client` are lost, so activation-service has been pinned to
1.29.1 (published 2023-06-21) while the client moved on 24 commits. That is what
has been blocking the 2.13.0 release (#1099).

`@threefold/tfchain_client` covers all four calls this service makes and is
actively published (2.9.2, 2025-11-25). Measured against devnet it is also
materially more robust on the two paths that matter here:

- Startup. `bin/www` awaits init() before server.listen(). The old client's
  `ApiPromise.create({provider})` never rejected, so when the chain was
  unreachable the service hung before listening, logged nothing and never became
  ready. connect() now rejects in ~1s and the existing catch reports it.
- Transfer outcome. The old `transfer` resolved on submission, so a failed
  funding transfer still returned 200. `apply()` waits for inclusion, detects
  system.ExtrinsicFailed and rejects, so failures surface as 5xx.

Balance reads are equivalent between the two clients, so reads are not a reason
to migrate; those two are.

Behaviour changes, all intentional:

- An invalid `substrateAccountID` returns 400, not 500. The constructed
  httpError(400) was never thrown, so execution fell through to `keyring.address`
  on an undefined variable. The response body for this case is now the standard
  error object instead of a bare string.
- Account ids must decode to a 32 byte public key. `0x1234` used to be accepted
  and funded as the derived address `25NbUg`, which nobody holds the key to.
  Validation is now explicit: the previous check was a side effect of
  `keyring.addFromAddress` throwing, and tfchain_client's transfer does not
  validate the address at all.
- A failed transfer is now an error response rather than a 200.
- `ACTIVATION_AMOUNT` is finally read, in whole TFT as readme.md and the chart
  have always documented it. It was required by bin/www since the service was
  written but never consumed, and the amount was hardcoded to 1000000 base units
  — 0.1 TFT, so the readme's "currently 1 TFT" overstated it tenfold. Unset it
  still defaults to 0.1 TFT, but the chart's `activation_amount: 1` now funds a
  whole TFT: ten times the previous amount wherever that value is deployed.
  Amounts below the chain's existential deposit are refused at startup.
- A failed startup exits 1. stopGraceful() exited 0 and the process.exit(1) after
  it was unreachable, so a failed boot reported success. Latent until now,
  because init() never used to reject.
- Chain errors map to status codes by constructor name: ValidationError to 400,
  ConnectionError and TimeoutError to 503, anything else 500.
- Dockerfile moves to node:22; @polkadot/api 15.x requires node >= 18.

The commented-out `validateActivation` is removed. Its route went away in #1103
and it referenced `client.verify`, which does not exist on the new client.

Tests: the service had none and no CI job. Adds unit tests for the funding
branches, the amount config, address validation, error mapping and the fail-fast
property, plus an opt-in devnet integration test whose targets are derived from
the funder (//as-ci//1..5) and swept back at both ends of the job so a crashed
run is reclaimed by the next one. `standard` is scoped to the server code; it was
failing on the committed frontend bundle, which made `npm test` red already.

Refs #1099

* fix(ci): stop the activation service workflow running twice and racing itself

Pushing a branch and then opening a PR from it triggered this workflow twice,
which was visible on #1105. Duplicate lint runs are only noise, but the
integration job is not safe to run concurrently: every run funds and sweeps the
same fixed pool of accounts derived from one shared mnemonic, so two overlapping
runs fight over the same balances — one run's start-of-job sweep empties an
account the other has just funded, and its "pool account should start empty"
assertion fails for a reason that has nothing to do with the code.

Restricts push to the default branch, since pull_request already covers branches,
and adds a single global concurrency group so runs serialise. The group queues
rather than cancelling, so a run already submitting extrinsics is left to reach
its end-of-job sweep instead of being killed mid-flight and stranding funds.

* fix(ci): scope working-directory per job so check-funder can run

The workflow-level `defaults.run.working-directory` applied to check-funder too,
but that job only reads a secret and never checks the repository out, so its step
ran against a path that does not exist and failed. Because integration is gated on
check-funder's output, the devnet test was skipped rather than run — visible on the
first run of this workflow on #1105.

Moved the setting onto the two jobs that actually run commands in the service
directory.

* fix(ci): scope the concurrency group to the job that shares devnet state

A workflow-level group with a fixed name meant every branch shared it, so a push
to any other branch touching activation-service cancelled a pending run instead of
queueing behind it — the run for dbf34ab was cancelled outright while older runs
were still going, leaving that commit with no CI result.

Only the integration job shares anything: the fixed pool of accounts derived from
the funder mnemonic. Moved the group there, so lint and unit results land on every
PR while devnet access still serialises. The group name deliberately has no ref in
it, since the pool is shared across branches regardless of which one is testing.

* fix(activation-service): stop the unit suite hanging on Node 22

`npm test` never finished on CI: the "Lint and run unit tests" step ran for 39
minutes on a job that takes a second locally. The fail-fast test in
test/startup.test.js was the cause. Its assertion passes — connect() does reject —
but something in the torn-down provider keeps the event loop alive, so the child
process never exits and `node --test` waits for it indefinitely.

It reproduces in node:22 and not on local Node 24, and only when the network is
up: offline the suite passes either way. So the difference between CI and a
developer's machine is what surfaced it.

Adds --test-force-exit, which exits once tests and hooks have finished rather than
waiting on stray handles, and gives the test an explicit timeout so a future
regression here fails the run instead of hanging it — which is precisely the
failure mode this test exists to catch.

Verified in node:22: npm test completes in 1s, 27 pass, 0 fail.

* docs(activation-service): correct an inaccurate comment on error logging

The comment claimed lodash's omit drops the non-enumerable message, so the chain
error would be lost from the response. It does not — omit(httpError(500, 'msg'),
['stack']) returns the message alongside expose: false, which is the leak #1107
fixes. Stated the real reason for logging instead.
@sameh-farouk
sameh-farouk deleted the branch fix/activation-service-migrate-tfchain-client July 26, 2026 15:47
@sameh-farouk

Copy link
Copy Markdown
Member Author

Superseded by #1109. GitHub closed this automatically when #1105 merged and its branch — the base of this PR — was deleted, and a closed PR cannot be reopened or retargeted once its base is gone. #1109 carries the same two commits rebased onto development.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant