Skip to content

Offer method collections to download and load in one step - #781

Merged
ccomb merged 7 commits into
mainfrom
method-catalog
Oct 31, 2026
Merged

ccomb merged 7 commits into
mainfrom
method-catalog

Conversation

@ccomb

@ccomb ccomb commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #780, which reads the padded method files the IMPACT World+ collections come in. Retarget to main once #780 is merged.

Problem

To score with a method collection the engine does not ship, a user has to find the publisher's file, download it, then upload it. Nothing tells them which collections are known to load, and nothing checks that the file they found is the one expected.

Solution

The engine carries a catalogue, data/method-catalog.toml, of collections it can fetch itself: Environmental Footprint 3.1, and the expert, midpoint and footprint versions of IMPACT World+ 2.2.1. Each entry states the publisher, version, licence (when the publisher states one), the page to cite, the file's https address, its size and its SHA-256.

  • GET /api/v1/method-collections/available lists the catalogue, each entry saying whether the engine already knows a collection of that name.
  • POST /api/v1/method-collections/available/{name}/install downloads the file, keeps it only when its size and SHA-256 match, stores it as an upload of that name and loads it.

The download follows no redirect, stops one byte past the stated size, gives up after a minute without data, and sends a User-Agent: one of the file servers refuses requests without one.

Answers: 403 on a read-only engine, 404 for a name not in the catalogue, 409 for a name already known, 413 when the file is above the plan's upload limit, 502 when the server cannot be reached or answers with an error status, 422 when the file received is not the one expected.

Wire revision 103. pyvolca gains available_method_collections() and install_method_collection(name).

Remarks

  • The catalogue is part of the reference data, so data/VERSION moves.
  • The new dependency is http-client-tls, listed in THIRD_PARTY_LICENSES.md. No method file is shipped with the engine, so none is listed there.
  • No MCP tool: uploads are not offered over MCP either.

How to test

./build.sh --test
# on a running engine with a writable data directory:
curl -s localhost:8080/api/v1/method-collections/available
curl -s -X POST "localhost:8080/api/v1/method-collections/available/IMPACT%20World%2B%202.2.1%20Footprint/install"
curl -s localhost:8080/api/v1/method-collections   # the collection is listed, loaded, with 6 categories

Done on a local engine: the four collections installed and loaded (Environmental Footprint 3.1 with 25 categories; IMPACT World+ Expert, Midpoint and Footprint with 36, 21 and 6). A second install answered 409; an unknown name, 404. From the REPL, fetchEntry refused a file of the wrong size, a file with a different SHA-256, a server answering 410 and an address it cannot parse, each with its reason.

ccomb added 5 commits October 30, 2026 23:37
Some exporters pad every line of a SimaPro method file to the widest row,
so a section marker reads `Impact category;;;;;` and a blank line `;;;;;`.
The reader compared whole lines, found no section, and loaded the file
with no category while reporting success. A line is now read without its
trailing separators, which also keeps them out of the method's name and
of the normalization-weighting set's name. A method file that yields no
impact category is refused instead of loading empty.
A collection whose files partly failed logged only their number, so a
file refused beside others that load never said which one or why.
An engine now carries a catalogue of method collections it can fetch
itself: Environmental Footprint 3.1 and the expert, midpoint and footprint
versions of IMPACT World+ 2.2.1. One call lists them with their publisher,
licence, size and whether they are already there; another downloads one,
keeps it only when its size and SHA-256 match the catalogue, stores it as an
upload and loads it. The download follows no redirect, stops one byte past
the stated size and says who asks, since some file servers refuse a request
that does not.

Wire revision 102; pyvolca gains available_method_collections() and
install_method_collection(name).
…dress with its reason

A download whose body stalls now ends after a minute of silence instead of
holding the request open. An address the HTTP library cannot parse is caught
with the other download failures. The test of the catalogue now requires the
lowercase SHA-256 the engine compares against, and the changelog names the
data version that carries the catalogue.
@ccomb ccomb added the agent-reviewed pr-cycle ran label Oct 31, 2026
Main took revision 102 for the basis of a score on its contributions.
@ccomb
ccomb changed the base branch from simapro-padded-columns to main October 31, 2026 17:08
@ccomb
ccomb merged commit 107863c into main Oct 31, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-reviewed pr-cycle ran

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant