Skip to content

zotero adapter discards volume/edition/series fields, making multi-volume sets indistinguishable #41

Description

@skishchampi

Summary

The Zotero adapter reads every field Zotero holds for an item, then keeps only seven of them. volume, edition, series, seriesNumber, numberOfVolumes, publisher, and place are fetched from SQLite and dropped on the floor.

For a multi-volume set — a collected-works edition where every volume shares one title, one date, and one set of creators — this makes search results unusable for citation. Every volume comes back as the same undifferentiated title, and the field that would tell them apart is sitting in the database, already loaded into memory.

Where

src/partial_recall/corpus/adapters/zotero.py

  • _fetch_item_fields() (~line 345) correctly returns all fields as a dict.
  • The consumer at ~lines 188-213 lifts only title, date, abstractNote, archive, archiveLocation, callNumber, libraryCatalog into the Item, and discards the rest.

Impact

  1. Search results can't be cited. semantic_search and search_fulltext return N identical titles for an N-volume set. A caller has to open the PDFs by hand to find out which volume a hit came from — which defeats the point of the index.
  2. get_item_details doesn't help. It surfaces library_location richness (archive, call number, catalog) but not the bibliographic fields, so there is no second call that recovers the volume either.
  3. Metadata hashes collide. _compute_metadata_hash(title, date, creators, abstract) is fed only fields that are identical across the whole set, so every volume of a multi-volume work hashes the same. Any logic keying on that hash treats 20 distinct volumes as one.

Filenames are not a workaround — they are assigned by the exporting client and frequently do not match the volume number. In the set that surfaced this, the file named ...speeches 18.pdf is Volume 17 Part One and ...speeches 20.pdf is Volume 17 Part Three. Trusting the filename produces a citation to a volume that does not exist.

Suggested fix

Carry the bibliographic fields through to Item and expose them in semantic_search / search_fulltext / get_item_details output, the same way the v0.2.4 library-location fields were added.

Minimum useful set: volume, edition, series, seriesNumber, numberOfVolumes.

Worth deciding separately whether volume/edition should join _compute_metadata_hash, since the collision above is arguably a distinct bug.

Repro

  1. Add a multi-volume work to Zotero as N book items sharing a title/date/creators, each with its volume field set.
  2. partial-recall index --source zotero
  3. Search for a phrase appearing in one volume.
  4. Results identify the work but not the volume; get_item_details on the hit does not return volume either.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions