Skip to content

data.ingest() silently stores empty objects for the NonReferenceInputs form its type signature accepts #456

Description

@g-despot

Summary

collection.data.ingest() accepts NonReferenceInputs<T> (the unwrapped { title: 'x' } form) in its type signature, but silently stores objects with no properties when given that form. The call reports success: UUIDs are returned, errors is empty, and hasErrors is false.

collection.data.insertMany() handles the same unwrapped input correctly, so the two batch entry points disagree on an input shape they both claim to accept.

This is silent data loss. A caller following the type signature gets a clean success response and an empty collection.

The type signature accepts the unwrapped form

src/collections/data/index.ts:88:

ingest: (objs: Iterable<DataObject<T> | NonReferenceInputs<T>>) => Promise<BatchObjectsReturn<T>>;

The implementation does not normalize it

src/collections/data/index.ts:251-257:

for (const obj of objs) {
  // eslint-disable-next-line no-await-in-loop
  await batching.addObject({
    collection: name,
    ...obj,          // <-- spread directly
    tenant,
  });
}

For a DataObject<T> the spread yields { collection, properties, tenant } and works. For a NonReferenceInputs<T> it yields { collection, title, tenant }properties is never set, so the object is created empty.

Compare insert in the same file, :287, which normalizes exactly this case:

parseObject(
  obj ? (DataGuards.isDataObject(obj) ? obj : ({ properties: obj } as InsertObject<T>)) : obj
),

DataGuards.isDataObject is defined at src/collections/serialize/index.ts:355 and is already the established guard for this discrimination. ingest never calls it.

Reproduction

import weaviate from 'weaviate-client';

const client = await weaviate.connectToLocal();

async function fresh(name) {
  if (await client.collections.exists(name)) await client.collections.delete(name);
  return client.collections.create({
    name,
    properties: [{ name: 'title', dataType: 'text' }],
    vectorizers: weaviate.configure.vectors.selfProvided(),
  });
}

// A — unwrapped NonReferenceInputs form
const a = await fresh('ReproIngestUnwrapped');
const ra = await a.data.ingest([{ title: 'alpha' }, { title: 'beta' }]);
console.log('uuids:', Object.keys(ra.uuids).length, 'errors:', Object.keys(ra.errors).length, 'hasErrors:', ra.hasErrors);
for await (const o of a.iterator()) console.log('STORED:', JSON.stringify(o.properties));

// B — wrapped DataObject form
const b = await fresh('ReproIngestWrapped');
await b.data.ingest([{ properties: { title: 'alpha' } }, { properties: { title: 'beta' } }]);
for await (const o of b.iterator()) console.log('STORED:', JSON.stringify(o.properties));

// C — insertMany with the same unwrapped form
const c = await fresh('ReproInsertMany');
await c.data.insertMany([{ title: 'alpha' }, { title: 'beta' }]);
for await (const o of c.iterator()) console.log('STORED:', JSON.stringify(o.properties));

Actual output

A: data.ingest([{ title: "..." }])          <- NonReferenceInputs form
  reported uuids   : 2
  reported errors  : 0
  hasErrors        : false
  STORED PROPERTIES: {}
  STORED PROPERTIES: {}

B: data.ingest([{ properties: { title } }]) <- DataObject form
  reported uuids   : 2
  STORED PROPERTIES: {"title":"beta"}
  STORED PROPERTIES: {"title":"alpha"}

C: data.insertMany([{ title: "..." }])      <- same unwrapped form
  STORED PROPERTIES: {"title":"alpha"}
  STORED PROPERTIES: {"title":"beta"}

Expected

Case A should store {"title":"alpha"} and {"title":"beta"}, matching cases B and C.

If the unwrapped form is not intended to be supported by ingest, then the type signature at :88 should drop NonReferenceInputs<T> so this fails at compile time rather than silently at runtime.

Suggested fix

Apply the same normalization insert already uses, inside the ingest loop:

for (const obj of objs) {
  await batching.addObject({
    collection: name,
    ...(DataGuards.isDataObject(obj) ? obj : { properties: obj }),
    tenant,
  });
}

Environment

  • weaviate-client 3.13.1
  • Weaviate server 1.38.0 (cr.weaviate.io/semitechnologies/weaviate:1.38.0, anonymous access, self-provided vectors)
  • Node.js v22.14.0, macOS

Note

This surfaced while updating the Weaviate quickstart documentation to use data.ingest(). Every plain property list had to be rewritten as .map((properties) => ({ properties })) to work, which is what led back to the signature mismatch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions