Skip to content

Relax thread-safety requirement when consuming pipelines #40

Description

@mzabani

It would be nice to allow different threads to consume the results of different statements from the same pipeline.

This might be achievable if we let consumers steal statements from a different thread. That way every statement always has an owning thread, so pipeline draining should still work. However, stealing must only be allowed once to avoid partial results and whatnot.

What is unclear is whether we can provide good error messages on misuse. Right now if a thread does not consume results in order, we throw an informative exception.
But with this change, we'll have to make threads block until it's their turn, instead.

This also means poll-checking thread liveness so we don't deadlock if threads die before consuming earlier results, and throwing informative exceptions in that case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions