Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 57 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,58 @@
# tesserift
Comment thread
d33bs marked this conversation as resolved.
Measure how bioimaging storage layouts shape performance, analysis, and biological discovery.

Bioimaging data can follow a format specification and still perform poorly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i recommend adding a bit more specificity to what you mean by "format specification". Also, what do you mean by "perform" and "poorly"? Perform what? Poorly measured how?


We will study around **six public datasets from three sources**, with two datasets from each source:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering how you decided six datasets - is there a reason not to scan through every dataset from these three sources?


* **Image Data Resource (IDR):** established OME bioimaging datasets with different image shapes and workloads.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider defining OME before first use.

Also, what is an "image shape" and what is an "image workload"?

* **SSBD:** multidimensional OME-Zarr data, including 3D and time-series images.
* **Cell Painting Gallery / JUMP:** large-scale imaging data with associated metadata and feature tables.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* **Cell Painting Gallery / JUMP:** large-scale imaging data with associated metadata and feature tables.
* **Cell Painting Gallery:** large-scale imaging data from the Cell Painting assay (e.g., JUMP) with associated metadata and morphology feature tables.


Two datasets from each source will help us measure variation both **within** and **between** public data services.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two is not sufficient to measure these things - how tough would it be to measure all datasets? (i imagine what we're measuring will answer this question!)


Unlike general-purpose Zarr/Parquet benchmarks, we tie storage properties to bioimaging-specific workloads and test causality directly, not just correlation.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too terse. What is a "general purpose Zarr/Parquet benchmark"? What do you mean by "tie" - what is a workload (same question as prior)? Causality of what? Poor performance? (again, going back to defining poor performance)


The work will focus on four goals:

* **Measure format skew.** Compare chunking, compression, file counts, metadata, schemas, row groups, and related storage properties.
* **Find costly patterns.** Identify storage properties that predict slow access or analysis.
* **Measure real workloads.** Test image reads, table scans, filtering, joins, and related bioimaging tasks.
* **Turn evidence into rules.** Add tested performance rules to `zarr-lint` and `arrow-lint`.
Comment on lines +15 to +20

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is effective! I recommend moving this up a section or two. Starting with goals is usually helpful!

It would also be helpful to add a "motivation" answering "why we're focusing on these goals"? Was there an observation that triggered this research?

Add a brief description of what these linting tools are and what they do


Cost means wall-clock latency and throughput, measured on a fixed compute and network setup.

We will not search for many examples of the same problem.
When we find an important pattern, we will create controlled versions of a dataset that change that property.
Comment thread
d33bs marked this conversation as resolved.

This lets us test whether the storage difference actually causes the observed performance change.
Comment thread
d33bs marked this conversation as resolved.

The main question is:

**How much do storage layouts vary across public bioimaging resources, and can that variation predict the cost of accessing and analyzing the data?**

The manuscript will also examine how these costs affect interactive exploration and large-scale biological analysis.
Comment thread
d33bs marked this conversation as resolved.

## Operational outline

Keep the complete study in one public GitHub repository.

* Write the manuscript with **Quarto Manuscripts**.
* Keep analysis and benchmark code beside the manuscript.
* Store generated figures and tables in the repository.
* Make benchmarks reproducible from scripts or workflows.
* Record dataset versions, inputs, and benchmark settings.
* Develop the manuscript and analysis together.
* Publish the completed manuscript as a **bioRxiv preprint**.
* Select a **right-size peer-reviewed journal** after the results and scope are clear.
Comment thread
d33bs marked this conversation as resolved.

The repository becomes the manuscript, analysis, figures, and reproducibility record.

## Timeline
Comment thread
d33bs marked this conversation as resolved.

Rough estimate, ~9 months total:

* Months 1-2: select datasets, set up benchmarking infrastructure.
* Months 3-4: measure format skew and workload cost.
* Months 5-6: build controlled dataset variants, test causality.
* Months 7-8: draft the manuscript, publish the bioRxiv preprint.
* Month 9: submit to a right-size journal.