Skip to content

feat: dataset creation ui - #386

Merged
Scr4tch587 merged 8 commits into
kai/proposal-api-clientfrom
kai/create-dataset-ui
Jul 28, 2026
Merged

feat: dataset creation ui#386
Scr4tch587 merged 8 commits into
kai/proposal-api-clientfrom
kai/create-dataset-ui

Conversation

@Scr4tch587

Copy link
Copy Markdown
Collaborator

What changed

New create view (`components/create-dataset.tsx`, reachable via the "new dataset" header button):

  • Proposer section (required): name, Wat Street team, discord user, and a "what is this dataset for?" description — all shown to reviewers in the PR body
  • Basics: name/version inputs, calendar + granularity selects, start-date
  • Schema: dynamic key/type rows (str · int · float · bool)
  • Dependencies: picker populated from `GET /datasets` plus optional lookback (`5d`-style, zod-validated)
  • Builder script: CodeMirror editor with Python highlighting and a template pre-filled
  • Extras: optional requirements.txt; env-vars checkbox reveals a `.env.template` field with a "secrets never go in the PR" note
  • zod + react-hook-form validation mirrors the server's rules (name regex, semver, date format, lookback format), and server-side 400/409 messages render inline on submit
  • success screen links the opened PR and repeats the manual-`.env` warning for env-vars datasets

The view is lazy-loaded (`React.lazy`) so CodeMirror's chunk (~540 kB) never loads during normal browsing. shadcn `select`/`checkbox`/`textarea` components added; new deps: `@uiw/react-codemirror`, `@codemirror/lang-python`.

Why

The UI half of the PR-based dataset-creation flow — pairs with the `POST /datasets` endpoint below this PR in the stack.

Benefit

Anyone on the team can propose a dataset without touching git; reviewers get a fully-attributed, pre-validated, pre-linted PR.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
schemaFields: z
.array(
z.object({
key: z.string().min(1, "field name required"),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI] schemaFields.key doesn't .trim(), unlike the other text fields (authorName, team, discordUser, description). A trailing space passes validation silently, and the backend will treat "price " as a different TOML key than "price" — builder output would then fail schema validation with a confusing "missing key" error. Suggest trimming this field too.

dependencies: dict[str, dict[datetime, list[dict]]],
timestamp: datetime,
) -> list[dict[str, Any]]:
# return one dict per row for this timestamp, matching the schema

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should add another (python) comment giving an example return so people have an idea how to structure the function

requirements_txt: values.requirementsTxt.trim()
? values.requirementsTxt
: undefined,
env_template: values.envTemplate.trim() ? values.envTemplate : undefined,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI] Unchecking "needs env vars" hides the envTemplate textarea (line 624) but doesn't clear its value, and toPayload here sends env_template whenever it's non-empty, regardless of envVars. A user can end up with env-vars = false in the generated config but a .env.template file still added. Suggest clearing envTemplate when envVarsEnabled flips false, or gating this line on values.envVars too.

@Blackgaurd

Copy link
Copy Markdown
Collaborator

[AI] The red validation hint text doesn't appear on first-time typing because useForm (create-dataset.tsx:132) doesn't set a mode, so react-hook-form defaults to mode: "onSubmit" — fields only validate after the first submit attempt. Consider mode: "onBlur" (or "onChange") so users get feedback as they fill out the form instead of only after clicking submit once.

@Blackgaurd

Blackgaurd commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

[AI] Reported error message looks truncated/garbled on submit failure (e.g. "builder script fails lint:" errors from ruff). Likely cause: the error <p> (create-dataset.tsx:652-654) doesn't set whitespace-pre-wrap, so ruff's multi-line stdout (one line per violation) collapses into a single run-on line under default HTML whitespace handling — nothing appears to be truncated server-side (backend passes the full message through with no length limit). Might be worth preserving newlines here so multi-line backend errors are readable.

That said, actually failing lint on this form isn't itself a problem — backend CI on the resulting PR will catch any lint errors regardless, and the proposer can revise their code from the PR. This is just about making the in-form error message readable, not about needing to prevent lint failures here.

</section>

<section className="space-y-4">
<SectionHeading>Builder script</SectionHeading>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI] Suggestion: add a "reset" button next to the builder script editor that restores field.value back to DEFAULT_BUILDER. Once someone starts editing/experimenting here it's easy to end up in a broken state with no quick way back to the known-good starting template.

calendar: values.calendar,
granularity: values.granularity,
start_date: values.startDate,
schema: Object.fromEntries(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AI] Object.fromEntries(values.schemaFields.map((field) => [field.key, field.type])) silently drops earlier rows if two schema fields share the same name (last one wins) — no duplicate-key check in the zod schema or before submit. Worth a .refine() on schemaFields checking all keys are unique, so a fat-fingered duplicate doesn't quietly disappear from the proposal.

Scr4tch587 and others added 2 commits July 27, 2026 22:06
docs: document dataset proposal flow in specs
…ordances

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Scr4tch587
Scr4tch587 merged commit 38da8cf into kai/proposal-api-client Jul 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants