feat: Create projects inside a project group on bt functions push - #395
Conversation
Definitions can declare a project group via the JS SDK's
`projects.create({ name, projectGroupName })`. The functions runner forwards
it as `project_group_name`, and push passes it to `POST /v1/project` when it
creates a missing project, so users who only hold project-creation permission
on a group can push into new projects.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7faf421477
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| kind: "function_event", | ||
| project_id: projectId, | ||
| project_name: projectName, | ||
| project_group_name: selector.project_group_name, |
There was a problem hiding this comment.
When a prompt or function event references a different missing project—for example, a legacy tool whose project came from projects.create({ name, projectGroupName })—resolver.resolve() serializes that nested reference only as project_id: "name:...", while this field carries the group only for the event's owning project. Rust discovers the nested name via collect_project_name_placeholders_checked, but finds no corresponding group and calls create_project with None, so users who can create projects only within that group still cannot push this valid definition. Carry group metadata for every project passed through the resolver, rather than only for the top-level entry.
Useful? React with 👍 / 👎.
|
lemme merge the python pr in, then we can do it for python too. |
|
Latest downloadable build artifacts for this PR commit
Available artifact names
|
- Braintrust organizes projects into "project groups", and a user can
have permission to create projects inside a group without being allowed
to create projects anywhere else in the organization.
- A recent unreleased change added a `projectGroupName` option to
`initLogger`, `init`, `initDataset` and `Eval`.
When the named project didn't exist yet, it was created inside that
group.
- The problem is that loggers resolve their project every time they
start up, not just once.
- Passing a group switches the backend from a cheap, cached lookup to an
uncached "create or verify group membership" call that locks the whole
group's database row.
- In production that means every logger cold start, across every service
logging into that group, waits on the same lock.
- If the project already exists outside the group, or is later moved out
of it, the backend rejects the call, so logging fails because of a
permissions change rather than a code change.
- Creating a project is really a one-time setup step, so this PR takes
`projectGroupName` back out of `initLogger`, `init`, `initDataset` and
`Eval`.
They now send the same requests as before the earlier change.
- The option now lives on the push and publish path instead:
`projects.create({ name: "my-project", projectGroupName: "my-group" })`.
- Previously `projects.create()` only took a project name or id, and
`braintrust push` / `project.publish()` always created missing projects
at the organization level.
- Now those commands pass the group along when they register the
project, so a missing project is created inside that group.
- The standalone `bt` CLI reads the same SDK project definitions, and
braintrustdata/bt#395 makes `bt functions push` pass the group along
too.
- The earlier change hasn't been released, so removing the option from
the logger, experiment and dataset entrypoints doesn't break anyone.
2 participants
bt functions pushcan create missing projects when you pass--create-missing-projects. Before this PR, it always created them at the top level of the org, even if the user's code said the project should be in a project group.project_group_name.projectGroupNamefrom the project reference, and Python readsproject_group_namefrom theProjectobject. On SDK versions without project groups, the field is simply missing and nothing changes.project_group_nameto the create-project API, so the project lands in the right group.bt projects create,bt switch, sync, datasets) work exactly as before and don't set a group.