Emit x-databricks-launch-stage for all stamped launch stages - #6516
Emit x-databricks-launch-stage for all stamped launch stages#6516Sankalp-Mittal wants to merge 7 commits into
Conversation
The schema generator only emitted x-databricks-launch-stage for private-preview fields, bundling two concerns in one branch: hiding private-preview fields from editor completions (DoNotSuggest) and emitting the machine-readable launch stage. Downstream codegen could therefore only distinguish private-preview from everything else. Split the two concerns: DoNotSuggest stays private-preview-only, while every field the contract stamps with a launch stage — GA, PUBLIC_BETA, PUBLIC_PREVIEW, PRIVATE_PREVIEW — now emits x-databricks-launch-stage so downstream tooling can read each field's stability. A field the contract leaves unstamped stays unmarked rather than defaulting to GA, via the new parseFieldLaunchStage (the enum path keeps dropping GA, unchanged). Regenerated jsonschema.json. pydabs codegen is unaffected (it branches only on PRIVATE_PREVIEW), so python/databricks/bundles is unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Waiting for approvalBased on git history, these people are best suited to review:
Eligible reviewers: Suggestions based on git history. See OWNERS for ownership rules. |
launchStageOverrides maps whole resource Go-types to a launch stage
(the Postgres* resources at Public Beta), but OverrideLaunchStage was
only applied to fields, so the type's own (self) schema stayed
unstamped. Apply the override to the type descriptor too: the contract
carries no type-level stage, so passing GA ("") returns the override
when one is set, else "". The self descriptor already flows through
assignAnnotation, so the type-level x-databricks-launch-stage now lands
in jsonschema.json.
This makes launchStageOverrides a per-resource stability registry that
tags both the type and its fields.
Regenerated jsonschema.json: the 7 Postgres* resource types gain the
type-level PUBLIC_BETA marker (and the [Beta] description prefix, matching
how their fields already render). pydabs codegen is unaffected.
Co-authored-by: Isaac <no-reply@databricks.com>
| // stability, not just previews. An empty stage means the contract assigns none; | ||
| // it stays empty (unmarked) instead of defaulting to GA, so only fields the | ||
| // contract actually stamps carry a stage. | ||
| func parseFieldLaunchStage(launchStage string) (clijson.LaunchStage, error) { |
There was a problem hiding this comment.
Can inline, the function is only used once.
There was a problem hiding this comment.
But then I'd have to remove the unit test that asserts the behaviour for this function, I think it's better to keep it
| assert.Error(t, err) | ||
| } | ||
|
|
||
| func TestParseFieldLaunchStage(t *testing.T) { |
There was a problem hiding this comment.
Tests are unnecessary. The clijson methods are already tested.
| require.NoError(t, err) | ||
|
|
||
| self := annotations[getPath(reflect.TypeFor[resources.PostgresRole]())].Self | ||
| assert.Equal(t, clijson.LaunchStagePublicBeta, self.LaunchStage) |
There was a problem hiding this comment.
Will this test break once postgres role moves of beta? Where is this value even coming from?
There was a problem hiding this comment.
For your context: postgres resources in DABs are pinned to Beta despite APIs being GA (#6226)
I assume when we remove that override eventually, this test will fail and be updated/removed alongside.
| require.NoError(t, err) | ||
|
|
||
| self := annotations[getPath(reflect.TypeFor[resources.PostgresRole]())].Self | ||
| assert.Equal(t, clijson.LaunchStagePublicBeta, self.LaunchStage) |
There was a problem hiding this comment.
For your context: postgres resources in DABs are pinned to Beta despite APIs being GA (#6226)
I assume when we remove that override eventually, this test will fail and be updated/removed alongside.
| // The contract carries no schema-level launch stage, so a type's stage | ||
| // comes only from the override map (launchStageOverrides), which stamps | ||
| // whole resources — e.g. Postgres* at Public Beta. Passing "" (GA, the | ||
| // least restrictive stage) returns the override when one is set for the | ||
| // type, else "". Enum schemas do carry per-value launch stages below. | ||
| typeStage := annotation.OverrideLaunchStage(basePath, "") |
There was a problem hiding this comment.
typeStage is confusing - let's maybe call it what it is: typeLaunchStage or typeOverriddenLaunchStage. But this also feels a bit hacky (mostly because it's confusing to find that OverrideLaunchStage is called in two locations). Why do we need this extra override here if no other resources have launch stage in the schema?
The comment is too verbose and not helping: Remove mention of override map (that's internal to OverrideLaunchStage).
There was a problem hiding this comment.
So this is to add the Launch Stage tag for resources in jsonschema.json, if not overriden each resource has release stage GA, so that's why I use this here, I did not want to hardcode overrides in 2 places
Integration test reportCommit: 994851a
Top 6 slowest tests (at least 2 minutes):
|
4 participants
Changes
The bundle schema generator only emitted
x-databricks-launch-stagefor private-preview fieldsAs a result, downstream codegen could only tell private-preview fields apart from everything else — public-preview, public-beta, and GA fields all looked identical (unmarked).
GA,PUBLIC_BETA,PUBLIC_PREVIEW,PRIVATE_PREVIEW— now emitsx-databricks-launch-stage, so downstream tooling can read each field's stability.parseFieldLaunchStage. The enum path keeps dropping GA (normalizeLaunchStage), unchanged.The human-readable
[Public Preview]/[Beta]description prefix (PreviewTag) was already emitted for all stages, so this only adds the machine-readable field.Keeping in line with the existing pattern
PRIVATE_PREVIEWhides from autocompleteResource types
launchStageOverridesmaps whole resource Go-types to a launch stage (thePostgres*resources at Public Beta). That override now applies to the resource type descriptor as well, not just its fields, so the 7Postgres*resource types carry a type-levelx-databricks-launch-stage: PUBLIC_BETA(and the[Beta]description prefix, matching how their fields already render). This makeslaunchStageOverridesa per-resource stability registry that tags both the type and its fields.Generated output
bundle/schema/jsonschema.jsonregenerated via./task generate-schema. The diff is large by design — many fields now carry an explicit stage marker. No existing markers were removed.Verification
pydabscodegen is a no-op: it branches only onPRIVATE_PREVIEW, sopython/databricks/bundles/**is unchanged after./task pydabs-codegen.TestExtractAnnotationsStampsTypeLaunchStagefor the type-level stamp.This pull request and its description were written by Isaac.