Skip to content

Commit 10bb4cb

Browse files
localstack-spiral[bot]spiralhovaesco
authored
LAV-4589: Round-trip storage-integration private-endpoint preferences (#4766)
Persist USE_PRIVATELINK_ENDPOINT as a custom Snowflake Boolean, defaulting to FALSE, and project its stored value through the existing DESCRIBE row. ALTER only updates it when supplied. Keep every existing SHOW column, unrelated DESCRIBE row, identity producer and endpoint/credential-selection path unchanged. Validate literal types using the existing storage-property compilation-error pattern. Parse storage-integration UNSET lists and reject the private-endpoint property before updating any state, including mixed-property lists. Azure platform rejections also precede CREATE OR REPLACE deletion and ALTER updates. Added 66 real-Snowflake snapshot cases; errors preserve the complete connector code, SQLSTATE and message including compilation headers and line breaks. State assertions include ENABLED/COMMENT values and all four cells of the USE_PRIVATELINK_ENDPOINT row. The unrelated ENABLED property_default drift is left to LAV-4592, matching the existing projection-test convention. Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud> ## Verification - Cloud capture: TEST_TARGET=SNOWFLAKE_CLOUD PYTEST_XDIST_ARGS='' make test-compat PYTEST_PATH=tests/queries/stages/test_storage_integration_privatelink.py PYTEST_ARGS='--snapshot-update' (66 passed). - make check (passed; vendored tests separately run because this target compares committed HEAD). - make test-vendored-sqlparser (passed, including new UNSET AST round-trip and malformed-list cases). - make test-compat PYTEST_PATH='tests/queries/stages/test_storage_integration_privatelink.py tests/queries/stages/test_storage_integrations.py tests/queries/stages/test_stages.py::test_get_stage_and_storage_integration tests/queries/ddl/test_integrations.py::test_desc_integration tests/queries/ddl/test_integrations.py::test_desc_integration_not_found' PYTEST_XDIST_ARGS='' (71 passed). - Vendored fuzz command attempted; unavailable because cargo-hfuzz is not installed (cargo: no such command: `hfuzz`). ## Deviations - Real Snowflake on the available AWS platform rejects explicit Azure TRUE and FALSE on CREATE and ALTER with 511300/0A000: "SQL compilation error: Privatelink endpoints are not supported with 'Azure Storage' locations on 'aws' platform." Implemented the captured rejection and preserved omitted FALSE metadata instead of inventing an Azure success path. Successful Azure explicit-preference round-trips are Cloud-platform-blocked; no infrastructure, credentials, edition gate or public configuration was added. - ADR 097 is accepted but explicitly implementation-deferred until the first stable release. Added the preference using the existing custom-typed store column and idempotent ADD COLUMN convention rather than introducing server-owned migrations ahead of that boundary change. ## Test matrix S3 CREATE None × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 CREATE TRUE × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-TRUE] S3 CREATE FALSE × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-FALSE] S3 ALTER SET TRUE × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 preference after SET TRUE × unrelated COMMENT/ENABLED change -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 DESCRIBE after SET TRUE × value and property_default -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 ALTER SET FALSE × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 preference after SET FALSE × unrelated COMMENT/ENABLED change -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 DESCRIBE after SET FALSE × value and property_default -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] S3 SHOW × unchanged six-column projection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[S3-None] AZURE CREATE None × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE CREATE TRUE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-TRUE] AZURE CREATE OR REPLACE TRUE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-TRUE] AZURE CREATE FALSE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-FALSE] AZURE CREATE OR REPLACE FALSE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-FALSE] AZURE ALTER SET TRUE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE preference after SET TRUE × unrelated COMMENT/ENABLED change -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE DESCRIBE after SET TRUE × value and property_default -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE ALTER SET FALSE × 511300 rejection, prior state preserved -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE preference after SET FALSE × unrelated COMMENT/ENABLED change -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE DESCRIBE after SET FALSE × value and property_default -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE SHOW × unchanged six-column projection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_roundtrip[AZURE-None] AZURE CREATE lowercase false × 511300 rejection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[AZURE-false] AZURE ALTER lowercase false × 511300 rejection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[AZURE-false] AZURE CREATE lowercase true × 511300 rejection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[AZURE-true] AZURE ALTER lowercase true × 511300 rejection -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[AZURE-true] S3 CREATE lowercase false × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[S3-false] S3 ALTER lowercase false × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[S3-false] S3 CREATE lowercase true × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[S3-true] S3 ALTER lowercase true × success -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_boolean_forms[S3-true] AZURE ALTER $$true$$ × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-$$true$$] AZURE ALTER 'false' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-'false'] AZURE ALTER 'invalid' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-'invalid'] AZURE ALTER 'true' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-'true'] AZURE ALTER (TRUE) × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-(TRUE)] AZURE ALTER (invalid='value') × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-(invalid='value')] AZURE ALTER -1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER--1] AZURE ALTER 0 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-0] AZURE ALTER 1.5 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-1.5] AZURE ALTER 1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-1] AZURE ALTER NULL × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-NULL] AZURE ALTER invalid × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-ALTER-invalid] AZURE CREATE $$true$$ × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-$$true$$] AZURE CREATE 'false' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-'false'] AZURE CREATE 'invalid' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-'invalid'] AZURE CREATE 'true' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-'true'] AZURE CREATE (TRUE) × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-(TRUE)] AZURE CREATE (invalid='value') × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-(invalid='value')] AZURE CREATE -1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE--1] AZURE CREATE 0 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-0] AZURE CREATE 1.5 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-1.5] AZURE CREATE 1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-1] AZURE CREATE NULL × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-NULL] AZURE CREATE invalid × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[AZURE-CREATE-invalid] S3 ALTER $$true$$ × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-$$true$$] S3 ALTER 'false' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-'false'] S3 ALTER 'invalid' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-'invalid'] S3 ALTER 'true' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-'true'] S3 ALTER (TRUE) × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-(TRUE)] S3 ALTER (invalid='value') × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-(invalid='value')] S3 ALTER -1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER--1] S3 ALTER 0 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-0] S3 ALTER 1.5 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-1.5] S3 ALTER 1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-1] S3 ALTER NULL × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-NULL] S3 ALTER invalid × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-ALTER-invalid] S3 CREATE $$true$$ × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-$$true$$] S3 CREATE 'false' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-'false'] S3 CREATE 'invalid' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-'invalid'] S3 CREATE 'true' × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-'true'] S3 CREATE (TRUE) × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-(TRUE)] S3 CREATE (invalid='value') × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-(invalid='value')] S3 CREATE -1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE--1] S3 CREATE 0 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-0] S3 CREATE 1.5 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-1.5] S3 CREATE 1 × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-1] S3 CREATE NULL × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-NULL] S3 CREATE invalid × complete invalid-value error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_invalid[S3-CREATE-invalid] AZURE UNSET COMMENT, USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[AZURE-COMMENT, USE_PRIVATELINK_ENDPOINT] AZURE UNSET STORAGE_AWS_OBJECT_ACL, USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[AZURE-STORAGE_AWS_OBJECT_ACL, USE_PRIVATELINK_ENDPOINT] AZURE UNSET USE_PRIVATELINK_ENDPOINT, COMMENT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[AZURE-USE_PRIVATELINK_ENDPOINT, COMMENT] AZURE UNSET USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[AZURE-USE_PRIVATELINK_ENDPOINT] S3 UNSET COMMENT, USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[S3-COMMENT, USE_PRIVATELINK_ENDPOINT] S3 UNSET STORAGE_AWS_OBJECT_ACL, USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[S3-STORAGE_AWS_OBJECT_ACL, USE_PRIVATELINK_ENDPOINT] S3 UNSET USE_PRIVATELINK_ENDPOINT, COMMENT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[S3-USE_PRIVATELINK_ENDPOINT, COMMENT] S3 UNSET USE_PRIVATELINK_ENDPOINT × complete 003118 error and prior-state preservation -> tests/queries/stages/test_storage_integration_privatelink.py::test_storage_private_endpoint_unset[S3-USE_PRIVATELINK_ENDPOINT] S3 omitted/FALSE × existing stage projection -> tests/queries/stages/test_stages.py::test_get_stage_and_storage_integration S3 omitted/FALSE × existing ALTER/DESCRIBE/SHOW projections -> tests/queries/stages/test_storage_integrations.py::test_create_alter_storage_integration Azure omitted × existing provider-identity projection -> tests/queries/stages/test_storage_integrations.py::test_create_describe_azure_storage_integration S3 omitted/FALSE × generic integration DESCRIBE projection -> tests/queries/ddl/test_integrations.py::test_desc_integration Azure explicit TRUE/FALSE × successful round-trip -> uncovered: Cloud AWS platform rejects these inputs with captured 511300/0A000; omission and rejection/state preservation are covered above. UNSET grammar × valid SET/UNSET AST round-trips and malformed lists -> vendor/sqlparser/tests/sqlparser_snowflake.rs::test_storage_integration_unset_options Other provider metadata/defaults, general lifecycle aliases and successful UNSET of other properties -> uncovered: explicitly separate siblings LAV-4592, LAV-4593, LAV-4594 and LAV-4595. Co-authored-by: spiral <spiral@localhost> Co-authored-by: Przemek Denkiewicz <przemyslaw.denkiewicz@localstack.cloud>
1 parent baad3a9 commit 10bb4cb

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

‎tests/sqlparser_snowflake.rs‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11450,3 +11450,21 @@ fn test_grant_create_backup_schema_privileges() {
1145011450
assert!(snowflake().parse_sql_statements(sql).is_err(), "{sql}");
1145111451
}
1145211452
}
11453+
11454+
#[test]
11455+
fn test_storage_integration_unset_options() {
11456+
for statement in [
11457+
"ALTER STORAGE INTEGRATION storage UNSET USE_PRIVATELINK_ENDPOINT",
11458+
"ALTER STORAGE INTEGRATION IF EXISTS \"UNSET\" UNSET COMMENT, USE_PRIVATELINK_ENDPOINT",
11459+
"ALTER STORAGE INTEGRATION storage SET USE_PRIVATELINK_ENDPOINT=true",
11460+
] {
11461+
snowflake().verified_stmt(statement);
11462+
}
11463+
for statement in [
11464+
"ALTER STORAGE INTEGRATION storage UNSET",
11465+
"ALTER STORAGE INTEGRATION storage UNSET USE_PRIVATELINK_ENDPOINT = true",
11466+
"ALTER STORAGE INTEGRATION storage UNSET USE_PRIVATELINK_ENDPOINT,",
11467+
] {
11468+
assert!(snowflake().parse_sql_statements(statement).is_err());
11469+
}
11470+
}

0 commit comments

Comments
 (0)