feat(semantic-layer): define credential readiness and backend errors in the SDK

UPDATING.md

@@ -24,6 +24,32 @@ assists people when migrating to a new version.
 
 ## Next
 
+- **Semantic-layer schema readiness and failures.** Both provider schema hooks
+  require keyword-only `readiness: SemanticCredentialReadiness` alongside
+  `transport`. This plain Enum is exported by `superset_core.semantic_layers`.
+  Discover metadata only when `readiness is SemanticCredentialReadiness.RESOLVED`
+  and required configuration fields are valid; otherwise return a static schema
+  without I/O. Do not interpret host masking placeholders. Upgrade host and
+  provider schema implementations together; omitted arguments fail explicitly.
+  `SemanticBackendError` signals execution or invalid-result faults with generic
+  host guidance (500); `ProviderRequestError` keeps safe transport detail (400),
+  and query refusal/completeness retain their separate meanings. Provider error
+  translations should suppress diagnostic chains. The semantic execution boundary
+  logs typed failures without diagnostic chains. Unknown failures retain their
+  type at ERROR and their full traceback/chain at DEBUG in server logs only. See the
+  [provider boundary guide](docs/developer_docs/extensions/semantic-provider-boundary.md).
+
+- **Semantic-layer SDK transport argument.** Provider implementations must accept
+  the required keyword-only `transport: ProviderTransport` in `from_configuration`,
+  `get_configuration_schema`, and `get_runtime_schema`. Superset supplies it even
+  for static schema discovery. Store it on the layer and pass it to views and HTTP
+  helpers; do not construct HTTP clients for provider endpoints. Every request now
+  requires an absolute monotonic deadline and a response byte cap. The default
+  host policy allows public HTTPS destinations only; the provider-specific
+  `SEMANTIC_LAYER_ALLOW_INTERNAL_HOSTS = True` setting also allows HTTP and internal
+  destinations. Both policies ignore environment proxies and preserve budgets and
+  TLS hostname verification. See the [provider transport guide](docs/developer_docs/extensions/semantic-provider-transport.md).
+
 - With a Redis Sentinel coordination backend (`DISTRIBUTED_COORDINATION_CONFIG`
   using `RedisSentinelCache`), coordination commands now honor `CACHE_REDIS_DB` and
   `CACHE_REDIS_USER`. Previously they always ran on DB 0 and without the ACL user.

docs/developer_docs/extensions/extension-points/semantic-layers.md

@@ -35,8 +35,10 @@ Extensions can register custom semantic layer implementations that allow Superse
 from typing import Any
 
 from superset_core.semantic_layers.context import SemanticCredentials, SemanticExecutionContext
+from superset_core.semantic_layers.config import SemanticCredentialReadiness
 from superset_core.semantic_layers.decorators import semantic_layer
 from superset_core.semantic_layers.layer import SemanticLayer
+from superset_core.semantic_layers.transport import ProviderTransport
 
 from my_extension.config import MyConfig
 from my_extension.view import MySemanticView
@@ -52,26 +54,34 @@ class MySemanticLayer(SemanticLayer[MyConfig, MySemanticView]):
 
     def __init__(
         self, configuration: MyConfig, *, cache_token: str,
-        context: SemanticExecutionContext,
+        context: SemanticExecutionContext, transport: ProviderTransport,
     ) -> None:
         self.configuration: MyConfig = configuration
         self.cache_token: str = cache_token
         self.context: SemanticExecutionContext = context
+        self.transport: ProviderTransport = transport
 
     @classmethod
     def from_configuration(
         cls, configuration: dict[str, Any], *, cache_token: str,
-        context: SemanticExecutionContext, credentials: SemanticCredentials | None = None,
+        context: SemanticExecutionContext, transport: ProviderTransport,
+        credentials: SemanticCredentials | None = None,
     ) -> "MySemanticLayer":
         config: MyConfig = MyConfig.model_validate(configuration)
-        return cls(config, cache_token=cache_token, context=context)
+        return cls(config, cache_token=cache_token, context=context, transport=transport)
 
     @classmethod
-    def get_configuration_schema(cls, configuration=None) -> dict:
+    def get_configuration_schema(
+        cls, configuration: MyConfig | None = None, *, transport: ProviderTransport,
+        readiness: SemanticCredentialReadiness,
+    ) -> dict[str, Any]:
         return MyConfig.model_json_schema()
 
     @classmethod
-    def get_runtime_schema(cls, configuration=None, runtime_data=None) -> dict:
+    def get_runtime_schema(
+        cls, configuration: MyConfig, runtime_data: dict[str, Any] | None = None,
+        *, transport: ProviderTransport, readiness: SemanticCredentialReadiness,
+    ) -> dict[str, Any]:
         return {"type": "object", "properties": {}}
 
     def get_semantic_views(self, runtime_configuration: dict) -> set[MySemanticView]:
@@ -96,9 +106,12 @@ Store the parsed configuration instance on the writable `self.configuration`
 attribute, using the provider's configuration type. Assigning a raw dictionary is
 a type error by design; a read-only `@property` is not supported.
 The host reads this attribute to request the runtime schema.
-`get_configuration_schema()` and `get_configuration_schema(None)` must
-return a static schema without I/O or a constructed provider: the host calls
-this form before credentials exist and when identifying secret fields.
+Both schema hooks require `transport` and `readiness` keyword arguments.
+`get_configuration_schema` with absent configuration or readiness other than
+`SemanticCredentialReadiness.RESOLVED` must return a static schema without I/O
+or a constructed provider. The host uses this form before credentials exist and
+when identifying secret fields. See [readiness and failures](../semantic-provider-boundary.md)
+for the schema contract and safe error signals.
 
 `get_values(dimension, filters)` returns its Arrow value column under
 `dimension.name` for nonempty results. The private `id` and display `verbose_name`
@@ -131,10 +144,20 @@ Import decorated provider modules **inside** a test or fixture that requests
 
 ```python
 def test_configuration(semantic_layer_registration: None) -> None:
+    from unittest.mock import create_autospec, MagicMock
+
+    from superset_core.semantic_layers.transport import ProviderTransport
+    from superset_core.semantic_layers.config import SemanticCredentialReadiness
+
     from my_extension.layer import MySemanticLayer
 
-    schema: dict[str, object] = MySemanticLayer.get_configuration_schema()
+    transport: MagicMock = create_autospec(ProviderTransport, instance=True)
+    schema: dict[str, object] = MySemanticLayer.get_configuration_schema(
+        transport=transport,
+        readiness=SemanticCredentialReadiness.UNRESOLVED,
+    )
     assert schema["type"] == "object"
+    transport.request.assert_not_called()
 ```
 
 The fixture substitutes only the SDK registration decorator and restores it after
@@ -150,6 +173,14 @@ explicitly refused, not claimed as a full provider implementation.
 See [Testing semantic providers](../../semantic-layer-provider-testing.md) for the
 reusable SDK conformance suite and its provider-owned fixtures.
 
+## HTTP transport
+
+Provider construction and both schema-discovery entry points also receive a required
+keyword-only `transport: ProviderTransport`. It is a separate host capability, not
+a field of the execution context or a component of binding identity. See the
+[provider transport guide](../semantic-provider-transport.md) for budgets, destination
+policy, errors, and a test fake.
+
 ## Execution context
 
 The unreleased SDK uses one `SemanticLayer.from_configuration` factory with

docs/developer_docs/extensions/semantic-provider-boundary.md

@@ -0,0 +1,105 @@
+---
+title: Semantic provider readiness and failures
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Semantic provider readiness and failures
+
+## Schema discovery
+
+Both schema methods require keyword-only `transport` and `readiness`. Import
+`SemanticCredentialReadiness` from `superset_core.semantic_layers.config` (also
+exported from `superset_core.semantic_layers`). It is a plain Enum, not a string.
+
+```python
+@classmethod
+def get_configuration_schema(
+    cls, configuration: MyConfig | None = None, *,
+    transport: ProviderTransport, readiness: SemanticCredentialReadiness,
+) -> dict[str, Any]:
+    schema: dict[str, Any] = MyConfig.model_json_schema()
+    if readiness is not SemanticCredentialReadiness.RESOLVED:
+        return schema
+    if configuration is None or not configuration.password:
+        return schema
+    # Discover only after provider-specific required-field checks.
+    # Use the supplied transport for any HTTP request.
+    return schema
+```
+
+Apply the same positive identity check in `get_runtime_schema(configuration,
+runtime_data=None, *, transport, readiness)`. Strings such as `"resolved"`, booleans
+and unrelated objects must not enable discovery. A static schema must perform no
+network activity or credential lookup and must not echo secrets in defaults/options.
+Readiness is neither complete configuration nor authenticated/authorized access.
+Providers still validate required fields and allowlisted runtime overrides.
+An empty configuration object has no unresolved secrets and is `RESOLVED`;
+providers with required fields must still reject incomplete settings themselves.
+
+The host checks submitted preview values before partial validation can discard an
+unresolved field. A preview does not fetch or restore saved secrets. Runtime
+schemas use the saved connection after the existing layer access check. The host
+alone understands its masking convention; client payload fields never select the
+trusted readiness keyword. Static masking and fallback calls explicitly pass
+UNRESOLVED. Direct SDK hosts must supply an explicit state using their own secret
+resolution rules, without adopting Superset's mask.
+
+This is a coordinated schema API upgrade. Update all overrides and direct callers
+with the host/core pin containing transport and readiness; there is no fallback
+that retries without safety arguments. `SemanticLayer.from_configuration` also
+gains the required keyword-only `transport: ProviderTransport` parameter; update
+provider factories and direct callers alongside the host/core pin. The existing
+`cache_token`, `context`, and `credentials` parameters retain their contracts.
+No SQL parser limit or memo policy is added to the SDK.
+
+## Error signals
+
+| Situation | Signal | Host behavior |
+| --- | --- | --- |
+| Invalid or unsupported query intent | `SemanticQueryRejectedError` with a `SemanticQueryErrorCode` | Fixed actionable guidance, HTTP 400 |
+| Transport failure | Original SDK `ProviderRequestError` | Its sanitized message, query HTTP 400 |
+| Terminal execution failure or invalid result | `SemanticBackendError()` | Fixed generic backend guidance, HTTP 500 |
+| Incomplete or unverifiable result coverage | `SemanticResultCompletenessError` | Existing completeness guidance; never partial success |
+
+Use INVALID_QUERY for a rejected query payload, including query compilation errors.
+Keep a result-page coverage failure separate from initial/poll query rejection.
+If a retrieval budget prevents coverage proof, retain the provider's completeness
+classification; other transport errors propagate unchanged. A failed remote status
+is not Python cancellation. Preserve security, OAuth and cancellation control flow.
+Never retry without a filter or return an empty/partial successful result on failure.
+Transport failures during query preparation (including provider construction and
+dimension/metric discovery), dispatch, and filter-value preparation use the same
+safe normalization. MCP `get_table` preserves its QueryError envelope with
+host-safe guidance. Semantic-layer structure and schema endpoints retain their
+own response contracts.
+
+Raise newly translated backend errors/refusals `from None`; never place backend
+responses, credentials or SQL in their public text. The host validates transport
+failure/status fields and reconstructs messages with a freshly sanitized provider
+label. Mutated fields and subclass formatters cannot supply public text; malformed
+transport fields become generic backend faults.
+
+Typed SDK failures log fixed event categories without causes or tracebacks.
+Unexpected exceptions log their type at ERROR and retain the full traceback and
+exception chain at DEBUG, server-side only. Keep these diagnostic logs restricted
+to trusted operators: they may contain provider details. Public responses remain
+fixed generic guidance and do not include the exception chain. The same logging
+policy applies to schema enrichment, runtime schemas and structure discovery.