Skip to content

Migrate Azure Container Apps Terraform template here, and fix ingress, unscanned routes, and image pinning - #28

Open
David Larsen (dc-larsen) wants to merge 15 commits into
mainfrom
migrate-azure-container-apps-terraform
Open

Migrate Azure Container Apps Terraform template here, and fix ingress, unscanned routes, and image pinning#28
David Larsen (dc-larsen) wants to merge 15 commits into
mainfrom
migrate-azure-container-apps-terraform

Conversation

@dc-larsen

@dc-larsen David Larsen (dc-larsen) commented Sep 5, 2026

Copy link
Copy Markdown

Migrates the Azure Container Apps Terraform template out of socketdev-demo/socket-firewall-azure-container-apps into this repo, with full commit history, and fixes three defects found while reviewing it for a customer deployment.

Same pattern as the helm/ migration. New home is terraform/azure-container-apps/, leaving room for the AWS ECS Fargate and GCP Cloud Run templates that also live in demo repos today.

Why now

A customer is standing up this template in Azure to cover GitHub-hosted runners. Reviewing it before handing it over turned up two problems that fail quietly, so they are worth fixing centrally rather than in one customer's copy.

The fixes

1. Ingress was unreachable from the VNet, and pointed at the wrong port.

ingress {
  external_enabled = false   # scopes ingress to the Container Apps env, not the VNet
  target_port      = 8443    # TLS listener
  transport        = "http"  # ...receiving cleartext
}

On an internal environment, external_enabled = false scopes ingress to the Container Apps environment only, so nothing else in the VNet can reach the firewall. Separately, transport = "http" against target_port = 8443 sends cleartext to the TLS listener (ports.http is 8080, ports.https is 8443 in the generated socket.yml).

The failure mode is the bad part: the health probes use transport = "HTTPS" on 8443 and keep passing, so the revision reports healthy while serving nothing. Now external_enabled = true (VNet-scoped) and target_port = 8080.

2. A route name that is not an ecosystem name silently passed traffic unscanned.

locals.routes set registry = name straight from the registries map key. The firewall's path-routing builder dispatches on that value and falls through to a plain streaming proxy for anything it does not recognize:

else:
    # Fallback to streaming proxy for unrecognized registries
    return streaming.build_location(path, upstream_host, upstream_scheme, needs_rewrite)

No package inspection on that route. Installs succeed, the deployment looks healthy, and the only signal is the absence of SOCKET_DECISION lines in the logs.

The template's own Artifactory example ("repository/npm-remote") produced exactly this shape, so anyone following the README got an unscanned route. Adds a registry_overrides map, uses it in the Artifactory example, and documents the failure mode on the variable.

3. Image defaulted to :latest.

Container Apps resolves a floating tag once at revision creation and does not re-resolve it, so :latest leaves the running version unknowable and lets a later apply roll a different build. Pinned to 2.4.1, which also carries the NuGet search fix from 2.3.0.

Also

  • environment_name derives a Key Vault name (kv-<environment_name>), which is globally unique across Azure and held through the soft-delete window. Called out in the example, since it is a likely first-apply failure on the default value.
  • Noted that Azure Cache auth must be an access key on the default user; Redis 6 ACL username/password pairs are not supported by the client.

Verification

  • terraform init -backend=false + terraform validateSuccess! The configuration is valid.
  • terraform fmt -check -recursive → clean
  • Both defects confirmed against the firewall config generator, not inferred from the template alone.

Not applied here: the transparent-routing work (host-based routing, customer certs, setup.sh) still sits on a branch in the demo repo. It is a larger change and worth its own review, which is a reason to archive that repo rather than delete it.

Follow-up

Once this merges, socketdev-demo/socket-firewall-azure-container-apps should be archived and its README pointed here. Holding that until merge, since the old URL is already in customers' hands.


Note

Medium Risk
Changes affect customer-facing firewall reachability and whether traffic is actually scanned; misconfiguration can still fail open with socket_fail_open = true, and secrets may remain in Terraform state unless Key Vault references are used.

Overview
Adds terraform/azure-container-apps/ as the new home for the Socket Registry Firewall on Azure Container Apps (migrated from the demo repo), and documents it in the root README.

The template provisions the usual stack (resource group, internal Container Apps Environment, Key Vault + managed identity, auto-generated socket.yml, optional Redis, self-signed cert/custom domains). This PR also fixes three quiet failure modes:

Ingress — Sets external_enabled = true so internal environments remain reachable from the VNet (runners, Front Door, jump boxes), and routes ingress to target_port = 8080 with transport = "http" instead of cleartext to the TLS listener on 8443 (which could look healthy via HTTPS probes but serve no traffic).

Package scanning — Introduces registry_overrides so non-ecosystem path names (e.g. Artifactory repository/npm-remote) map to npm/pypi in generated routes; without it those paths fall through to an unscanned streaming proxy. README and tfvars examples call out the missing SOCKET_DECISION signal.

Image — Defaults firewall_image to socketdev/socket-registry-firewall:2.4.1 instead of a floating tag.

Also includes Terraform moved blocks for Key Vault secrets after count, optional out-of-band Key Vault secret IDs, and expanded ops docs (Front Door SNI, tarball rewriting, Redis access-key note).

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

…ents

Replace manual socket_yml_content input with auto-generation from a
registries variable and domain hostname. The firewall's path_routing
domain must match the Host header clients send, so this is now a
required variable.

Changes:
- Generate socket.yml from registries map and domain variable using
  yamlencode, removing need for manual base64-encoded config
- Remove SOCKET_FAIL_OPEN env var (now set in generated socket.yml)
- Add .gitignore for tfstate, tfvars, .terraform/
- Add Registries, Verify the deployment, and Troubleshooting to README
- Update terraform.tfvars.example with new variables
Azure Container Apps limits failure_count_threshold to 1-10.
The previous value of 30 caused terraform plan to fail with a
validation error.
… config

- Remove api_url from socket.yml (defaults to api.socket.dev)
- Add commented Artifactory/upstream route examples to main.tf and tfvars
- Add debug_logging_enabled and debug_user_agent_filter variables
- Add recently_published_enabled_ecosystems variable
- Update README with new variables and upstream mode documentation
…outputs

- Add generate_self_signed_cert option that creates a proper server cert
  with SANs derived from the domain variable (covers Front Door private
  link setups where cert subject name validation is required)
- Add SOCKET_FAIL_OPEN, SOCKET_LOG_LEVEL, SOCKET_DEBUG_LOGGING_ENABLED,
  and SOCKET_DEBUG_USER_AGENT_FILTER env vars (the firewall reads these
  from env vars, not socket.yml)
- Add log_level variable (set to "debug" for TLS handshake details)
- Add ssl_cert_sans output to verify cert SANs after deploy
- Add troubleshooting output with deployment-specific debug commands
- Document Front Door SSLMismatchedSNI, tarball URL rewriting, and
  secret propagation issues in README
…ader (#1)

The Container Apps ingress rejects requests with Host headers that don't
match the default FQDN. When Front Door sends Host: <custom-domain>, the
ingress returns 404 before nginx sees the request. This adds:

- tls_pkcs12_archive to convert the self-signed cert to PFX format
- azurerm_container_app_environment_certificate to register the cert
- azurerm_container_app_custom_domain for each hostname in the domain var

All three layers of hostname matching are now handled by the template:
Container Apps ingress, Front Door cert validation, and nginx server_name.
Adds Redis auth and TLS for Azure Cache for Redis: redis_password (Key Vault-backed REDIS_PASSWORD secret) and redis_ssl (REDIS_SSL, default true). No-ops while redis_enabled = false.

Adds *_key_vault_secret_id variants for socket_api_token, ssl_cert/ssl_key, and redis_password so secrets can be created out-of-band and referenced by ID, keeping values out of Terraform state. Preconditions enforce token presence and block SSL references with self-signed generation. moved blocks keep existing deployments from recreating their Key Vault secrets.

Also fixes terraform validate on fresh clones: tls_pkcs12_archive does not exist in hashicorp/tls; replaced with chilicat/pkcs12 pkcs12_from_pem.
…32117ffbf3dc387b7d344a'

git-subtree-dir: terraform/azure-container-apps
git-subtree-mainline: 61b4d54
git-subtree-split: 91ddad4
external_enabled = false on an internal environment scopes ingress to the
Container Apps environment rather than the VNet, so nothing outside the
environment (developer machines, CI runners, Front Door) can reach the
firewall.

target_port 8443 with transport "http" also sent cleartext to the TLS
listener. The HTTPS health probe on 8443 still succeeds, so the revision
reports healthy while serving no traffic, which is a slow failure to
diagnose.

Points ingress at the plaintext listener (8080) and scopes it to the VNet.
…ames stay inspected

routes derived its ecosystem from the registries map key. A key that is not
a valid ecosystem name (an Artifactory path such as repository/npm-remote,
or a route named nuget-v2) is served by the firewall's fallback streaming
proxy, which forwards traffic with no package inspection. The deployment
looks healthy and packages install normally; the only tell is that no
SOCKET_DECISION lines appear in the logs.

The README's own Artifactory example produced exactly this shape.

registry_overrides maps such route names onto a real ecosystem.
…example

Default the image to 2.4.1 rather than :latest. Container Apps resolves a
floating tag once at revision creation, so :latest leaves the running
version unknowable and lets a later apply roll a different build.

2.4.1 also carries the NuGet search fix from 2.3.0, which affects anyone
fronting api.nuget.org.

The Artifactory example now sets registry_overrides. As written it produced
route names the firewall does not treat as ecosystems, which meant the
example itself demonstrated an unscanned passthrough.
…and Key Vault naming

Adds the registry_overrides example alongside the Artifactory registries
block, since that is where the omission causes an unscanned route.

Notes that environment_name derives a globally unique Key Vault name, which
is a common first-apply failure, and that Azure Cache auth must be an access
key on the default user rather than a Redis 6 ACL user.

Also records terraform/ in the repository layout.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 5 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

  • ✅ Fixed: Ingress targets HTTP redirect port
    • Disabled the container HTTPS listener and pointed health probes at HTTP 8080 so ACA TLS termination no longer hits nginx's HTTP-to-HTTPS redirect.
  • ✅ Fixed: Custom domains include platform FQDN
    • Custom-domain bindings now omit localhost and *.azurecontainerapps.io while those names remain available as certificate SANs.
  • ✅ Fixed: BYO cert skips domain bindings
    • Environment certificate and hostname bindings are created whenever a PEM is available to Terraform, including bring-your-own ssl_cert/ssl_key.

Create PR

Or push these changes by commenting:

@cursor push 93cecd1610
Preview (93cecd1610)
diff --git a/terraform/azure-container-apps/README.md b/terraform/azure-container-apps/README.md
--- a/terraform/azure-container-apps/README.md
+++ b/terraform/azure-container-apps/README.md
@@ -142,6 +142,7 @@
 ```hcl
 domain = "registry.company.com ca-socket-fw.xxxxx.eastus.azurecontainerapps.io"

+The Container App FQDN (*.azurecontainerapps.io) is valid as a SAN so origin host-header checks succeed. Do not expect it to be bound as a custom domain — Azure already owns that name, and the template skips it when creating hostname bindings.

Tarball URLs point to the Container App FQDN instead of the customer-facing domain
The firewall rewrites tarball URLs using the Host header it receives. If Front Door's origin host header is set to the Container App FQDN, tarball URLs will use that FQDN, and npm clients will try to download tarballs directly (bypassing Front Door), which fails with ECONNRESET.
@@ -162,7 +163,7 @@

Notes

-Custom domain binding: When generate_self_signed_cert = true, the template automatically registers each hostname in the domain variable as a custom domain on the Container App. This is required so the Container Apps ingress accepts requests with those Host headers. Without it, requests from Front Door (or any client using a custom hostname) get a 404 from the ingress layer before reaching nginx.
+Custom domain binding: When a certificate PEM is available to Terraform (generate_self_signed_cert = true, or ssl_cert / ssl_key with generation disabled), the template registers each customer hostname in the domain variable as a custom domain on the Container App. Platform FQDNs (*.azurecontainerapps.io) and localhost are omitted from those bindings. This is required so the Container Apps ingress accepts requests with those Host headers. Without it, requests from Front Door (or any client using a custom hostname) get a 404 from the ingress layer before reaching nginx. Key Vault secret IDs alone are not enough to upload the ingress certificate; pass ssl_cert and ssl_key as well if you need custom-hostname bindings.

Azure Container Apps mounts secrets as files in a shared volume at /mnt/config/. The template sets the CONFIG_FILE env var so the firewall reads socket.yml from the correct path. SSL certificate paths in the generated socket.yml reference /mnt/config/ssl-cert and /mnt/config/ssl-key.

diff --git a/terraform/azure-container-apps/main.tf b/terraform/azure-container-apps/main.tf
--- a/terraform/azure-container-apps/main.tf
+++ b/terraform/azure-container-apps/main.tf
@@ -50,6 +50,10 @@
ports = {
http = 8080
https = 8443

  •    # Container Apps HTTP ingress terminates TLS and forwards cleartext.
    
  •    # The image's HTTP listener redirects to HTTPS when that listener is
    
  •    # enabled, so disable it and serve on 8080.
    
  •    disable_https = true
     }
     socket = {
       fail_open = var.socket_fail_open
    

@@ -215,9 +219,9 @@
}

resource "pkcs12_from_pem" "server" {

  • count = var.generate_self_signed_cert ? 1 : 0
  • cert_pem = tls_self_signed_cert.server[0].cert_pem
  • private_key_pem = tls_private_key.server[0].private_key_pem
  • count = local.has_ingress_certificate ? 1 : 0
  • cert_pem = local.ssl_cert_pem
  • private_key_pem = local.ssl_key_pem
    password = ""
    }

@@ -225,8 +229,16 @@
ssl_cert_pem = var.generate_self_signed_cert ? tls_self_signed_cert.server[0].cert_pem : var.ssl_cert
ssl_key_pem = var.generate_self_signed_cert ? tls_private_key.server[0].private_key_pem : var.ssl_key

  • Custom domains: all hostnames from the domain variable except "localhost"

  • custom_domains = [for d in split(" ", var.domain) : d if d != "localhost"]
  • Cert SANs use every hostname in var.domain (plus localhost). Ingress

  • bindings cannot include localhost or the platform FQDN Azure already owns.

  • custom_domains = [
  • for d in split(" ", var.domain) : d
  • if d != "" && d != "localhost" && !endswith(d, ".azurecontainerapps.io") && d != "azurecontainerapps.io"
  • ]
  • Environment certificate + bindings need a PEM in Terraform. Self-signed

  • generation or ssl_cert/ssl_key both qualify; Key Vault-only IDs do not.

  • has_ingress_certificate = var.generate_self_signed_cert || (var.ssl_cert != "" && var.ssl_key != "")
    }

resource "azurerm_key_vault_secret" "ssl_cert" {
@@ -288,7 +300,7 @@

host header (which it must, so tarball URLs are rewritten correctly).

resource "azurerm_container_app_environment_certificate" "server" {

  • count = var.generate_self_signed_cert ? 1 : 0
  • count = local.has_ingress_certificate ? 1 : 0
    name = "cert-${local.env_name}"
    container_app_environment_id = azurerm_container_app_environment.this.id
    certificate_blob_base64 = pkcs12_from_pem.server[0].result
    @@ -364,10 +376,10 @@

    developer machines, CI runners, jumpboxes, or Front Door.

    external_enabled = true
  • 8080 is the plaintext listener (see ports.http in socket_yml above).

  • TLS is terminated by Container Apps ingress, so target_port must match

  • transport. Pointing "http" transport at 8443 sends cleartext to the

  • TLS listener: the revision passes its health probes and serves nothing.

  • HTTP ingress terminates TLS and forwards cleartext. 8443 is TLS-only, so

  • targeting it with transport "http" sends cleartext to nginx's HTTPS

  • listener. 8080 redirects to HTTPS unless that listener is disabled

  • (see disable_https in socket_yml). Serve the app on 8080 instead.

    target_port = 8080
    transport = "http"

@@ -471,8 +483,8 @@
# ── Liveness probe ────────────────────────────────────────────────

   liveness_probe {
  •    transport               = "HTTPS"
    
  •    port                    = 8443
    
  •    transport               = "HTTP"
    
  •    port                    = 8080
       path                    = "/health"
       initial_delay           = 15
       interval_seconds        = 30
    

@@ -483,8 +495,8 @@
# ── Readiness probe ───────────────────────────────────────────────

   readiness_probe {
  •    transport               = "HTTPS"
    
  •    port                    = 8443
    
  •    transport               = "HTTP"
    
  •    port                    = 8080
       path                    = "/health"
       interval_seconds        = 10
       timeout                 = 3
    

@@ -495,8 +507,8 @@
# ── Startup probe ─────────────────────────────────────────────────

   startup_probe {
  •    transport               = "HTTPS"
    
  •    port                    = 8443
    
  •    transport               = "HTTP"
    
  •    port                    = 8080
       path                    = "/health"
       interval_seconds        = 5
       timeout                 = 3
    

@@ -523,7 +535,7 @@

Host: to the Container App.

resource "azurerm_container_app_custom_domain" "domains" {

  • for_each = var.generate_self_signed_cert ? toset(local.custom_domains) : toset([])
  • for_each = local.has_ingress_certificate ? toset(local.custom_domains) : toset([])

    name = each.value
    container_app_id = azurerm_container_app.firewall.id

diff --git a/terraform/azure-container-apps/outputs.tf b/terraform/azure-container-apps/outputs.tf
--- a/terraform/azure-container-apps/outputs.tf
+++ b/terraform/azure-container-apps/outputs.tf
@@ -57,15 +57,15 @@

 # ── Test health endpoint from inside container ───────────────────
 # (run from console)
  • curl -s http://localhost:8080/health

    ── Test npm route from inside container ─────────────────────────

    (run from console)

  • curl -s http://localhost:8080/npm/lodash | head -c 200

    ── Check tarball URL rewriting ──────────────────────────────────

    (run from console) Tarball URLs should use your domain, not the Container App FQDN

EOT
}

diff --git a/terraform/azure-container-apps/terraform.tfvars.example b/terraform/azure-container-apps/terraform.tfvars.example
--- a/terraform/azure-container-apps/terraform.tfvars.example
+++ b/terraform/azure-container-apps/terraform.tfvars.example
@@ -20,7 +20,8 @@

IMPORTANT: If you use Azure Front Door with private link and certificate subject

name validation, include ALL hostnames that need to match the cert's SANs.

-# Separate multiple hostnames with spaces:
+# Separate multiple hostnames with spaces. The Container App FQDN is a valid SAN
+# but is not bound as a custom domain (Azure already owns *.azurecontainerapps.io):

domain = "registry.company.com ca-socket-fw.happystone-xxxx.eastus.azurecontainerapps.io"

domain = "registry.company.com"


</details>


<sub>You can send follow-ups to the cloud agent <a href="https://cursor.com/agents/bc-59ac80c8-c872-4717-a9ec-618e5b5d75e7">here</a>.</sub>
<!-- BUGBOT_AUTOFIX_REVIEW_FOOTNOTE_END -->

_Comment `@cursor review` or `bugbot run` to trigger another review on this PR_


<sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 97a2bef07fbc9b42c9996449d91193c352aa5c5f. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>

# transport. Pointing "http" transport at 8443 sends cleartext to the
# TLS listener: the revision passes its health probes and serves nothing.
target_port = 8080
transport = "http"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ingress targets HTTP redirect port

High Severity

Container Apps ingress terminates TLS and forwards cleartext to target_port 8080, while the firewall's HTTP listener redirects that traffic to HTTPS. Health probes still call 8443 directly and stay green, so the revision looks healthy while client requests redirect-loop or fail.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

container_app_id = azurerm_container_app.firewall.id
container_app_environment_certificate_id = azurerm_container_app_environment_certificate.server[0].id
certificate_binding_type = "SniEnabled"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Custom domains include platform FQDN

High Severity

domain is reused for certificate SANs and azurerm_container_app_custom_domain. The Front Door guidance adds the Container App FQDN to domain, so Terraform tries to bind *.azurecontainerapps.io as a custom hostname. Azure already owns that name, and apply fails.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

container_app_id = azurerm_container_app.firewall.id
container_app_environment_certificate_id = azurerm_container_app_environment_certificate.server[0].id
certificate_binding_type = "SniEnabled"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BYO cert skips domain bindings

High Severity

Environment certificate and custom-domain resources are created only when generate_self_signed_cert is true. Bring-your-own certificates therefore get no ingress hostname bindings, and Container Apps returns 404 on custom Host headers before nginx sees the request.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

} : {},
length(var.recently_published_enabled_ecosystems) > 0 ? {
recently_published_enabled_ecosystems = var.recently_published_enabled_ecosystems
} : {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recently published key nested wrong

Medium Severity

recently_published_enabled_ecosystems is merged at the top level of the generated socket.yml. The firewall and the Helm chart read this key under socket, so the Terraform variable is ignored and ecosystem scoping never takes effect.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

variable "vnet_id" {
description = "Resource ID of the VNet for internal networking"
type = string
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required vnet_id is unused

Low Severity

vnet_id is required and documented as network configuration, but no resource reads it. Callers must supply a value that does not change the plan.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 97a2bef. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants