Repository navigation
Add Deploy-to-Azure Bicep template - #1477
Josh Duffney (duffney) wants to merge 25 commits into
Conversation
Provisions AKS (OIDC issuer, workload identity, native Key Vault Secrets Provider add-on, Cilium network policy), a VNet with private endpoints for Key Vault/Cosmos DB/Redis/Storage, Cosmos DB (MongoDB API only), Azure Cache for Redis (Entra ID auth preferred), a blob+queue storage account, and an ACR with AcrPull granted to the AKS kubelet identity. Explicitly excludes azd, FluxCD/GitOps, ASO, ESO, KEDA-via-Flux, CorpNet IP allowlisting, GitHub App/Entra bootstrap scripts, and Windows node pools. Does not touch the existing infra/ (azd dev CosmosDB) or root azure.yaml. Includes: - main.bicep orchestrator + 7 modules (networking, aks, keyvault, cosmosdb, redis, storage, acr), using Azure Verified Modules where available - main.parameters.json with sensible defaults - Compiled azuredeploy.json for the Deploy-to-Azure button, plus a CI workflow (.github/workflows/deploy-azure-drift.yml) that fails if it drifts from main.bicep - createUiDefinition.json with a Basics/Advanced portal UI flow - metadata.json for the template gallery - deploy/azure/README.md documenting prerequisites, the CLI deploy path, and what this does NOT automate (the Helm app install, BYO secrets) - Deploy-to-Azure button + link added to the root README All Bicep files compile cleanly via 'az bicep build' with zero errors and zero warnings. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Upgrade avm/res/container-service/managed-cluster from 0.9.0 to 0.14.0. The pinned 0.9.0/0.13.0 module versions generated a managedClusters resource at API version 2024-09-02-preview, which Azure has since retired (not present in the supported api-versions list), causing deployments to fail with: NoRegisteredProviderFound: No registered resource provider found for location 'westus3' and API version '2024-09-02-preview' for type 'managedClusters'. 0.14.0 generates the resource at 2025-10-01 (current, non-preview). Updated param usage to match this version's schema: enablePrivateCluster moved under apiServerAccessProfile, and enableWorkloadIdentity moved under securityProfile.workloadIdentity.enabled. All module outputs (resourceId, name, oidcIssuerUrl, kubeletIdentityObjectId, systemAssignedMIPrincipalId) are unchanged between versions. Verified: deploy/azure/modules/aks.bicep and deploy/azure/main.bicep both compile with zero errors/warnings via 'az bicep build'; confirmed 2024-09-02-preview no longer appears anywhere in the compiled azuredeploy.json. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…t kind The AVM database-account module (br/public:avm/res/document-db/database-account:0.10.0) derives the account's 'kind' property from whether mongodbDatabases is non-empty, independent of capabilitiesToAdd. Passing an empty mongodbDatabases array left the account at kind=GlobalDocumentDB (SQL API) even with EnableMongo set, which then rejected the MongoDB-groupId private endpoint with: BadRequest: Call to Microsoft.DocumentDB/databaseAccounts failed. Error message: GroupId MongoDB is not supported Fixed by provisioning a default database (name configurable via the new 'databaseName' param, default 'scope') so the module correctly resolves kind=MongoDB. The child mongodb-database module already skips setting throughput when EnableServerless is present, so this has no cost impact under the default serverless configuration. Verified: deploy/azure/modules/cosmosdb.bicep and the full deploy/azure/main.bicep both compile with zero errors/warnings via 'az bicep build'; confirmed the compiled azuredeploy.json's kind expression now resolves to 'MongoDB' when mongodbDatabases is populated. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Wassim Chegham (manekinekko)
left a comment
There was a problem hiding this comment.
Thanks for the PR, Josh! The module separation, private-endpoint wiring, and secure connection-string outputs are a good foundation. I've left three initial comments on resource naming, the portal wizard, and the post-deployment instructions.
|
|
||
| var vnetName = 'vnet-${environmentName}-${resourceToken}' | ||
| var aksName = 'aks-${environmentName}-${resourceToken}' | ||
| var keyVaultName = take('kv-${environmentName}-${resourceToken}', 24) |
There was a problem hiding this comment.
[P2] Preserve a valid name and the uniqueness suffix when truncating.
A 20-character environmentName is accepted by the template, but this expression produces a Key Vault name ending in a hyphen. For example, abcdefghijklmnopqrst becomes kv-abcdefghijklmnopqrst-, which Azure rejects. With a 19-character name, only one character of the uniqueness suffix survives.
Could we truncate the environment portion before appending the suffix, rather than truncating the completed resource name? The Storage account naming expression has the same suffix-truncation problem and should preserve its uniqueness suffix too.
| } | ||
| ] | ||
| }, | ||
| "outputs": { |
There was a problem hiding this comment.
[P2] Put the output mappings inside parameters and connect this wizard to the buttons.
outputs is currently at the document root, but the CreateUiDefinition schema requires parameters.outputs to bind the controls to the ARM template parameters.
Also, both Deploy to Azure buttons reference only azuredeploy.json; neither references this UI definition, so users won't see the custom wizard or its validation. Could we move these mappings into parameters and wire the UI definition into both deployment links?
|
|
||
| - **Installing Scope itself.** This template provisions infrastructure only. The | ||
| Scope application (API, workers, Judge, Portal, Token Manager) is installed | ||
| separately via Helm charts against the AKS cluster this template creates. See the |
There was a problem hiding this comment.
[P2] Point to an available application-install procedure.
This directs users to install Scope via Helm charts and sends them to the documentation site for instructions, but I couldn't find a Helm chart or Helm installation instructions. Is this part of a separate PR?
…rite to race cache creation The primaryKey output called listKeys() against a resourceId() built purely from the redisName parameter, with no symbolic reference to the redis AVM module. Bicep only infers an implicit dependsOn from symbolic-name references, so this output had no dependency edge on the cache's actual provisioning - ARM could evaluate listKeys() before the long-running Redis create finished, which explains reports of the redis-primary-key secret being missing from Key Vault after otherwise-successful deployments (the cache itself provisions fine; only the secret write silently failed). Fixed by declaring an existing resource reference for the cache (a compile-time-known ID, valid as a listKeys() target) with an explicit dependsOn: [redis] to force correct ordering. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…e connectivity The private DNS zone for the Cosmos DB MongoDB API private endpoint was named 'privatelink.mongo.cosmos.azure.net' - wrong TLD, likely a copy/paste mix-up with the Key Vault zone (privatelink.vaultcore.azure.net). The canonical zone name, per the public DNS CNAME chain (*.mongo.cosmos.azure.com -> *.privatelink.mongo.cosmos.azure.com), is 'privatelink.mongo.cosmos.azure.com' (.com). With the wrong zone name, pods resolved the Cosmos Mongo hostname to its public IP (no matching records in the misnamed zone), and Cosmos's firewall rejected the connection since publicNetworkAccess is Disabled - MongoServerSelectionError connecting to the public IP on port 10255. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…ad identity The shared workload identity only had Key Vault Secrets User (read-only), which is correct for the Helm chart's Secrets Store CSI mounts but blocks the Token Manager service (apps/token-manager), which writes BYO credentials (GitHub/Anthropic API keys, etc.) into the vault at runtime via SecretClient.setSecret() using this same shared identity. Read-only access 403s that write path, leaving the portal's built-in credential registration UI (POST/DELETE /api/v1/keys) non-functional out of the box on a freshly provisioned environment. Key Vault Secrets Officer is a superset of Secrets User (adds set/delete alongside get/list on secrets, still scoped to secrets only - no key/cert or permission-management access), so this replaces rather than stacks alongside the previous role assignment. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
… Data Owner Storage Blob Data Contributor's data actions do not include Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags/write, so the app's practice of setting Blob Index Tags (requestId/runId/iteration) via the x-ms-tags header on per-iteration workspace snapshot uploads 403s with AuthorizationPermissionMismatch under Contributor alone. 100% reproducible, confirmed via raw PUT with/without x-ms-tags using a storage-scoped AAD token exchanged from the workload identity's federated token. Storage Blob Data Owner includes the tags/write data action alongside everything Contributor already grants, so this keeps tagged snapshot uploads working without the app needing to drop tags. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…ed Redis
Classic Azure Cache for Redis (Microsoft.Cache/redis) is being retired for
new deployments, causing DeploymentFailed errors on the redis sub-deployment
("Azure Cache for Redis is retiring, create Azure Managed Redis instance
instead").
Rewrote redis.bicep to use the AVM module avm/res/cache/redis-enterprise:0.5.0
(Microsoft.Cache/redisEnterprise), targeting the "Balanced" Azure Managed
Redis SKU tier:
- Replaced the Basic/Standard/Premium skuName + C/P skuFamily + integer
skuCapacity parameter trio with a single skuName enum of AMR SKU strings
(Balanced_B0/B1/B3/B5/B10; capacity is now encoded in the SKU name itself).
Default is Balanced_B1.
- Private endpoint service/groupId changed from redisCache to redisEnterprise;
private DNS zone changed from privatelink.redis.cache.windows.net to
privatelink.redisenterprise.cache.azure.net.
- Entra ID (Azure AD) data-plane auth is now granted via an access policy
assignment on the database submodule (accessPolicyAssignments), replacing
the classic module's redisConfiguration 'aad-enabled' flag. The shared
workload identity's principal ID is threaded through from main.bicep (same
pattern already used for storage.bicep).
- Access-key auth (accessKeysAuthentication: 'Enabled') is kept as a
compatibility fallback; the primary access key is still surfaced as a
@secure() output and written to the redis-primary-key Key Vault secret, now
sourced directly from the AVM module's own output instead of a manual
listKeys() call — the AMR database submodule's output is correctly ordered,
so the existing-resource + explicit dependsOn workaround from the prior
dependsOn fix is no longer needed.
- Updated main.bicep (params, redisName max length 63->60 per
Microsoft.Cache/redisEnterprise naming rules, module wiring, outputs),
main.parameters.json, createUiDefinition.json (single SKU dropdown, capacity
slider removed), metadata.json, and README.md to match.
Verified with `az bicep build` on the module and full orchestrator (clean,
zero errors/warnings), `az bicep lint`, and a rebuild-and-diff drift check
against the committed azuredeploy.json (no drift).
Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Live deployment hit:
Unable to process template language expressions for resource
'.../redis-apa-0'... 'The language expression property name doesnt
exist, available properties are userObjectId.' (Code: InvalidTemplate)
Root cause: the AVM redis-enterprise module's nested access-policy-assignment
deployment reads parameters('accessPolicyAssignments')[i].name directly
(not via tryGet), even though name is documented as optional/nullable on
accessPolicyAssignmentType. ARM strips omitted optional properties from the
JSON object entirely (there is no null placeholder), so direct property
access on a missing key throws InvalidTemplate. Our accessPolicyAssignments
array item only set userObjectId, omitting name as the (incorrect) docs
imply is fine.
Fix: explicitly set name: 'workload-identity-access' on the access policy
assignment entry in redis.bicep's database block, working around the
module's property-access bug.
Verified with az bicep build (module + full orchestrator, clean), az bicep
lint (clean), and a rebuild-and-diff drift check against the committed
azuredeploy.json (no drift) - confirmed the compiled ARM template now
includes the explicit name property on the access policy assignment.
Co-authored-by: Copilot App <[email protected]>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Live Helm deploy testing (#1481) found that every app service (api, report-generator, both ACP workers) failed to connect to Redis with connect ETIMEDOUT, even though the cache resource itself was healthy. Root cause: redis.bicep linked the private endpoint's DNS zone group to privatelink.redisenterprise.cache.azure.net, which has zero matching record sets for this resource. Azure Managed Redis (the Balanced_* SKU tier used here) and the older "Redis Enterprise" product branding share the same underlying resource type (Microsoft.Cache/RedisEnterprise) and private endpoint subresource (redisEnterprise), but publish their hostnames under different private DNS zones: - Redis Enterprise (Enterprise/EnterpriseFlash SKUs): privatelink.redisenterprise.cache.azure.net - Azure Managed Redis (Balanced/ComputeOptimized/FlashOptimized/ MemoryOptimized SKUs, i.e. what this template deploys): privatelink.redis.azure.net With the wrong zone attached, in-cluster DNS fell through to the public record, which resolves to a public IP - unreachable since publicNetworkAccess is Disabled on the cache. Confirmed authoritative zone name against the Microsoft Learn private endpoint DNS reference table (https://learn.microsoft.com/azure/private-link/private-endpoint-dns), which lists both product brandings separately. Also validated live on a test environment per the linked issue - after relinking the private endpoint's DNS zone group to privatelink.redis.azure.net, Redis connectivity came up cleanly with no other changes. Fix: change redis.bicep's privateDnsZoneName var from privatelink.redisenterprise.cache.azure.net to privatelink.redis.azure.net. The private endpoint's service/groupId (redisEnterprise) is unchanged - only the DNS zone name was wrong. Verified with az bicep build (module + full orchestrator, clean), az bicep lint (clean), and a rebuild-and-diff drift check against the committed azuredeploy.json (no drift) - confirmed the compiled ARM template now references privatelink.redis.azure.net. Fixes #1483. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…t isolation ACP worker pods execute benchmark-controlled/untrusted agent code in the same container as the workload identity used for Azure resource access. The previously single shared identity had Key Vault Secrets Officer (write access), meaning a compromised worker process could read or tamper with Key Vault secrets it has no legitimate need to touch. Fix: split into two user-assigned managed identities, each with its own federated credential and Kubernetes service account subject: - Backend identity (scope-workload-identity, unchanged name/behavior): Key Vault Secrets Officer, Redis Entra ID access, Storage Blob/Queue data-plane access. Used by trusted components (api, Token Manager, scheduler, etc.). - Worker identity (scope-worker-workload-identity, new): Storage Blob/Queue data-plane access ONLY - no Key Vault role, never wired into keyvault.bicep. storage.bicep now accepts a second optional principal ID (workerWorkloadIdentityPrincipalId) and grants the same Storage Blob Data Owner + Storage Queue Data Contributor roles to both identities that are non-empty. New outputs: workerWorkloadIdentityClientId, workerWorkloadIdentityServiceAccountNamespace, workerWorkloadIdentityServiceAccountName - for the Helm chart to annotate the worker pods Kubernetes service account. New params (with defaults, exposed in createUiDefinition.json and main.parameters.json): workerWorkloadIdentityServiceAccountNamespace, workerWorkloadIdentityServiceAccountName. Verified: az bicep build (clean), az bicep lint (clean), rebuild-and-diff against committed azuredeploy.json (no drift). Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…Key Vault isolation" This reverts commit a28ed98.
Both ACP workers run benchmark-controlled agent code and should not share the backend workload identity (Key Vault Secrets Officer) or its mounted secrets file tree with untrusted execution. - Drop the azure.workload.identity/use label and kv-secrets CSI volume/mount from both worker Deployments, so no federated Azure AD token or Key Vault file tree reaches the untrusted agent subprocess. - Keep the common-secret envFrom (Mongo/Redis creds), since the worker still needs those to claim/lock Storage Queue work — the queue-processor and agent executor are one process today, so a chart change alone can't separate them further. - Document the residual limitation and a worker-scoped managed identity (Storage Blob/Queue only, planned on the Bicep side) in WORKERS.md. Addresses part of the review finding in #1481 (comment). Full trusted/untrusted separation needs worker code changes and a dedicated worker identity from deploy/azure/, which is still being worked out on the Bicep PR (#1477) after an earlier attempt there was reverted. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 094bb93a-ec65-4f80-8896-0dd041aedeff
…app breakage) The previous split (a28ed98, reverted in 4568107) gave the worker identity NO Key Vault role at all. That broke the app: worker pods rely on the shared Secrets Store CSI SecretProviderClass to mount their own BYO credentials (GitHub/Anthropic API keys) from Key Vault, which requires at least read access - removing Key Vault access entirely from the worker identity broke that CSI mount for every worker pod. Corrected approach: keyvault.bicep now grants two tiers of access: - Backend identity (unchanged): Key Vault Secrets Officer (read/write/delete), for Token Manager's runtime credential-registration writes. - Worker identity (new param `readOnlyPrincipalIds`): Key Vault Secrets User (read-only) - enough to mount its own CSI-synced secrets, but it can never write, delete, or tamper with vault contents. This still satisfies the original security review's intent (workers must not have write access to Key Vault) while keeping the CSI secret mount functional. Everything else from the original split is unchanged: a separate workerWorkloadIdentity module/federated credential, worker Storage Blob/Queue access via storage.bicep's generalized role-assignment logic, and the same output names (workerWorkloadIdentityClientId, workerWorkloadIdentityServiceAccountNamespace, workerWorkloadIdentityServiceAccountName). Verified: az bicep build (clean), az bicep lint (clean), rebuild-and-diff against committed azuredeploy.json (no drift). Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Bicep PR #1477 landed a worker-specific managed identity (Storage Blob Data Owner + Queue Data Contributor only — no Key Vault access), exposed as workerWorkloadIdentityClientId/ workerWorkloadIdentityServiceAccountName. - values.yaml: add azure.workerWorkloadIdentityClientId and azure.workerServiceAccountName (default scope-worker-workload-identity). - templates/serviceaccount.yaml: add a second ServiceAccount for the worker identity, rendered only when a worker is enabled. - _helpers.tpl: add scope.workerServiceAccountName helper. - Both ACP worker Deployments now use the worker ServiceAccount (with azure.workload.identity/use re-added, needed for their direct Storage Blob/Queue SDK access via DefaultAzureCredential) instead of the shared backend ServiceAccount. The kv-secrets CSI mount stays removed from their own pods — the worker's GITHUB_TOKEN/ ANTHROPIC_API_KEY fallback values already reach them via envFrom.secretRef on the common K8s Secret, which the api pod's own CSI mount keeps synced; the workers don't need a mount of their own for that. - scripts/generate-values-from-bicep.sh: read the two new outputs, tolerating deployments that don't have them (older Bicep deployments, or ones that never provisioned the worker identity). - WORKERS.md: document the real worker-identity wiring and keep the known-limitation note (full trusted/untrusted process separation needs worker code changes). Continues the fix for #1481 (comment). Co-authored-by: Copilot App <[email protected]> Copilot-Session: 094bb93a-ec65-4f80-8896-0dd041aedeff
Per the Helm chart (PR #1481, commit 6b4c847), ACP worker pods no longer mount the Secrets Store CSI SecretProviderClass directly - they get GITHUB_TOKEN/ANTHROPIC_API_KEY via envFrom/secretRef against a common Kubernetes Secret that the api pod's CSI mount populates. Worker pods never call Key Vault, so the read-only `Key Vault Secrets User` grant added in b897d9a is unnecessary. Reverts keyvault.bicep to a single role assignment (Key Vault Secrets Officer for the backend identity only) and removes the worker identity's `readOnlyPrincipalIds` wiring in main.bicep. The worker identity keeps its Storage Blob Data Owner + Storage Queue Data Contributor roles and output names (workerWorkloadIdentityClientId, etc.) are unchanged. Verified: az bicep build (clean), az bicep lint (clean), rebuild-and-diff against committed azuredeploy.json (no drift). Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Several coworkers hit AKS vCPU-quota errors and Azure Managed Redis InsufficientCapacity failures deploying this template on new/free subscriptions. Rework the defaults to prioritize "it deploys" over performance/HA for this quickstart template, while keeping every value overridable via parameters: - AKS: default VM size for both node pools changed from Standard_D4s_v5 to Standard_B2ms (2 vCPU/8GB burstable), and default node count from 3+2 to 1+1. Cuts the default footprint from 20 vCPUs (often exceeding a subscription's 10-vCPU total regional quota, or its Dsv5-family quota specifically) down to 4 vCPUs on a generally less-contended VM family. - Redis: default SKU changed from Balanced_B1 to Balanced_B0 (smallest available), and added a new redisHighAvailability param defaulting to Disabled - this roughly halves the underlying node allocation Azure Managed Redis needs, reducing the odds of region-level InsufficientCapacity errors (a known constraint on this SKU family, independent of subscription quota). - ACR: default SKU changed from Standard to Basic - sufficient for a single worker image and avoids unnecessary cost (does not affect quota). Updated main.parameters.json (used by the Deploy-to-Azure button) and createUiDefinition.json (portal UI defaults/tooltips/new HA dropdown) to match. Added a new "Sizing and quota" README section explaining the rationale and how to size up for production. Verified: az bicep build (clean), az bicep lint (clean), rebuild-and-diff against azuredeploy.json (no drift), cross-checked main.parameters.json keys against the compiled template parameters (exact match). Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Avoids naming collisions when multiple people deploy the template into the same tenant/subscription concurrently, similar to how the Azure portal suggests a randomized default resource group name. Uses createUiDefinition's client-side guid()/substring() functions; actual resource names still get their own uniqueString()-based suffix in main.bicep regardless of what's typed here. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
The button link only included the template uri (azuredeploy.json), so the portal's Custom Deployment screen used the raw ARM template parameter form instead of the curated createUiDefinition.json UI — explaining why the prepopulated Environment name field (and the Basics/Advanced grouping) wasn't showing up when testing the link. Adds the required /createUIDefinitionUri/<encoded-uri> segment pointing at createUiDefinition.json alongside the existing /uri/<encoded-uri> template segment, in both README.md and deploy/azure/README.md. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
outputs was a top-level sibling of parameters instead of nested inside it (parameters.outputs), which is invalid per the classic CreateUiDefinition (Microsoft.Azure.CreateUIDef / 0.1.2-preview) schema. The Azure portal's Custom deployment page failed to parse it as the classic schema, silently fell back to trying the newer Form view (uiFormDefinition) schema instead, and surfaced that fallback parser's error: 'The provided uiFormDefinition does not have a parameters property under $.view.outputs.' No content changes — basics, steps, and all 19 output mappings are unchanged, just correctly nested. Re-validated as valid JSON and cross-checked output keys against main.bicep's parameter names. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Reverts the behavior from d6a9dc7 (defaultValue back to the static 'scope', tooltip back to the original wording) while keeping the parameters.outputs nesting fix from fe87207, which is unrelated and still needed. This is a manual inverse edit rather than 'git revert' because fe87207 reformatted the whole file afterward and would conflict. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Restores README.md, deploy/azure/README.md, and deploy/azure/createUiDefinition.json to their exact content as of 1cd20c1, undoing d6a9dc7 (env-name random suffix), e8d216b (wiring createUIDefinitionUri into the button link), fe87207 (outputs-nesting fix), and f846714 (partial manual revert). The createUIDefinitionUri change kept surfacing portal errors ('uiFormDefinition does not have a parameters property under $.view.outputs') that didn't clear even after the underlying createUiDefinition.json structure was fixed, likely compounded by raw.githubusercontent.com CDN caching. Reverting to the plain Microsoft.Template/uri/<azuredeploy.json> button link (no custom UI) restores the last confirmed-working deploy experience. The portal falls back to its default ARM-parameter form, which is less polished but functional; revisiting the custom createUiDefinition.json UI is left for a future iteration. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…lity The AVM managed-cluster module defaults every agent pool's availabilityZones to [1,2,3] when not explicitly set. Not all regions support AKS availability zones (e.g. centralus), so deployments there failed preflight validation with: AvailabilityZoneNotSupported: The zone(s) '1' for resource 'system' is not supported. The supported zones for location 'centralus' are ''. Both the system and user node pools now explicitly request an empty availabilityZones array, trading zone-redundant HA (already not guaranteed with the 1-node-per-pool quota-friendly defaults) for working deployments in any region. Set availabilityZones explicitly per node pool for a production environment in a region that supports them. Verified: az bicep build clean, az bicep lint clean, rebuild-and-diff shows no drift in azuredeploy.json, and confirmed both agent pool profiles compile to "availabilityZones": [] in the ARM template. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Standard_B2ms (classic B-series) is not an allowed VM size in every subscription - a live deployment failed preflight validation with: The VM size of Standard_B2ms is not allowed in your subscription in location 'centralus'. Standard_D2s_v5 (general-purpose, non-burstable) has the same 2 vCPU / 8 GiB memory profile as the previous default, is on this subscription's allowed-SKU list, and is a far more broadly available family across subscriptions/regions than burstable B-series sizes (some subscriptions only allow the newer "_v2" B-series, not classic B2ms, so guessing another B-series variant risks the same failure mode elsewhere). Updated: aks.bicep/main.bicep param defaults + comments, main.parameters.json, createUiDefinition.json tooltips/recommendedSizes, and deploy/azure/README.md sizing guidance. Verified: az bicep build + lint clean, zero drift in azuredeploy.json, all JSON files parse, no remaining Standard_B2ms default references. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Live-tested by the Helm-deploy session: with 1 System + 1 User node (both Standard_D2s_v5, 2 vCPU), the core app alone (api/portal/judge/ token-manager/scheduler/report-generator) already consumes ~85% of the User node's allocatable CPU. An optional Phase 3 ACP worker pod (coder-acp-copilot: 500m main + 250m MCP-gateway sidecar = 750m) then fails to schedule with 'Insufficient cpu' on both nodes. Bumping aksUserNodeCount default to 2 (same Standard_D2s_v5 size, no pool recreation needed) resolves it - confirmed live by scaling the user pool to 2 nodes, after which the worker pod scheduled and came up 2/2 Running. Total default footprint is now 3 nodes / 6 vCPUs, still comfortably inside the common 10-vCPU regional quota. Users running only the core app (no ACP workers) can still set aksUserNodeCount=1. Updated: aks.bicep/main.bicep param defaults + comments, main.parameters.json, createUiDefinition.json slider default/tooltip, and deploy/azure/README.md sizing guidance. Verified: az bicep build + lint clean, zero drift in azuredeploy.json (only the intended defaultValue/description diffs + template hashes), all JSON files parse. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
A coworker's live deployment failed with: ServiceUnavailable: Sorry, we are currently experiencing high demand in Central US region for the zonal redundant (Availability Zones) accounts, and cannot fulfill your request at this time. Root cause: cosmosdb.bicep didn't pass a 'locations' param to the AVM document-db/database-account module, so its internal fallback (defaultFailoverLocation) applied - which hardcodes isZoneRedundant: true. This silently requested a zone-redundant Cosmos DB account, which some regions (e.g. centralus) intermittently lack capacity for, independent of subscription quota. Fix: explicitly pass a single-region 'locations' entry with isZoneRedundant: false, opting out of zone redundancy by default - same rationale/pattern as the earlier AKS availabilityZones: [] fix. Also documented in README.md that the companion Redis 'ComputeProvisioningFailed' error reported alongside this is an unrelated, separate transient Azure capacity issue: confirmed via the compiled AVM template that 'zones' only ever applies to Enterprise/EnterpriseFlash Redis SKUs, not the Balanced/AMR tier this template uses, so it is not caused by anything in this template. Verified: az bicep build + lint clean, zero drift in azuredeploy.json beyond the intended locations/isZoneRedundant diff and template hashes, confirmed isZoneRedundant: false compiles into the output. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
…eardown Add a "Good to know before you deploy" section right after the Deploy to Azure button that: - Frames occasional AKS zone / Cosmos DB zonal / Azure Managed Redis capacity errors as expected, transient, and easily resolved by retrying or switching regions (links to the existing detailed Sizing and quota section for specifics). - Encourages deleting the resource group once done exploring, framed as a lean-by-design template that's easy to try and easy to tidy up, to avoid surprise cost from a left-running dev environment. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
Summary
Adds a "Deploy to Azure" portal button that provisions the Azure infrastructure needed to run Scope: AKS, networking, Key Vault, Cosmos DB, Redis, Storage, and a container registry. This gives anyone a one-click way to stand up a ready-to-use environment, as a sanitized, minimal, resource-group-scoped Bicep template under a new
deploy/azure/folder.The existing
infra/folder (azd dev-CosmosDB setup) and rootazure.yamlare untouched; this is a separate, independent deployment path intended for new environments, not local/dev workflows.Approach
deploy/azure/main.biceporchestrates 7 modules underdeploy/azure/modules/(networking,aks,keyvault,cosmosdb,redis,storage,acr), built against a shared parameter/output contract and using Azure Verified Modules (AVM) where available:main.bicepis compiled todeploy/azure/azuredeploy.json(the file the Deploy-to-Azure button URL points at), alongsidemain.parameters.json,createUiDefinition.json(Basics + Advanced portal UI flow), andmetadata.jsonfor the template gallery. A new CI workflow (.github/workflows/deploy-azure-drift.yml) rebuildsmain.bicepand fails ifazuredeploy.jsonhas drifted, plus lints all module files.deploy/azure/README.mddocuments prerequisites, the CLI deploy path, and explicitly what this does not automate: the Helm-based app install itself, and BYO secrets the user must populate in Key Vault before/after deploying. The rootREADME.mdlinks to it with the Deploy-to-Azure button.Notable assumptions (documented in
deploy/azure/README.md)az aks get-credentials+kubectl/helmwork without a jumpbox; only data-plane services (Key Vault, Cosmos, Redis, Storage) are private.Testing
az bicep buildonmain.bicepand each of the 7 modules individually: all compile with zero errors and zero warnings.azuredeploy.jsonmatches a fresh rebuild ofmain.bicepexactly (no drift), matching the new CI check's logic.createUiDefinition.jsonandmain.parameters.jsonas syntactically valid JSON.az deployment group validate/what-if against a real Azure subscription (no Azure credentials available in this environment).Documentation and compatibility
deploy/azure/README.mdadded; rootREADME.mdupdated with a "Deploy to Azure" section and button.infra//azd setup.Checklist
up()/down()and keep it CosmosDB-compatible. (N/A)NOTICE/NOTICE-REVIEW.txtwithpnpm noticeas needed. (N/A - no dependency changes)