This directory contains CI/CD workflows for the CUDly project, providing automated testing, deployment, and operations across AWS, GCP, and Azure.

Note: Frontend deployment is handled automatically via Terraform as part of the backend deployment workflows.

File: ci.yml

Runs comprehensive quality checks on every pull request and push to main branch.

1. Lint - golangci-lint, go vet
2. Unit Tests - Go tests with race detection, coverage reporting
3. Integration Tests - Tests with real PostgreSQL
4. Docker Build - Build and test Docker image
5. Terraform Validate - Validate all Terraform configs (AWS, GCP, Azure)
6. Security Scan - gosec, trivy, tfsec
7. Snyk Scan - Dependency vulnerability scanning
8. E2E Tests - Docker Compose end-to-end tests
9. Cost Estimate - Infracost cost estimation (PR only)

• Pull requests to main or develop
• Pushes to main or develop
• Manual dispatch

• SNYK_TOKEN (optional - for Snyk scanning)
• INFRACOST_API_KEY (optional - for cost estimation)

• GO_VERSION (default: 1.26.6)

# Automatically runs on PR
git push origin feature-branch

# Or trigger manually
gh workflow run ci.yml

File: deploy-aws-lambda.yml

Deploy CUDly to AWS Lambda with Function URL. Serverless, event-driven platform.

1. Prepare - Determine environment and image tag
2. Build & Push - Build Docker image, push to ECR
3. Deploy - Deploy with Terraform
4. Test - Health check and smoke tests

• Push to main (deploys to dev)
• Release creation (deploys to prod)
• Manual dispatch with environment selection

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY

• AWS_REGION (default: us-east-1)
• AWS_ACCOUNT_ID
• ECR_REPOSITORY (default: cudly)

# Deploy to dev
gh workflow run deploy-aws-lambda.yml -f environment=dev

# Push to main also deploys to dev
git push origin main

# Deploy to prod
gh release create v1.0.0

• Function URL: https://<id>.lambda-url.us-east-1.on.aws
• Deployment info artifact

File: deploy-aws-fargate.yml

Deploy CUDly to AWS ECS Fargate with ALB. Always-on containerized platform.

1. Build & Push - Build Docker image, push to ECR
2. Deploy - Deploy with Terraform (Fargate mode)
3. Test - Health check verification

• Manual dispatch only

• Same as AWS Lambda

# Deploy to staging with Fargate
gh workflow run deploy-aws-fargate.yml -f environment=staging

File: deploy-gcp.yml

Deploy CUDly to GCP Cloud Run. Serverless container platform.

1. Build & Deploy - Build, push to Artifact Registry, deploy with Terraform
2. Test - Health check and smoke tests

• Manual dispatch
• Called by deploy-all.yml

• GCP_SA_KEY (Service Account JSON with permissions)
• GCP_PROJECT_ID

• GCP_REGION (default: us-central1)
• ARTIFACT_REGISTRY_REPO (default: cudly)

# Deploy to GCP dev
gh workflow run deploy-gcp.yml -f environment=dev

• Service URL: https://cudly-<hash>-uc.a.run.app

File: deploy-azure.yml

Deploy CUDly to Azure Container Apps. Serverless container platform with built-in HTTPS.

1. Build & Deploy - Build, push to ACR, deploy with Terraform
2. Test - Health check and smoke tests

• Manual dispatch
• Called by deploy-all.yml

• AZURE_CREDENTIALS (Service Principal JSON)
• AZURE_SUBSCRIPTION_ID

• AZURE_LOCATION (default: eastus)
• ACR_NAME (default: cudlyacr)

# Deploy to Azure staging
gh workflow run deploy-azure.yml -f environment=staging

• App URL: https://<app-name>.<region>.azurecontainerapps.io

File: deploy-all.yml

Orchestrate deployment to multiple cloud providers in parallel.

1. Determine Strategy - Choose which clouds to deploy to
2. Deploy AWS Lambda - Parallel deployment
3. Deploy AWS Fargate - Parallel deployment (optional)
4. Deploy GCP - Parallel deployment
5. Deploy Azure - Parallel deployment
6. Notify - Aggregate results

• Manual dispatch with provider selection
• Release creation (deploys to all clouds in prod)

• All secrets from individual deployment workflows

• all - Deploy to AWS, GCP, and Azure
• aws-only - AWS Lambda only
• gcp-only - GCP Cloud Run only
• azure-only - Azure Container Apps only
• aws-gcp - AWS and GCP
• aws-azure - AWS and Azure
• gcp-azure - GCP and Azure

# Deploy to all clouds (staging)
gh workflow run deploy-all.yml -f environment=staging -f deploy_to=all

# Deploy to AWS and GCP (prod)
gh workflow run deploy-all.yml -f environment=prod -f deploy_to=aws-gcp

# Automatic on release
gh release create v1.0.0

• Disaster Recovery - Multi-cloud redundancy
• Cost Optimization - Compare costs across providers
• Testing - Validate across all platforms
• Global Reach - Deploy to optimal regions per cloud

File: database-migration.yml

Apply or rollback database schema migrations across cloud providers.

1. Validate - Safety checks
2. Migrate AWS - Run golang-migrate on Aurora
3. Migrate GCP - Run golang-migrate on Cloud SQL
4. Migrate Azure - Run golang-migrate on Flexible Server

• Manual dispatch only (safety measure)
• Can be called by deployment workflows

• DB_PASSWORD_AWS
• DB_PASSWORD_GCP
• DB_PASSWORD_AZURE
• Cloud credentials (same as deployment workflows)

• Database endpoints per environment

• up - Apply migrations (default)
• down - Rollback migrations (DANGEROUS)

# Apply all migrations to AWS dev
gh workflow run database-migration.yml \
  -f cloud=aws \
  -f environment=dev \
  -f direction=up

# Rollback last 2 migrations on GCP staging
gh workflow run database-migration.yml \
  -f cloud=gcp \
  -f environment=staging \
  -f direction=down \
  -f steps=2

# Rollback 1 migration on AWS prod (requires typed confirmation)
gh workflow run database-migration.yml \
  -f cloud=aws \
  -f environment=prod \
  -f direction=down \
  -f steps=1 \
  -f confirm=rollback-prod

# Apply to all clouds
gh workflow run database-migration.yml \
  -f cloud=all \
  -f environment=prod \
  -f direction=up

• Validation - Checks migration files exist before running
• Explicit steps required - direction=down requires an explicit positive steps value; steps=0 (the default, which would run down -all and drop the entire schema) is rejected
• Production confirmation - direction=down on environment=prod additionally requires typing rollback-prod in the confirm input; omitting or mistyping it blocks the run
• Defense in depth - each migrate job independently re-validates the positive-steps constraint, so a future validate regression cannot reach down -all
• Audit Trail - Records all migrations in the step summary

File: rollback.yml

Quickly rollback to a previous deployment version by redeploying a known-good Docker image.

1. Validate - Validate image tag and construct image URI
2. Rollback - Confirm the image exists in the registry, then deploy it with Terraform
3. Summary - Create audit record

Image existence is verified inside each rollback job rather than in a standalone job. A separate verify job would have to assume the same cloud deploy role while carrying no environment: binding, which is exactly the ungated-but-credentialed shape that made the workflow exploitable. The tradeoff is that a rollback to a nonexistent tag now fails after the environment approval rather than before it.

• Manual dispatch only (safety measure)

• Cloud credentials (same as deployment workflows)

# Rollback AWS Lambda production to previous version
gh workflow run rollback.yml \
  -f cloud=aws-lambda \
  -f environment=prod \
  -f image_tag=sha-abc123 \
  -f reason="Critical bug in v1.2.3"

# Rollback GCP staging
gh workflow run rollback.yml \
  -f cloud=gcp \
  -f environment=staging \
  -f image_tag=v1.2.2

• Image Verification - Confirms image exists before deploying
• Audit Trail - Records all rollbacks (365 day retention)
• Reason Tracking - Requires reason for accountability
• Manual Only - Cannot be triggered automatically

# AWS ECR
aws ecr list-images --repository-name cudly

# GCP Artifact Registry
gcloud artifacts docker images list <region>-docker.pkg.dev/<project>/<repo>/cudly

# Azure ACR
az acr repository show-tags --name cudlyacr --repository cudly

AWS:

# Create secrets
gh secret set AWS_ACCESS_KEY_ID
gh secret set AWS_SECRET_ACCESS_KEY
gh secret set DB_PASSWORD_AWS

GCP:

# Create service account and download JSON
gcloud iam service-accounts create cudly-cicd --project=<project-id>

# Grant permissions
gcloud projects add-iam-policy-binding <project-id> \
  --member="serviceAccount:cudly-cicd@<project-id>.iam.gserviceaccount.com" \
  --role="roles/run.admin"

# Create and download key
gcloud iam service-accounts keys create key.json \
  --iam-account=cudly-cicd@<project-id>.iam.gserviceaccount.com

# Set secrets
gh secret set GCP_SA_KEY < key.json
gh secret set GCP_PROJECT_ID -b"<project-id>"
gh secret set DB_PASSWORD_GCP

Azure:

# Create service principal
az ad sp create-for-rbac --name cudly-cicd --sdk-auth > azure-credentials.json

# Set secrets
gh secret set AZURE_CREDENTIALS < azure-credentials.json
gh secret set AZURE_SUBSCRIPTION_ID -b"<subscription-id>"
gh secret set DB_PASSWORD_AZURE

Optional:

gh secret set SNYK_TOKEN
gh secret set INFRACOST_API_KEY

# AWS
gh variable set AWS_REGION -b"us-east-1"
gh variable set AWS_ACCOUNT_ID -b"123456789012"
gh variable set ECR_REPOSITORY -b"cudly"

# GCP
gh variable set GCP_REGION -b"us-central1"
gh variable set ARTIFACT_REGISTRY_REPO -b"cudly"

# Azure
gh variable set AZURE_LOCATION -b"eastus"
gh variable set ACR_NAME -b"cudlyacr"

# Frontend
gh variable set CLOUD_PROVIDER -b"aws"
gh variable set FRONTEND_BUCKET -b"cudly-frontend-prod"
gh variable set CLOUDFRONT_DISTRIBUTION_ID -b"E1234567890"
gh variable set API_URL -b"https://api.cudly.example.com"

GitHub Environments provide deployment protection and environment-specific secrets. GitHub auto-creates a referenced environment on first use with no protection rules and no branch policy, so relying on that instead of provisioning the list below produces exactly the gap this section exists to prevent (see #141). The list below is generated from the actual environment: expressions each workflow binds, not from what the environment happens to be named -- keep the two in sync when a workflow's binding changes.

1. Go to Settings → Environments

2. Create environments matching every environment: binding in .github/workflows/*.yml:

   • dev, staging, prod -- bound by deploy-aws-lambda.yml, deploy-gcp.yml, deploy-azure.yml, and three of rollback.yml's four jobs (rollback-aws-lambda, rollback-gcp, rollback-azure -- reusing the deploy environments rather than having their own, see #139). deploy-aws-fargate.yml and rollback.yml's rollback-aws-fargate job both use aws-fargate-<env> instead (below), matching each other rather than this group.
   • aws-fargate-dev, aws-fargate-staging, aws-fargate-prod -- deploy-aws-fargate.yml
   • aws-db-dev, aws-db-staging, aws-db-prod -- database-migration.yml (AWS)
   • gcp-db-dev, gcp-db-staging, gcp-db-prod -- database-migration.yml (GCP)
   • azure-db-dev, azure-db-staging, azure-db-prod -- database-migration.yml (Azure)
   • staging -- also bound by cleanup-staging.yml's destroy jobs (same staging environment as above, not a separate one)
   • dev -- also bound by destroy-fargate-dev.yml (same dev environment as above)
   • frontend-aws-dev, etc. -- if/when frontend deploy workflows gain an environment: binding

3. Configure protection rules:

   • Production (prod, aws-fargate-prod, aws-db-prod, gcp-db-prod, azure-db-prod): require approvals, restrict deployment_branch_policy to main
   • Staging (staging, aws-fargate-staging, aws-db-staging, gcp-db-staging, azure-db-staging): optional approvals, restrict to main
   • Dev (dev, aws-fargate-dev, aws-db-dev, gcp-db-dev, azure-db-dev): no restrictions

   Configuring protection rules here is necessary but not sufficient on its own: each cloud's OIDC trust must independently allowlist the environment:<name> subject for every name above, and the three clouds do this three different ways:

   • Azure (terraform/environments/azure/ci-cd-permissions/): the github_environments Terraform variable -- add the name there and re-apply, or azure/login fails with AADSTS70021.
   • AWS (terraform/environments/aws/ci-cd-permissions/role.tf): a literal sub list inlined in the role's assume-role policy, not a variable -- add the subject there and re-apply, or configure-aws-credentials fails with AssumeRoleWithWebIdentity denied.
   • GCP (terraform/environments/gcp/ci-cd-permissions/github_oidc.tf): ref-based, not environment-based -- its attribute_condition checks repository/ref only, never sub, so no GCP-side change is needed for a new environment name (see #141 for why that's its own, separate gap).

Unit tests fail:

# Run locally
make test-unit

Integration tests fail:

# Run with testcontainers
make test-integration

Security scan fails:

# Run locally
make security-scan-all

AWS - Image not found:

# Check ECR
aws ecr describe-images --repository-name cudly --region us-east-1

# Re-push image
docker push <account>.dkr.ecr.us-east-1.amazonaws.com/cudly:latest

GCP - Permission denied:

# Check service account permissions
gcloud projects get-iam-policy <project-id>

# Grant missing roles
gcloud projects add-iam-policy-binding <project-id> \
  --member="serviceAccount:<sa>@<project>.iam.gserviceaccount.com" \
  --role="roles/run.admin"

Azure - Resource not found:

# Verify resource group exists
az group show --name cudly-rg

# Create if missing
az group create --name cudly-rg --location eastus

Connection timeout:

• Check database security groups/firewall rules
• Verify VPN/bastion access if required
• Check database is running

Migration already applied:

# Check current version
migrate -path migrations -database <url> version

# Force version (use with caution)
migrate -path migrations -database <url> force <version>

• Require CI to pass before merging
• Require code reviews
• Restrict direct pushes to main

• Dev: Auto-deploy on push to develop branch
• Staging: Auto-deploy on push to main
• Prod: Manual approval required, deploy on release

• Keep last 10 images in each registry
• Document rollback procedures
• Test rollback in staging first

• Set up CloudWatch/Cloud Logging alerts
• Monitor deployment success rates
• Track deployment frequency

• Rotate secrets regularly
• Use environment protection rules
• Enable secret scanning
• Review security scan results

# View recent workflow runs
gh run list --limit 50

# View specific workflow
gh run list --workflow=ci.yml --limit 20

• Target: Multiple deployments per day
• Track via GitHub Actions insights

• Use rollback workflow for quick recovery
• Target: < 15 minutes

• Unit tests: ~5 min
• Integration tests: ~3 min
• Security scans: ~2 min
• Total: ~10 min target

• GitHub Actions Documentation
• AWS ECR Documentation
• GCP Artifact Registry
• Azure Container Registry
• golang-migrate
• Terraform Cloud