SEMOSS-deployment

SEMOSS can be deployed as a service running in a Kubernetes cluster on a cloud provider. The SEMOSS deployment requires the following infrastructure resources:

• Storage bucket.
• Database (PostgreSQL)
• Kubernetes cluster

The Following diagram details the Kubernetes and cloud provider resources that will be created (exmaple for Azure).

 

Before deploying the SEMOSS container, ensure that the pod has access to both the storage bucket (e.g., S3, Azure Blob, Google Bucket) and the database. Once the necessary policies are in place, the pods and services can be deployed to the Kubernetes cluster.

The details of these cloud resources are passed to the SEMOSS pods as environment variables. These are centralized in the semoss-config-and-secrets.yml manifest — a ConfigMap for non-sensitive settings and a Secret for credentials — which the SEMOSS Deployment consumes via envFrom rather than inlining every variable.

There are two ways to bring up the whole stack in dependency order. Both read their configuration from semoss-config-and-secrets.yml, so edit that file first — fill in the database, storage, image, and auth placeholders.

Prerequisites: an external PostgreSQL database and a storage bucket must already exist (see Database configuration), and an NGINX ingress controller should be installed (see Ingress service).

Option A — declarative, one command (Kustomize):

kubectl apply -k .

Creates objects in a dependency-friendly order (Namespace → ConfigMap/Secret → Services → workloads → Ingress). It does not wait for readiness between objects; SEMOSS retries its connections until the database and coordination backend are up.

Option B — staged, with readiness gating:

./deploy.sh                            # Redis + Sentinel backend (default)
CLUSTER_BACKEND=zookeeper ./deploy.sh  # ZooKeeper ensemble instead
TIMEOUT=300s ./deploy.sh               # override the per-stage readiness timeout

Applies each stage and blocks on kubectl rollout status until the coordination backend is Ready before starting SEMOSS.

Both default to the Redis + Sentinel coordination backend. To use ZooKeeper instead, see Cluster coordination backend. Whichever backend you choose in the manifests must match the one enabled in semoss-config-and-secrets.yml.

The sections below walk through the same resources individually for manual / customized deploys.

If you are using a managed database service, the following databases or schemas need to be created.

Databases/Schemas that need to be created:

• security
• localmaster
• scheduler
• themes
• user_tracking
• prompt_hub

It is best to make these separate database instances for independent scaling

• model_logs
• audit_logs

The names used for the schemas/databases can be changed to fit the project's naming standards.

If you are using a PostgreSQL database, you can use psql to create the databases or schemas.

apt install postgresql

sudo dnf update -y
sudo dnf install postgresql -y

psql -h mydb.xxxxxxx.us-east-1.rds.amazonaws.com -p 5432 -U myuser -d postgres

CREATE DATABASE <DATABASE_NAME>;

\l

\q

Once SEMOSS pods are launched, they will connect to the databases and initialize tables upon its first launch.

We use a load balancer to expose the SEMOSS application, and a way to control the load balancer creation in a cloud environment is by using Ingress Controllers. The Ingress-NGINX Ingress controller is recommended to externally expose the SEMOSS service.

The Ingress-Nginx can launch a load balancer and manage it using annotations. The instructions on how to deploy the Ingress controller in the cloud environment can be found in the project's guide page.

Note: The Ingress-NGINX controller can use annotations from a cloud provider's controller, such as the AWS Load Balancer Controller. Please see the documentation for your cloud provider for more information and the Ingress-NGINX documentation for compatibility and usage information.

First create the semoss namespace (the Quick deploy commands do this for you):

kubectl apply -f namespace.yml     # or: kubectl create ns semoss

SEMOSS uses a coordination backend to synchronize state across pods in a cluster. Choose one of the two options below — the backend you deploy must match the one enabled in semoss-config-and-secrets.yml.

SEMOSS supports three Redis topologies — standalone, Sentinel (HA with automatic failover), and Redis Cluster (sharded). Enable one by setting the matching keys in the ConfigMap (REDIS_SENTINEL_ENABLED / REDIS_CLUSTER_ENABLED); the shared keys apply to all modes:

Shared keys (all modes): SEMOSS_IS_CLUSTER_REDIS, REDIS_ENABLED, REDIS_TIMEOUT_MS, REDIS_POOL_MAX_TOTAL, REDIS_POOL_MAX_IDLE, REDIS_POOL_MIN_IDLE.

The default is a highly-available Sentinel setup: a master + 2 replicas (a StatefulSet) with 3 Sentinels (a StatefulSet). Deploy it with:

kubectl apply -f redis-headless-service.yml -f redis-config-configmap.yml -f redis-statefulset.yml
kubectl apply -f sentinel-config-configmap.yml -f redis-sentinel-service.yml -f sentinel-statefulset.yml

The Sentinel keys are already set in the ConfigMap (REDIS_SENTINEL_NODES lists the three Sentinel pods). Set REDIS_PASSWORD / REDIS_SENTINEL_PASSWORD in the Secret if your Redis/Sentinel require auth.

Simpler option: for a single-node (standalone) Redis (dev / smoke tests), see the filled-in example/semoss-with-postgres-minio-redis stack. A Redis Cluster is typically an externally-managed service; point REDIS_CLUSTER_NODES at its endpoints.

A 3-node ZooKeeper ensemble (tolerates one node failing). Deploy its headless service (for peer discovery), client service, and the StatefulSet (which gives each node persistent /data and /datalog volumes):

kubectl apply -f zookeeper-headless-service.yml -f zookeeper-service.yml -f zookeeper-statefulset.yml

The ensemble is self-configuring: each pod derives a unique ZOO_MY_ID from its ordinal, and ZOO_SERVERS lists all three members via the headless service DNS. Then, in semoss-config-and-secrets.yml, disable the Redis block (SEMOSS_IS_CLUSTER_REDIS: "false" and remove the Redis keys) and uncomment the ZooKeeper block (SEMOSS_IS_CLUSTER_ZK, ZK_SERVER). Use one backend or the other, not both.

Simpler option: for a single-node ZooKeeper, see example/semoss-with-postgres-minio-zk. The older single-pod Deployment under legacy/ is superseded by the StatefulSet.

SEMOSS configuration lives in semoss-config-and-secrets.yml, split into a ConfigMap (non-sensitive settings) and a Secret (credentials). The SEMOSS Deployment pulls all of these in with envFrom instead of declaring each variable inline:

      envFrom:
        - configMapRef:
            name: semoss-config
        - secretRef:
            name: semoss-secrets

Edit semoss-config-and-secrets.yml to set your database connection details, storage bucket/provider, authentication providers, and cluster backend. That file is organized into commented sections — backing databases, authentication & security (SSO providers), R/Python execution, file limits, clustering, object storage, and runtime/sandboxing — with placeholders (<...>) for the values you need to fill in.

Key groups to review:

Fully filled-in, runnable stacks live under example/ — copy one as a starting point:

These examples run Postgres and MinIO inside the cluster. In production you'll more often point SEMOSS at managed / SaaS services instead — the section below lists the env vars that change.

When the database and object storage are managed services (e.g. AWS RDS + S3, Cloud SQL + GCS, Azure Database + Blob) rather than in-cluster pods, you don't deploy the Postgres/MinIO manifests at all — you only change connection details in semoss-config-and-secrets.yml.

Databases (managed Postgres). There is one set of keys per database (security, localmaster, scheduler, themes, user_tracking, model_logs, prompt_hub, audit_logs). Non-sensitive keys live in the ConfigMap; credentials and the URL live in the Secret. <db> below is the per-database prefix (e.g. CUSTOM_SECURITY_...).

model_logs and audit_logs are high-write; if you want to scale them independently, put them on their own managed instance by giving those two a different host in their CONNECTION_URL.

Object storage (managed buckets). Set SEMOSS_STORAGE_PROVIDER and the matching keys; access/secret keys go in the Secret. Omit S3_ENDPOINT (that key is only for pointing S3-mode at MinIO):

_{* credentials belong in the Secret, not the ConfigMap.}

On a cloud cluster you can skip storage keys entirely and let the pod authenticate through the cloud's pod-identity mechanism (AWS IRSA / GCP + Azure Workload Identity) — the recommended approach in production, since no credentials live in the manifests and they rotate automatically. The pod runs as the serviceaccount.yml ServiceAccount bound to a cloud IAM role instead of using S3_ACCESS_KEY / S3_SECRET_KEY. See extra-cloud-config/aws-semoss-deployment.md for the full walkthrough (IAM role, trust policy, and the GCP/Azure annotation equivalents).

Other values you'll typically set (managed or not):

For the full list of configuration parameters, refer to the AI Server Configuration Parameters.

After updating the configuration, deploy SEMOSS. With the Quick deploy commands the Deployment and Service are applied for you; to do it manually, apply the SEMOSS-deployment and then expose the pods with the SEMOSS-service YAML file:

kind: Service
metadata:
  name: semoss-service
  namespace: semoss
spec:
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/instance: semoss
    app.kubernetes.io/name: semoss
  sessionAffinity: None
  type: ClusterIP

Use the SEMOSS-ingress YAML file to create the ingress resource and define the rules for routing traffic to the backend SEMOSS service.

Note: The ingress resource can be configured to have externally-reachable URLs and terminate SSL / TLS. Please see the Kubernetes documentation for more information.

kubectl -n semoss create secret tls <testsecret-tls> --cert=path/to/tls.crt --key=path/to/tls.key    

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: semoss-ingress
  namespace: semoss
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 500m
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
    nginx.ingress.kubernetes.io/session-cookie-name: semoss-nginx-sticky
    nginx.ingress.kubernetes.io/session-cookie-path: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.org/client-max-body-size: 500m
spec:
  tls:
  - hosts:
      - demo.semoss.org # URL
    secretName: testsecret-tls
  rules:
    - host: demo.semoss.org # URL
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: semoss-service
                port:
                  number: 8080

  

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/app-root: /SemossWeb
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/hsts: "true"
    nginx.ingress.kubernetes.io/hsts-include-subdomains: "true"
    nginx.ingress.kubernetes.io/hsts-max-age: "31536000"
    nginx.ingress.kubernetes.io/hsts-preload: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 500m
    nginx.ingress.kubernetes.io/proxy-cookie-path: ~*^/.* /
    nginx.ingress.kubernetes.io/proxy-read-timeout: "350"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "350"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
    nginx.ingress.kubernetes.io/session-cookie-name: semoss-nginx-prod
    nginx.ingress.kubernetes.io/session-cookie-path: /
    nginx.ingress.kubernetes.io/session-cookie-secure: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.org/client-max-body-size: 500m
  name: semoss-ingress
  namespace: semoss
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: semoss-service
            port:
              number: 8080
        path: /
        pathType: ImplementationSpecific

After the ingress resource has been created get the Load balancer's Address.

$ kubectl -n semoss get ingress -o wide
NAME             CLASS    HOSTS             ADDRESS                      PORTS   AGE
semoss-ingress   <none>   demo.semoss.org   ID.elb.REGION.amazonaws.com   80      87d

The address value will be used to replace the INGRESS_DNS placeholder mentioned in the REDIRECT key value.

• AWS