Skip to content

Repository files navigation

Go-Micro: Distributed Services Ecosystem

A production-oriented microservices ecosystem built with Go 1.21+, demonstrating resilient distributed system design using a hybrid communication model:

  • Synchronous Request–Response (HTTP/RPC)
  • Asynchronous Event-Driven Messaging (AMQP / RabbitMQ)

The system is designed for scalability, decoupling, and operational clarity.


🏗 Architecture Overview

The ecosystem follows the API Gateway (Broker) Pattern.

The Broker Service acts as the single entry point for all external traffic. It decouples clients from internal services while enabling:

  • Centralized request orchestration
  • Protocol translation (JSON ↔ RPC/AMQP)
  • Logging redirection
  • Future authentication and rate limiting

System Topology

graph TD
    subgraph "External"
        Client
    end
    Client -->|REST/JSON| Broker

    subgraph "Synchronous Layer"
        Broker -->|HTTP| Auth
        Broker -->|RPC| Logger
        Broker -->|HTTP| Mail
        Auth -->|SQL| Postgres
    end

    subgraph "Asynchronous Layer"
        Broker -->|AMQP Publish| RabbitMQ
        RabbitMQ -->|Topic Subscription| Listener
        Listener -->|Internal Logic| Logger
    end
Loading

🧱 Core Technologies

  • Language: Go 1.21+

  • Concurrency Model: Goroutines (non-blocking I/O)

  • Messaging: RabbitMQ (AMQP 0.9.1, Topic Exchanges)

  • Databases:

    • PostgreSQL (Authentication / Relational data)
    • MongoDB (Structured logging)
  • Email Testing: MailHog (SMTP trapping)

  • Containerization: Docker / Docker Compose


🎯 Architectural Principles

1️⃣ Resiliency via Exponential Backoff

Services implement retry strategies during startup to prevent crash loops in containerized environments.

Example pattern used:

time.Sleep(time.Duration(math.Pow(float64(attempt), 2)) * time.Second)

This ensures dependent services (e.g., RabbitMQ, Postgres) have time to initialize before connection attempts exhaust.

Production Consideration: In larger deployments, retries would ideally be handled via:

  • Kubernetes readiness/liveness probes
  • Service mesh (Istio / Linkerd)
  • Circuit breaker patterns

2️⃣ Communication Duality

The system intentionally uses two communication paradigms:

🔹 Immediate Consistency (HTTP/RPC)

Used for:

  • Authentication
  • Mail service

These require real-time confirmation to the client.

🔹 Eventual Consistency (AMQP)

Used for:

  • Logging
  • Background workflows

This allows the Broker to remain responsive even if consumers (e.g., Logger) are temporarily unavailable.


3️⃣ Strict Separation of Concerns

Service Responsibility
Broker Protocol translation, orchestration, external gateway
Auth Identity management, password hashing (bcrypt), DB persistence
Logger Centralized structured logging (RPC/HTTP → MongoDB)
Mail Email abstraction layer
Listener Background event consumer

Each service is independently deployable and loosely coupled.


📂 Service Registry

Service Port Protocol Data Store
Broker 8080 HTTP —
Auth 8081 HTTP PostgreSQL
Logger 8083 RPC / HTTP MongoDB
Mail 8084 HTTP MailHog
Listener — AMQP —

⚠️ Current Constraints

  1. RPC Implementation

    • Uses Go’s net/rpc
    • Stable but not actively evolving
    • Recommended future migration: gRPC
  2. Service Discovery

    • Relies on Docker Compose DNS
    • Production-ready alternative: Kubernetes DNS / Consul
  3. Asynchronous Error Handling

    • No Dead Letter Exchange (DLX) implementation yet
    • Failed messages are logged but not re-queued

🚀 Running the Ecosystem

The workspace is fully containerized.

Start All Services

make up

View Running Containers

make ps

Stop & Clean

make down

🔎 Manual Verification

  • Broker Health Check

    GET http://localhost:8080
    
  • MailHog UI

    http://localhost:8025
    
  • RabbitMQ Management

    http://localhost:15672
    guest / guest
    

🛠 Development Workflow

Adding a New Broker Action

  1. Update RequestPayload contract.
  2. Add a new switch case in the submission handler.
  3. Implement communication logic (HTTP / RPC / AMQP).

Updating Auth Models

When modifying authentication models:

  • Respect dbTimeout (currently 3s)
  • Always use context.WithTimeout
  • Ensure connections close cleanly

📈 Roadmap

  • gRPC migration (Protobuf contracts)
  • Graceful shutdown with os.Signal
  • Distributed tracing (OpenTelemetry)
  • Dead Letter Exchange (DLX) for RabbitMQ
  • Circuit breaker implementation
  • Structured logging across all services
  • JWT-based authentication at Broker level

🧠 Engineering Focus

This project demonstrates:

  • Distributed system fundamentals
  • Hybrid communication modeling
  • Resilient startup strategies
  • Protocol translation
  • Service decoupling
  • Containerized development workflow

About

go microservice

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages