Skip to content

brian5Home/Reconciliation-Service

Repository files navigation

Reconciliation Service

A .NET 8 service that runs in Docker and reconciles data across multiple HTTP API endpoints. It uses a factory pattern for environment-specific logging and endpoint definitions, and config-driven reconciliation rules so you can add new endpoint pairs and discrepancy handling without code changes.

Architecture

  • Host: ASP.NET Core (WebApplication) hosting both a minimal API and a background worker.
  • Factory pattern: IEnvironmentConfigFactory returns EnvironmentOptions (log level, endpoint base URLs) per environment (Development, Staging, Production). Add new environments by adding config sections; no code change.
  • Config-driven reconciliation: Each job is defined in config (Reconciliation:Definitions) with source/target systems and paths, and ComparisonRules (exclude fields, numeric tolerance, how to treat missing data). New reconciliations = new config entries.
  • Orchestration: IReconciliationOrchestrator runs all (or one) definitions; IEndpointReconciler fetches from both endpoints and compares JSON using the rules.

Quick Start

# Build and run (Production env)
docker compose up -d

# Run with Development profile (different ports, verbose logging)
docker compose --profile dev up -d reconciliation-service-dev

API (default port 8080):

  • GET /health — health check
  • GET /reconciliation/definitions — list configured reconciliation jobs
  • POST /reconciliation/run — run all reconciliations
  • POST /reconciliation/run/{id} — run one reconciliation by id

Configuration

  • Environment: Set ASPNETCORE_ENVIRONMENT (e.g. Development, Staging, Production). The factory uses Environment:{EnvironmentName} for log level and EndpointBaseUrls.
  • Reconciliation: appsettings.jsonReconciliation section:
    • IntervalSeconds: background run interval (0 = only on-demand via API).
    • Definitions: array of reconciliation jobs (see example in appsettings.json).

Per-definition you configure:

  • SourceSystem / SourcePath, TargetSystem / TargetPath (systems resolve to base URLs from environment config).
  • ComparisonRules: ExcludeFields, IncludeFields, NumericToleranceByField, and whether to report missing source/target as discrepancies.

Recommendations

  1. Secrets: Keep API keys and sensitive base URLs out of appsettings in repo. Use User Secrets (dev), environment variables, or a secret store (e.g. Azure Key Vault) and bind to EndpointBaseUrls or a dedicated options class.
  2. Authentication: For authenticated APIs, register typed HttpClients per system (e.g. AddHttpClient("SystemA", client => { ... }).AddHttpMessageHandler<AuthHandler>()) and have EndpointReconciler (or a factory) use the right client per definition.
  3. Resilience: Add Polly to the HttpClient (retries, circuit breaker) so transient failures don’t mark reconciliations as failed unnecessarily.
  4. Structured logging: Serilog is already in place; use structured properties (e.g. DefinitionId, DiscrepancyCount) so logs can be queried in your log aggregator.
  5. Extensibility: For complex comparison logic (e.g. key-based matching of arrays), consider a strategy or rule chain per definition type, or a small scripting/config layer (e.g. JsonPath or a custom expression) in ComparisonRules.
  6. Scheduling: For more advanced schedules (cron, time windows), replace the simple delay loop in Worker with a scheduler (e.g. Quartz.NET or Hangfire) or run the worker only when triggered by an external scheduler calling POST /reconciliation/run.

Documentation

Document Description
docs/ARCHITECTURE.md Components, data flow, and factory pattern.
docs/API.md HTTP API reference (health, definitions, run).
docs/CONFIGURATION.md Configuration reference (Environment, Reconciliation, ComparisonRules).

API types are documented with XML comments; build with GenerateDocumentationFile to produce the XML doc file.

Testing

Unit tests use xUnit and Moq and live in tests/ReconciliationService.Tests/.

Run all tests:

dotnet test

Run with coverage (coverlet):

dotnet test --collect:"XPlat Code Coverage"

Test layout:

  • Factory: EnvironmentConfigFactory (environment-specific config from IConfiguration), ReconciliationConfigProvider (enabled definitions from options).
  • Services: EndpointUrlResolver (URL resolution from base URLs), EndpointReconciler (HTTP + comparison with fake handler), ReconciliationOrchestrator (run all / run one, error handling, case-insensitive id).

Project Layout

src/ReconciliationService/
├── Configuration/          # EnvironmentOptions, ReconciliationDefinition, ComparisonRules
├── Factory/                # IEnvironmentConfigFactory, EnvironmentConfigFactory, IReconciliationConfigProvider
├── Models/                 # ReconciliationResult, DiscrepancyRecord
├── Services/               # Orchestrator, EndpointReconciler, EndpointUrlResolver
├── Program.cs              # Host, API, DI, Serilog
└── Worker.cs               # Background reconciliation loop

tests/ReconciliationService.Tests/
├── Factory/                # EnvironmentConfigFactoryTests, ReconciliationConfigProviderTests
└── Services/                # EndpointUrlResolverTests, EndpointReconcilerTests, ReconciliationOrchestratorTests

docs/
├── ARCHITECTURE.md
├── API.md
└── CONFIGURATION.md

Build (local)

dotnet build
dotnet run --project src/ReconciliationService

Set ASPNETCORE_ENVIRONMENT to change environment (and thus logging/endpoints).

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors