Skip to content

Add DynamoDB Accelerator (DAX) support for sub-millisecond reads #6059

@abhijeet-dhumal

Description

@abhijeet-dhumal

Is your feature request related to a problem? Please describe.

DynamoDB online store latency is fundamentally limited by network round-trip time to the DynamoDB service (~50-80ms per request). Even with all code optimizations applied, achieving <60ms p99 latency for production workloads (50 entities × 200 features) is not possible with direct DynamoDB access.
Benchmarks results here shows: https://github.com/abhijeet-dhumal/featurestore-benchmarks/tree/perf-with-6optimisations/fs-online-benchmark/locust-bechmarking/feast-benchmarking

  • Current latency: 169ms p99 at 50 entities × 200 features
  • Target SLA: 60ms p99
  • Gap: 109ms that cannot be closed with code optimizations alone

This is frustrating for users who need low-latency feature serving for real-time ML inference but want to use DynamoDB for its scalability and managed infrastructure.

Describe the solution you'd like
Add native support for DynamoDB Accelerator (DAX) - AWS's in-memory caching layer for DynamoDB. This would allow users to configure a DAX endpoint for sub-millisecond cached reads:

online_store:
  type: dynamodb
  region: us-east-1
  use_dax: true
  endpoint_url: "daxs://my-cluster.abc123.dax-clusters.us-east-1.amazonaws.com:8111"

When enabled, Feast would use amazon-dax-client instead of boto3 for read operations, providing:

  • <1ms latency for cached reads (vs 50-80ms for DynamoDB)
  • 10x+ throughput improvement
  • Transparent caching with no application changes

Describe alternatives you've considered

  1. VPC Endpoints: Reduces latency by ~10-20ms but still limited by DynamoDB service latency.
  2. Application-level caching (Redis sidecar): Adds operational complexity, requires cache invalidation logic, and duplicates the online store concept.
  3. Switch to Redis online store: Achieves lower latency (~102ms at same scale) but loses DynamoDB's serverless scaling, managed infrastructure, and AWS integration benefits.
  4. Reduce batch size: Requesting fewer entities per call meets latency benchmark but increases total request count and doesn't solve the underlying latency issue.
  5. ElastiCache for DynamoDB: Not a direct integration; requires custom implementation.

None of these alternatives provide the combination of low latency + DynamoDB's managed infrastructure that DAX offers natively.

Configuration P99 Latency (50 entities × 200 features)
DynamoDB (current Feast master branch) 169ms
DynamoDB + VPC Endpoint ~150ms (expected)
DynamoDB + DAX ~40ms (expected)

Implementation scope:

  • Add use_dax: bool config option
  • Conditional client initialization using amazon-dax-client
  • Optional dependency in pyproject.toml
  • No changes to read/write API signatures

DAX characteristics:

  • Fully API-compatible with DynamoDB for reads
  • Write-through cache (writes go to both DAX and DynamoDB)
  • Cost: ~$0.25/hr for dax.r5.large node
  • Runs inside VPC (same network as application pods)

References:

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions