-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Is your feature request related to a problem? Please describe.
DynamoDB online store latency is fundamentally limited by network round-trip time to the DynamoDB service (~50-80ms per request). Even with all code optimizations applied, achieving <60ms p99 latency for production workloads (50 entities × 200 features) is not possible with direct DynamoDB access.
Benchmarks results here shows: https://github.com/abhijeet-dhumal/featurestore-benchmarks/tree/perf-with-6optimisations/fs-online-benchmark/locust-bechmarking/feast-benchmarking
- Current latency: 169ms p99 at 50 entities × 200 features
- Target SLA: 60ms p99
- Gap: 109ms that cannot be closed with code optimizations alone
This is frustrating for users who need low-latency feature serving for real-time ML inference but want to use DynamoDB for its scalability and managed infrastructure.
Describe the solution you'd like
Add native support for DynamoDB Accelerator (DAX) - AWS's in-memory caching layer for DynamoDB. This would allow users to configure a DAX endpoint for sub-millisecond cached reads:
online_store:
type: dynamodb
region: us-east-1
use_dax: true
endpoint_url: "daxs://my-cluster.abc123.dax-clusters.us-east-1.amazonaws.com:8111"
When enabled, Feast would use amazon-dax-client instead of boto3 for read operations, providing:
- <1ms latency for cached reads (vs 50-80ms for DynamoDB)
- 10x+ throughput improvement
- Transparent caching with no application changes
Describe alternatives you've considered
- VPC Endpoints: Reduces latency by ~10-20ms but still limited by DynamoDB service latency.
- Application-level caching (Redis sidecar): Adds operational complexity, requires cache invalidation logic, and duplicates the online store concept.
- Switch to Redis online store: Achieves lower latency (~102ms at same scale) but loses DynamoDB's serverless scaling, managed infrastructure, and AWS integration benefits.
- Reduce batch size: Requesting fewer entities per call meets latency benchmark but increases total request count and doesn't solve the underlying latency issue.
- ElastiCache for DynamoDB: Not a direct integration; requires custom implementation.
None of these alternatives provide the combination of low latency + DynamoDB's managed infrastructure that DAX offers natively.
| Configuration | P99 Latency (50 entities × 200 features) |
|---|---|
| DynamoDB (current Feast master branch) | 169ms |
| DynamoDB + VPC Endpoint | ~150ms (expected) |
| DynamoDB + DAX | ~40ms (expected) |
Implementation scope:
- Add use_dax: bool config option
- Conditional client initialization using amazon-dax-client
- Optional dependency in pyproject.toml
- No changes to read/write API signatures
DAX characteristics:
- Fully API-compatible with DynamoDB for reads
- Write-through cache (writes go to both DAX and DynamoDB)
- Cost: ~$0.25/hr for dax.r5.large node
- Runs inside VPC (same network as application pods)
References:
- AWS DAX Documentation
- amazon-dax-client on PyPI
- Benchmark results from Feast online store performance testing