Boost ML Feature Serving Latency by up to 29x with Pre-computed Feature Vectors in Feast
A new opt-in feature in Feast's online store pre-computes feature vectors to cut read latency drastically, supporting faster real-time ML applications without API changes.
Practical Summary
This update introduces an optional pre-computed feature vector mode in Feast's online store, which bundles all features for an entity into a single blob. By switching from multiple per-feature-view reads to one consolidated read, latency is reduced dramatically—up to 29x for batch operations. The feature is store-agnostic, auto-refreshes with standard materialization, and includes strict error handling and schema validation.
Why It Matters
For teams building revenue-growth workflows with real-time ML—such as personalized recommendations, dynamic pricing, or fraud detection—faster feature serving directly translates to better user experiences, higher conversion rates, and lower infrastructure costs. This optimization allows scaling ML applications without proportional increases in database load or latency bottlenecks.
Understanding Pre-computed Feature Vectors
Traditionally, when Feast's `get_online_features()` request involves multiple feature views, it issues separate database reads for each view. Pre-computed feature vectors solve this by storing all features for a given entity (e.g., a user or product) as a single serialized protobuf blob. At query time, the server fetches one blob per entity instead of N reads, reducing the operation to O(1).
Key Design Decisions
The implementation is opt-in (set `precompute_online=True` on your FeatureService), works across all supported backends (Redis, DynamoDB, PostgreSQL, etc.), and automatically refreshes vectors during standard materialization (`materialize`, `materialize-incremental`, `push`). There's no silent fallback—if the pre-computed vector is missing or stale, a `RuntimeError` is raised, ensuring problems are visible.
Schema changes are detected via a fingerprint of feature names, and stale vectors are rejected with column-order-independent comparison. Individual feature view TTLs are still enforced within the blob, maintaining data freshness.
Performance Impact Benchmarks
Benchmarks on a tuned server (4 CPU, 9 workers) with a service having 6 feature views and 29 features show significant latency reductions: