Feature Stores and Training-Serving Skew
A feature has one definition and two readers. Training reads its history as of each label time, and serving reads its latest value. When the two readers disagree, the model sees inputs that it never trained on.
100 minutesBuildPythonMembers
Learning Objectives
- Describe the offline store, the online store, entities, event timestamps, and TTLs, and the job of each one
- Implement a point-in-time correct join and show how a naive join copies future values into training data
- Name four causes of training-serving skew: two code paths, stale features, different defaults, and time travel
- Build a feature registry in which one definition feeds both the training path and the serving path
- Backfill a new feature over past dates without reading data from after each date
- Detect skew by comparing logged serving features with the training path for the same entity and time