Just-In-Time Learning for Fast and Flexible Inference