LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Models
Researchers have introduced LiveHouse-TS, an open-world living benchmark for Time Series Foundation Models (TSFMs). Unlike traditional benchmarks that rely on static data with fixed historical test windows, LiveHouse-TS evaluates models prequentially on real future data in open-world environments. This allows for a more accurate assessment of a model's performance over time, including its ability to adapt to seasonal variations and distribution shifts. The authors demonstrate
Researchers have introduced LiveHouse-TS, an open-world living benchmark for Time Series Foundation Models (TSFMs). Unlike traditional benchmarks that rely on static data with fixed historical test windows, LiveHouse-TS evaluates models prequentially on real future data in open-world environments. This allows for a more accurate assessment of a model's performance over time, including its ability to adapt to seasonal variations and distribution shifts. The authors demonstrate the effectiveness of their benchmark by evaluating 11 domains across 17 datasets, showing that static rankings can change dramatically under a live protocol.
---
Why it matters: This matters because it provides a more realistic evaluation framework for TSFMs, which are increasingly being used in real-world applications such as weather forecasting and financial modeling. By assessing model performance over time, researchers can better understand their strengths and weaknesses and develop more robust models that can adapt to changing conditions.
Source: https://arxiv.org/abs/2608.17299
This article was originally published at: https://arxiv.org/abs/2608.17299