Researchers have studied how the AdamW optimizer affects training dynamics beyond a single update cycle. They found that past gradient information stored in the optimizer's state can influence future updates. This delayed effect is due to the way AdamW stores and uses model parameters and moment estimates. The study used linearization and error decomposition to understand how this works, revealing patterns in how gradients affect loss over time. --- Why it matters: This research matters because it sheds light on how popular optimizers like AdamW can impact training dynamics, potentially leading to more accurate models or better understanding of optimization algorithms. Source: https://arxiv.org/abs/2608.19762