Researchers have proposed TreeWY, a method for verifying and rolling back speculative decoding in hybrid models like Gated DeltaNets. This is done by applying a tree-structured WY transform to the gated delta rule, which reduces memory usage and allows for wider acceptance trees without increasing memory requirements. The approach has been tested on two large-scale hybrid models, resulting in improved throughput and lower time-to-first-token when memory becomes a bottleneck. --- Why it matters: This matters because it addresses a significant limitation of current hybrid models, which struggle with speculative decoding due to high memory usage. By reducing this overhead, TreeWY enables the development of more efficient and scalable hybrid architectures. Source: https://arxiv.org/abs/2608.20961