GRAFT: Adaptive DLM-Based Draft Tree Construction with Target-Distilled Edge Scoring
Researchers have proposed a new framework called GRAFT for constructing draft trees in language models. The existing method, DDTree, has limitations when used with diffusion language models (DLMs) like DFlash. GRAFT addresses these issues by introducing two key components: Target-Distilled Edge Scoring (TDES), which selects target-compatible edges based on parent-child preferences distilled from target-model traces; and State-Aware Budget Allocation (SABA), which dynamically
Researchers have proposed a new framework called GRAFT for constructing draft trees in language models. The existing method, DDTree, has limitations when used with diffusion language models (DLMs) like DFlash. GRAFT addresses these issues by introducing two key components: Target-Distilled Edge Scoring (TDES), which selects target-compatible edges based on parent-child preferences distilled from target-model traces; and State-Aware Budget Allocation (SABA), which dynamically adjusts the tree budget to balance draft gain and verification cost. The authors claim that GRAFT achieves significant speedup over autoregressive decoding, with a $2.13 imes$--$6.36 imes$ end-to-end speedup across multiple models and tasks, while adding minimal overhead.
---
Why it matters: This matters because it could improve the efficiency of language model-based text generation systems, which are increasingly used in applications such as chatbots, virtual assistants, and natural language processing pipelines.
Source: https://arxiv.org/abs/2608.20375
This article was originally published at: https://arxiv.org/abs/2608.20375