AI

Entropy-Constrained Adaptive Stochastic Quantization

Researchers have proposed a new method called Entropy-Constrained Adaptive Stochastic Quantization (ECASQ) to improve the efficiency of data and machine learning workloads. ECASQ jointly selects quantization values to minimize Mean Squared Error (MSE) under an entropy budget and an unbiasedness constraint. The authors provide optimal and approximate dynamic programs for solving the ECASQ problem, which can be implemented on GPUs. They also present an iterative refinement proc
Researchers have proposed a new method called Entropy-Constrained Adaptive Stochastic Quantization (ECASQ) to improve the efficiency of data and machine learning workloads. ECASQ jointly selects quantization values to minimize Mean Squared Error (MSE) under an entropy budget and an unbiasedness constraint. The authors provide optimal and approximate dynamic programs for solving the ECASQ problem, which can be implemented on GPUs. They also present an iterative refinement procedure that achieves near-optimal results while retaining a speed advantage over their solver for the optimal solution. --- Why it matters: This work matters to researchers in AI because it addresses the communication and memory bottlenecks of modern data and machine learning workloads, allowing for more efficient compression and storage of data. The proposed method can be applied to various applications, including model, gradient, and KV-cache compression, and nearest-neighbor search. Source: https://arxiv.org/abs/2608.18147

This article was originally published at: https://arxiv.org/abs/2608.18147