AI

Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Task

Researchers have developed a method to reduce the energy consumption of large language models (LLMs) used in software engineering tasks. The method, called energy-aware knowledge distillation, uses direct estimates of CPU and GPU energy consumption instead of relying on Floating Point Operations (FLOPs). This approach can decrease inference energy consumption by up to 90% and memory usage by 86%, with only minor trade-offs in accuracy. The study focuses on LLMs applied to clo
Researchers have developed a method to reduce the energy consumption of large language models (LLMs) used in software engineering tasks. The method, called energy-aware knowledge distillation, uses direct estimates of CPU and GPU energy consumption instead of relying on Floating Point Operations (FLOPs). This approach can decrease inference energy consumption by up to 90% and memory usage by 86%, with only minor trade-offs in accuracy. The study focuses on LLMs applied to clone detection, vulnerability prediction, and code summarization tasks. --- Why it matters: This research is important for engineers working with large language models, as it provides a more accurate way to estimate energy consumption and optimize model efficiency. By using direct energy surrogates, developers can create more sustainable and deployable LLMs on consumer hardware and resource-constrained platforms. Source: https://arxiv.org/abs/2608.17515

This article was originally published at: https://arxiv.org/abs/2608.17515