AI

Natural Language Code Retrieval for 1C:Enterprise: An Open Benchmark and Efficient Bi-Encoder

Researchers have developed an open benchmark for natural language code retrieval in the 1C:Enterprise ecosystem. This system combines Russian syntax with highly domain-specific terminology, making it challenging to develop specialized models. The team created a comprehensive pipeline consisting of a benchmark dataset, evaluation harness, and bi-encoder model. To overcome limited labeled data, they fine-tuned their model on synthetic triplets generated from public code reposit
Researchers have developed an open benchmark for natural language code retrieval in the 1C:Enterprise ecosystem. This system combines Russian syntax with highly domain-specific terminology, making it challenging to develop specialized models. The team created a comprehensive pipeline consisting of a benchmark dataset, evaluation harness, and bi-encoder model. To overcome limited labeled data, they fine-tuned their model on synthetic triplets generated from public code repositories using Matryoshka Representation Learning (MRL) and a privacy-aware tokenizer. --- Why it matters: This work matters to researchers in AI because it addresses the challenge of developing models for domain-specific languages like 1C:Enterprise. The proposed pipeline and benchmark dataset can be used as a starting point for future research on natural language code retrieval, enabling the development of more efficient and effective models. Source: https://arxiv.org/abs/2608.19957

This article was originally published at: https://arxiv.org/abs/2608.19957