Who Leads Now? Token-Level Modality Arbitration for Chart-to-Code Generation
Researchers have developed a new model called MoCA for chart-to-code generation. This task requires a model to read visual details of a chart and write executable code that reproduces it. Existing methods either train visual and coding abilities separately or fine-tune on chart-to-code data with both abilities entangled, which doesn't account for their distinct nature. MoCA separates these abilities using a Cross-modal Arbitration Block (CAB) and a lightweight arbiter. The mo
Researchers have developed a new model called MoCA for chart-to-code generation. This task requires a model to read visual details of a chart and write executable code that reproduces it. Existing methods either train visual and coding abilities separately or fine-tune on chart-to-code data with both abilities entangled, which doesn't account for their distinct nature. MoCA separates these abilities using a Cross-modal Arbitration Block (CAB) and a lightweight arbiter. The model is trained in two stages: a supervised warm-up and reinforcement learning. Analysis shows that the arbiter learns structured allocations of visual and code contributions. MoCA delivers competitive performance on three benchmarks, with gains attributed to its joint contributions of complementary initialization and input-conditioned arbitration.
---
Why it matters: This matters because chart-to-code generation is an important task in AI research, particularly for applications like data analysis and visualization. A model that can effectively separate visual understanding from coding abilities could improve the accuracy and efficiency of this process.
Source: https://arxiv.org/abs/2608.15510
This article was originally published at: https://arxiv.org/abs/2608.15510