about
ChartNet: A High-Quality Multimodal Dataset for Robust Chart Understanding (arxiv.org)
5 points by droidjj 122 days ago | hide | past | pdf | discuss on HN

In plain words: A training set pairs a chart with the code that drew it, its table, a summary, and questions, so models link pictures, numbers, and words. Its 1.5 million examples span 24 chart types, and fine-tuning on them beats the usual training on chart tests.

Abstract · ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding

Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language models (VLMs) remain limited. We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advance chart interpretation and reasoning. ChartNet leverages a novel code-guided synthesis pipeline to generate 1.5 million diverse chart samples spanning 24 chart types and 6 plotting libraries. Each sample consists of five aligned components: plotting code, rendered chart image, data table, natural language summary, and question-answering with reasoning, providing fine-grained cross-modal alignment. To capture the full spectrum of chart comprehension, ChartNet additionally includes specialized subsets encompassing human annotated data, real-world data, safety, and grounding. Moreover, a rigorous quality-filtering pipeline ensures visual fidelity, semantic accuracy, and diversity across chart representations. Fine-tuning on ChartNet consistently improves results across benchmarks, demonstrating its utility as large-scale supervision for multimodal models. As the largest open-source dataset of its kind, ChartNet aims to support the development of foundation models with robust and generalizable capabilities for data visualization understanding. The dataset is publicly available at https://huggingface.co/datasets/ibm-granite/ChartNet

Jovana Kondic, Pengyuan Li, Dhiraj Joshi, Isaac Sanchez, Ben Wiesel, Shafiq Abedin, Amit Alfassy, Eli Schwartz, Daniel Caraballo, Yagmur Gizem Cinar, Florian Scheidegger, Steven I. Ross, et al.
arXiv:2603.27064 · cs.CV, cs.AI, cs.CL · submitted Mar 28, 2026 · updated Apr 14, 2026
abstract · pdf · html · Accepted at CVPR 2026

add comment on HN