about
Multi-Modal vs. Text-Based: Benchmarking LLM Strategies for Invoice Processing (arxiv.org)
1 point by PaulHoule on Sep 25, 2025 | hide | past | pdf | discuss on HN

In plain words: AI models from three families were tested on public invoice datasets, either reading the invoice image directly or converting it to text first. Feeding the image straight in generally worked better than the text conversion route.

Abstract · Multi-Modal Vision vs. Text-Based Parsing: Benchmarking LLM Strategies for Invoice Processing

This paper benchmarks eight multi-modal large language models from three families (GPT-5, Gemini 2.5, and open-source Gemma 3) on three diverse openly available invoice document datasets using zero-shot prompting. We compare two processing strategies: direct image processing using multi-modal capabilities and a structured parsing approach converting documents to markdown first. Results show native image processing generally outperforms structured approaches, with performance varying across model types and document characteristics. This benchmark provides insights for selecting appropriate models and processing strategies for automated document systems. Our code is available online.

David Berghaus, Armin Berger, Lars Hillebrand, Kostadin Cvejoski, Rafet Sifa
arXiv:2509.04469 · cs.CL, cs.AI · submitted Aug 29, 2025
abstract · pdf · html

add comment on HN