about
Towards Optimizing SQL Generation via LLM Routing (arxiv.org)
11 points by PaulHoule on Nov 27, 2024 | hide | past | pdf | 1 comment on HN

In plain words: A router looks at each question and picks the cheapest AI model likely to write the correct database query, instead of always using the biggest one. It matched the strongest model's accuracy while lowering cost.

Abstract

Text-to-SQL enables users to interact with databases through natural language, simplifying access to structured data. Although highly capable large language models (LLMs) achieve strong accuracy for complex queries, they incur unnecessary latency and dollar cost for simpler ones. In this paper, we introduce the first LLM routing approach for Text-to-SQL, which dynamically selects the most cost-effective LLM capable of generating accurate SQL for each query. We present two routing strategies (score- and classification-based) that achieve accuracy comparable to the most capable LLM while reducing costs. We design the routers for ease of training and efficient inference. In our experiments, we highlight a practical and explainable accuracy-cost trade-off on the BIRD dataset.

Mohammadhossein Malekpour, Nour Shaheen, Foutse Khomh, Amine Mhedhbi
arXiv:2411.04319 · cs.DB, cs.AI, cs.LG · submitted Nov 6, 2024
abstract · pdf · html · Table Representation Learning Workshop at NeurIPS 2024

add comment on HN

Now this is how I see LLMs being properly leveraged. Certain databases I've come across in the past make aggregating data from various tables tough to do elegantly. Huge nasty SQL queries that are tough to write, but not necessarily tough to understand.