about
Search-Based Optimization of LLM Learning Shots for Story Point Estimation (arxiv.org)
2 points by zkirby on Mar 14, 2024 | hide | past | pdf | 1 comment on HN

In plain words: A search routine picks how many example tasks, and which ones, to show a language model before it estimates story points for new agile tasks. Across three datasets, these tuned examples cut the model's average estimation error by 59.34% compared with giving it no examples.

Abstract · Search-based Optimisation of LLM Learning Shots for Story Point Estimation

One of the ways Large Language Models (LLMs) are used to perform machine learning tasks is to provide them with a few examples before asking them to produce a prediction. This is a meta-learning process known as few-shot learning. In this paper, we use available Search-Based methods to optimise the number and combination of examples that can improve an LLM's estimation performance, when it is used to estimate story points for new agile tasks. Our preliminary results show that our SBSE technique improves the estimation performance of the LLM by 59.34% on average (in terms of mean absolute error of the estimation) over three datasets against a zero-shot setting.

Vali Tawosi, Salwa Alamir, Xiaomo Liu
arXiv:2403.08430 · cs.SE, cs.AI · submitted Mar 13, 2024
abstract · pdf · html · 6 pages, Accepted at SSBSE'23 NIER Track

add comment on HN

Improvement to story point estimation using LLMs. We're not too far from a world where an AI, starting with a simple ticket, can estimate how difficult that ticket is and then immediately solve and PR-it.