about
Spam-T5: Benchmarking Large Language Models for Few-Shot Email Spam Detection (arxiv.org)
2 points by PaulHoule on Apr 5, 2023 | hide | past | pdf | 1 comment on HN

In plain words: A text model called Spam-T5 was tuned to label emails as spam or not, and tested against other language models and classic spam filters on four public datasets. It beat the usual filters and other models in most cases, especially with few labeled examples.

Abstract

This paper investigates the effectiveness of large language models (LLMs) in email spam detection by comparing prominent models from three distinct families: BERT-like, Sentence Transformers, and Seq2Seq. Additionally, we examine well-established machine learning techniques for spam detection, such as Naïve Bayes and LightGBM, as baseline methods. We assess the performance of these models across four public datasets, utilizing different numbers of training samples (full training set and few-shot settings). Our findings reveal that, in the majority of cases, LLMs surpass the performance of the popular baseline techniques, particularly in few-shot scenarios. This adaptability renders LLMs uniquely suited to spam detection tasks, where labeled samples are limited in number and models require frequent updates. Additionally, we introduce Spam-T5, a Flan-T5 model that has been specifically adapted and fine-tuned for the purpose of detecting email spam. Our results demonstrate that Spam-T5 surpasses baseline models and other LLMs in the majority of scenarios, particularly when there are a limited number of training samples available. Our code is publicly available at https://github.com/jpmorganchase/emailspamdetection.

Maxime Labonne, Sean Moran
arXiv:2304.01238 · cs.CL, cs.AI · submitted Apr 3, 2023 · updated May 7, 2023
abstract · pdf · html

add comment on HN

A nice case study that compares transformer models to classical models for a typical text classification class, useful for the practitioner.