about
Pre-Trained Security LLM 8B (arxiv.org)
6 points by YoOnoAP on May 1, 2025 | hide | past | pdf | discuss on HN

In plain words: A general-purpose language model was further trained on a hand-picked collection of cybersecurity text so it understands security topics better. On cybersecurity tests it matched a much larger general model and a leading small commercial one, despite being far smaller.

Abstract · Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report

As transformer-based large language models (LLMs) increasingly permeate society, they have revolutionized domains such as software engineering, creative writing, and digital arts. However, their adoption in cybersecurity remains limited due to challenges like scarcity of specialized training data and complexity of representing cybersecurity-specific knowledge. To address these gaps, we present Foundation-Sec-8B, a cybersecurity-focused LLM built on the Llama 3.1 architecture and enhanced through continued pretraining on a carefully curated cybersecurity corpus. We evaluate Foundation-Sec-8B across both established and new cybersecurity benchmarks, showing that it matches Llama 3.1-70B and GPT-4o-mini in certain cybersecurity-specific tasks. By releasing our model to the public, we aim to accelerate progress and adoption of AI-driven tools in both public and private cybersecurity contexts.

Paul Kassianik, Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Aufiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, Aman Priyanshu, Adam Swanda, et al.
arXiv:2504.21039 · cs.CR, cs.AI · submitted Apr 28, 2025
abstract · pdf · html

add comment on HN