 |
ML Newsnew | past | best | rss | submit | about |
| Detecting Malicious Source Code in PyPI Packages with LLMs (arxiv.org) | |
| 2 points by abhisek on Apr 23, 2025 | hide | past | pdf | 1 comment on HN | |
|
In plain words: Tested whether feeding an AI code assistant extra security knowledge from rule books and past attacks helps it spot malicious Python packages. Showing a few known examples worked far better, reaching 97% accuracy, while the knowledge lookup only gave mediocre results. Abstract · Detecting Malicious Source Code in PyPI Packages with LLMs: Does RAG Come in Handy?Malicious software packages in open-source ecosystems, such as PyPI, pose growing security risks. Unlike traditional vulnerabilities, these packages are intentionally designed to deceive users, making detection challenging due to evolving attack methods and the lack of structured datasets. In this work, we empirically evaluate the effectiveness of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and few-shot learning for detecting malicious source code. We fine-tune LLMs on curated datasets and integrate YARA rules, GitHub Security Advisories, and malicious code snippets with the aim of enhancing classification accuracy. We came across a counterintuitive outcome: While RAG is expected to boost up the prediction performance, it fails in the performed evaluation, obtaining a mediocre accuracy. In contrast, few-shot learning is more effective as it significantly improves the detection of malicious code, achieving 97% accuracy and 95% balanced accuracy, outperforming traditional RAG approaches. Thus, future work should expand structured knowledge bases, refine retrieval models, and explore hybrid AI-driven cybersecurity solutions. | |
| add comment on HN |
Over 1,200 malicious Python packages were evaluated using three approaches:
– Zero-shot LLM prompts (no prior examples)
– Retrieval-Augmented Generation (RAG) with threat intelligence
– Fine-tuned LLMs on labeled malicious behavior patterns
RAG underperformed across all tests—even when enhanced with YARA rules, GitHub Advisories, and known malware code snippets. It failed to meaningfully improve detection in any setup.
In contrast, fine-tuning LLaMA-3.1-8B on behavior-based features (e.g., os.system, subprocess.Popen, eval) reached 97% accuracy and 95% balanced accuracy, outperforming both zero-shot and RAG methods.