about
Qwen OSS 7B,14B models; techreport - Code-Qwen, Math-Qwen [sep28'23] (arxiv.org)
1 point by saurabh20n on Nov 30, 2023 | hide | past | pdf | 1 comment on HN

In plain words: Qwen is a family of language models in several sizes, with chat versions tuned by human feedback and special versions for coding and math. Its base models beat open models on many tasks, while the coding and math versions trail paid ones only slightly.

Abstract · Qwen Technical Report

Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans. In this work, we introduce Qwen, the first installment of our large language model series. Qwen is a comprehensive language model series that encompasses distinct models with varying parameter counts. It includes Qwen, the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. The base language models consistently demonstrate superior performance across a multitude of downstream tasks, and the chat models, particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The chat models possess advanced tool-use and planning capabilities for creating agent applications, showcasing impressive performance even when compared to bigger models on complex tasks like utilizing a code interpreter. Furthermore, we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat, which are built upon base language models. These models demonstrate significantly improved performance in comparison with open-source models, and slightly fall behind the proprietary models.

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, et al.
arXiv:2309.16609 · cs.CL · submitted Sep 28, 2023
abstract · pdf · html · 59 pages, 5 figures

add comment on HN

Discussion of the 72B model happening here: https://news.ycombinator.com/item?id=38475501