In plain words: It teaches a chat model to think step by step, explore next steps like a game tree, and double-check its answers, so it can handle problems with no single correct answer. Unlike training that needs clear right answers and easy-to-score rewards, it targets real-world questions.
Abstract · Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions
Currently OpenAI o1 sparks a surge of interest in the study of large reasoning models (LRM). Building on this momentum, Marco-o1 not only focuses on disciplines with standard answers, such as mathematics, physics, and coding -- which are well-suited for reinforcement learning (RL) -- but also places greater emphasis on open-ended resolutions. We aim to address the question: ''Can the o1 model effectively generalize to broader domains where clear standards are absent and rewards are challenging to quantify?'' Marco-o1 is powered by Chain-of-Thought (CoT) fine-tuning, Monte Carlo Tree Search (MCTS), reflection mechanisms, and innovative reasoning strategies -- optimized for complex real-world problem-solving tasks.
Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, Kaifu Zhang
arXiv:2411.14405 · cs.CL · submitted Nov 21, 2024 · updated Nov 25, 2024
abstract · pdf · html