 |
ML Newsnew | past | best | rss | submit | about |
| DiVeRSe - On the Advance of Making Language Models Better Reasoners (arxiv.org) | |
| 2 points by lajamerr on Jun 8, 2022 | hide | past | pdf | 1 comment on HN | |
|
In plain words: The model solves a math problem several different ways, then a checker scores each reasoning step and votes on the best final answer. Scoring steps one at a time beat checking whole solutions, reaching 83.2% on grade-school math. Abstract · Making Large Language Models Better Reasoners with Step-Aware VerifierFew-shot learning is a challenging task that requires language models to generalize from limited examples. Large language models like GPT-3 and PaLM have made impressive progress in this area, but they still face difficulties in reasoning tasks such as GSM8K, a benchmark for arithmetic problems. To improve their reasoning skills, previous work has proposed to guide the language model with prompts that elicit a series of reasoning steps before giving the final answer, achieving a significant improvement on GSM8K from 17.9% to 58.1% in problem-solving rate. In this paper, we present DIVERSE (Diverse Verifier on Reasoning Step), a novel approach that further enhances the reasoning capability of language models. DIVERSE has three main components: first, it generates diverse prompts to explore different reasoning paths for the same question; second, it uses a verifier to filter out incorrect answers based on a weighted voting scheme; and third, it verifies each reasoning step individually instead of the whole chain. We evaluate DIVERSE on the latest language model code-davinci-002 and show that it achieves new state-of-the-art results on six of eight reasoning benchmarks (e.g., GSM8K 74.4% to 83.2%). | |
|
add comment on HN
Also discussed: Jun 2022 (2 points, 0 comments)
|
*Large language models such as GPT-3 and PaLM have shown remarkable performance in few-shot learning. However, they still struggle with reasoning tasks such as the arithmetic benchmark GSM8K. Recent advances deliberately guide the language model to generate a chain of reasoning steps before producing the final answer, successfully boosting the GSM8K benchmark from 17.9% to 58.1% in terms of problem solving rate. In this paper, we propose a new approach, DiVeRSe (Diverse Verifier on Reasoning Step), to further advance their reasoning capability. DiVeRSe first explores different prompts to enhance the diversity in reasoning paths. Second, DiVeRSe introduces a verifier to distinguish good answers from bad answers for a better weighted voting. Finally, DiVeRSe verifies the correctness of each single step rather than all the steps in a whole. We conduct extensive experiments using the latest language model code-davinci-002 and demonstrate that DiVeRSe can achieve new state-of-the-art performance on six out of eight reasoning benchmarks (e.g., GSM8K 74.4% to 83.2%), outperforming the PaLM model with 540B parameters.
Personal opinion: A lot of development lately for machine learning/language models is using existing big models that has had the resources by enterprises to train it with a prohibitively big dataset/computation requirement. Then others build upon those models and tailor it to the types of tasks they want solved. In a way this is democratization of Big AI because the subsequent training isn't as compute intensive as what it took to generate the original big models.