In plain words: A language model trained on a huge mix of Bloomberg's financial text and general writing, so it can handle finance jobs like judging sentiment and answering questions. It beat other models on finance tasks without losing accuracy on general ones.
Abstract
The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. We release Training Chronicles (Appendix C) detailing our experience in training BloombergGPT.
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, Gideon Mann
arXiv:2303.17564 · cs.LG, cs.AI, cs.CL, q-fin.GN · submitted Mar 30, 2023 · updated Dec 21, 2023
abstract · pdf · html · Updated to include Training Chronicles (Appendix C)
I use ChatGPT to keep track of tasks and Todo lists. It works phenomenally well for me, and the natural language back-and-forth helps keep me motivated. I give it a set of tasks, with time estimates, and it organizes these tasks for me, and I tell it when I complete them, and it updates my task list.
The one funny mistake it makes is that when it groups my tasks (say I have 3 "Work" tasks and 2 "Personal" tasks") it sums up the total estimated time for each task group, but the totals are often wrong, especially when I start adding new tasks or completing tasks.
When so much of finance requires numeric accuracy, I'm curious how BloombergGPT handles numbers.