weights available here: https://huggingface.co/chaoyi-wu/PMC_LLAMA_7B/

Brief tweetalysis: https://twitter.com/omarsar0/status/1652840470878232576?s=20

It said to be trained on 75 Billion tokens with 8 A100 in 7 days.

However, in the paper they say that for each of the 5 epochs, they sample 512 tokens from each paper randomly.

This would come down to an effective training size of 12 billion tokens.

Also, if I am not reading this wrong, the paper shows that when Fine-tuning on a task, this additional Medical training (7 days on A100) essentially does nothing (benefit of maybe 1% over standard LLaMA).

Zero shot comparisons are not shown.

This looks like a failed experiment to me?

With 75B tokens, would it be better to train from scratch, then apply instruction fine-tuning? Training from llama seems like it could introduce a lot of unexpected behavior.

75B tokens is not really enough data to make an intelligent model. Llama was trained on over 1 trillion tokens.

And yes, training on top of LLaMA could introduce a lot of unexpected behavior, but that's just where the State-of-the-Art is today

can someone explain the PEFT vs Zero-shot results to me? (in the table) How can Llama go from 1.8 to around 68?

AIUI they (1) finetuned LLaMA on 75B tokens of medical papers, and called the resulting model PMC-LLaMA, (2) finetuned that model on task-specifc data, then (3) separately finetuned the base model they started with in (1) on the same task specific data for comparison. So, they're reporting 6 separate conditions: {LLaMA, PMC-LLaMA} x {Zero-shot, Finetune, PEFT}.

Could definitely use some more exposition around that though.

Would you trust a LLM giving medical advice in the wild?

I would if it's shown to be better than doctors at their own game.

Wouldn't you?

If you asked me in 1995 would I bet my money on World Chess Champion or a computer? ask that again 4 years later.

At some point I will trust cars to drive us more than us to drive cars.

These "gotcha" statements aren't helpful.

Chess is a game of perfect information. Healthcare outsiders really don't understand that information retrieval is _the_ big issue in medical automation, and that reasoning on said data is the smaller problem. See, docs have one major advantage: they walk around the corridors. Doing so, they acquire a lot of info that is as of now out of reach for computers. Until that's changed, we won't see chatGPT for medicine.

Sure, but that wasn't the question. The question was an "ever" one with no restrictions. So... I answered it.

In software development sphere, there is a review process for all code that is written. No code gets into production without an explicit approval of some peer or supervisor.

When you get diagnosed by a doctor, usually no one checks the diagnosis. I would love if my health was checked by medical professional committee. Human and/or AI.

That's just not correct on both sides. Code reviews happen sometimes in some environments - it's not a given. On the other side, doctors do run their questions by other doctors if they're not sure about the results. (Again, not enforced) Similar for pathology tests results, but with the added layer of the tests themselves being reviewed too.

You'd have to join the social network groups for doctors to see that, but there's a lot of double checking happening both with local peers and wider groups. For the more interesting cases anyway... you're not getting a second review if you come in with runny nose and test positive for RSV. (Edit: and even in more complex cases you may not get it for a lot of reasons)

My last two ENTs did not run their answers by anyone. The first failed to correctly record the details of my issue, to the point of writing a summary that said exactly the opposite. The second failed at applying high-school level logic and came to a conclusion that is logically impossible (I tried to walk him through the failure in his reasoning several times before giving up and seeking another opinion). So there may well be a lot of double checking, but there are also a lot of cases of doctors being confidently wrong and not having their reasoning tested.

(But, yeah, we have plenty of software developers confidently checking in untested, unreviewed code too)

When there is a review process in place, it is usually followed. Mistakes should be expected in the tiniest commits and there should be processes that catch them. As you suggested, medical community does this ad-hoc and only for interesting/unsure cases and this is the result:

> The researchers estimate that 7.4 million misdiagnosis errors are made every year, 2.6 million people receive a harm that could have been prevented, and another 370,000 are permanently disabled or die because of the misdiagnosis.

Source: https://edition.cnn.com/2022/12/15/health/hospital-misdiagno...

While that's true, we have to keep in mind: 1. We don't know yet what the misdiagnosis rate would be with a review. (There are reasons it could go up) 2. At least in the area I'm familiar with, on many days there just wouldn't be enough staff to do it - people are already sometimes waiting for hours. 3. Issues raised in the article.

Why do you think that the ratio of misdiagnosis would go up if there was a dialogue or a review process?

2. and 3. are interlinked. In my mind, medicine is the number one application for ML. Every doctor should have the best AI at their finger tips. One person cannot digest all the research that is being done; computer can. The benchmarks will go up.

Potential issues from reviews:

First person thinks it's X, second person doesn't, first person changes mind but it really was X.

First person thinks it's X, second person finds potential issue Y which becomes more important, but X was actually the biggest issue.

> if they're not sure

The thing about mistakes is that you often don't expect you're making one. To my mind there is still value in having a built-in second opinion.

The amount of times I've heard doctors tell female friends that their issue is "hormonal" is frankly offensive. Perhaps if we can create an AI (assuming it also doesn't say women have the modern equivalent of "hysterics" for every problem) they will finally get a real answer.

Don't get me wrong, the system is not perfect and I'm sure it failed many people. It's just the comparison as stated in the original comment that's made up.

Genuine question: how do they not violate HIPAA? Are they just crazy careful to abstract and anonymize any identifiable details?

I'm not based in the US so maybe someone else knows the details there. But HIPAA qualifies the privacy requirements with "and that identifies the individual or for which there is a reasonable basis to believe it can be used to identify the individual." - normally the questions I see are extremely generic, with the sex/age being mentioned at most.

At some point it becomes a serious tradeoff. It will definitely excel in finding niche solutions, in bringing up that one mechanism that's rarely used but very relevant in some case. It cannot say "I don't know" and will start babbeling instead, but if you know the downsides the upsides can be significant

If I was given the choice of no medical help at all versus take the risk and ask AI, well, I guess I would take the chance?

Given my last encounter with an ENT specialist that failed to apply basic logic, I'd be tempted to ask in response why you would trust a doctor giving medical advice in the wild.

And the answer is the same: Thorough testing and vetting and auditing out any adverse outcomes is what makes or would make either safe enough. Once an LLM is thoroughly tested and shown to perform at least as well as a real doctor, without any worse worst-case outcomes, then sure.

So, not yet.

It is a nice second opinion source, for sure.

women and minorites receive notoriously dismissive diagnosis from human physicians

which means over 75% of the population would chose this immediately

and of the people that are content with their medical attention, a good portion of them would also choose this

tl;dr yes

*Some amount of women and minorities attribute dismissive doctors to their sex or race.

I’m a straight white man and I’ve had plenty of dismissive doctors over the years too. It turns out arrogance, laziness, or a lack of time have little to do with the patient themselves.

This is a hilarious example of how you don't get it. People in your position think because they have experienced a mild version of something or experienced something once they now know exactly how any other minority experiences the world.

This doesn't make you a bad person btw, just means you are not aware of how small your social context is.

Mild?

When I was 17 I went in to the doctor because of extreme fatigue. He just slapped me on some antidepressants and called it a day. Years later I happened to read about low testosterone symptoms, which matched me very well. He eventually acquiesced to letting me test, and it came back 2 points above the low end of normal - which probably by itself qualified as low for my age. He wouldn't let me do a quick trial of testosterone supplementation nor test again, but I begged him to refer me to an endocrinologist.

The endocrinologist (2 hours away) was worse. 100% dismissive, straight up told me I didn't have low testosterone. He did however let me test again, and this time I was 150 below normal. I asked to be put on medications that would maintain my fertility. He told me I didn't need to worry about losing my fertility on straight up testosterone, which is 100% false.

I'm going to cut the story off there, but it keeps going. My dad almost died because doctors didn't diagnose his lupus. My sister's friend lost her father when we were young because of an infection after some dental work where they dentist completely dismissed his pain after the fact.

Everyone is unaware of how "small their social context is," which is exactly my point. If I were a minority I very well might be blaming my story on that.

You are completely missing the multidimensional aspect of this. I really didn't want to anger you, and that's why I chose to write my comment carefully. Apparently not careful enough. But imagine this dismissal, but in every part of your life. Your dad got misdiagnosed for lupus? They don't even believe that we have can a headache. Just chill man and accept that some people can have more problems than you and some communities have more problems than straight white males. If the only thing that will stop you from dismissing this is hard core data, i think you should recalibrate your empathy circuits.

As a straight white man, on one hand I've also seen this. On the other hand, having had a black female partner and seen first hand the dismissive attitudes she often had to deal with from doctors until she started pointing out to them every chance she got that she's a lawyer: What you and I deal with is nothing like what they women and minorities deal with from doctors. Yes, we could benefit from better doctors too, but the difference is stark.

I think its fine as written because we no longer have to do any empirical analysis anymore just to accommodate your view of the outcome, LLMs are here

This comment is so painfully predictable I don't even need a fancy transformer model to predict it. A bash script would be good enough

Let me make it more plain.

Somebody you love is in an life threatening condition. You need advice what to do or you lose them.

You have a one shot option to get advice from a human expert or a LLM.

Which one you pick.

This is a stupid hypothetical; only in an extreme emergency where your loved one only had a few seconds to live would you not have time to consult both a human expert and an LLM.

You are evading the question because it does not suit your narrative.

Countless ink has been spent on discussing the potential risks and dilemmas of automated decision support, e.g., the autopilot situation having to choose whom to save.

It is not "stupid" to isolate the essence of the challenges we are facing and discuss them without hidden agendas.

Incidentally such discussions also help cast much needed light on tech bro culture and morality.

This is a trolley type question. It's a fun philosophy game about morality. But nobody ever needs to answer either question with the stated limits. In both cases it's answer C: apply breaks on the trolley and ask both an expert and AI about the issue and discuss AI's response with the expert if you need to.

LLM on medical forum posts was more empathic and precise than the physicians. Thats the only data we have so far.

I’ll pick the LLM. The humans are too random. I’ll take my chances with “hallucination” from something that’s also read every research paper in every language. Its just like Flowers for Algernon but even better.

Why not both. Doctors can sieve hallucinations and random BS and cross check with peers anyway, and correct the diagnosis description. This can save a lot of typical paperwork and increase healthcare availability by decreasing bureaucratic work. It's not about diagnosis but the work around it. To be clear, I just mean the autocomplete for diagnosis description entry, not the diagnosis itself.

Good luck and good night.

What exactly are you trying to reason about with this hypothetical? Do you only buy processors which can be sent to space?

LLMs are exposed to exactly the same information that doctors are. If the system starts with llama, it will get all the biases that exist on the internet as well. See the recent examples of chatgpt refusing to accept a judge can be a woman. Or the "come up with punishment given variables race and sex" questions.

There's no good reason yet to expect that the 75% would receive better diagnosis from from the AI.

We need a source for that "75% would choose this immediately".

Our group, sure! Our generation, and people in our shoes, and with our level of knowledge about LLMs. But maybe not others.

tl;dr Not yet

Consider three things:

1. What people prefer when it comes to intimate/private issues that a physician diagnoses.

2. Tainted source datasets, resulting in the same "dismissive diagnosis" from the LLM

3. Lack of Trust held within a new system

1. For starters, there are certainly a large subset of people that simply "don't trust machines". Alternatively, they "don't trust major corporations". They would likely pick local doctors over machines, due to a perceived lack of connection/humanity in the LLM. Combine that with the recent increased use of LLM / AI models used to make scam calls, and you have a potentially deep seated (and at times illogical) distrust of any LLM diagnosis.

Some would say the machine doesn't and cannot understand their unique situation. And refuses to give them what they need. So they would go to a doctor. Some would have difficulty being honest with a machine. Would you trust a human or a machine to keep your medical records safe? (I honestly don't know which I would trust. Both at the same time preferably?)

2. Others would say the dataset is made up of biased diagnoses specifically tailored towards the majority, and fail to diagnose the minority correctly. You are assuming that the LLM cannot be tainted by biased or dismissive diagnosis made by doctors against the very same women and minorities. The datasets have to come from somewhere, and some of that data will without a doubt be biased.

3. There is a lack of trust. Automated systems are making great strides yes. But absolutely no one trusts them fully. Whether it be planes, cars, logistics, data, etc Absolutely no one trusts them fully. Especially with GPT like systems that simply cannot say "No" and start making stuff up.

Remember the fiasco with Bard gaslighting someone about the current year? Imagine if that happens during an LLM diagnosis.

Come full circle, those same people who don't trust a human physician due to discrimination? "Why the hell would I trust an AI machine made specifically to make money!". What is there that actually makes a corporate AI more trust worthy than the local physician? What is there that actually prevents the LLM from discriminating?

This is really my main concern: Nothing stops the LLM from discriminating. All it takes, is a dataset with built in biases.

It isn't nearly as clear cut as you make it out to be.

If you have a hammer, everything looks like a nail. Medicine isn't a good nail to hammer with a LLM, though. I struggle to see how a stochastic token completion system, prone to hallucination, let alone all the attack vectors described over the last months and years, would do any good in the medical domain.

A medical paper isn't just a digestible blog post or conversational op-ed, it's full of technical jargon and statistics that even require expert readers to do some unpacking in order to evaluate how the presented findings fit into a larger picture. Indeed, it's a skill and often a form of art to be able to read and understand medical papers, even from within one's field of expertise. Aside from that, medical knowledge is never truly established, it is always in flux, both in respect to ongoing research and in the skill that derives from decade long practice of negotiating this with often more complex and nuanced real-world situations, i.e. with the patients that come through the doctor's office doors.

Doctors already face(d) the toll of diagnosis within a Google search's reach: misinformed, anxious and often outright overconfident patients often stand in their own way. That is not to say, that medical literacy and knowledge for the masses would be a detrimental thing, quite the contrary is true. However, with the high stakes faced in medicine, any misinformation, be it by Google or LLM can lead to grave harms.

I wonder would it be better off using sources like the Cochrane library? Smaller data set, meta-analysis.

While doctors face google, patients face poor doctors. My father is 96, so lots of things have inevitably started to go wrong. Knees, heart, bladder, prostate, blood pressure. His doctor is absolutely shit. He started feeling light-headed a few months back and the doctor confidently pronounced that he's just old. My mother intervened. Turns out the handfuls of pills he's been prescribing my father included blood pressure pills, and if he ha bothered taking routine blood pressure readings the doctor would have realized that my father's blood pressure was dangerously low. He quit those pills and bounced back. His doctor has been useless, lazy and shamelessly apathetic for 10 years. Terrible doctors are everywhere. My wife's doctor told her that her extremely heavy periods just happen to some women. Oh, and that her chronic anemia and B12 deficiency were normal in some women, so he'd just keep giving her those shots. She demanded a scan, which revealed a fibroid but the doctor told her: "oh, you can just go on the pill!" It took another doctor to schedule a procedure to remove the fibroid. She's been fine for a decade since. But has met many women online who went through hell with untreated fibroids. One woman said that it ruined her life because her anemia was disabling, her heavy periods left her housebound, and when the fibroid finally reached the size of a grapefruit she had to have a hysterectomy. Up to that point her doctor called her a hypochondriac.

Oh, and when my sister told her doctor that she had "new" pains 2 months after giving birth, he told her it was normal, take pain killers and go home. It took several attempts before she convinced him to schedule a scan. Turns out it was terminal cancer! I don't even know that catching it a few weeks earlier would have changed anything, but still. Err on the side of caution.

The thing about a decent LLM doctor, if one were to exist, is that it won't get bored, apathetic, depressed, tired, hateful etc.

> The thing about a decent LLM doctor, if one were to exist, is that it won't get bored, apathetic, depressed, tired, hateful etc.

It will, however, have had the standard biases RLHF'd into it.

It's too late, it's out there now.

It's just a matter of time before we see a HN Launch for a startup that's going to disrupt the medical industry with LLM.

Why not? People said the same thing about programming and LLMs are usually quite helpful for a lot of tasks.