Say it with me, now: chatgpt is not a doctor.
Now, louder for the morons in the back. Altman! Are you listening?!
this post was submitted on 23 Jun 2025
159 points (97.0% liked)
Technology
71885 readers
4336 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
ChatGPT is not a doctor. But models trained on imaging can actually be a very useful tool for them to utilize.
Even years ago, just before the AI “boom”, they were asking doctors for details on how they examine patient images and then training models on that. They found that the AI was “better” than doctors specifically because it followed the doctor’s advice 100% of the time; thereby eliminating any kind of bias from the doctor that might interfere with following their own training.
Of course, the splashy headline “AI better than doctors” was ridiculous. But it does show the benefit of having a neutral tool for doctors to utilize, especially when looking at images for people who are outside of the typical demographics that much medical training is based on. (As in mostly just white men. For example, everything they train doctors on regarding knee imagining comes from images of the knees of coal miners in the UK some decades ago)
LLMs are not Large Medical Expert Systems. They are Large Language Models, and are evaluated on how convincing their output is, instead of how accurate or useful it is.
Why are they... why are they having autocomplete recommend medical treatment? There are specialized AI algorithms that already exist for that purpose that do it far better (though still not well enough to even assist real doctors, much less replace them).
Because sycophants keep saying it's going to take these jobs, eventually real scientists/researchers have to come in and show why the sycophants are wrong.
Their analysis also revealed that these nonclinical variations in text, which mimic how people really communicate, are more likely to change a model’s treatment recommendations for female patients, resulting in a higher percentage of women who were erroneously advised not to seek medical care, according to human doctors.
This is not an argument for LLMs (which people are deferring to an alarming rate) but I’d call out that this seems to be a bias in humans giving medical care as well.
Of course it is, LLMs are inherently regurgitation machines - train on biased data, make biased predictions.
large language model deployed to make treatment recommendations
What kind of irrational lunatic would seriously attempt to invoke currently available Counterfeit Cognizance to obtain a "treatment recommendation" for anything...???
FFS.
Anyone who would seems a supreme candidate for a Darwin Award.
There's a potentially justifiable use case in training one and evaluating its performance for use in, idk, triaging a mass-casualty event. Similar to the 911 bot they announced the other day.
Also similar to the 911 bot, i expect it's already being used to justify cuts in necessary staffing so it's going to be required in every ER to ~~maintain higher profit margins~~ just keep the lights on.