"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
An AI that lacks intelligence is only ever going to do what mathematics does. If predictive mathematics were accurate, we'd literally be able to see the future and we wouldn't call it "predictive".
"Hallucinations" has always been an inaccurate term, but I think it was picked to imply intelligence was there when it never was.
The precise, scientific term is "bullshitting".
The best use case for this is on social media to use it to manipulate public opinion. That's why all social media companies are heavily invested in it.
Thus, bullshit, in contrast to mere nonsense, is something that implies but does not contain adequate meaning or truth.
We argue that an important adjutant of pseudo-profound bullshit is vagueness which, combined with a generally charitable attitude toward ambiguity, may be exacerbated by the nature of recent media.
The concern for “profundity” reveals an important defining characteristic of bullshit (in general): that it attempts to impress rather than to inform; to be engaging rather than instructive.
I think even bullshitting isn't a good term for it because to me it implies intent.
It's just a text predictor that can predict text well enough to be conversational and trick people interacting with it enough to pass the Turing test (which IMO was never really a good test of intelligence, though maybe shines a spotlight on how poorly "intelligence" is defined in that context, because despite not being a good test, it might still be one of the best I've heard of).
All of its "knowledge" is in the form of probabilities that various words go together, given what words preceded them. It has no sense of true, false, or paradox.
Misinformation farms ADORE it
Predictive mathematics is highly accurate and quite useful at predicting the future already for many types of problems.
As one example: we can use math models to predict where the planets in the solar system will be.
The problem with LLM hallucinations is not a general limitation of mathematics or linear algebra.
The problem is that the LLMs fall into bullshit, in the sense of On Bullshit. The deal is that both truthtellers and liars care about what the real truth is, but bullshit ters simply don't care at all whether they're telling the truth. The LLMs end up spouting bullshit, because bullshit is designed to be a pretty good solution to the natural language problem; and there's already a good amount of bullshit in the LLM training data.
LLM proponents believed that if you put enough compute power at the problem of predicting the next token, then the model will be forced to learn logic and math and everything else to keep optimizing that next token. The existence of bullshit in natural language prevents this from happening, because the bullshit maximizes the objective function at least as well as real content.
LLM takes this idea of Bullshit and takes it even further. The model has no concept of truth or facts. It can only pick the most likely word to follow the sequence it has.
A perfect illustration of this for me personally was when I tried early on in the LLM hype cycle (in like 2023? maybe?) playing around with an autocomplete example that said something like "Paris is the capital of France" with a high degree of confidence (which seems impressive until you mess with it) and changing the wording slightly to be a different city....still a high degree of confidence.
I tried telling chatgpt that Versailles was he capital of france to test how it reacts. It corrected me, but what got me was this ending:
For a time [...], Versailles was the center of political power in France, but Paris has always remained the official capital.
Let me know if you’re referring to a specific historical period — that might change the context a bit.
Of course, "things might be different in a different period" is a perfectly normal and reasonable thing to say when talking about history, so I imagine it might be common too. If you asked me about the capital of Germany, I'd ask about the period first because that very much changes the answer from "wherever the King happens to be at the moment" to "what Germany?", "Frankfurt, kinda", "which part?", "Berlin", "which part?" and back to "Berlin".
I imagine that's why ChatGPT would add that note: it's a thing historians are likely to say when asked a question where the answer depends on the exact period. But regardless of whether it is true or not, saying "always" followed by "might change" is a wonderful demonstration that it has no ducking clue why they would say that. If Paris always remained the capital, changing the context won't fucking change the truth.
Hallucinations are investor / booster speak for errors.
Yeah it's a pretty good hand wavey term for a real issue
It's a weird case. As the paper says, this is inherent to LLMs. They have no concept of true and false, and rather produce probabilistic word streams. So is producing an untrue statement an error? Not really. Given these inputs (training data, model parameters and quiet), it's correct. But it's also definitely not a "hallucination", that's a disingenuous bogus term.
The problem however is that we pretend these probabilistic language approaches are somehow a general fit for the programs they're put in place to solve.
If the system (regardless of the underlying architecture and technical components) is intended to produce a correct result, and instead produces something that is absurdly incorrect, that is an error.
Our knowledge about how the system works or its inherent design flaws does nothing to alter that basic definition in my opinion.
but I think it was picked to imply intelligence was there when it never was.
Bingo bango. This is why the humanizing words used with these algorithms are so insidious. They have all been adopted and promoted to subtly suggest and enforce the idea that there is intelligence, or even humanity, where there is none. It's what sells the hype and inflates the bubble.
To be fair, some AI is just people in a data center manually generating responses.
Yep. Its one of those "um actually" things that at surface can make you seem annoying, but unfortunately the nuance is really important.
In order to hallucinate it'd need to be capable of proper thought first.
In the same way people ask of their software "why doesn't it just work?!" Well... It actually DOES work. Its doing exactly as it's been programmed to do.
Whether the issue is because the dev didnt think of an angle you use it on, the QA didnt test it enough, or you yourself have a weird expectation, etc, it is doing exactly what it is only capable of doing in the situation that you see as "it isnt working right".
Its then on you, the human, to recognize that and proceed.
This dissonance even happens from human to human conversation. "Oh i thought you meant this."
If you go to an agriculturist and start asking them about the culture of another country, they'd probably stop you to point out the issue. They could also just start giving you agriculture info and leave you confused. The nuance is important and what lets our biological brain computer figure it out where the metal brain needs to be specifically told to make sure they meant land agriculture.
ah yes, psychohistory. math predicting the future is the initial idea that gets asimov's "foundation" rolling.
I was trying to debug a programming issue yesterday and resorted to Google since I couldn't find a solution on DDG. The AI summary garbage literally just made up a bunch of details about the software I was working with that had no bearing on reality whatsoever.
What I love is when the AI just literally says the same shit as the top returned result. Wow! Free plagiarism! Just what I need clogging up my search results!
The AI summarizes the AI blog posts from the result and everything just turns in to total unusable slop in the end
I only read official documentation and man pages these days
The top result is probably plagiarism because good writers don't have time for SEO. So you got plagiarism with one less click. Progress.
Yep, had the same experience trying to troubleshoot something in AutoCAD, complete with hallucinated source links that 404 on the autodesk site.
DDG uses bing. Or did. So that explains why it’s gone to shit. Most of my web searches start at ddg. And then almost instantly followed by google cause of the shit results.
But I found nothing on Google either, except hallucinated bullshit that wasted my time
I believe it. They are both terrible now. I find myself hitting pages 5 - 10 on Google way more than I ever have.
It appeared in my case, I couldn't find anything except the hallucination because my problem wasn't a solved one. Better to think "I can't do it this way" than to find a fake AI solution and spend thirty minutes trying it out, only to realize it's not a real thing.
I use chat it's a lot more. They seem to give better results then the AI on search and they are definitely better then google. It is very odd maybe that is there plan. But even then you do get bull shit sometimes.
Took a look cause, as frustrating as it'd be, it'd still be a step in the right direction. But no, they're still adamant that it's just a "quirk".
Conclusions
We hope that the statistical lens in our paper clarifies the nature of hallucinations and pushes back on common misconceptions:
Claim: Hallucinations will be eliminated by improving accuracy because a 100% accurate model never hallucinates. Finding: Accuracy will never reach 100% because, regardless of model size, search and reasoning capabilities, some real-world questions are inherently unanswerable.
Claim: Hallucinations are inevitable. Finding: They are not, because language models can abstain when uncertain.
Claim: Avoiding hallucinations requires a degree of intelligence which is exclusively achievable with larger models. Finding: It can be easier for a small model to know its limits. For example, when asked to answer a Māori question, a small model which knows no Māori can simply say “I don’t know” whereas a model that knows some Māori has to determine its confidence. As discussed in the paper, being “calibrated” requires much less computation than being accurate.
Claim: Hallucinations are a mysterious glitch in modern language models. Finding: We understand the statistical mechanisms through which hallucinations arise and are rewarded in evaluations.
Claim: To measure hallucinations, we just need a good hallucination eval. Finding: Hallucination evals have been published. However, a good hallucination eval has little effect against hundreds of traditional accuracy-based evals that penalize humility and reward guessing. Instead, all of the primary eval metrics need to be reworked to reward expressions of uncertainty.
Infuriating.
Finding: Accuracy will never reach 100% because, regardless of model size, search and reasoning capabilities, some real-world questions are inherently unanswerable.
Translation: PEBKAC. You asked the wrong question.
Basically "You must be prompting it wrong!"
We don't have mathematical proof that this technology will find adequate results. You must have faith in the technology.
Got a link or a title I can google to find the full paper? I'd be really interested in reading it.
Bing is now completely useless because of them now. Like utterly useless. How did it come to this?
Google too very soon, they said they're planning to make AI search the default.
I <3 DuckDuckGo
When it was coming out I never imagined it would actually be better than Google.
I guess technically google is now worse than DDG
All the results are still made with a good dose of ai slop
noai.duckduckgo.com
You misunderstand me. Pretty much all of the stuff with high seo these days are generated
When the whole web is AI garbage what is the point of a search engine. Even noticing lots of YouTube videos especially the voice overs the voices are AI.
By design. Web search wasn't making line go up. Must enshittify. Must sell ads. Must steal data.
It was never a good search engine to begin with.
I feel like dead internet theory comes from the right's refusal to acknowledge the popularity of leftism online.
And here I thought it was because nearly all the content is focused onto a couple of social media sites populated by a bunch of bots (a lot of which actually spout right wing opinions)
Literally every reddit username nowadays is WordWord1234
Kinda sus, ngl
Step one: just lie or make up an answer
Step two: when it can't be de ied anymore just say its true and that you always knew it was true
Companies should be ranked on how much lies their C suite pushes out there and the worst offenders should just have the entire C suite jailed for a few years to set an example
This was posted on mastodon. The fact that we have to screenshot it, to bring it to lemmy...
"Make no mistake" Makes mistake Da fuk?
I can't say I remember when this person specifically got shit for this, no. Who is this?
I have never heard anyone claim that LLM's would not hallusinate. who would be that stupid, it is the basic principle how they work
