Ask Lemmy

TLDR: The result of current LLMs will be very bandlimited and one-directional.

I hope that means something to you, because otherwise I'm going to try to explain this very specific thing, and I'm afraid I might not be able to express it in very understandable terms (sorry):

Firstly, one-directionality: when a human wants to write a story, we usually think about the plot twist beforehand and then pave the way by hinting at the upcoming twist without giving too much away. It's just nice when a first time reader is surprised, but struggles on a second time how they missed all the obvious clues.

This process requires a lot of back-and-forth while writing. Humans do this naturally. LLMs and other transformer networks have a huge problem with this. I often hear LLMs referred to as text prediction machines. This is not entirely accurate, but a similar enough. And to keep with this analogy: text prediction doesn't really work backwards to suggest a better start to the sentence, does it? LLMs tend to take a path, from start to finish, even in great detail, but that's it. There's no setup. It's very flat writing.

Secondly, bandlimiting: Over time LLMs tend to mush different characterizations and continuity into a smooth paste, leaving little grit to it. I really struggle to not say the word derivative (like in math). But LLMs just write average characters who do average things in an average way. And then spell out how everything was totally unpredictable, important and meaningful, while using superficially eloquent language. Nothing just is everything serves as. It's a poor writing style that often misses the appropriate tone, trying to sound sophisticated.