this post was submitted on 15 May 2025
86 points (84.1% liked)
Technology
71447 readers
2262 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If this really is lossless, it is incredible. I'm skeptical until I see it in action though.
Lossless is the big claim that nobody is fixating on because "AI" discussions only ever run one set of talking points.
I get how semantic understanding would trade performance for file size when doing compression. I don't get how you can deterministically use it to always get the exact same complete output from a partial input. I'd love to go over the full paper. And even then the maths would probably go way, way over my head.
So... crystal ball, I don't have access to the paper either. Think arithmetic coders as neural nets are function approximators. You send an initial token and the NN will start to generate deterministically, once you detect a divergence from the lossless ideal you send another token to put it on track again. Make it a sliding window so things don't become too computationally expensive. You architect the model not to be smart but to need little guidance following "external reasoning" so to speak.
The actual disadvantage of this kind of thing will be the model size, yes you might be able to transmit a book in a kilobyte (100x or more compression) but both encoder and decoder will need access to gigabytes of neural weights, and that's just for text. It's also not going to be computationalliy cheap, though probably cheaper than PAQ.
Arithmetic coding is one of my favorite algorithms. Any token predictor can be converted into an entropy encoder!