Programming

26625 readers

174 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 2 years ago

MODERATORS

snowe@programming.dev

Ategon@programming.dev

UlrikHD@programming.dev

bugsmith@programming.dev

Spyro@programming.dev

Anthropic Mythos shaping up as nothingburger (www.theregister.com)

submitted 1 day ago by HaraldvonBlauzahn@feddit.org to c/programming@programming.dev

15 comments fedilink hide all child comments

[...]

That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers.

"So far we've found no category or complexity of vulnerability that humans can find that this model can't," Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: "We also haven't seen any bugs that couldn't have been found by an elite human researcher." In other words, it's like adding an automated security researcher to your team. Not a zero-day machine that's too dangerous for the world.

you are viewing a single comment's thread
view the rest of the comments

[–] Quetzalcutlass@lemmy.world 48 points 1 day ago (3 children)

As much as I hate everything about the rise of LLMs, saying this isn't impressive because it can be matched by "an elite security researcher" isn't very reassuring to me. It's still an agent being pointed at a codebase and finding hundreds of vulnerabilities. Even if only a twentieth turn out to be exploitable in practice, that's still a terrifying tool to imagine in the hands of hackers who might otherwise lack the skills to find these vulnerabilities.

Most hacking groups buy exploits off of dark markets and indiscriminately target servers until they find one that's vulnerable. The number that can actually develop those hacks is far smaller, but if you can simply ask an LLM to find a vulnerability then that bar is lifted. Hell, you could probably coerce it into writing the actual exploit too by claiming you need a proof-of-concept for a CVE writeup.

[–] theunknownmuncher@lemmy.world 21 points 1 day ago* (last edited 1 day ago) (1 children)

Most all of the reporting about this is purely misinformation. If you actually read the papers that Anthropic published instead of the marketing material, you'll find that:

it was actually claude opus that discovered many of the vulnerabilities, not mythos, which undermines the "MyThOs Is ToO dAnGeRoUs" narrative. All of these capabilities are already out there for anyone to use
the researchers guided mythos to the vulnerabilities, not the other way around

[–] Quicky@piefed.social 5 points 1 day ago

That’s actually mentioned in this article tbf.

Additionally, the "'thousands of severe vulnerabilities' extrapolates from 198 manually reviewed reports. The Linux kernel bug was found by Opus 4.6, the public model, not Mythos," Devansh said.

[–] Grandwolf319@sh.itjust.works 2 points 1 day ago

I’m so proud of lemmy for fully calling our nuance cases and not letting our bias get the best of us.

[–] CultLeader4Hire@lemmy.world 1 points 1 day ago

I agree, and is it even true if “elite security researchers” didn’t actually find these problems? They didn’t find them because they weren’t looking for them is the obvious answer but it’s still a glaring inconsistency