this post was submitted on 19 Jul 2025
461 points (96.6% liked)

Technology

73035 readers
3621 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
461
Study finds AI tools made open source software developers 19 percent slower (arstechnica.com)
submitted 3 days ago by Aatube@kbin.melroy.org to c/technology@lemmy.world
24 comments fedilink hide all child comments
 

Coders spent more time prompting and reviewing AI generations than they saved on coding. On the surface, METR's results seem to contradict other benchmarks and experiments that demonstrate increases in coding efficiency when AI tools are used. But those often also measure productivity in terms of total lines of code or the number of discrete tasks/code commits/pull requests completed, all of which can be poor proxies for actual coding efficiency. These factors lead the researchers to conclude that current AI coding tools may be particularly ill-suited to "settings with very high quality standards, or with many implicit requirements (e.g., relating to documentation, testing coverage, or linting/formatting) that take humans substantial time to learn." While those factors may not apply in "many realistic, economically relevant settings" involving simpler code bases, they could limit the impact of AI tools in this study and similar real-world situations.

you are viewing a single comment's thread
view the rest of the comments
[–] HubertManne@piefed.social 31 points 3 days ago (8 children)

This does not seem surprising to me:

"Overall, the developers in the study accepted less than 44 percent of the code generated by AI without modification. A majority of the developers reported needing to make changes to the code generated by their AI companion, and a total of 9 percent of the total task time in the "AI-assisted" portion of the study was taken up by this kind of review."

It sounds about right. The AI should be acting as an assistant. The big question to me is if the code that comes out 19% slower is at all of higher quality. Since the coder is doing more correction and review does it act a bit like a second set of eyes or a pho sort of collaboration. If so it could still be helpful. Granted my experience so far is that most of what it does can be done with plugins to an ide but like it is sorta handy to have it all set and going after an installation without having to find and start using the plugins. Im still worried about energy usage with these things but hoping that can be worked out and honestly im not sure if the energy usage for something integrated with an ide or such is as bad.

[–] Zachariah@lemmy.world 21 points 3 days ago (6 children)

pho

faux

[–] ThoGot@feddit.org 18 points 3 days ago (1 children)

Maybe they're making soup

[–] kautau@lemmy.world 7 points 3 days ago

You can tell it’s code soup by the smell

https://en.wikipedia.org/wiki/Code_smell

load more comments (4 replies)
load more comments (5 replies)