I mean.. At best it's a stack overflow/google replacement.
Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
There's some real perks to using AI to code - it helps a ton with templatable or repetitive code, and setting up tedious tasks. I hate doing that stuff by hand so being able to pass it off to copilot is great. But we already had tools that gave us 90% of the functionality copilot adds there, so it's not super novel, and I've never had it handle anything properly complicated at all successfully (asking GPT-5 to do your dynamic SQL calls is inviting disaster, for example. Requires hours of reworking just to get close.)
But we already had tools that gave us 90%
More reliable ones.
Deterministic ones
Fair, I've used it recently to translate a translations.ts file to Spanish.
But for repetitive code, I feel like it is kind of a slow down sometimes. I should have refactored instead.
Some code is boilerplate and can’t be distilled down more. It’s nice to point an AI to a database schema and say “write the Django models, admin, forms, and api for this schema, using these authentication permissions”. Yeah I’ll have to verify it’s done right, but that gets a lot of the boring typing out of the way.
This is a thing people miss. "Oh it can generate repetitive code."
OK, now who's going to maintain those thousands of lines of repetitive unit tests, let alone check them for correctness? Certainly not the developer who was too lazy to write their own tests and to think about how to refactor or abstract things to avoid the repetition.
If someone's response to a repetitive task is copy-pasting poorly-written code over and over we call them a bad engineer. If they use an AI to do the copy-paste for them that's supposed to be better somehow?
I found that it only does well if the task is already well covered by the usual sources. Ask for anything novel and it shits the bed.
That's because it doesn't understand anything and is just vomiting forth output based on the code that was fed into it.
At absolute best.
My experience is it's the bottom stack overflow answers. Making up bullshit and nonexistent commands, etc.
“No Duh,” say senior developers everywhere.
I'm so glad this was your first line in the post
No duh, says a layman who never wrote code in his life.
Oddly enough, my grasp of coding is probably the same as the guy in the middle but I still know that LLM generated code is garbage.
Yeah, I actually considered putting the same text on all 3, but we gotta put the idiots that think it's great somewhere! Maybe I should have put it with the dumbest guy instead.
The most immediately understandable example I heard of this was from a senior developer who pointed out that LLM generated code will build a different code block every time it has to do the same thing. So if that function fails, you have to look at multiple incarnations of the same function, rather than saying “oh, let’s fix that function in the library we built.”
Yeah, code bloat with LLMs is fucking monstrous. If you use them, get used to immediately scouring your code for duplications.
AI coding is the stupidest thing I've seen since someone decided it was a good idea to measure the code by the amount of lines written.
More code is better, obviously! Why else would a website to see a restaurant menu be 80Mb? It's all that good, excellent code.
Imagine if we did "vibe city infrastructure". Just throw up a fucking suspension bridge and we'll hire some temps to come in later to find the bad welds and missing cables.
Almost like its a desperate bid to blow another stock/asset bubble to keep 'the economy' going, from C suite, who all knew the housing bubble was going to pop when this all started, and now is.
Funniest thing in the world to me is high and mid level execs and managers who believe their own internal and external marketing.
The smarter people in the room realize their propoganda is in fact propogands, and are rolling their eyes internally that their henchmen are so stupid as to be true believers.
Might be there someday, but right now it’s basically a substitute for me googling some shit.
If I let it go ham, and code everything, it mutates into insanity in a very short period of time.
I'm honestly doubting it will get there someday, at least with the current use of LLMs. There just isn't true comprehension in them, no space for consideration in any novel dimension. If it takes incredible resources for companies to achieve sometimes-kinda-not-dogshit, I think we might need a new paradigm.
A crazy number of devs weren't even using EXISTING code assistant tooling.
Enterprise grade IDEs already had tons of tooling to generate classes and perform refactoring in a sane and algorithmic way. In a way that was deterministic.
So many use cases people have tried to sell me on (boilerplate handling) and im like "you have that now and don't even use it!".
I think there is probably a way to use llms to try and extract intention and then call real dependable tools to actually perform the actions. This cult of purity where the llm must actually be generating the tokens themselves... why?
I'm all for coding tools. I love them. They have to actually work though. Paradigm is completely wrong right now. I don't need it to "appear" good, i need it to BE good.
Glad someone paid a bunch of worthless McKinsey consultants what I could’ve told you myself
It is not worthless. My understanding is that management only trusts sources that are expensive.
it's slowing you down. The solution to that is to use it in even more places!
Wtf was up with that conclusion?
It remains to be seen whether the advent of “agentic AIs,” designed to autonomously execute a series of tasks, will change the situation.
“Agentic AI is already reshaping the enterprise, and only those that move decisively — redesigning their architecture, teams, and ways of working — will unlock its full value,” the report reads.
"Devs are slower with and don't trust LLM based tools. Surely, letting these tools off the leash will somehow manifest their value instead of exacerbating their problems."
Absolute madness.
I have been vibe coding a whole game in JavaScript to try it out. So far I have gotten a pretty ok game out of it. It's just a simple match three bubble pop type of thing so nothing crazy but I made a design and I am trying to implement it using mostly vibe coding.
That being said the code is awful. So many bad choices and spaghetti code. It also took longer than if I had written it myself.
So now I have a game that's kind of hard to modify haha. I may try to setup some unit tests and have it refactor using those.
Senior Management in much of Corporate America is like a kind of modern Nobility in which looking and sounding the part is more important than strong competence in the field. It's why buzzwords catch like wildfire.
I code with LLMs every day as a senior developer but agents are mostly a big lie. LLMs are great for information index and rubber duck chats which already is incredible feaute of the century but agents are fundamentally bad. Even for Python they are intern-level bad. I was just trying the new Claude and instead of using Python's pathlib.Path it reinvented its own file system path utils and pathlib is not even some new Python feature - it has been de facto way to manage paths for at least 3 years now.
That being said when prompted in great detail with exact instructions agents can be useful but thats not what being sold here.
After so many iterations it seems like agents need a fundamental breakthrough in AI tech is still needed as diminishing returns is going hard now.
I miss the days when machine learning was fun. Poking together useless RNN models with a small dataset to make a digital Trump that talked about banging his daughter, end endless nipples flowing into America. Exploring the latent space between concepts.
This article sums up a Stanford study of AI and developer productivity. TL;DR - net productivity boost is a modest 15-20%, or as low as negative to 10% in complex, brownfield codebases. This tracks with my own experience as a dev.
Are you trying to tell me that the people want to sell me their universal panacea for all human endeavours were... lying...? Say it ain't so.
I use AI as an entryway to learning or for finding the name or technique that I'm thinking of but can't remember or know it's name so then i can look elsewhere for proper documentation. I would never have it just blindly writing code.
Sadly search engines getting shitter has sort of made me have to use it to replace them.
Then it's also good to quickly parse an error for anything obviously wrong.
LLMs work great to ask about tons of documentation and learn more about high-level concepts. It's a good search engine.
The code they produce have basically always disappointed me.
It turns every prototyping exercise into a debugging exercise. Even talented coders often suck ass at debugging.
I work adjacent to software developers, and I have been hearing a lot of the same sentiments. What I don't understand, though, is the magnitude of this bubble then.
Typically, bubbles seem to form around some new market phenomenon or technology that threatens to upset the old paradigm and usher in a new boom. Those market phenomena then eventually take their place in the world based on their real value, which is nowhere near the level of the hype, but still substantial.
In this case, I am struggling to find examples of the real benefits of a lot of these AI assistant technologies. I know that there are a lot of successes in the AI realm, but not a single one I know of involves an LLM.
So, I guess my question is, "What specific LLM tools are generating profits or productivity at a substantial level well exceeding their operating costs?" If there really are none, or if the gains are only incremental, then my question becomes an incredulous, "Is this biggest in history tech bubble really composed entirely of unfounded hype?"
From what I've seen and heard, there are a few factors to this.
One is that the tech industry right now is built on venture capital. In order to survive, they need to act like they're at the forefront of the Next Big Thing in order to keep bringing investment money in.
Another is that LLMs are uniquely suited to extending the honeymoon period.
The initial impression you get from an LLM chatbot is significant. This is a chatbot that actually talks like a person. A VC mogul sitting down to have a conversation with ChatGPT, when it was new, was a mind-blowing experience. This is a computer program that, at first blush, appears to be able to do most things humans can do, as long as those things primarily consist of reading things and typing things out - which a VC, and mid/upper management, does a lot of. This gives the impression that AI is capable of automating a lot of things that previously needed a live, thinking person - which means a lot of savings for companies who can shed expensive knowledge workers.
The problem is that the limits of LLMs are STILL poorly understood by most people. Despite constructing huge data centers and gobbling up vast amounts of electricity, LLMs still are bad at actually being reliable. This makes LLMs worse at practically any knowledge work than the lowest, greenest intern - because at least the intern can be taught to say they don't know something instead of feeding you BS.
It was also assumed that bigger, hungrier LLMs would provide better results. Although they do, the gains are getting harder and harder to reach. There needs to be an efficiency breakthrough (and a training breakthrough) before the wonderful world of AI can actually come to pass because as it stands, prompts are still getting more expensive to run for higher-quality results. It took a while to make that discovery, so the hype train was able to continue to build steam for the last couple years.
Now, tech companies are doing their level best to hide these shortcomings from their customers (and possibly even themselves). The longer they keep the wool over everyone's eyes, the more money continues to roll in. So, the bubble keeps building.
The biggest value I get from AI in this space is when I get handed a pile of spagehtti and ask for an initial overview.
I'd be inclined to try using it if it was smart enough to write my unit tests properly, but it's great at double inserting the same mock and have 0 working unit tests.
I might try using it to generate some javadoc though.. then when my org inevitably starts polling how much ai I use I won't be in the gutter lol
I personally think unit tests are the worst application of AI. Tests are there to ensure the code is correct, so ideally the dev would write the tests to verify that the AI-generated code is correct.
I personally don't use AI to write code, since writing code is the easiest and quickest part of my job. I instead use it to generate examples of using a new library, give me comparisons of different options, etc, and then I write the code after that. Basically, I use it as a replacement for a search engine/blog posts.
I always need to laugh when I read "Agentic AI"
I am jack's complete lack of surprise.
I have never seen an AI generated code which is correct. Not once. I've certainly seen it broadly correct and used it for the gist of something. But normally it fucks something up - imports, dependencies, logic, API calls, or a combination of all them.
I sure as hell wouldn't trust to use it without reviewing it thoroughly. And anyone stupid enough to use it blindly through "vibe" programming deserves everything they get. And most likely that will be a massive bill and code which is horribly broken in some serious and subtle way.
But will something be done about it?
NOooOoOoOoOoo. As long as it is still the new shiny toy for techbros and executive-bros to tinker with, it'll continue.