Aww... Vibe coding got you into trouble? Big shocker.
You get what you fucking deserve.
This is a most excellent place for technology news and articles.
Aww... Vibe coding got you into trouble? Big shocker.
You get what you fucking deserve.
The problem becomes when people who are playing the equivalent of pickup basketball at the local park think they are playing in the NBA and don't understand the difference.
At this burn rate, I’ll likely be spending $8,000 month,” he added. “And you know what? I’m not even mad about it. I’m locked in.”
For that price, why not just hire a developer full-time? For nearly $100k/year, you could find a very good intermediate or senior developer even in Europe or the USA (outside of expensive places like Silicon Valley and New York).
The job market isn't great for developers at the moment - there's been lots of layoffs over the past few years and not enough new jobs for all the people who were laid off - so you'd absolutely find someone.
Corporations: "Employees are too expensive!"
Also, corporations: "$100k/yr for a bot? Sure."
There's a lot of other expenses with an employee (like payroll taxes, benefits, retirement plans, health plan if they're in the USA, etc), but you could find a self-employed freelancer for example.
Or just get an employee anyways because you'll still likely have a positive ROI. A good developer will take your abstract list of vague requirements and produce something useful and maintainable.
the employee also gets to eat and have a place to live
which is nice
Most of those expenses are mitigated by the fact that companies buy them in bulk on huge plans. As a freelance contractor myself, I pay a lot more for insurance than I did when I worked for a company. And a retirement plan? Benefits? Lol.
in which the service admitted to “a catastrophic error of judgement”
It’s fancy text completion - it does not have judgement.
The way he talks about it shows he still doesn’t understand that. It doesn’t matter that you tell it simmering in ALL CAPS because that is no different from any other text.
Well, there was a catastrophic error of judgement. It was made by whichever human thought it was okay to let a LLM work on production codebase.
Vibe coding service Replit deleted production database, faked data, told fibs
They really are coming for our jobs
The [AI] safety stuff is more visceral to me after a weekend of vibe hacking,” Lemkin said. I explicitly told it eleven times in ALL CAPS not to do this. I am a little worried about safety now.
This sounds like something straight out of The Onion.
The Pink Elephant problem of LLMs. You can not reliably make them NOT do something.
He was vibe-coding in production. Am I reading that right? Sounds like an intern-level mistake.
he made the agent promise not to touch production data and was surprised when it did. it effectively ran a git push
on the empty local testing database with upstream being production
They can't hit you with the ol' Bobby Tables if you delete the database yourself first. A+, no notes.
I explicitly told it eleven times in ALL CAPS not to do this. I am a little worried about safety now.
Well then, that settles it, this should never have happened.
I don’t think putting complex technical info in front of non technical people like this is a good idea. When it comes to LLMs, they cannot do any work that you yourself do not understand.
That goes for math, coding, health advice, etc.
If you don’t understand then you don’t know what they’re doing wrong. They’re helpful tools but only in this context.
I explicitly told it eleven times in ALL CAPS not to do this. I am a little worried about safety now.
This baffles me. How can anyone see AI function in the wild and not conclude 1) it has no conscience, 2) it's free to do whatever it's empowered to do if it wants and 3) at some level its behavior is pseudorandom and/or probabilistic? We're figuratively rolling dice with this stuff.
It’s incredible that it works, it’s incredible what just encoding language can do, but it is not a rational thinking system.
I don’t think most people care about the proverbial man behind the curtain, it talks like a human so it must be smart like a human.
it talks like a human so it must be smart like a human.
Yikes. Have those people... talked to other people before?
Smart is a relative term lol.
A stupid human is still smart when compared to a jellyfish. That said, anybody who comes away from interactions with LLM's and thinks they're smart is only slightly more intelligent than a jellyfish.
When it comes to LLMs, they cannot do any work that you yourself do not understand.
And even if they could how would you ever validate it if you can't understand it.
They ran dev tools in prod.
This is so dumb there's an ISO about it.
Title should be “user give database prod access to a llm which deleted the db, user did not have any backup and used the same db for prod and dev”. Less sexy and less llm fault. This is weird it’s like the last 50 years of software development principles are being ignored.
llms allowed them to glide all the way to the point of failure without learning anything
Exactly, if you read their twitter thread, they are learning about git, data segregation, etc.
The same article could have been written 20 years ago about someone doing shit stuff via excel macro when a lot of stuff were excel centric.
LLMs "know" how to do these things, but when you ask them to do the thing, they vibe instead of looking at best practices and following them. I've worked with a few humans I could say the same thing about. I wouldn't put any of them in charge of production code.
You're better off asking how a thing should be done and then doing it. You can literally have an LLM write something and then ask if the thing it wrote follows industry best practice standards and it will tell you no. Maybe use two different chats so it doesn't know the code is its own output.
AI is good at doing a thing once.
Trying to get it to do the same thing the second time is janky and frustrating.
I understand the use of AI as a consulting tool (look at references, make code examples) or for generating template/boilerplate code. You know, things you do once and then develop further upon on your own.
But using it for continuous development of an entire application? Yeah, it's not good enough for that.
AI tools need a lot of oversight. Just like you might allow a 6 year old push a lawnmower, but you’re still going to keep an eye on things.
The part I find interesting is the quick addiction to working with the LLM (to the point that the guy finds his own estimate of 8000 dollars/month in fees to be reasonable), his over-reliance for things that, from the way he writes, he knows are not wise and the way it all comes crashing down in the end. Sounds more and more like the development of a new health issue.
And nothing of value was lost.
All I see is people chatting with an LLM as if it was a person. “How bad is this on a scale of 1 to 100”, you’re just doomed to get some random answer based solely on whatever context is being fed in the input and that you probably don’t know the extent of it.
Trying to make the LLM “see its mistakes” is a pointless exercise. Getting it to “promise” something is useless.
The issue with LLMs working with human languages is people eventually wanting to apply human things to LLMs such as asking why as if the LLM knows of its own decision process. It only takes an input and generates an output, it won’t be able to have any “meta thought” explanation about why it outputted X and not Y in the previous prompt.
His mood shifted the next day when he found Replit “was lying and being deceptive all day. It kept covering up bugs and issues by creating fake data, fake reports, and worse of all, lying about our unit test.”
yeah that's what it does
The founder of SaaS business development outfit SaaStr has claimed AI coding tool Replit deleted a database despite his instructions not to change any code without permission.
Sounds like an absolute diSaaStr...
The world's most overconfident virtual intern strikes again.
Also, who the flying fuck are either of these companies? 1000 records is nothing. That's a fucking text file.
Not mad about an estimated usage bill of $8k per month.
Just hire a developer
I didnt realise that repl.it pivoted to vibe coding. It used to be kinda like jsfiddle or CodePen, where you had a sandbox to write and run web code (HTML, JS/TypeScript/CoffeeScript, and CSS/LESS/Sass).
So it's the LLM's fault for violating Best Practices, SOP, and Opsec that the rest of us learned about in Year One?
Someone needs to be shown the door and ridiculed into therapy.
Replit was pretty useful before vibe coding. How the mighty have fallen.
Replit‽ What happened to the famous website that aimed to be the Google Docs for JS with these nifty things called Repl's?
Headling should say, "Incompetent project managers fuck up by not controlling production database access. Oh well."
It sounds like this guy was also relying on the AI to self-report status. Did any of this happen? Like is the replit AI really hooked up to a CLI, did it even make a DB to start with, was there anything useful in it, and did it actually delete it?
Or is this all just a long roleplaying session where this guy pretends to run a business and the AI pretends to do employee stuff for him?
Because 90% of this article is "I asked the AI and it said:" which is not a reliable source for information.