The AI black box

One piece of advice I heard recently is not to automate something in your business with AI until you know how to do it manually. I would say it's sound advice, but not always practical, depending on how complex your business needs are.

People are vibe coding and spinning up apps left and right "without coding experience," but is this really a good thing? I noticed in my own experience with SQL that I much prefer a query that I wrote that AI helped me modify, vs a query written purely by AI. It's one thing to let AI write your app, but what happens if a user finds a bug in production and you're out of tokens to fix it?

For simple tasks like maintaining my directory, I can always fall back to the manual method if needed. The problem, as AI scales to handle more and more complex tasks, is that we don't understand all the moving parts. We let AI create a black box, and all is well and good as long as it's working smoothly. Maybe the juice is ultimately worth the squeeze, but I am hesitating to jump into the vibe coding world until I have learned some fundamental skills to peer into the black box.