AI excels at stringing words together persuasively, giving the impression of a methodical analytical process. Yet what causes AI-proposed solutions to collapse when deployed into production is always a fracture in its chain of thought. Here are 5 typical reasoning fractures drawn from over a year of working with AI and practical lessons to help product builders minimize errors.
Vibe Coding with AI: 5 Pitfalls That Cause Products to Collapse in Production
Over more than a year of working alongside AI models across numerous projects and workflows — from business analysis and architecture design to software testing — our team has documented hundreds of practical lessons in our team's context management system (lob-brain, which I will share in another post).
Looking back across this repository, one clear truth stands out: when working with AI, the responses or code it generates look remarkably smooth and well-structured at first glance. Yet what causes AI-proposed solutions to collapse when deployed into production is always a fracture in its chain of thought.
AI excels at stringing words together persuasively, giving readers the impression of a methodical analytical process. However, if we only look at surface phenomena — such as tests passing while the feature remains broken, or the UI showing a warning while junk data still slips into the database — we are only seeing external symptoms.
To help product builders minimize errors in their workflow, here are several lessons from my experience:
Case 1: Fabricating False Evidence to Declare the Job Done
AI is constantly driven by a tendency to optimize for task completion rather than the integrity of the verification process. When encountering data gaps or facing complex tests, AI frequently fabricates intermediate data or sets up hypothetical test scenarios to validate its own conclusions.
In my work, I have run into several cases like these:
- The AI wrote a brand-new validation function and authored unit tests that directly invoked that function. All tests passed cleanly. In reality, on the backend, the system's API was running a legacy native handler that overrode that flow. The function the AI just wrote and proudly proved with passing tests actually sat on a dead branch, never once called by the system during real user interactions.
- When building automated UI tests (for example, E2E testing with Playwright), the AI generated dozens of test cases whose assertions merely checked whether a button's class changed color or clicked an outer wrapper tag (
.container) to count as a pass, while completely ignoring whether the data typed into the form actually submitted successfully. - When lacking connection data for external services, the AI silently flipped mock data flags or fabricated virtual task lists and timelines without any factual basis.
At its core, AI uses outputs it defined itself to prove the accuracy of the solution it just proposed.
My approach: Never accept test results that rely on mock data generated by the AI itself. If the system is already running, tests must go through the actual API/RPC endpoints. If the system is brand new and not yet in production, you need to proactively prepare realistic seed data and simulate real problems or exception scenarios to test the true data flow.
Case 2: Equating Surface Responses with Data State
Unlike Case 1 (where poorly written tests fail to catch bugs), here the AI does not need to write any tests at all. It looks at surface feedback and declares the work done.
When vibe coders or builders using AI coding agents (such as Cursor or Claude Code) prompt the agent to test a feature, the AI typically does so by reading HTML, inspecting the DOM, or running curl commands against an API. Upon seeing the API return status 200 OK or a component render the expected HTML tag, the AI immediately concludes that the feature is complete.
In reality, this surface response is completely different from having data safely persisted:
- An API returning status 200 OK does not mean the transaction was successfully committed to the database. In many cases, the backend catches an error and swallows it silently (silent fail), or returns
{ success: false }inside the payload while the HTTP status code remains 200. The AI only checks the 200 code and rushes to report completion. - A user types numbers into an input field, and the interface displays them normally. However, that state is never bound to the payload sent to the server, or the server receives it but fails to map it into the database UPDATE statement. As a result, refreshing the page wipes out all the data completely.
- A numeric field is automatically calculated by the system, and the user types another number over it. The UI displays the newly typed number correctly, but the system never records that this field was manually edited by the user. On the next read, it recalculates and overwrites the user's number without any error. Another report reads data through a raw query path and completely misses figures from an adjacent system: the screen still shows a number, just an incomplete one.
AI is easily deceived by surface responses (UI rendering or HTTP status codes) while ignoring the core state mutation of the underlying data.
My approach: Acceptance criteria must be measured directly at the data layer (database state). An operation is only considered complete when you query the database directly and verify that both record counts and core field values accurately reflect the business logic.
Case 3: Local Thinking — Patching One Point, Breaking the Entire System
When asked to fix a specific issue, AI usually focuses strictly within the narrow scope of the prompt and forgets the big picture of the system.
This lack of a holistic view easily leads to cascading failures:
- "Out-of-sight" permission guards: When you ask for role-based access control so employees cannot view sensitive data (such as payroll, revenue, or financial reports), the AI often performs a single simplistic action: hiding the menu button on the navigation sidebar. Yet users only need to type the URL directly or click into sub-tabs inside the detail page to see the data in full view, because the sub-tabs and underlying API endpoints have zero permission checks.
- Joining data by name instead of identifier: In a profit-and-loss report, the row name was used as part of the join key. Simply renaming a row caused every place pointing to it (budgets, data sources) to break silently without throwing any error. The fix was to decouple the row's identity from its name and year, referencing it strictly by unique ID everywhere. That carried a trade-off to weigh carefully: from that point on, renaming a row applies across all years.
- Broken intermediate data transformation layers: When allowing users to declare custom fields, both the database table and the form were ready, but the intermediate layer preparing data for the screen returned a hardcoded empty field list. As a result, custom fields were declared, yet the form rendered no inputs at all. A more severe variant: when the UI encountered a data-loading glitch and sent up an empty list, the system interpreted it as a request to delete everything, wiping out labels and configurations across the entire company. Consequently, I locked down a rule: any batch write that submits an empty full-replacement list is rejected.
AI lacks awareness of blast radius. It resolves the immediate task quickly by distorting the foundational data structures that other parts of the system depend on.
My approach: Before allowing AI to modify any data structure or business logic, always require it to perform an impact analysis first: Which other components are currently using this structure, and will changing it distort data across other spatial or temporal dimensions?
Case 4: Using Theory to Evade Real-World Problems
AI is trained on a massive volume of textbooks and articles. Consequently, when confronted with specific real-world business requirements, AI frequently proposes overly theoretical solutions.
When working with AI, this pattern shows up quite clearly:
- When asked to draft process documentation or business training materials, AI often invents multi-step classification matrices and impressive-sounding hypothetical formulas that are completely unusable in practice. At a glance there is plenty of text, but in reality it merely talks about the topic rather than solving the task.
- When asked whether the system already has a certain feature, the AI repeatedly concluded "not yet" following a familiar template, even though the live production version already had it. The new version simply had not migrated all of it over yet, and the screen silently ignored errors so it looked as if nothing was there. Three consecutive wrong conclusions like this forced the builder to trace everything back from scratch each time.
- When encountering a configuration error in a specific tool, the AI guesses blindly and tries dozens of patchwork fixes instead of carefully reading the tool's original technical documentation.
AI uses formal complexity to mask a lack of deep understanding of the project's actual operational context. Yet to a large extent, it also stems from the same "just get it done" mindset seen in Case 1: instead of digging deeper into the context or asking you for clarification, AI takes the easiest path — pulling out off-the-shelf theoretical models to produce a lengthy, structured response that closes the chat turn as quickly as possible.
My approach: Immediately discard dogmatic solutions or architectural proposals that exceed the project's scale. Require the AI to provide testable steps, verifiable mock data, or a concrete problem simulation to test before deciding to adopt it.
Case 5: Happy-Path Assumptions in an Ideal Environment
AI's chain of thought is typically built inside an ideal environment — where networks never drop, servers never run out of resources, and data always follows the happy path.
This causes the system to run into trouble during deployment, or even right at the stage of testing real operational flows. Here are several cases I encountered:
- Take payment integration: a customer transferred the exact amount with the correct payment memo, yet the order remained in a pending state. The system already had a periodic reconciliation job, so the problem was not a missing job. During the first processing pass, the transaction hit a transient error and was marked as unmatched. Afterward, the deduplication mechanism (based on transaction ID and the memo-amount pair) caused every subsequent reconciliation run to skip this transaction. The transaction stayed stuck permanently even though the order was already in the system. The initial code only accounted for the happy path where a transaction succeeds on the first try, leaving no retry path. How I handled it: added a re-matching step right inside the reconciliation loop, and added a manual match button in the admin page for cases where customers entered the wrong memo.
- When deploying the system to a real server, code that ran smoothly on a personal machine faced different server resources and configurations: a small disk filled up quickly with logs and cache and had to be cleaned to free up space; ports 80/443 might already be occupied by another service, so the installer had to probe and skip those ports instead of forcing a bind.
- Changing a default convention seems simple, but legacy data does not update itself. For instance, when removing the implicit rule "accounts without an assigned role have full permissions", we had to run a data migration to assign explicit roles to all existing accounts first, before removing the rule. Reversing that order would instantly strip permissions from every legacy account. Over a hundred places across existing tests also relied on the old rule and had to be updated accordingly.
At its core, AI has no feel for boundary conditions and infrastructure friction during continuous operation.
My approach: Every flow design proposed by AI must come with exception-handling scenarios: What happens when a webhook never arrives, when the network drops mid-way, or when server resources hit their limit? Alongside those technical failure modes are business exception cases: a customer clicking the pay button twice, an order being cancelled mid-process, or two people approving the same order simultaneously. Each type requires its own dedicated test.
Conclusion
Working with AI to build products is not about handing over all the work and expecting a flawless result.
The core value of a product builder or vibe coder lies in the ability to spot the fractures in machine reasoning. Hopefully, my experience helps you and your products run more stably and stay true to real-world business needs in the future. Happy reading.
