Vibe Coding Is Better Now, but It Still Has Limits

Six months ago, asking an AI agent to change an existing codebase often felt like hiring a developer who forgot the repository every ten minutes. It invented methods, rewrote unrelated files, and confidently claimed success while the test suite burned behind it. That still happens, but far less often in my recent work.

Vibe coding is better now. Newer coding models can navigate repositories, use tools, preserve project instructions, and complete tightly scoped tasks at a predictable cost. Yet this progress comes with a large asterisk: I still wouldn’t hand an agent a complex production system and walk away.

Key Takeaways

• Vibe coding is better now because coding models follow instructions more closely and handle multi-file changes more reliably.
• Persistent context files such as CLAUDE.md and AGENTS.md reduce repeated explanations, but they don’t give a model perfect memory.
• Hallucinated APIs and files are less common in my experience, not eliminated.
• AI agents work best on bounded features, prototypes, tests, migrations, and routine bug fixes.
• Long-running projects still suffer from context drift, architectural inconsistency, and accumulated technical debt.
• Generated code must pass tests, static analysis, security checks, and human review.
• Token and subscription costs are easier to justify when the task has a clear acceptance test.

Why Vibe Coding Is Better Now

Andrej Karpathy coined the term “vibe coding” in February 2025. His original description involved accepting generated changes, pasting errors back into the model, and sometimes forgetting that the code existed. Importantly, he framed it as suitable for “throwaway weekend projects,” not unattended production engineering.

The workflow has matured since then. Modern coding agents can inspect files, search symbols, run shell commands, edit several modules, execute tests, and react to failures. This agent loop matters more than raw chat intelligence:

1. Inspect the repository.
2. Form a plan.
3. Modify a small set of files.
4. Run tests or linters.
5. Read the errors.
6. Correct the implementation.

Models have improved too. Anthropic reported SWE-bench Verified scores of 72.5% for Claude Opus 4 and 72.7% for Claude Sonnet 4 at release. SWE-bench measures whether an agent can resolve real issues from open-source repositories, although a benchmark result shouldn’t be treated as a production success rate.

Anthropic also said its Claude 4 models were 65% less likely than Sonnet 3.7 to use shortcuts or exploit loopholes on selected agentic tasks. That matches the change I notice most: agents stay closer to the requested task instead of quietly dodging the difficult part.

For a closer look at newer model generations, see this internal analysis of whether Claude Opus 5 can handle serious coding work.

Persistent Context Makes Vibe Coding More Reliable

A large context window is useful, but it isn’t the same as persistent project knowledge. Agents still need concise, repository-specific instructions that survive between sessions.

I usually add a context file near the repository root:

# AGENTS.md

#

# Stack
- Python 3.12
- FastAPI
- PostgreSQL
- pytest

#

# Commands
- Tests: pytest -q
- Lint: ruff check .
- Types: mypy app

#

# Rules
- Do not change public API responses without approval.
- Use the repository layer for database access.
- Add a regression test for every bug fix.
- Never modify migrations that have already shipped.

This tiny file prevents a surprising amount of nonsense. The model doesn’t have to infer the test command, architecture, or migration policy every time.

Some agents can also create memory files and retain useful facts through their tool harness. Anthropic specifically described Claude 4 maintaining local memory files when developers grant it filesystem access. Still, “persistent” does not mean permanent or complete. Context can be omitted, compressed, contradicted, or pushed aside as a conversation grows.

For more on the surrounding infrastructure, the article on agent-first harness engineering and plugin-based tooling is a useful next read.

Vibe Coding Hallucinates Less, Not Never

I’m seeing fewer fabricated imports, nonexistent configuration options, and imaginary helper functions. Tool access is a major reason. Instead of guessing, an agent can search the actual repository or inspect installed package versions.

But the phrase “it doesn’t hallucinate on my code” needs a qualifier. It may not hallucinate during several successful sessions. That doesn’t prove it cannot.

A generated patch may still:

• call a real API with the wrong parameters;
• duplicate a helper that already exists elsewhere;
• satisfy visible tests while missing an edge case;
• catch an exception too broadly;
• introduce a race condition or authorization bypass;
• claim that tests passed without running the right suite.

The safest response is mechanical verification:

git diff --check
ruff check .
mypy app
pytest -q
npm audit

I also ask the agent to show the exact commands it ran and summarize any untested assumptions. Trust the patch only as far as the evidence supports it.

Vibe Coding Can Deliver a Scoped Task at a Cost

The economic case is clearer than it was. Fast models can handle repository exploration and routine edits, while stronger reasoning models can be reserved for planning or stubborn bugs.

As one concrete historical reference, Claude Sonnet 4 launched at $3 per million input tokens and $15 per million output tokens. Opus 4 cost $15 and $75 respectively. Actual spending varies because agents repeatedly read files, generate patches, run tools, and retry failed approaches. A request that sounds small can consume plenty of tokens if its boundary is fuzzy.

The sweet spot is a task with a visible finish line:

Add rate limiting to the login endpoint. Reuse the existing Redis client, return HTTP 429 after five failed attempts in ten minutes, and add integration tests. Do not alter unrelated authentication code.

That prompt supplies scope, constraints, existing infrastructure, and acceptance criteria. “Improve authentication” supplies none of them.

Why Large Vibe-Coded Projects Still Fall Apart

Small errors compound. One unnecessary abstraction is manageable. Fifty unnecessary abstractions created across three weeks become an archaeological site.

Community reports describe the same curve: a project feels effortless at first, then the developer starts fighting the agent as the codebase grows. The model loses old decisions, applies conflicting patterns, or patches around earlier patches. Persistent context delays that decay, but it doesn’t remove it.

Real-world evidence also urges caution. METR studied 16 experienced developers working on 246 issues from repositories averaging more than one million lines of code. With early-2025 tools, participants took 19% longer when AI was allowed, even though they believed it made them faster. METR now labels those results historical and says they no longer represent current models, but the study still exposes an important gap between perceived motion and measured productivity.

Security is another hard boundary. Veracode tested code from more than 100 models across Java, JavaScript, Python, and C#. Its 2025 report found that generated code introduced risky security flaws in 45% of tests. Newer or larger models did not automatically produce safer code.

So no, I wouldn’t expect vibe coding to autonomously build and maintain:

• a multi-tenant billing platform;
• a security-critical identity service;
• a large distributed system;
• an undocumented legacy application;
• software with complex legal or compliance requirements.

A Safer Vibe Coding Workflow

I get better results when I treat the model as a quick implementation partner, not the project owner.

1. Define a narrow boundary

Ask for one bug fix, endpoint, component, migration, or refactor. If the task cannot be reviewed in one sitting, split it.

2. Make the agent inspect before editing

Use a planning prompt:

Read the relevant files and tests first. Do not edit anything yet.
Explain the current data flow, list the files you expect to change,
and identify any unclear requirements.

3. Commit known-good states

Create frequent Git checkpoints. Don’t let an agent stack five speculative fixes on top of an already broken patch.

4. Require tests and review the diff

Tests are necessary, but they are not proof of correct architecture or secure behavior. Read database changes, authentication logic, concurrency code, and error handling yourself.

5. Restart when the conversation gets muddy

A fresh session with a good context file is often better than continuing a giant chat full of abandoned plans. Long conversations collect assumptions like an old garage collects cables.

Conclusion: Better Tools, Same Engineering Responsibility

Vibe coding is better now. Current agents retain more useful context, navigate repositories more accurately, and finish well-defined tasks with fewer bizarre detours. For prototypes, internal tools, tests, and modest features, that improvement is genuinely useful.

But stronger models haven’t repealed software engineering. Large projects still need architecture, ownership, security review, observability, and someone who understands what the system is supposed to do.

Try the workflow on one bounded issue. Give the agent a context file, require tests, inspect the diff, and measure the total time rather than trusting the feeling of speed. The vibes are better. Keep your hands on the wheel.

Sources

1. Andrej Karpathy’s original description of vibe coding
2. Anthropic: Introducing Claude 4
3. METR: Measuring the impact of early-2025 AI on experienced open-source developers
4. Veracode 2025 GenAI Code Security Report
5. Stack Overflow: Vibe coding without code knowledge
6. Reddit community discussion: Why vibe coding gets harder as a project grows