Things I've Learned Building AI Agents

Agents are where LLMs stop being "a chat box" and start doing real work: looking things up, making decisions, and taking actions. It's also where things can go sideways fast.

I've been building small agents in a very casual "ship it and learn" way, and the biggest lesson is this:

You don't always need an agent.

Some of these are just personal tools I tinker with, and some are agents I build for businesses. The lessons end up being basically the same.

Sometimes what you actually need is a boring workflow with one or two AI steps inside it. It will be cheaper, more reliable, and easier to debug.

Workflows vs Agents (What's the Difference?)

Workflow: You know the steps ahead of time.

• "Do A, then B, then C."
• Great when the task is repeatable.
• Easier to test and trust.

Agent: The system decides the steps as it goes.

• "Here's the goal. Figure it out."
• Great when the task has lots of variation or unknowns.
• Needs stronger guardrails, logging, and stop conditions.

My rule of thumb: start with a workflow. Add agent behavior only where the workflow hits an "unknown" state.

The Core Loop (All Frameworks Reduce To This)

Whether you use Google ADK, OpenAI's tooling, LangChain, or something else, the same loop shows up. Even tools like Claude Code and Codex are built around some version of it.

• Visibility: What can the agent see? APIs, logs, documents, calendars, databases, browser pages. If it can't see it, it will guess.
• Decision-making: How does it choose what to do next? The best results usually come from a simple flowchart for the known parts, and "LLM reasoning" only when the state is unclear.
• Action: What can it do in the world? Send an email, create a ticket, update a doc, file a PR, schedule an event. Actions should be logged and ideally reversible.

That's it: observe, decide, act, observe again. Repeat until done.

What Usually Fails In The Real World

This is the stuff that makes agents feel "cool in a demo" but painful in production:

• Fuzzy goals: "Help me with my week" turns into randomness. Agents need a crisp success condition.
• Bad tools: One flaky API call will make the whole agent feel unreliable.
• Too much freedom: If the agent can do everything, it will eventually do something you didn't mean.
• No stop condition: Agents get stuck in loops or keep "trying one more thing" forever.
• No paper trail: If you can't see what it did and why, you can't fix it.
• Context bloat: Stuffing everything into the prompt makes it slower, more expensive, and sometimes worse.
• Silent failures: Tools fail but the agent continues as if nothing happened.

A few "I learned this the hard way" moments:

• The calendar optimizer: I asked an agent to "make room for a meeting" and it tried to reshuffle half the week. Now I use "propose changes" by default and require approval for anything that touches real schedules.
• The mega tool trap: I built one email_tool that could read, draft, and send. The agent got confused fast, so I split it into smaller tools and made "send" a separate, gated action.
• The context firehose: I dumped a giant doc into the prompt and wondered why the agent still missed the point. Now I start with the one page that matters, then let it pull more only when it can say what it's looking for.
• The retry spiral: An API got rate limited, and the agent just kept trying. I added step budgets, backoff, and a simple rule: fail twice, then ask a human instead of burning the whole run.
• The "success" that did nothing: A tool returned "ok" but didn't actually create the event (bad auth). I started verifying important actions ("did it really change?") before moving on.

Tool Design (Boring Is Good)

Tools are just functions the agent can call. The trap is building "mega tools" that do 12 things.

Better pattern:

• One tool does one job: search_docs, get_calendar_events, send_email, create_issue.
• Return clean, structured results (not walls of text).
• Fail loudly with useful errors ("auth failed", "rate limited", "not found").
• Prefer actions that are safe to retry when possible.
• Add a "dry run" mode for dangerous actions.

Memory & Context (Don't Stuff The Whole World In The Prompt)

Agents run over time, so they need to remember:

• What they already tried
• What they learned from tool results
• What decision they made and why

I use the prompt/context window for the current task, and external memory for what happened so far.

External memory can be simple: a file, a database row, a run log, or a searchable knowledge base. The key is that it's retrievable and relevant, not just "more text."

An example of this is how I use tools like Windsurf and Codex. I'll outline a project, generate a simple spec, and keep an actionable sprint list.

Then the agent can read the spec, the current sprint, and any key notes from past sprints and start working. I don't need to dump every file I've ever touched or my entire code history just so it can decide what to do next.

Guardrails (Enforced, Not Requested)

One misconception: don't just tell the agent "don't delete things."

The system around the agent should enforce it.

A pattern I like:

• Default to a no-destruction posture (read/create/update, no deletes).
• Only allow specific tools (an explicit list), not arbitrary code execution.
• Add budgets: max steps, max time, max spend.
• Require approval for high-impact actions (send email, create PR, schedule meetings).
• Prefer reversible actions when you can (drafts, staging, "propose changes" mode).

If the agent decides deleting your whole calendar is the best way to make you available for a meeting, it should simply be blocked and forced to find another path.

Validation & "Am I Making Progress?"

Agents need a way to prove they're getting closer to done:

• Define what "done" looks like up front.
• Validate outputs before acting on them.
• Detect loops ("already tried this") and bail out.
• If the agent is stuck, it should ask a human for clarification instead of guessing.

How I Build Agents (My High-Level Checklist)

If you want a simple build process that actually works, this is what I follow:

1. Write the goal + success condition in plain English (what does "done" mean?).
2. Start as a workflow and mark the unknown steps (where the AI earns its keep).
3. List what the agent needs to see (visibility: inputs, data sources, current state).
4. Design tools that are small, reliable, and testable on their own.
5. Pick decision logic (simple states/steps for the routine path; model reasoning for unknowns).
6. Add guardrails (permissions, budgets, approvals, no-destruction defaults).
7. Add validation + stop conditions (progress checks, loop detection, "ask for help" triggers).
8. Run a few real scenarios (happy path, missing data, tool failure, ambiguous request, risky action) every time you change the agent.

Do that and you're 80% of the way to an agent you can actually trust.

Evolving "Skills" Over Time

My agents get better when I treat "skills" like something I can improve, not magic prompts.

What tends to work:

• Start with a manual checklist you would follow as a human.
• Turn each step into a tool or a small workflow component.
• Capture failures as new rules ("when X happens, do Y instead").
• Version your prompts and tools like code, because they are code.

Frameworks

Frameworks like Google ADK, OpenAI's tooling, and LangChain can help with orchestration, tool calling, and memory patterns. They're great, but they're not the hard part.

The hard part is designing the loop, tools, guardrails, and "definition of done" so the agent behaves like a responsible teammate instead of a chaotic intern with API keys.

Closing

Agents are powerful, but reliability comes from the boring stuff: clear goals, simple tools, enforced restrictions, and a paper trail.

Start small. Ship one useful agent. Let it fail in a controlled way. Then improve the skill.

Written from home, fueled by coffee and error logs.