← All handbooks

AI Agentic Handbook

Agentic AI end to end — turning an LLM from a text generator into a system that reasons, calls tools, retrieves knowledge, remembers, and acts toward a goal. Patterns, pitfalls, evaluation, and guardrails.

01 LLM basics

• An LLM predicts the next token given prior tokens. “Reasoning” emerges from this + scale + post-training (RLHF / preference tuning).
• Context window = max tokens (prompt + output) the model can attend to. Everything the agent “knows in the moment” must fit here.
• Sampling: temperature (randomness), top-p. Low temp for tools/structured output, higher for ideation.
• Models hallucinate (confident wrong answers) and have a training cutoff — ground them with tools & retrieval.

02 Prompting

• System prompt sets role, rules, format, and tool policy. Be explicit and bounded.
• Few-shot examples teach format/behavior; chain-of-thought (“think step by step”) improves multi-step reasoning.
• Demand structured output (JSON schema / function calling) when another system consumes it — don’t parse prose.
• Prompt-injection is real: untrusted retrieved/tool text can try to override instructions — never blindly trust it (see Guardrails).

03 What an agent is

# the core agent loop
state = init(goal)
while not done(state):
    thought, action = llm(state)        # decide
    if action.is_tool:
        obs = run_tool(action)           # act
        state = state + action + obs     # observe
    else:
        return action.final_answer
    if steps > MAX: break            # stop condition

This is the ReAct pattern (Reason + Act). Levels of autonomy rise from: single call → chain (fixed steps) → router → tool-using agent → planning agent → multi-agent. Use the least autonomy that solves the task — more loops = more cost, latency, and failure modes.

04 Tool use (function calling)

{
  "name": "query_sales",
  "description": "Run a read-only SQL query against the sales DB",
  "parameters": { "sql": {"type":"string"} }
}

• You expose tools (functions/APIs) with name + description + JSON schema. The model picks one and emits arguments; your code executes and returns the result.
• Good tools are narrow, well-described, validated, and side-effect-aware. Validate every argument server-side — the model can emit anything.
• MCP (Model Context Protocol) standardizes how agents discover & call external tools/data sources.
• Gate destructive actions behind confirmation; prefer read-only + dry-run; least privilege per tool.

05 RAG & retrieval

Retrieval-Augmented Generation grounds answers in your data instead of model memory.

chunk docs → embed → store in vector DB
query → embed → similarity search (top-k) → rerank
→ stuff retrieved context into the prompt → generate (cite sources)

• Chunking matters: semantic/structural chunks with overlap; keep metadata for filtering & citations.
• Embeddings map text → vectors; vector DB (pgvector, Pinecone, FAISS) does ANN search. Add hybrid search (keyword + vector) and a reranker for precision.
• Cite sources; show retrieved snippets; say “I don’t know” when retrieval is weak. RAG > fine-tuning for fresh/changing facts.
• Agentic RAG: the agent decides when and what to retrieve, can re-query and refine.

06 Memory & context management

• Short-term = the conversation in the context window. It’s finite → manage it.
• Summarize / compact older turns; keep a rolling summary + recent verbatim turns.
• Long-term = external store (vector/DB) of facts, preferences, prior outcomes; retrieve relevant bits per turn (this is RAG over memory).
• Scratchpad / state: persist intermediate results (plan, tool outputs) outside the prompt; reference by id to save tokens.

07 Planning & control loops

• ReAct — interleave reasoning and tool calls (default).
• Plan-and-execute — draft a multi-step plan, then execute (and re-plan on failure). Better for complex tasks.
• Reflection / self-critique — the agent reviews its own output and retries; pairs with evals.
• Always set stop conditions: max steps, budget, time, and a confidence/exit check — agents loop forever without them.
• Decompose: break the goal into sub-tasks; verify each before proceeding.

08 Multi-agent systems

• Orchestrator–worker (a planner delegates to specialists), pipeline (hand-offs), debate/critic (one proposes, one reviews).
• Each agent gets a focused role, its own tools, and a narrow prompt → better than one giant agent.
• Costs compound (more calls, more latency, coordination bugs). Start single-agent; add agents only when a clear role boundary helps.
• Frameworks: LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Claude Agent SDK — but the patterns matter more than the library.

09 Evaluation

• You can’t improve what you don’t measure. Build an eval set of real tasks with expected outcomes early.
• Metrics: task success rate, tool-call accuracy, retrieval precision/recall, faithfulness (grounded vs hallucinated), latency, cost/task.
• Methods: exact/fuzzy match, rubric-based LLM-as-judge (validate the judge!), human review for high-stakes.
• Trace every run (inputs, prompts, tool calls, outputs) — LangSmith / Langfuse / Phoenix. Regression-test on prompt/model changes.

10 Guardrails & safety

• Input: validate/scope user input; detect prompt-injection; strip/neutralize instructions embedded in retrieved or tool content.
• Output: schema-validate; content/PII filters; refuse out-of-scope; keep a human in the loop for irreversible actions.
• Tool safety: least privilege, allow-lists, read-only defaults, confirmations for writes/deletes, sandboxing, rate limits.
• Treat all model output and external text as untrusted. Don’t let the model escalate its own permissions.

11 Ops & cost

• Cost scales with tokens × calls. Trim context, cache (prompt caching, retrieval results), pick the smallest model that passes evals, cap loop steps.
• Latency: stream responses, parallelize independent tool calls, use smaller/faster models for routing.
• Reliability: timeouts, retries with backoff, fallbacks, idempotent tools, structured-output validation + repair.
• Observability: log traces, token usage, tool errors; monitor success rate & drift; version prompts.

12 Interview drills

• What makes something an “agent” vs a single LLM call; describe the ReAct loop.
• RAG pipeline end to end; why RAG over fine-tuning; how to reduce hallucination.
• How function calling works; how you’d make tools safe.
• Short-term vs long-term memory; how to handle a full context window.
• How you’d evaluate an agent; what is LLM-as-judge and its risks.
• What prompt injection is and how to defend; when to add a second agent.