GenAI — Cheat Sheet

LLM/GPT = a next-token predictor: given text, it outputs a probability over the next token, samples one, appends, repeats (autoregressive).
Tokens = sub-word chunks (~4 chars). Text → tokens → embeddings (vectors capturing meaning). Positional encoding adds word order.
Attention (Q/K/V): each token's Query scores every Key; softmax weights mix the Values → context-aware representation. The "Transformer" core.
Training = learn weights on huge text (slow, costly, frozen after). Inference = run frozen model to generate. Model has no memory + a knowledge cutoff.
Decoding knobs: temperature (↑ = more random), top-p (nucleus sampling), top-k, max_tokens (length cap). softmax turns logits → probabilities.

Roles: system (rules/persona), user (task), assistant (model replies). System steers behaviour.
Zero-shot = ask directly; few-shot = give examples in the prompt to set format/style.
Chain-of-Thought (CoT): "think step by step" → better reasoning on multi-step problems.
Structured output: demand JSON/schema for reliable, parseable results.
Be specific: give context, constraints, examples, and an output format. Garbage in → garbage out.

Agent = LLM + tools + a loop. Tools let it act (search, run code, call APIs) beyond its frozen knowledge.
Tool/function calling: model emits a structured call → your code runs it → result fed back → model continues.
ReAct loop: Thought → Action → Observation, repeat until done. Reason and act interleaved.
Planning vs reactive: reactive = decide one step at a time; planning = lay out full steps first, then execute.
Agents add autonomy but cost more tokens/latency and can loop — cap iterations.

RAG = Retrieval-Augmented Generation: fetch relevant docs, stuff into the prompt so the LLM answers from your data (beats cutoff + hallucination).
Chunking: split docs into passages (with overlap) so each fits and retrieves cleanly.
Embeddings: encode each chunk + the query as vectors; similar meaning → near in space.
Vector DB (Pinecone, Chroma, pgvector): stores vectors, does fast similarity search (cosine / kNN).
Flow: chunk → embed → store, then embed query → retrieve top-k → generate with context.

Query rewriting: rephrase/expand the user query so retrieval matches better.
HyDE: generate a hypothetical answer, embed that, and retrieve with it.
Re-ranking: a cross-encoder re-scores top-k hits for relevance before generating.
CRAG (Corrective RAG): grade retrieved docs; if weak, fall back to web search / re-query.
Also: hybrid search (keyword + vector), metadata filtering, better chunking, citations to cut hallucination.

LangChain: framework to wire LLMs, prompts, tools, retrievers, memory into apps.
LCEL (LangChain Expression Language): compose components with | pipes → runnable chains (stream/batch/async free).
LangGraph: model agents as a stateful graph of nodes + edges → loops, branches, cycles.
Good for control over flow vs a single agent loop; supports human-in-the-loop and persistence.
Building blocks: prompt templates, output parsers, retrievers, tools, memory.

Agent SDK: opinionated, minimal way to build agents — define an agent with instructions + tools and run it; loop handled for you.
Lifecycle: input → LLM picks tool → tool runs → result back → repeat → final output.
Handoffs: one agent delegates to another specialist agent (multi-agent orchestration).
Guardrails: validate inputs/outputs; structured outputs for typed results.
Less boilerplate than hand-wiring LangGraph for common agent patterns.

Thread = a conversation's running history so the agent keeps context across turns.
Tracing: record every LLM call, tool call, input/output, latency, token cost in a tree.
Observability (LangSmith, etc.): debug why an agent did X, find failures, measure cost & speed.
Spans nest under a trace; tag runs to filter and compare.
Essential for production: without traces, agents are black boxes.

Model is stateless — memory is engineered around it.
Short-term = the context window: recent messages re-sent each turn (finite, costs tokens).
Long-term: persist facts in a DB / vector store and retrieve them when relevant.
Manage the window: summarize old turns, trim, or store + recall key facts.
Types: conversational buffer, summary memory, semantic (vector) memory.

Knowledge graph: data as nodes (entities) + edges (relationships) — stores connections, not just facts.
Neo4j = graph DB; query with Cypher to traverse relationships (multi-hop "who connects to what").
GraphRAG: retrieve over the graph for multi-hop / relational questions vectors miss.
Great when answers depend on how entities relate, not text similarity alone.

Pipeline: STT (speech→text) → LLM (reason) → TTS (text→speech).
Latency is the enemy — every stage adds delay; aim for natural turn-taking.
Tricks: streaming partial transcripts/audio, interrupts (barge-in), VAD to detect speech end.
Speech-to-speech models cut hops vs the 3-stage pipeline.
Handle accents, noise, and conversational repair gracefully.

MCP (Model Context Protocol): a standard way to expose tools/data to any LLM via tool servers — plug-and-play, no custom glue per app.
MCP server offers tools, resources, prompts; clients (agents) connect and call them.
A2A (Agent-to-Agent): protocol for agents to discover & talk to each other across systems.
Together: standardized tool access + agent interop → composable ecosystems.

Ship the app as an API/service (e.g. on AWS); containerize + a managed model/API behind it.
Scaling: horizontal replicas + load balancing; handle bursty traffic.
Caching: cache embeddings & frequent LLM responses to cut cost + latency.
Rate limiting: protect against abuse and provider quota limits.
CI/CD for safe deploys; monitor cost, latency, errors; manage secrets/keys.

GenAI — Quick Revision