You Can Learn AI Agent Harness & Loop Engineering In 19 Min | LLM Ops, Eval, Tracing, RAG
Beginner
An LLM is an incredibly powerful brain that knows everything about humanity and nothing about you or the software you run. This video walks through the four buzzwords that turn that raw brain into a system you can trust: agent harness, loop engineering, LLM Ops, and eval. No jargon, step by step, technical or not.
You can build anything. You can learn anything. 💪
Timeline
0:00 Intro: the 4 AI agent buzzwords
0:42 What an AI agent run actually is
1:52 The memory system: procedural, semantic, episodic
3:37 What "harness" really means (the horse)
5:07 Storing and updating memory (databases, skills, summarizer agent)
8:28 Retrieval: RAG, SQL vs semantic search
10:00 Tool calling and why agents loop
10:52 Loop engineering and end-loop guardrails
13:21 A Claude Code hooks example
14:18 Eval and LLM Ops: why you need them
15:55 Tracing every run (Langfuse, LangSmith)
16:43 Evaluation: LLM as a judge
17:20 Diagnosing what broke
18:16 The gate: ship the fix or fix the bug
18:57 Zoom out: the full system
19:53 Wrap up
Launch Full-Stack App Template: https://github.com/ShenSeanChen/launch-mvp-stripe-nextjs-supabase
My AI Agent Startup AutoManus.io is an AI sales lead manager for made-to-order product sales. We embed directly where conversations happen (WhatsApp, email, web chat) to capture inbound, automate follow‑ups, and kill CRM busywork; live with 4 pilots and 2 contracts in motion, pre‑seed backed by Character VC. Focused on US/UK/Germany, raising to expand engineering and scale deployments.
☕️ Buy me a coffee: https://buy.stripe.com/5kA176bA895ggog4gh
- X: https://x.com/ShenSeanChen
- Sean's Discord: https://discord.gg/tvECErKcFr
- AutoManus Discord: https://discord.gg/5HhcNjCR
- LinkedIn: https://linkedin.com/in/shen-sean-chen
- Github Link: https://github.com/ShenSeanChen
- Instagram: https://www.instagram.com/sean_ai_stories
Sub Count when launched: 13,050
Top Github Repos
- LaunchMVP NextJS,Supabase,Stripe (1015 stars): https://github.com/ShenSeanChen/launch-mvp-stripe-nextjs-supabase
- ReactNextJSTips (114 stars): https://github.com/ShenSeanChen/yt-react-nextjs-tips
- DeepResearchAgentBackend (70 stars): https://github.com/ShenSeanChen/yt-DeepResearch-Backend
- SupabaseTipsKanban (64 stars): https://github.com/ShenSeanChen/yt-supabase-tips-kanban
- RAGVectorDB (56 stars): https://github.com/ShenSeanChen/yt-rag
- KimiK2Claude (37 stars): https://github.com/ShenSeanChen/yt-kimi-k2-claude
- DeepResearchAgentFrontend (33 stars): https://github.com/ShenSeanChen/yt-DeepResearch-Frontend
- AgenticRAG (13 stars): https://github.com/ShenSeanChen/yt-agentic-rag
You Can Build Anything Playlist (Full-Stack Dev Without CS Degrees)
- Core7: 95% React/NextJS Use Cases https://www.youtube.com/watch?v=vZzFlAjz4rA&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=16
- Core6: 95% Supabase Use Cases https://www.youtube.com/watch?v=Pnoq2tIif3s&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=15
- Core5: StripeNextJSSupabase https://www.youtube.com/watch?v=ad1BxZufer8&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=8
- Core4: AuthSupabaseGoogle https://www.youtube.com/watch?v=AZAU3ClAWWE&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=6
- Core3: FastapiGCPPro https://www.youtube.com/watch?v=u3sfqpzzo-U&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=5
- Core2: SupabaseHero https://www.youtube.com/watch?v=V0kmgkyPsVQ&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=4
- Core1: LaunchAnApp https://www.youtube.com/watch?v=iQ2izvGG4tQ&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=3
- Launch WhereToMeet w/ GoogleMaps, Calendar, Supabase, Cursor: https://www.youtube.com/watch?v=5xMjX1QxGUY&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR&index=2
- Launch EncourageMe w/ Cursor, Supabase, Vercel https://www.youtube.com/watch?v=hnvL4xa2DHA&list=PLE9hy4A7ZTmpGq7GHf5tgGFWh2277AeDR
Sean‘s AI Stories
