Build
Call a model with plain HTTP, give it tools, and write the agent loop yourself. No framework, so you see every moving part.
Build, Evaluate & Secure an AI Agent
A hands-on, vendor-neutral workshop. Build an AI agent from scratch, measure it with evals, then break it and make it safe. Runs on your laptop with open models, or on any provider you like.
What you’ll do
Section titled “What you’ll do”Evaluate
Write a test set, add automatic checks and an LLM judge, and find out how good your agent really is.
Attack
Play capture-the-flag against your own agent. Leak secrets, hijack it with poisoned data, make it act for someone else.
Defend
Add five layers of defense and prove they work by turning your attacks into tests.
Built for every community
Section titled “Built for every community”Vendor neutral
Works with any OpenAI-compatible endpoint: Ollama, LM Studio, vLLM or any cloud provider. Change one line in a .env file.
Free and offline-friendly
The default path runs a small open model on your own laptop. No API keys, no credits, no cloud account.
Modular
Run it as a 2-hour meetup, a half day or a full day. Organizers choose the modules that fit their event.
Ready to run
Starter and solution code for every module, facilitator notes, timing plans and promo copy for organizers.
Maintained by Stephen Simon (X). If this workshop is useful, star the repository.