Use any model provider
The workshop code talks to models through the OpenAI-compatible chat completions API: one POST to /chat/completions. Almost every model server and cloud provider supports it, so switching providers means changing three lines in code/.env:
LLM_BASE_URL=...LLM_API_KEY=...LLM_MODEL=...Then run python -m agentlab.check to confirm it works.
Local servers
Section titled “Local servers”These run on your own machine. Free, private and offline.
| Server | LLM_BASE_URL |
LLM_API_KEY |
|---|---|---|
| Ollama (default) | http://localhost:11434/v1 |
anything, for example ollama |
| LM Studio | http://localhost:1234/v1 |
anything |
| llama.cpp server | http://localhost:8080/v1 |
anything |
| vLLM | http://localhost:8000/v1 |
anything, unless you set one |
Cloud providers
Section titled “Cloud providers”Use the base URL from your provider’s documentation for its OpenAI-compatible endpoint. Some common ones, in alphabetical order:
| Provider | LLM_BASE_URL |
|---|---|
| Anthropic | https://api.anthropic.com/v1 |
| Azure OpenAI / Microsoft Foundry | https://YOUR-RESOURCE.openai.azure.com/openai/v1 |
| GitHub Models | https://models.github.ai/inference |
| Google Gemini | https://generativelanguage.googleapis.com/v1beta/openai |
| Groq | https://api.groq.com/openai/v1 |
| Hugging Face Inference Providers | https://router.huggingface.co/v1 |
| Mistral | https://api.mistral.ai/v1 |
| OpenAI | https://api.openai.com/v1 |
| OpenRouter | https://openrouter.ai/api/v1 |
Put your provider’s API key in LLM_API_KEY, and a model name from its docs in LLM_MODEL. Endpoints and model names change over time, so check the provider’s current documentation if something doesn’t work.
Choosing a model
Section titled “Choosing a model”The model must support tool calling (also called function calling). Check your provider’s model list. For local models, good options are:
| Model | Size | Notes |
|---|---|---|
qwen3:4b-instruct |
about 2.5 GB | Workshop default. Fast, good at tools |
qwen2.5:1.5b |
about 1 GB | For laptops with 8 GB of RAM. Less accurate |
llama3.1:8b |
about 5 GB | Needs 16 GB of RAM. More capable |
Avoid “thinking” or reasoning models for the workshop on a laptop. They think before every reply, which makes each step much slower.
A different judge
Section titled “A different judge”In Module 4, the judge can be a different model, or even a different provider, from the agent. A stronger judge gives more reliable grades. Set JUDGE_MODEL in .env. To use a different provider for the judge, you’d extend agentlab/llm.py with a second base URL, which makes a nice extra exercise.