The experimentation platform for agent organizations
Run a team of AI agents from text files. Change one line. Run it again.
An agent is a Markdown file, a team is a folder, and the org chart is the directory tree. No Python and no framework to learn. Edit the prose, run one command, and every prompt, reply and cost is kept.
pip install teamrunner
Mac, Windows and Linux. Works with OpenAI, Anthropic and free local models through Ollama.
hiring_committee/ ├── team.yaml ← this folder is a team: the job and the packet ├── chair.md ← how the meeting is run ├── chen.md ← a member: settings on top, prompt below ├── diaz.md ← each knows something the others don't ├── okafor.md ├── clerk.md ← records the decision ├── models/ │ └── member.yaml ← one model for the whole committee └── runs/ ← every run, logged here
A question from the study of groups
When the key facts are spread across a committee, does it pick the best candidate?
In 1985 Stasser and Titus showed that human groups mostly discuss what they all already know, and so miss the facts that would change their decision. It is one of the most replicated findings about how groups decide. With TeamRunner you can ask whether a committee of AI agents does the same, and what fixes it.
Three members, three candidates, and facts no one person has
The committee is hiring a Director of Operations for a food bank. Everyone has read the same packet, which makes Avery look best. Each member also holds two facts the others don't.
Alone, each member would still choose Avery. With all six facts on the table, Blake is the clear choice.
chen.md reference calls
Avery Former manager would not say whether they would rehire.
Blake “The best operations person I have worked with.”
diaz.md finance office
Avery Part of one under-budget year came from a one-time grant.
Blake Cost per meal delivered fell 18% in two years.
okafor.md human resources
Avery Team turnover 28%, against 18% across the sector.
Blake Regular volunteers grew from 250 to 400.
Change how the meeting is run, in one line
A well-known fix from the research is to have everyone report what they know before anyone gives an opinion. In TeamRunner the way a meeting runs lives in one file, chair.md. Moving one # switches from open discussion to share-first. Nothing else changes.
Run both versions. Then run them fifty times.
Every turn records whom each member would hire right now, so you can see when minds change and whose facts changed them. One run is an anecdote, because models are random. With experiments, one file describes both versions and the number of runs, and you get one CSV for R, Stata or pandas.
teamrunner new committee \ --from hiring_committee teamrunner run committee teamrunner runs
base: ../hiring_committee
factors:
meeting:
- {label: open, patch: {}}
- label: share_first
patch:
chair: file:variants/share_first.md
replications: 25
seed: 42
Why it is needed
Teams of agents are a new laboratory for studying organizations. It has been built for programmers.
Built for the people who study organizations
Management scientists and behavioural economists know how structure changes a team's output. They shouldn't have to write hundreds of lines of orchestration code to ask their first question.
Structure is a one-line change
Who reports to whom, who sees which facts, who decides. Each is a value in a file, and anything in a file can be varied in an experiment.
One run is an anecdote
Most published multi-agent results come from a single run. TeamRunner seeds what it controls, replicates what the model makes random, and records everything.
A study you can send someone
A team folder is a replication package. Zip it, put it in git, cite it in a methods section. Anyone can rerun it with the exact version you used.
Concepts
Five ideas cover the whole format
An agent is a Markdown file
Settings at the top, the prompt as prose below. Edit it in any text editor.
judge.mdA folder is a team
Any folder with a team.yaml is a team. A team inside a team is a subcommittee. The file browser is the org chart.
team.yamlEvery team has a chair
Someone always decides who speaks next: a script you wrote, or members in name order. Every decision is logged.
chair.mdTools sit behind a toolkeeper
Like a factory's tool crib: one member holds the web search or the code sandbox and decides who may use it, on what terms.
web.mdExperiments are overlays 0.2
An experiment file sits outside the team and patches it. The team itself stays clean and runnable.
experiment.yamlInstall
Running in five minutes, even without Python
The installer, uv, fetches the right version of Python on its own. You need a terminal and an API key, or Ollama for free local models.
- Open Terminal (press ⌘ Space, type “Terminal”) and install uv:
curl -LsSf https://astral.sh/uv/install.sh | sh - Close Terminal, open it again, then install TeamRunner:
uv tool install teamrunner - Check that it works:
teamrunner new hello --from round_robin
- Open PowerShell from the Start menu and install uv:
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" - Close PowerShell, open it again, then install TeamRunner:
uv tool install teamrunner - Check that it works:
teamrunner new hello --from round_robin
- In a terminal, install uv:
curl -LsSf https://astral.sh/uv/install.sh | sh - Open a new terminal, then install TeamRunner:
uv tool install teamrunner - Check that it works:
teamrunner new hello --from round_robin
- Python 3.10 or later:
pip install teamrunner - Check that it works:
teamrunner new hello --from round_robin
~/.teamrunner/.env, a file in your home folder that is never inside a team. TeamRunner offers to put it there the first time it needs one.What every run keeps
A run folder is a full record, on your own disk
Plain files you can open, diff and query. No dashboard you have to log in to, and no agent can read or change the log.
runs/2026-10-12_143201_debate/ ├── run.json ← when, what, versions, cost ├── events.jsonl ← every turn, one line each ├── output.md ← what the team produced ├── inputs/ ← copies of files it was given └── work/ ← what its code wrote
- Every prompt and every reply
- Exactly what each agent could read, what it said, which model answered, tokens, cost and time.
- Every decision about who acts next
- Who made it and why. There are no hidden if-statements.
- Every tool call
- Who asked, whether the toolkeeper said yes, and the full result.
- The exact version that ran
- Your 2026 study can be rerun in 2028.