Minecraft PvP · trained like a language model

An AI that learns to fight
by reading games as text.

Every tenth of a second of a Minecraft fight becomes a line of tokens: what the player sees, then what they did. A small transformer is pretrained on those games the way LLMs are pretrained on the internet, fine-tuned on the best players, then trained with reinforcement learning on wins.

Pipeline

The same stages a chat model goes through, applied to PvP. Live status from the training machine.

Latest game

A top-down replay of the most recent recorded fight, drawn from the recording itself.

0:00

Training

Loss is how surprised the model is by the next token (lower is better). Action accuracy is how often its top guess for a player's next action matches what the player really did, measured on held-out games.

Loss

trainingheld-out games

Next-action accuracy (held-out)

top-1 action token

Runs

GameLM vs the engines

Settings

Four strengths, from someone new to PvP up to a bot with no human limits at all. Ratings are measured from real fights between the engine bots on the local server, on the Elo scale with human fixed at 1500.

Head to head (kills, row vs column)

Recent engine games

Data

Pretraining data comes from engine-vs-engine games at random settings on random modes. Fine-tuning data comes from real people playing on the local server.

By mode

By setting (player-games)

Machine

How it works

Text, not pixels: the model reads the game state, the same facts a player sees on screen, with nothing a player couldn't see.

One step of a game, as tokens

About 55 tokens per tenth of a second. The game header comes once at the start, like a system prompt, and tells the model who it is playing as.

selfa visible playerblocks aroundactions (what it learns to write)

Training, the LLM way

PretrainingNext-token prediction on every token of every game, states and actions, so it learns how Minecraft and players behave.
Supervised fine-tuningLoss on action tokens only, from the players worth copying.
Reinforcement learning on winsIt plays the engines on the local server and is rewarded for winning fights, with a pull back toward the fine-tuned model so it stays sensible.
Live playTen decisions a second. Each one, it writes its 7 action tokens one at a time: movement, jump, sprint, sneak, turn, look up or down, click.

Model

TypeDecoder-only transformer
Size10.8M parameters · 6 layers · width 384 · 6 heads
PartsRMSNorm, rotary positions, SwiGLU, tied embeddings
Context1,024 tokens, about 2 seconds of play with full detail
Vocabulary397 tokens, each one a binned game fact
OptimizerAdamW (0.9, 0.95), warmup + cosine, clip 1.0
SpeedAbout 40 ms per decision on a laptop

Fair play

Only what a player could seeOther players appear in its input only when they are in front of it or very close, with a clear line of sight. No seeing through walls.
Real game rulesMovement, reach and attack cooldowns are the real server's. The all-out setting has no human limits, but it still cannot do anything the game itself doesn't allow.
Own server onlyIt plays on a local test server, and only there.

Lab log

What changed, in order, newest first.