Did the NPC Establish a Useful Minecraft Work Point?
This write-up describes the project state completed on June 30, 2026.
The larger research question is still open. The project has not yet picked a final claim such as action-consequence prediction, social behavior, or model comparison. The current work is more basic: make a natural Minecraft village setup where those questions can later be tested without relying on a staged toy world.
For this check, I ran four provider-backed model lanes through the same small Minecraft LLM-agent harness: two ModelScope lanes using Qwen Plus and Qwen Max, plus OpenAI and Gemini reference lanes.
Outside ModelScope, I stayed within available free-tier or low-quota access. I did not run frontier paid comparisons such as GPT-5.5 or Claude Opus 4.8 for this post, so the results should be read as a harness check, not a frontier-model ranking.
This is not a leaderboard, not a broad model claim, and not a decision about whether interaction history can beat an LLM prior. The point was narrower: check whether the current environment can support concrete Minecraft work, repeated screenshots, inventory review, and blocker analysis across multiple model lanes.
Minecraft natural village - environment check
One practical Minecraft goal, four attempts to reach it.
Each lane started from the same natural village scenario. The actor was asked to create a useful village-adjacent work point: collect wood, craft basic materials, make or use a crafting table, recover from movement blockers, and continue toward simple material work instead of only describing a plan.
Experiment Setup
The run used natural-village-spawn-v1 with seed 4167799982467607063. Each model
received a fresh world lane, 30 cycles, the report visual profile, and Minecraft
server version 1.21.4 for prismarine-viewer screenshots.
The actor-facing objective was intentionally practical:
From a fresh natural village-adjacent start, establish a useful work point for
continued material work. Collect wood, craft basic materials, make or use a
crafting table, recover from blockers, and leave the world in a state that can
be reviewed from inventory, screenshots, and runtime records.
This is why the run looks like basic Minecraft work instead of an explicit social-simulation demo. The aim was to check whether the environment can support later research questions, not to declare the research target solved.
Screenshots are included because they make the run easier to inspect. They are not used by themselves to decide block identity or material progress. For those claims, I checked inventory, observe/world-state scans, runtime status, and action records.
Practical Goal State Over Time
C00Village start
C02Wood acquired
C03Planks/sticks
C04Table step
C30Pickaxe retainedC00Village startC02Wood acquiredC03Table stepC18Pickaxe retained
C306 cobblestone!C30Camera caveatC00Village startC06Wood acquiredC12Materials craftedC30Table heldC00Village startC03Wood acquiredC05Table step!C15Recovery stalls
C30Sticks/planks retainedThe graph does not reward repeated actions. It only moves when the actor leaves behind a more useful material state: wood, basic materials, a table step, a retained tool, or stone follow-through. That is closer to the question this run actually asks: did the NPC autonomously leave the village-adjacent world in a more useful state than it found it?
Model Cards
ModelScope
Qwen Plus
Qwen Plus via ModelScope API
Reached a usable wood-level work point.
It built the wood -> table -> pickaxe chain, then ended without stone follow-through.
- Highest goal state
- Tool retained
- Final material state
- Crafting table progress plus wooden pickaxe retained; no cobblestone.
- Blocker recovery
- Some blocked/no-progress cycles remained, but the final material state stayed useful.
- Cost footprint
- 38 provider records, 1.13M tokens
- planksx3
- logsx3
- sticksx2
- wooden pickaxex1
Cycle diagnostics
- Observed progress
- 12/30 40%
- Blocked
- 8/30 27%
- No progress
- 10/30 33%
ModelScope
Qwen Max
Qwen Max via ModelScope API
Reached the deepest practical goal state.
It kept the wood -> table -> pickaxe chain and continued into stone material work.
- Highest goal state
- Stone follow-through
- Final material state
- Wooden pickaxe retained and 6 cobblestone recorded at the end.
- Blocker recovery
- It still had blocked cycles and a final camera caveat, but the material chain continued.
- Cost footprint
- 38 provider records, 1.09M tokens
- planksx1
- sticksx6
- wooden pickaxex1
- cobblestonex6
Cycle diagnostics
- Observed progress
- 15/30 50%
- Blocked
- 8/30 27%
- No progress
- 7/30 23%
OpenAI API
GPT-5.4 mini
gpt-5.4-mini
Reached materials and held a table, but did not settle the work point.
It gathered useful materials and held a crafting table, then failed to close the table/work-point state.
- Highest goal state
- Crafting table
- Final material state
- Crafting table in inventory, but no retained tool or stone follow-through.
- Blocker recovery
- Movement recovery displaced the practical goal after early crafting progress.
- Cost footprint
- 204 provider records, 1.12M tokens
- crafting tablex1
- logsx3
- planksx6
- sticksx4
Cycle diagnostics
- Observed progress
- 8/30 27%
- Blocked
- 9/30 30%
- No progress
- 13/30 43%
Gemini API
Gemini 3.1 Flash Lite
gemini-3.1-flash-lite
Reached table-level progress, then stalled around recovery.
It reached planks, sticks, and a table step, but the second half was dominated by blocked recovery.
- Highest goal state
- Crafting table
- Final material state
- Logs, planks, and sticks retained; no tool or stone follow-through.
- Blocker recovery
- Blocked movement and generated-action rejection dominated the second half.
- Cost footprint
- 67 provider records, 1.71M tokens
- logsx1
- planksx4
- sticksx8
Cycle diagnostics
- Observed progress
- 11/30 37%
- Blocked
- 19/30 63%
- No progress
- 0/30 0%
Did The NPC Reach The Practical Goal?
The useful result is not that one provider "won" a benchmark. The practical question is whether the actor made the natural village more workable for continued material work.
- Qwen Max reached the deepest final state: table step, retained wooden pickaxe, and 6 cobblestone.
- Qwen Plus reached a stable wood-level work point with a retained wooden pickaxe, but did not continue into stone.
- GPT-5.4 mini gathered useful materials and held a crafting table, but did not settle the work point.
- Gemini 3.1 Flash Lite reached table-level progress, then spent much of the run stuck around recovery.
Provider usage still matters, but only as a cost caveat. It should not be read as the research outcome:
| Lane | Highest practical goal state | Provider records | Total tokens |
|---|---|---|---|
| Qwen Plus | Tool retained | 38 | 1.13M |
| Qwen Max | Stone follow-through | 38 | 1.09M |
| GPT-5.4 mini | Crafting table held/unresolved | 204 | 1.12M |
| Gemini 3.1 Flash Lite | Crafting table step, recovery stalled | 67 | 1.71M |
What The NPC Actually Managed To Do
The useful result is modest: the harness can now produce model-driven material work in a natural village without a hand-staged toy arena. The records show logs becoming planks and sticks, table progress appearing, tools showing up in inventory, and one lane reaching cobblestone.
That does not make this a general statement about Minecraft competence or about a provider family. It only says the environment is now useful enough to design sharper natural-village runs where material behavior is the input to a harder question, not the claimed research result.
Important boundary: this run mostly evaluates one-actor material work continuity in a natural village. It does not prove social emergence, long-run settlement behavior, or a learnable action-consequence target beyond LLM prior.
How I Read The Evidence
The model cards show screenshots because they help humans inspect the run. I did not use pixels alone to decide progress. Material claims in this post come from inventory, observe/world-state scans, runtime status, and action records.
That matters most for the Qwen Max lane: the final third-person camera produced a known terrain cross-section artifact, so I use the cleaner cycle-20 image for visual context while treating the stone result as an inventory and world-state claim. The same rule applies to every lane: screenshots are context, not ground truth.
What Still Failed
The failures are more useful than another broad model comparison. They point to the next experiment design:
- movement recovery can dominate after early crafting progress;
- table crafting and recipe handling still create friction;
- generated action-skill candidates can be rejected before a Minecraft action executes;
- the objective mixed workbench, navigation recovery, material continuity, and longer follow-through into one loose task.
For the next version of this experiment, I would make the target sharper:
Place a crafting table and maintain a village-adjacent work point with bounded
material follow-through.
That would still produce useful material records without mixing workbench, shelter, storage, navigation recovery, and long-horizon continuity into one run.
What This Does Not Prove
GEMINI_MIN_REQUEST_INTERVAL_MS=15000.Excluded lanes Gemini 2.5 Flash Lite, Gemma 4 31B, and unpaced Gemini 3.1 were not comparable completed lanes.Run Notes
This article is based on the repository state completed on June 30, 2026. The repo keeps the full HTML report, combined model-comparison data, per-lane summaries, provider preflight notes, run reports, and screenshot folders. I am not listing raw internal file paths here because the public post should explain the result rather than make readers chase local repository paths.
The main lesson is modest: the environment is now close enough to run small natural village material-work comparisons, but the research target is still being narrowed. The next runs should use sharper tasks before making claims about action-consequence learning or social behavior.