Network architecture¶
A design. This page is the plan for Coney's online play, from two friends in the original's own co-op to dedicated servers that run their own scripts. What is built, and the exact protocol, are on Network multiplayer, which owns the layers, the messages and the sessions; this page links there and does not repeat them. When a part of this design is built, it moves to that page (or to the subsystem's research page) and this page keeps only a link. Questions only a maintainer can answer are in Maintainer decisions.
Every engineer reads The networking checklist before writing a subsystem: it is short, and it is what keeps everything below possible without rewriting the game later.
Contents¶
- Where we start: the original's own two-player game, and what Coney has.
- Goals and phases: from the original's co-op over a network to scripted servers.
- The authority model: who decides what.
- Replication: network ids, snapshots, events, interest, interpolation and prediction.
- Subsystem by subsystem: what each part of the game syncs and how, and what in today's code is in the way.
- Story, missions and cutscenes in co-op.
- Scripting across the network: server and client resources, events and state.
- Security: the trust model, validation, encryption and cheats.
- Transport: ENet now, and what changes later.
- Saves and persistence.
- Tick rate: should the simulation run at 60 Hz?
- Testing.
- The networking checklist, for every engineer.
- Roadmap: the work in order.
- Maintainer decisions and Prior art and sources.
Where we start¶
The original already has two-player co-op¶
The Warriors has drop-in, drop-out two-player co-op in story mode, on one console with a dynamic split screen, and two-player co-op and versus in Rumble and Armies of the Night. That is the most important fact for this design: two players in the story are faithful, not an enhancement. The original's own code and scripts already handle a second player everywhere a network game needs them to.
| What the original does with two players | Where | Evidence |
|---|---|---|
Player 2 joins when a second pad presses START and joining is allowed (CfgMultiplayerJoin): GameState_SyncPlayers takes the player gang's next human by hand-over priority, makes him a player and calls the level's multiplayer callback (GameState_CallMultiplayerCallback). He drops out only through the pause menu's Player 2 quit; a pulled pad raises the controller error instead (Player_UnbindPad 0x0022a980) |
0x0041a460, 0x0041b148, PauseMenu_QuitPlayerTwo 0x001df3a8 (Scripting: player 2) |
confirmed (code) |
| Whether player 2 is in play is kept per mode (story, Rumble, Armies) | GameState_IsTwoPlayerActive 0x00419a80 (Front end) |
confirmed (code) |
The level scripts guard player 2's steps with HuIsAPlayer(player2) and register SetPlayer2Callback to resume, place or hide the second Warrior. Shown for level99's three scripts, the ones disassembled; the other levels are expected to follow the same pattern (not checked) |
level99.lua, level99_lesson1.lua, level99_lesson2.lua (Scripting) |
inferred (script disassembly) |
| The cameras split and merge the screen: merged when both players are in view within 11 m, else split, with 0.72 s timers and a flash | Cam_Follow_DecideSplit 0x00126c40, Cameras_LayoutViews 0x00122ed0 (Cameras) |
confirmed (code) |
| The AI knows about two players: Warriors far from player 1 defend player 2, reactions and enemy counting change in co-op | DefendTactic_GiveGoals 0x00310288, WarriorFollowTactic_GiveGoals 0x00310f38, HideTactic_RegiveMember 0x00313028 (AI code) |
confirmed (code) |
| Spawners place new humans where no player's camera can see them | Camera_AnyPlayerCanSeePoint 0x001202e8 (AI: spawners) |
confirmed (code) |
| The pause menu offers a co-op quit (player 2 or all) | PauseMenu_QuitPlayerTwo 0x001df3a8 (Pause menu) |
confirmed (code) |
So the first online game is the original's two-player story with player 2's pad and screen on another machine. The host simulates exactly what a PS2 with two pads would; nothing in the simulation changes, only where the second pad's samples come from and where the second view is drawn. More than two players is Coney's own, and comes later behind a session rule (Phases).
What Coney has¶
- The foundation:
coney_net(bit stream, transport interface, loopback, link conditioner, protocol, server and client sessions) and the ENet transport (Network multiplayer). - A loop built for this: a fixed 1/30 s step at any frame rate, bit for bit the same at every rate, game time
counted in steps,
render()that only draws and blends the last two steps, pads read once per step, no clock outside the frame pacer, a headless lockstep test mode (Update and render). - The game's own random numbers in one place (
GameRandom), as the original's one shared table walk. - One player. Coney does not yet build the original's two-player game; the pause menu leaves out the co-op quit
and the scripts'
HuIsAPlayer(player2)is answered for one player. That is the first blocker (Readiness).
Goals and phases¶
The goal: online story co-op with friends first, then servers that anyone can host, whose own scripts drive the game and talk to scripts on the players' machines, in the spirit of FiveM. Each phase is usable on its own and none rewrites the one before. P4 and P5 are ConeyMP, a separate later product on the same network stack (Two products, one stack).
flowchart LR
P0["P0 Foundation<br/>coney_net, ENet<br/>(built)"] --> P1["P1 Online two-player<br/>the original's co-op,<br/>player 2 remote"]
P1 --> P2["P2 Smooth and safe<br/>prediction, encryption,<br/>LAN discovery"]
P2 --> P3["P3 Dedicated server<br/>headless coney-server,<br/>2-4 players"]
P3 --> P4["P4 Scripted servers<br/>server and client resources,<br/>events, state"]
P4 --> P5["P5 Large sessions<br/>many players, routing<br/>buckets (ConeyMP)"]
| Phase | What a player gets | Faithful? | Depends on |
|---|---|---|---|
| P0 Foundation (built) | nothing visible: the library, ENet, the tests | n/a | |
| P1 Online two-player | a friend joins the story, Rumble or Armies as player 2 over LAN or a direct address; the host's game is the one true game | yes: the original's two-player game, only the pad and the view travel | the original's two-player game built offline first |
| P2 Smooth and safe | the joining player's own Warrior answers at once (prediction); the session is encrypted; a host on the same network is found by LAN discovery, one across the internet by its address and a forwarded port | yes (presentation and transport only) | P1 |
| P3 Dedicated server | a server program with no window hosts a session; players join and leave at any time; three or four players | two players: yes; more: no, Coney's rule set | the simulation out of src/platform/ (Readiness) |
| P4 Scripted servers (ConeyMP) | a server runs its own resources: server scripts that drive the game, client scripts that add screens and effects, events between them | no: Coney's own, a modded session | P3, the script split (Asset and script architecture) |
| P5 Large sessions (ConeyMP) | dozens of players, several worlds on one server | no | P4 |
Faithful first. The original's game is complete before anything beyond it (AGENTS.md, Rules). P1 needs no exception: the second player is the original's. P2 adds nothing to the simulation. From P3 on, everything beyond two players or the original's levels runs only in a session marked with Coney's rule set, which a test mode and a single-player game never turn on. The rule set is the one switch for every limit raised beyond the original's (Limits).
Single-player is untouched. With no session open, no network code runs and the game is the same, bit for bit, as a build without networking; a test proves it (Testing). The networking hooks in the loop cost one branch when no session exists.
The authority model¶
One machine, the server, runs the one true simulation. In P1 and P2 it is the host's own game (a listen server); from P3 it may be a dedicated server with no player of its own. The other machines are clients: they send what their players do and show what the server says.
flowchart TB
subgraph Server["Server (host's game or coney-server)"]
Sim["Simulation, 30 Hz step<br/>humans, AI, scripts, missions,<br/>physics, cameras, spawns, GameRandom"]
Repl["Replication<br/>snapshots, events, HUD state"]
Sim --> Repl
end
subgraph Client["Client (each joining player)"]
Pad["Pad"] --> Cmd["Input commands"]
Buf["Jitter buffer"] --> Pres["Presentation<br/>interpolated world, own view,<br/>HUD, sound, effects"]
Pred["Prediction (P2)<br/>own Warrior only"]
end
Cmd -- "unreliable, redundant" --> Sim
Repl -- "snapshots: sequenced<br/>events: reliable or not" --> Buf
Repl --> Pred
| The server owns | The client owns |
|---|---|
| every human's position, state, health and animation; every brain, gang, formation and goal | its pad, and nothing else of the game |
| every script (the original's level scripts and, from P4, server resources) | its client resources (P4): screens and effects, never game state |
| missions, checkpoints, flags, story progress, the save | its options, its display, its audio mix |
GameRandom and every gameplay draw |
its own cosmetic random numbers, seeded per client |
| physics, collision, thrown objects, doors, props, cars, spawns | particles and decals from the server's events |
| every player's camera (the simulation reads cameras: spawners, scripted shots, the split decision) | drawing its player's view from the server's camera, smoothed; P2: its own camera predicted for feel |
| HUD state: messages, objectives, counters, radar blips, prompts | drawing the HUD from that state |
| when a cutscene or movie starts and ends | playing it, and saying when it has finished or been skipped |
Why server-authoritative and not peer-to-peer lockstep or rollback:
- The world is large and busy. Up to 60 humans (the original's human pool, Characters), brains, gangs, scripts and streaming. Lockstep makes every machine wait for the slowest and needs bit-exact floats across compilers and platforms; rollback must copy and re-simulate all of that several times a frame. Both are right for a fighting game with two characters (GGPO), not here.
- Joining late. Snapshots let a player drop in mid-level, as the original's player 2 does; lockstep needs the whole history.
- The trust model. One authority decides every hit, item and mission step; a client that lies about its position or its money has nothing to lie with (Security).
- One code path. A listen server and a dedicated server are the same session code; only the front end differs (Model).
Determinism is still kept, but for other reasons: replays, the tests, prediction (a client's re-simulation of its own Warrior should match the server's on the same inputs) and the runtime comparison. Nothing depends on two machines computing the same floats.
Replication¶
Network ids¶
Nothing crosses the wire as a pointer, an index into a pool or a script handle. Every replicated thing gets a
NetId from the server: 16 bits of slot and a few bits of generation, so an id reused after a human dies is not
mistaken for the old one. Each side keeps a table from NetId to its own handle. The server assigns ids when a
thing becomes replicated (spawned, or a placed object first touched) and frees them when a client has acknowledged
its removal.
| Kind | Identified by | Notes |
|---|---|---|
| Humans (players and AI) | NetId |
the largest share of every snapshot |
| World objects (doors, props, glass, pickups, loose objects) | NetId, mapped from the object's script handle |
replicated only once changed from the level's data |
| Cars | NetId |
|
| Cameras | the player's slot, and the camera's kind | one active camera per player |
| Players | slot 0-3 (the host's player is 0, as the original's player 1) | from the handshake (Sessions) |
| Sounds, particles, messages | none: they are events | |
| Assets (clips, models, sounds, texts) | the game's own name hashes and ids, the same on both sides | both sides load the same disc |
Both sides must load the same simulation data: the handshake compares the store's simulation digest (Asset and script architecture) and refuses a client whose disc region, executable or simulation overrides differ.
Three kinds of data¶
Following Halo Reach's split (sources), each piece of data travels as one of three kinds, each with its own delivery:
| Kind | What | Delivery | Example |
|---|---|---|---|
| State | what a thing is now; only the newest matters; eventually consistent | snapshots: UnreliableSequenced, delta against an acknowledged baseline | a human's position, heading, clip and health; a door's angle |
| Events | something happened at a step; cosmetic ones may be lost, important ones may not | Unreliable for cosmetic (a spark, a footstep), ReliableOrdered for the rest (a scene starts, a message shows, a mission ends) | SoundPlayed(id, emitter, step), SceneStarted(scene, step) |
| Control | the session itself | ReliableOrdered | join, leave, ready, load level, pause |
Input commands go the other way (Input).
Snapshots¶
Quake 3's model (sources): the server keeps, for each client, the last 32 snapshots it sent; each new snapshot is written as a delta against the newest one that client has acknowledged, or against nothing (a full snapshot) when there is none. Loss costs nothing to repair: the next delta is just against an older baseline.
- Per entity: a "changed" bit, then a changed bit per field group. Fields are declared once in a table (name,
bits, range), the way Quake 3's
netField_tand our bit stream quantized floats work, so the encoder, the decoder and the tests read one definition. - Quantization: positions to 1 cm in the level's bounds (18 bits a coordinate in a 2 km level), heading to 10 bits, health to its own range, a playhead to the clip's frame count.
- Budget: one snapshot fits one datagram of at most 1,200 bytes of payload; what does not fit waits, ordered by a priority accumulator.
- Rate: every step (30 Hz) on a LAN, 15 Hz as the default across the internet, chosen per client from its link quality (Link quality).
A rough budget, to show it fits: a moving human is about 15-25 bytes as a delta (position, heading, clip and playhead, a few flags), a still one 1-2 bytes. Sixty humans of which twelve move near a player: about 340 bytes a snapshot, about 10 KB/s at 30 Hz per client, 82 kbit/s. The host's upstream for three clients is about 250 kbit/s.
Interest management¶
Not everything goes to everyone. A client gets an entity when it is relevant to its player:
- Always: every player, the player's own gang, everything a running scene or the mission names.
- By distance: within 70 m of the client's player, the radius the original already uses to spawn objects around a
player (
ObjectTaskManager_UpdateSpawns, Tasks), on a grid of level cells. - Never: what the client cannot see or hear and no rule above keeps.
Within the relevant set, each entity's priority grows every step it is not sent (faster when near the player, on screen, or in a fight with him) and resets when it is sent; the snapshot fills in order of priority until the budget is spent. Far humans thus update a few times a second and near ones every snapshot, with no rule naming a rate.
With two to four players in one level, most of the level is relevant to someone; interest matters for bandwidth on the internet and for large sessions (P5), where routing buckets, FiveM's separate worlds on one server, keep groups of players apart.
Interpolation¶
A client draws everything the server owns in the past: a jitter buffer of snapshots, and the world drawn between
two of them, about 100 ms behind the newest (three snapshots at 30 Hz, two at 15 Hz), so one lost snapshot still
leaves two to blend. This is the same blend Coney already does between the last two steps
(Update and render): Interpolated<T> and DrawnPoses get their two ends from
the jitter buffer instead of the last two steps. A thing that jumps (a teleport, a new clip) is marked in the snapshot
and drawn unblended, as reset() already does offline.
Input commands¶
A client sends its pad, never its result. Each step's command is the pad sample the simulation would read on that step (Input: commands): buttons, pressures and both sticks, tagged with the step it belongs to. Each packet repeats the last several commands, so a lost packet costs nothing. The server keeps a short buffer per client and applies each command on its step to that player's pad record, the place where the original's per-player record already reads a pad; from there on, the server's game cannot tell a remote pad from a local one. This is the step in progress on pad port 2 (The plan).
If a command is late, the server repeats the last one (a held button stays held), and tells the client to send a little earlier; if commands pile up, a little later (Overwatch's command-frame tuning, sources).
Prediction (P2)¶
Until P2, a client sees its own Warrior after a round trip: fine on a LAN (a few ms), sluggish across the internet. P2 adds prediction for the client's own player only: the client runs its Warrior's step from its own commands at once, keeps the commands the server has not acknowledged, and when a snapshot arrives resets its Warrior to the server's state for that step and replays the rest. Small differences are blended away over a few frames; large ones snap. Everything else, including every AI human, is interpolated and never predicted.
Prediction needs the human's step to run on a client against a partial world (the other humans as the snapshot has them), which is the hardest requirement on the human code. Hits are never predicted as decided: a client may play its own swing and a spark at once, but whether it hit and what it did is the server's, checked against what the attacker saw (lag compensation).
Lag compensation¶
With interpolation, a client sees the others about 100 ms plus half its round trip in the past. In a brawler the question "did my punch connect" is settled by the server at the step it runs the attack's sweep. P2 adds Source's answer (sources), restricted to strikes: when a remote player's attack reaches its hit frames, the server tests it against the targets' positions as that client drew them (rewound by the client's interpolation delay and round trip, at most 200 ms), then restores them. Movement, grabs and throws are never rewound.
Subsystem by subsystem¶
What each part of the game must sync, who owns it, how it travels, and what in today's code is in the way. Surveyed
on main at 96644c00a; paths are under src/. The summary first:
| Ready now | In the way |
|---|---|
| One fixed 1/30 s step, no clock outside the frame pacer, game time from the step count | One player is built into the level glue: PlayLevelMode::m_player, pad port 0, HUD panel 0, the radar, crew, hiding, tagging, revive, uncuff |
Pads sampled once per step through one InputSource and the pad record; the AI writes the same PlayerRecord |
Simulation inside src/platform/: about 4,800 lines of play_level_*.cpp mix player-1 gameplay (pickups, throw aim, mugging, object attacks) with drawing |
Script handles from one deterministic counter, and ordered std::map<double, ...> almost everywhere |
No copy or restore of simulation state; Human, Brain and Player are not copyable, and humans link to each other by raw pointer |
Per-human and per-brain CombatRandom streams; cosmetic effects on their own streams |
The shared GameRandom stream is also drawn by presentation-adjacent code (system music, radios), in subsystem order |
| A headless mode that runs the whole simulation, and input scripts | Sounds, particles and HUD changes are direct calls from the simulation, with no event that could be sent |
PlayerSnapshot and TargetSnapshot, ready-made per-human presentation state |
Tuning singletons the debug menus change live, and debug tools that change the game out of band |
Loop, time and input¶
- Sync: nothing directly: the server's step number is the clock every message carries.
- Today:
GameModeStack::runUntilEmpty(gamemodes/game_mode_stack.cpp) asksFrameClockfor 0-4 steps a frame; each step samples the pads, advances the timer and updates the top mode.--headlessruns the whole game on librw's null renderer, lockstep. Work in progress addsbeforeStepandafterStephooks and aNetInputsource that puts the guest's pad on port 2. - In the way: an overloaded
FrameClockdrops time rather than catching up. On a server that is right (the clients follow its step numbers), but a client's jitter buffer must then expect gaps in step numbers. The mode stack freezes everything under a pushed mode (pause, movie, menus); with one simulation on the server that is the original's behaviour for both players and is kept (Story).
Humans and players (human/)¶
- Sync: position, heading, the animation layers (clip id and playhead, fade), health, stamina, held object, state flags. Owner: server. How: snapshots.
- Today:
Humanowns itsPlayerRecord, animator,Fighter, stamina and body;Humans::updatesteps them in five passes;Playerbundles one human, theHumansstepper and one follow camera. OnlyPlayer::updatereads a pad; everything below sees thePlayerRecord, the cleanest seam in the code.Humansalready accepts several pad-controlled humans. - In the way: the level holds one
Player.Fighterkeeps raw pointers to its target, the held and the thrown object; snapshots need them as ids.PlayerRecord::cameraForwardcomes from that player's follow camera, so the server needs each player's camera (it simulates them, as the original does). Five tuning singletons (bodyTuning,climbTuning,jumpTuning,locomotionTuning,staminaTuning) are mutable statics.
Combat (combat/)¶
- Sync: the attack chain's state, power and rage meters, grabs, mugging, the hits. Owner: server. How: meters and grab state in the snapshot; a hit as an event (who, by whom, which reaction) for sparks and sounds.
- Today: pure functions and a
CombatInputstruct, no globals exceptcombatTuning(), randomness per human. Ready. - In the way: prediction (P2) runs a player's combat step on the client, which needs the same tuning; the tuning is then part of the session's handshake.
Animation (animation/, characters/)¶
- Sync: per human, each active layer's clip id, playhead and fade; not bones. Owner: server. How:
snapshots; the client evaluates poses from the same disc clips and blends playheads with
lerpLooping. - Today:
AnimTaskStack(up to 12 layers) insideHumanAnimator; clip events drive combat timing and root motion drives movement, so animation is simulation on the server and presentation on a client. - In the way: layers hold
const AnimClip*; replication needs the clip's id (animId), which every layer has.DynamicClipsloads clips by name on demand: a client must load a clip the first time a snapshot names it.
AI (ai/)¶
- Sync: nothing of the brains. Their results are humans, and humans replicate. Radar blips and HUD prompts the brains cause go through the HUD state. Owner: server only.
- Today:
Brainsown brains, gangs and formations; thinking is staggered by index; randomness per brain and per gang, seeded by slot; spawner placement draws fromGameRandomand reads the player's camera. - In the way:
m_playerBraininAiHumansandScriptedBrains::m_playerare single pointers, used acrossai/(21 lines nameplayerBrainorplayer_brain, counted with grep onsrc/ai). The original's two-player rules (a Warrior defends the other player) are not built.
Physics, collision and paths (physics/, raycast/, world/)¶
- Sync: nothing on its own: humans and objects carry their results. The collision mesh's triangle flags and the path map's edges change when doors open and panes break; the server owns them, and a client needs them only when it predicts (P2), from the replicated door and pane states. Owner: server.
- Today: stateless sweep kernels;
CollisionMeshandPathMapare mutated by object logic and scripts throughmutableTriangles()andmutableEdges();ObjectWorldkeeps the writes to re-apply them. - In the way: for P2 prediction, the client must apply the same writes in the same order; derived from replicated
door and pane states, not sent separately.
CollisionMesh::m_checkedis mutable scratch, so queries are not re-entrant (one simulation per thread).
Cameras (camera/)¶
- Sync: each player's current camera: kind, position, orientation, field of view, and the split decision. Owner: server (the simulation reads cameras). How: per-player camera state in that player's snapshot, interpolated; scene and scripted cameras are cameras like any other.
- Today: one
Camerasper level holding oneFollowCamera(owned byPlayer), the scripted cameras by handle, shake and slow motion. The camera feeds the simulation in four places: the stick's direction, slow motion's step, the world objects' streaming view, and the spawners' visibility test. - In the way: one follow camera; the original's per-player cameras and the split decision are not built.
followTuning()andfollowDefaults()are mutable statics. Camera shake and sway draw from their own streams, so a client can play them as presentation. - Presentation choice: the server decides the split exactly as the original; each client then draws its own player's view. Full screen (default online) or the original's split layout is a display option of each machine.
World objects (world_objects/)¶
- Sync: doors (angle, state), props and glass (broken or not), loose and thrown objects (position, rotation, holder), pickups, cars, radios, tag spots; the triggers stay on the server. Owner: server. How: snapshots keyed by the object's handle; a break or a pickup also as an event for its sound and particles.
- Today:
SpawnRecords,LevelObjects,LooseObjects,Pickups,Carsand the rest, keyed bydoublehandles in ordered maps (the best replication keys in the code); doors and props tick at 60 Hz, two ticks a step. - In the way: doors, glass, props and radios draw from the shared
GameRandomthroughObjectWorld::random: fine on the server, so a client must never run that logic itself. Player-specific object code (carrying, the object attack) lives inPlayLevelModefor player 1.
Effects (effects/)¶
- Sync: nothing of their state. Owner: each machine. How: events (an object broke, a hit landed, a script started particle system n at a place) start them on the client.
- Today: particles, glints, fog, litter, smoke and light tasks each have their own fixed-seed stream; nothing in
human/,ai/orcombat/includeseffects/. Ready. - In the way: particle starts are calls from simulation code, not events; they need the one call the checklist asks for.
Scripts (scripting/)¶
- Sync: nothing of the Lua state; its effects. Owner: server only (Scripting).
- Today: one Lua 4.0 state per level; 632 of the original's 956 natives built; bindings hold about 28 pointers
into the game through
BindingContext; callbacks by name (message handlers, pad handlers, anim callbacks, triggers,ScheduleFunc); handles from one counter per state. - In the way: the Lua state is game state no snapshot can copy, which is why scripts stay on the server. The
original's two-player natives do nothing yet:
CheckMultiplayeris a no-op (scripting/hub_world_bindings.cpp), andSetMultiplayerCallbackandCfgMultiplayerJoinonly store their callback name and flag (scripting/player_bindings.cpp). Natives that show something on screen or play a sound must go through the HUD state and sound events.
Missions, story and saves (warriors/, gamemodes/)¶
- Sync: the level and checkpoint, mission success or failure, per-player inventory and statistics (two slots already), money. Owner: server. How: control messages for level loads and endings; per-player state in that player's snapshot.
- Today:
GameStateowns the level table, checkpoint,GameRandom,PlayerState(inventory, stats, two slots), story and hub state, and thetwoPlayersflag (set by the front end, read by nothing in play). Level jumps happen inbeginFrame, outside any step. The save is the profile record only. - In the way: the two-player game state paths are unbuilt; level changes need a load barrier (Story).
HUD, menus and pause (hud/, gui/, gamemodes/)¶
- Sync: each player's panel (health, meters, money, items), messages, objectives, counters, radar blips, prompts. Owner: server. How: a per-player HUD state in that player's snapshot, and messages as reliable events; the client draws it.
- Today:
HUDhas two player panels, of which only panel and radar 0 are driven. Menus read pad port 0; the pause menu leaves out the co-op quit. - In the way: panel 1 and the second radar; a menu driven by whichever player opened it (the original's pause stores the player).
Audio (audio/, platform/audio_output)¶
- Sync: what plays: sound id, emitter (
NetIdor position), start step; music changes; the listener is each machine's own player. Owner: server decides, client plays. How: events, unreliable for one-shots, reliable for music and loops. - Today: the simulation calls
GameSoundand the sound engine directly; the listener is player 1; system music draws fromGameRandom, and the sound engine's randomisation calls back into it. - In the way: no sound event can be recorded today; the music's draws must stay on the server, with the client playing what the server picked.
Movies and scenes (movies/, scenes/)¶
- Sync: scene start and end (its roles are humans, its camera a camera); movie start, skip and end. Owner: server. How: reliable events (Story).
- Today:
SceneSystemsteps inside the simulation and holds the player;MovieModestops the game clock and any pad's button skips it. - In the way: nothing beyond the events and the barrier.
Debug tools (debug/)¶
- Today: the Lua console, the native caller, teleport, time control and the tunables change the game outside the step.
- Plan: in a session they run on the server only (the host's debug menus), go through the step like any input, and are refused on a client; time control stays local to an offline game. A session shares the server's tunables in the handshake.
Story, missions and cutscenes in co-op¶
The rule for P1 and P2 is the original's own: the host's game is the story, and the second player is a guest in it, exactly as player 2 on one PS2 is. Every decision the original makes for "any player's pad" is kept, with the pad now possibly remote.
| Moment | What the original does with two pads | Online (P1, P2) |
|---|---|---|
| Joining | player 2 presses START when joining is allowed; GameState_SyncPlayers makes the gang's next human a player and calls the level's callback |
a client connecting is a second pad coming in: its first command with START is the same press, through the same SyncPlayers path. Until joining is allowed, the client waits in a lobby view |
| Leaving | player 2 drops out only through the pause menu's Player 2 quit, which releases the human and calls the callback to hide him; a pulled pad puts up the controller error and waits (Scripting: player 2) | a client choosing to leave is the Player 2 quit, on that step. A client that drops or times out is not a pulled pad on the host (the controller error would stop the host's game): Coney's choice is to run the same Player 2 quit for it, so the host plays on alone |
| Pause | either player's START pauses the whole game (Pause) | the same: a client's START pauses the host's game, both see the menu, the pauser's pad drives it. The co-op quit's Player 2 entry disconnects the guest |
| Scenes (in-engine cutscenes) | the scene runs in the simulation; either player's cross or START skips a skippable scene after 2 s (Scenes: skipping) | the scene runs on the host as any simulation; the client sees it through snapshots and its view's scene camera. Either player skips, as in the original |
| Movies (Bink FMV) | the game stops while a movie plays; any button on any pad skips a skippable one (Movies) | a reliable MovieStarted(name, skippable) event; each machine plays its own copy from its own disc; any player's skip ends it for both; the host resumes when its movie has ended and every client has said it has, or after a time-out |
| Checkpoints, mission fail, restart | the host's game decides; both players restart | the same, sent as reliable control; the client reloads with the host |
| Level change | the next level loads | reliable LoadLevel(level, checkpoint); the server waits until every client has loaded (a barrier with a time-out that drops a client who never finishes) before the first step |
| Story progress, unlockables, the save | the one profile on the memory card | the host's profile only (Saves) |
Scene and movie timing. Both start on a server step; a client that joins mid-scene starts drawing it from the current snapshot (the roles are humans like any other, the scene camera is replicated as the player's camera), so there is no separate "cutscene sync".
P3 and beyond (Coney's rule set). With more than two players, the original's two-player rules decide nothing for
players 3 and 4. Coney's own rules, to design with the maintainer before P3: which human each extra player takes
(the gang's next by hand-over priority, as Gang_PickNextPlayer does for player 2, is the natural extension), what
the AI's two-player rules do with more (a Warrior defends the nearest player), and how cameras decide views when there
is no shared screen (every client always has its own view; the split decision is skipped). A skip in that rule set
may become a vote. None of it changes a one- or two-player session.
What the original reads from cameras stays simulation. Spawners place humans where no player's camera sees them and the split decision depends on both players being in view. The host therefore simulates both players' cameras exactly as the original does, split decision included, even though each machine then shows only its own player's view. Whether a client draws its view full screen or the original's split layout is a presentation choice (Cameras).
Scripting across the network¶
The original's scripts (P1-P3)¶
The original's level scripts are simulation: they spawn humans, move the story on, start scenes, set the HUD. They
run on the server only, in the one Lua 4.0 state per level Coney already has (Asset and script
architecture). A client runs no level script; what the scripts change
reaches it as replicated state and events like anything else. That keeps one script run, so GameRandom draws, the
order of natives and every callback are exactly the original's.
Three consequences for script bindings, today:
- A native whose effect a player sees on a client must leave a trace the replication can carry: a state change on a replicated thing, or an event. A native that draws straight to the screen or plays a sound directly (bypassing the HUD state or the sound events) cannot be seen by a client. The checklist asks for this.
- A native that reads a player's state (
HuIsAPlayer, the pad, the camera) must take the player's slot, not assume player 1; the original's take a handle or a player number already. - Handles that scripts hold are the server's own; they never cross the wire (Network ids).
Coney resources (P4)¶
From P4 a server runs resources, FiveM's idea in Coney's terms: a folder with a manifest, server scripts, client scripts and data. They are Coney's own content, so a session that runs any is a modded session (Asset and script architecture) and never a faithful one.
flowchart LR
subgraph Server
SR["Server resource<br/>Lua 5.4 sandbox"] -- "natives: game state" --> G["Simulation"]
SR -- "writes" --> RS["Replicated state:<br/>entity, player, global"]
end
subgraph Client
CR["Client resource<br/>Lua 5.4 sandbox"] -- "natives: screens, effects" --> UI["Presentation"]
end
SR -- "events, server to client" --> CR
CR -- "request events, validated,<br/>rate-limited" --> SR
RS -- "replicated, read-only on clients" --> CR
| Piece | Rule |
|---|---|
| Manifest | names the resource, its version, its server, client and shared files, the events it accepts from clients with their argument schemas, its dependencies |
| Runtime | a modern Lua (5.4) beside the original's 4.0, behind the same native API (Asset and script architecture); one state per resource, never sharing globals |
| Server natives | the game's own: spawn, move, give, set mission state, start scenes, send events. Only server scripts have them |
| Client natives | presentation only: draw screens, play local sounds and effects, read replicated state, send a request event. No client native writes game state |
| Events | named (resource:name) with a declared argument schema; a client-to-server event the receiving resource has not declared is dropped; the server always knows the sender's slot, which a script cannot forge; a client-to-server event carries a request ("buy item 3"), never a result ("I have 500 dollars") |
| Replicated state | key-value state per entity, per player and global, written only by the server (FiveM's strict mode as the only mode), read on clients, with change callbacks |
| Large data | a rate-capped reliable stream for big payloads (FiveM's latent events), so one resource cannot flood the reliable channel |
| Client files | everything in a client script is public: it is downloaded and runs on the player's machine. Secrets live in server scripts |
Sandboxing¶
Both sides run code they did not write: a server runs its operator's resources, a client runs what a server sends
it. Every resource state opens only Coney's allowed libraries (no io, os, debug, package, no loading of
bytecode, no dofile or require outside the resource), has its own environment over a read-only API table, an
instruction budget per call (a count hook that aborts the call) and a memory cap (an allocator that refuses growth
past it). A client script can never open a file, a socket or another resource's state. The original's Lua 4.0 state
has none of these limits and needs none: it runs only the disc's scripts.
Security¶
The trust model¶
| Who | Trusts | Does not trust |
|---|---|---|
| Server | its own simulation and its own resources | any byte from a client: every message is decoded strictly, checked against ranges and rates, and read only as a request |
| Client in P1-P2 (friends) | the host's game, as a player trusts the friend holding the other pad | anything else on the network |
| Client in P3+ (public servers) | the server's game state | the server's client scripts: they run sandboxed and can only draw, play and request |
| Everyone | other clients: a client never talks to another client and never learns another client's address |
The topology is a star: every client talks only to the server. No client learns another's address, which removes the class of attacks on other players' connections that peer meshes such as GTA Online's suffer (sources).
What a client can try, and the answer¶
| Cheat class | Possible? | Why not, or how it is limited |
|---|---|---|
| Teleport, speed, fly, god mode, infinite health | no | the client sends a pad; the server moves the human and keeps the health |
| Hit from afar, hit through walls | no | the server runs every sweep; lag compensation rewinds at most 200 ms and only the targets' positions |
| Item, money, flag or mission injection | no | no message sets state; requests are checked against the game (FiveM's classic amount exploit has no message to ride) |
| Impossible input (a stick past its range) | no | commands are clamped to the pad's range; a pad can only do what a pad can do. Turbo and macros remain possible, as on a PS2 |
| Seeing what is hidden (wall hacks) | partly | a client gets only what is relevant to it, but anything within 70 m is on the client. Accepted for co-op |
| Combat bots | partly | a bot plays through a pad; the protocol cannot prevent it. Out of scope for co-op |
| Flooding: packets, events, chat | limited | per-client rate limits on every message kind; over the limit is dropped, far over it is a kick |
| Malformed packets, decoder exploits | no (by design) | strict decoding, every read checked, limits checked before allocating, fuzzed (Testing) |
| Spoofed source address, amplification | fixed in P2 | ENet's handshake is unauthenticated; P2 adds a challenge before any state is kept, and replies never larger than requests |
| Eavesdropping, tampering | fixed in P2 | P2 encrypts and authenticates every packet |
| A malicious server | limited from P4 | sandboxed client scripts with presentation-only natives; downloads are scripts and data under size limits, never native code |
| A malicious host in P1 | yes | the host's game is the story: a host can cheat in his own game as on his own PS2. Accepted |
Encryption and authentication (P2)¶
ENet sends plain UDP and its connect handshake is not authenticated. From P2, a challenge comes before the server keeps any state for a peer, and every later packet is encrypted and authenticated with libsodium (Encryption and the connect challenge, issue #561). ConeyMP servers (P4) authenticate players the way their admins choose (passwords, allow lists); Coney runs no account or token service.
Limits a client cannot raise¶
Every pool the original sizes for one level's play (brains, gangs, formations, routes, enemies, shards) keeps the original's size in a faithful session. Raising one belongs to Coney's rule set only (Asset and script architecture, memory hardcodes). A server sets its rule set; a client cannot ask for more.
Transport¶
Decided: ENet over UDP stays the transport, with libsodium for encryption and a connect challenge, behind the
Transport interface (Transport and delivery). Players connect by address or by LAN
discovery; across the internet the host forwards one UDP port (27615 by default). Coney runs no services: no
signaling, no relay, no server list.
Why ENet: it has the least delay a UDP transport can have, it is small, and it is what FiveM itself is built on (FiveM
vendors its own copy of ENet, vendor/citizen_enet in its repository). The
alternative studied, Valve's GameNetworkingSockets (BSD-3), would have brought built-in encryption and hole punching;
with joining by address and a forwarded port, hole punching is not needed, and encryption is a small layer of our
own over libsodium. Nothing above Transport knows which library carries the packets, so a later change of library
touches one file in src/platform/ and its tests.
Joining¶
| How | Where | What the host does |
|---|---|---|
| LAN discovery | the same local network | nothing: the host answers a broadcast on the local network with its name, players and protocol; the guest picks it from a list (issue #562) |
| Direct address | anywhere | gives the guest an address (IPv4, or IPv6 where both have it) and, across the internet, forwards the UDP port on the router once |
UPnP or NAT-PMP (the host's game asking the router to forward the port itself) would make the second row easier; it is not planned and would be its own decision.
Encryption and the connect challenge¶
On top of ENet, with libsodium (ISC licence; issue #561):
- Challenge before state. A new peer's first packet gets a small challenge back (a token the server can check without remembering it); only a peer that returns it is given a slot. A spoofed source address therefore costs the server nothing, and no reply is larger than the packet that caused it (no amplification).
- Key exchange. X25519 in the handshake; each side derives a key per direction.
- Every later packet sealed with an AEAD (XChaCha20-Poly1305), the packet's sequence as nonce, and a replay window. A packet that fails is dropped and counted; many failures close the connection.
- An optional session password, checked inside the encrypted channel.
The handshake's Hello keeps its magic and version first, unencrypted, so a client of another protocol is still told "version mismatch" (Protocol).
Two products, one stack¶
Co-op and ConeyMP share one network stack and are kept apart by protocol:
| Coney co-op | ConeyMP | |
|---|---|---|
| What | the original's two-player game online (P1, P2), later Coney's three and four players (P3) | a separate, later product in the spirit of FiveM: dedicated servers running server and client scripts (P4, P5) |
| Hosting | a player's game hosts; guests join by LAN discovery or address | port-forwarded dedicated servers running all the time, with admins |
| Shared | transport, encryption, network ids, snapshots, input commands, interest, the tests | the same |
| Kept apart | its own protocol id and version in Hello | its own protocol id and version |
A co-op client cannot join a ConeyMP server or the other way round: the server answers a Hello of another protocol id with a Reject, as it does a version mismatch. Finding ConeyMP servers (a server list) is a later, separate decision.
Other transport needs¶
| Need | When | Plan |
|---|---|---|
| IPv6 | P2 | ENet 1.3.18 is IPv4; the enet6 fork (SirLynix/enet6, MIT) adds dual stack, or our own patch |
| MTU | P1 | payloads of at most 1,200 bytes, below every common path's MTU, VPNs and IPv6 included |
| Bandwidth control | P1 | the snapshot budget and rate per client from link quality |
| Authentication of players on ConeyMP servers | P4 | a server admin's own choice (passwords, allow lists); no Coney service |
Saves and persistence¶
- The host's profile is the story. In P1-P2 the host's game saves to the host's profile, as the original's one memory card does (Profiles and saving). The guest's own profile is not touched: no story progress, unlockables or statistics move to it, which is what the original gives player 2.
- Later, in Coney's own store (never the original's profile record): a guest's co-op record (levels finished as a guest) may be kept, never merged into the guest's own story. A maintainer decision before it is built.
- A dedicated server's world (P3+) saves on the server, in the server's own folder, never on a client.
- Resources' data (P4) is saved by the server in a key-value store per resource, never by a client.
- Copying the simulation (save and restore the whole state in memory) is a checklist item for every subsystem: it gives late join a full state and tests a reset.
Host migration (another player takes over when the host leaves) is not planned: the host's game is the story, and it ends with the host, as the original's ends when the console is switched off.
Tick rate: should the simulation run at 60 Hz?¶
The question: should Coney move its simulation to a fixed 60 Hz everywhere, and would that help networking? Decided: no, 30 Hz, subject to a measurement. Keep the original's rates, already built: a 30 Hz step with the original's two 60 Hz object ticks inside it, drawn at any frame rate. A 60 Hz simulation cannot be made identical to the original, and in networked play the step is a small share of the delay, which prediction removes for the player's own Warrior.
The original's rates¶
All on NTSC-U SLUS_212.15.
| What | Rate | Evidence |
|---|---|---|
| The play tick | TaskManager_TickGame (0x003a2ea0) once per frame; while the bus clock holds at least two 60 Hz ticks (0x960000 COP0 counts) it runs a pair of 60 Hz ticks, so a long frame catches up with several pairs; the rest carries over. At 29.97 frames a second that is one pair a frame, and two in about one frame of 1,000 |
confirmed (code) and (runtime), Tasks: the play tick |
| Frames | 29.97 a second, two fields each | confirmed (runtime), same page |
| Characters: movement, combat, animation, brains | Humans_Update (0x00249108) is called on both ticks, its body runs on every second: 30 Hz |
confirmed (code) and (runtime), same page |
| AI thinking | inside the characters' step; some rules count steps (a think every fifth step) | confirmed (code), AI |
| Scheduled objects: doors, props, particles, cars, scene tasks | the timing wheel, one bucket per 60 Hz tick, each object every interval ticks |
confirmed (code) and (runtime), The wheel |
| Physics step | every 60 Hz tick; it settles landed objects and refreshes a bound, and moves no human | confirmed (code), Physics: 30 or 60 Hz |
| Cameras | at the end of each characters' step, 30 Hz | confirmed (code), Cameras |
| Game state, spawns, HUD, the level scripts' update and schedule | once per frame, 30 Hz | confirmed (code), Tasks, Level loading |
| Scenes | in 1/30 s frames | confirmed (code), Scenes: timing |
| Slow motion | changes the characters' dt (1/60, 1/120 or 1/150 s), not the step rate |
confirmed (code), Cameras |
| PAL | the tick counts the CPU's bus clock, not video fields, so at 25 frames a second it would run on average 1.2 pairs a frame and keep the same game speed | inferred from the NTSC-U code; no PAL executable studied, so speculative for PAL itself |
Coney's rates today¶
One fixed step of exactly 1/30 s at any display rate, bit for bit the same at every rate
(Update and render). Inside it the objects take the original's two 60 Hz ticks
(src/platform/play_level_objects.cpp, LooseObjects::kTicksPerStep, the doors', glints' and light tasks' ticks);
the humans, cameras, scripts and HUD step once. Drawing blends the last two steps, so 60, 144 or 240 frames a second
already look smooth.
What a 60 Hz simulation would change¶
Everything the player feels. A human's step is a short sequence of float operations (velocity, then position; an acceleration per step²; a clip advanced by a step's time); halving the step does not halve each result, it changes it, and the difference compounds. Every step-counted rule (the five updates after a block, a think every fifth step, 60 updates before a scene may be skipped) would have to be rescaled, and an odd count has no exact half. The runtime comparison against the original, step by step, would stop being possible. A faithful game at 60 Hz is not a reachable target; a 60 Hz game is a different game that is close.
What a player wants from "60 Hz" is smooth motion and quick response. Smooth motion is already built (interpolated drawing). Response gains 16.7 ms on average (below).
Where the tick shows in response time¶
A press waits for the next step that reads the pad, and what that step changes is drawn by blending the last two steps (Update and render), which shows the newest step up to one step late. Both waits scale with the step length T: on average T/2 each, at worst T each.
| 30 Hz (T = 33.3 ms) | 60 Hz (T = 16.7 ms) | Difference | |
|---|---|---|---|
| Wait for the step that reads the press | 16.7 ms on average, 33.3 ms at worst | 8.3 ms, 16.7 ms | 8.3 ms, 16.7 ms |
| Drawing between steps | 16.7 ms on average, 33.3 ms at worst | 8.3 ms, 16.7 ms | 8.3 ms, 16.7 ms |
| Together | 33.3 ms on average, 66.7 ms at worst | 16.7 ms, 33.3 ms | 16.7 ms on average, 33.3 ms at worst |
Arithmetic from the step lengths, not a measurement. These waits are the same offline and online, and the original on a PS2 has the same 30 Hz waits, so at 30 Hz Coney responds as the original did (inferred from the play tick: the original reads the pads once a frame and draws the frame it has just stepped).
What the network adds on top, for the guest's own Warrior:
| Path | Round trip | Source |
|---|---|---|
| Same house (LAN) | under 1 ms to a few ms | general knowledge |
| Same region (one country in Europe, one US region) | 10-45 ms | Verizon's backbone targets: 30 ms within Europe, 45 ms within North America (Verizon latency) |
| Across the US | 60-80 ms | general knowledge (not re-checked) |
| Across the Atlantic | about 90 ms | Verizon: London to New York (same page) |
- Without prediction (P1): the guest's Warrior answers after the round trip, plus the server's step wait, plus the wait for the next snapshot, plus the interpolation delay (about 100 ms). Across a region that is about 150-200 ms at 30 Hz. A 60 Hz step would take about 17-25 ms off it (the step wait and a snapshot sent every step), about a tenth; the round trip and the interpolation delay, which the tick rate does not change, are the rest.
- With prediction (P2): the guest's own Warrior runs on the guest's machine from the guest's pad, so its response is the offline one above, whatever the round trip. The network then shows only in the others (drawn about 100 ms in the past) and in hits, which the server decides with lag compensation. The tick rate's share is the same 16.7 ms average difference as offline.
- The host plays with no network at all: his response is the offline one.
What players notice. Studies of action games put noticeable delay at about 50-100 ms for precise, fast actions (Claypool and Claypool, Latency and player actions in online games, CACM 2006, not re-checked; a survey of the thresholds finds the most time-critical games degrade from about 50 ms, Hagen et al.). The 16.7 ms between 30 and 60 Hz is below those thresholds and is already part of the original's feel; the round trip across a region is not, and prediction is what removes it.
What shipped games use¶
Tick (simulation) and update (snapshot) rates of online games, for scale. Marked secondary when the figure comes from a measurement or write-up rather than the developer, and not re-checked when it could not be fetched.
| Game | Server tick | Updates to clients | Source |
|---|---|---|---|
| Apex Legends | 20 Hz | a snapshot each tick | Respawn, Servers and netcode deep dive |
| Call of Duty: Warzone | about 20-24 Hz | secondary, Sportskeeda on measurements | |
| Fortnite | 30 Hz (8 Hz at launch) | 75 a second (2018) | secondary, GamingBolt citing Battle(non)sense |
| Battlefield 4 | 30 Hz after patches (10 Hz at launch); 60 and 120 Hz tested | secondary, GamePro | |
| Battlefield 1, Battlefield 6 | 60 Hz | secondary, MP1st | |
| Overwatch | about 62.5 Hz (16 ms command frames) | secondary, Edgegap on Tim Ford's GDC 2017 talk | |
| Counter-Strike 2 | 64 Hz with sub-tick timing of inputs | Valve, CS2; 64 Hz secondary | |
| Valorant | 128 Hz | Riot, Peeking into Valorant's netcode | |
| Rocket League | 120 Hz physics | Jared Cone, GDC 2018 (secondary, not re-checked) | |
| GTA Online | peer-to-peer hybrid; no published rate | none found |
So 20-30 Hz servers are common in big third-person and battle-royale games, and 60-128 Hz in competitive shooters, where a few milliseconds decide who saw whom first. Apex's developers give the cost in their own terms: at 50 ms of ping, 20 Hz against 60 Hz is about two displayed frames of extra delay, for three times the bandwidth. A four-player co-op brawler against AI is far from the shooters' case.
Input sampled at the display rate¶
Coney already keeps every press between steps, so a tap shorter than a step is never lost, and samples the pads just before each step (Update and render). Sampling faster than the step cannot make the step act sooner: the original's game reads the pad once per step and acts on what it reads. Two things do help, and both are in the plan:
- The guest's client sends each step's command as soon as it is sampled, with no wait for its own frame, so the command reaches the server as early as the link allows (Input commands).
- Drawing at the display rate (built) means the result appears on the first displayed frame after the step, not at the next 30 Hz frame boundary.
Counter-Strike 2's sub-tick idea (timestamping a press inside the step) changes what a step decides, so it is not faithful and is not planned.
How we would measure it¶
So the decision can be revisited with evidence (issue #568): a headless test, in CI once the client's world and prediction exist (issues #555 and #559), measures press to screen, the time from a pad press to the first drawn frame where the pressing player's Warrior has changed:
- A server and a client over the loopback, behind the link conditioner, at round trips of 0, 20, 50, 100 and 150 ms with 5 ms of jitter and 1% loss.
- The client draws at 240 frames a second in test mode (simulated frame times, no clock); an input script presses a button (a punch, a step of the stick) at random sub-step moments, 1,000 presses per run, seeded.
- Each press's time to screen is recorded on the client, for the host's player and for the guest's, with and without prediction, and split into its parts: step wait, network, server buffer, snapshot wait, interpolation, drawing.
- The same network path is run at a 60 Hz step: the network layer counts in steps and is told the step rate in Welcome (Protocol), so the sessions, commands and snapshots run at 60 Hz with a stand-in simulation of the same size; the step-bound parts are then compared directly.
- The report gives the median and the 95th percentile for each case. The rule: 30 Hz stays while, with prediction, the guest's median press to screen at 30 Hz is within 20 ms of the 60 Hz path and of the offline figure, at every round trip. If not, the result goes back to the maintainer.
Decision¶
30 Hz, the original's rates, everywhere: offline, host and server, subject to the measurement above. It was decided on that condition; the measurement is issue #568.
- Keep high frame rates as drawing only, as built; what is not yet blended is listed on Enhancements.
- Recover response online with prediction (P2), not with the tick rate.
- Keep the door open at no cost: every duration named as steps or as time (Update and render), so a later Coney-only 60 Hz mode, if ever wanted, has its list to rescale.
Moving to 60 Hz would cost a rescale and re-verification of every gameplay constant and rule in every subsystem (months of the whole team's work), the loss of the step-by-step comparison with the original, and a game that no longer plays exactly as the original.
Testing¶
Networking is tested like the rest of the engine: no real time, no window, no human (Testing). What each phase adds:
| Test | What it proves | Phase |
|---|---|---|
| Encodings, refusals, sessions | built: Testing | P0 |
| Offline is untouched | a level played by an input script gives the same state digest with networking compiled in and no session as before; run by every networking pull request | P1 |
| Loopback co-op | a server and a client in one process over the loopback, each with an input script: the server's digest equals a one-machine, two-pad run of the same inputs (the remote pad changes nothing) | P1 |
| What a client sees | once the jitter buffer settles, every replicated field on the client equals the server's at the drawn step, within quantization | P1 |
| Bad links | the same runs behind the link conditioner (latency, jitter, 5% loss, duplicates, reordering): the server's game is unchanged and the client converges | P1 |
| Two processes | coney --headless --host and coney --headless --join 127.0.0.1, each with an input script, over ENet, in CI |
P1 |
| Replays | the server records input commands and session events; replaying them headless gives the same digests | P1 |
| Disc-backed | one level with two players over the loopback, printing counts and hashes only (No game data) | P1 |
| Bandwidth | bytes per snapshot and per second in a busy level, against a budget a test pins | P1 |
| Prediction | a client's predicted Warrior equals the server's after corrections, with and without loss | P2 |
| Fuzzing | random and mutated packets into every decoder and session: no crash, no hang, only clean refusals | P2, nightly |
The state digest is the key tool: a hash of the replicated state at each step, printed by the headless server and the client, compared by the tests and shown on the Network page. When two digests differ, a debug dump names the first entity and field that differ.
The networking checklist¶
For every engineer writing any subsystem, faithful work included. Each item is cheap now and expensive to retrofit. It is a rule, not a suggestion: AGENTS.md cites it under Code, the code review checks it, and Update and render points here.
- Say who owns each piece of state: simulation (the server decides; it lives in the game's state and changes
only in
update()) or presentation (each machine's own; it may read the simulation, never write it). Write it in the class's doc comment when it is not obvious. - No hidden mutable globals. Simulation state lives in objects the game mode owns, so a server can run a session,
copy it, or run two in one process. A
staticor file-scope variable that changes during play breaks all three. - Never assume one player. Take a player slot or a pad record; never read "the player", pad 0 or camera 0 directly. The original already has two players; write for a player count.
- Ids, not pointers, for anything another machine names. Keep a stable handle or id for every human, object, car and camera; never keep a raw pointer across steps where an id will do, and never send one.
- Gameplay randomness from
GameRandomonly. Presentation randomness (particles, idle sounds, decals) comes from its own stream, never fromGameRandom, so a client's effects cannot shift the game's draws. - No real clock. Time is steps and game time, already enforced (Update and render);
name durations
...Stepsor...Ms. - Input only through the pad record. A subsystem reads the pad record it is given, filled once per step; a remote pad must be indistinguishable from a local one.
- Make visible effects observable. What the simulation causes a player to see or hear (a sound, a particle burst, a HUD message, a screen flash, a rumble) goes through one call that can be recorded as an event, never straight to the renderer or the audio device from inside the simulation.
- Keep presentation out of the simulation. The simulation never reads the window, the display's field of view,
the frame rate or a GPU result;
render()never writes whatupdate()reads. - State that can be copied. Prefer plain data that can be copied and compared; say on the research page what a subsystem's state is, so replication knows what to send.
- Iterate in a stable order. No iteration over an unordered container, or by pointer value, where the order changes the result.
- Validate what comes from outside. Anything decoded from a file or a packet returns
std::expectedand is range-checked before use.
Roadmap¶
The work in order; each row is about one pull request. Each is an issue, a sub-issue of #547.
| # | Unit | Phase | Needs | Issue |
|---|---|---|---|---|
| 1 | The Network page, --host and --join (in progress) |
P1 | the foundation | in progress |
| 2 | The remote player's pad on pad port 2 (in progress) | P1 | 1 | in progress |
| 3 | The original's two-player game, offline: GameState_SyncPlayers, player 2's human, HuIsAPlayer, the multiplayer callbacks, the co-op quit |
P1 | faithful work with two local pads | #548 |
| 4 | Two players' cameras: the split decision and the layout, offline | P1 | 3 | #549 |
| 5 | The AI's two-player rules | P1 | 3 | #550 |
| 6 | Players by slot everywhere: the single-player assumptions in gameplay, HUD and bindings | P1 | 3 | #551 |
| 7 | Input commands: tagged, redundant, the server's buffer and timing feedback | P1 | 2 | #552 |
| 8 | Network ids and the replicated-field tables | P1 | #553 | |
| 9 | Snapshots: humans first, deltas against acknowledged baselines, the budget | P1 | 8 | #554 |
| 10 | The client's world: jitter buffer, interpolation through Interpolated<T> and DrawnPoses, a client game mode with no simulation |
P1 | 9 | #555 |
| 11 | Events: sounds, particles, HUD messages, scenes, movies | P1 | 9 | #556 |
| 12 | Level loads, checkpoints and the loading barrier | P1 | 11 | #557 |
| 13 | The state digest, two-process tests in CI, replays | P1 | 9 | #558 |
| 13a | Measure press to screen at 30 and 60 Hz (Tick rate) | P2 | 10, 14 | #568 |
| 14 | Prediction of the client's own Warrior | P2 | 10 | #559 |
| 15 | Lag compensation for strikes | P2 | 14 | #560 |
| 16 | Encryption and the connect challenge over ENet, with libsodium | P2 | #561 | |
| 17 | LAN discovery and joining by address (IPv4 and IPv6) | P2 | 16 | #562 |
| 18 | The simulation out of src/platform/, so a server needs no renderer |
P3 | #563 | |
| 19 | coney-server, the headless dedicated server |
P3 | 18 | #564 |
| 20 | Coney's rule set: three and four players | P3 | maintainer decision | #565 |
| 21 | Resources: manifest, Lua 5.4 sandbox, events, replicated state | P4 | the script split | #566 |
| 22 | ConeyMP large sessions: routing buckets, many players; a server list only if the maintainer decides one | P5 | #567 |
Maintainer decisions¶
Decided: the networking checklist is adopted; the tick rate stays 30 Hz, subject to the measurement; the original's two-player game is built offline first (roadmap 3-6); the transport stays ENet with libsodium, players join by LAN discovery or address with a forwarded port, Coney runs no services, and ConeyMP is a separate product on the same stack (Transport). Open:
- Coney's rule set (P3): more than two players, which humans they take, how the AI and cameras treat them.
- A guest's co-op record in Coney's own store (Saves).
- A second Lua runtime (5.4) for resources (P4).
- A server list for ConeyMP (later, a separate decision): Coney runs no service until it is decided.
Prior art and sources¶
Ideas only: no code is taken from any of these. FiveM's code is under the Rockstar Games Creator Platform License Agreement, not an open-source licence, as its repository's README states, so only its public documentation was read. Sources marked (not re-checked) could not be fetched when this page was written.
| Idea used here | Source |
|---|---|
| Entity ownership, routing buckets, strict mode | FiveM, OneSync |
| Server-written replicated state; the risk of client-writable state | FiveM, State bags |
| Events; latent events with a rate cap | FiveM, Triggering events |
| Never trust event arguments | FiveM, Server security |
| Snapshots, prediction, interpolation delay, lag compensation | Valve, Source Multiplayer Networking and Latency Compensating Methods (not re-checked) |
| A transport with encryption and hole punching built in (studied, not chosen) | Valve, GameNetworkingSockets (BSD-3) |
| ENet under a large scripted-server platform | FiveM's vendored ENet, vendor/citizen_enet in its repository |
| RPC kinds, validation that disconnects, relevancy, the replication graph | Epic, Remote procedure calls, Replication Graph |
| Deltas against the last acknowledged snapshot; field tables | Fabien Sanglard, Quake 3 network model |
| Command frames, tuning the server's input buffer | Tim Ford, Overwatch Gameplay Architecture and Netcode, GDC 2017 |
| State, events and control; priority under a budget | David Aldridge, I Shot You First, GDC 2011 |
| Choosing a model; snapshot compression; fixed steps; connect tokens | Glenn Fiedler, What every programmer needs to know, Snapshot compression, Fix your timestep, netcode standard |
| Rollback, and why it does not fit here | GGPO (MIT) |
| Host-owned story with guests as visitors; the risks of a peer mesh | GTA Online, Dark Souls, Borderlands (general knowledge, no single source) |
| Lua sandboxing: instruction hook, allocator cap | Lua 5.4 manual, lua-users: Sand boxes |