File I/O¶
Verified against: SLUS_212.15 (NTSC-U, SHA1 e9cb2cc49aa046b9e494313dce2f5038ed17b2f4). No runtime claims.
Purpose¶
How the game turns a file name such as level1.lev into bytes in memory. Every asset lives in WARRIORS.WAD and is
found by name through WARRIORS.DIR (WARRIORS.DIR / .WAD); the bytes are fetched by the game's
own IOP module, either synchronously through a 384 KB buffer or asynchronously through a request queue serviced once
per frame. The chunk system reads from these streams; the boot path creates them.
For Coney most of this layer is replaced by plain reads from the player's disc image, but the interfaces (a file
system with Exists, GetSize, Open, Close; a file with Read, Seek, Tell, GetSize) and the two read
paths are what the rest of the game is written against.
Original structure¶
| File | Class (role) | Address range | Evidence |
|---|---|---|---|
Device/ps2/DS_PS2Device.cpp |
DS_PS2Device: the platform object; creates and owns the file systems |
0x001483e8-0x00148580 (+ methods 0x00145790-0x001458b0, 0x00148230, 0x00148820-0x001488f8) |
confirmed (code) |
Device/ps2/DS_PS2FileSys.cpp |
PS2FileSys (host0: debug files) and PS2DbgFile; also PS2StreamFileSys and its stream file |
0x00148608-0x00148e00 |
confirmed (code) for the anchors; the stream classes' file is inferred from position |
Device/ps2/fileio/DVDWadIndexPS2.cpp |
DVDWadIndex: the name index |
0x00149040-0x00149248 |
confirmed (code) |
DVDWadIndexPS2.cpp (by position) |
the stream request for one WAD entry | 0x00149410-0x00149500 |
inferred |
(unnamed, after sound/msaudiodevice.cpp's stub 0x0014d508) |
the IOP command layer: commands to IOP.IRX, shared with audio |
0x0014da38-0x00151c70 |
inferred |
FileIO/ (unnamed, before FS_MemoryFile.cpp) |
FS_FSToStreamFSFileSys and its file |
0x00153f60-0x00154168 |
confirmed (code) for the allocation tag; file inferred |
FileIO/FS_MemoryFile.cpp |
FS_MemoryFile: a file in memory |
0x001541e0-0x00154440 |
confirmed (code) |
FileIO/StreamManager.cpp |
the buffered reader, and FileManager (asynchronous requests, the File Stream Buffer) |
0x001544c0-0x00155b10 |
confirmed (code) |
World/ (just before WorldLevel.cpp's anchor 0x0040c7f0) |
the WAD object: opens WARRIORS.DIR and WARRIORS.WAD, wraps Find |
0x0040c5e0-0x0040c688 |
confirmed (code) for the code; file inferred |
Key functions (names ours):
| Address | Name | Role | Evidence |
|---|---|---|---|
0x001483e8 |
DS_PS2Device::CreateStreamFileSys |
PS2StreamFileSys → device +0x94 |
confirmed (code) |
0x001484d8 |
DS_PS2Device::CreateDefaultFileSys |
FS_FSToStreamFSFileSys over the stream file system, mounted as current (+0x90) |
confirmed (code) |
0x00148460 |
DS_PS2Device::CreateHostFileSys |
PS2FileSys("debug/") → device +0x8c |
confirmed (code) |
0x00148aa0 |
PS2StreamFileSys::Open |
start an IOP read of a WAD entry | confirmed (code) |
0x00148c90 |
PS2StreamFile::Wait |
block until the read is done, servicing pads | confirmed (code) |
0x00149160 |
DVDWadIndex::DVDWadIndex(path) |
read the index | confirmed (code) |
0x001490b8 |
DVDWadIndex::Find(name) |
name → entry | confirmed (code) |
0x00149410 |
IopStream::Start |
send the read commands to the IOP | confirmed (code); name inferred |
0x00153fa8 |
FSToStreamFile::Read |
one blocking read | confirmed (code) |
0x001544c0 |
BufferedStream::BufferedStream(file, buf, size) |
confirmed (code) | |
0x00154630 |
BufferedStream_Read |
confirmed (code) | |
0x001547b0 |
FileManager creation |
confirmed (code) | |
0x00154d50 |
FileManager_Request |
queue an asynchronous whole-file read | confirmed (code) |
0x00154ae0 |
FileManager_Service(fm, block) |
start/finish queued reads | confirmed (code) |
The other functions of FileIO/ (0x00153f60-0x00155b30), names ours. The vtable slots are those of
the file interface and the file system interface. Several one- and
two-instruction methods had no function in Ghidra and were made there (marked made).
| Address | Name | Role | Evidence |
|---|---|---|---|
0x00153f60 |
FSToStreamFile_Init(file, name, streamFs) |
the embedded file's open: copies the name (63 chars), keeps the stream file system, position 0 | confirmed (code) |
0x00154068 |
FSToStreamFile_Write (made) |
slot +0x58: does nothing |
confirmed (code) |
0x00154070 / 0x00154078 |
FSToStreamFile_Seek / _Tell (made) |
slots +0x60 / +0x68: set / return the position +0x04 |
confirmed (code) |
0x00154080 |
FSToStreamFile_GetSize |
slot +0x70: the stream file system's GetSize(name) |
confirmed (code) |
0x001540b0 |
FSToStreamFile_Close |
slot +0x78: does nothing |
confirmed (code) |
0x001540c0 |
FSToStreamFSFileSys_Init(fs, streamFs) |
keeps the stream file system, marks the file free | confirmed (code) |
0x001540d0 / 0x00154100 |
FSToStreamFSFileSys_Exists / _GetSize |
slots +0x50 / +0x58: forward the name to the stream file system |
confirmed (code) |
0x00154130 |
FSToStreamFSFileSys_Open(fs, name) |
slot +0x60: marks the one embedded file in use and opens it (0x00153f60); never refuses, even when it is already open |
confirmed (code) |
0x00154168 |
FSToStreamFSFileSys_Close |
slot +0x68: marks the file free |
confirmed (code) |
0x00154530 |
BufferedStream_Clear |
zeroes the fields before the set-up | confirmed (code) |
0x00154550 |
BufferedStream_Attach(s, file, buffer, size) |
the set-up of BufferedStream: reads the file's size through its slot +0x70 |
confirmed (code) |
0x001545a0 |
File_Destroy(file, flags) |
the file base class's destructor (vtable 0x00534298), slot +0x08 of BufferedStream; frees itself when bit 0 is set |
confirmed (code) |
0x001545d0 |
BufferedStream_Take(s, dst, n) |
copies n bytes from the buffer cursor and moves the cursor, the bytes left and the position |
confirmed (code) |
0x001546f8 / 0x00154700 |
BufferedStream_Write / _Seek (made) |
slots +0x58 / +0x60: do nothing (reads are forward-only) |
confirmed (code) |
0x00154708 / 0x00154710 |
BufferedStream_Tell / _GetSize (made) |
slots +0x68 / +0x70: +0x14 / +0x18 |
confirmed (code) |
0x00154718 |
BufferedStream_Detach |
slot +0x78: forgets the buffer (the s.detach() of Synchronous reads); 7 loaders call it |
confirmed (code) |
0x001547b0 |
FileManager_Create |
allocates the 0x48-byte FileManager in the system pool, sets up the queue, calls 0x001548c0, stores it in g_FileManager (0x005e5340) |
confirmed (code) |
0x001548c0 |
FileManager_Init |
takes the device's stream file system, allocates the File Stream Buffer |
confirmed (code) |
0x00154950 |
FileManager_FreeBuffer |
frees the File Stream Buffer (from the destructor 0x004dd9b8, link-once code) |
confirmed (code) |
0x001549a8 |
FileManager_IsBusy |
+0x08 |
confirmed (code) |
0x001549b0 |
FileManager_IsInFlightDone |
busy and the stream file's IsDone |
confirmed (code) |
0x001549f8 |
FileManager_WaitInFlight |
the stream file's Wait |
confirmed (code) |
0x00154a28 |
FileManager_GetFileSize(fm, name) |
the stream file system's GetSize(name); the scene cache sizes its scene_list.cnk request with it (0x003535e8) |
confirmed (code) |
0x00154a68 |
FileManager_QueueLength |
requests waiting in the queue (a deque of 0x54-byte requests, 6 to a node); the world streamer and resource loaders poll it | confirmed (code) |
0x00154ee8 |
Gpu_DmaHangCallback(active, value) |
the callback the graphics driver calls while it waits for the GPU (Graphics): keeps value in 0x0050c6a8, or prints DMA Hang when it is 0xcdcdcdcd |
confirmed (code) |
0x00155a90 / 0x00155b10 |
StreamManager_StaticInit / its stub |
the unit's static initialiser: sets 22 words 0x005e5348-0x005e539c to 1 (the world toggles, The streamed world); the stub ends StreamManager.cpp |
confirmed (code) |
The Device/ps2/ file code and its neighbours, names ours. The pads and player commands in the same region are on
Input; the game clock on Boot. The host0: calls (0x00441910 and its neighbours)
are unnamed in Ghidra; that they are the SDK's open, close, read, write and lseek is inferred from their
arguments (0x602 = write, create, truncate).
| Address | Name | Role | Evidence |
|---|---|---|---|
0x00145888 |
DS_PS2Device_MountFileSys(dev, fs) |
appends to the mounted list (+0x04, +0x08) and makes it current (+0x90) |
confirmed (code) |
0x001458b0 |
DS_PS2Device_SaveCurrentFileSys |
device slot +0x88: +0x88 = +0x90 |
confirmed (code) |
0x001458c0 |
Device_AlwaysTrue |
returns 1; asked by the script loaders before each file (ScriptSystem_DoFileLua, InitLevel) |
confirmed (code) |
0x001458c8 / 0x001458d8 |
Loading_HasCallback / Loading_SetCallback(fn) |
the loading-screen callback 0x0050b728 that PS2StreamFile_Wait and the world preload call while they block; setting one calls it once at once |
confirmed (code) |
0x00148230 |
DS_PS2Device_InitIop |
the IOP reboot and modules (Boot) | confirmed (code) |
0x001482d8 / 0x001482e0 / 0x00148358 |
Iop_LoadRetryHook / Iop_LoadModuleRetry(path) / Iop_LoadModules |
load one module, retrying until it loads (the hook between tries is empty); the six modules in order | confirmed (code) |
0x001483e8 / 0x00148460 / 0x001484d8 |
DS_PS2Device_CreateStreamFileSys / _CreateHostFileSys / _CreateDefaultFileSys |
above | confirmed (code) |
0x00148580 / 0x001485a8 |
DS_PS2Device_StaticInit / its stub |
the static device's vtable 0x00537c80; the stub ends DS_PS2Device.cpp |
confirmed (code) |
0x00148820 |
DS_PS2Device_Slot68 |
device slot +0x68: empty |
confirmed (code) |
0x00148828 |
DS_PS2Device_GetHeap(dev, out) |
slot +0x70: fills 2 KB at 0x01fef800 with 0xcd and gives the main heap's size, 0x018d7c94 (Memory) |
confirmed (code) |
0x001488f8 |
DS_PS2Device_InitInput |
slot +0xa0: Pads_Init, then AudioManager_Reset |
confirmed (code) |
0x001485c8 |
PS2FileSys_Construct(fs, root) |
vtable 0x00537d38, root path (255 chars) at +0x04 |
confirmed (code) |
0x00148708 |
PS2FileSys_MakePath(fs, name) |
root + name into +0x104 |
confirmed (code) |
0x00148748 / 0x001487b0 |
PS2FileSys_Exists / _GetSize |
slots +0x50 / +0x58: open a PS2DbgFile on the stack, close it |
confirmed (code) |
0x00148608 / 0x001486c0 |
PS2FileSys_Open(fs, name, mode) / _Close |
slots +0x60 / +0x68: allocates a PS2DbgFile (vtable 0x00537ef8, tag PS2DbgFile) / releases and frees it |
confirmed (code) |
0x00149248 |
PS2DbgFile_Open(file, name, mode) |
opens host0:<name>, read (mode 0) or write, create, truncate (mode 1); the handle at +0x04 |
confirmed (code) |
0x001492c8 / 0x001492e8 / 0x00149308 |
PS2DbgFile_Close / _Read / _Write |
close (called directly), and slots +0x50, +0x58 |
confirmed (code) |
0x00149328 / 0x00149348 / 0x00149370 |
PS2DbgFile_Seek / _Tell / _GetSize |
slots +0x60, +0x68: lseek from the start / from the current position; the size by seeking to the end and back |
confirmed (code) |
0x00148920 |
PS2StreamFileSys_Construct |
the pool of 10 stream files (PS2StreamFileSys) | confirmed (code) |
0x00148a50 / 0x00148a70 |
PS2StreamFile_Construct / _Destroy |
vtable 0x00537db0, request −1 / back to the file base vtable, frees itself when bit 0 is set |
confirmed (code) |
0x00148aa0 |
PS2StreamFileSys_Open |
above | confirmed (code) |
0x00148ba8 / 0x00148bf0 |
PS2StreamFileSys_GetSize / _Exists |
./ee_files/ + name, looked up in the WAD (Wad_GetEntrySize, Wad_HasEntry) |
confirmed (code) |
0x00148c38 |
PS2StreamFile_IsDone |
slot +0x50: the IOP stream 0 is not busy |
confirmed (code) |
0x00148c60 |
PS2StreamFileSys_Update |
slot +0x70: services the IOP client (0x0050bce4, its slot +0x10) |
confirmed (code) |
0x00148c90 |
PS2StreamFile_Wait |
Synchronous reads | confirmed (code) |
0x00148db0 |
PS2StreamFileSys_Close(fs, file) |
cancels the stream (an empty call), clears the open file, destroys the file in place and returns it to the free list | confirmed (code) |
0x00148e30 |
Device_SetCombination(n, text) |
copies text (128 chars) to 0x0050b830 and stores n with the name Combination4 to Combination7 (n = 1-4) at 0x0050b7a8; nothing calls it and nothing reads them |
confirmed (code); role unknown |
0x00148f60 / 0x00148f98 |
Timer_Construct / Timer_Reset |
vtable 0x00537e98; total 0, last count = now (Timers) |
confirmed (code) |
0x00148fc8 / 0x00148fd8 |
Timer_ReadCount / Timer_GetTicks |
COP0 Count; the 64-bit total |
confirmed (code) |
0x00149040 |
DVDWadIndex_Create(path) |
allocates the 8-byte index (tag DVDWadIndex) and reads it |
confirmed (code) |
0x00149160 / 0x001490b8 |
DVDWadIndex_Construct / _Find |
DVDWadIndex | confirmed (code) |
0x00149410 |
IopStream_Start |
above | confirmed (code) |
0x001494d0 / 0x001494d8 |
IopStream_Cancel / IopStream_IsBusy |
empty / stream 0's busy flag (MS_GetStreamInfo) |
confirmed (code) |
0x00149500 |
IopStream_SetName |
copies the entry's name into the request | confirmed (code); role inferred |
The same unit also holds the push-or-pop helpers of several game modes, documented on their own pages:
PauseMenu_Toggle (0x00154f28, Pause), MissionSelect_Show (0x00155180: pushes mode 0x10 unless
it is on top, after storing the two Lua callbacks), GameStats_Show (0x001551e0: mode 0x13 with a close callback),
RumbleMenu_Show (0x00155228) and ProfileManager_Show (0x001552b0, Front end),
Autosave_Request (0x00155308, Save), SSMC_StartLoadSequence / SSMC_StartDeleteSequence
(0x00155378 / 0x001553c0: push the memory-card mode 6 set to load or delete, Front end),
MissionFailed_Toggle (0x00155408, mode 0xc, Pause), RumbleResult_Toggle (0x00155648, mode
0x14, Rumble) and ANGameOver_Toggle (0x001557f8, mode 0xd, the Armies of the Night game over). Each
pushes its mode when it is not on top and otherwise pops it and acts on the result. Confirmed (code). That they sit in
StreamManager.cpp rather than a unit of their own is inferred from position (the next stub is 0x00155b10).
Data¶
All classes use the GCC 2 vtable layout (8-byte {delta, fn} slots, see Compiler); slot offsets
below are from the vtable start. Slots +0x08 to +0x48 are the common base class's (destructor and allocation
helpers) and are left out.
File system interface¶
| Slot | Method | Evidence |
|---|---|---|
+0x50 |
bool Exists(name) |
confirmed (code), 0x00148bf0, 0x00148748 |
+0x58 |
u32 GetSize(name) |
confirmed (code), 0x00148ba8, 0x001487b0 |
+0x60 |
File* Open(name, ...): the stream file system takes (name, dest, length, offset); the others (name, mode) with mode 0 = read, 1 = write |
confirmed (code) |
+0x68 |
Close(File*) |
confirmed (code) |
+0x70 |
Update(): stream file system only, drives the IOP client |
confirmed (code), 0x00148c60 |
File interface¶
| Slot | Method | Evidence |
|---|---|---|
+0x50 |
Read(dst, n) |
confirmed (code) |
+0x58 |
Write(src, n) |
confirmed (code) |
+0x60 |
Seek(position) (absolute) |
confirmed (code) |
+0x68 |
u32 Tell() |
confirmed (code) |
+0x70 |
u32 GetSize() |
confirmed (code) |
+0x78 |
release: frees or detaches the buffer | confirmed (code) |
+0x88 |
void* GetData() (FS_MemoryFile only) |
confirmed (code) |
None of the implementations checks bounds or reports a short read.
DS_PS2Device¶
The platform object, a static at 0x005de160 (vtable 0x00537c80) reached through 0x0050b788. File-system fields,
confirmed (code):
| Offset | Meaning |
|---|---|
+0x04 |
index of the last mounted file system, -1 at start |
+0x08 |
array of mounted file systems (0x00145888 appends and makes the new one current) |
+0x88 |
copy of +0x90 (0x001458b0) |
+0x8c |
PS2FileSys for host0: debug files |
+0x90 |
current file system: the FS_FSToStreamFSFileSys; every game load goes through this one |
+0x94 |
PS2StreamFileSys |
PS2StreamFileSys¶
0x1c bytes, vtable 0x00537e18, confirmed (code) at 0x00148920:
| Offset | Meaning |
|---|---|
+0x04 |
pool of 10 stream-file records, 20 bytes each |
+0x08 |
free list: array of 10 record pointers |
+0x0c |
pool capacity (10) |
+0x10 |
records in use |
+0x14 |
the open stream file, or null: only one read can be in flight |
A stream file record (vtable 0x00537db0): +0x04 IOP request handle (-1 when idle), +0x0c 0, +0x10 owning file
system. Slot +0x50 is IsDone(), slot +0x58 is Wait().
FS_FSToStreamFSFileSys¶
0x58 bytes, two vtables (0x00538118 for the file system, 0x00538190 for the file it embeds), confirmed (code) at
0x001484d8:
| Offset | Meaning |
|---|---|
+0x04 |
the PS2StreamFileSys |
+0x08 |
the one embedded file: +0x04 position, +0x08 name (64 bytes), +0x48 the stream file system |
+0x54 |
in use (1 between Open and Close) |
So the default file system can have one file open at a time.
DVDWadIndex¶
8 bytes, confirmed (code) at 0x00149160: +0x00 entry count, +0x04 pointer to the entries (12 bytes each,
{wadOffset, size, nameHash}, allocation tag DVDWadEntry). Layout of the file: WARRIORS.DIR.
The WAD object at 0x006f39e8 (reached through 0x005147a4) holds the index pointer at +0.
BufferedStream¶
Built on the caller's stack (0x20 bytes, vtable 0x005382b8), confirmed (code) at 0x00154550:
| Offset | Meaning |
|---|---|
+0x04 |
buffer |
+0x08 |
buffer size |
+0x0c |
bytes left in the buffer |
+0x10 |
read cursor in the buffer |
+0x14 |
position in the file (Tell) |
+0x18 |
file size (from the file's GetSize at construction) |
+0x1c |
the underlying file |
FileManager¶
0x48 bytes, tag FileManager, global 0x005e5340, confirmed (code) at 0x001547b0 and 0x001548c0:
| Offset | Meaning |
|---|---|
+0x00 |
the stream file system (device +0x94) |
+0x04 |
stream file of the request in flight |
+0x08 |
busy (a request is in flight) |
+0x0c |
last request id (ids count up from 1) |
+0x10-+0x38 |
queue of requests (an STL deque: +0x1c front, +0x2c back) |
+0x3c |
File Stream Buffer: 0x60000 bytes (393,216), aligned to 128 |
+0x40 |
its size |
+0x44 |
vtable 0x00538348 |
A request is 0x54 bytes, confirmed (code) at 0x00154d50:
| Offset | Meaning |
|---|---|
+0x00 |
id |
+0x04 |
name (64 bytes, without the ./ee_files/ prefix) |
+0x44 |
destination; the File Stream Buffer when the caller passes 0 |
+0x48 |
bytes to read; the buffer's size when the caller passes 0 |
+0x4c |
completion callback (request*, user) |
+0x50 |
user value for the callback |
FS_MemoryFile¶
0x18 bytes, vtable 0x00538220, confirmed (code) at 0x001541e0:
| Offset | Meaning |
|---|---|
+0x04 |
owns the data (1 when it allocated it) |
+0x08 |
data |
+0x0c |
size: what GetSize returns; raised by Write past it |
+0x10 |
capacity |
+0x14 |
position |
Its functions, all in FileIO/FS_MemoryFile.cpp (names ours, in the FS_MemoryFile:: style of the class tag),
confirmed (code):
| Address | Name | Slot | Behaviour | Evidence |
|---|---|---|---|---|
0x001541e0 |
FS_MemoryFile(capacity) |
allocates capacity bytes from the current pool (tag unsigned char, align 16); size = capacity = capacity, position 0, owns the data |
confirmed (code) | |
0x00154268 |
FS_MemoryFile(buffer, size, capacity) |
wraps an existing buffer without owning it | confirmed (code) | |
0x00154290 |
~FS_MemoryFile |
+0x08 |
frees owned data into the current pool (Memory) | confirmed (code) |
0x00154320 |
Read(dst, n) |
+0x50 |
copies from data + position, advances | confirmed (code) |
0x00154370 |
Write(src, n) |
+0x58 |
copies to data + position, advances, raises the size | confirmed (code) |
0x001543c8 |
Seek(position) |
+0x60 |
sets the position | confirmed (code) |
0x001543d0 |
Tell() |
+0x68 |
the position | confirmed (code) |
0x001543d8 |
GetSize() |
+0x70 |
the size | confirmed (code) |
0x001543e0 |
Release() |
+0x78 |
frees owned data; data = null | confirmed (code) |
0x00154438 |
GetData() |
+0x88 |
the data pointer | confirmed (code) |
None checks bounds. A file made with a capacity starts with its size equal to that capacity, not 0. The last
function of the file, 0x00154440 (Stream_SkipBytes(file, n)), is not a method: it skips n bytes of any file by
reading them into a 256-byte stack buffer through slot +0x50, 256 at a time (confirmed (code); its file inferred
from its position, the source map's last function of FS_MemoryFile.cpp).
Behaviour¶
Opening the WAD at boot¶
0x0040c5e0, called from initialisation, confirmed (code):
new DVDWadIndex("cdrom0:\WARRIORS.DIR;1"): open with the SCE file API (sceOpen, read-only), read the 4-byte count, seek to0x10, allocatecount × 12bytes, read the entries, close. This is the only EE-side, blocking disc read the game makes through the standard file API.- Send the IOP command
0x1fwith the pathcdrom0:\WARRIORS.WAD;1(0x001500a8): the game's IOP module opens the WAD and keeps it open for every later read. Command meaning inferred.
Resolving a name¶
Confirmed (code) at 0x00148bf0, 0x00148ba8, 0x00148aa0, 0x001490b8:
- The caller passes a bare file name, for example
level1.levorwarriors.glr. - The stream file system prefixes it:
"./ee_files/" + name, in a 256-byte stack buffer. All game files live in that one flat folder. DVDWadIndex::Findlowercases the buffer in place, computes the CRC-32 of the whole path (Name hashing) and scans the entries linearly for that hash. No match returns null.Existsis "Find returned an entry";GetSizeis the entry'ssize(a missing name crashes, there is no check).
The host0: file system (PS2FileSys, root debug/) never sees game files; it is for debug output on a
development kit (PS2DbgFile, sceOpen("host0:<name>"), write mode 0x602 = create, truncate, write). Inferred
that a retail game never uses it.
The IOP stream¶
Every WAD read is done by the game's own IOP module IOP.IRX (loaded at boot), not
by the EE. The EE side builds a command buffer and sends it (0x0014da38 starts a command, 0x0014dcf8 adds a word,
0x0014dc88 a half-word, 0x0014dd78 a string, 0x0014daa8 sends it). IopStream::Start (0x00149410) sends, for
a read of length bytes at byte offset of a WAD entry into EE memory at dest:
| Command | Arguments | Meaning (inferred) |
|---|---|---|
0x29 |
0, entry.wadOffset + offset, length |
set the source range in the open WAD |
0x31 |
dest |
set the EE destination |
0x01 |
channel/mode words, 0x7e |
start the transfer |
0x00150838 reads the stream's status; the stream file's IsDone is "not busy". The same module and command set
play the streamed audio (the 0x01 command with other modes); that belongs on an audio page. Confirmed (code) for the
command numbers and arguments; their meanings are inferred.
Synchronous reads¶
The common path, used by level loading (0x0040c688, 0x0040d900) and resource packs (0x00187c38). Confirmed
(code):
fs = device.currentFileSystem # FS_FSToStreamFSFileSys
file = fs.Open(name) # marks the one embedded file in use, position 0
s = BufferedStream(file, fileManager.buffer, 0x60000)
ChunkSystem_LoadContainer(s) # or any reader of s
s.detach(); fs.Close(file)
BufferedStream.Read(dst, n) copies from its buffer and, whenever the buffer is empty, refills it with
file.Read(buffer, min(bufferSize, fileSize - position)). Reads are forward-only (Seek does nothing).
FSToStreamFile::Read(dst, n) (0x00153fa8) turns every refill into one complete IOP transfer:
streamFile = streamFs.Open(name, dst, n, position) # IOP read of n bytes at position
position += n
streamFile.Wait()
streamFs.Close(streamFile)
if discErrorState == 3: push the error mode # 0x005e5580, see below
Wait (0x00148c90) loops until IsDone: each pass it calls the stream file system's Update, yields
(0x004aa9f0(0)), and every 67 ms (0x0050b720) reads the pads (0x001454a8) and, if a callback is installed at
0x0050b728, calls it: the loading screen's tick during a level load, or the
preload indicator. Confirmed (code).
So a synchronous load of a file of size S costs ceil(S / 384 KB) blocking IOP transfers, each into the shared
File Stream Buffer, plus a copy into the chunk's own allocation.
Asynchronous reads¶
FileManager_Request(fm, name, length, callback, user, dest) (0x00154d50) fills a request (defaults as in the
table above), appends it to the queue, calls FileManager_Service(fm, 0) and returns the request id. When dest is
the shared buffer and length is larger than it, an assertion was compiled into a store through a null pointer
(the game would crash; inferred).
FileManager_Service(fm, block) (0x00154ae0), confirmed (code):
if already servicing: return # re-entrancy guard 0x0050c64c
streamFs.Update()
do:
if busy and block: inFlight.Wait()
if not busy:
if queue empty: break
r = queue.front
inFlight = streamFs.Open(r.name, r.dest, r.length, 0); busy = inFlight != null
else if inFlight.IsDone():
if discErrorState == 3: push the error mode (only when blocking); return
r = pop queue.front
r.callback(&r, r.user)
busy = false; streamFs.Close(inFlight)
while block
So without block each call starts at most one read or finishes at most one; with block it drains the whole queue.
Every game-mode frame calls it once without blocking (step 11 of one frame); loaders that need
the data at once call it with block = 1 right after their request (for example 0x0016e8f0, 0x003535e8).
Callbacks usually parse the buffer they were given by wrapping it in an FS_MemoryFile and running the
chunk system on it (the resource manager's 0x00186710 does this). Because the default
destination is the shared File Stream Buffer, the data must be consumed inside the callback.
The two paths share the buffer and the stream file system's single slot. A synchronous read started while a queued
read is in flight would get no stream file (Open returns null) and crash; the code avoids this by draining the
queue first (inferred from the callers; no lock exists).
Disc errors¶
0x005e5580 holds a device error state; the value 3 makes the readers push the error mode (0x00157930(errorMode,
1, 3)). The sound device's update sets it (0x0014c140) from MultiStream's disc-error check: 4 for error -1, 5 for
-3, 3 for any other (Sound: the PS2 sound device). Confirmed (code); what -1 and -3 are (the
IOP's error codes) is not traced.
Buffer sizes and limits¶
| What | Size | Evidence |
|---|---|---|
File Stream Buffer |
0x60000 bytes (384 KB), 128-byte aligned |
confirmed (code) at 0x001548c0 |
| name buffer for a WAD path | 256 bytes, ./ee_files/ prefix included |
confirmed (code) |
| request name | 64 bytes | confirmed (code) |
| stream file records | 10, but only one open at a time | confirmed (code) |
| files open on the default file system | 1 | confirmed (code) |
| pad service interval while waiting | 67 ms | confirmed (code) at 0x0050b720 |
Coney's implementation¶
Written from this page and WARRIORS.DIR / .WAD, in src/fileio/ (namespace coney::io):
stream.h: the file interface asStream(read,seek,tell,size,skip), where a read delivers every byte or fails withconey::Error;MemoryStreamover a buffer (the read side ofFS_MemoryFile);SubStream, a window of another stream that the chunk system hands to stream readers.file_stream.h:FileStream, a byte range of a host file read on demand with standard C++ file I/O.disc.h:Disc, the player's disc as a folder or an ISO 9660 image (primary volume descriptor and root directory only, 2048-byte sectors), with files found by name in any letter case, asconey-toolsdoes.wad_index.h:WadIndex, the parsedWARRIORS.DIRwith lookup by name (./ee_files/prefix, lowercased CRC-32) through a hash table, and by hash. It refuses a file whose size is not16 + count x 12and an entry that ends past the end ofWARRIORS.WAD.wad.h:Wad, the disc plus its index;lookuptakes a name or a0xhash andopenEntryreturns a stream over one entry. Reads are synchronous; there is no IOP, no 384 KB staging buffer and no one-file-open limit.
Not done yet: the file-system interface (Exists, GetSize, Open by name for the game's own callers), the
BufferedStream and the asynchronous FileManager queue. None is needed until a subsystem loads files by itself.
TODO for research, found while implementing: the FS_MemoryFile functions have no names on this page, so
Coney's equivalents carry no @orig tag; Stream_SkipBytes is cited with file (unknown) (answered: the names and
Stream_SkipBytes's file, FileIO/FS_MemoryFile.cpp (inferred), are in FS_MemoryFile). The WAD
opener at 0x0040c5e0 is named Wad_Open here (our name; no string names it). It opens cdrom0:\WARRIORS.DIR;1 through
0x00149040 and WARRIORS.WAD through 0x001500a8, and its strings sit just before WorldLevel.cpp's
("WorldLevel", "%s.lev"), so it is probably the first unit of World/ (inferred); cite it as
@orig 0x0040c5e0 Wad_Open (World/unknown).
Open questions¶
- What the disc error states 4 and 5 (
0x005e5580) change for the readers, and the IOP's error codes -1 and -3. - The exact
IOP.IRXcommand protocol (command words, the channel argument of0x01, status layout at0x00150838); only needed if Coney ever emulates the IOP side, which it does not plan to. - Whether any game file is read from outside
./ee_files/through the stream file system (movies usecdrom0:\%s;1through Bink, and the sound banks are opened by the IOP fromcdrom0:\IOP\). RockWadIndexPS2.cpphas a path string but no code: a second WAD index format, compiled out.