It’s been a while since my last post, but I’ve made significant progress in understanding and rendering Argentum Online’s maps. The journey has been challenging, involving a lot of digging through old code and some complex problem-solving.
In the old days, because we did not have 999GB of RAM we needed to divide in this case the map in chunks in order to just load enough to offer a good playable experience.
The current AO map is subdivided into 317 chunks, When a player reaches the border of one chunk, the game loads the neighboring chunk based on the direction they’re moving. For example, if the player is walking north, the northern chunk is loaded. The process usually takes between 100 ms to 2 seconds, depending on your computer’s power and disk speed.
This is an example of how a map looks. As you can see, it is built with tiles and layers. This is a very common way of building maps because it is really easy and allows us to layer elements on top of each other to create complex maps.
Textures on top of other textures are what we call layers, and we can have as many as we want. Each layer can also be used to mark spawn points or collisions, etc. It’s up to the developer to decide what to do and how it best suits their needs.
Okay now that I introduced a bit lets get dirty :)
Information gathering.
Becase we have the code we can easily read what’s happening behind the scenes.
First we need to know how are they structured, how many tiles per map and how many layers, type of layers etc, etc.
After some search we can see a module called TileEngine.bas where we can find the following definition:
'Map sizes in tiles
Public Const XMaxMapSize As Byte = 100
Public Const XMinMapSize As Byte = 1
Public Const YMaxMapSize As Byte = 100
Public Const YMinMapSize As Byte = 1
This tells us that each chunk is a 100x100 grid of tiles. But what about layers? Let’s look at a snippet from general.bas to see how layers are managed:
(some code was removed to simplify it)
For Y = YMinMapSize To YMaxMapSize
For X = XMinMapSize To XMaxMapSize
ByFlags = fileBuff.getByte()
'Layer 1
.Blocked = (ByFlags And 1)
.Graphic(1).GrhIndex = fileBuff.getLong()
Call InitGrh(.Graphic(1), .Graphic(1).GrhIndex)
'Layer 2 used?
If ByFlags And 2 Then
.Graphic(2).GrhIndex = fileBuff.getLong()
Call InitGrh(.Graphic(2), .Graphic(2).GrhIndex)
End If
'Layer 3 used?
If ByFlags And 4 Then
.Graphic(3).GrhIndex = fileBuff.getLong()
Call InitGrh(.Graphic(3), .Graphic(3).GrhIndex)
End If
'Layer 4 used?
If ByFlags And 8 Then
.Graphic(4).GrhIndex = fileBuff.getLong()
Call InitGrh(.Graphic(4), .Graphic(4).GrhIndex)
End If
'Trigger used?
If ByFlags And 16 Then
.Trigger = fileBuff.getInteger()
End If
If ByFlags And 32 Then
Call General_Particle_Create(CLng(fileBuff.getInteger()), X, Y)
End If
Next X
Next Y
Oh WOW! (imagine me jumping out of my seat reading some very old code) this is exactly what we need and more!
Decoding
As we can see, we iterate through the 100x100 tiles, and at first glance, we have 6 layers. It’s very clever how they are stored because we don’t have 6 loops to read each layer. Instead, we run a single cell just once, and everything is encoded there, making it very performant (remember, we’re talking about a 1999 game).
Notice that for each tile, we first read a byte:
ByFlags = fileBuff.getByte()
This byte gives us a lot of information. In short, this byte is used as flags, where each bit represents something for that tile. This means that if we perform an AND operation between a bit mask and this byte, we will get either a 1 or 0, depending on the bit value.
This is used for the various If conditions that are executed to handle different properties of the tile:
- First bit: This tile is blocked.
- Second bit: There is layer 2 information.
- Third bit: There is layer 3 information.
- Fourth bit: There is layer 4 information.
- Fifth bit: This tile has some kind of trigger.
- Sixth bit: This tile has particles.
- The rest of the bits are not used.
This byte is really important because it tells us how many bytes we need to read per cell!
(A Long is always read for the first layer, and other reads depend on this byte.)
For instance, if the byte is 00000010, it means that we have a second layer, and we will have to read:
- Long() bytes for layer 1
- Long() bytes for layer 2
- With this, we can assume that each cell contains a byte used as flags, at least one Long, and up to 3x Long() and 2x Integer().
Another example: 00000110:
- Long() bytes for layer 1
- Long() bytes for layer 2
- Long() bytes for layer 3
But now, we need to understant what is stored what is that Long()?
Well, this Long() that we are reading is the GrhIndex we talked about in the Graphics post—it’s the index for that graphic. (This is true for layers 1 through 4, and for now, we will just focus on these four layers and ignore the last two).
This means that each cell will contain a minimum of 1 texture or a maximum of 4 textures, with 2 possible triggers. The game stores this information in something called MapBlock, and so, in a single map, we will have 100x100 MapBlocks.
'Tipo de las celdas del mapa
Public Type MapBlock
Graphic(1 To 4) As Grh
CharIndex As Integer
ObjGrh As Grh
Damage As DList
NPCIndex As Integer
OBJInfo As obj
TileExit As WorldPos
Blocked As Byte
Trigger As Integer
Engine_Light(0 To 3) As Long 'Standelf, Light Engine.
Particle_Group_Index As Long 'Particle Engine
fX As Grh
FxIndex As Integer
End Type
Moving that to C#
We now have the knowledge and the power to start rendering this, BUT, like everything, there are some extra nuances to consider.
Each map will have a header that, luckily, is always 273 bytes.
Here’s an extract from the original code:
mapInfo.MapVersion = fileBuff.getInteger
With MiCabecera
.Desc = fileBuff.getString(Len(.Desc))
.CRC = fileBuff.getLong
.MagicWord = fileBuff.getLong
End With
fileBuff.getDouble
Something that happened over time is that computers became more powerful, and instead of a 32-bit address space, we now have 64-bit architectures with 64 bit address space. This means that the byte sizes for each type also changed! For example, a Long() in VBA6 is 4 bytes, whereas in C# and 64-bit architectures, that’s an int32. This means we must be extremely careful to use the correct byte sizes, because reading just one extra byte will shift everything and throw off our data alignment.
Here is an small extract of the reader ported to C#
string filePath = Path.Combine(Application.streamingAssetsPath, "Maps/Mapa"+map+".map");
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read)){
using (BinaryReader reader = new BinaryReader(fs)) {
int version = reader.ReadInt16();
// read header
byte[] description = reader.ReadBytes(255);
long crc = reader.ReadInt32();
long magicWord = reader.ReadInt32();
// padding?
reader.ReadBytes(8);
for(int y = 1; y <= 100; y++){
for(int x = 1; x <= 100; x++){
byte flag = reader.ReadByte();
MapBlock tile = new MapBlock();
tile.blocked = (flag & 1) == 1;
long grhIndex = reader.ReadInt32();
GrhData grhData = GetGraphic(grhIndex);
grhData.LoadSprite();
tile.grh[0] = grhData;
Vector3Int position = new Vector3Int(x, y - (grhData.pixelHeight / 2) / 32, 0);
Tile tilee = ScriptableObject.CreateInstance<Tile>();
tilee.sprite = grhData.sprite;
tilemap.SetTile(position, tilee);
// layer 2 used?
if((flag & 2) == 2) {
grhIndex = reader.ReadInt32();
[...]
But I’ve been hiding something from you all this time… Notice this line
Vector3Int position = new Vector3Int(x, y - (grhData.pixelHeight / 2) / 32, 0);
WTF is that y transformation?. Well remember at the begining of the post that we have a plot twist so yeah we need to twist things, we have a coordinate problem :)
Coordinates Origin
If we pressed the play button to render our scene without our transformation we will se the map rendered like this…
As we can see something looks like a bit off, it’s like is rotated 180º on the Y and 180º the Z!
Also we have our houses a bit weird we can see the interior and some images are on invalid spots… like on the wall?
WHAT IS GOING ON HERE!
The original game was built on top a game engine called ORE v.0.4 from what I could search it was a game engine built by the developers of Argentum Online, and they decided to have the X,Y origin at the top left of the screen.
This is not a problem on itself but it is if you also encode your maps expecting the origin to be there, this creates an issue for us, because we are also reading the maps like the original code, but our y:0 is not on top of the screen is instead to the bottom!
The issue and the solution
Because the Y-axis is flipped when comparing Unity and ORE, it also means that apart from the map having flipped coordinates, we also have flipped texture coordinates.
One might ask, why does the map almost look good, but some sprites appear a bit weird? Why are the grass, paths, and other sprites correct, while some seem off?
That’s because Argentum does not use 32x32 tiles for building everything on the maps. It’s a combination of 32x32 textures (mainly used on layer 1), but the rest could be anything, like this:
Well, this is a BIG problem! Normally, all games have their textures cropped like this:
But in this game, the textures do not always use the full size of the texture and are instead placed in the top-left corner.
Notice the separation between the top of the texture and the first pixel of the house—it has a separation of 32px…
(Yes I was dying internally while I was discovering this, just image having to do all of this shit just to render a png on an screen on 1999, I think the problem was not the hardware)
That is what is fucking us up!, all of the alignment issues are caused by this shitty decision.
To fix this issue (once you reach this conclusion, it’s simple: move the Y coordinate to place the origin as expected):
y = y - (grhData.pixelHeight / 2) / 32
(I’m not going to explain why it looks like that, but I’ll leave it to you :3)
If we add our fix, then the map will look like this:
Again, everything is flipped because of the coordinates. We have two options: re-encode the map by flipping each cell’s Y coordinate, or simply use Unity’s transform…
Finally, we have everything we wanted! It’s done; the rendering is finally complete. It took me about 6 months, on and off, to notice all of these issues. On this gist, you can see the rendering code I ended up using. https://gist.github.com/saulbensach/a3da8ee4c0913e41a3a52691d0c9e943
As you can see, is a bit slow because I’m actually reading the files and building everything at runtime. My next goals are a big refactor for all maps in order to bring them closer to home, and who knows, maybe use some tools that already exists to work with them.
Bye. Thanks for reading.