Inner workings of Graal Reborn explained (a little bit)

Ok, I will try to summarize part of what I have learn about Graal Reborn (GR).
Mostly, inside architecture.
I suspect what I will say will be known by most, I am sorry, I came to GR without passing by Graal, and I am old.
So I must have learn very slowly. In fact, this will probably show how much I don’t understand things,
as much as it could help a newbie learn about GR.

First let’s present most components very very very briefly:
-client: written with an old directx version, allows you to play the game
-gserver: that’s the game server. It load levels, and knows about NPC (non-player characters).
-listserver: As the name says, it contains the list of servers, and allows to connect and edit players profiles

-The remote control. As the name says, it allows to control the gserver remotely (from an other computer).

-The NPC server, is optional, and not really used in Graal Reborn yet because developing versions are incomplete.
It is a special client, that execute scripts for the NPCs for all clients, rather than having each client execute them.
In fact, scripts can have a server-side and a client-side. NPC server would execute the server-side part of scripts.
NPC are not only monsters, bar tenders, shop tenders, but can be any visual object that react to others (players or npc).
In fact, GR only allows to attach scripts to npc. And so a lot of NPC are not visible, and are created only to set some
properties of a level, or tell to do things when a player enter the level.

So, let’s summarize again the very big picture.

At the start of the world, the listserver is launched. Normaly, the Graal Reborn forums, give instructions how to
get and setup your listserver account with same name as GR forum, but with a possibly, hopefully different password.
The listserver wait for clients, or gservers to connect to it.

Gserver(s) when launched, connect to the listserver.
And disconnect (or are disconnected by listserver if not responding for too long) before quitting.
Because of that, listservers have a list of running gservers.

Then clients can connect to the unique listserver (the listserver url is hardwired in the client).
Client ask listserver to verify their account (username + password).
BTW, the current clients sent in the clear (not encrypted account info), but our listserver have a “new” feature
to check account information in encrypted format, which our future own client could use.
So, if the listsever is happy with the account (by example it is not globally banned -ban on listserver is global ban-),
then the client will have the possibility to request the list of servers from the listserver.
In fact, our listserver can be compiled with an option that avoid using MySQL server. In this mode, the listserver is
always happy with the username and password whatever the password given. Hopefully, the listserver GR use, is not compiled
with this NO_MYSQL option.

I now believe it keep connected to the listserver (by example, to update player profile).
So, the listserver knows about the list of running gservers, but also of connected clients (players).

The human player choose one of the server he wish to play, and the client open a connection to the corresponding gserver.

From there, the client and the gserver will be connected together by a (TCP?) connection.

So at least these three connections exist for GR:
client <–> listserver
client <–> gserver
gserver <–> listserver

Optional connections are:
remotecontrol <–> gserver
npcServer <–> gserver

If the npcServer <–> gserver connection exist, the gserver will not send the serverside part of the scripts to the clients.
But rather send them to the npcServer to be executed.

Components communicate through packets. Which is basically a special string (CString) that knows how to receive, parse, send, compress and uncompress packets. The first byte of each packet identify the type (command) of the packets, so that in theory up to 255 type of commands may exist.

gserver will receive (in packets) from each client (player), and from the listserver (often refer to simply as SV in the code).
listserver will reseive (in packets) from each gserver and each clients.
gserver can and do send a numbers of (out packets) to each clients (players), and to the listserver.
listserver can and do send a numbers of (out packet) to each gserver and each clients (players).

It make very much sense to expect that for each in packets from a component, there exist an out packet in the other component.
It could possibly make code better if the .h file would refrain to call them in or out packets, and use the same .h file.
Code could call the function stored in inDispatcher[id] … thinking. There is no outDispatcher, because packet are sent when needed, but packets are received “asynchronously”.

I intend to show where to find these in the code, in a future post.

I like what you’ve done here. It surprises me, but to my knowledge no one has collected this information into one place like this. Good job!

Ok. So let’s go a bit further inside the code. In part to see if this is true that components that send packets should send the same kind of packets that the receiver that receive it.
A word about our components.

Most of our code (eh all?) is at github.com. It used to be at GoogleCode but it was transferred to github before GoogleCode decide to close and automatically move everything at github.com. So we now have stuff at github.com in double. The latest stuff, is the automatically added repositories from GoogleCode:
https://github.com/OpenGraal/gs2emu-googlecode Basically, it contains everything as it was left on GoogleCode, before we switched, so it is probably not to the latest version, and should be avoided.

When moved from GoogleCode to Github.com, every sub-project have move inside it’s own repository. Now, I believe it was not done really right, and projects files for Visual Studio were broken. It is part of my recent fixes to fix these projects files so to work fine. Alas as far as I know, no one have have pickup my fixs for this. Well, I did not learn, bother to make pull requests, so this could be considered my fault.
So, right now, the best code for Graal Reborn are mine:
https://github.com/dufresnep?tab=repositories
rather than the official one:
https://github.com/OpenGraal
That said, I will use official one here, as the code itself is really the same, and the official one should stay longer in theory.

I will begin with NPC servers, because we have two (one in C++ and the other in C#) and this will allows me to compare them:

https://github.com/OpenGraal/NpcServer-v1 : This is the C++ NPC server. It used to compile GameMonkey scripts (http://www.gmscript.com/) which can be quite similar to GS2 (Graal Script version 2).

https://github.com/OpenGraal/OpenGraal.NpcServer-cs/tree/master/OpenGraal.NpcServer : OurC# written NPC server. (We have a C++ one) Wish I would know more. Seems most of the time, NPC is abbreviated to NC in code. It used to use .NET compiler to compile C# scripts, see: https://msdn.microsoft.com/en-us/library/system.codedom.compiler%28v=vs.110%29.aspx I have said before I worry about security issues related to this. (Seems almost no limit on what an NPC script can do). That said, it is/was considered the most complete promising NPC server. But more recently, it use scripting functionality from Common-cs, which means it use JavaScript.

Next, let’s see what other components other than NPC servers, we have:

https://github.com/OpenGraal/OpenGraal.RemoteControl-cs : Well, never built it. Seems to have code written with Gtk. Quite enough code in C# to do something usefull. Need to investigate it more.

https://github.com/OpenGraal/GServer-v2 : That’s our Server. Written in C++ and using libBoost which is a very known library to add containers, manage threads, and way lot more to C++.

https://github.com/OpenGraal/GServer-v1 : An obsolete gserver kept for archive purpose.

https://github.com/OpenGraal/OpenGraal.Common-cs : From the readme.md: “OpenGraal.Common is a helper class library for .NET/Mono written in C# for handling Graal Online related assets. It currently contains classes for manipulating animations, maps, levels and players. It also contains some classes related to scripting for an npc-server and a client that uses these assets.” Scripting is nowadays, done with Microsoft ClearScript, configured to use Google V8 (Javascript) engine. This is huge (about 2GB), but is believed to give fast runtime, and allows to send plain javascript to clients (which might be considered to clear, but it the easiest, compare to compiling to some predefined bytecode).

https://github.com/OpenGraal/OpenGraal.Common.Interfaces-cs: The only almost useful thing in there would be this small description of what a gani is:https://github.com/OpenGraal/OpenGraal.Common.Interfaces-cs/blob/master/OpenGraal.Common.Interfaces/Animations/IGaniObj.cs

https://github.com/OpenGraal/OpenGraal.Core-cs/tree/master/OpenGraal.Core : This one is really interesting and used by almost if not all other C# projects. The most needed and important one is https://github.com/OpenGraal/OpenGraal.Core-cs/blob/master/OpenGraal.Core/CString.cs that deals, with making and decoding packets among other things, and compressing and decompressing them.

https://github.com/OpenGraal/OpenGraal.Common.Connections-cs I believe it is not used yet.
This is mostly this file: https://github.com/OpenGraal/OpenGraal.Common.Connections-cs/blob/master/OpenGraal.Common.Connections/Client/GraalServer.cs that seems like a nice canvas for building a new GServer in C#.

https://github.com/OpenGraal/OpenGraal.Common.Scripting-cs/tree/master/OpenGraal.Common.Scripting : This one is a compiler for C# compiler (used for compiling NPC scripts). This seems to be the system described in:
http://forums.opengraal.com/showthread.php?7088-Imperim-Standard-Engine-(OpenGraal-Client)/page5 for the Imperim engine. Is it used by OpenGraal.NpcServer-cs?

https://github.com/OpenGraal/OpenGraal.MapGenerator-cs/tree/master/OpenGraal.MapGenerator : Well, I am like you, I can guess what it makes by the name.

https://github.com/OpenGraal/RemoteControl-v1 Well, according to it?s name, this one is a Remote Controller. It seems to be made with QT (it has a .pro to compile with qmake). And reuse a lot of other C++ files. Never tried to compiled it, and never heard about.

https://github.com/OpenGraal/GraalServerListDumper I guess this is a program to list the running gservers (so connected to the listserver). The ?simple seems to give full details. This is written in C++. Did read somewhere on the forum that there was some problems when built on Linux, but have no confirmation at all. I never compiled it.

https://github.com/OpenGraal/GraalImageSearch : Ok, here is my guess about what it does. Inside levels files (mostly .nw nowadays, but there was .graal files too), there are NPCs. Some NPC have a predefined image associated. So I believe this programs read all the levels files from the current directory, and extract the images it found for the NPC in these, and write them out in the directory.

https://github.com/OpenGraal/ServerList-v2 This is the C++ code for our (mostly only one) listserver.

https://github.com/OpenGraal/serverlist-v3 This seems to be an “empty” shell for a future listserver.

https://github.com/OpenGraal/OpenGraal.LevelEditor-cs/tree/master/LevelEditor : our C# gtk Level editor. It works, but is very minimal at the moment. It is in competition with Gonstruct at http://londeroth.org/~fry/gonstruct/ with the code at:
https://github.com/fry/graal-gonstruct (check forks, they might be more evolved? did not look yet).