Questions about architecture

Programmers discuss here anything related to FreeOrion programming. Primarily for the developers to discuss.

Moderators: Committer, Committer

Post Reply
Message
Author
User avatar
Rydra
Space Squid
Posts: 55
Joined: Fri Jun 28, 2013 6:15 pm

Questions about architecture

#1 Post by Rydra » Fri Oct 04, 2013 6:25 pm

Hi there,

Right now I'm performing an UML modeling of the whole freeorion architecture (which I think it will be useful in order to give a higher degree of visualization and documentation). However, while analyzing the code, I've seen that several classes that are interconnected (e.g. Empire, Diplomacy, EmpiresManager, Ship, etc...) when associating them they store the ID integer instead of a pointer to the object (e.g. storing which empire built a Ship). Why store the ID int instead of a Empire* pointer? What's the reason?

Thank you and greetings.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12529
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Questions about architecture

#2 Post by Geoff the Medio » Fri Oct 04, 2013 7:09 pm

Serialization, robustness, and inertia. Lots of things reference other things by ID instead of pointer in FreeOrion code. This is useful in cases where there might be multiple instances in memory of the relevant object (not applicable to empires), and it makes state dumping easier than would a pointer. There are also cases where the in-memory object associated with a particular ID might be replaced with a new object at a new location in memory (such as when turn updates come in). By using IDs instead of pointers, there's no need to update anything that references the updated object. Storing everything with IDs also makes it easier to do safety checks, as one can look up the object with the relevant ID and check that a non-null pointer was returned, but could not so easily verify that a pointed-to memory location still contained a valid object.

User avatar
Rydra
Space Squid
Posts: 55
Joined: Fri Jun 28, 2013 6:15 pm

Re: Questions about architecture

#3 Post by Rydra » Fri Oct 04, 2013 9:05 pm

Geoff the Medio wrote:Serialization, robustness, and inertia. Lots of things reference other things by ID instead of pointer in FreeOrion code. This is useful in cases where there might be multiple instances in memory of the relevant object (not applicable to empires), and it makes state dumping easier than would a pointer. There are also cases where the in-memory object associated with a particular ID might be replaced with a new object at a new location in memory (such as when turn updates come in). By using IDs instead of pointers, there's no need to update anything that references the updated object. Storing everything with IDs also makes it easier to do safety checks, as one can look up the object with the relevant ID and check that a non-null pointer was returned, but could not so easily verify that a pointed-to memory location still contained a valid object.
Allow me to perform some questions for the sake of understanding and learning. I've read the following topic about serialization:

http://www.boost.org/doc/libs/1_39_0/li ... ation.html

It says that a "A type T is Serializable if and only if one of the following is true: it is a pointer to a Serializable type." If that pointer is of a Serializable type... doesn't it serve its purpose? Why would using an ID instead of a pointer ease the serialization?

Then I'm also dubious about robustness. Doesn't having to manage multiple IDs everywhere (instead of pointers pointing to the same object) create duplicities? I mean, imagine (just as an example) you change the ID of an Empire. With a pointer you'd just change the object's ID. By storing ids instead of pointers, you have to change it everywhere. Am I wrong?

I understand however your point about the management of placing the object from the server at another side of the memory. But... couldn't this somehow be managed by, for instance, reserving memory heap with "new"s instead of using the memory stack?

Also, I've noted the mixed use of classes and structs. Given that their differences are minimum at best (default visibility), why mixing them instead of using one sole type of structure declaration? Just wondering.

Forgive me if I do noobie questions. I come from a .NET/Java/PHP/SOAP world and C++ is a complete new world and a heavy challenge with those pointers and memory considerations.
Last edited by Rydra on Sat Oct 05, 2013 7:00 am, edited 3 times in total.

User avatar
Dilvish
AI Lead, Programmer
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: Questions about architecture

#4 Post by Dilvish » Fri Oct 04, 2013 10:31 pm

Although it might very well be possible to use pointers in most places instead of int IDs, keep in mind that most objects exist as a multiplicity -- the Server's ObjectMap has the 'Gold Standard' copy of the object with 100% accurate info, but the server also maintains distinct object maps for each Empire, having distinct copies of each object visible to that Empire, and with varying degrees of information (and up-to-date-edness) for each object. Although serialization of pointers to serializable objects might work fine for archival purposes, I think it would be difficult at best to make that work with our visibility system, and the incremental update process that we hope to eventually implement. That's not to say things couldn't be changed, it's just a bit more detail behind the 'inertia' mentioned by Geoff.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Bigjoe5
Designer and Programmer
Posts: 2058
Joined: Tue Aug 14, 2007 6:33 pm
Location: Orion

Re: Questions about architecture

#5 Post by Bigjoe5 » Sun Oct 06, 2013 12:54 am

Rydra wrote:It says that a "A type T is Serializable if and only if one of the following is true: it is a pointer to a Serializable type." If that pointer is of a Serializable type... doesn't it serve its purpose? Why would using an ID instead of a pointer ease the serialization?
Serializing pointers incurs significantly more overhead than serializing value types, since you ('you' meaning boost, in this case) need to keep track of every object that's pointed to by anyone, and make sure that you create that object only once, and have all the right objects on the other end pointing to that same object. Furthermore, there are plans to move away from serializing/deserializing the entire universe each turn, and instead only send the diff between the client and the server. This would certainly wreak havoc if the diffs contained pointers to objects, which would then be deserialized on the other end, creating whole new duplicate objects, when the intention was really just to point to a different, previously existing object.
Rydra wrote:Then I'm also dubious about robustness. Doesn't having to manage multiple IDs everywhere (instead of pointers pointing to the same object) create duplicities? I mean, imagine (just as an example) you change the ID of an Empire. With a pointer you'd just change the object's ID. By storing ids instead of pointers, you have to change it everywhere. Am I wrong?
An object's or empire's ID is invariant by design. While I'm reasonably certain that an Empire will never change memory location, that invariant is not explicitly guaranteed by our architecture, so I would say that it is actually more likely that an empire's address will change (invalidating any pointers to that empire) than that an empire's ID will change (invalidating any stored empire IDs referencing that empire).
Rydra wrote:I understand however your point about the management of placing the object from the server at another side of the memory. But... couldn't this somehow be managed by, for instance, reserving memory heap with "new"s instead of using the memory stack?
This is already handled by boost::serialization, and we don't need to deal with it.
Rydra wrote:Also, I've noted the mixed use of classes and structs. Given that their differences are minimum at best (default visibility), why mixing them instead of using one sole type of structure declaration? Just wondering.
It's just a matter of taste and communicating intention, really. If a type doesn't have a responsibility (aside from holding field-based data), I like to call it a struct. I think most of the code in FO is more or less consistent with this philosophy as well.
Rydra wrote:Forgive me if I do noobie questions. I come from a .NET/Java/PHP/SOAP world and C++ is a complete new world and a heavy challenge with those pointers and memory considerations.
No worries - ask away.
Warning: Antarans in dimensional portal are closer than they appear.

User avatar
Rydra
Space Squid
Posts: 55
Joined: Fri Jun 28, 2013 6:15 pm

Re: Questions about architecture

#6 Post by Rydra » Mon Oct 07, 2013 8:02 pm

Bigjoe5 wrote:Serializing pointers incurs significantly more overhead than serializing value types, since you ('you' meaning boost, in this case) need to keep track of every object that's pointed to by anyone, and make sure that you create that object only once, and have all the right objects on the other end pointing to that same object. Furthermore, there are plans to move away from serializing/deserializing the entire universe each turn, and instead only send the diff between the client and the server. This would certainly wreak havoc if the diffs contained pointers to objects, which would then be deserialized on the other end, creating whole new duplicate objects, when the intention was really just to point to a different, previously existing object.
I'm not sure of what I will say, but the matter of diffs... could it be dealt by making the client store the stream he recieves from the server, perform turn actions and then send to the server the diff between the previous received stream and the new turn status, generated in the client? Just asking.
An object's or empire's ID is invariant by design. While I'm reasonably certain that an Empire will never change memory location, that invariant is not explicitly guaranteed by our architecture, so I would say that it is actually more likely that an empire's address will change (invalidating any pointers to that empire) than that an empire's ID will change (invalidating any stored empire IDs referencing that empire).
Can't argue as I'm still learning C++ memory management
Rydra wrote:I understand however your point about the management of placing the object from the server at another side of the memory. But... couldn't this somehow be managed by, for instance, reserving memory heap with "new"s instead of using the memory stack?
This is already handled by boost::serialization, and we don't need to deal with it.
If you say boost already deals with this... why worry about memory location then :o

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 12529
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: Questions about architecture

#7 Post by Geoff the Medio » Mon Oct 07, 2013 8:42 pm

Rydra wrote:If you say boost already deals with this... why worry about memory location then :o
Boost ensures that if you deserialize an archive, all the pointer references to objects in the archive will be restored sensibly. It doesn't ensure that any external pointers (ie. not part of the archive) that previously contained addresses of related objects that were deleted when the archive was deserialized will still be valid.

Post Reply