Recently, it became clear that most developers prefer to write nearly everything in C++, as opposed to previous plans (IIRC) to, for example, write the AI part in Python.
I understand that most of you don't have the time and/or will to learn another language right now, and I don't want to step on anybodies toes with this. I'm just wondering how much we want to expose the program internals to python, as doing so would make mods more easy and some tedious non-timecritical parts could be written in python by those of us how want to.
As I'm waiting for tzlaine to finish the EditControl in GG so I can do the chatwindow, I could start to write the interface using boost_python and also prepare the configure-scripts for this task.
I want to know, if I should do that, and how much of the interface we want to be exposed.
Python integration?
Moderator: Committer
-
drekmonger
- Space Kraken
- Posts: 146
- Joined: Mon Jul 07, 2003 10:17 am
The UI doesn't need to be exposed. In fact, nothing on client side should be exposed.
Universe, empire, and "trait" style objects ought to be exposed. (trait objects include technologies, specials, race picks, and facilities)
In my own little fantasy, rather than listing out trait style objects in a text (er xml) file and hardcoding the effects, each trait would be defined by it's own class. Futher, these classes ought be written out in a high level lanaguage, one class per file. An aspect of the class ought to tell the program where and how the trait fits into the game.
For example, plantary special trait classes would have the list of weights used to randomly place the trait into the universe. The program would read all the traits stashed in the correct directory, instance a single object for each class, and include wieghts and pointers in a manager object.
edit: erm, one more thing.
Traits would interact with the core engine though an event model, like GUI widgets or Inform game objects.
Anyway, this is just my smalltallk experience talking. I know from experience that the doing OO the "right way" is often too cumbersome for real world problems...
...but in this instance I'm 83.24% certain that it will save us lots of work and time in the end.
Universe, empire, and "trait" style objects ought to be exposed. (trait objects include technologies, specials, race picks, and facilities)
In my own little fantasy, rather than listing out trait style objects in a text (er xml) file and hardcoding the effects, each trait would be defined by it's own class. Futher, these classes ought be written out in a high level lanaguage, one class per file. An aspect of the class ought to tell the program where and how the trait fits into the game.
For example, plantary special trait classes would have the list of weights used to randomly place the trait into the universe. The program would read all the traits stashed in the correct directory, instance a single object for each class, and include wieghts and pointers in a manager object.
edit: erm, one more thing.
Traits would interact with the core engine though an event model, like GUI widgets or Inform game objects.
Anyway, this is just my smalltallk experience talking. I know from experience that the doing OO the "right way" is often too cumbersome for real world problems...
...but in this instance I'm 83.24% certain that it will save us lots of work and time in the end.
Re: Python integration?
I think that what you really need to ask is what does Python buy you that you can't get in C++? What is its learning curve? Then toss in the small factor that this is a volantary project and some of the programmers actually have lives outside of programming.Yoghurt wrote:Recently, it became clear that most developers prefer to write nearly everything in C++, as opposed to previous plans (IIRC) to, for example, write the AI part in Python.
I understand that most of you don't have the time and/or will to learn another language right now, and I don't want to step on anybodies toes with this. I'm just wondering how much we want to expose the program internals to python, as doing so would make mods more easy and some tedious non-timecritical parts could be written in python by those of us how want to.
As I'm waiting for tzlaine to finish the EditControl in GG so I can do the chatwindow, I could start to write the interface using boost_python and also prepare the configure-scripts for this task.
I want to know, if I should do that, and how much of the interface we want to be exposed.
Thumper
-
drekmonger
- Space Kraken
- Posts: 146
- Joined: Mon Jul 07, 2003 10:17 am
Over the past week, I've discovered that the learning curve for python is minimal. Anyone who knows C should have no problems at all picking it up; plus it seems like a useful thing to know professionally, decent resume fodder.
The real question is whether or not there's a significant RIO for the time expended implementing and debugging the interface. I believe the answer may be "yes" because:
1: We'll be able to change scripts without having to relink the entire program. As the program grows in size, the time wasted here might become significant.
2: High level languages are much easier to debug than C. I'll go out on a limb and say "much much much easier".
3: We'll be to give inexperienced coders a place in the development process. I, for example, feel no joy at all when working with C++ or assembly--would be much more likely to contribute significant amounts of code if I could work with a high level language.
There have been a few people who obviously don't know jack about C, but still would like to help out. I'd feel better about these inexperienced coders working with python and a FreeOrion API than messing around with low level code--they'd be able to get up to speed faster and have less of a chance introducing game crashing errors to the core engine.
The real question is whether or not there's a significant RIO for the time expended implementing and debugging the interface. I believe the answer may be "yes" because:
1: We'll be able to change scripts without having to relink the entire program. As the program grows in size, the time wasted here might become significant.
2: High level languages are much easier to debug than C. I'll go out on a limb and say "much much much easier".
3: We'll be to give inexperienced coders a place in the development process. I, for example, feel no joy at all when working with C++ or assembly--would be much more likely to contribute significant amounts of code if I could work with a high level language.
There have been a few people who obviously don't know jack about C, but still would like to help out. I'd feel better about these inexperienced coders working with python and a FreeOrion API than messing around with low level code--they'd be able to get up to speed faster and have less of a chance introducing game crashing errors to the core engine.
I would encourage everyone out there to quickly check an online tutorial for Python. It really is quite easy to pick up.
Like these ones:
http://www.python.org/doc/current/tut/tut.html
http://honors.montana.edu/~jjc/easytut/easytut/
Like these ones:
http://www.python.org/doc/current/tut/tut.html
http://honors.montana.edu/~jjc/easytut/easytut/
I am very much in favor of using python to this end, as I've expressed before 
I am planning to learn it very soon, and so if you want to look into that Yoghurt I, at least, would appreciate it.
At the very least putting the foundation there if that's possible so we can expand where needed. Though I don't actually understand much of what's involved to expose and integrate with python.
I am planning to learn it very soon, and so if you want to look into that Yoghurt I, at least, would appreciate it.At the very least putting the foundation there if that's possible so we can expand where needed. Though I don't actually understand much of what's involved to expose and integrate with python.
-
drekmonger
- Space Kraken
- Posts: 146
- Joined: Mon Jul 07, 2003 10:17 am
The ability to tweak scripts between turns and see the results just by pushing the "next turn" button would be a godsend for debugging and play-balancing. Compared to recompiling, linking, and re-running the program if it's all c++....
The programmers should seriously consider moving as much game logic as possible over to python as early in the development process as possible. I'm betting that coding the game logic in python will shave monthes (if not years) off of v1.0's release date.
(Of course, objects like "planet" that need to be shared by the client and server should remain in c++.)
The programmers should seriously consider moving as much game logic as possible over to python as early in the development process as possible. I'm betting that coding the game logic in python will shave monthes (if not years) off of v1.0's release date.
(Of course, objects like "planet" that need to be shared by the client and server should remain in c++.)
First of all, let me state that I'm not against using scripting for some things (story elements or event triggers, for example), but IMO all this talk about migrating all the game logic to python is going way too far...
dim x as object
set x = nothing
' do other stuff, forgetting to initialize x
' Oooops this is a fatal runtime error
x.CallSomeMethod()
This kind of thing is just as easy to debug in C or C++ as it is in VB if you know what you're looking for. Just because there are people who want to help but dont work well in C++ is not, in my mind, a good reason to complicate things by making the app bilingual.
It is going to be much much much harder to debug a bilingual application, because when we code the C++ bindings to the interpretter, we'd essentially have to assume that we have no idea what that thing is going to be doing. It is much easier to make a module crash resistant if you know what all the code will be doing than if you dont.
Here's the main problem with migrating turn processing to python:
The core data structures (empire and universe stuff), should be in C++, because they're used in the client, and the client HAS to be in C++, because there's no way you're ever going to convince Zach to re-write GG in python
The way things are now, a lot of the turn processing code for the universe side is stubbed out in the form of virtual methods in the objects themselves. For instance, Planet::PopulationGrowthPhase() (or whatever its called). If you want to move just the turn processing code to python, its going to involve recoding the universe in python, OR giving arbitrary python code access to the private sections of the universe objects to manipulate them, which in my mind is a flagrant sin against OO programming.
If you recode the universe in python, it means you're going to have to either put a python interpretter in the client, or maintain two different universe versions in two different languages. Neither of these options are acceptable to me. We shouldn't have to put a python interpretter in the client just because the server has one, and I, for one, refuse to maintain anything that's duplicated in two languages.
You're talking about the entire Universe and Empire modules here. All of these things (Empires, planets, fleets, systems) are shared by client and server. If you want to expose the universe and empire modules in python, then you need to re-write all of these.drekmonger wrote:
(Of course, objects like "planet" that need to be shared by the client and server should remain in c++.)
This works both ways. It would be good for the C people to learn python, but it would be just as good (better IMHO) for the python people to learn C. There is much, much, much more existing code out there in C than in Python, so in terms of resume filler, C is better anyway.Anyone who knows C should have no problems at all picking it up; plus it seems like a useful thing to know professionally, decent resume fodder.
I wouldn't feel too good about inexperienced coders working with anything. We can let them help, but they shouldn't be tasked with writing large, important pieces of code unless we know they're capable of doing it well. You can introduce bugs no matter which language you're working in, and the inexperienced coders will always introduce more bugs than the experienced ones.
There have been a few people who obviously don't know jack about C, but still would like to help out. I'd feel better about these inexperienced coders working with python and a FreeOrion API than messing around with low level code--they'd be able to get up to speed faster and have less of a chance introducing game crashing errors to the core engine.
I do not see what the difference is. A logic error is a logic error, whether it's coded in C++, python, Java, even LISP. It's perfectly easy to write game breaking errors in high level languages. Here's an example, in VB:2: High level languages are much easier to debug than C. I'll go out on a limb and say "much much much easier".
dim x as object
set x = nothing
' do other stuff, forgetting to initialize x
' Oooops this is a fatal runtime error
x.CallSomeMethod()
This kind of thing is just as easy to debug in C or C++ as it is in VB if you know what you're looking for. Just because there are people who want to help but dont work well in C++ is not, in my mind, a good reason to complicate things by making the app bilingual.
It is going to be much much much harder to debug a bilingual application, because when we code the C++ bindings to the interpretter, we'd essentially have to assume that we have no idea what that thing is going to be doing. It is much easier to make a module crash resistant if you know what all the code will be doing than if you dont.
This is, I will admit, an advantage of python. This is actually my biggest complaint right now, as I have never seen a compiler work as slowly as DevC++ has been on my system. Once you build the entire app though, the time it takes to recompile one module and relink is significantly less.1: We'll be able to change scripts without having to relink the entire program. As the program grows in size, the time wasted here might become significant.
Here's the main problem with migrating turn processing to python:
The core data structures (empire and universe stuff), should be in C++, because they're used in the client, and the client HAS to be in C++, because there's no way you're ever going to convince Zach to re-write GG in python
The way things are now, a lot of the turn processing code for the universe side is stubbed out in the form of virtual methods in the objects themselves. For instance, Planet::PopulationGrowthPhase() (or whatever its called). If you want to move just the turn processing code to python, its going to involve recoding the universe in python, OR giving arbitrary python code access to the private sections of the universe objects to manipulate them, which in my mind is a flagrant sin against OO programming.
If you recode the universe in python, it means you're going to have to either put a python interpretter in the client, or maintain two different universe versions in two different languages. Neither of these options are acceptable to me. We shouldn't have to put a python interpretter in the client just because the server has one, and I, for one, refuse to maintain anything that's duplicated in two languages.
Empire Team Lead
I agree with Josh completely. To toss out good OO practices on such a large project in favor of lowering barriers to entry for people who will probably write borken code is the worst kind of insanity to me. I posted some stuff on the thread about planet specials/features that applies here, but now I can't find that thread.
If the motivating factor is faster turnaround times when tweaking game settings (avoiding recompiles and relinking), then we should put all our settings into config files whenever possible, to achieve the same thing. The code itself should not be changed "between turns" anyway, even if that code is in an interpreted language. Running code in this situation does not verify how it will work in your game, but how it will work in a situation that will never exist in your game, since you are running it under starting conditions provided by old code that will not be executed in the final version, after the change.
If the motivating factor is faster turnaround times when tweaking game settings (avoiding recompiles and relinking), then we should put all our settings into config files whenever possible, to achieve the same thing. The code itself should not be changed "between turns" anyway, even if that code is in an interpreted language. Running code in this situation does not verify how it will work in your game, but how it will work in a situation that will never exist in your game, since you are running it under starting conditions provided by old code that will not be executed in the final version, after the change.
No! Just as you don't need to rewrite your library if some program does link to it, we only need to define the interface to python, and we'd be able to create a new SpamProductionCenter by using something likeYou're talking about the entire Universe and Empire modules here. All of these things (Empires, planets, fleets, systems) are shared by client and server. If you want to expose the universe and empire modules in python, then you need to re-write all of these.
Code: Select all
from Universe import ProdCenter
class SpamProductionCenter(ProdCenter):
def AdjustWorkforce(workforce):
....I agree completely with you in this matter. But theh fact that someone is coding in Python does not mean that he's an inexperienced coder.I wouldn't feel too good about inexperienced coders working with anything. We can let them help, but they shouldn't be tasked with writing large, important pieces of code unless we know they're capable of doing it well. You can introduce bugs no matter which language you're working in, and the inexperienced coders will always introduce more bugs than the experienced ones.
In this example the high-level language would raise an exception, and you could theoretically examine the code using the python console and continue working, C++ would coredump, and you'll have to use gdb to track the error down, and recompile everything.I do not see what the difference is. A logic error is a logic error, whether it's coded in C++, python, Java, even LISP. It's perfectly easy to write game breaking errors in high level languages. Here's an example, in VB:
The reason for using python is not to make development easier for newbies, it's the possibility to write working code faster, making mods easier is a nice side-effect. IMHO, this is well worth the effort. (Of course we need to specify which parts are written in python and which are not)Just because there are people who want to help but dont work well in C++ is not, in my mind, a good reason to complicate things by making the app bilingual.
Hmm, I think I'm missing something here, but the same thing applies the C++ code that #includes something.It is going to be much much much harder to debug a bilingual application, because when we code the C++ bindings to the interpretter, we'd essentially have to assume that we have no idea what that thing is going to be doing.
Did you compile with the -O2 switch? It really slows down compile-time.This is, I will admit, an advantage of python. This is actually my biggest complaint right now, as I have never seen a compiler work as slowly as DevC++ has been on my system. Once you build the entire app though, the time it takes to recompile one module and relink is significantly less.
If we wanted to, we could expose GG to Python without needing to change a single line of GG code.The core data structures (empire and universe stuff), should be in C++, because they're used in the client, and the client HAS to be in C++, because there's no way you're ever going to convince Zach to re-write GG in python
If the virtual PopulationGrowthPhase in a subclassed Planet don't have access to the private members, why should Python-subclasses?The way things are now, a lot of the turn processing code for the universe side is stubbed out in the form of virtual methods in the objects themselves. For instance, Planet::PopulationGrowthPhase() (or whatever its called). If you want to move just the turn processing code to python, its going to involve recoding the universe in python, OR giving arbitrary python code access to the private sections of the universe objects to manipulate them, which in my mind is a flagrant sin against OO programming.
I probably misunderstood something here...
OK, if you are against including a python interpreter into the client, there's nothing we can do. But why are you against it? Most of our users will probably also install the server along with the client, which already needs Python.dll, the client would simple use this.If you recode the universe in python, it means you're going to have to either put a python interpretter in the client, or maintain two different universe versions in two different languages. Neither of these options are acceptable to me. We shouldn't have to put a python interpretter in the client just because the server has one, and I, for one, refuse to maintain anything that's duplicated in two languages.
What is your argument about including python into the client?
-
drekmonger
- Space Kraken
- Posts: 146
- Joined: Mon Jul 07, 2003 10:17 am
It's not tweaking game settings that I'm talking about--it's debugging and tweaking working code. Not sure if you programed in any high level language extensively--the ability to change code real time is a huge advantage in the debugging process. High level coders use this technique extensively: it might sound scary to a C++ coder, but it does work and it works fast.If the motivating factor is faster turnaround times when tweaking game settings (avoiding recompiles and relinking), then we should put all our settings into config files whenever possible, to achieve the same thing.
Many interpretted languages can be debugged real time.I do not see what the difference is. A logic error is a logic error, whether it's coded in C++, python, Java, even LISP. It's perfectly easy to write game breaking errors in high level languages. Here's an example, in VB:
Due to their brevity compared to C++, high level languages are easier to read (and it's faster to type out code to begin with). It's the same type of difference between debugging assembly and C: it's just easier for a human to read C.
It would be the height of stuipidity to move any part of the client to python, for security and performance reasons. By extension, we should keep the shared objects/code in C++ on the server so that we don't have to maintain two copies of the same code.The core data structures (empire and universe stuff), should be in C++, because they're used in the client, and the client HAS to be in C++, because there's no way you're ever going to convince Zach to re-write GG in python
I've got a model worked out half way worked out in my head that maintains the integrity of UniverseObject while adding in "Traits". Traits are simlair to the widgets contained by a GUI window. While the traits could be implemented in C++, the idea is to give an obvious place to hook python code.The way things are now, a lot of the turn processing code for the universe side is stubbed out in the form of virtual methods in the objects themselves. For instance, Planet::PopulationGrowthPhase() (or whatever its called). If you want to move just the turn processing code to python, its going to involve recoding the universe in python, OR giving arbitrary python code access to the private sections of the universe objects to manipulate them, which in my mind is a flagrant sin against OO programming
The methods/events onMovePhase, on EconomyPhase, onWhateverPhase would still be called by the engine to each Universeobject and empire. The difference is, these objects would pass the event and a reference to themselves to their scripted traits. The traits would act though the same public methods available to C++ (but limited to objects and methods we write interfaces for). Most of the working game logic would be contained by traits.
er, yeah. That seems obvious to me now that I've some time to reflecte on this subject. (my earilier call to rewrite the entire server in python was an error on my part).If you recode the universe in python, it means you're going to have to either put a python interpretter in the client, or maintain two different universe versions in two different languages. Neither of these options are acceptable to me.
Because the client is purely in C++ (as it should be) the results of the game logic has to be stored in C++. Many results are stored in various member properites of the UniverseObject, many more could be stored in lists of Descriptors.
For example, the player needs to see what planet specials are on a world. So a planet special would add it's name, OwnerID, sorting priority, tooltip text, and HelpID to a Descriptor attached (in a list) to a planet. Given the various properities of a Descriptor, the client will be able to parse out where and how the information should be displayed.
One small advantage: there's very little game logic floating around the client. The client doesn't need to know what "Gravity" does to a planet--it just needs to display the results.
This is by far my biggest worry about using Python to script large parts of the program code. Putting the code for multiple aspects of an object in different places that all perform operations on that object is more C-style than C++-style (more procedural than OO). The problem I have with this is that by making this change we will be 1) breaking the OO idiom we use throughout the rest of the project, and moreover 2) chucking several principles that are not necessarily OO out the window, such as data hiding, encapsulation, and modularity.I've got a model worked out half way worked out in my head that maintains the integrity of UniverseObject while adding in "Traits". Traits are simlair to the widgets contained by a GUI window. While the traits could be implemented in C++, the idea is to give an obvious place to hook python code.
Also, the issue of debugging has come up several times. While I'm sure that it's easier in many cases to debug an interpretted language, I'm not so sure it is as easy to do so when your interpretter is buried in one of your executables.
That aside, bilingual code makes it difficult for anyone who is not bilingual to debug anything, since she cannot say for sure that the code she is looking at in the language she doesn't know is working as she thinks it does. I am not going to become as expert in Python as I am in C++. It just won't happen; I use C++ every day, and have no use for Python other than this part-time project. That means that I fall into this category of people who are disadvantaged by the bilingual code. Very frequently, errors in newbie code comes from simple misunderstandings of the language. Looking things up in language references takes up a lot of newbie time while debugging. I can't remember the last time I had to refer to The C++ Programming Language when debugging code. I don't just say all this for selfish reasons. I believe that most or all of us, and most or all of those who may join us later know one of C++ or Python better than the other. Asking them to know both equally expertly is too large a burden, no matter how easy the core functionality of either is to learn when you know the other.
-
drekmonger
- Space Kraken
- Posts: 146
- Joined: Mon Jul 07, 2003 10:17 am
Adding a button to a GUI window doesn't break the OO model. These "traits" I'm talking about are "game logic widgets". In any case, I believe the plan all along was to make facilities, technolgies, planet specials seprate objects "attached" to parents. What I'm saying here is derive all of these various classes from a single virtual class: a "trait".This is by far my biggest worry about using Python to script large parts of the program code. Putting the code for multiple aspects of an object in different places that all perform operations on that object is more C-style than C++-style (more procedural than OO).
A trait would basically be a wrapper for a python class. And even if the program is 100% C++, I still think that "game logic widgets" are the best way of expressing the ideas we need to express.
(I thought "traits" up while trying to figure out how to implement a tech tree with rich behaviors without burdening non-tech objects with a bunch of special case code....that breaks encapsulation. We want all the effects of a tech, a planet special, facilitiy, race pick, or whatever contained within the trait itself rather than spewing the code all over the program in various classes.)
In addition, other pieces of game logic (such as planet size, enviroment) could be traits. The end result is that no effect is hardcoded into an object: we can add and remove effects real time. (for example, when a player builds a building or researches a new tech, a trait is added to the approprate object).
I don't expect heavy c++ coders to learn python extensively. There'd be some people responsible for the python aspects and others for the C++. Python code is readable by anyone who knows C++--it's only the nuisances that the python experts would have to debug.
In a large program it's not required that a single person to be intimately aware of how every piece of the program works.
-
OceanMachine
- Pupating Mass
- Posts: 95
- Joined: Thu Jun 26, 2003 12:09 pm
- Location: Chicago
I agree that we want to keep the main core of the game in c++. I do think it might be nice to have some of the AI modules done with python to encourage tinkering by players who may not want to bother with setting up their environment and trying to recompile everything, which can be a bit of a chore, as we've seen.
But before we commit to doing anything in python I'd like to see a demo of python integration so we can all see a working example of how these interfaces would need to be set up. Yoghurt, were you volunteering to do something like this when you started the thread?
But before we commit to doing anything in python I'd like to see a demo of python integration so we can all see a working example of how these interfaces would need to be set up. Yoghurt, were you volunteering to do something like this when you started the thread?
Programming Lead
Yep, I will do it. I'm away till Thursday, but then I'll start.OceanMachine wrote:But before we commit to doing anything in python I'd like to see a demo of python integration so we can all see a working example of how these interfaces would need to be set up. Yoghurt, were you volunteering to do something like this when you started the thread?
Anyone who wants to help and/or has suggestion how we should do it (drek for example), drop me a PM, I'll respond as soon as I'm back.