Potential replacement of FOCS with Python
Moderator: Committer
Re: Potential replacement of FOCS with Python
I thought Python was not meant to be used as an engine to run the effects, but as a way to construct c++ effects data structures. As such, it would be ran only at the start, right?
https://github.com/mmoderau
[...] for Man has earned his right to hold this planet against all comers, by virtue of occasionally producing someone totally batshit insane. - Randall Munroe, title text to xkcd #556
[...] for Man has earned his right to hold this planet against all comers, by virtue of occasionally producing someone totally batshit insane. - Randall Munroe, title text to xkcd #556
Re: Potential replacement of FOCS with Python
I've managed to get AST from file in https://github.com/freeorion/freeorion/pull/3211 so I would like if someone from Python coders could provide Python look of game_rules.focs.txt.
It's better if it can pass CI checks for python.
I've prepared test but the only thing it can do is to validate if file have python syntax.
It's better if it can pass CI checks for python.
I've prepared test but the only thing it can do is to validate if file have python syntax.
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
dunno. like this(?):
Code: Select all
class RealGameRule():
def __init__(self, name, description, category, default, min, max):
self.name = name
self.description = description # could be defaulted to name + _DESC here
self.category = category # could be stronger typed
self.default = default
self.min = min
self.max = max
BALANCE="BALANCE"
game_rules_list = [
RealGameRule("RULE_SHIP_HULL_COST_FACTOR",
description="RULE_SHIP_HULL_COST_FACTOR_DESC",
category = BALANCE,
default = 1.0,
min = 0.1,
max = 10.0 ),
RealGameRule("RULE_SHIP_PART_COST_FACTOR",
description = "RULE_SHIP_PART_COST_FACTOR_DESC",
category = BALANCE,
default = 1.0,
min = 0.1,
max = 10.0 ),
]
for rule in game_rules_list:
print("Parsed ", rule.name)
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.
Look, ma... four combat bouts!
Look, ma... four combat bouts!
Re: Potential replacement of FOCS with Python
Code is only parsed as Python, not executed as Python.Ophiuchus wrote: ↑Sat Oct 31, 2020 1:25 pmdunno. like this(?):Code: Select all
class RealGameRule(): def __init__(self, name, description, category, default, min, max): self.name = name self.description = description # could be defaulted to name + _DESC here self.category = category # could be stronger typed self.default = default self.min = min self.max = max BALANCE="BALANCE" game_rules_list = [ RealGameRule("RULE_SHIP_HULL_COST_FACTOR", description="RULE_SHIP_HULL_COST_FACTOR_DESC", category = BALANCE, default = 1.0, min = 0.1, max = 10.0 ), RealGameRule("RULE_SHIP_PART_COST_FACTOR", description = "RULE_SHIP_PART_COST_FACTOR_DESC", category = BALANCE, default = 1.0, min = 0.1, max = 10.0 ), ] for rule in game_rules_list: print("Parsed ", rule.name)
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
Code: Select all
game_rules_list = [
RealGameRule("RULE_SHIP_HULL_COST_FACTOR",
description="RULE_SHIP_HULL_COST_FACTOR_DESC",
category = "BALANCE",
default = 1.0,
min = 0.1,
max = 10.0 ),
RealGameRule("RULE_SHIP_PART_COST_FACTOR",
description = "RULE_SHIP_PART_COST_FACTOR_DESC",
category = "BALANCE",
default = 1.0,
min = 0.1,
max = 10.0 ),
]
I am not sure why you want to use AST though (if that is what you want). To save the bytecode conversion/time spent on interpretation?
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.
Look, ma... four combat bouts!
Look, ma... four combat bouts!
Re: Potential replacement of FOCS with Python
Mostly because it could be unsafe to execute python code from untrusted sources and planned modding support could provide amount of untrusted sources.Ophiuchus wrote: ↑Sat Oct 31, 2020 7:57 pmI would prefer some execution before using the structure. So one can e.g. define BALANCE (making sure it is everywhere the same). Or use a loop to create multiple definitions.Code: Select all
game_rules_list = [ RealGameRule("RULE_SHIP_HULL_COST_FACTOR", description="RULE_SHIP_HULL_COST_FACTOR_DESC", category = "BALANCE", default = 1.0, min = 0.1, max = 10.0 ), RealGameRule("RULE_SHIP_PART_COST_FACTOR", description = "RULE_SHIP_PART_COST_FACTOR_DESC", category = "BALANCE", default = 1.0, min = 0.1, max = 10.0 ), ]
I am not sure why you want to use AST though (if that is what you want). To save the bytecode conversion/time spent on interpretation?
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
legitimate!
I am a bit wary of using the AST instead of processing python structures - i think it would lead to lots of boilerplate and hard to maintain code (happy to be proven wrong)
we could use AST to restrict the program to a safe subset of python (e.g. no imports,file access,...) or even a minimal subset. bail out if an unsupported feature is encountered and interpret if the given program is safe.
Any code or patches in anything posted here is released under the CC and GPL licences in use for the FO project.
Look, ma... four combat bouts!
Look, ma... four combat bouts!
- Geoff the Medio
- Programming, Design, Admin
- Posts: 13603
- Joined: Wed Oct 08, 2003 1:33 am
- Location: Munich
Re: Potential replacement of FOCS with Python
If you are so concerned about unsafe code, then why not use lua, which is much simpler to execute in a protected way?
You could also require that all code used in multiplayer games is downloaded from a central server that you manage and not from the server, that way you can make sure both that the code is alright and that the code is available.
You could also require that all code used in multiplayer games is downloaded from a central server that you manage and not from the server, that way you can make sure both that the code is alright and that the code is available.
Re: Potential replacement of FOCS with Python
Except CVE in Python parser it's impossible to make changes in filesystem or send files with passwords via network with parsing.
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
In case of adding new dependency we can use YAMLThinkSome wrote: ↑Sat Oct 31, 2020 11:39 pm If you are so concerned about unsafe code, then why not use lua, which is much simpler to execute in a protected way?
You could also require that all code used in multiplayer games is downloaded from a central server that you manage and not from the server, that way you can make sure both that the code is alright and that the code is available.
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
There nice thread I've just read about it https://stackoverflow.com/questions/306 ... ure-python and it looks like there bunch of holes in tries to execute code.
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Re: Potential replacement of FOCS with Python
YAML is not a programming language, hence very limiting in what one can do.o01eg wrote: ↑Sun Nov 01, 2020 6:41 amIn case of adding new dependency we can use YAMLThinkSome wrote: ↑Sat Oct 31, 2020 11:39 pm If you are so concerned about unsafe code, then why not use lua, which is much simpler to execute in a protected way?
You could also require that all code used in multiplayer games is downloaded from a central server that you manage and not from the server, that way you can make sure both that the code is alright and that the code is available.
Re: Potential replacement of FOCS with Python
I'm confused - we already do execute Python code (from potentially untrusted sources) - the universe generation scripts, the turn events scripts and the AI scripts are all executed Python code.
And all that code resides in the "default" folder, meaning it belongs to the user customizable/moddable stuff, where also all the FOCS stuff is. Replacing FOCS with Python just migrates the part of the user costomizable stuff that is now scripted in FOCS to Python.
And the idea to replace FOCS wih Python is not to change the FOCS stuff to Python syntax and then again use some tools to parse that stuff ourselves, but to actually execute those Python scripts and construct all the condition and effects objects via Python (as we already use Python that way for the other stuff, so we're not really adding anything or any new dependencies, just reuse it to also do the things we now use FOCS for).
Re: Potential replacement of FOCS with Python
That is why I proposed long ago to split it https://freeorion.org/forum/viewtopic.php?f=9&t=11538Vezzra wrote: ↑Sun Nov 01, 2020 12:21 pm I'm confused - we already do execute Python code (from potentially untrusted sources) - the universe generation scripts, the turn events scripts and the AI scripts are all executed Python code.
And all that code resides in the "default" folder, meaning it belongs to the user customizable/moddable stuff, where also all the FOCS stuff is. Replacing FOCS with Python just migrates the part of the user costomizable stuff that is now scripted in FOCS to Python.
Also if user doesn't host the game he doesn't execute all those python code and doesn't affected by its vulnerability.
As I understand if we have someVezzra wrote: ↑Sun Nov 01, 2020 12:21 pm And the idea to replace FOCS wih Python is not to change the FOCS stuff to Python syntax and then again use some tools to parse that stuff ourselves, but to actually execute those Python scripts and construct all the condition and effects objects via Python (as we already use Python that way for the other stuff, so we're not really adding anything or any new dependencies, just reuse it to also do the things we now use FOCS for).
Code: Select all
def func():
return 2+3
Code: Select all
Operation(OpType::PLUS, std::make_unique<Constant>(2), std::make_unique<Constant>(3))
Gentoo Linux x64, gcc-11.2, boost-1.78.0
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm
Ubuntu Server 22.04 x64, gcc-12, boost-1.74.0
Welcome to the slow multiplayer game at freeorion-lt.dedyn.io.Version 2024-04-14.ad50e93.
Donations're welcome:BTC:bc1q007qldm6eppqcukewtfkfcj0naut9njj7audnm