Page 3 of 5

Re: Python3 migration

Posted: Wed Jan 15, 2020 8:48 pm
by Cjkjvfnby
My progress so far.

PR with tests compatiable with Python 3 is ready for review. https://github.com/freeorion/freeorion/pull/2711

TODO:
  • Work with problem places reported by pylint (Here is an example. https://github.com/freeorion/freeorion/pull/2709). Blocked by pr with tests.
  • Setup mypy and check problem places. (It has different check than pylint) Blocked by pr with tests.
  • Add flake8 for python 3 code checks, we will need it after migration. Probably blocked by pr with tests.
PS. I will be away this weekend.

Re: Python3 migration

Posted: Fri Jan 17, 2020 11:29 am
by Vezzra
adrian_broher wrote: Tue Jan 14, 2020 8:27 amFYI the majority of Linux Platforms are dropping Python2 with their next release. MacOS will drop support with 10.16.
Which basically means we're actually way overdue with the migration to Python3, and 0.4.9 won't be included in the repos for upcoming Linux distro releases.

Well, not much we can do about that, but the more urgent it will be to get the migration done within the next release cycle.

I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?

Re: Python3 migration

Posted: Fri Jan 17, 2020 11:31 am
by Vezzra
o01eg wrote: Tue Jan 14, 2020 9:13 amConsidering https://fedorapeople.org/groups/schedul ... tasks.html looks like freeorion will be missed in Fedora 32.
Here we go...

Re: Python3 migration

Posted: Fri Jan 17, 2020 11:46 am
by o01eg
Vezzra wrote: Fri Jan 17, 2020 11:29 am I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?
I would like to get Goverment merged and make empire readiness refactoring.

Re: Python3 migration

Posted: Fri Jan 17, 2020 11:58 am
by Vezzra
o01eg wrote: Fri Jan 17, 2020 11:46 amI would like to get Goverment merged and make empire readiness refactoring.
The Government branch is about to be merged, as far as I understand, so that I expect to happen anyway.

Regarding the empire readiness refactoring, as far as I understood, that is supposed to address certain issues, so can be seen as a bug fixing effort. Those can and should go in as well of course.

But aside from such things it might be a good idea to get another release out with the Python3 migration dealt with as soon as possible (by summer maybe?).

Re: Python3 migration

Posted: Fri Jan 17, 2020 1:16 pm
by Geoff the Medio
Vezzra wrote: Fri Jan 17, 2020 11:58 am
o01eg wrote: Fri Jan 17, 2020 11:46 amI would like to get Goverment merged and make empire readiness refactoring.
The Government branch is about to be merged, as far as I understand, so that I expect to happen anyway.
I'd like to get the Government branch merged soon as well, and could just click twice to do that. But realistically it's a substantial new feature that will take a lot of time to get adequately fleshed out with content, balanced, and integrated with the AI.

Given the importance of getting an update with Python3 support, and given that substantial progress towards that is currently happening, I'm somewhat inclined to hold off on the Government merge and to try to get a Python3 update out as soon as possible after v0.4.9.

It's a bit of a hassle to keep the branch separate, but interested people can check out the Government branch now and start providing feedback and help with further development of it without merging into master.

Re: Python3 migration

Posted: Fri Jan 17, 2020 1:28 pm
by Ophiuchus
Geoff the Medio wrote: Fri Jan 17, 2020 1:16 pm It's a bit of a hassle to keep the branch separate, but interested people can check out the Government branch now and start providing feedback and help with further development of it without merging into master.
I could ship fresh government based snaps if v0.4.9 is out in parallel to the master weekly builds.
But probably only if somebody explicitly asks for it - i never got any feedback on the builds i did in the past.

Re: Python3 migration

Posted: Fri Jan 17, 2020 1:35 pm
by o01eg
Vezzra wrote: Fri Jan 17, 2020 11:29 am
adrian_broher wrote: Tue Jan 14, 2020 8:27 amFYI the majority of Linux Platforms are dropping Python2 with their next release. MacOS will drop support with 10.16.
Which basically means we're actually way overdue with the migration to Python3, and 0.4.9 won't be included in the repos for upcoming Linux distro releases.

Well, not much we can do about that, but the more urgent it will be to get the migration done within the next release cycle.

I'm wondering if we should actually focus only on the migration to Python3 now and get out another release as soon as that's done - thoughts, opinions?
Technically I suppose it possible to backport Python-only changes to release branch and make 0.4.9.1 release with python2->3 port. They will be network compatible, but default/ files from 0.4.9 won't be compatible with default/ files from 0.4.9.1.

Re: Python3 migration

Posted: Sun Jan 19, 2020 2:34 pm
by Vezzra
Geoff the Medio wrote: Fri Jan 17, 2020 1:16 pmI'd like to get the Government branch merged soon as well, and could just click twice to do that. But realistically it's a substantial new feature that will take a lot of time to get adequately fleshed out with content, balanced, and integrated with the AI.

Given the importance of getting an update with Python3 support, and given that substantial progress towards that is currently happening, I'm somewhat inclined to hold off on the Government merge and to try to get a Python3 update out as soon as possible after v0.4.9.
That sounds reasonable, and if you prefer to hold off on (bigger) new features and do a quick release cycle focused on the migration to Phyton3, I'm very much inclined to second that.

Re: Python3 migration

Posted: Sun Jan 19, 2020 2:48 pm
by Vezzra
o01eg wrote: Fri Jan 17, 2020 1:35 pmTechnically I suppose it possible to backport Python-only changes to release branch and make 0.4.9.1 release with python2->3 port. They will be network compatible, but default/ files from 0.4.9 won't be compatible with default/ files from 0.4.9.1.
In theory that is of course possible, but I don't think it's really feasible. On Windows and macOS we do not rely on a Python environment provided by the OS (as it's the case on Linux), but ship the Python framework with the installer. Also, the SDKs for Windows and macOS have the Python framework packaged.

That means, to do a 0.4.9.1 release with Python3 we need to switch SDKs on the release branch. This is something I definitely do not want to do. Changes that are big enough to require a new SDK version go beyond the scope of bugfix releases and should require a new major release.

Aside from that, the switch to Python3 requires changes to practically all Python scripts (AI, universe generation and turn events). Separating these adjustments from all the other changes to the Python scripts that went into master (and those that most likely will go into master) after the 0.4.9 release branch had been created, to cherry pick them to the release branch sounds like a nightmare. One I do not want to deal with.

I have considered that option myself (backporting the Python3 migration to the 0.4.9 release branch), but for the reasons cited I discarded that idea.

Re: Python3 migration

Posted: Thu Jan 23, 2020 7:23 pm
by Cjkjvfnby
I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?

Re: Python3 migration

Posted: Fri Jan 24, 2020 1:50 pm
by Vezzra
Cjkjvfnby wrote: Thu Jan 23, 2020 7:23 pm I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?
I see. Well, I can try to squeeze it in... depending on if we declare the official release on Sunday. If we do, you'll get new test builds of master on Tuesday anyway. If not, I can see what I can do.

You're on Windows, right?

Re: Python3 migration

Posted: Tue Jan 28, 2020 8:25 pm
by Cjkjvfnby
Vezzra wrote: Fri Jan 24, 2020 1:50 pm
Cjkjvfnby wrote: Thu Jan 23, 2020 7:23 pm I miss master build for my playtesting. My build is not compatiable with FOCS from master anymore. Can you provide a new one for me?
I see. Well, I can try to squeeze it in... depending on if we declare the official release on Sunday. If we do, you'll get new test builds of master on Tuesday anyway. If not, I can see what I can do.

You're on Windows, right?
Yes I am on windows.

Re: Python3 migration

Posted: Sun Feb 02, 2020 1:39 pm
by Vezzra
Sorry for the delay, have been quite busy last week.

I provided an inofficial test build (build of latest master as of the time of this post), I uploaded the binaries only package (assuming you don't need the full installer):

https://sourceforge.net/projects/freeor ... p/download

Re: Python3 migration

Posted: Sun Feb 09, 2020 6:01 pm
by Cjkjvfnby
Migration to python3 will be delivered in the next release.

Do we plan to release any other features?

I try to list all the required actions to get it ready:
  • Python 2, 3 compatibilities.
    I am working on it. I think we can do it in a week.
  • C++ changes.
    Looks like it is almost done for AI, what about UniverseGeneration and TurnEvents?
  • Linux, mac builds.
    Do we need to do something here or it will work from the box?
  • Windows build.
    We need to ship python binaries together with the installer.
  • Remove compatibility from the python code
    I don't know if we want to do it in the release or later.
  • Linux distributives
    Some distributives excluded FreeOrion from their repositories, need to fix that. I am not familiar with that world.