fr_stringtable

Discuss, plan, and make Translations for FreeOrion
Message
Author
User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#226 Post by Dilvish »

Geoff the Medio wrote:
Dilvish wrote:...please test these additional tweaks out to see if they solve it for you.
They do not.
Could you try changing the comment definition to

Code: Select all

    const sregex COMMENT = keep('#' >> keep(*(~_n)) >> _n);
and see if that does it (based on a suggestion from here)
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13586
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: fr_stringtable

#227 Post by Geoff the Medio »

Dilvish wrote:Could you try changing the comment definition...
With that, starting up with fr.txt selected causes the game window to pop up, but it is all white and not responding, and using a lot of cpu. The log file is empty. After a minute or so I killed it.

My own attempt:

Code: Select all

*(space | +COMMENT) >>
Has a similar result.

In case it was missed at the end of the page...
Geoff the Medio wrote:After some additional debug logging and selective string uncommenting, it seems to be failing on the long string of comment lines in the Name Lists section.
Edit: Adding a non-commented entry near the end seems to fix it.

Code: Select all

MIDWAY_IGNORED
Ignored
inserted on line 12662 lets it parse. Just adding spaces doesn't help (eg. breaking up the big list of star names, if all those lists remain commented out).

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#228 Post by Dilvish »

Geoff the Medio wrote:Edit: Adding a non-commented entry near the end seems to fix it.

Code: Select all

MIDWAY_IGNORED
Well, that's certainly better than no fix at all. Ouaz, you could just manually add that, deleting a corresponding number of commented-out names from the starname list, which won't really matter.

That will probably muck with Cj trying to run his scripts on the file again without a fair bit of monkeying on his part, but he doesn't really need to so I guess that's ok.

I do have one more idea for a possible coding fix, though Geoff, if you could try this when you have a few spare minutes. Please try making the second line of the ENTRY definition be

Code: Select all

keep(*(space | COMMENT)) >>
and if that causes the weird hang again then try deleting the keep() from the COMMENT definition (but leaving this new keep in ENTRY).
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13586
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: fr_stringtable

#229 Post by Geoff the Medio »

Dilvish wrote:Please try making the second line of the ENTRY definition be

Code: Select all

keep(*(space | COMMENT)) >>
and if that causes the weird hang again...
It doesn't hang, but it still produces the stack space exhausted error.
...then try deleting the keep() from the COMMENT definition (but leaving this new keep in ENTRY).
Same result.

Edit:

Code: Select all

    const sregex COMMENT = '#' >> *(~_n) >> _n;
[...]
    const sregex ENTRY =
        keep(*(space | +COMMENT)) >>
        KEY >> *blank >> (_n | COMMENT) >>
        (("'''" >> MULTI_LINE_VALUE >> "'''" >> *space >> _n) | SINGLE_LINE_VALUE >> _n);
Seems to work without errors.

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#230 Post by Ouaz »

Dilvish wrote:
Geoff the Medio wrote:Edit: Adding a non-commented entry near the end seems to fix it.

Code: Select all

MIDWAY_IGNORED
Well, that's certainly better than no fix at all. Ouaz, you could just manually add that, deleting a corresponding number of commented-out names from the starname list, which won't really matter.
I did this (line 12652, deleting two lines before). It fixes the problem, but new errors appears in the log file:

Code: Select all

2015-04-18 16:55:21.049588 [debug] Client : Logger initialized
2015-04-18 16:55:21.049588 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_BEGINNER in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_TURTLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_CAUTIOUS in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_TYPICAL in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_AGGRESSIVE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_MANIACAL in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.080788 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.096388 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.096388 [error] Client : Unresolved reference: INFRASTRUCTURE_TITLE in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt.
2015-04-18 16:55:21.423988 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE
If i remove the comment before [INFRASTRUCTURE_TITLE] (line 3684), and before each [AI_CAPITOL_NAMES] (line 10675, 681, 685, 689, 694, 698), the log is fine again.

But like you said, the Name lists section is not needed, i.e. I could delete it from the FR file. The number of lines before it would be preserved and still be matched with the english file.

So I deleted all the Name Lists section and launched FO:

- The links in the Pedia are broken again (key instead of value)
- The greek letters (system and planet names) are not displayed when I launch a game.
- Only one new error in the log (INFRASTRUCTURE_TITLE and AI_CAPITOL_NAMES are gone):

Code: Select all

2015-04-18 17:40:56.749167 [debug] Client : Logger initialized
2015-04-18 17:40:56.749167 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 17:40:56.764767 [error] Client : StringTable file "C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt" is malformed around line 10802
2015-04-18 17:40:57.045568 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE
Line 10802 being the last line of the fr file.
broken_display.PNG
broken_display.PNG (253.5 KiB) Viewed 6239 times
With my working FR file, it is Thorne α , Thorne β , Nitzer β, and the Pedia links is correctly displayed.

By the way, if I use the "cleaned" fr file that is in Git main repository (https://github.com/freeorion/freeorion/ ... les/fr.txt), the same problem happens (except the Pedia links which are fine, as there are no commented lines).

On the both files, if I add only the STAR_GROUP_NAMES and STAR_GROUP_CHARS at the end of the file, the greek letters are displayed again, but I still have the same log error than above ("malformed around line *), and still the broken pedia links (key instead of value).

What a mess, I'm sorry to bother you with this. :?

EDIT:
Geoff the Medio wrote: Edit:

Code: Select all

    const sregex COMMENT = '#' >> *(~_n) >> _n;
[...]
    const sregex ENTRY =
        keep(*(space | +COMMENT)) >>
        KEY >> *blank >> (_n | COMMENT) >>
        (("'''" >> MULTI_LINE_VALUE >> "'''" >> *space >> _n) | SINGLE_LINE_VALUE >> _n);
Seems to work without errors.
I will wait the next build then.
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#231 Post by Dilvish »

Ouaz wrote:I did this (line 12652, deleting two lines before). It fixes the problem, but new errors appears in the log file:

Code: Select all

2015-04-18 16:55:21.049588 [debug] Client : Logger initialized
2015-04-18 16:55:21.049588 [debug] Client : v0.4.4+  [build 2015-04-13.d489d44] MSVC 2010
2015-04-18 16:55:21.080788 [error] Client : Unresolved key expansion: AI_CAPITOL_NAMES_BEGINNER in: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt
If i remove the comment before [INFRASTRUCTURE_TITLE] (line 3684), and before each [AI_CAPITOL_NAMES] (line 10675, 681, 685, 689, 694, 698), the log is fine again.
Hmm, that sounds familiar, that lookups had to be in the same stringtable. I think we had discussed this issue before; I think there is a potential solution via loading the default stringtable first as a backup for these lookups, and then providing that backup list into the routine that reads the stringtable. Now that we have started trying to using more macro subsitutions wherever possible it seems more important than ever. I'll try to follow up on that this weekend. Thanks much for your patience with this-- this is a good issue to get sorted out, with significance beyond the FR stringtable.
The greek letters (system and planet names) are not displayed when I launch a game.
Ah, yes, we had seen that quite some time back with the DE stringtable, and then forgot about that issue recently. Until we think of a solution the greek letter list will need to be an exception to the comment-out-if-not-translated rule.
...What a mess, I'm sorry to bother you with this. :?
No, no, we need to get these problems sorted out, even if you didn't care about how the FR stringtable was structured.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Cjkjvfnby
AI Contributor
Posts: 539
Joined: Tue Jun 24, 2014 9:55 pm

Re: fr_stringtable

#232 Post by Cjkjvfnby »

Dilvish wrote:
Ouaz wrote:Hmm, that sounds familiar, that lookups had to be in the same stringtable. I think we had discussed this issue before; I think there is a potential solution via loading the default stringtable first as a backup for these lookups, and then providing that backup list into the routine that reads the stringtable.
My bad. This is unexpected behavior for me. I like you potential solution. Should I wait for it or make pull request with add of this keys?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#233 Post by Dilvish »

Cjkjvfnby wrote:Should I wait for it or make pull request with add of this keys?
I said I would try to get to it this weekend (i.e., today or tomorrow, my time), but no promises on that. I expect very very few people are updating their stringtables over the weekend, but if you want to submit the pull request I won't argue with it, though I would hope to revert it within a day.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#234 Post by Dilvish »

ok, I have put in a github PR for a solution. I explain a basic test in the comments there. Any suggestions on ways to improve it before I commit to the main repo?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#235 Post by Ouaz »

Hi,

I tested the latest build:

- With the current FR stringtables in the main repo, everything works fine (values instead of keys in the Pedia) but the greek letters don't display (known issue). The AI_CAPITOL_NAMES and INFRASTRUCTURE_TITLE are checked with your "fallback" patch:

Code: Select all

2015-04-21 16:34:20.450366 [debug] Client : Key expansion: AI_CAPITOL_NAMES_MANIACAL not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
2015-04-21 16:34:20.465966 [debug] Client : Key reference: INFRASTRUCTURE_TITLE not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
- With my work-in-progress "commented" FR stringtables, the greek letters are displayed, but the Pedia links are broken:

Code: Select all

2015-04-21 16:29:28.992109 [debug] Client : Logger initialized
2015-04-21 16:29:28.992109 [debug] Client : v0.4.4+  [build 2015-04-20.e356dec] MSVC 2010
2015-04-21 16:29:29.070109 [error] Client : Exception caught regex parsing Stringtable: Regex stack space exhausted
2015-04-21 16:29:29.070109 [error] Client : Last and prior keys matched: , HOTKEY_FOCUS_NEXT_WND
2015-04-21 16:29:29.382109 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE
I fixed it by deleting all STAR_NAMES and all the stuff after. The FR file ended now with:

Code: Select all

10816 #ω'''
10817
10818 #STAR_GROUP_NAMES
10819 #Centauri
10820 
I kept "Centauri" as last line because it's an unique string, so it will be easy to check if the line number are matched with the english one (currently line 10819 on both files).

The log still mentions the fallback though:

Code: Select all

2015-04-21 16:45:03.492804 [debug] Client : Logger initialized
2015-04-21 16:45:03.492804 [debug] Client : v0.4.4+  [build 2015-04-20.e356dec] MSVC 2010
[...]
2015-04-21 16:45:03.570804 [debug] Client : Key expansion: AI_CAPITOL_NAMES_MANIACAL not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
2015-04-21 16:45:03.586405 [debug] Client : Key reference: INFRASTRUCTURE_TITLE not found in primary stringtable: C:\Program Files (x86)\FreeOrion\default\stringtables\fr.txt; checking in fallback fileC:\Program Files (x86)\FreeOrion\default\stringtables\en.txt
[...]
2015-04-21 16:45:03.836005 [debug] Client : OpenAL initialized. Version 1.1Renderer SoftwareVendor Creative Labs Inc.
Extensions: EAX EAX2.0 EAX3.0 EAX4.0 EAX5.0 EAX3.0EMULATED EAX4.0EMULATED AL_EXT_OFFSET AL_EXT_LINEAR_DISTANCE AL_EXT_EXPONENT_DISTANCE
.

Otherwise, evererything is OK.

I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37

Thanks.
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Cjkjvfnby
AI Contributor
Posts: 539
Joined: Tue Jun 24, 2014 9:55 pm

Re: fr_stringtable

#236 Post by Cjkjvfnby »

Ouaz wrote:Hi,
I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37
Thanks.
In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#237 Post by Dilvish »

Cjkjvfnby wrote:In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.
Ah, yes, you are probably right about that, but I have already accepted the PR. We could revert it & split it up, but I don't think it's really worth doing that, is it?

Ouaz wrote:The log still mentions the fallback though:
It still has to get those keys from the fallback en.txt, doesn't it?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#238 Post by Ouaz »

EDIT: Oops, didn't see there was new posts.
Cjkjvfnby wrote:
Ouaz wrote:Hi,
I pull a request to update the FR stringtable in the main repo: https://github.com/freeorion/freeorion/pull/37
Thanks.
In that case it will be better to have two commits:
1) bulk update with commented lines (update by script)
2) translation work.
It will be more easy to track changes.
You mean separate PRs each time? One for the "structure" modification (added, removed or modified lines in en.txt, merged in fr.txt) and one for updated french translation?

It will complicate things again...
Dilvish wrote:
Ouaz wrote:The log still mentions the fallback though:
It still has to get those keys from the fallback en.txt, doesn't it?
Yes. However, for the INFRASTRUCTURE_TITLE, it could be avoided as it is actually translated. See below.


----------------------------------
Thanks for the merge.

Now... ^^
Dilvish wrote:
2) As I said, some french words are the same than in english (e.g. production = production). So, there is a bunch of "translated strings" that are commented: These three french words are the same than in english. So, can I "uncomment" them? (to mark that is actually translated)
Since the translations are the same as in en.txt, there is no real harm in them being commented out, it seems. But to avoid confusion it probably would be better to flag such things (whether or not Cj's script were adjusted to recognize the flag). It would be nice to be able to simply add a "// translated" on the same line as the key for these entries that are properly unchanged when translated. Unfortunately, right now it appears that adding a comment to the end of a stringtable line for a key breaks the stringtable formatting. Geoff, being able to have a comment on the same line after a stringtable key would probably be reasonable to implement, would you have any objections? Since adding a comment line before the key would throw off the line matching you are trying to maintain, I think we should just leave such entries commented out.
What about this?

It seems Geoff and you have worked on this, but I don't understand very well (as always...)

Can you give me an example on how to flag these lines, to "mark" them as translated?

For example, for "Infrastructure" being "Infrastructure" also in french:

Code: Select all

#INFRASTRUCTURE_TITLE
#Infrastructure
INFRASTRUCTURE_TEXT
L'Infrastructure englobe toutes les structures et organisations qui assurent la pérennité et la production des populations de l'empire : l'énergie, les transports, les télécommunications, les services sociaux, etc.
Thanks again.
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#239 Post by Dilvish »

Ouaz wrote:Can you give me an example on how to flag these lines, to "mark" them as translated?
The following should now be fine for the parser:

For example, for "Infrastructure" being "Infrastructure" also in french:

Code: Select all

INFRASTRUCTURE_TITLE  # translated
Infrastructure
(you may want to test and confirm that to be sure)

Then if Cj were to plan on running a script to help with processing the file, he could have the script look for the extra "# translated" markers and know to leave those entries even if they are not otherwise noticeably translated. I think there is no rush for him to do this processing again, but it would be helpful if you mark as "# translated" any of the entries you uncomment. Please note that the "# translated" only goes after the entry's key, on the same line.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#240 Post by Ouaz »

Dilvish wrote:
Ouaz wrote:Can you give me an example on how to flag these lines, to "mark" them as translated?
The following should now be fine for the parser:

For example, for "Infrastructure" being "Infrastructure" also in french:

Code: Select all

INFRASTRUCTURE_TITLE  # translated
Infrastructure
(you may want to test and confirm that to be sure)
It works. Great. 8)

Just to be sure:

[key][space][space][#][space][translated] : that's right? (two spaces needed before #)
Then if Cj were to plan on running a script to help with processing the file, he could have the script look for the extra "# translated" markers and know to leave those entries even if they are not otherwise noticeably translated. I think there is no rush for him to do this processing again, but it would be helpful if you mark as "# translated" any of the entries you uncomment. Please note that the "# translated" only goes after the entry's key, on the same line.
No problem. I'm working on it.

Thanks!
I release every updated file under the CC-BY-SA 3.0 license.

Post Reply