fr_stringtable

Discuss, plan, and make Translations for FreeOrion
Message
Author
User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#211 Post by Ouaz »

Vezzra wrote:
Dilvish wrote:If Cj adjusted his cleanup script so that it did not quite totally remove untranslated entries, but instead commented them out (both key and value) then it seems to me that both sets of concerns would be met.
That sounds like a reasonable compromise to me.
Oh, I finally understand: just add a "#" before the untranslated strings (keys) or not yet translated (values).

Totally OK for me, that would work like a charm (as the file structure is preserved).
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Cjkjvfnby
AI Contributor
Posts: 539
Joined: Tue Jun 24, 2014 9:55 pm

Re: fr_stringtable

#212 Post by Cjkjvfnby »

Ouaz wrote:
Cjkjvfnby wrote:
Yes, but it won't work for me (then again :P ).

When I translate, I translate "in-game", i.e. Freeorion is started, and I edit my latest FR file directly in the game repository. Then I can use the "Flush Stringtables" button in the options, to check if the translation fits within the UI for example, or to see if the translation context is good (sometimes, without seeing the sentences in the game itself, it's hard to understand what it is talking about ^^)
It dumps result directly to fr.txt at your command.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Cjkjvfnby
AI Contributor
Posts: 539
Joined: Tue Jun 24, 2014 9:55 pm

Re: fr_stringtable

#213 Post by Cjkjvfnby »

Vezzra wrote:
Dilvish wrote:If Cj adjusted his cleanup script so that it did not quite totally remove untranslated entries, but instead commented them out (both key and value) then it seems to me that both sets of concerns would be met.
That sounds like a reasonable compromise to me.
The idea with full files was to not check en.txt when you start translation. Open file and replace english with you language.

https://github.com/Cjkjvfnby/freeorion/ ... les/fr.txt
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#214 Post by Dilvish »

Thanks for putting that together, Cj. I notice though that it doesn't preserve line number matching between the FR and EN entries-- 14035 total lines for the FR version vs. 14112 for EN, so at least some of the entries must be offset. Ouaz, that should be easy enough for you to go through and add commented blank lines to fr.txt to get the entries to match up, yes?

It sounds to me like we may as well go ahead and put this version into the main repo. Ouaz can then simply work with the main repo copy and submit a PR when he has new translations to add. It sounds like Ouaz will be happy enough to keep manually checking the commented-out entries to see if the en.txt version has changed, or if en.txt has new entries he can also copy them into fr.txt (commented out with a leading '#' until he actually translates them).

There could be some benefit for Cj to periodically run his script to make sure that changed EN entries don't get overlooked (if the script were augmented to also preserve key line number matching), but I think it would be ok to leave it up for Ouaz to decide if he wants to coordinate on that extra bit of assistance.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Cjkjvfnby
AI Contributor
Posts: 539
Joined: Tue Jun 24, 2014 9:55 pm

Re: fr_stringtable

#215 Post by Cjkjvfnby »

Dilvish wrote:
Thanks for putting that together, Cj. I notice though that it doesn't preserve line number matching between the FR and EN entries-- 14035 total lines for the FR version vs. 14112 for EN, so at least some of the entries must be offset. Ouaz, that should be easy enough for you to go through and add commented blank lines to fr.txt to get the entries to match up, yes?
My script removes double blank lines. But PR with it to en.txt is not merged.
Dilvish wrote:It sounds to me like we may as well go ahead and put this version into the main repo.
Do you know that you can create pull request from my branch yourself?
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#216 Post by Dilvish »

Cjkjvfnby wrote:
Dilvish wrote:It sounds to me like we may as well go ahead and put this version into the main repo.
Do you know that you can create pull request from my branch yourself?
Yes, I did know. I meant to invite discussion (or at least concurrence :D ) rather than simply saying "Please submit a PR."
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#217 Post by Ouaz »

Dilvish wrote:
Thanks for putting that together, Cj. I notice though that it doesn't preserve line number matching between the FR and EN entries-- 14035 total lines for the FR version vs. 14112 for EN, so at least some of the entries must be offset. Ouaz, that should be easy enough for you to go through and add commented blank lines to fr.txt to get the entries to match up, yes?
Yes, these 75 lines are the double blank lines not yet removed in EN.txt. No problem to deal with it in the FR file until the PR is accepted for EN.txt. :wink:
It sounds to me like we may as well go ahead and put this version into the main repo. Ouaz can then simply work with the main repo copy and submit a PR when he has new translations to add. It sounds like Ouaz will be happy enough to keep manually checking the commented-out entries to see if the en.txt version has changed, or if en.txt has new entries he can also copy them into fr.txt (commented out with a leading '#' until he actually translates them).
Yes, I will be happy. :P

However, there are some things I don't understand:

1) I thought that each key will be commented. Currently, the commented keys are only those for which the value is not yet translated (or the french word is the same than in english, e.g. infrastructure = infrastructure):
comment_untranslated.PNG
comment_untranslated.PNG (22.47 KiB) Viewed 6268 times
As you see, the Artificial Planet description is commented (both key and value, because the value is not yet translated), but the Subterranean Habitation keys aren't (because the value is already translated). Wouldn't be more logic to comment each key (for which their values are translated or not)? -although I keep them up-to-date regularly - :mrgreen:


2) As I said, some french words are the same than in english (e.g. production = production). So, there is a bunch of "translated strings" that are commented:
french_same_english.PNG
french_same_english.PNG (15.76 KiB) Viewed 6268 times
These three french words are the same than in english. So, can I "uncomment" them? (to mark that is actually translated)


3) For Cj's: sometimes, some strings are not commented, whereas it should be (not a problem though, I can manually fix it):
missing_comment.PNG
missing_comment.PNG (17.25 KiB) Viewed 6268 times
As you see, Endomorphic IV and Fractal I are commented (both key and value, because I kept the same name in the french file), but Gravitator I isn't. Happens with some tech description stuff too.



Otherwise, everything is perfect. I can work easily and quickly as before, with this "commented" FR file.

Thanks to all of you, guys. 8)
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#218 Post by Dilvish »

Ouaz wrote:1) I thought that each key will be commented. Currently, the commented key are only those for which the value is not yet translated (or the french word is the same than in english, e.g. infrastructure = infrastructure): As you see, the Artificial Planet description is commented (both key and value, because the value is not yet translated), but the Subterranean Habitation keys aren't (because the value is already translated). Wouldn't be more logic to comment each key (for which their values are translated or not)?
No, the keys are how the entries are found-- commenting them out totally disables them, so we do not want to comment out entries you have actually translated (with perhaps the limited exception discussed below).

2) As I said, some french words are the same than in english (e.g. production = production). So, there is a bunch of "translated strings" that are commented: These three french words are the same than in english. So, can I "uncomment" them? (to mark that is actually translated)
Since the translations are the same as in en.txt, there is no real harm in them being commented out, it seems. But to avoid confusion it probably would be better to flag such things (whether or not Cj's script were adjusted to recognize the flag). It would be nice to be able to simply add a "// translated" on the same line as the key for these entries that are properly unchanged when translated. Unfortunately, right now it appears that adding a comment to the end of a stringtable line for a key breaks the stringtable formatting. Geoff, being able to have a comment on the same line after a stringtable key would probably be reasonable to implement, would you have any objections? Since adding a comment line before the key would throw off the line matching you are trying to maintain, I think we should just leave such entries commented out.

3) For Cj's: sometimes, some strings are not commented, whereas it should be (not a problem though, I can manually fix it):
As you see, Endomorphic IV and Fractal I are commented (both key and value, because I kept the same name in the french file), but Gravitator I isn't. Happens with some tech description stuff too.
You may have overlooked-- the en.txt stringtable entry for this key is no longer the same as what is in your fr.txt entry-- that's the exact problem we are trying to avoid, and (not to wag fingers, except, well, perhaps a little bit), it shows that the review process for changed en.txt entries you have been trying to do is difficult to really be reliable about if you are doing it manually.

**edit: so on this last issue, untranslated entries that the script doesn't currently detect because the en.txt has already changed, unless Cj adjusts his script as I inquired about above (a somewhat ambitious prospect), then these entries all need to be manually updated-- but I think that is what you have already said you want to do as a general matter.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13586
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: fr_stringtable

#219 Post by Geoff the Medio »

Dilvish wrote:Unfortunately, right now it appears that adding a comment to the end of a stringtable line for a key breaks the stringtable formatting. Geoff, being able to have a comment on the same line after a stringtable key would probably be reasonable to implement, would you have any objections?
No objections, seems like a useful addition.

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#220 Post by Ouaz »

Dilvish wrote:No, the keys are how the entries are found-- commenting them out totally disables them, so we do not want to comment out entries you have actually translated (with perhaps the limited exception discussed below).
Oh, OK, I missed that. No problem then.

Since adding a comment line before the key would throw off the line matching you are trying to maintain, I think we should just leave such entries commented out.
Ok, there are just a few cases, so it's not a problem.

You may have overlooked-- the en.txt stringtable entry for this key is no longer the same as what is in your fr.txt entry-- that's the exact problem we are trying to avoid, and (not to wag fingers, except, well, perhaps a little bit), it shows that the review process for changed en.txt entries you have been trying to do is difficult to really be reliable about if you are doing it manually.
8) It's not something I missed, I remember now. "Gravitating" means nothing in French, unlike "Gravitator", which has a connotation even in french (big "gravitating" ship). That's why I kept "Gravitator" in first place. ^^ Actually, it's like it is translated.

For the others strings not commented (values in Tech descriptions), it's because I haven't checked them yet (the manual sync with the english is finished from line 1 to 7376 -and updated again when the EN file is modified-, line 7376 being currently the point I am regarding the translation/proofreading/sync).

I thought that Cj's script would have commented them anyway (after line 7376), even if they are different from the english ones. But I understand now why it didn't (although I don't understand now how his previous script removed all the untranslated strings, even if there were differences with the EN file... Whatever :p).

So, no need of finger-wagging. :mrgreen:
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#221 Post by Dilvish »

Ouaz wrote:I thought that Cj's script would have commented them anyway (after line 7376), even if they are different from the english ones.
I'm quite sure it did not. He mentioned he has done some additional manual cleanup of those kinds of entries for ru.txt; if you are sure they were deleted from some version of fr.txt he gave you then he must have done that manually as well.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Ouaz
Dyson Forest
Posts: 232
Joined: Wed Aug 13, 2014 7:21 pm
Location: France

Re: fr_stringtable

#222 Post by Ouaz »

Me again...

Finally, using the FR "commented" file (https://github.com/Cjkjvfnby/freeorion/ ... les/fr.txt) directly in-game, it seems there is a problem:
stringtable_problem.PNG
stringtable_problem.PNG (4.81 KiB) Viewed 6248 times
that causes to display the keys for all the Pedia Links instead of the values, only into the description stuff:
pedia_link_error.PNG
pedia_link_error.PNG (48.78 KiB) Viewed 6248 times
Here the "Detection Range" pedia link (key instead value).

If I revert to the previous fr.txt (the latest one without comments > https://github.com/Cjkjvfnby/freeorion/ ... ile_fr.txt), everything works fine again.

:?
I release every updated file under the CC-BY-SA 3.0 license.

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13586
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: fr_stringtable

#223 Post by Geoff the Medio »

Geoff the Medio wrote:
Dilvish wrote:Unfortunately, right now it appears that adding a comment to the end of a stringtable line for a key breaks the stringtable formatting. Geoff, being able to have a comment on the same line after a stringtable key would probably be reasonable to implement, would you have any objections?
No objections, seems like a useful addition.
Done, I think.
Ouaz wrote:...it seems there is a problem: [Stack Space Exhausted]
I also get that. There is one other mention of it on the forums, and various boost mailing list posts. I don't know if it's easily fixable.

Adding some debug output, the last matched key is right before the "Name Lists" section, which is a big long series of commented out lines. I'm guessing that many comment lines in a row is causing a problem? Normally the regex wouldn't see something like that.

User avatar
Dilvish
AI Lead and Programmer Emeritus
Posts: 4768
Joined: Sat Sep 22, 2012 6:25 pm

Re: fr_stringtable

#224 Post by Dilvish »

Geoff the Medio wrote:Done, I think.
I notice though that your change allows the post-key whitespace to include newlines rather than just blanks and tabs, and also leaves the parser sensitive to whitespace after a key if it is not also followed by a comment (which could easily happen if someone removes a comment). I would suggest that instead of

Code: Select all

        KEY >> (_n | *space >> COMMENT) >>
we use

Code: Select all

        KEY >> *blank >> (_n | COMMENT) >>
Ouaz wrote:...it seems there is a problem: [Stack Space Exhausted]
I also get that. There is one other mention of it on the forums, and various boost mailing list posts. I don't know if it's easily fixable.
To be clear, Ouaz, this is not any kind of inherent bug with the modified stringtable -- it works fine for me. It's just that it apparently causes a bit more processing demands, just enough to cause a problem for you. Shutting down other applications might free up enough stack space for you.

It seems that boost::xpressive default is to do even more backtracking than boost::spirit does. "keep" can be used to limit backtracking for a subexpression (perhaps it works out the same as spirit '>' versus '>>'), and I tested that it appears to work fine for for the COMMENT, REFERENCE and KEYEXPANSION definitions. It seems likely to help reduce stack demands, so I committed these changes. Ouaz, after the next weekly builds please try this commented stringtable file again and see if your parser can handle it then (if simply shutting down other apps wasn't enough).

**edit: Geoff, I just saw that apparently you can reproduce the problem, so please test these additional tweaks out to see if they solve it for you.
If I provided any code, scripts or other content here, it's released under GPL 2.0 and CC-BY-SA 3.0

User avatar
Geoff the Medio
Programming, Design, Admin
Posts: 13586
Joined: Wed Oct 08, 2003 1:33 am
Location: Munich

Re: fr_stringtable

#225 Post by Geoff the Medio »

Dilvish wrote:...please test these additional tweaks out to see if they solve it for you.
They do not.

After some additional debug logging and selective string uncommenting, it seems to be failing on the long string of comment lines in the Name Lists section.

Post Reply