[i18n] how to convert .po files to "=" separated .txt files

User avatar
Hamlet
Member
 
Posts: 452
Joined: Sat Jul 29, 2017 21:09
Location: Italia
IRC: H4mlet

[i18n] how to convert .po files to "=" separated .txt files

by Hamlet » Thu Feb 01, 2018 13:11

A foreword: please do not turn this thread into a debate/flame about ".po Vs .txt", please bring that somewhere else.

Using Intllib's xgettext.sh one can swiftly parse a/multiple .lua file(s) to generate a template.pot file, which can be easily edited to support multiple language translations, this is the structure of .po(t) files:

Code: Select all
#case 1 (used for short strings)

msgid "hi"
msgstr "ciao"

#case 2 (used for long, descriptive strings)

msgid ""
"this is a"
"longer string"
"containing a lot"
"of informations"
msgstr ""
"questa è una"
"stringa più lunga"
"contentente un sacco"
"di informazioni"


However nowadays localisation seems to rely on simple .txt files where the original string and the target language's translated string are separated by a "=", e.g.:

Code: Select all
hi = ciao
this is a longer string containing a lot of informations = questa è una stringa più lunga contenente un sacco di informazioni


Is there any script like Intllib's xggettext.sh able to natively parse .lua files to generate template.txt files? Or at least able to convert a template.pot into a template.txt?

Thanks.
 

User avatar
Linuxdirk
Member
 
Posts: 1588
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk

Re: [i18n] how to convert .po files to "=" separated .txt fi

by Linuxdirk » Thu Feb 01, 2018 14:00

I wrote a conversion program in Lua for that.

https://github.com/4w/xtend/tree/5486f9 ... to_tr_file

I tested with my own PO files and it did what I expected it to do. I did not test if the resulting translations work. See source code of the script for more information. Longer strings and newlines resulting in the same layout are not supported.
 

User avatar
Hamlet
Member
 
Posts: 452
Joined: Sat Jul 29, 2017 21:09
Location: Italia
IRC: H4mlet

Re: [i18n] how to convert .po files to "=" separated .txt fi

by Hamlet » Thu Feb 01, 2018 19:15

Linuxdirk wrote:[snip] Longer strings and newlines resulting in the same layout are not supported.


Thanks for your reply, hopefully these unsupported features can be added to your script.
Until now I had to manually copy & paste lines from .po to .txt: a nightmare for files having more than a dozen of entries.
 

User avatar
Linuxdirk
Member
 
Posts: 1588
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk

Re: [i18n] how to convert .po files to "=" separated .txt fi

by Linuxdirk » Fri Feb 02, 2018 05:09

I tried, but I was not able, yet. This would basically result in a complete rewrite, I guess ...

It's a pity that MT does not use an already existing and widely supported translation standard, but cobbles together an own solution incompatible to anything else ...
 

User avatar
TumeniNodes
Member
 
Posts: 2444
Joined: Fri Feb 26, 2016 19:49
Location: in the dark recesses of the mind
GitHub: TumeniNodes
IRC: tumeninodes
In-game: TumeniNodes

Re: [i18n] how to convert .po files to "=" separated .txt fi

by TumeniNodes » Fri Feb 02, 2018 05:59

rather old article but, maybe a little helpful?
http://www.linuxjournal.com/content/int ... sh-scripts

if it has nothing to do with what you are looking for I apologize.

I never got into the depths of i18n, but have always been aware of the process, and sometimes looked over conversations related to it in dev groups I was part of.

This is an extremely important part of any development group but I think sometimes it is approached backwards. Not intentionally though.

But, I do not wish to spark any debate as you have stated.
Good luck with your work, I do know this can be very tedius

also a quick search page on the topic (Im sure you did this already but, just in case)
https://www.google.com/search?source=hp ... AUQqEsOhKg
My brain, is AES256 encrypted, even I don't know wth I'm thinking...
 

User avatar
Hamlet
Member
 
Posts: 452
Joined: Sat Jul 29, 2017 21:09
Location: Italia
IRC: H4mlet

Re: [i18n] how to convert .po files to "=" separated .txt fi

by Hamlet » Fri Feb 02, 2018 11:10

Maybe I have found the solution, it is based on "sed", from its "man" page:
"sed - stream editor for filtering and transforming text"

Look at the second example at the following url, I have tried it (save the last two commands) and it did the job in no time:
https://stackoverflow.com/questions/426 ... c-txt-file

What I did:

Code: Select all
#!/usr/bin/env bash

# - 1 - REMOVE COMMENTS
sed -r -e '/^#/ d' < INPUT.po > OUTPUT.txt

# - 2 - MERGE STRINGS
sed -i OUTPUT.txt -r -e ':L;/"$/{N;s/"\n"//;b L}'

# - 3 - DELETE GETTEXT HEADER
sed -i -e '1,2 d' OUTPUT.txt


What I got:

INPUT FILE (Italian Minetest's .po):
Code: Select all
msgid ""
"Map generation attributes specific to Mapgen v6.\n"
"The 'snowbiomes' flag enables the new 5 biome system.\n"
"When the new biome system is enabled jungles are automatically enabled and\n"
"the 'jungles' flag is ignored.\n"
"Flags that are not specified in the flag string are not modified from the "
"default.\n"
"Flags starting with 'no' are used to explicitly disable them."
msgstr ""
"Attributi di generazione della mappa specifici per Generatore mappa v. 6.\n"
"Quando i biomi di neve sono abilitati le giungle sono abilitate "
"automaticamente, l'impostazione 'jungles' è ignorata.\n"
"Le impostazioni predefinite impostate nel motore sono: biomeblend, mudflow\n"
"La stringa delle impostazioni modifica le impostazioni predefinite del "
"motore.\n"
"Le impostazioni che non sono specificate nella stringa mantengono i valori "
"predefiniti.\n"
"Le impostazioni che iniziano con \"no\" sono usate per disabilitarle "
"esplicitamente."


OUTPUT FILE:
Code: Select all
msgid "Map generation attributes specific to Mapgen v6.\nThe 'snowbiomes' flag enables the new 5 biome system.\nWhen the new biome system is enabled jungles are automatically enabled and\nthe 'jungles' flag is ignored.\nFlags that are not specified in the flag string are not modified from the default.\nFlags starting with 'no' are used to explicitly disable them."
msgstr "Attributi di generazione della mappa specifici per Generatore mappa v. 6.\nQuando i biomi di neve sono abilitati le giungle sono abilitate automaticamente, l'impostazione 'jungles' è ignorata.\nLe impostazioni predefinite impostate nel motore sono: biomeblend, mudflow\nLa stringa delle impostazioni modifica le impostazioni predefinite del motore.\nLe impostazioni che non sono specificate nella stringa mantengono i valori predefiniti.\nLe impostazioni che iniziano con \"no\" sono usate per disabilitarle esplicitamente."


Pretty good, isn't it?
I'll read sed's man page to eventually make it generate "original text=translated text" lines.
 

User avatar
Linuxdirk
Member
 
Posts: 1588
Joined: Wed Sep 17, 2014 11:21
Location: Germany
In-game: Linuxdirk

Re: [i18n] how to convert .po files to "=" separated .txt fi

by Linuxdirk » Fri Feb 02, 2018 14:07

Hamlet wrote:Pretty good, isn't it?

Yes, with sed it's pretty easy. Maybe I re-implement it in Python. I just want it "self-contained" (no external tools or modules used) and independent from the operating system (Lua, Python, etc.).
 


Return to Problems



Who is online

Users browsing this forum: No registered users and 1 guest