Skip to content

i18n - extraction tooling - #32912

Closed
petebacondarwin wants to merge 7 commits into
angular:masterfrom
petebacondarwin:i18n-extraction
Closed

petebacondarwin wants to merge 7 commits into
angular:masterfrom
petebacondarwin:i18n-extraction

Conversation

@petebacondarwin

@petebacondarwin petebacondarwin commented Sep 30, 2019 •

Copy link
Copy Markdown
Contributor

This PR implements a new message extraction tool, which will eventually be integrated into the CLI.

Sits on top of #37114, which should land first.

FW-2073

Closes #37202

@petebacondarwin petebacondarwin added area: i18n Issues related to localization and internationalization target: major This PR is targeted for the next major release comp: ivy labels Sep 30, 2019
@ngbot ngbot Bot modified the milestone: needsTriage Sep 30, 2019
@petebacondarwin
petebacondarwin force-pushed the i18n-extraction branch 4 times, most recently from 4020c4a to 9f3d350 Compare October 4, 2019 08:46
@petebacondarwin
petebacondarwin force-pushed the i18n-extraction branch 2 times, most recently from 7a3b68a to 093f8d5 Compare October 6, 2019 14:53
@thetric

thetric commented Oct 14, 2019

Copy link
Copy Markdown

FYI #32881 was merged 5 days ago in 2cdb3a0

@petebacondarwin
petebacondarwin force-pushed the i18n-extraction branch 5 times, most recently from a0d1090 to bdc258e Compare October 15, 2019 07:48
@sistla001

sistla001 commented Sep 3, 2020 •

Copy link
Copy Markdown

I see 10.1.0 is released, so upgraded my project and did ng xi18n project --format xlf --output-path src/locale, this command didn't translate translations from typescript, but including --ivy flag extracted translations from typescript. Is this expected? I have other translations from Kendo libraries (from node_modules), these are getting extracted fine without --ivy flag, but not with --ivy flag. Am I missing anything? I want to keep both the translations. I have "postinstall": "ngcc" and didn't help. Any thoughts?

@rsivan

rsivan commented Sep 3, 2020

Copy link
Copy Markdown

It appears using the --ivy flag is a must... not sure that is documented though.

@jaska45

jaska45 commented Sep 3, 2020 •

Copy link
Copy Markdown

Extraction from .ts seems working when using --ivy. However the extracted XLIFF no longer contain category="location" unless either meaning, comment or both are passed to i18n.

@petebacondarwin

petebacondarwin commented Sep 3, 2020 •

Copy link
Copy Markdown
Contributor Author

At the moment we are hiding the new extractor behind the --ivy flag until we have ironed out all the bugs.

@jaska45 - please can you create a new issue with a reproduction of your problem?

@sistla001 - currently the new message extractor only looks at the application code - not library code. We need to come up with a proper story for this. In the meantime you could extract them "manually" by running the ivy message extractor directly on the code of the library... Something like:

node --inspect-brk node_modules/.bin/localize-extract  -s "node_modules/@progress/**/dist/fesm2015/index.js" -f xlf -o kendo-messages.xlf

(obviously you will have needed to run ngcc on these libraries fiirst)

@sistla001

Copy link
Copy Markdown

Here is what I did to get it to work. I am using xlf-merge to merge kendo-messages.xlf and messages.xlf (project with --ivy flag) into new messages.xlf then using "@ngx-i18nsupport library to generate messages.es.xlf and messages.fr.xlf from messages.xlf. Does @angular/cli already has support to these functionalities? If not, it would be nice if these too were included.

"extract-i18n": "npm run extract-project && npm run extract-kendo && npm run merge-messages && npm run translate-messages && npm run translate-kendo",
"extract-project": "ng xi18n project --ivy --format xlf --output-path src/locale/xlfs/messages.xlf",
"extract-kendo": "node_modules/.bin/localize-extract  -s node_modules/@progress/**/dist/fesm2015/index.js -f xlf -o src/locale/xlfs/kendo-messages.xlf",
"merge-messages": "xlf-merge src/locale/xlfs/*.xlf -o src/locale/messages.xlf",
"translate-messages": "ng run project:xliffmerge",
"translate-kendo": "npx kendo-translate src/locale/messages.es.xlf --locale es-PR --force && npx kendo-translate src/locale/messages.fr.xlf --locale fr-CA --force",

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

That's really cool @sistla001 I had not been aware of the kendo-translate tool.

If Kendo provided these messages in a form that we could use (like XLIFF) then you could short-circuit the extraction process and just merge their translations when building the app: you are able to provide an array of translation files for each locale in the angular.json file. But it seems that the kendo-angular-messages package only contains YAML files that only reference the message's "meaning" rather than its "id".

@bastienmoulia

bastienmoulia commented Sep 4, 2020 •

Copy link
Copy Markdown

I am using xlf-merge to merge kendo-messages.xlf and messages.xlf

Maybe you don't have to merge the files because the i18n/locales attributes in angular.json accept an array of filepath.

"i18n": {
  "locales": {
    "fr": ["src/locale/messages.fr.xlf", "src/locale/kendo-messages.fr.xlf"]
  }
}

@sistla001

Copy link
Copy Markdown

I am using xlf-merge to merge kendo-messages.xlf and messages.xlf

Maybe you don't have to merge the files because the i18n/locales attributes in angular.json accept an array of filepath.

"i18n": {
  "locales": {
    "fr": ["src/locale/messages.fr.xlf", "src/locale/kendo-messages.fr.xlf"]
  }
}

Good to know.. Thanks!!

@manzonif

manzonif commented Sep 10, 2020 •

Copy link
Copy Markdown

Is it possible that the meaning and description in the templates are not extracted? The source file also does not appear.
The $localize instead is extracted as expected.

"extract-i18n2": "ng xi18n project --ivy --format xlf2 --output-path src/i18n",

<button i18n="Follow User button|click to follow user@@FollowUser">Follow User</button>

give me this result:

    <unit id="FollowUser">
      <segment>
        <source>Follow User</source>
      </segment>
    </unit>
 const title = $localize`:welcome message|home page title@@HomeTitle:You are Welcome`;

this:

    <unit id="HomeTitle">
      <notes>
        <note category="location">src/app/main/pages/home/home.component.js:65</note>
        <note category="description">home page title</note>
        <note category="meaning">welcome message</note>
      </notes>
      <segment>
        <source>You are Welcome</source>
      </segment>
    </unit>

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

I just tried that template HTML with the latest Angular and CLI releases (i.e. 10.1.1) and I got:

    <unit id="FollowUser">
      <notes>
        <note category="location">src/app/app.component.html:3</note>
        <note category="description">click to follow user</note>
        <note category="meaning">Follow User button</note>
      </notes>
      <segment>
        <source>Follow User</source>
      </segment>
    </unit>

Can you check your versions @manzonif

@manzonif

Copy link
Copy Markdown

@petebacondarwin thanks, now it work correctly.

@sistla001

Copy link
Copy Markdown

Is there a way to translate text from [title] from HTML without moving the condition to typescript? There is no reason why I shouldn't move the condition to typescript but just curious to know if there is a way.

<custom-component [title]="condition?'Home':'Dashboard'"></custom-component>

I tried - i18n-title but didn't extract any messages.

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

@sistla001 - We do not support i18n messages on dynamic input bindings.

@jaska45

jaska45 commented Sep 10, 2020

Copy link
Copy Markdown

@sistla001You have to handle the string in .ts code. For example in your component code

title = condition ? $localize`Home` : $localize`Dashboard`;

Then you use the title variable in your template

<custom-component [title]="{{title}}"></custom-component>

@JanneHarju

Copy link
Copy Markdown

How about normal plural syntax? Is It still working? So in html title="here some plural syntax" i18n.title="@@someTitle"

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

@JanneHarju - what do you mean by "plural syntax"? Do you mean a plural ICU expression? e.g.

<div title="{value, plural, =1 {one} other {many}}" i18n.title="@@someTitle">...</div>

This still works.

@JanneHarju

Copy link
Copy Markdown

@petebacondarwin that is exactly what I ment. And its was answer for @sistla001 when he ask is conditional translation possible. With plural syntax it is possible no need to move code to ts side.

@sistla001

sistla001 commented Sep 11, 2020 •

Copy link
Copy Markdown

@petebacondarwin that is exactly what I ment. And its was answer for @sistla001 when he ask is conditional translation possible. With plural syntax it is possible no need to move code to ts side.

Thanks @JanneHarju, @petebacondarwin - I always overlooked this plurals. Thanks for pointing out. It works!!
Just want to point out that i18n-title worked not i18n.title

@JanneHarju

Copy link
Copy Markdown

How about documentation? When we get it to angular.io? Thanks.

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

Good question! I need to work with @aikidave and @kapunahelewong on this. We will also need to nail down the public API as well.

@PowerKiKi

Copy link
Copy Markdown
Contributor

In my migration from Angular 9 to Angular 10, I am trying leverage $localize to translate string from components.

I noticed that running ng xi18n will extract those string with the new ID format (19 digits), instead of the legacy format (40 hexa). Even though templates string keep using the legacy IDs. This is problematic because code and templates may share the same string, so I would like them to share the same ID, to avoid to have to translate them twice.

In order to avoid this situation, I'd like to permanently migrate everything to the new ID format. But I failed to find a way to specify this option to cli tool ng xi18n, or in angular.json.

I did find the --useLegacyIds flag on the lower level tool. So I can run something like:

./node_modules/.bin/localize-extract --locale fr --useLegacyIds false --source dist/my-lib/fesm2015/my-lib.js --format xlf --outputPath dist/messages.xlf

Is there a more proper way to opt-in the new ID format, without tinkering with low-level things ?

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

Actually there is! Currently the CLI xi18n job does not support the --useLegacyIds option.

But If you disable legacy message ids in the Angular compiler it will never generate them in the first place, and so the message extractor will have to use the non-legacy ids.

tsconfig.json

{
  ...
  "angularCompilerOptions": {
    "enableI18nLegacyMessageIdFormat": false
  }
}

Check out this repo: https://github.com/petebacondarwin/issue32912-i18n-no-legacy-test

Note that ngcc always enables legacy message ids at the moment, so any localized messages in templates of library components will still have the old message ids.

@PowerKiKi

Copy link
Copy Markdown
Contributor

Thank you for your quick reply. That allowed me to keep using the standard ng xi18n, while already moving to latest ID format.

Our library will now ship their own translations as part of their package, and the consuming application will includes the application translation and the library translation, via the array configuration you mentioned in #32912 (comment). So, for us, the ngcc part is irrelevant, but it's always good to know.

@sistla001

sistla001 commented Sep 28, 2020 •

Copy link
Copy Markdown

@JanneHarju - what do you mean by "plural syntax"? Do you mean a plural ICU expression? e.g.

<div title="{value, plural, =1 {one} other {many}}" i18n.title="@@someTitle">...</div>

This still works.

@petebacondarwin, @JanneHarju - So I tried the ICU or the plural syntax.
this works:

<div i18n>{value, plural, =1 {one} other {many}}</div>

but this doesn't work: It displays the full plural syntax

<input value="{value, plural, =1 {one} other {many}}" i18n-value="@@someTitle"/>

example:
https://stackblitz.com/edit/angular-plurals-angular9-5xkaza?file=src%2Fapp%2Fapp.component.html

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

@sistla001 - this is a known issue: #21615

@sistla001

sistla001 commented Oct 2, 2020 •

Copy link
Copy Markdown

Is there any angular recommended approach for multi-language support in Angular Libraries? I am thinking of creating messages files for each library and them merge these message files in the SPA.

@PowerKiKi

Copy link
Copy Markdown
Contributor

This is not an official recommendation, but it seems to be working for us. Our lib maintain localized messages (translated via Weblate) and ship them in the package. Then our app consume them with the implicit merging mechanism. Here is an extract of angular.json that allow that:

{
    "projects": {
        "my-app": {
            "i18n": {
                "sourceLocale": "fr",
                "locales": {
                    "de": [
                        "node_modules/@ecodev/natural/i18n/messages.de.xlf", 
                        "client/i18n/messages.de.xlf"
                    ],
                    "it": [
                        "node_modules/@ecodev/natural/i18n/messages.it.xlf", 
                        "client/i18n/messages.it.xlf"
                    ],
                    "en": [
                        "node_modules/@ecodev/natural/i18n/messages.en.xlf", 
                        "client/i18n/messages.en.xlf"
                    ]
                }
            }
        }
    }
}

@petebacondarwin

Copy link
Copy Markdown
Contributor Author

We are looking into ways that these pre-packaged translations in a library can be automatically discovered by the CLI when translating. It would require some specification in the Angular Package Format, I believe.

@angular-automatic-lock-bot

Copy link
Copy Markdown

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: i18n Issues related to localization and internationalization cla: yes target: major This PR is targeted for the next major release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include context/context-group tags in i18n extraction tooling