Suggested support tool to minify XLIFFs

[johannks]

Hello Martin

Thanks again for an excellent tool. There is an option/functionality that we use which could be of interest to others as well. When we build a project, we have added a step to minify the XLIFF files to remove (what we think are) unnecessary tags.

We've added the following to a pre-build.js:

var fs = require('fs');

const files = fs.readdirSync('src/locale', {encoding: 'utf8'});
files.forEach(file => {
  if (file.match(/^(messages\.[a-z]{2,2}(-[A-Z]*)*)(\.xlf)$/)) {
    const minified = file.replace(/^(messages\.[a-z]{2,2}(-[A-Z]*)*)(\.xlf)$/g, '$1.min$3');
    console.log(`Minifying src/locale/${file} > src/locale/${minified}...`);
    if (fs.existsSync(`src/locale/${minified}`)) {
      fs.unlinkSync(`src/locale/${minified}`);
    }
    fs.writeFileSync(`src/locale/${minified}`,
      fs.readFileSync(`src/locale/${file}`, {encoding: "utf8"})
        .replace(/<!--[\s\S]*?-->/g, '') // remove html comments
        .replace(/<context-group[\S\s]*?context-group>/g, '') // remove context-group tags with content
        .replace(/<source[\S\s]*?source>/g, '') // remove source tags with content
        .replace(/\sstate=\".*?\"/g, '') // remove state attributes
        .replace(/\sdatatype=\".*?\"/g, '') // remove datatype attributes
        .replace(/(>)\s*(<)/g, '$1$2'), // lastly, linearize
      {encoding: 'utf8'}
    );
  }
});

and are including the resulting *.min.xlf files (<50% size) instead of the original translated files in the build.

We have not run into any problems with this approach, and perhaps it could be a welcome addition to your package?

/Johan

[martinroob]

Hello Johan,
thanks for your suggestion, but to be honest I am not really convinced.
First of all I do not expect to have a real win by minimizing the XLIFFs. The build process will not be measurably faster, I guess.
Second, the informations you are removing are not used by the angular build process, but they can be used by translation tools and you will lose this by removing them.
For example the source tags are a hint, where the translation is used and this can be a valuable information for translators.

Regards
Martin

[johannks]

Hello Martin

It is not the build process itself that we aim to optimise. It will be slower, but the minification of seven language files is done in a fraction of a second. However, even if it took significantly longer, the duration of the production build process is insignificant to us. We're currently minifying the XLIFFs on every build, but would really only need to for distribution/production builds if it was necessary to optimise development builds.

Regarding removal of information, the minification does not affect the "normal" files used for "offline" translation, and we have not seen any tools which would allow a translator to translate the app at runtime.

Our aim with the minification (as with the minification of the JavaScript) is to cut out everything not needed at runtime for the optimisation of the app's loading time, for example:

      <trans-unit id="actionbar.command-save.tooltip" datatype="html">
        <source>Save (Ctrl+S)</source>
        <target state="final">Speichern (Strg+S)</target>
      </trans-unit>

where the only thing really needed at runtime seems to be the id and the target

<trans-unit id="actionbar.command-save.tooltip"><target>Speichern (Strg+S)</target>

By minification of ~850 strings (expected to grow even further), we save >100 kB per language bundle, an opportunity the Angular team would probably jump at.

The generated minified files are excluded from GIT using .gitignore with /src/locale/*.min.xlf as they are generated on demand, and are included in the architect/build/configurations of angular.json as for example:

            "de-prod": {
              "i18nFile": "src/locale/messages.de.min.xlf",
              "i18nFormat": "xlf",
              "i18nLocale": "de"
            },

To us this makes completely sense and we have only seen an upside with the approach so far.

/Johan

[johannks]

Forgot to mention that this would be another support tool / command, and not an option to xliffmerge, since the normal offline XLIFFs are needed for translation and the minification would be performed after the translation has been completed.

[GeorgDangl]

@johannks, the XLIFF files are only used by the build process, they are not part of the output. When you build for a locale and supply an XLIFF file, the translations are applied during the build process.
Minimizing the XLIFF file itself should have no impact on the generated bundles.

[johannks]

@GeorgDangl, in our setup (following the instructions in @ngx-translate/i18n-polyfill) it does get included into the main.js.

As a matter of fact, I only just now noticed that ALL our 7 XLIFF files get included in the main.js of every language prod-bundle, which actually compounds our problem.

Are we doing something wrong? Is anyone else having the same issue?

[GeorgDangl]

Oh, I see, the XLIFF is directly loaded for dynamic translations. I thought you'd create a separate bundle for
every target language. Maybe that's an option you can evaluate if you really want to trim down on the bundle size:)

Personally, I build a .json file for every language target with the strings I need and statically reference that during the build, so only the actual values (and their keys) are present, not the whole XLIFF. I can later take a look at how exactly I've implemented it.