Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Initial version #1

Merged
merged 35 commits into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2aa1fc9
refactor: wip boilerplate
radiovisual May 30, 2024
80e226d
chore: update dependencies
radiovisual May 30, 2024
ba78dc7
build: init build system
radiovisual May 30, 2024
412c51b
feat: add no-unstranslated-messages rule
radiovisual May 31, 2024
b9d7f8e
test: add tests for no-untranslated-messages
radiovisual May 31, 2024
add5e97
test: add tests for no-untranslated-messages
radiovisual May 31, 2024
f1b2188
refactor: tests for no-untranslated-messages
radiovisual May 31, 2024
a3c8da1
ci: run unit tests via github actions
radiovisual May 31, 2024
99de097
docs: add ci badge
radiovisual May 31, 2024
c4a79db
docs: minor readme tweaks
radiovisual May 31, 2024
4607e8c
build: add editor configs
radiovisual May 31, 2024
72e2dca
feat: force config to match the locale with filename
radiovisual May 31, 2024
596eb6c
build: init an esbuild script
radiovisual May 31, 2024
c5949b8
build: remove module type
radiovisual Jun 1, 2024
7b73d5c
docs: fix readme typos
radiovisual Jun 1, 2024
a46e230
feat: add no-empty-messages rule
radiovisual Jun 1, 2024
1c665cf
feat: add ignoreKeys configuration and documentation
radiovisual Jun 1, 2024
0ff0474
docs: update advanced config section
radiovisual Jun 1, 2024
1a7824a
feat: add no-invalid-variables rule
radiovisual Jun 1, 2024
03e6bde
docs: update rule documentation
radiovisual Jun 2, 2024
e710d59
docs: minor readme tweaks
radiovisual Jun 2, 2024
b157ff6
build: populate missing package.json fields
radiovisual Jun 2, 2024
6496b64
feat: no-html-messages
radiovisual Jun 2, 2024
7b13567
docs: update readme
radiovisual Jun 2, 2024
56e8305
docs: update readme
radiovisual Jun 2, 2024
073f5a5
docs: more readme tweaks
radiovisual Jun 2, 2024
a5debde
feat: add rule no-invalid-configuration
radiovisual Jun 3, 2024
450fbe6
feat: add no-missing-keys rule
radiovisual Jun 3, 2024
f0deeab
feat: exit when enabled = false
radiovisual Jun 3, 2024
762ebb8
feat: track ignored problems
radiovisual Jun 3, 2024
1d2fbe6
test: fix tests and improve coverage
radiovisual Jun 3, 2024
0c0be97
ci: add code coverage
radiovisual Jun 3, 2024
79ff326
refactor: move fixture files outside of src
radiovisual Jun 3, 2024
79feca1
docs: update readme with milestone notes
radiovisual Jun 3, 2024
40cc03f
feat: clean up problem logger
radiovisual Jun 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
root = true

[*]
indent_style = tab
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.yml]
indent_style = space
indent_size = 2
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* text=auto eol=lf
25 changes: 25 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: CI
on:
- push
- pull_request
jobs:
test:
name: Node.js ${{ matrix.node-version }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
node-version:
- 16
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm test

- name: Upload coverage reports to Codecov
uses: codecov/[email protected]
with:
token: ${{ secrets.CODECOV_TOKEN }}
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=false
151 changes: 145 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,154 @@
# i18n-validator
# i18n-validator [![CI](https://github.com/radiovisual/i18n-validator/actions/workflows/ci.yml/badge.svg)](https://github.com/radiovisual/i18n-validator/actions/workflows/ci.yml) [![codecov](https://codecov.io/github/radiovisual/i18n-validator/graph/badge.svg?token=G4X3X08FB6)](https://codecov.io/github/radiovisual/i18n-validator)

> Fast, configurable and extendable CLI to look for common problems in your translated source files.
> Configurable CLI validation tool to look for common problems in your translated source files.

## Why?

> Have you ever shipped "buggy" translation files to your users in proudction? Probably. I know I have!
> Have you ever shipped "buggy" translation files to your users in production? **Probably!**. I know I have!

Translated files in your software project are an often-overlooked source of problems that can affect the usability and reliability of your applications. These translated files are often edited manually, built automatically (without any integrity checks) or outsourced to a third party to provide translations, and then these files typically do not pass through any of your automated tests, or get skipped in your manual tests...which means the hidden problems get shipped to your users in production. If you are reading this right now, you have probably shipped "buggy" translation files to real users in production, am I right?
Translated files in your software project are an often-overlooked source of problems that can affect the usability, reliability and reputation of your applications. These translated files are often edited manually, built automatically (without any integrity checks) or outsourced to third parties to provide translations. These files typically do not pass through any (or most) of your automated tests, or they get skipped in your manual tests...which means the hidden problems get shipped to your real users in production.

Furthermore, there are best practices we want to adhere to with our translated files, and these best practices should be enforceable with an integrity check.

## WIP
## Roadmap to Version `1.0.0`

Check the branch called `initial-version` for latest progress.
We are getting close to the initial `1.0.0` release. :tada: Check out [this milestone](https://github.com/radiovisual/i18n-validator/milestone/1) for details.

## Configuration

For each project where you want to run the i18n-validator, you will need to have a file named `i18n-validator.config.json` with the following format:

```json5
{
/**
* The locale your translations uses as the default or source language.
* Example: 'en'
*
**/
defaultLocale: "en",
/**
* The filename of the file where your app's default/source language is defined.
* Example: "default.json" | "source.json" | "en.json"
*
**/
sourceFile: "en.json",
/**
* An object describing the locale and its assosiated translation file name.
* Example: {"de": "de.json", "fr": "fr.json" }
*
**/
translationFiles: {
de: "de.json",
fr: "fr.json",
},
/**
* The path, relative to the root directory where your i18n files can be located.
* Example: 'relative/path/to/i18n/files'.
*
* Note: This relative path will be combined with the filenames you supplied in
* the translationFiles object to create the path to each translation file.
*
**/
pathToTranslatedFiles: "i18n",
/**
* The rules configuration.
*
* You can set each rule to a severity level of: 'error' | 'warn' | 'off'
* Note: If you do not provide a severity setting for a rule, then the
* rule's default setting will apply.
*
* You can also pass in advanced configuration for some rules if you pass in
* an object like:
*
* "rule-name": {
* "severity": "error",
* "ignoreKeys": ["foo", "bar"]
* }
*
* Check the rule's documentation for advanced configuration options.
*
**/
rules: {
"no-untranslated-messages": "error",
"no-empty-messages": "error",
"no-html-messages": "error",
"no-invalid-variables": "error",
"no-missing-keys": "error",
},
/**
* Set this dryRun setting to true to get all the same logging and reporting
* you would get in a real validation check, but no errors will be thrown if
* errors are found. Great for getting your i18n-validation setup in CI/CD
* pipelines without breaking your builds.
*
**/
dryRun: false,
/**
* Enable or disable this entire i18n-validator.
*
**/
enabled: true,
}
```

# Rule Defaults

Each rule can have a default setting of `error`, `warn` or `off`, these defaults will apply if you do not provide a configuration for the rule in the configuration file.

| Rule name | Default |
| ------------------------ | ------- |
| no-untranslated-messages | `error` |
| no-empty-messages | `error` |
| no-invalid-variables | `error` |
| no-html-messages | `error` |
| no-missing-keys | `error` |

# Overriding Rules

Some rules can have advanced configuration passed in. Each rule might be different, so you want to read the documentation for each rule, but here are some of the more general overrides available:

## ignoreKeys

For validation rules that read through each key of a translated file, you can pass in an `ignoreKeys` array of strings that represent keys that should be ignored for the specific rule. Meaning, the rule will still run as expected on all keys, except for the keys specified. For example, if you have translation files that looks like this:

```json
{
"en": { "ok": "OK" },
"fr": { "ok": "OK" }
}
```

Then you might want the rule `no-untranslated-messages` to ignore this key `'ok'` because the translation of "ok" might also be "OK" and trigger a false positive in the rule.

To configure the `ignoreKeys` array you can assign an object to the rule name in the configuration file.

```json
{
"rules": {
"no-untranslated-messages": {
"severity": "error",
"ignoreKeys": ["ok"]
}
}
}
```

> [!IMPORTANT]
> Be careful when ignoring keys for specific rules, ignored keys can lead to ignored problems!

> [!TIP]
> If you find that you are ignoring a lot of keys, consider moving some strings outside of your translated files (for example in a common messages library, etc)...

# Tracking Ignored Problems

A feature of this CLI tool is that whenever you intentionally skip/ignore a known problem, these problems are still tracked and reported by default (as of now, there is also no way for you to opt-out of seeing ignored problems in the log output, pull requests welcome!), **but any ignored errors won't fail your build, they are merely reported in the logs**.

One of the main motivations with this CLI is to _expose_ (not hide) the problems in your translation files. I believe that ignoring problems and not showing these ignored problems in the logs prevents new problems from creeping into the files and also helps encourage best practices, additional benefits include: tracking technical debt and showing the consequences/risks you might be taking by ignoring actual problems.

# Contributing

Contributions to this project are always welcome! :heart: Feel free to open issues, pull requests, etc and let's make this project better together.

---

:rainbow: :heart: :hamburger:
11 changes: 11 additions & 0 deletions build/cli.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import * as esbuild from 'esbuild';

await esbuild.build({
entryPoints: ['./src/**/*.ts'],
bundle: true,
outdir: 'dist',
platform: 'node',
legalComments: 'inline',
packages: 'external',
target: 'node16'
});
14 changes: 14 additions & 0 deletions fixtures/i18n/de.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"untranslated-message": "Hi!",
"empty-string": "",
"only-empty-in-en": "[DE] not empty",
"declares-a-valid-variable": "Hallo, {firstName}",
"declares-an-unbalanced-variable-in-en": "[DE] unbalanced {variable}",
"declares-another-unbalanced-variable-in-en": "[DE] another unbalanced {variable}",
"declares-a-missing-variable-in-fr": "[DE] missing {variable}",
"declares-a-single-missing-variable-in-fr": "[DE] should be two {one} {two}",
"declares-a-malfomed-variable-in-de": "[DE] malformed {variable",
"html-message": "[DE] Hi, <b>{firstName}</b>",
"missing-key-in-fr": "[DE] this key should not be found in fr.json",
"unexpected-key": "[DE] this key is not defined in the source file en.json"
}
13 changes: 13 additions & 0 deletions fixtures/i18n/en.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"untranslated-message": "Hi!",
"empty-string": "",
"only-empty-in-en": "",
"declares-a-valid-variable": "Hi, {firstName}",
"declares-an-unbalanced-variable-in-en": "[EN] unbalanced variable}",
"declares-another-unbalanced-variable-in-en": "[EN] another unbalanced {variable",
"declares-a-missing-variable-in-fr": "[EN] missing {variable}",
"declares-a-single-missing-variable-in-fr": "[EN] should be two {one} {two}",
"declares-a-malfomed-variable-in-de": "[EN] malformed {variable}",
"html-message": "Hi, <b>{firstName}</b>",
"missing-key-in-fr": "[EN] this key should not be found in fr.json"
}
13 changes: 13 additions & 0 deletions fixtures/i18n/fr.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"untranslated-message": "Hi!",
"empty-string": "",
"only-empty-in-en": "[FR] not empty",
"declares-a-valid-variable": "Salut, {firstName}",
"declares-an-unbalanced-variable-in-en": "[FR] unbalanced {variable}",
"declares-another-unbalanced-variable-in-en": "[FR] another unbalanced {variable}",
"declares-a-missing-variable-in-fr": "[FR] missing",
"declares-a-single-missing-variable-in-fr": "[FR] should be two {one}",
"declares-a-malfomed-variable-in-de": "[FR] malformed {variable}",
"html-message": "[FR] Hi, <b>{firstName}</b>",
"unexpected-key": "[FR] this key is not defined in the source file en.json"
}
18 changes: 18 additions & 0 deletions i18n-validator.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"defaultLocale": "en",
"sourceFile": "en.json",
"translationFiles": {
"de": "de.json",
"fr": "fr.json"
},
"pathToTranslatedFiles": "fixtures/i18n",
"rules": {
"no-untranslated-messages": "error",
"no-empty-messages": "error",
"no-html-messages": "error",
"no-missing-keys": "error",
"no-invalid-variables": "error"
},
"dryRun": false,
"enabled": true
}
9 changes: 9 additions & 0 deletions jest.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"extensionsToTreatAsEsm": [".ts"],
"preset": "ts-jest",
"testEnvironment": "node",
"testPathIgnorePatterns": ["dist"],
"collectCoverage": true,
"coverageDirectory": "coverage",
"coverageReporters": ["text", "lcov"]
}
56 changes: 56 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"name": "i18n-validator",
"version": "0.0.1",
"main": "dist/index.js",
"description": "Configurable CLI validation tool to check for common problems in your translated source files.",
"author": {
"name": "Michael Wuergler",
"email": "[email protected]",
"url": "https://github.com/radiovisual"
},
"bugs": {
"url": "https://github.com/radiovisual/i18n-validator/issues"
},
"scripts": {
"build:prod": "rm -rf dist/* && node build/cli.mjs",
"build:dev": "rm -rf dist/* && npm run typecheck && tsc && node build/cli.mjs",
"start": "npm run build:dev -- --watch",
"test": "jest",
"typecheck": "tsc --noEmit"
},
"bin": {
"i18n-validate": "dist/index.js"
},
"files": [
"dist/**/*"
],
"devDependencies": {
"@types/jest": "^29.5.12",
"@types/node": "^20.12.13",
"esbuild": "0.21.4",
"jest": "^29.7.0",
"ts-jest": "^29.1.4",
"ts-node": "^10.9.2",
"typescript": "^5.4.5"
},
"dependencies": {
"@formatjs/icu-messageformat-parser": "^2.7.8",
"chalk": "^4.0.0"
},
"keywords": [
"cli",
"validation",
"check",
"i18n",
"translation",
"integrity",
"internationalization",
"locale",
"locales",
"json",
"automation",
"ci",
"cd"
],
"license": "MIT"
}
Loading