Fall back to source language instead of showing undefined

[KraXen72]

Problem

when a key is not defined, there should be an option in project.inlang.json that makes it so that instead of showing undefined, all keys would fallback to source language. if a source language doesen't have a key, but other languages do, then it's okay to show undefined, since that's a big mistake.

Expected behavior

optionally, translations should fallback back to source language

Reproduction

make a new key
only include source language
switch to other language
see that it renders 'undefined'

Other information

Vivaldi latest stable
Windows 10
inlang ext 1.20.0

[samuelstroschein]

We can't fallback to the source language.

Once per language splitting is implemented, falling back to the source language is not possible (client-side). We strongly recommend setting up CI/CD that lints missing translations via the inlang CLI and use the machine translate command of the CLI to avoid missing translations altogether.

Returning undefined is a bit annoying though. We should (and had at one point) return the message id instead to enable CMD + F searches for the message id

[KraXen72]

message id's sound good, but i think there should be atleast file/route level fallbacks - if a page with 10 routes only has 9 of them translated, is it still worth not allowing that translation to be used? this can especially happen with community translations, when people translate only as much as they currently have time for

[samuelstroschein]

@KraXen72 if a page with 10 routes only has 9 of them translated, is it still worth not allowing that translation to be used?

Maybe in the future if more users ask for it.

In the meantime, integrate inlang machine translate into your build pipeline, and you won't have missing translations. We suspect that the tooling we provide like machine translate will make the need for fallbacks irrelevant in the long run.

[Theta-Dev]

Machine translation is unsuitable for translating software because of the missing context. And for a lot of applications, most users prefer English instead of bad machine translations.

Abozt the code splitting issue: loading 2 translations instead of 1 should not add a significant amount of code to the bundle. I agree that this should not be the default option but you should be able to configure a fallback language if necessary.

[LorisSigrist]

Reopening in response to https://discord.com/channels/897438559458430986/1083724234142011392/1186525642540453928

@samuelstroschein
We could avoid the problems with the language-splitting by putting the fallback into the language-specific file, instead of putting the fallback logic in the shared file.

For Example if de is missing message_a but the source language has it, then we could put a message_a function into paraglide/messages/de.js that returns the source language. That way all message-files would have identical interfaces and act as drop-in replacements for one-another.

This may cause some confusion as to if messages are present, so I would also suggest somehow warning when this happens.

[samuelstroschein]

@LorisSigrist let's move this to a feature request in the discussions

[ccdexcom]

It's useful to know when a translation is undefined.

One situation could be:

• "greeting": "Welcome to our app" is the current text to be translated, and translations exist across all locales
• The template/jsx/etc/ is updated to use "greetingNew": "Welcome to our app, it's awesome!"
• We might not have the "greetingNew" translated across all locales, and in that case the existing is okay in the meantime:

// fallback if necessary
greetingNew() ?? greeting()

What I'm seeing right now is something like

export { greetingNew } from "./en-US.js"

Which makes it impossible to know if it's translated or not. Comparing to an the english string explicitly isn't suitable either because it could be that the translation overlaps between english and another language. There's probably other situations where it's valuable to know if the translation at source exists or not.

[LorisSigrist]

There is a lint rule for checking that translations are defined in all languages.
https://inlang.com/m/4cxm3eqi/messageLintRule-inlang-missingTranslation

Click "Install", Follow the Manual Installation instructions and run npx @inlang/cli lint. You can also add this to your CI pipeline.

We could consider logging a warning to the console when the compiler uses a fallback language, but it's impractical to log a warning at runtime.

[ccdexcom]

It's not so much an issue of knowing that we don't have the translation, in this case we know which translations are missing because they're in flight (and possibly coming back in multiple batches). At runtime we want to fallback to a completely different message id because we know showing that is better than showing english. This gives the authors of the content control of what to do in this case.

[samuelstroschein]

@ccdexcom this is a complex use case. to me, such fallbacks signal that the localization pipeline should be improved rather than fixing something in code by falling back to other message ids. falling back to other messages does not seem to make sense. how would you know if some other message id is suited to displayed as fallback without manual work?

[ccdexcom]

In the case we've been working with it's like the simplistic example above greetingNew() ?? greeting(). It's a situation we've been faced with for several years. We know that greetingNew is preferable but greeting is acceptable. A better pipeline would definitely help but due to the strictness of our translations we're unable to even rely on machine/ai translations as an interim solution. We don't want to slow down our development so we have translation fallbacks.

Outside of this use case it's handy from a data perspective to be able to do our own runtime checks, like you you mentioned adding logging, but if this was exposed optionally it's something authors could log themselves in user land.

Something roughly like this could work with the assumption that the fallback function is re-exported from the source language:

import { sourceLanguageTag } from '$paraglide/runtime';

export const hello = (params = {}, options = { allowFallback: true }) => {
	const messages = {
		"de-DE": de_DE.hello,
		"fr-CA": fr_CA.hello,
		"fr-FR": fr_FR.hello,
	};
	
	const outputLanguageTag = [options?.languageTag ?? languageTag()];

      	if (!options.allowFallback && messages[sourceLanguageTag] === messages[outputLanguageTag]) {
		return undefined;
	}
	
	return messages[outputLanguageTag]()
}

There's likely other options that could be done on the code generation side that could be non-breaking, too.

I appreciate the quick responses and bouncing back and forth. I wanted to throw our use case out there even if it's an edge case. I'll post a bit of code that will be our workaround in case others stumble across this edge case as well.