Crimean Tatar (crh)

[arysin]

Add Crimean Tatar Language (crh)

[danielnaber]

common_words.txt should probably be limited to ~10,000 lines. The lang detection algorithm doesn't normalize for the length of the common_words file, I think, so all of them should roughly have the same number of words.

[arysin]

So (at least for now) crh dictionary contains words duplicated in Latin and Cyrillic, so we have 10k + 10k words. If we should trim it to 10k total, I can cut it to 5k + 5k

[arysin]

@danielnaber based on what you said about common_words, it could be that in this (unique) case with two alphabets it makes sense for 10k+10k, as the text would be either in Latin or Cyrillic and in each case only 10k of either set of words would be in play. So this would work in the algorithm is based on absolute values. Unless I am missing something.

[danielnaber]

I guess 10k+10k would be okay

[danielnaber]

I guess this is ready to be merged now? Just as a heads-up: while I can merge this, please don't expect the language to show up in the UI. Adapting the UIs is a lot of manual work, and adding one more language makes the usability of the long drop-down even worse.

[arysin]

Yes, thank you. The team understands it'll take some time to reach the UI.