[question] Why bsl language is not supported?

[nixel2007]

Hello! I've noticed that BSL language is excluded. Could you explain why? Does it use some feature that is not supported in libprisma? We could try to adjust the grammar to make it work with your lib.

Thanks in advance!

[exclued]

Seems it have something to do with unsupported UTF-16 characters in the language grammar.

function sanitize(pattern) { // Unsupported: // UTF-16 ranges
from generate.js

Compare this with BSL definition in https://github.com/PrismJS/prism/blob/master/components/prism-bsl.js

[nixel2007]

Thanks, I'll take a look. For most cases these utf-16 sequences can be simplified to [а-яё], which are in utf-8 range

[nixel2007]

prism.js v1 does not merge any PRs these days. Will you approve a .patch file with patch to bsl grammar and additions to generate.js/github workflow to apply the patch it at place?

[nixel2007]

Hello, team! I'm looking into the new way to include bsl language into libprisma introduced recently. I want to clarify what does UTF-16 ranges in sanitize mean? Do you not support /uXXXX sequences at all or some concrete range (\uD800+ for example)?
If /uXXXX sequences are not supported at all, is there any way to add cyrillic letters into regex? will regular [а-яё] work?

/cc @FrayxRulez

[nixel2007]

Anyone? :)

[FrayxRulez]

Hi @nixel2007 sorry for the delay.
Please, check Boost.Regex specifications in regards of UTF-16 support, as there are limitations in different areas (we use string and not wstring, and Boost regex syntax is slightly different and more limited than JS one).
It's important that this works on all platforms, UNIX and Windows have different behaviors when dealing with strings.
You can probably test this by generating a new grammars.dat file with only bsl enabled (you'll have to edit the generation script) and try to load it using the library. You'll see that Boost will crash right away while trying to interpret the patterns.

[nixel2007]

Hi, @FrayxRulez !

Thanks for your input, it's clear. I'll take a look.

[bapho-bush]

we use string and not wstring, and Boost regex syntax is slightly different and more limited than JS one

Could you please describe reasons why don't you use std::wstring and boost::wregex?

[mm8191]

دقیقا یادم نیس پیام مال چی بوده ولی توکن و دارایی من گم شده باید چکارکنم کی مسئول است عایا باید ب پاول دولف پیام بدم

[mm8191]

سلام من دارایی ام گم شده چکارکنم و یک خساب تلگرامم سیمکارتم گم شده وصل نمیتونم چکارکنم در تاریخ جمعه ۲۰ سپتامبر ۲۰۲۴، ۸:۴۴ بعدازظهر Nikita Fedkin < ***@***.***> نوشت:

…

Hi, @FrayxRulez <https://github.com/FrayxRulez> ! Thanks for your input, it's clear. I'll take a look. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BJNVG4E2ZVUDDYZQDZS4S33ZXRJXFAVCNFSM6AAAAABNJEW3QGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRUGE3DQNBXGY> . You are receiving this because you commented.Message ID: ***@***.***>