AraBERT and Cross-domain tasks

[YomnaEid]

Is AraBERT, like BEERT a task-agnostic, which lacks domain awareness and can not distinguish the characteristic of source
and target domain when transferring knowledge?

[WissamAntoun]

It's not that it lacks domain awareness. The model was trained (v2 and v02) on news articles, Wikipedia, and whatever was in OSCAR (basically blogs and random Arabic web content). The twitter models have some tweets added with emojis to better suit dialects.

So if you want to use the model on medical arabic text the performance wouldn't be optimal compared to a model that already has medical text in the pertaining data