Researchers Expose Vulnerabilities in Bengali Language AI Models and Propose Defense Strategy

Transformer-based language models have become the standard for Bengali sentiment analysis, but their ability to withstand adversarial attacks has never been systematically tested. Researchers have now filled this gap by developing destroR, a comprehensive framework that both tests and reinforces Bengali text classifiers against carefully crafted perturbations designed to fool the models while preserving meaning.

The Attack Methods#

The team designed three distinct adversarial techniques that attack models by subtly altering input text while maintaining semantic integrity. These include paraphrasing passages into alternative phrasings, back-translating through intermediate languages, and swapping individual words while preserving grammatical structure. Unlike simple noise-based attacks, these approaches generate fluent, natural-sounding Bengali sentences that confuse the models without obviously corrupting the underlying meaning.

Testing the Models#

The research benchmarked five major transfer learning models—BanglaBERT, BanglishBERT, XLM-RoBERTa, MuRIL, and IndicBERTv2—evaluating their robustness across four datasets. The newly developed attacks were pit against two established word-substitution baselines, TextFooler and BAE, under consistent testing conditions. The results revealed that word-swapping techniques proved surprisingly effective, with BAE achieving a 54.2% success rate in fooling the classifiers.

Hardening Models Through Training#

To counteract these vulnerabilities, the researchers applied adversarial training—exposing models to attack examples during the learning process. This defensive approach successfully reduced residual susceptibility across all tested models, regardless of which specific attack was used against them. Notably, MuRIL, a multilingual model designed for Indian languages rather than Bengali specifically, demonstrated markedly greater robustness than dedicated Bengali models.

Open Research and Reproducibility#

All models, datasets, adversarial examples, and implementation code have been made publicly available to enable further research and adoption of these defenses by the broader community.