A hybrid transformer-BiLSTM model for punctuation restoration in low-resource languages: Uzbek as a case study
2026, vol.18 , no.4, pp. 79-90
Article [2026-04-08]
Automatic punctuation restoration is an important natural language processing task that improves the readability of unpunctuated text and supports automatic speech recognition, machine translation and text editing systems. This study proposes a language-adaptable hybrid Transformer+BiLSTM framework for token-level punctuation restoration in low-resource languages and evaluates it using Uzbek as a case study. The task was formulated as a sequence-labeling problem with eleven classes. RoBERTa, XLM-RoBERTa, BERTbek and TahrirchiBERT were fine-tuned also BiLSTM layers were added to selected transformer encoders. The models were evaluated using accuracy, precision, recall and F1-score. XLM-RoBERTa+BiLSTM achieved the best overall results, with 0.8588 accuracy, 0.8493 precision, 0.8588 recall and 0.8393 F1-score. The findings indicate that bidirectional sequential modelling can complement transformer representations in low-resource punctuation restoration.
NLP, transformer models, Punctuation restoration, BiLSTM, low-resource languages
https://doi.org/10.59035/LJKP6387
Maksud Sharipov, Hushnudbek Adinaev. A hybrid transformer-BiLSTM model for punctuation restoration in low-resource languages: Uzbek as a case study. International Journal on Information Technologies and Security, vol.18 , no.4, 2026, pp. 79-90. https://doi.org/10.59035/LJKP6387