A multidimensional evaluation of AI-generated Arabic translations of English social media posts
Main Article Content
Abstract
The extensive implementation of AI in social media has revolutionized multilingual communication by allowing users to generate automatic translations of their content in different languages. Though significant progress has been made in neural machine translation, few studies have explored whether the translations generated by AI can adequately convey communicative meanings in real digital settings, especially in translating from English to Arabic. This study examines the quality of automatic translations of English-language social media messages on Facebook and X (previously Twitter) in Arabic. A qualitative corpus-based method was used to analyze 60 authentic messages evenly distributed over six areas of communication: politics, health care, tourism, business, entertainment, and everyday social communication in equal proportion for each system, 30 for each. The translation was assessed according to a multidimensional evaluation framework based on the MQM model, pragmatic analysis, cultural analysis, and multimodal discourse analysis. The results suggest that AI translation systems are efficient in preserving the semantics of content, terminology, and facts of the message, especially for institutional and informative posts. Nonetheless, common issues were encountered when translating humor, metaphors, political language, culture-specific expressions, and multimodal content, leading to lower communicative effectiveness despite a high level of lexical accuracy. Moreover, Facebook posts posed higher pragmatic challenges than X posts because of their longer and more persuasive discourse construction. This study contributes to media and communication research in terms of conceptualizing the phenomenon of automatic translation as AI-mediated multilingual communication and suggesting an evaluation framework for translation quality assessment.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This open-access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
No additional restrictions You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
How to Cite
Share
References
Abdu, F. J., Mughaus, R., Abudalfa, S., Ahmed, M., & Abdelali, A. (2025). An empirical evaluation of arabic text formality transfer: a comparative study. Language Resources and Evaluation, 59(4), 4093-4153.
Abu-Rayyash, H., & Haider, A. S. (2024). Introducing Transvisio: A customizable AI-powered subtitling tool. Research Journal in Advanced Humanities, 5(4), 618-632. https://doi.org/https://orcid.org/0000-0002-7763-201X
Aburahme, M. (2021). The Life of the Arabic Language, its Merit, and Means of its Development in the Thought of Muhammad al-Khadr Hussein. Jordan Journal of Applied Science-Humanities Series, 27(2), 110-121. https://doi.org/ https://doi.org/10.35192/jjoas-h.v27i2.8
Ahmad, M., Haider, A. S., & Saed, H. (2025). Assessing AI-driven dubbing websites: Reactions of Arabic native speakers to AI-dubbed English videos in Arabic. Research Journal in Advanced Humanities, 6(1), 1-20. https://doi.org/https://doi.org/10.58256/64dz1c72
Akasheh, W. M., Haider, A. S., Al-Saideen, B., & Sahari, Y. (2024). Artificial intelligence-generated Arabic subtitles: insights from Veed. io's automatic speech recognition system of Jordanian Arabic. Texto Livre, 17, e46952-e46952. https://doi.org/10.1590/1983-3652.2024.46952.
Al-Darabee, M., Farghal, M., & Haider, A. S. (2025). Netflix Versus Google Translate: A Case Study of the English-Arabic Translation of Scatological Terms. Studies in Linguistics Culture and FLT, 13(2), 77-95.
Al-Momani, A., Haider, A. S., Dagamseh, M., & Akasheh, W. M. (2025). Audience responses to cultural and linguistic gaps in English–Arabic auto-subtitles on YouTube. Research Journal in Advanced Humanities, 6(3).
Al-Salman, S., & Haider, A. S. (2024). Assessing the accuracy of MT and AI tools in translating humanities or social sciences Arabic research titles into English: Evidence from Google Translate, Gemini, and ChatGPT International Journal of Data and Network Science, 8(4), 2483-2498. https://doi.org/https://doi.org/10.5267/j.ijdns.2024.5.009
Al Maaytah, S. A. (2025). A CRITICAL EVALUATION OF GOOGLE TRANSLATE IN ENGLISH-ARABIC TRANSLATION: ACCURACY AND LIMITATIONS.
AlAfnan, M. A. (2025). Artificial intelligence and language: Bridging Arabic and English with technology. Journal of Ecohumanism, 4(1), 240-256.
Alharbi, M. A. (2025). Navigating AI-Driven Translation in Saudi Arabia's Media: Challenges and Opportunities. Arabic Journal for Translation Studies, 4(12), 10-31.
Almahasees, Z., & Jaccomard, H. (2020). Facebook Translation Service (FTS) usage among Jordanians during COVID-19 lockdown. Advances in Science, Technology, Engineering Systems Journal, 5(6), 514-519.
Ammari, R., & Al-Ahmad, W. (2023). A corpus-based study of English synonyms: Small, little, tiny, and petite. Jordan Journal of Applied Science-Humanities Series, 36(2), 1-20. https://doi.org/https://doi.org/10.35192/jjoas-h.v36i2.542
Baker, M. (2018). In other words: A coursebook on translation. Routledge.
Behravan, M., Mohammadrezaei, E., Azab, M., & Gračanin, D. (2024). Multilingual standalone trustworthy voice-based social network for disaster situations. 2024 IEEE 15th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON),
Benammar, S. (2024). The pragmatic effects of figurative language in social media Université de Haute Alsace-Mulhouse].
Bravo, C., Castells, V. B., Zietek-Gutsch, S., Bodin, P.-A., Molony, C., & Frühwein, M. (2022). Using social media listening and data mining to understand travellers’ perspectives on travel disease risks and vaccine-related attitudes and behaviours. Journal of Travel Medicine, 29(2), taac009.
Castilho, S., & Knowles, R. (2025). A survey of context in neural machine translation and its evaluation. Natural Language Processing, 31(4), 986-1016.
Chen, S., & Lin, Y. (2025). A multidimensional comparison of ChatGPT, Google Translate, and DeepL in Chinese tourism texts translation: fidelity, fluency, cultural sensitivity, and persuasiveness. Frontiers in Artificial Intelligence, 8, 1619489.
Costa-Jussà, M. R., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., & Maillard, J. (2022). No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672.
Darwish, N., Haider, A., Rummana, B. T. R. N. A., Alantari, D., Saed, H., & Dagamseh, M. (2025). A Reception Study of AI-Translated Idioms and Proverbs between Arabic and English. Research Journal in Advanced Humanities, 6(3), 1-12. https://doi.org/https://doi.org/10.58256/k4d6pp20
Desjardins, R. (2016). Translation and social media: In theory. In Translation and Social Media: In Theory, in Training and in Professional Practice (pp. 35-66). Springer.
Desjardins, R. (2019). Translation, pragmatics and social media. In The Routledge Handbook of Translation and Pragmatics (pp. 375-393). Routledge.
Ding, J. (2024). Corpus-based translation studies: Examining media language through a linguistic lens. SHS Web of Conferences,
El-Siddig, M., & Mohamed, M. (2025). Comparing human and machine translation quality in student educational and social media texts. In: Dragoman.
Elhadary, T. (2023). Linguistic and cultural differences between English and Arabic languages and their impact on the translation process. International Journal of Language and Translation Research, 3(2), 103-117.
Farghal, M., & Haider, A. S. (2025). A Cogno-Prosodic Approach to Translating Arabic Poetry into English: Human vs. Machine. 3L, Language, Linguistics, Literature, 31(1), 255-271. https://doi.org/https://doi.org/10.17576/3l-2025-3101-17
Farghal, T. M., & Ammari, R. M. G. (2025). Evaluating English-Arabic Automated Subtitling on Social Media Reels and Shorts: Linguistic Challenges and Performance Metrics♦.
Fazil, A. W., Hakimi, M., Akrami, K., Akrami, M., & Akrami, F. (2024). Exploring the role of social media in bridging gaps and facilitating global communication. Studies in Media, Journalism and Communications, 2(1), 13-21.
Feng, Z., Cao, S., Ren, J., Su, J., Chen, R., Zhang, Y., Xu, Z., Hu, Y., Wu, J., & Liu, Z. (2025). Mt-r1-zero: Advancing llm-based machine translation via r1-zero-like reinforcement learning. arXiv preprint arXiv:2504.10160.
Gwasmeh M, Riziq DK, Al-Abbas LS, Almubasher ZM, Haider AS, Sahari Y, & WM, A.-D. (2026). AI in translation classrooms: examining student practices in Jordanian and Iraqi universities. Front. Educ., 1-14.
Hadley, B. (2017). Theatre, social media, and meaning making (Vol. 7). Springer.
Haider, A. S., Obeidat, M. M., Hamdan, Y., AlZghoul, O., & Abu-Rayyash, H. (2025). SDL Trados Live Cloud vs. Traditional Subtitling Workflows: An Empirical Evaluation of Performance. Research Journal in Advanced Humanities, 6(3), 1-11.
Haider, A. S., Obeidat, M. M., Hamdan, Y., AlZghoul, O., & Abu-Rayyash, H. (2026). Tech-driven advances in audiovisual translation: developing a cloud-based English-Arabic subtitle corpus for training and practice. Texto Livre, 18(2025), 1-19.
Han, C. (2025). Quality assessment in multilingual, multimodal, and multiagent translation and interpreting (QAM3 T&I): Proposing a unifying framework for research. Interpreting and Society, 5(1), 27-55.
Hatab, W. A., Khaleel, M., Mohammed, A., & Al-Tarawneh, A. (2024). A contrastive study on the thematic structure functions of English and Arabic short stories. World Journal of English Language, 14(2).
Imam, S. (2025). Evaluating Translation Quality in Dubbed and Subtitled Translations: A Quantitative Study Using a Functional Holistic Model Itä-Suomen yliopisto].
Khatatbah, D., & Ashour, M. (2024). The Role of School Administration in Activating the Use of Information Technology at Schools in Irbid Governorate. Jordan Journal of Applied Science-Humanities Series 38(2), 121-137. https://doi.org/https://doi.org/10.35192/jjoas-h.v38i2.718
Kruk, M., & Kałużna, A. (2025). Investigating the role of AI tools in enhancing translation skills, emotional experiences, and motivation in L2 learning. European Journal of Education, 60(1), e12859.
Lang, C., Su, Q., Xia, S., & Zboja, J. (2025). The effectiveness of social media communication of sustainable fashion brands: evidence from consumer engagement. Atlantic Journal of Communication, 33(2), 237-256.
Li, B. (2025). Crowdsourced Translation as Immaterial Labour: Emerging Communities of Practice (CoP) in the TED Translation Project Université d'Ottawa/University of Ottawa].
Lommel, A., Uszkoreit, H., & Burchardt, A. (2014). Multidimensional quality metrics (MQM): A framework for declaring and describing translation quality metrics. Tradumàtica(12), 0455-0463.
Mahadin, D., Olimat, S. N., & Almahasees, Z. (2025). Artificial Intelligence in translation studies: a cross-sectional survey of Jordanian academics’ knowledge, attitudes and practices. Texto Livre, 18, e57583.
Manystighosa, A., Junining, E., & Rusmawati, R. (2022). Indonesian user perception on the usefulness of auto-translate feature on social media. Journey: Journal of English Language and Pedagogy, 5(2), 149-159.
Megdadi, S., Almahasees, Z., Al-Harahsheh, A., Al Rousan, R., Al-Taher, M. A., & Husienat, I. (2026). Translating food idioms across the English-Arabic language divide: A com-parative study of human translators, ChatGPT, and Gemini. Research Journal in Advanced Humanities, 7(2), 1-15.
Meqdad, S., Al-Bayyari, M., & Al-Taher, M. (2023). Semantic Disparity between Amr (Command) and Nahy (Negative Command) in the Noble Quran: An Interpretive/Contextual Study. Jordan Journal of Applied Science-Humanities Series, 36(1), 1-10. https://doi.org/https://doi.org/10.35192/jjoas-h.v36i1.516
Naib, A. (2025). Algerian Viewers’ Attitudes toward Taboo Language: A Case Study of Mourad Oudia’s Humorous YouTube Content. Jordan Journal of Applied Science-Humanities Series, 44(1), 167-191. https://doi.org/https://doi.org/10.35192/jjoas-h.v44i1.1830
Nesterenko, T., Kozii, O., Varych, N., Shulzhenko, A., & Tsypina, D. (2024). Language technologies impact on modern communication: analysis of new formats, challenges, and education.
Obeidat, M. M., Haider, A. S., Tair, S. A., & Sahari, Y. (2024). Analyzing the Performance of Gemini, ChatGPT, and Google Translate in Rendering English Idioms into Arabic. FWU Journal of Social Sciences, 18(4), 1-18. https://doi.org/http://doi.org/10.51709/19951272/Winter2024/1
Oshima, J., Oshima, R., & Taiki Kawakubo, A. J. (2025). Multi‐Layered Temporal Network Analysis: Idea Improvement Processes in Knowledge‐Building Practice Leading to High Learning Performance. Journal of Computer Assisted Learning, 41(3), e70063.
Oshin, A. A. (2023). Machine Translators and Indigenous Knowledge in Hausa Language: A Case for Google and Facebook Translators. NIU Journal of Humanities, 8(1), 27-32.
Paul, B. (2023). Advancements and perspectives in machine translation: A comprehensive review. 1st-International Conference on Recent Innovations in Computing, Science & Technology.
Qiu, J., & Pym, A. (2025). Fatal flaws? Investigating the effects of machine translation errors on audience reception in the audiovisual context. Perspectives, 33(5), 913-929.
Rashid, M. N. (2026). Challenges of Translating Social Media'Trends' and Modern Slang Expressions: The Dilemma of Maintaining Cultural Impact and News Speed. Journal of The Iraqi University, 75(2), 993.
Robertson, S. P. (2018). Social media and civic engagement: History, theory, and practice. Morgan & Claypool Publishers.
Rothwell, A., Moorkens, J., Fernández-Parra, M., Drugan, J., & Austermuehl, F. (2023). Translation tools and technologies. Routledge.
Rüdiger, S., & Mühleisen, S. (2022). Introduction: Formality and informality in online performances. In (Vol. 5, pp. 1-11): John Benjamins Publishing Company Amsterdam/Philadelphia.
Sallam, M., Al-Mahzoum, K., Alshuaib, O., Alhajri, H., Alotaibi, F., Alkhurainej, D., Al-Balwah, M. Y., Barakat, M., & Egger, J. (2024). Language discrepancies in the performance of generative artificial intelligence models: an examination of infectious disease queries in English and Arabic. BMC Infectious Diseases, 24(1), 799.
Savoldi, B., Bastings, J., Bentivogli, L., & Vanmassenhove, E. (2025). A decade of gender bias in machine translation. Patterns, 6(6).
Shahmerdanova, R. (2025). Artificial intelligence in translation: Challenges and opportunities. Acta Globalis Humanitatis Et Linguarum, 2(1), 62-70.
Singh, S., Chaubey, D. S., Raj, R., Kumar, V., Paliwal, M., & Mahlawat, S. (2025). Social media communication, consumer attitude and purchase intention in lifestyle category products: a PLS-SEM modeling. Marketing Intelligence & Planning, 43(2), 272-296.
Suram, P., Banik, D., & Sharma, A. K. (2026). Large language model based machine translation for universal multilingual understanding and translation quality enhancement. Discover Computing, 29(1), 217.
Tannous, B., & Haider, A. S. (2025). Assessing the Accuracy of AI Tools (Google Translate and Gemini) in Translating Arabic Idiomatic Expressions and Proverbs into English. International Journal on Artificial Intelligence Tools, 34(04n05), 2550016. https://doi.org/https://doi.org/10.1142/s0218213025500162
Tavares, C., Tallone, L., Oliveira, L., & Ribeiro, S. (2023). The challenges of teaching and assessing technical translation in an era of neural machine translation. Education Sciences, 13(6), 541.
Wang, Y., Zhang, J., Shi, T., Deng, D., Tian, Y., & Matsumoto, T. (2024). Recent advances in interactive machine translation with large language models. Ieee Access, 12, 179353-179382.
Wells, N. (2022). Translation as Culture in the Age of the Machine. Wasafiri, 37(3), 77-80.
Zakaryia, A., Mohammad, A.-T., & Jaccomard, H. Evaluation of Facebook Translation Service (FTS) in Translating Facebook Posts from English into Arabic in Terms of TAUS Adequacy and Fluency during Covid-19. In: Academic Press.
Zhang, X., Rajabi, N., Duh, K., & Koehn, P. (2023). Machine translation with large language models: Prompting, few-shot learning, and fine-tuning with QLoRA. Proceedings of the Eighth Conference on Machine Translation.