ChatGPT vs. Google Translate: Comparative Analysis of Translation Quality
Abstract
The rise of large language models and their use in machine translation has prompted the need to examine the quality of their translations and compare them with other systems. This study aimed to assess the quality of literary translation from Persian to English using ChatGPT and Google Translate. A Persian short story was chosen, and both tools were used to generate translations. The translations were evaluated using Sofyan and Tarigan's (2019) functional holistic model, resulting in scores of 56% and 40% respectively. Additionally, a critical error analysis was conducted to identify areas where the tools struggled with effective translation, highlighting their strengths and weaknesses. These scores indicate that both machine translation systems have limitations in terms of accuracy, equivalence, and text function, particularly in literary translation. Moreover, the findings of this study emphasize the importance of human translators in achieving high-quality translations that effectively convey cultural nuances and idiomatic expressions in Persian to English literary translations, despite the convenience offered by machine translation systems for quick translations.
References
Adawiyah, A. R., Baharuddin, B., Wardana, L. A., & Farmasari, S. (2023). Comparing post-editing translations by Google NMT and Yandex NMT. TEKNOSASTIK, 21(1), 23–34.
Aghai, M. (2023). Quality asessment of online translation agencies in Iran. Iranian journal of translation studies, 20(80).
Boitet, C., Blanchon, H., Seligman, M., & Bellynck, V. (2009). Evolution of MT with the Web. In Proceedings of the conference on machine translation (Vol. 25, pp. 1–13).
Bonyadi, A. (2020). Exploring linguistic modifications of machine-translated literary articles: The case of Google Translate. Journal of Foreign Language Teaching and Translation Studies, 5(3), 89–104. doi: 10.22034/efl.2020.250576.1057
Cheng, Y., & Cheng, Y. (2019). Semi-supervised learning for neural machine translation. Joint training for neural machine translation, 25–40.
Dong, Q., Li, L., Dai, D., Zheng, C., Wu, Z., Chang, B. & Sui, Z. (2022). A survey for in-context learning. arXiv preprint arXiv:2301.00234
Holmes, J.S. (1988). The name and nature of translation studies. In Holmes, Translated! Papers on literary translation and translation studies. Rodopi.
Khoshafah, F. (2023). ChatGPT for Arabic-English translation: Evaluating the accuracy.
Liu, L., Utiyama, M., Finch, A., & Sumita, E. (2016). Neural machine translation with supervised attention. arXiv preprint arXiv:1609.04186.
Moneus, A. M., & Sahari, Y. M. (2023). Artificial intelligence and human translation: A contrastive study based on legal texts. Artificial intelligence and human translation: A contrastive study based on legal texts.
Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., & Tao, D. (2023). Towards making the most of ChatGPT for machine translation. arXiv preprint arXiv:2303.13780.
Sofyan, R., & Tarigan, B. (2019). Developing a holistic model of translation quality assessment. In Eleventh conference on applied linguistics (CONAPLIN 2018) (pp. 266–271). Atlantis Press.
Turovsky, B. (2016). Found in translation: More accurate, fluent sentences in Google Translate. Blog. Google. November, 15.
Vinay, J. P., & Darbelnet, J. (1995). Comparative stylistics of French and English: A methodology for translation (Vol. 11). John Benjamins.
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., & Dean, J. (2016). Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.
Xie, Y., Yu, C., Zhu, T., Bai, J., Gong, Z., & Soh, H. (2023). Translating natural language to planning goals with large-language models. arXiv preprint arXiv:2302.05128
Downloads
Published
Versions
- 2024-05-11 (2)
- 2024-05-11 (1)
How to Cite
Issue
Section
DOR
License
Copyright (c) 2024 Mohammad Aghai
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright Licensee: Iranian Journal of Translation Studies. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution–NonCommercial 4.0 International (CC BY-NC 4.0 license).
