ChatGPT for Arabic-English Translation: Evaluating

the Accuracy
Faten Khoshafah (  [email protected] )
Ministry of Education, Yemen

Research Article

Keywords: ChatGPT, Translation, Human translation, Language, Arabic and English, Translation accuracy

Posted Date: April 17th, 2023

DOI: https://doi.org/10.21203/rs.3.rs-2814154/v2

License:   This work is licensed under a Creative Commons Attribution 4.0 International License.
Read Full License
KHOSHAFAH F. 2023

Article

ChatGPT for Arabic-English Translation: Evaluating the Accuracy

Faten Khoshafah*1
1. Ministry of Education, Yemen

*Corresponding Author Abstract: Cross-cultural communication has become more

Faten Khoshafah
[email protected]
accessible with the advent of ChatGPT as a translator for multiple
language pairings, including English and Arabic. However, the
accuracy of ChatGPT's translation needs to be evaluated due to
limitations in its understanding of domain-specific terminology
and cultural context. This article aims to evaluate the translation
accuracy of ChatGPT 3.5 by comparing its outputs with
professional translations of various text genres, such as historical,
literary, media, legal, and scientific. The results indicate that
ChatGPT can be used as a translator for simple content, though it
struggles with complexities that require human intervention.
Despite ChatGPT generally providing accurate translations, its
limitations make it unsuitable for some texts, such as legal
documents, medical reports, scientific studies, and literary works.
Therefore, users should be cautious while using ChatGPT for such
texts, and human intervention is required to ensure accuracy. The
article further offers recommendations for students and translators
who frequently use ChatGPT as a translator. Overall, ChatGPT is a
valuable tool for cross-cultural communication, but its limitations
should be kept in mind while using it as a translator.

Keywords: ChatGPT, Translation, Human translation, Language,

Arabic and English, Translation accuracy.
Copyright © 2023 The Author: This work is licensed under a CC BY 4.0 License.

INTRODUCTION

Globalization and technology have raised the demand for international communication
and information exchange in many fields. Translation has changed drastically. Due to the
requirement for instant translation and rapid development of new technology, machine
translation has greatly facilitated cross-cultural communication. Although machine translation
systems are best suited for translating instruction manuals and their translations are high-
quality, machine translation of other texts, especially complex texts, is still problematic, but
society's growing needs require its development.

https://doi.org/10.21203/rs.3.rs-2814154/v2 1
KHOSHAFAH F. 2023

The potent language model ChatGPT that OpenAI created uses deep learning to process
and produce natural language text (Possati, 2021; Schulman et al., 2022). It has been trained on
a massive amount of data, allowing it to understand and generate human-like text with
remarkable accuracy (Tate et al., 2023). ChatGPT can assist with research, generate text for
various types of documents, and provide feedback on the grammar, style, and coherence of the
written texts. ChatGPT uses a transformer-based architecture that allows it to learn the patterns
and syntactic structures of different languages. Unlike traditional rule-based methods, ChatGPT
can learn from a large corpus of text data, including books, articles, and websites, and generate
output that is contextually relevant and grammatically accurate. By encoding the English text
into a vector representation and then decoding it into Arabic text, ChatGPT is able to translate a
sentence from English to Arabic, for instance. This process ensures that the output is fluent and
coherent, and it also allows ChatGPT to handle complex language pairs that have different
grammar, syntax, and vocabulary.

In light of the recent meteoric rise of the language translation market. By 2021, the
global market for language services and technology is expected to be worth $56.2 billion, as
reported by Common Sense Advisory (Keen, 2020). The advent of AI and ML has upended the
translation market by making formerly inefficient processes more efficient, cheaper, and more
accurate. If you need a high-quality translation between languages, look no farther than
ChatGPT, one of the most cutting-edge AI-powered language models available. Hence, using
AI-based alternatives like ChatGPT might revolutionise the translation industry. ChatGPT's
ability to understand the context of a given prompt and generate appropriate responses makes it
an effective tool for translation. It can also generate translations that are difficult to distinguish
from human translations, making it a valuable asset for businesses and organisations that
require accurate and reliable translations.
It is also worth noting that AI-based models like ChatGPT will help foreign language
learners (Yan, 2023). For instance, an artificial neural network (ANN) in Google Translator
(GT) system has taken the place of traditional digital lexicographic dictionaries. As a result, the
researchers who conducted a study in an English as a Foreign Language (EFL) course at a
Chilean high school recommended using Google (AI-based) Translate to help students improve

https://doi.org/10.21203/rs.3.rs-2814154/v2 2
KHOSHAFAH F. 2023

their syntax and grammar. (Cancino & Panes, 2021). In the same manner, Tsai's (2019) study
also highlighted the efficacy of Google Translate in minimising typos and elongating texts.
(Tsai, 2019). With its "Google Neural Machine Translation" (GNMT) AI models and cutting-
edge training methods, Google's translation system mimics the way neurons in the human brain
process information by "taking in data, coding it, putting it into some sort of context, and then
re-coding it for output (King, 2019).
ChatGPT's versatility as a translator between several language pairings, including
English and Arabic, has made it an invaluable resource for learning, cross-cultural conversation
and understanding. Is AI-based modelling, then, a magical stick? Is ChatGPT going to provide
solutions for all the problems in machine translation, particularly for the complex language
pairs? The answer is "no"! This is due to certain obvious and hidden limitations, aside from the
ethical concerns and the potential bias of GPT algorithms (Possati, 2021), and the ChatGPT
cannot provide accurate translations for some sorts of texts, like literature, historical, legal, or
religious texts. This is because these types of texts have specific jargon and terms that can be
difficult to translate accurately without a deep understanding of the subject matter and its
context. Due to its inability to reason and analyse, Chat GPT may present incorrect information
and change the meaning of the original text. Therefore, human supervision has to be applied to
the translated text, particularly for scientific and legal documents, to circumvent this issue.
This paper focuses on demonstrating the potential capabilities of ChatGPT models in
translation between the English and Arabic languages. The purpose is to do determine if AI
language models like ChatGPT can accurately translate complicated messages from English to
Arabic and vice versa. Translators, editors, linguists, researchers, lecturers, and students in
translation studies, translation technologies, computational linguistics, and related fields, as well
as language engineers, information technology, and artificial intelligence specialists, may find
this research useful.

METHODS
This study focuses on different types of texts, including general texts from the media,
literary, scientific, religious, and legal texts, with the aim of comparing the accuracy of
ChatGPT translation in these different contexts. These texts have been selected from various

https://doi.org/10.21203/rs.3.rs-2814154/v2 3
KHOSHAFAH F. 2023

sources, such as books and websites, and are presented in the results tables alongside the chosen
text, as shown in the next section. Although ChatGPT offers a wide range of translations from
English into other languages, the author focused on Arabic and English due to the significant
number of English language users in Arab countries who require translations for different
purposes and contexts. The ChatGPT translations are rated based on their clarity and accuracy
compared to the human translations mentioned in the sources of these texts. The search process
is depicted in Figure 1.
The selected texts are translated using the ChatGPT translator, and the system is
explicitly prompted to translate them with high accuracy (refer to Appendix 1). Table 1 presents
an example of the selected sentence in its original form and the initial translation provided by
ChatGPT. Then, the human translation is entered into the chat, and the initial translation is
revised based on the human translation. The final modified translation is presented in the last
row of the tables. The new translation is then evaluated and discussed by the researcher.

Figure 1. The translation process

https://doi.org/10.21203/rs.3.rs-2814154/v2 4
KHOSHAFAH F. 2023

RESULTS & DISCUSSION

Table.1: Religious & historical (Ar-Eng)-Source (2, p. 237)

Original text ‫لمر ماتركته حتى يظهره هللا‬
‫وهللا لو وضعوا الشمس في يميني والقمر في يساري على ان اترك هذا ا أ‬
.(‫ محمد بن عبدهللا )صلى هللا عليه وسلم‬."‫او اهلك دونه‬
ChatGPT 1st "By Allah, if they were to place the sun in my right hand and the moon in my left
translation hand, I would never give up this matter until Allah makes it triumph or I die striving
for it." - Prophet Muhammad (peace be upon him)
Human “I swear by God (Allah), if they placed the sun in my right hand and the moon in my
translation: left hand, to cause me to renounce this task, I would not desist it, until God (Allah)
made manifest His cause, or I perish in an attempt.
Mohammed Ibn Abdellah (peace be upon him)”
Revised AI "I swear by Allah, even if they were to place the sun in my right hand and the moon in
translation: my left hand, I would never abandon this matter until Allah makes it manifest or I
by ChatGPT perish without it." - "Prophet Muhammad (peace be upon him)"

In this text, both translations convey the same meaning, but there are some
dissimilarities in style and choosing of words. Translation by human is more formal and
includes additional context as "His Cause". In this translation, context demands to be wider to
clarify meaning and in order to retain the meaning of the original text. The ChatGPT translation
is more succinct and straight-forward, but may not give the full depth of the source Arabic text.
Moreover, the use of parentheses in translation by human renders certain phrases or words are
clear and understandable as (Allah), while translation by ChatGPT depends on dashes. Human
translation uses the word "swear" to convey strong emphasis and "manifest" to capture the
meaning of "‫ "يظهره هللا‬in the original text. In ChatGPT translation there is phrase which is in the
source text disappears during the translation process and are absent in the target text like Ibn
Abdellah. It adds the word prophet which is not found in the original text.

Table.2: Religious & historical text. (Ar-Eng)-Source (2, p. 211)

Original text ‫ وعند جبل‬،‫ خرج رسول هللا صلى هللا عليه وسلم مع اكثر من مائة الف من المسلمين‬،‫في هذا العام‬
ً
‫ مناديا بالمساواة بين‬.‫عرفات القى الرسول خطبته الخالدة التي تعتبر دستور للدولة الأسلامية الجديدة‬
ً
‫ل القاكم بعد عامي‬ ‫اني لأ ادري لعلى أ‬H ‫ ف‬،‫ يا ايها الناس اسمعوا قولي‬: ‫لأسلام وقال‬
H ‫ مبينا قواعد ا‬.‫البشر‬
M M
..‫ن اكرمكم عند هللا اتقاكم‬H‫ ا‬.‫ كلكم لأدم و ا دم من تراب‬،‫ن اباكم واحد‬H‫ وا‬،‫ن ربكم واحد‬H‫ ا‬.. ‫هذا‬
.‫ اللهم فاشهد‬..‫ الأ هل بلغت‬.‫لأ بالتقوى‬H‫ل فضل لعربي على اعجمي ا‬
‫أ‬
ChatGPT 1st In this year, the Messenger of Allah, peace be upon him, went out with more than one
translation hundred thousand Muslims. At the mountain of Arafat, the Messenger delivered his
immortal sermon, which is considered a constitution for the new Islamic state. He

https://doi.org/10.21203/rs.3.rs-2814154/v2 5
KHOSHAFAH F. 2023

called for equality among human beings and explained the rules of Islam. He said, "O
people, listen to my words, for I do not know whether I will meet you again after this
year. Indeed, your Lord is one, and your father is one. All of you are from Adam, and
Adam is from dust. The most honorable among you in the sight of Allah is the most
righteous among you. There is no superiority for an Arab over a non-Arab except by
righteousness. Have I conveyed the message? O Allah, bear witness."
Human The Farwell Pilgrimage (631A. D -10th H)
translation: That year the Messenger peace be upon him went out with more than 100,000 Moslem.
At the Arafat Mountain where he delivered his immortal speech which all Moslems
consider a permanent constitution for the new Islamic State. He claimed equality of
people and indicated the rules of Islam saying "Oh people your God is one, yours
father is one. You belong to Adam and Adam was created out of dust. The best one of
you to Allah is the one who is devout and pious. An Arab is not better than a non-Arab
save by piety. (O! My Lord! I told them, O! My Lord! You do witness.)
Revised AI In the year of the Farewell Pilgrimage (631 AD - 10th H), the Messenger of Allah,
translation: peace be upon him, went out with more than one hundred thousand Muslims. At
by ChatGPT Mount Arafat, he delivered his immortal speech, which is considered a constitution for
the new Islamic state. He called for equality among people and explained the rules of
Islam, saying: "O people, listen to my words, for I do not know if I will meet you again
after this year. Your Lord is one, and your father is one. All of you are from Adam, and
Adam is from dust. The most honorable among you in the sight of Allah is the most
righteous among you. There is no preference for an Arab over a non-Arab except by
righteousness. Have I conveyed the message? O Allah, bear witness."

In this religious & historical text and through my evaluation of accuracy and
intelligibility of human and GPT translation, I have found that the human translation is more
detailed and accurately conveys the historical context and significance of event such as the
perfect date of the Pilgrimage. Furthermore, human translation contains more complete
translation of the main points of the speech, including the need for equality among people and
the special importance to piety as the measure of superiority, while the GPT translation is
satisfying, acceptable, and sufficient to get a good meaning for text. There are differences
between these two translations in words and style; human translation uses the words "belong",
"created out", and "piety".which make translation impeccable and accurate. ChatGPT uses
simple sentences without illustration that indicates that AI translation is generally accurate, but
lacks the depth of translation by human and some of the nuance. Finally, quality of GPT
translation is high enough to understand the source text, whereas human translation ensures
accuracy that conforms to the original text without ambiguity.

Table.3: media text (Eng-Ar)-Source (2, p. 134)

https://doi.org/10.21203/rs.3.rs-2814154/v2 6
KHOSHAFAH F. 2023

Original text Peace gives us a golden chance to carry out useful projects. It saves the money spent
on wars and destructive weapons. In peace, this money can be used for building new
factories, improves health care and solving the problems of housing and transport.
ChatGPT 1st ‫ فهو يوفر المال الذي ينفق على الحروب والأسلحة‬،‫السلام يعطينا فرصة ذهبية لتنفيذ مشاريع مفيدة‬
translation
،‫ وتحسين الرعاية الصحية‬،‫ يمكن استخدام هذا المال لبناء مصانع جديدة‬،‫ وفي زمن السلام‬.‫الدمارية‬
.‫لأسكان والنقل‬
H ‫وحل مشاكل ا‬
Human
translation:
‫لموال التي تنفق على‬
‫ حيث يوفر لنا ا أ‬،‫يوفر لنا السلام الفرصة الذهبية لتنفيذ المشروعات النافعة‬
‫ وكذلك‬،‫لموال لبناء مصانع جديدة‬
‫ وبالسلام يمكن استخدام هذه ا أ‬،‫الحروب والأسلحة المدمرة‬
.‫لأسكان والمواصلات وتحسين العناية الصحية‬
H ‫حل مشكلات ا‬
Revised AI ‫ يوفر المال الذي ينفق على الحروب والأسلحة‬.‫السلام يمنحنا فرصة ذهبية لتنفيذ مشاريع مفيدة‬
translation:
‫لأسكان‬
H ‫المدمرة ويمكن استخدامه في بناء مصانع جديدة وتحسين الرعاية الصحية وحل مشاكل ا‬
by ChatGPT
.‫والنقل‬

In this text I noticed that two translations are good and accurate and there is no
significant difference between them. However, some small improvements can be made. For
instance, translating by human uses the word "‫ "يوفر‬which means "provides" instead of "‫"يعطينا‬
which means "gives us".
The word "‫ "يوفر‬is more accurate and precise than the AI translation "‫"يعطينا‬.Human
translation uses the word "‫ "وبالسلام‬which means "in peace" instead of "‫ "وفي زمن السلام‬which is
used in the AI translation, here the source text doesn't talk about the period of peace, but it talks
about the peace at self which is a state of harmony among people. Generally, the GPT
translation is well and captures the meaning of the source text, however it could benefit from
some minor modulations to make it more correct and natural. Besides, this text translated by
GPT is surprisingly high and it does not deviate significantly from the text translated by human
interpretation.

Table.4: media text (Eng-Ar)-Source (2, p. 127)

Original text Many people say that they are too busy to go swimming or to play football. But they
do not have to do special exercises to be fit. In this case, walking or even cleaning the
house is just as good for them as practicing games.
ً
ChatGPT 1st ‫لى‬H‫ لكنهم لأ يحتاجون ا‬.‫نهم مشغولون جدا للذهاب للسباحة او اللعب في كرة القدم‬H‫كثيرون يقولون ا‬
translation
‫ المشي او حتى تنظيف المنزل يعد بمثابة‬،‫ في هذه الحالة‬.‫ممارسة تمارين خاصة للحفاظ على لياقتهم‬
.‫التمارين الرياضية بالنسبة لهم‬

https://doi.org/10.21203/rs.3.rs-2814154/v2 7
KHOSHAFAH F. 2023

Human
translation: ً
"‫يقول الكثير من الناس انهم مشغولون جدا لدرجة انهم لأيمكنهم ان يذهبوا للسباحة "يمارسوا السباحة‬
ً
‫وفي‬، ‫لداء تمرينات خاصة لكي يكونوا لأئقين صحيا‬ ‫ولكنهم ليسوا مضطرين أ‬، ‫او لعب كرة القدم‬
ً
.‫ن المشي او تنظيف المنزل مفيد لهم تماما مثل ممارسة الألعاب الرياضية‬H‫هذه الحالة فا‬
ً
Revised AI .‫ وهم لأ يستطيعون نهاية للسباحة او اللعب في كرة القدم‬، ‫نهم يشربون جدا‬H‫كثيرون يقولون ا‬
translation:
‫ يمكن للمشي او حتى تنظيف المنزل ان يكونا‬، ‫ في هذه الحالة‬.‫معدات تمارين خاصة لتحسين لياقتهم‬
by ChatGPT
.‫مفيدان بنفس القدر كم الألعاب الرياضية‬

The translated text from English to Arabic by human and GPT that the two translations
are very similar, nevertheless there are a few minor differences in choosing of words for
example, using the word "they" in AI translation instead of "many people" and "fit" instead of
ً
"physically fit".Additionally AI translation doesn't translate the word "just" which means " ‫"تماما‬
as human translation does. Automatic translation uses the word "‫ "بمثابة‬instead of "‫ "مثل‬which is
in translation by man and the word "exercises" which means "‫ "تمارين‬instead of "games" which
means "‫"العاب‬. In short, the two translations effectively convey the intended message in the
source text. The text translated by GPT does not decrease in the quality of the original text even
if there are changes in the style or words. I think that GPT provides accurate translation for
simple sentences.

Table.5: media text (Eng-Ar)-Source (2, p. 124)

Original text The government tries always to increase production in all fields. In agriculture, for
instance, it has prepared a plan for land reclamation and cultivation. It has also
obtained loans in order to be able to use machines in agricultural projects.
ً
ChatGPT 1st ‫ اعدت خطة‬،‫ على سبيل المثال‬،‫ في الزراعة‬.‫لنتاج في جميع المجالأت‬ ‫ أ‬H ‫تحاول الحكومة دائما زيادة ا‬
translation M
‫للأت في‬ ‫ كما حصلت على قروض من اجل القدرة على استخدام ا أ‬.‫لعادة الأراضي وزراعتها‬ ‫أ‬H
.‫المشاريع الزراعية‬
Human
translation: ً
‫ ففي مجال الزراعة على سبيل المثال اعدت‬.‫لأنتاج في كل المجالأت‬
H ‫لى زيادة ا‬H‫تسعى الحكومة دائما ا‬
‫الحكومة خطة لأستصلاح الأراضي وزراعتها كما حصلت على القروض اللازمة حتى تتمكن من‬
M
.‫دخال الألأت في مجال المشروعات الزراعية‬H‫ا‬
ً
Revised AI ،‫ على سبيل المثال‬،‫ ففي مجال الزراعة‬،‫لأنتاج في جميع المجالأت‬H ‫لى زيادة ا‬H‫تسعى الحكومة دائما ا‬
translation: M
‫ كما حصلت على قروض لتمكينها من استخدام الألأت‬،‫لراضي وزراعتها‬ ‫اعدت خطة لأستصلاح ا أ‬
by ChatGPT
.‫في مشاريعها الزراعية‬

https://doi.org/10.21203/rs.3.rs-2814154/v2 8
KHOSHAFAH F. 2023

In the translation of this text to Arabic by human and GPT-3, the translated text by GPT
is grammatically correct, but lacks the proper Arabic phrasing (way of use phrases and words)
for instance, the phrase "be able to use machines in agricultural projects " is translated as" the
ability to use machines in a agricultural projects". Also the phrase "land reclamation and
cultivation" is translated as "reclamation and cultivation of land", in this example there is no a
big mistake, but looks a bit awkward in Arabic. In general human translation sounds more
accurate, concise, and polished. It gives a broader image for the meaning of the original text.
Although AI translation still can't perform better than human translation, it can correctly convey
the meaning of the text.

Table.6: literature text (Ar-Eng)-Source (2, p. 239)

Original text ‫ن جاوز الشفتان ذاع‬H‫السر ا‬
ChatGPT 1st ".If a secret surpasses the lips, it becomes known
translation
Human ”The only way to keep a secret is to say nothing “
translation:
Revised AI ".The secret, once it goes beyond the lips, becomes known "
translation:
by ChatGPT

In this text, the human translation is more precise and gives the most basic and
important quality of original Arabic text in a simple and straightforward manner. Actually, GPT
translation communicates the meaning of the source Arabic text accurately, but it lacks the
clarity and conciseness. This translation could be improved by using easier language and direct
sentence structure to convey the intended sense clearly. Overall, I noticed that ChatGPT
translation could indeed be used for process of converting text from one language to another
correctly, but human translation is better at translating content for retaining the real meaning of
the original text.

Table.7: literature text (Ar-Eng)-Source (1)

Original text :‫جحا وابنه والحمار‬
ً
"‫لى هذا الرجل الذي خلا قلبه من‬H‫ انظروا ا‬:‫ركب جحا حمارا ومشي ابنه خلفه ومر امام جماعة فقالوا‬
‫ انظروا‬:‫ ومر على جماعة فقالوا‬.‫ فنزل جحا ومشى واركب ابنه‬.‫الشفقة يركب هو ويترك ابنه يمشي‬

https://doi.org/10.21203/rs.3.rs-2814154/v2 9
KHOSHAFAH F. 2023

‫ فمرا‬.‫ فركب جحا وابنه الحمار وسارا‬.‫لى هذا الغلام غير المودب يركب الحمار ويترك اباه يمشي‬H‫ا‬
‫ فنزل جحا وابنه‬.‫لى هذا الرجل القاسي يركب هو وابنه ولأ يرفقان بالحمار‬H‫ انظروا ا‬:‫من جماعة وقالوا‬
‫لى هذين المغفلين يتعبان من المشي وامامهم الحمار‬H‫ انظروا ا‬:‫ فمرا من جماعة جديدة فقالوا‬.‫ومشيا خلفه‬
‫ وسارا به ومروا بجماعة جديدة فضحكوا منهما‬.‫ حمل جحا وابنه الحمار‬،‫ وبعد ا ن جاوازاهم‬.‫ل يركبانه‬ ‫أ‬
ً
‫ قال جحا‬.‫ذن انزلأه‬H‫ وحين ا‬.‫لى هذين المجنونين يحملان الحمار بدلأ من ان يحملهما‬H‫ انظروا ا‬:‫وقالوا‬
ً ً
‫ل‬
‫رضاء الناس غاية أ‬H‫ قم بما تراه مناسبا فا‬،‫ل تستطيع ان تظفر برضى الناس جميعا‬
‫نك أ‬H‫ اسمع يا بنيه ا‬:‫لأبنه‬
‫"تدرك‬
st
ChatGPT 1 Joha, his son, and the donkey:
translation "Joha rode the donkey while his son walked behind him. They passed by a group of
people who said, 'Look at this man who has no pity in his heart. He rides while his son
walks.' So, Joha got off the donkey and let his son ride. They passed by another group
of people who said, 'Look at this rude boy who rides the donkey while his father
walks.' Then, Joha and his son both rode the donkey. They passed by another group of
people who said, 'Look at these cruel people. They both ride the donkey and show no
mercy.' So, Joha and his son got off the donkey and walked behind it. They passed by
another group of people who laughed at them and said, 'Look at these fools. They carry
the donkey instead of letting it carry them.' Eventually, Joha and his son took the
donkey down. Joha said to his son, 'Listen, my son. You cannot please everyone. Do
what you think is right because pleasing people is an impossible goal."
Human “Juha, His Son and the Donkey:
translation: Juha rode a donkey, and his son walked behind him. They passed by a group that said:
Look at this man, whose heart is devoid of pity. He rides and lets his son walk. Juha
got down to walk and had his son ride. He passed by a group of people that said: Look
at this impolite boy riding a donkey and letting his father walk. So Joha and his son
both rode the donkey and continued on their way. They passed by a group that said:
Look at this cruel man riding with his son. They are not kind to the donkey. So Joha
and his son got down from the donkey and walked behind it. They passed by a new
group that said: Look at these two idiots who are tired of walking and in front of them
is a donkey that they do not ride. After passing them, Juha and his son carried the
donkey. They walked with it and passed a new group, who laughed at them and said:
Look at these two madmen carrying the donkey instead of it carrying them. Just then,
he put it down. Juha said to his son: Listen, my son, you cannot win the approval of all
people. Do what you see fit, for pleasing people is an unattainable goal.” Source 1
Revised AI Juha rode the donkey, and his son walked behind him. They passed by a group who
translation: said: "Look at this man, whose heart is devoid of pity. He rides and lets his son walk."
by ChatGPT So Juha got down and let his son ride. They passed by another group who said, "Look
at this impolite boy riding a donkey and letting his father walk." Then, Juha and his
son both rode the donkey. They passed by another group who said, "Look at this cruel
man riding with his son. They are not kind to the donkey." So, Juha and his son got off
the donkey and walked behind it. They passed by another group who said, "Look at
these two fools who are tired of walking and have a donkey that they do not ride."
After passing them, Juha and his son carried the donkey. They walked with it and
passed a new group who laughed at them and said, "Look at these two madmen
carrying the donkey instead of it carrying them." Just then, they put it down. Juha said

1- Juha, His Son and the Donkey ‫( جحا وابنه والحمار‬transparent.com)

https://doi.org/10.21203/rs.3.rs-2814154/v2 10
KHOSHAFAH F. 2023

to his son: "Listen, my son, you cannot win the approval of all people. Do what you
see fit, for pleasing people is an unattainable goal."

In this literary text translated from Arabic to English by a human and GPT, I noticed that
the human translation is more polished and flows better than the AI translation. The ChatGPT
translation seems to have some grammatical errors, such as "Joha got off the donkey and let his
son ride" instead of "got down to walk and had his son ride". Furthermore, it misses the nuance
of language. It uses the sentence "look at this man who has no pity in his heart" instead of "look
at this man whose heart is devoid of pity" as translated by the human. Overall, the human
translation is more accurate and natural, conveying the meaning of the source Arabic text. In the
second revised translation, ChatGPT made some changes based on the comparison with the human
translation. It adjusted some of the word choices to make the meaning clearer and more accurate, and it
added a few missing details to help the story flow better.

Table.8: scientific text (Ar-Eng)-Source (3, p. 247~250)

Original text ‫العشرا حد النباتات المشھورة في المملكة وھو نبات شجیري معمر دائم الخضرة؛‬
،‫لوراق كبیرة‬
‫ا أ‬،‫ لحاوھا اسفنجي‬،‫ افرع النبات متخشبة ھشة‬.‫یصل ارتفاعه الى خمسة امتار‬
.‫ تحتوي جمیع اجزاء النبات عصارة لبنیة غزیرة‬. ‫لحمیة ذات لون اخضر مزرق لیس لھا عنق‬
.‫الأزھار مخضرة من الداخل بنفسجیة من الخارج وتوجد على مدار العام‬
ChatGPT 1st One of the famous plants in the Kingdom is the evergreen, long-lived, and shrubby
translation plant. Its height reaches up to five meters. The plant's branches are woody, its bark is
spongy, and its leaves are large and fleshy, with a blue-green color and no neck. All
parts of the plant contain abundant milky sap. The flowers are green inside and purple
outside and can be found throughout the year.
Human {Apple of Sodom} is one of the best-known plants of Saudi Arabia. It is a {perennial}
translation: {dendritic} {evergreen}4 shrub, which reaches five metres in height. The branches are
{woody} and brittle, and it has a {spongy} bark. Apple of Sodom has large, grey-green
{succulent} leaves without {petioles}. All parts of the plant contain a thick milky sap.
The flowers are greenish in the middle, and purple on the outside and grow throughout
the year. P.248 (Thinking Arabic Translation_ A Course in Translation Method_ Arabic
to English
Revised AI "Apple of Sodom" is one of the well-known plants in Saudi Arabia. It is a perennial,
translation: dendritic, evergreen shrub that can grow up to five meters in height. Its branches are
by ChatGPT woody and brittle, and its bark is spongy. The plant has large, succulent leaves with a
greenish-blue color and no petioles. All parts of the plant contain abundant milky sap.
The flowers are green on the inside and purple on the outside and can be found
throughout the year.

https://doi.org/10.21203/rs.3.rs-2814154/v2 11
KHOSHAFAH F. 2023

In this example, the AI translation is very poor, with some major issues and awkward
phrasing. For example, the name of the plant "‫( "عشر‬Apple of Sodom) is missing. The phrase
"evergreen, long-lived, and shrubby plant" is not as smooth as "perennial dendritic evergreen
shrub". The phrase "with a blue-green color and no neck" is not accurate, as the human
translation states "grey-green succulent leaves without petioles". Additionally, the phrase
"abundant milky sap" is not as precise as "thick milky sap" in the human translation. This
scientific text shows how ChatGPT performs in such specialized terms that require experts in
the field. Therefore, as recommended below, students and researchers should not rely on
ChatGPT translations for scientific content. It needs significant scaffolding to understand and
translate the text correctly, as shown in the second revised translation.

Table. 9: legal text. (Ar-Eng)-Source (4, p. 248)

Original text ‫يعاقب بالحبس مدة لأ تقل عن سنة كل من قام بنفسه او بواسطة الغير باستعراض القوة او التلويح‬
‫بالعنف او التهديد بايهما او استخدامه ضد المجنى عليه او مع زوجه او احد اصوله او فروعه وذلك‬
‫لأضرار بممتلكاته او سلب ماله او‬
H ‫لحاق اي اذى مادى او معنوي او ا‬H‫بقصد ترويعه او التخويف با‬
‫رادته لفرض السطو عليه‬H‫الحصول على منفعه منه او التاثير في ا‬

ChatGPT 1st Anyone who personally or through another person demonstrates the use of force,
translation threatens to use it, or intimidates with it against the victim, their spouse, their parents,
or their children, with the intent of frightening or scaring them or causing them
material or moral harm, or damaging their property, or robbing them, or benefiting
from them, or influencing their will to impose robbery on them, shall be punished by
imprisonment for no less than one year.
Human Whosoever demonstrates the use of force or insinuates violence or threatens to use
translation: either of them himself or via another shall be punished by imprisonment for not less
than one year for using them against the victim or against his spouse or one of his
ascendants or descendants, with a view to terrify or intimidate them through any
material or moral harm or damage to his possessions or stealing his money or getting
a benefit from him or influence his will to impose robbery upon him. (article 1)
(Book’s author translation, p.48
Revised AI Whosoever demonstrates the use of force, insinuates violence, or threatens to use
translation: either of them, either personally or through another person, against the victim, their
by ChatGPT spouse, parents, or children, with the intent to intimidate, terrify, or cause them any
material or moral harm, damage to their property, theft of their belongings, obtaining
benefits from them, or influencing their will to force them into robbery, shall be
punished with imprisonment for a period not less than one year.

https://doi.org/10.21203/rs.3.rs-2814154/v2 12
KHOSHAFAH F. 2023

In this legal text translated from Arabic to English, the human translation is very
thorough and precise, using accurate legal terminology and phrasing. It conveys the meaning of
the source text clearly and precisely, and it is grammatically correct. In comparison, the
ChatGPT translation is also accurate in conveying the meaning of the original text but falls
short of the human translation. It includes a number of redundant phrases and awkward
phrasing. For instance, "their parents or their children" is not an accurate translation of "‫اصوله‬
‫"وفروعه‬, which refers to a person's ascendants and descendants. Moreover, the phrase "with the
intent of frightening or scaring them" is redundant, as "to terrify or intimidate them" conveys
the same meaning. In general, whereas the machine translation is understandable, it could be
improved by using more accurate terminology and phrasing and eliminating redundancies.

Table. 10: legal text. (Eng-Ar)-Source (4, p. 140)

Original text

• No one shall be presumed guilty of a criminal offence before a judgment of

conviction has entered into force with regard to him or her.
• It is the case when somebody gives authority to another to act in his name as
his agent. If a provision is not included therein, then reference shall be made
to the Civil Code, otherwise provisions of commercial practice and guidance
by judicial, jurisprudent interpretation and equality principles shall be applied.
• Every fault which causes damage to another obliges that who has committed
this fault to repair it.
• So long as the Loan Agreement is in effect the warranties herein shall be true
and correct.
• The contractor shall not assign the Contract or any part thereof or benefit or
interest therein without the prior consent of the Employer.

ChatGPT 1st .‫لأدانة النهائي بشانه‬

H ‫دانة اي شخص بارتكاب جناية قبل صدور حكم ا‬H‫لأيجوز افتراض ا‬ •
translation M َّ ُ‫يحدث هذا عندما ي‬
‫ذا لم يتم تضمين اي‬H‫ ا‬.‫فوض شخص ما ا خر للتصرف نيابة عنه كوكيل‬ •
‫ل سيتم تطبيق احكام الممارسة‬
‫ أ‬H‫ وا‬،‫لى القانون المدني‬H‫ فسيتم الرجوع ا‬،‫بند في هذا الشان‬
.‫لى التفسير القضائي والفقهي ومبادى المساواة‬H‫التجارية والتوجيهات المستندة ا‬
ً M
.‫ملزما بتعويضها‬ ‫حداث اضرار لشخص ا خر يجعل المرتكب‬H‫كل خطا يتسبب في ا‬ •
‫ يجب ان تكون الضمانات المذكورة فيها صحيحة‬،‫طالما ان اتفاقية القرض سارية المفعول‬ •
.‫ودقيقة‬
‫ل يحق للمقاول ان ينتقل بالعقد او جزء منه او اي منفعة او مصلحة منه دون موافقة صاحب‬
‫أ‬ •
‫العمل المسبق‬

https://doi.org/10.21203/rs.3.rs-2814154/v2 13
‫‪KHOSHAFAH F. 2023‬‬

‫‪Human‬‬
‫‪translation:‬‬
‫لأدانة في حقه‬
‫ل يجوز افتراض ا‪H‬دانة اي شخص بارتكاب جريمة ما قبل نفاذ حكم ا ‪H‬‬
‫أ‬ ‫•‬
‫‪M‬‬
‫ا‪H‬نها الحالة التي يمنح فيها شخص السلطة لشخص ا خر للتصرف باسمه كوكيله‪.‬‬ ‫•‬
‫ل فتطبق احكام‬
‫فا‪H‬ن لم يوجد فيه حكم يتناول هذا الأمر يرجع ا‪H‬لى القانون المدني و ا‪ H‬أ‬
‫العرف التجاري و الأسترشاد بالأجتهادات القضائية والفقهية ومبادى المساواة ‪.‬‬
‫ً‬
‫كل خطا يسبب ضررا للغير يلزم من ارتكبه بالتعويض‪.‬‬ ‫•‬
‫طالما ان اٍتفاقية القرض سارية المفعول فاٍ ٍ ن الضمانات الواردة في هذا الأتفاق صحيحة‬ ‫•‬
‫يتعهد المقاول بعدم التنازل عن العقد او اي جزء منه او اي منفعة او مصلحة فيه او بموجبه‬ ‫•‬
‫بدون موافقة مسبقة من صاحب العمل‪(p.140 ) .‬‬
‫‪Revised AI‬‬ ‫لأدانة النهائي بحقه‪.‬‬
‫لأيجوز افتراض ا‪H‬دانة اي شخص بارتكاب جريمة ما قبل صدور حكم ا ‪H‬‬ ‫•‬
‫‪translation:‬‬ ‫‪M‬‬
‫تحدث هذه الحالة عندما يتم منح شخص ا خر السلطة للتصرف نيابة عنه كوكيل‪ .‬وا‪H‬ذا لم‬ ‫•‬
‫‪by ChatGPT‬‬
‫ل فا‪H‬ن احكام‬
‫لأشارة ا‪H‬لى هذا الأمر في اي بند‪ ،‬فا‪H‬ن القانون المدني هو الذي يطبق‪ ،‬وا‪ H‬أ‬
‫يتم ا ‪H‬‬
‫العرف التجاري وتوجيهات الفقهية والقضائية ومبادى المساواة ستطبق‪.‬‬
‫‪M‬‬ ‫ً‬
‫ضررا بشخص ا خر مسوولية تعويضه‪ ،‬طالما‬ ‫يجب ان يتحمل المتسبب في اي خطا يلحق‬ ‫•‬
‫انه يسبب الأضرار‪.‬‬
‫عندما تكون اتفاقية القرض سارية المفعول‪ ،‬يجب ان تكون الضمانات المنصوص عليها في‬ ‫•‬
‫الأتفاقية صحيحة ودقيقة‪.‬‬
‫لشكال او بيع اي جزء منه او الحصول‬
‫ل يحق للمقاول التنازل عن العقد باي شكل من ا أ‬
‫أ‬ ‫•‬
‫على اية منافع ا و مصالح بموجبه دون الحصول على موافقة صاحب العمل المسبقة‪.‬‬

‫‪In this legal text translated from English to Arabic, I found that the ChatGPT translation‬‬
‫‪is pretty accurate and captures the essence of the source text. However, there are some areas‬‬
‫" ‪where the GPT translation could be improved. For example, in the second sentence, the word‬‬
‫‪" means "grants" or‬يمنح" ‪", which is not an accurate translation.‬يحدث" ‪" is translated as‬يمنح‬
‫‪" means "happens" or "occurs". So the correct translation would be‬يحدث" ‪"gives", whereas‬‬
‫‪"this is the case where one person grants authority to another to act on their behalf as their‬‬
‫‪" which means‬الممارسة التجارية " ‪agent". In the third sentence, the AI translation uses the phrase‬‬
‫"العرف التجاري " ‪"commercial practice", but a more accurate translation in this context would be‬‬
‫‪which means "commercial custom". In short, although the GPT translation is effective and‬‬
‫‪exhibits improvements made in automatic translation, there is still room for improvement in‬‬
‫‪accuracy and nuance.‬‬

‫‪https://doi.org/10.21203/rs.3.rs-2814154/v2‬‬ ‫‪14‬‬
KHOSHAFAH F. 2023

CONCLUSION

Generally, there are some commonly used online translation tools, such as Google
Translate, Microsoft Translator, and others, which all provide great accuracy and cover many
languages. However, currently all eyes are on OpenAI's ChatGPT strengths make it a good
translator. Firstly, it can handle many complex and rare language pairs, such as Arabic and
Chinese. Companies that need to communicate with multilingual consumers or partners may
benefit from this. Secondly, ChatGPT can perform simultaneous translation, which is crucial for
time-sensitive operations such as customer support, emergency response, and multinational
conferences. Thirdly, ChatGPT may learn from user feedback and update and improve the
quality of the translation provided, as is evident in the translated texts in the previous section.
Based on the evaluation of the results presented in the previous section, we find that the
translations by ChatGPT and human translation carry the same meaning to some extent, but
there are some differences in style and words, as in the text of "‫ " حجةالوداع‬translation by
human uses the words"belong ", "created out", and "piety", which make translation impeccable
and precise. However, ChatGPT translation uses simple sentences make translation generally
accurate, whereas lacks the depth of translation by human and some of the nuance. Also in the
fourth text AI translation uses the word "fit" instead of "physically fit", the word "‫ "بمثابة‬instead
of "‫"مثل‬. Here, the translated text by GPT does not lower the quality of the original text even if
there are changes in style or words. Similarly, there are other texts translated by ChatGPT that
are grammatically correct but lack proper Arabic phrasing as presented in Table 5.
One of the major challenges in Arabic-English translation is the issue of word order. In
Arabic, the verb usually comes before the subject, while in English, it comes after; this can lead
to confusion and ambiguity in the translation process. Also, Arabic can omit pronouns, but
English must. Arabic pronouns vary by gender and sentence place. Translating pronouns may
require context. Idiomatic expressions are another challenge for ChatGPT. Arabic contains
many colloquial terms that are not present in English. ChatGPT may also struggle with Arabic
diacritical marks, which indicate the sounds of vowels and pronunciation. False cognates, words
that appear or sound similar but have different meanings in both languages, can also cause
translation problems. Moreover, the rules, sentence structure, and cultural background differ
between Arabic and English. Words change based on their function in Arabic, which is heavily

https://doi.org/10.21203/rs.3.rs-2814154/v2 15
KHOSHAFAH F. 2023

inflected. This makes Arabic sentences longer and more complex than English sentences.
Translation from Arabic to English may sometimes be challenging due to cultural differences.
Translating words and concepts with distinct meanings into different languages requires a
detailed awareness of the cultural background of the text.
Therefore, to make the most of ChatGPT as a translator, users should be aware of the
limitations of ChatGPT. Firstly, ChatGPT may not be suitable for translating highly technical or
specialised content that requires domain-specific knowledge, such as legal documents, medical
reports, scientific studies, religious, historical, and literary terminology. Secondly, ChatGPT
may produce culturally insensitive outputs, especially when translating idioms, jokes, and
colloquial language. ChatGPT may not understand the cultural significance of some phrases or
terms because it lacks the cultural background and context in which these expressions or terms
were used. In short, ChatGPT translation is not as accurate as human translation. Although it is
most effective, it is not useful for translating some long-form writing and jargon; it is used to
translate short documents. It seems to produce very good, fluent translations, but not necessarily
the most accurate ones. Human translation is best. Human translators translate and proofread
manuscripts.

RECOMMENDATIONS

To sum up, To ensure that users of ChatGPT can make the most of this translation tool,
it is important to take into account the following suggestions and recommendations, particularly
for those who are researchers, students, or teachers.

1- When using ChatGPT for translation, it is advisable to reference the responses provided
by the tool. While machine translation can help save time, it is not always perfect and
may experience delays during peak usage hours.
2- ChatGPT should not be relied on solely for translations on sensitive or significant topics,
as it is still a machine and may not always provide complete or accurate translations.
3- While human input was used to train ChatGPT, it is not infallible and can produce
biased content. It is suggested that ChatGPT users should not exclusively rely on it for
translations and should consider consulting a professional translator.

https://doi.org/10.21203/rs.3.rs-2814154/v2 16
KHOSHAFAH F. 2023

4- Translated texts produced by automatic translation should be monitored by professional

translators to mitigate any potential negative results.
5- ChatGPT is useful for translating short documents but lacks the nuance that human
translators provide for longer pieces, such as novels.
6- It's vital to keep in mind that ChatGPT might not be able to understand and come up
with as many novel thoughts or viewpoints as humans' minds could.
7- Users should be informed that ChatGPT has not yet been perfected, as it is still in beta.
However, as advancements in machine learning continue, ChatGPT and other similar
tools will shape the future of translation.

TEXTS SOURCES:

(1) Arabic language archive. (2021, August 9). Arabic Language Blog | Language and
Culture of the Arabic-Speaking World. https://blogs.transparent.com/arabic/archives-
and-categories/
(2) Ashraf Moawad Mustafa. (2000). Your guide to the correct translation (1st ed.). Ibn
Sina Library for Publishing and Distribution.
(3) Dickins, J., Hervey, S., & Higgins, I. (2016). Thinking Arabic translation: A course in
translation method: Arabic to English. Taylor & Francis.
(4) El-Farahaty, H. (2015). Arabic-English-Arabic legal translation (1st ed.). Routledge.

https://doi.org/10.21203/rs.3.rs-2814154/v2 17
KHOSHAFAH F. 2023

REFERENCES

Keen, C. (2020). The Global Market for Language Services and Technology. Common Sense
Advisory. Retrieved from https://csa-research.com/Reports/Market-Sizing/The-Global-
Market-for-Language-Services-and-Technology
King, K. (2019). Can Google translate be taught to translate literature? A case for humanists to
collaborate in the future of machine translation. Translation Review, 105(1), 76e92.
https://doi.org/10.1080/07374836.2019.1673268
Luca Possati (2021): “The algorithmic unconscious: how psychoanalysis helps in understanding
AI” (Routledge). AI & Soc (2022)
Schulman, J., Zoph, B., Kim, C., Hilton, J., Menick, J., Weng, J., Uribe, J. F. C., Fedus, L.,
Metz, L., Pokorny, M., Lopes, R. G., Zhao, S., Vijayvergiya, A., Sigler, E., Perelman, A.,
Voss,C., Heaton, M., Parish, J., Cummings, R. N., … Ryder, N. (2022, Nov. 30). ChatGPT:
Optimizing language models for dialogue. OpenAI.com. https://openai.com/blog/chatgpt/
Shu-Chiao Tsai (2019) Using google translate in EFL drafts: a preliminary investigation,
Computer Assisted Language Learning, (32) 5-6, 510-526,
https://doi.org/10.1080/09588221.2018.1527361
Tate, T., Doroudi, S., Ritchie, D., Xu, Y., & Uci, M. W. (2023). Educational Research and AI-
Generated Writing: Confronting the Coming Tsunami. EdArXiv.
https://doi.org/10.35542/osf.io/4mec3
Yan, D. (2023) Impact of ChatGPT on learners in a L2 writing practicum: An exploratory
investigation. Educ Inf Technol. https://doi.org/10.1007/s10639-023-11742-4

Funding: The author did not receive any financial support for the research, authorship, and
publication of this study.

Conflict of Interest: The author has declared no potential conflicts of interest in relation to the
research, authorship, and publication of this report.

Ethics Approval Statement: Ethics approval was not required for this report.

Data availability and sharing policy: All the data (texts) used in this article/appendix are
shared.

Acknowledgments: Not applicable.

https://doi.org/10.21203/rs.3.rs-2814154/v2 18
Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.

App1.pdf