International Conference on Innovative Computing and Communication (ICICC 2020)

A Review on Implementation Issues of Rule-based Chatbot Systems

Dr. Sandeep A. Thorata ,Vishakha D. Jadhavb*

a
Rajarambapu Institute of Technology, Islampur,Sangli 415414, Maharashtra, India
b
Rajarambapu Institute of Technology, Islampur,Sangli 415414, Maharashtra, India

Abstract: Communication between a human and computer is area of interest for many researchers since past many years. There are different ways to
perform the communication between human and computers. Chatbots have become popular mechanism to perform such communication. Chatbot is a
computer program which makes this interaction easy and interesting. Existing artificial intelligence methods show poor performance while giving
most appropriate response to the user’s query. Hence at present rule-based Chatbot systems are more popular among the industries. In this paper, we
present detailed study on implementing rule-based Chatbot systems. The paper discusses performance measurement parameters for Chatbot systems.
The paper presents comparison between Google Dialogflow and IBM Watson, which are two most popular rule-based Chatbot implementation
frameworks. The paper ends by listing expectations from future Chatbot systems.

Keywords: Chatbots Rule-based Chatbots Platforms and Frame-works Google Dialogflow, IBM Watson Performance Parameters.

The question is how to make Chatbot system smart enough. There

1. Introduction and Motivation
are two ways of doing so:
A Chatbot also called as Smartbot or Chatterbot is a computer
program which performs interaction among human and machine using  Rule-based Chatbots:
audio or messaging methods. Chatbots are generally dialog systems Here a Chatbot system works on basis of certain rules. However when
having various purposes like customer service, information acquisition, input pattern does not match with any predefined rule then this Chatbot
automated information retrieval etc. End users ask their queries then system is inefficient to answer the question. Developers use AIML
Chatbot systems give appropriate answer to the user. Chatbot make life (Artificial Intelligence Markup Language) to write rules for Chatbot
easier for end user and are available for 24/7 hours. Now a day’s System. AIML is XML based language. Composing rules for various
Chatbot system can easily communicate like human being. Chatbots are situations is very tedious job and it is di cult to write rules for each and
gaining lot of popularity in various sector and it reduces human efforts. every possible situation. These rule based Chatbot system can deal with
These benefits triggered increase in number of Chatbot systems and its’ straightforward questions but it is crucial to manage complex questions.
users [15]. Businesses and entrepreneurs are in a race to develop Most social media Chatbot systems are rule-based.
Chatbots for improving user engagement and giving personalized
experience to the users. Now Customers can search and shop more  Self-learning Chatbots:
easily with the help of Chatbot system. Customers can order physical These Chatbots use machine learning algorithms which enable them
goods like cloths and also buy gifts conveniently. Chatbot system also to learn things. These bots can be of two types: Using Retrieval Based
solve travel related problems. Mariott International’s Chatbot is used for Models: These bots are trained for lot of inquiries and their possible
online provision purpose. Best banking Chatbot application is HDFC answers. For each question, the bot can locate the most important
bank’s EVA (Electronic Virtual assistant). It uses the AI (Artificial answers from the set of every conceivable answer. Likewise, there is no
Intelligence) and NLP (Natural language processing) to understand issue with the language and sentence structure as the appropriate
user’s input. Eva has addressed in excess of 5 million inquiries with responses are pre-decided and it can’t turn out badly in sentence
more than 85% accuracy [24]. Chatbot interact with user through structure way. Using Generative Models: Generative models do not
messaging applications, websites, and mobile application also through replies with same answer from set of answer. They take word by word
the telephone. Chatbot system perform two different task as analysing from the inquiry and give appropriate responses. These models should be
request given by user and giving appropriate response to the user. In prepared more exactly because it can handle spelling and grammar errors
first task, Chatbot analyses user’s input then identify intent to extract easily.
appropriate entities. As part of second task Chatbot gives appropriate
response to the user. Once the user’s intent recognized correctly then The self-learnable Chatbots are yet to become mature. Many times
Chatbot system should give response to the user. Here answer can be in such Chat-bots are giving irrelevant responses to user’s queries, which
generic and predefined text or in the form of knowledge base [5]. may causes irritating experience to the business customers. Hence,
businesses and industries are preferring rule-based Chatbot systems as of

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 1

Electronic copy available at: https://ssrn.com/abstract=3567047

International Conference on Innovative Computing and Communication (ICICC 2020)
now. This paper focus on various issues related with design, email and others. Microsoft Bot Framework has its own software
development and implementation of rule-based Chatbot systems. The development kit (SDK).
major contributions of this paper are as follow:

 This paper present critical analysis of various existing rule based 3. Popular Chatbot Platforms and Frameworks
Chatbot system
 The paper discusses issues in implementing Chatbot systems and it While implementing rule based Chatbot systems two key terminologies
guides organizations about choosing appropriate Chatbot platform are Chat-bot development frameworks, and Chatbot platforms. Many
and framework times people get confused in these two terminologies. Below table 1
 The paper presents case studies on two most popular rule based shows differences between the frame-work and platform.
Chatbot systems viz. Google Dialogflow and IBM Watson. The
paper compares both frameworks on various parameters Table 1 - Difference between Chatbot development frameworks and
Chatbot platforms. [21]

This paper is organized as follows: Section 2 discusses historical Chatbot Frameworks Chatbot Platforms
development of Chatbots. Section 3 describes and compares Chatbot It contain set of predefined On-line ecosystem where Chatbot
implementation platforms and development frameworks. Section 4 functions and classes. can be deployed and interact.
explains parameters to measure performance of the Chatbot system. Chatbot framework helps to Chatbot platform provide base to
Detail the design of Google Dialogflow and IBM Watson Chatbot develop and bind components of deploy and run the Chatbot
system and its limitations are discussed in Section 5. Section 6 gives Chatbot system. It give tool that application.
Conclusion of this study and also discusses expectations from future help to write code in better and
faster manner.
Chatbot system.
It requires programming languages It is easy and simple to understand
to develop. to make a Chatbot with the help of
2. History of Chatbot Systems simple drag and drop approach.

Chatbot Framework provide the Chatbot publishing platform is the

All figures should be numbered ELIZA was first Chatbot which is
flexibility to store-data, produce way through which the Chatbot
created by Joseph Weizenbaum in 1966. ELIZA uses simple pattern analytic. can access and used by user.
matching. ELIZA was designed to work as a therapist who ask questions
Used by developers and coders to Used by beginners or non-
and give respond to user [3, 4].Kenneth Colby created PARRY in 1972.
build Chatbot using programming technical users. User can develop
PARRY have same response pattern structure as ELIZA but having language. a Chatbot without coding.
advanced control structure. It is used to simulate the disease. This Example: Microsoft Bot Example: Chattypeople, Botsify,
Chatbot system also having language understanding capabilities. Framework, Api.ai., Wit.ai, Chatfuel, Motion.ai, etc.
PARRY has effective variables like anger, fear, and mistrust [13]. Google Dialogflow, IBM Watson,
ALICE (Artificial Linguistic Internet Computer Entity) was created by RASA NLU etc.

Richard Wallace in 1995.It was inspired by ELIZA. It is an open source This research work focuses on study of Chatbot development
natural language handling Chatbot program that interact with a human frameworks. As per Google Trends IBM Watson and Google Dialogflow
by using some pattern matching rules. ALICE is situated in XML are being most popular Chatbot development framework system at
knowledge base. It coordinates the client contribution against predefined present. Hence in next part we are going to focus on detailed analysis of
set of reactions. As it has a predefined set QA reactions, it can’t answer Google Dialog flow and IBM Watson. Section IV explains which
every one of the inquiries satisfactorily it is feasible for ALICE bots to parameters affect performance of the Chatbot system.
grow their insight bases through a XML. Utilizing this, An ALICE bot
can be intended to be a specialist in any space specific data [2].Google 4. Performance Evaluation of Chatbot Systems
Dialogflow [10] is Google-owned Chatbot system was developed in
2010. It is based on natural language conversations. These Chatbot are There is need to evaluate and compare performance of a Chatbot
able to respond voice commands. Dialogflow is Google service that systems. There are number of dimensions on which we can compare
execute on Google cloud platform. User can interact with Chatbot Chatbot systems. This section discusses various performance
system via voice and text-based interface. Watson is a Q&A framework measurement parameters proposed by [16], [17] and [18]. As per [16]
[8] created in IBM’s Deep Q&A venture by the exploration group. IBM following parameters should be considered to evaluate performance of
Watson According to the exploration study by Mindbowser in Chatbot system - Variety of tasks done by the Chatbot, Number of
relationship with Chatbot Journal IBM Watson is the main decision as a channels (Facebook, Skype, Telegram etc.) supported, Support for NLP
bot-building stage for 61% of organizations. Microsoft Bot Frame-work and Speech, Collect data from user intelligently through interaction,
[9] help user that connect Chatbot with different channels such as SMS, Tools available to build Chatbot, etc. [17] suggests following
parameters to evaluate performance of Chatbot system -Total users

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 2

Electronic copy available at: https://ssrn.com/abstract=3567047

International Conference on Innovative Computing and Communication (ICICC 2020)
using Chatbot, Number of messages exchanged per day, New users Following table shows comparison of two Chatbot systems based on
registered recently, Number of times Chatbot gave don't know reply, natural language understanding features.
Rating given by user at end of conversation, etc. [18] listed following
parameters for performance measurement - Price of system, Credibility
of the service , User friendliness, Exit strategy when Chatbot is not able
to answer, etc. After summarizing above parameters, we think that
following parameters are most important while evaluating performance
of a Chatbot system: Multitasking, Multi-channel support, Support for
NLP (Natural Language Programming), Price, User Friendliness,
Programming language/Apps/Integration, Free to Explore, Flexibility, Table 2 – Comparison of Dialogflow and IBM Watson. [23]
and Text-based query support, Voice-based query support, Total Users, Platform ML Based ML Based Dictionary
Engaged Users, Retention Rate, Fall Back Rate, Usability, Elasticity and Intent Entities Based
Scalability of the system. Most of these features are supported by Classification Recognition Entities
Google Dialogflow and IBM Watson Chatbot system. We present Recognition
detailed case study on these two Chatbot systems in next section.
Dialogflow Yes No Dictionary +
Auto expansion
5. Case Study on Google Dialogflow and IBM Watson IBM Watson Yes Yes Yes

In this section we are going to present case study on working of two 1.2. Limitations of Google Dialogflow and IBM Watson
most popular Chatbot frameworks viz. Google Dialogflow and IBM
Watson. As part of our research we identified limitations of both Google
Google Dialogflow is a natural language understanding engine Dialogflow and IBM Watson. In this sub-section we are going to discuss
developed by Google. Dialogflow help users to build intelligent Chatbot these identified limitations. Google Dialogflow Limitations: Dialogflow
system. IBM Watson is a question and answer system that provides agent is not open source. On free edition Dialogflow provide some
interaction between the machine and users. Following subsection limited services. Google Dialogflow does not support WhatsApp as
discusses comparative study of Google Dialogflow and IBM Watson target platform. Also it is hard to manage complex project logic. If a
Chatbot system. context is present then Google Dialogflow doesn't block the matching of
an intent. Dialogflow doesn't have interactive UI. With the help of this
1.1. Comparing Google Dialogflow and IBM Watson framework when we create web Chatbot system it does not look
attractive and good. Here Python SDK is not well documented and there
FinancesOnline [19] show comparison between Dialogflow vs IBM are some lack of examples and tutorials. IBM Watson Limitations: IBM
Watson. In this Web page they use unique SmartScore system for Watson is used in some limited areas only. It does not process structure
evaluation purpose. This score is based on nine components as - Main data directly. There is no any relational databases. It having limited
Functionality (20%), Collaboration Features (20%), Customization resources. As compare to others platform this is more expensive
(15%), Integration (15%), and Ease of use (10%), General Impression platforms. At some point IBM Watson requires some complex coding for
(5%), Help & Support (5%), Security (5%), and Mobility (5%). proper integration. Dialogues sometimes take a long time to give
SmartScore system considers IBM Watson framework is better than its respond, they need to be re-configured. If answers is not known then
counterpart. The system assigns total score of 8.6 to Google Dialogflow IBM Watson does not give any response. It gives response only answer
and total score of 9.3 to IBM Watson. At Kommunicate [20], they when the system is 100% sure.
assessed numerous Chatbot stages for different use cases, E.g. client
support and service. From Dzone [21] webpage show comparison 6. Experimental Result
between various Chatbot platform. Globalme language and technology
[22] web page compare Google and IBM to show which Chatbot In this section for analysis purpose we create Chatbot system for
platform is easiest to train. Here they use three key aspects for educational college website.
comparison purpose as Documentation provided, Configuration
Experience, Testing and Interface. As per [22] Google Dialogflow 1.3. Google Dialogflow Result
documentation is better whereas IBM Watson takes less time for
configuration. As per them testing and interface of both systems are We collect college training data for Both Google Dialogflow and IBM
equally good. Most important component of conversation is Watson Chatbot system. In Google Dialogflow first we create
understanding natural language. Medium web page [23] show Dialogflow account and Facebook Developers account. Here we create
comparison between different platforms depends on its NLU features. agent name as RIT_smartchat. In that agent we create more than 15

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 3

Electronic copy available at: https://ssrn.com/abstract=3567047

International Conference on Innovative Computing and Communication (ICICC 2020)
intents and add some entities so it is easy to perform pattern matching. Fig. 2 – (a) Information about college after click on button.

Both fig. 2 (a) and (b) show same result whenever it get different user
input in the format of text or button click. With the help of intent and
entities it is easy to perform pattern matching for this Chatbot.

Fig. 2 – (b) Information about college after users’ text input.

Integrating with Facebook messenger we create Facebook page. Using

Facebook Chatbot can easily interact with user through text, image or
video. Bellow fig. 1 show working of RIT_smartchat on Facebook. Here
Chatbot start conversation with greeting and some information.

Fig. 1 – Welcome page of RIT_smartchat.

Fig. 1 show that how RIT_smartchat welcome to the user. Here there are
some buttons provided or chatbot can get input from user in the form of
text also.

Bellow fig. 2 show information about college and college structure

image. Here user can ask question in different way to the chatbot or can
simply press buttons given in option.

Fig. 3 –Web demo of RIT_smartchat.

Above fig. 3 show web demonstration of RIT_smartchat. This chatbot

give response in textual form only. As compare to Facebook integration
it doesn’t support for any unstructured data. For comparison purpose we
create URL as follow: https://bot.dialogflow.com/RIT_Smartchat.
Whenever Chatbot does not understand user input it give default answer
randomly like: 1. Sorry, could you say that again. 2. Sorry, I didn’t get
that. While integrating with Google assistant it support audio as well as
textual method. Only difference is that it does not show any quick
buttons. Here Google Dialogflow does not give attractive UI.

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 4

Electronic copy available at: https://ssrn.com/abstract=3567047

International Conference on Innovative Computing and Communication (ICICC 2020)
1.4. IBM Watson Result on. After click on option Chatbot give response with the help of text and
image.
We perform comparison of web demo between Google Dialogflow and
Fig. 5–RIT_smartchat on IBM Watson about college.
IBM Watson. In prevision section fig. 3 show that Google Dialogflow
does not give response in images, buttons and so on.
7. Conclusion and Expectations from Future Chatbots
Bellow fig.4 show how Chatbot work on IBM Watson. Whenever
user ask about college in different way this Chatbot also give same
In this paper initially we explained about Chatbot system and its real
answer all times.
time applications. Next, we provide some basic information about
Chatbot development frameworks and publishing platform. There are a
number of dimensions on which these platforms can be compared as
Features, programming languages and many more. From all above
comparison in this paper we conclude that every Chatbot framework
have its unique features. We presented detail study and analysis on
Google Dialogflow and IBM Watson. At present IBM Watson is more
popular than Dialogflow Chatbot system. Expectations from future
Chatbots are given as follow. These expectations gives guidelines to
researchers working in this domain in order to enhance performance and
behaviour of Chatbot system. Currently Chatbots do not support all
languages and are unable to understand informal speech. Hence there is a
good scope for evacuating such language barriers from Chatbots.
Embedding AI helps the computer system to perform just like as a
human. Chatbot with AI makes our Chatbot capable and intelligent to
answer complex queries. With the help of AI Chatbot can improve
current response using previous interaction. Such Chatbot system will
perform smart interactions and will help for saving customer's time. In
coming five years, there is a high probability that about 80% of the
correspondences will be _finished with the assistance of robots. As there
is an expansion in the notoriety of menial helpers like Amazon Alexa
and Google Assistant. In coming years speech recognition technology is
Fig. 4–RIT_smartchat on IBM Watson. really improving and will go on to get enhanced. Voice activated
Fig. 5 show whenever user ask question then chatbot give some option Chatbots are better than text-based Chatbot. It is based on voice. They
like college past and future, organization structure, programmes and so accept command in an oral or written form and respond through voice
like Alexa and Google Home.

REFERENCES

Sameera A, Abdul-Kader, Dr. John Woods (2015).Survey on Chatbot Design

Techniques in Speech Conversation Systems. IJACSA, vol. 6 Issue 7.
Aditya Deshpande, Alisha Shahane, Darshana Gadre, Mrunmayi Deshpande,
Prof. Dr. Prachi M. Joshi (2017).A survey of various chatbot
implementation techniques. Vol. 11.
Prof .Shailaja Pede, Shreya Joshi, Aishwarya Bornare, Akash Ghadge,
Dhiraj Bhujade (2018).A Comparative Study of Chatbot Implementation
Techniques. (IJSRET), ISSN 2278 Volume 7, Issue 10.
Prof. Dr. Ute Schmid (2016).How does IBM Watson understand natural
language.
Paul Greenberg Author (2018). Chatbots conversation for all of us. CRM at
the Speed of Light, 4th Edition; Managing Principal, The 56 Group, LLC.
Rivas, P., Holzmayer, K., Hernandez, C., & Grippaldi, C. (2018). Excitement
and Concerns about Machine Learning-Based Chatbots and Talkbots: A
Survey. IEEE International Symposium on Technology and Society
(ISTAS)

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 5

Electronic copy available at: https://ssrn.com/abstract=3567047

International Conference on Innovative Computing and Communication (ICICC 2020)
The history of Chatbot, https://onlim.com/en/the-history-of-Chatbots/.
Story of ELIZA, The First Chatbot developed in 1966,
https://analyticsindiamag.com/story-eliza- rst-Chatbot-developed-
1966/.
Parry: The AI Chatbot from 1972, https://phrasee.co/parry-the-a-i-
chatterbot- from-1972/.
Dialogflow, https://dialog ow.com/.
Deconstructing Chatbots: Getting Started with Dialogflow,
https://medium.com/google-cloud/deconstructing-Chatbots-getting-
started- with-dialog ow-4f91deb32135.
DialogflowReview-CompareCamp.com,
http://comparecamp.com/dialogflow-review-pricing-pros-cons-
features/.

Dr. Sandeep A. Thorat ,Vishakha D. Jadhav 6

Electronic copy available at: https://ssrn.com/abstract=3567047