Testing Chatbots Using Metamorphic Relations - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2019

Testing Chatbots Using Metamorphic Relations

Josip Bozic
  • Function : Author
  • PersonId : 994797
Franz Wotawa
  • Function : Author
  • PersonId : 994796


Modern-day demands for services often require an availability on a 24/7 basis as well as online accessibility around the globe. For this sake, personalized software systems, called chatbots, are applied. Chatbots offer services, goods or information in natural language. These programs respond to the user in real-time and offer an intuitive and simple interface to interact with. Advantages like these makes them increasingly popular. Chatbots can even act as substitutes for humans for specific purposes. Since the chatbot market is growing, chatbots might outperform and replace classical web applications in the future. For this reason, ensuring correct functionality of chatbots is of high and increasing importance. However, since different implementations and user behavior result in unpredictable results, the chatbot’s output is difficult to predict and classify as well. In fact, testing of chatbots represents a challenge because of the unavailability of a test oracle. In this paper, we introduce a metamorphic testing approach for chatbots. In general, metamorphic testing can be applied to situations where no expected values are available. In addition, we discuss how to obtain test cases for chatbots, i.e. sequences of interactions with a chatbot, in an according manner. We demonstrate our approach using a hotel booking system and discuss first experimental results.
Fichier principal
Vignette du fichier
482770_1_En_3_Chapter.pdf (268.18 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02526355 , version 1 (31-03-2020)





Josip Bozic, Franz Wotawa. Testing Chatbots Using Metamorphic Relations. 31th IFIP International Conference on Testing Software and Systems (ICTSS), Oct 2019, Paris, France. pp.41-55, ⟨10.1007/978-3-030-31280-0_3⟩. ⟨hal-02526355⟩
110 View
214 Download



Gmail Facebook X LinkedIn More