TY - GEN
T1 - Vision powered conversational AI for easy human dialogue systems
AU - Basnyat, Bipendra
AU - Singh, Neha
AU - Roy, Nirmalya
AU - Gangopadhyay, Aryya
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/12
Y1 - 2020/12
N2 - In this paper, we propose an end to end goal-oriented conversational AI agent that can provide contextual information from a potential hazard site. We posit the conversational agent as a FloodBot capable of seeing, sensing, assessing hazard condition, and ultimately conversing about them. We present our domain-specific FloodBot design-solution and learning-experience from the real-time deployment in a flash flood devastated city that uses state-of-the-art deep learning models. We specifically used computer vision and pertinent natural language processing technologies to empower the conversation power of the FloodBot. To deliver such practical and usable AI, we chain multiple deep learning frameworks and create a human-friendly question-answer based dialogue system. We present our deployment details from the last five months and validate the results using ongoing COVID19's impact on the area as well.
AB - In this paper, we propose an end to end goal-oriented conversational AI agent that can provide contextual information from a potential hazard site. We posit the conversational agent as a FloodBot capable of seeing, sensing, assessing hazard condition, and ultimately conversing about them. We present our domain-specific FloodBot design-solution and learning-experience from the real-time deployment in a flash flood devastated city that uses state-of-the-art deep learning models. We specifically used computer vision and pertinent natural language processing technologies to empower the conversation power of the FloodBot. To deliver such practical and usable AI, we chain multiple deep learning frameworks and create a human-friendly question-answer based dialogue system. We present our deployment details from the last five months and validate the results using ongoing COVID19's impact on the area as well.
KW - Chatbot
KW - Computer Vision
KW - Deep Learning
KW - Mobile computing
KW - Natural Language Processing
UR - https://www.scopus.com/pages/publications/85102171761
U2 - 10.1109/MASS50613.2020.00088
DO - 10.1109/MASS50613.2020.00088
M3 - Conference contribution
AN - SCOPUS:85102171761
T3 - Proceedings - 2020 IEEE 17th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2020
SP - 684
EP - 692
BT - Proceedings - 2020 IEEE 17th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 17th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2020
Y2 - 10 December 2020 through 13 December 2020
ER -