CyberBOT: Towards Reliable Cybersecurity Education via Ontology-Grounded Retrieval Augmented Generation
Project Overview
The document discusses the innovative use of generative AI in education through the introduction of CyberBOT, an ontology-grounded question-answering system tailored for cybersecurity education. Utilizing retrieval-augmented generation (RAG) techniques alongside a specialized ontology, CyberBOT aims to deliver reliable and contextually relevant responses to students' inquiries. Deployed in a graduate-level course, the system enhances the learning experience by providing personalized and validated answers, which can significantly improve educational outcomes. However, the initial evaluations also point out several challenges, including issues related to accuracy, the scalability of deployment, and the dependence on curated knowledge bases. Overall, the integration of generative AI in educational contexts like CyberBOT demonstrates promising potential, while also necessitating careful consideration of its limitations and areas for improvement.
Key Applications
CyberBOT - a question-answering chatbot
Context: Graduate-level course (CSE 546: Cloud Computing) at Arizona State University with over 100 students
Implementation: Integrated as a web-based platform for students to submit queries and receive validated responses.
Outcomes: Enhanced reliability and trustworthiness of responses, improved learning engagement, and support for self-paced inquiry.
Challenges: Dependence on the quality of the knowledge base and ontology, potential for incomplete coverage of rapidly evolving cybersecurity threats.
Implementation Barriers
Technical limitations
The system's accuracy is dependent on the quality and coverage of its curated knowledge base and ontology, which may not be comprehensive or up-to-date.
Proposed Solutions: Continuous updates of the ontology and knowledge base, and the development of robust methods to address out-of-scope queries.
Scalability challenges
The computational overhead of real-time retrieval and validation may pose practical challenges for broader adoption across larger student cohorts.
Proposed Solutions: Exploring resource-efficient deployment solutions to handle larger scales without compromising performance.
Project Team
Chengshuai Zhao
Researcher
Riccardo De Maria
Researcher
Tharindu Kumarage
Researcher
Kumar Satvik Chaudhary
Researcher
Garima Agrawal
Researcher
Yiwen Li
Researcher
Jongchan Park
Researcher
Yuli Deng
Researcher
Ying-Chih Chen
Researcher
Huan Liu
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Chengshuai Zhao, Riccardo De Maria, Tharindu Kumarage, Kumar Satvik Chaudhary, Garima Agrawal, Yiwen Li, Jongchan Park, Yuli Deng, Ying-Chih Chen, Huan Liu
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai