Date
AI Overview
Why it matters
Academic theses contain valuable knowledge, but access is often hindered by inefficient physical libraries and digital repositories. Creating conversational systems can optimize search time and facilitate natural access to this information.
Study at a glance
- Question: How effectively can an AI conversational system answer natural language questions about university geology theses?
- Setting and data: Universidad Central del Ecuador, 244 geology undergraduate thesis records up to December 2024, surveyed 55 potential users.
- Method: System development of a RAG-SQL chatbot, followed by a potential user survey and benchmark text evaluation (BLEU metric).
- AI tools: Geolog-IA, Llama 3.1, Gemini 2.5 Flash Lite
Key findings
- Geolog-IA achieved an average BLEU score of 0.87 when comparing its generated answers to actual database query results.
- A survey of 55 potential university users revealed that most viewed the conversational system favorably and highlighted its utility for finding academic information.
- The structured database approach allowed the system to successfully answer both quantitative queries and descriptive, qualitative questions based on thesis summaries.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
This study presents the development of Geolog-IA, a novel conversational
system based on artificial intelligence that responds naturally to questions
about geology theses from the Central University of Ecuador. Our proposal uses
the Llama 3.1 and Gemini 2.5 language models, which are complemented by a
Retrieval Augmented Generation (RAG) architecture and an SQLite database. This
strategy allows us to overcome problems such as hallucinations and outdated
knowledge. The evaluation of Geolog-IA's performance with the BLEU metric
reaches an average of 0.87, indicating high consistency and accuracy in the
responses generated. The system offers an intuitive, web-based interface that
facilitates interaction and information retrieval for directors, teachers,
students, and administrative staff at the institution. This tool can be a key
support in education, training, and research and establishes a basis for future
applications in other disciplines.
Research Categories
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
AI tools
Publication Source
Publisher
arXiv
