Querying the Depths: Unveiling the Strengths and Struggles of Large Language Models in SPARQL Generation

Adrián Ghajari, Salvador Ros, Álvaro Pérez

Resumen


The emergence of the Semantic Web has precipitated a proliferation of structured data manifested in the form of knowledge graphs, underscoring the imperative of natural language interfaces to enhance accessibility to these repositories of information. The capacity to articulate queries in natural language and subsequently retrieve data through SPARQL queries assumes paramount importance. In the present investigation, we have scrutinized the efficacy of in-context learning based on an agent-based architecture in facilitating the construction of SPARQL queries. Contrary to initial expectations, the augmentation of in-context learning prompts through agent-based mechanisms has been found to diminish the efficacy of Language Model-based Systems (LLMS), as it is perceived as extraneous "noise," thereby delineating the constraints inherent in this approach. The results highlight the need to delve deeper into the intricacies of model training and fine-tuning, focusing on the relational aspects of ontology schemas.

Texto completo:

PDF