Quality over Quantity: Ontology Dataset Selection for Efficient Triple Generation in LLMs

Resumen

The integration of structured semantic knowledge into Large Language Models (LLMs) is increasingly relevant for knowledge-intensive Natural Language Processing tasks. Ontologies play a central role in organizing this knowledge, yet their manual construction and maintenance become inefficient as digital information grows. While LLMs are increasingly used for ontology learning, their ability to generate coherent structural knowledge remains limited, particularly in small open-source models needed for efficient and sustainable deployment. This work investigates a controlled ontology-to-ontology generation setup to evaluate LLMs’ structural generalization abilities and introduces a data-centric continual pretraining methodology based on ontology quality. Two open-source models, LLaMA 3.2-1B and GEMMA 3-1B, are trained and evaluated using structural and formatting criteria. Results show that selecting only high-quality ontology data substantially improves triple generation while reducing training time by an average of 67% for 1 epoch.