publications

2025

1. EMNLP 2025

   Schema Generation for Large Knowledge Graphs Using Large Language Models

   Bohui Zhang, Yuan He, Lydia Pintscher, and 2 more authors

   Conference on Empirical Methods in Natural Language Processing, 2025

   Abs arXiv Code

   Schemas play a vital role in ensuring data quality and supporting usability in the Semantic Web and natural language processing. Traditionally, their creation demands substantial involvement from knowledge engineers and domain experts. Leveraging the impressive capabilities of large language models (LLMs) in tasks like ontology engineering, we explore schema generation using LLMs. To bridge the resource gap, we introduce two datasets: YAGO Schema and Wikidata EntitySchema, along with novel evaluation metrics. The LLM-based pipelines utilize local and global information from knowledge graphs (KGs) to generate schemas in Shape Expressions (ShEx). Experiments demonstrate LLMs’ strong potential in producing high-quality ShEx schemas, paving the way for scalable, automated schema generation for large KGs. Furthermore, our benchmark introduces a new challenge for structured generation, pushing the limits of LLMs on syntactically rich formalisms.

2. SWJ 2025

   Towards Explainable Automated Knowledge Engineering with Human-in-the-loop

   Bohui Zhang, Albert Meroño Peñuela, and Elena Simperl

   Semantic Web, 2025

   Abs DOI Code

   Knowledge graphs (KGs) are crucial in human-centered AI as they provide large labeled machine learning datasets, enhance retrieval-augmented generation, and generate explanations. However, knowledge graph construction has evolved into a complex, semi-automatic process that increasingly relies on black-box deep learning models and heterogeneous data sources to scale. The knowledge graph lifecycle is not transparent, accountability is limited, and there are no accounts of, or methods to determine, how fair a knowledge graph is in downstream applications. KGs are thus at odds with AI regulation, for instance, the EU’s AI Act, and with efforts elsewhere in AI to audit and debias data and algorithms. This paper reports on work towards designing explainable human-in-the-loop knowledge graph construction pipelines. Our work is based on a systematic literature review, in which we study tasks in knowledge graph construction that are often automated, as well as methods to explain how they work and their outcomes, and an interview study with 13 experts from the knowledge engineering community. To analyze the literature, we introduce use cases, related goals for explainable AI (XAI) methods in knowledge graph construction, and the gaps in each use case. To understand the role of XAI models in practice and reveal requirements for improving current methods, we designed interview questions covering broad transparency topics, along with example discussion sessions using examples from the review. From practical knowledge engineering experience, we identify user requirements, propose design blueprints, and outline directions for future research.

3. JoWS 2025

   Knowledge Prompting: How Knowledge Engineers Use Generative AI

   Elisavet Koutsiana, Johanna Walker, Michelle Nwachukwu, and 3 more authors

   Journal of Web Semantics, 2025

   Abs DOI

   Despite many advances in knowledge engineering (KE), challenges remain in areas such as engineering knowledge graphs (KGs) at scale, automating tasks, and keeping pace with evolving domain knowledge. KE has used NLP demonstrating notable advantages in knowledge-intensive tasks, but the most effective use of generative AI to support knowledge engineers across the KE activities is still in its infancy. To explore how generative AI may enhance KE and change existing KE practices, we conducted a multi-method study during a KE hackathon. We investigated participants’ views on the use of generative AI, the challenges they face, the skills they may need to integrate generative AI into their practices, and how they use generative AI responsibly. We found participants felt LLMs could indeed contribute to improving efficiency when engineering KGs, but presented increased challenges around the already complex issues of evaluating KE task success. We discovered prompting to be a useful but undervalued skill for knowledge engineers working with LLMs, and note that NLP skills may become more relevant across more roles in KE workflows. Integrating generative AI into KE tasks needs to be done with awareness of potential risks and harms. Given the limited ethical training most knowledge engineers receive, solutions such as our proposed ’KG Cards’ based on Data Cards could be a useful guide for KG construction. Our findings can support designers of KE AI copilots, KE researchers, and practitioners using advanced AI to develop trustworthy applications, propose new methodologies for KE and operate new technologies responsibly.

2024

1. ESWC 2024

   OntoChat: A Framework for Conversational Ontology Engineering using Language Models

   Bohui Zhang, Valentina Anita Carriero, Katrin Schreiberhuber, and 4 more authors

   European Semantic Web Conference, 2024

   Abs DOI arXiv Code

   Ontology engineering (OE) in large projects poses a number of challenges arising from the heterogeneous backgrounds of the various stakeholders, domain experts, and their complex interactions with ontology designers. This multi-party interaction often creates systematic ambiguities and biases from the elicitation of ontology requirements, which directly affect the design, evaluation and may jeopardise the target reuse. Meanwhile, current OE methodologies strongly rely on manual activities (e.g., interviews, discussion pages). After collecting evidence on the most crucial OE activities, we introduce \textbfOntoChat, a framework for conversational ontology engineering that supports requirement elicitation, analysis, and testing. By interacting with a conversational agent, users can steer the creation of user stories and the extraction of competency questions, while receiving computational support to analyse the overall requirements and test early versions of the resulting ontologies. We evaluate OntoChat by replicating the engineering of the Music Meta Ontology, and collecting preliminary metrics on the effectiveness of each component from users. We release all code at https://github.com/King-s-Knowledge-Graph-Lab/OntoChat.

2023

1. ISWC 2023

   Using Large Language Models for Knowledge Engineering (LLMKE): A Case Study on Wikidata

   Bohui Zhang, Ioannis Reklos, Nitisha Jain, and 2 more authors

   LM-KBC Challenge @ International Semantic Web Conference, 2023

   Abs arXiv Code

   In this work, we explore the use of Large Language Models (LLMs) for knowledge engineering tasks in the context of the ISWC 2023 LM-KBC Challenge. For this task, given subject and relation pairs sourced from Wikidata, we utilize pre-trained LLMs to produce the relevant objects in string format and link them to their respective Wikidata QIDs. We developed a pipeline using LLMs for Knowledge Engineering (LLMKE), combining knowledge probing and Wikidata entity mapping. The method achieved a macro-averaged F1-score of 0.701 across the properties, with the scores varying from 1.00 to 0.328. These results demonstrate that the knowledge of LLMs varies significantly depending on the domain and that further experimentation is required to determine the circumstances under which LLMs can be used for automatic Knowledge Base (e.g., Wikidata) completion and correction. The investigation of the results also suggests the promising contribution of LLMs in collaborative knowledge engineering. LLMKE won Track 2 of the challenge. The implementation is available at https://github.com/bohuizhang/LLMKE

2. HHAI 2023

   Towards Explainable Automatic Knowledge Graph Construction with Human-in-the-loop

   Bohui Zhang, Albert Meroño Peñuela, and Elena Simperl

   International Conference on Hybrid Human-Artificial Intelligence, 2023

   Abs DOI Code

   Knowledge graphs are important in human-centered AI because of their ability to reduce the need for large labelled machine-learning datasets, facilitate transfer learning, and generate explanations. However, knowledge-graph construction has evolved into a complex, semi-automatic process that increasingly relies on opaque deep-learning models and vast collections of heterogeneous data sources to scale. The knowledge-graph lifecycle is not transparent, accountability is limited, and there are no accounts of, or indeed methods to determine, how fair a knowledge graph is in the downstream applications that use it. Knowledge graphs are thus at odds with AI regulation, for instance the EU’s upcoming AI Act, and with ongoing efforts elsewhere in AI to audit and debias data and algorithms. This paper reports on work in progress towards designing explainable (XAI) knowledge-graph construction pipelines with human-in-the-loop and discusses research topics in this space. These were grounded in a systematic literature review, in which we studied tasks in knowledge-graph construction that are often automated, as well as common methods to explain how they work and their outcomes. We identified three directions for future research: (i) tasks in knowledge-graph construction where manual input remains essential and where there may be opportunities for AI assistance; (ii) integrating XAI methods into established knowledge-engineering practices to improve stakeholder experience; as well as (iii) evaluating how effective explanations genuinely are in making knowledge-graph construction more trustworthy.

2022

1. ISWC 2022

   Enriching Wikidata with Linked Open Data

   Bohui Zhang, Filip Ilievski, and Pedro Szekely

   Wikidata Workshop @ International Semantic Web Conference, 2022

   Abs arXiv Code

   Large public knowledge graphs, like Wikidata, contain billions of statements about tens of millions of entities, thus inspiring various use cases to exploit such knowledge graphs. However, practice shows that much of the relevant information that fits users’ needs is still missing in Wikidata, while current linked open data (LOD) tools are not suitable to enrich large graphs like Wikidata. In this paper, we investigate the potential of enriching Wikidata with structured data sources from the LOD cloud. We present a novel workflow that includes gap detection, source selection, schema alignment, and semantic validation. We evaluate our enrichment method with two complementary LOD sources: a noisy source with broad coverage, DBpedia, and a manually curated source with a narrow focus on the art domain, Getty. Our experiments show that our workflow can enrich Wikidata with millions of novel statements from external LOD sources with high quality. Property alignment and data quality are key challenges, whereas entity alignment and source selection are well-supported by existing Wikidata mechanisms. We make our code and data available to support future work.