Assessing and Improving Knowledge Graph Quality in GraphRAG applications
A Talk by Panos Alexopoulos (Founder and Principal Educator, OWLTECH)
About this Talk
The rapid rise of Graph Retrieval-Augmented Generation (Graph RAG) has introduced an innovative approach to grounding Large Language Models (LLMs) with factual information from knowledge graphs (KGs). This is an approach that has gained significant traction due to its ability to enhance the factual accuracy and contextual relevance of LLM outputs better than document-based RAG.
However, many of the knowledge graphs employed in Graph RAG applications are automatically constructed using LLM-based techniques and, quite often, lack the necessary accuracy, completeness, and consistency for reliable knowledge grounding.
This masterclass will focus on the critical issue of knowledge graph quality for Graph RAG applications and will teach participants practical methods and techniques for assessing graph quality and identifying gaps and inaccuracies that prevent them from better supporting LLMs in generating accurate and reliable responses.
Key Topics
- Graph Retrieval Augmented Generation
- Knowledge Graph Quality
Target Audience
- Data scientists and engineers involved in Knowledge Graph construction and management.
- Professionals interested in leveraging Knowledge Graphs for enhancing the quality and accuracy of language model outputs.
- Developers and architects designing retrieval-augmented systems that rely on high-quality factual grounding.
Goals
By the end of the masterclass, participants will be able to:
- Understand the importance of high-quality Knowledge Graphs in Graph RAG applications and their role in grounding LLM outputs.
- Identify common challenges in LLM-based Knowledge Graph construction, including issues of accuracy, completeness, and relevance.
- Apply methods and metrics for assessing the quality of Knowledge Graphs, including error detection, validation, and refinement techniques.
- Implement best practices and strategies to improve Knowledge Graph quality, enhancing the overall performance of Graph RAG applications.
Session outline:
Introduction to Graph RAG and Knowledge Graphs (20 minutes)
- Brief introduction to knowledge graphs
- Overview of Graph RAG and its role in grounding LLMs with factual data.
- The importance of high-quality knowledge graphs in Graph RAG applications.
- Common issues with LLM-based knowledge graph construction.
Understanding Knowledge Graph Quality (20 mins)
- Key quality dimensions
- Common quality Issues and causes
Assessing Knowledge Graph Quality (50 mins)
- Quality assessment techniques
- Error detection and validation
- Interactive case study: evaluating and debugging an LLM-derived knowledge graph
Improving Knowledge Graph Quality (20 mins)
- Improving the knowledge graph
- Improving the knowledge graph construction process
Q&A and Wrap-up (10 mins)
- Summary of key takeaways.
- Questions and discussions on specific use cases or challenges.
Format
Lecture-based, with a coding-based case study
Level
Beginner - Intermediate
Prerequisite Knowledge
Prior exposure to RAG applications and/or knowledge graphs is desirable but not necessary
About The Speakers
We’re a cloud tech company that provides organisations around the world with computing infrastructure and software to help them innovate, unlock efficiencies and become more effective. We also created the world’s first – and only – autonomous database to help organise and secure our customers’ data.
