Why build Knowledge graphs
Unstructured data, like text documents and web pages, holds a wealth of valuable information. The challenge is figuring out how to tap into those insights and connect the dots across disparate sources.
Knowledge graphs turn these unstructured data into a structured representations. They map out the key entities, relationships, and patterns – enabling advanced semantic analysis, reasoning, and inference.
DIY
- Extract Nodes and relationships
- Pass the input data to the LLM and let it decide which nodes and relationships to extract.
- Ask the LLM to return the extracted entities in a specific format
- Chunk the input data to overcome context window restrictions
- Clean the data
- Asking LLM to extract nodes and relationships on chunks would lead to having duplicates
- Ask the LLM to merge the data
- Load into Neo4j
- Parse the LLM generated text into a csv file
- Use the Neo4j Data Importer tool to load the data into Neo4j