LOT4KG

Knowledge Graph Implementation. The knowledge graph implementation activity aims to construct and validate the knowledge graph. It is composed by a set of sub-activities that transform the input data, which can be of any type and format (e.g., tabular in CSV, text in PDF, etc.), into the knowledge graph and its later validation over a set of constraints. The output of this activity is the implemented knowledge graph (virtual or materialized) and the associated rules for constructing (e.g., RML, SPARQL-Anything, etc.) and validating it (e.g., ShEx or SHACL shapes). The latter may also include a validation report as output.

• Recommended reads:
  • R2RML: RDB to RDF Mapping Language
  • RML: RDF Mapping Language
  • SHACL: Shapes Constraint Language

Recommended Tools (based on the survey):

• Data Preparation:
  • Triply ETL - TriplyETL allows you to create and maintain production-grade linked data pipelines.
  • Pandas - A fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Oxigraph - A graph database implementing the SPARQL standard.
  • Dremio - A lakehouse built natively on Apache Iceberg, Polaris, and Arrow - providing flexibility, preventing lock-in, and enabling community-driven innovation.
  • DBT - An open-source command line tool that helps analysts and engineers transform data in their warehouse more effectively.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
  • Open Refine - A powerful free, open source tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.
  • Cline - A fully collaborative AI partner that's open source, fully extensible, and designed to amplify developer impact.
  • GPT - A a generative artificial intelligence chatbot.
• Mapping Generation:
  • YARRRML - A human readable text-based representation for declarative Linked Data generation rules.
  • YARRRML Parser - A library allows to convert YARRRML rules to RML or R2RML rules.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Cline - A fully collaborative AI partner that's open source, fully extensible, and designed to amplify developer impact.
  • Yatter - A tool for translating mapping rules from YARRRML in a turtle-based serialization of RML or R2RML.
  • RML - A generic mapping language, based on and extending R2RML
  • Chimera - A framework implemented on top of Apache Camel offering components to define schema and data transformation pipelines based on Semantic Web solutions.
  • SPARQL Anything - A system for Semantic Web re-engineering that allows users to query anything with SPARQL.
  • ShExML - A language based on ShEx to map and merge heterogeneous data sources.
  • Ontopic Studio - An environment for building knowledge graphs from relational data.
  • Tiny RML - An implementation of a subset of RML and R2RML with some helpful extended features.
  • R2RML-F - An R2RML Implementation.
  • Pandas - A fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language.
  • RDF4J - A powerful Java framework for processing and handling RDF data.
  • GPT - A a generative artificial intelligence chatbot.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
• Data Transformation:
  • RMLMapper - A tool that executes RML rules to generate Linked Data.
  • Morph-KGC - An engine that constructs RDF knowledge graphs from heterogeneous data sources with the R2RML and RML mapping languages.
  • Chimera - A framework implemented on top of Apache Camel offering components to define schema and data transformation pipelines based on Semantic Web solutions.
  • SPARQL Anything - A system for Semantic Web re-engineering that allows users to query anything with SPARQL.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • py-SUbyT - A Python library for Semantic Uplifting by Templates.
  • Morph-xR2RML - An implementation of the xR2RML mapping language that enables the description of mappings from relational or non relational databases to RDF.
  • SPARQL Micro-Services - An architecture that enables querying Web APIs with SPARQL, as well as assigning dereferenceable URIs to Web API resources that do not have a URI in the first place.
  • Jena - A free and open source Java framework for building Semantic Web and Linked Data applications.
  • HDT - A a compact data structure and binary serialization format for RDF.
  • Cline - A fully collaborative AI partner that's open source, fully extensible, and designed to amplify developer impact.
  • GPT - A a generative artificial intelligence chatbot.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
• Constraint Generation:
  • SPARQL - A language for querying RDF data.
  • SHACL - A language for validating RDF graphs against a set of conditions.
  • Astrea - A tool that produces the SHACL shape that can be infered from one, or more, ontologies.
  • Python - A programming language that lets you work quickly and integrate systems more effectively.
  • ShExML - A language based on ShEx to map and merge heterogeneous data sources.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Metaphactory - Transforms your data into consumable, contextual & actionable knowledge and drives continuous decision intelligence using knowledge graphs and AI.
  • Cline - A fully collaborative AI partner that's open source, fully extensible, and designed to amplify developer impact.
  • GPT - A a generative artificial intelligence chatbot.
• Data Validation:
  • Jena SHACL - An implementation of the W3C Shapes Constraint Language (SHACL).
  • pySHACL - A Python validator for SHACL.
  • GPT - A a generative artificial intelligence chatbot.
  • Cline - A fully collaborative AI partner that's open source, fully extensible, and designed to amplify developer impact.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.

Knowledge Graph Publication. The high-level KG publication activity is a counterpart to the ontology publication activity. It captures the necessary tasks and steps which are taken to document and make the KG available online. The output of this activity is the documented KG and the online accessible resource. The lower-level activities part of the KG publication are documentation and data publication. During the documentation step, the mappings, RDF data, SHACL shapes and validation report are used to document the process and output of the implementation. The output is the HTML documentation, which can then be published during the data publication step alongside the online KG.

• Recommended reads:
  • OBA: An ontology-based framework for creating REST APIs for knowledge graph
  • A survey of RDF stores & SPARQL engines for querying knowledge graphs
  • RMLdoc: Human-readable documentation for RML Mappings

Recommended Tools (based on the survey):

• Documentation:
  • Github - A proprietary developer platform that allows developers to create, store, manage, and share their code.
  • GitLab - A comprehensive AI-powered DevSecOps Platform.
  • Zenodo - A general-purpose open repository.
  • Confluence - A web-based corporate wiki .
  • Metaphactory - Transforms your data into consumable, contextual & actionable knowledge and drives continuous decision intelligence using knowledge graphs and AI.
  • Markdown - A lightweight markup language for creating formatted text using a plain-text editor.
• Publication:
  • Triply DB Data Stories - A way of communicating information about your linked data along with explanatory text while also being able to integrate query results.
  • Github - A proprietary developer platform that allows developers to create, store, manage, and share their code.
  • GitLab - A comprehensive AI-powered DevSecOps Platform.
  • Zenodo - A general-purpose open repository.
  • Virtuoso - A database function within the Virtuoso database system specifically designed for managing and querying RDF data.
  • GraphDB - An enterprise ready Semantic Graph Database.
  • W3id - A secure, permanent URL re-direction service for Web applications.
  • GraphQL - A data query and manipulation language that allows specifying what data is to be retrieved or modified.
  • brwsr - A Lightweight Linked Data Browser.
  • Jena - A free and open source Java framework for building Semantic Web and Linked Data applications.
  • Metaphactory - Transforms your data into consumable, contextual & actionable knowledge and drives continuous decision intelligence using knowledge graphs and AI.
  • Ontodia - A JavaScript library that allows to visualize, navigate and explore data in the form of an interactive graph based on underlying data sources.

Knowledge Graph Maintenance. This task is modelled along the same lines as the ontology maintenance task. It is specifically aimed at fixing bugs in the already published KG. This step does not capture proper evolution, the changes within the ontology or the data sources. Therefore, the detailed activity is named bug detection. The output of the activity is the issues and bugs to be fixed by backtracking in the process. The new data requirements task refers to updates in the input data sources that trigger ontology and/or KG update activities.

Recommended Tools (based on the survey):

• Bug Detection:
  • Git - A distributed version control system that tracks versions of files.
  • SPARQL - A language for querying RDF data.
  • GRLC - Builds a web API from SPARQL queries hosted on GitHub to accessing triple store data.
  • Apache airflow - A web-based corporate wiki .
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • SHACL - A language for validating RDF graphs against a set of conditions.

Change Detection and Impact Analysis Step. To be able to update an already existing KG, the changes applied to the ontology need to be examined and analysed against the KG. We define two sub-activities: detect delta (optional) and assess change impact. The main output of this activity is a list of relevant changes, relevant for the update of the KG.

• Recommended reads:
  • COnto–Diff: generation of complex evolution mappings for life science ontologies

Recommended Tools (based on the survey):

• Delta Detection:
  • Git - A distributed version control system that tracks versions of files.
  • UNIX Diff - A language for querying RDF data.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.

Knowledge Graph Update. This activity, just as the one for updating the ontology, serves as a mirror to the KG implementation activity. Its purpose is to bring the KG up to date with regards to any types of changes in the ontology or source data. Hence, the KG update activity can be triggered from the change detection activities which provide the list of ontology changes, or from changes to the source data, depicted by the arrow connecting the KG maintenance activity with KG update. The KG update activity has four sub-activities that make it possible to update the KG, each of them associated with the corresponding assets of the KG implementation (mappings, RDF graph, constraints and validation report). The high-level output is the updated RDF graph and its associated assets (i.e. mappings, data constraints, and validation report), which are to be published using the KG publishing activity.

• Recommended reads:
  • Propagating Ontology Changes to Declarative Mappings in Construction of Knowledge Graphs
  • IncRML: Incremental Knowledge Graph Construction from Heterogeneous Data Sources

Recommended Tools (based on the survey):

• Mappings Update:
  • YARRRML - A human readable text-based representation for declarative Linked Data generation rules.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Yatter - A tool for translating mapping rules from YARRRML in a turtle-based serialization of RML or R2RML.
• Transformation:
  • Morph-KGC - An engine that constructs RDF knowledge graphs from heterogeneous data sources with the R2RML and RML mapping languages.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
• Constraints Update:
  • SHACL - A language for validating RDF graphs against a set of conditions.
  • RDFlib - A Python library for working with RDF, a simple yet powerful language for representing information.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.
• Validation:
  • GRLC - Builds a web API from SPARQL queries hosted on GitHub to accessing triple store data.
  • SPARQL - A language for querying RDF data.
  • SHACL - A language for validating RDF graphs against a set of conditions.
  • pySHACL - A Python validator for SHACL.
  • Metaphacts ETL Pipeline - Provides a means to convert structured data to RDF, perform post-processing steps, and ingest it into a graph database.