Use Apache Jena Fuseki as Semantic Triple Store for T0 Base Model Storage¶

Context and Problem Statement¶

The platform must ingest and manage a massive Common Information Model (CIM) grid model containing approximately 40 million triples (~3GB) at time \(T_0\) (Base Model). This core dataset forms the foundational semantic truth of the power network, representing structured electrical topologies across multiple profiles (such as Equipment, Topology, Steady State Hypothesis, and Geographical profiles) defined by the IEC 61970 standard.

The primary challenge is finding a database system that natively handles standard W3C Semantic Web specifications, enables parallel multi-profile isolated transactional ingestion without locking issues, supports complex declarative multi-graph merging operations, and provides standard declarative validation capabilities natively.

How can we store the initial \(T_0\) complete grid model while respecting native CIM semantics, ensuring full transactional isolation across sub-profiles, and permitting rapid, cross-profile schema validation?

Decision Drivers¶

• Native CIM Specification Alignment: Strict adherence to RDF/XML, JSON-LD, or Turtle serialization formats natively defined by the IEC 61970 standard.
• Ingestion and Scaling Performance: Ability to handle rapid stream-based data loading of millions of triples simultaneously without experiencing high memory foot-prints or lockouts.
• Multi-Profile Graphical Separation: Requirement to store each profile (EQ, TP, SSH, G, etc.) into its own transactionally isolated drawer or graph partition, merging them seamlessly later.
• Native Semantic Validation: Direct support for declarative validation standards such as SHACL (Shapes Constraint Language) to enforce rules across interconnected grid structures.
• Query and Schema Flexibility: Capability to execute cross-profile virtual queries and relationships without altering underlying physical data or causing performance degradation.

Considered Options¶

• Option 1: Semantic Triple Store (Apache Jena Fuseki with TDB2 engine)
• Option 2: Property Graph Database (e.g., Neo4j / FalkorDB)
• Option 3: Relational Database with Hybrid Document/JSON capabilities (e.g., PostgreSQL with JSONB)

Decision Outcome¶

Chosen option: "Option 1: Semantic Triple Store (Apache Jena Fuseki with TDB2 engine)", because it provides native support for the W3C RDF standards underlying the CIM specification, offers out-of-the-box transactionally isolated Named Graphs, and natively supports dynamic SPARQL merging along with complete cross-profile SHACL validation. This guarantees that the system's "Source of Truth" is directly compliant with standard semantic web tooling without requiring custom object-graph translation mappings.

Consequences¶

• Good: Eliminates the need for an intermediate translation or mapping layer, allowing the dotnetRDF streaming parser to dump incoming triples directly via GZIP compressed HTTP streams.
• Good: Native support for Named Graphs permits fully concurrent, transactionally isolated ACID writes across different CIM profiles without table or database level locks.
• Good: Cross-profile validations are cleanly written via standard SHACL rules (.ttl files) and executed directly on a virtualized unified master graph.
• Bad: Raw analytical and real-time operational simulation traversal speeds are slower compared to memory-optimized graph matrices, requiring a downstream projection layer (e.g., FalkorDB + Microsoft Orleans).

Confirmation¶

The compliance with this ADR will be validated through automated performance tests executing the dotnetRDF streaming classification pipeline, verifying that all incoming profile chunks are processed asynchronously via Microsoft Orleans clusters and correctly committed to distinct graph drawers inside Apache Jena Fuseki. Additionally, successful validation reports generated by cross-profile SHACL execution on the graph://models/base/t0/master path will act as the system's official stamp of a healthy base model genesis.

Pros and Cons of the Options¶

Option 1: Semantic Triple Store (Apache Jena Fuseki with TDB2 engine)¶

An open-source Java framework for building Semantic Web and Linked Data applications that provides a robust SPARQL server along with a transactional, high-performance triple store engine (TDB2).

• Good, because it natively stores data as semantic subject-predicate-object triples, eliminating any impedance mismatch with the standard CIM RDF structures.
• Good, because it features native HTTP endpoints that support Content-Encoding: gzip, allowing an Orleans actor cluster to stream zipped data into the engine, unpacking on-the-fly and drastically cutting down I/O bottlenecking.
• Good, because it supports Named Graphs, letting each profile (EQ, TP, SSH, G, etc.) be cleanly written into isolated graph drawers simultaneously without blocking each other.
• Good, because complex merge scenarios can be easily resolved using a dynamic query factory that builds standard SPARQL UNION statements to dynamically construct a single virtualized master graph view.
• Good, because it has built-in engines for SHACL rules validation to catch invalid, unmapped, or loose semantic links across profiles.
• Bad, because graph traversal algorithms over multi-million nodes on disk are not optimized for sub-millisecond real-time operational simulations.

Option 2: Property Graph Database (e.g., Neo4j / FalkorDB)¶

A database system that uses graph structures with nodes, edges, and properties to represent and store highly connected transactional or analytical data.

• Good, because graph traversals and topology matrix manipulations are exceptionally fast, making it ideal for operational calculations, short-circuit calculations, and real-time path discovery.
• Good, because data structures match physical network graphs (substations, lines, breakers, transformers) intuitively.
• Bad, because it does not natively understand W3C Semantic Web standards (RDF/XML, Triples, Namespaces), requiring a complex ETL translation service to map every incoming triple statement into nodes, relations, and property attributes.
• Bad, because partitioning the database into isolated, transient sub-profiles while maintaining deep relationships is highly complex and error-prone without native Named Graph abstractions.
• Bad, because there is no native support for executing standard SHACL constraints, meaning validation logic must be manually coded in custom validation scripts or custom query expressions.

Option 3: Relational Database with Hybrid Document/JSON capabilities (e.g., PostgreSQL with JSONB)¶

A classic object-relational database management system that supports both structured relational schemas and advanced unstructured/semi-structured binary JSON objects.

• Good, because it is a highly mature technology with advanced transaction management, widespread ecosystem support, and exceptional indexing features.
• Good, because it is incredibly easy to store profile definitions and general metadata tables alongside the raw ingested network strings.
• Bad, because representing deep hierarchical structures and dynamic many-to-many topological electrical graph links requires heavy join operations or expensive JSONB attribute lookups.
• Bad, because standard CIM relationships spanning across multiple separate tables or JSON blocks lack native type and structural enforcement out of the box.
• Bad, because it possesses zero out-of-the-box knowledge of semantic graph definitions, SPARQL query protocols, or SHACL schema alignment, forcing the engineering team to construct an expansive, custom-made orchestration and validation subsystem.

More Information¶

Naming and Path Convention Summary¶

To support multi-scenario planning, multiple active grid versions, and isolated what-if simulation capabilities, every ingested base model is assigned a unique {ModelId} (e.g., a UUID or structured version code such as base-v2026-q1). The storage addresses inside the Triple Store must strictly adhere to the following parameterized URI convention:

Related Decisions¶

• Downstream Multi-Model Projection: Downstream projection mechanics must be scheduled immediately following a successful SHACL validation report for a specific {ModelId}. The system will flatten and synchronize the validated data model into the operational cache (FalkorDB + Microsoft Orleans Cluster) and the read model index (Elasticsearch), appending the {ModelId} as a partition key or state prefix. This ensures the Microsoft Orleans cluster can isolate and simulate multiple independent grid networks concurrently.
• Parameterized Ledger Event Streams: KurrentDB (EventStoreDB) must bypass generic, non-versioned stream addresses. Instead, it must open a dedicated, isolated event stream formatted as cim-base-stream-{ModelId} immediately upon the completion of the ingestion flow. This stream will record a secure, immutable BaseModelImported genesis event ledger log, embedding the unique {ModelId}, importation Timestamp, and provisioning metadata into the event payload to prevent cross-contamination of historical base data.