Connector Data Process Flow

The connector data process flow describes the key stages executed within a NEXUS connector when data is submitted by a client over REST or when the connector is triggered on a scheduled basis.

Process Flow Overview

The process flow consists of a series of steps which prepare, validate, structure and submit data into NEXUS:

Explanation of Steps

Source

The Source step is where the connector either receives data directly or is triggered to retrieve data from various configurable data sources. The connector collects this data into what is referred to as an in-memory data lake. Once data acquisition has completed successfully, the data is passed to the next step in the process.

Classification

The Classification step performs data identification. The connector inspects each data item as a structured object, analysing data types and values and applying validation rule sets. This process determines the corresponding NEXUS item type for each piece of data.

Merge

The Merge step is an optional normalisation stage. In cases where clients are unable to provide de-duplicated data, this step allows multiple instances of the same entity to be merged. This ensures that parent entities, or entities that should only be processed once, are handled correctly.

Hierarchy

The Hierarchy step resolves relationships within the data as they should exist in NEXUS. Each data item is associated with a NEXUS entity, and this step defines the relational structure and processing order used when submitting data through the IC-WEB interface. Based on the classification results, the connector also determines the correct NEXUS item type and position within the hierarchy.

Target

The Target step processes the hierarchical data from the previous step and submits each classified data item to the appropriate location within NEXUS.