Collibra Connect deployment

In the following schema, you see a typical deployment of Collibra Connect, Collibra Data Governance Center and any external system.

Collibra Connect consists of two major components:

  • Integration flow, often referred to as (integration) templates, defining the mapping between Collibra DGC and the external system.
  • Connector which communicates with the Collibra DGC REST API.

Collibra Connect linking an external system with Collibra DGC

In the schema, you can distinguish the following elements:

Element Description
1

Collibra Data Governance Center
2

Collibra Connect.

In the production environment, it is recommended to install the Mule ESB (Collibra Connect) server on a separate machine (physical or virtual). This ensures that Collibra DGC and Collibra Connect do not share memory resources or CPU processing power. Additionally, the port on the Collibra Connect server to which Collibra Connect has to listen, must be open and reachable by the Collibra DGC server, to allow the triggering of integration flows. You can decide and configure which port that is, for example, 443 for standard HTTPS.
3

Collibra Connect integration templates communicate with Collibra DGC, using the Collibra DGC Connector. The communication can go over HTTP (default port 80) or HTTPS (default port 443).

The Collibra DGC Connector encapsulates the REST API calls to Collibra DGC to provide session management support and streamline operations. It is also possible for Collibra DGC workflows to communicate (start integration flows or receive BPMN Message Events) with Collibra Connect, see Connecting to the gateway from a workflow.
4

Integration flows.

Each integration flow performs integration logic (migration, broadcast ...) between Collibra DGC and one or more external systems. You can find the documentation and download integration templates on the delivery site.
5

External systems that are to be integrated with Collibra DGC. The communication with the external system can happen through:

  • A custom connector to the external system; for example IBM BG Connector.
  • A REST API of the external system.
  • A Relational Database connector to connect to a relation database.
  • An other file-based connector, such as FTP, to read a file from a remote location.
  • Any other method that is allowed by the system.

In general, you can integrate any system with accessible APIs with Collibra DGC through Collibra Connect.
6

Starting the integration.

There are numerous ways of starting the integration automatically:

  • An integration starts periodically with a configurable frequency.
  • The integration starts automatically after a change in Collibra DGC or in the external system:
    • A workflow in Collibra DGC is started automatically after a workflow-starting event, for example when an asset is added or modified. In turn, the started workflow triggers the required integration flow.
    • With a built-in mechanism, an integration flow periodically checks the external system. If it detects a change, it starts the actual integration flow.
    • An external system itself triggers the call to start the required integration flow when necessary. This is only possible if the external system can execute a REST call or register an event callback.
  • A user starts a workflow in Collibra DGC that in turn calls the integration flow in Collibra Connect.
  • A workflow in Collibra DGC calls the integration flow in a recurring interval.
  • A user triggers the integration flow manually when the most up-to-date view of the data is required.
    The user can do this by sending the trigger request (REST call) to the Gateway. This request can be made from any computer, using tools like cURL or POSTMAN.

7

Deployment of integration flows. The ARM agent fetches the integration flows from the Anypoint Runtime Manager and deploys them on the Mule ESB Standalone server.

The communication between ARM and the ARM agent is a REST API call and is highly recommended to be secure, for example by using SSL.