Importing CSV data

In this step, the external system is queried to retrieve the data. This can be done through an SQL connector, a custom connector or something similar.

Sometimes it is useful to create a Java Class (POJO), so that the retrieved data is instantiated as Java Objects. That makes it easier to test and map the data to the Collibra DGC format in future steps. It also improves the robustness of the solution.

This is the most complex step. In this step:

1. The unique ID of the external asset is identified.
2. The mapping between the external asset and the Collibra DGC asset is created or updated.
3. The asset is created.
4. The two Delete cases are handled, either on the Collibra DGC side or on the external system side.

For more information about enriching assets with mapping, see Enriching assets with mapping

In this step, the external data is transformed to the format that Collibra DGC understands. Usually a CSV format is used, because this is the most performing way to update the assets in Collibra DGC. The import only updates the existing assets in Collibra DGC and does not create new assets. This is necessary to cope with the following scenarios: Move assets in Collibra DGC and Update asset in the external system.

For more information about converting data to CSV format, see Converting data to CSV.