Importing JSON files

Whether it is the actual input file or the CSV or Excel template, the import API works with data formatted in JSON.

The import API accepts an array of objects where each object represents a command to be executed. The implicit operation for a command is MERGE. A merge creates or updates a resource in Collibra depending on the result of a query based on the parameters provided in the identifier field within the JSON object.

[
  {command 1},
  {command 2},
  ...
]

Order of execution

By default, the importer splits commands into batches. Then, for each batch, the commands are sorted for optimal performance.

If a command depends on the result of a previous command, the previous command has to appear in the input data before the command that depends on it. Commands created from templates, such as DataWeave, should be sorted in the same way.

Although not guaranteed, in some situations, the import may still work even if the order of the commands is not correct.

When you use a single file to import a community and a domain that belongs to it, the command to import the community should appear first. Additionally, if an asset, a domain or a community is created and mapping also needs to be created for a given resource, the mapping command should be second.

[
  {command for Community 1},
  {command for Community 2},
  ...
  {command for Community N},
  {command for Domain 1},
  ...
  {command for Domain M},
  {command for Asset 1},
  ...
  {command for Asset P},
]

Command merging

During the import process, multiple subsequent commands with the same identifier are merged and executed as a single import command. This means that an imported asset has attributes and relations from all merged commands.

Only commands that immediately follow each other can be merged, otherwise the last command in the file can override the results of the previous commands with the same identifier.

For consistent results, use one import command per unique identifier.