> For the complete documentation index, see [llms.txt](https://developer.collibra.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.collibra.com/tutorials/collibra-connect-upsert-operation.md).

# Collibra Connect upsert operation

{% hint style="warning" %}
Collibra Connect is End of Life and therefore no longer supported. This documentation is no longer maintained and is likely out of date, but it remains available for reference purposes.
{% endhint %}

This tutorial contains instructions on how to add data from a CSV file to Collibra Data Governance Center (Collibra DGC) by performing an upsert operation using Collibra Connect. An upsert is an operation where a resource is either updated if it already exists in Collibra DGC or created if it does not exist.

### Prerequisites

* Access to a Collibra DGC environment
* You have [installed](/tutorials/getting-started-with-collibra-connect.md#installation) MuleSoft Anypoint Studio with the CollibraDGC connector.

### Prepare the CSV file

The source of the upsert operation is a CSV file. We are using a short list of acronyms in this example as sample data.

For the operation to succeed you have to provide the following mandatory elements.

To create a new business term:

* The business term name.
* The domain the business terms belongs to.

  <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The domain can be identified by name or the universally unique identifier (UUID).</p></div>

If the specified domain does not exit, it will be created and you have to provide the following mandatory elements.

To create a new domain:

* The domain name.
* The domain type.
* The community the domain belongs to.

  <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The domain can be identified by name or UUID.</p></div>

If the specified community does not exist, it will be created and you have to provide:

* The community name.

The following table and text show the structure of the sample file:

| Name | Domain             | Domain Type | Community               |
| ---- | ------------------ | ----------- | ----------------------- |
| DGC  | New Business Terms | Glossary    | Data Governance Council |
| FAQ  | New Business Terms | Glossary    | Data Governance Council |
| GDPR | New Business Terms | Glossary    | Data Governance Council |

```
Name,Domain,Domain Type,Community
FAQ,New Business Terms,Glossary,Data Governance Council
DGC,New Business Terms,Glossary,Data Governance Council
GDPR,New Business Terms,Glossary,Data Governance Council
```

{% hint style="success" %}
Save the above as a CSV file to use later in the project
{% endhint %}

### Create a MuleSoft project

Open Anypoint Studio and create a new project:

{% stepper %}
{% step %}
Click **File** → **New** → **Mule Project**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-fcfec526fe0f45391b6a6a3e7ed3af6f2dfd86cd%2Fdocs_tutorial_start-anypoint-project-new.png?alt=media" alt="" width="50%">
{% endstep %}

{% step %}
Enter a name for your project and click **Next**. We are using *csv\_upsert* in this example.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-23013742b01cc063acd4157506d3fca46abd12fb%2Fdocs_tutorial_upsert-anypoint-project-new-name.png?alt=media" alt="" width="375">

{% hint style="info" %}
Blank spaces are not supported as part of the project name.
{% endhint %}
{% endstep %}

{% step %}
Leave all the default values and click **Next**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-d10d4b8b2485ea11ef12218e0b7ed59d5ab485ca%2Fdocs_tutorial_upsert-anypoint-project-new-java.png?alt=media" alt="" width="375">
{% endstep %}

{% step %}
Click **Finish**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-957f62abb31ef551a18f9d37ce89b825e1efd7e0%2Fdocs_tutorial_upsert-anypoint-project-new-finish.png?alt=media" alt="" width="375">
{% endstep %}
{% endstepper %}

<i class="fa-chevrons-right">:chevrons-right:</i> The project is created. The **Package Explorer**, **Palette** and **Connections Explorer** are populated.

### Create the connect flow

{% stepper %}
{% step %}
From the **Palette**, drag the **File** connector to the **Canvas**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-46b21c22ccb52e0c39cdb8590009d0df09c7fc3c%2Fdocs_tutorial_upsert-anypoint-flow-file-add.png?alt=media" alt="" width="375">
{% endstep %}

{% step %}
From the **Palette**, drag the **Transform Message** component to the **Canvas**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-786bc43d5ad1ae464427ffe0e88fd4c192de58d0%2Fdocs_tutorial_upsert-anypoint-flow-transform-add.png?alt=media" alt="" width="375">
{% endstep %}

{% step %}
From the **Palette**, drag the **CollibraDGC** connector to the **Canvas**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-bf58a34c3b8aa700ba1c1dbe27917c895c875d19%2Fdocs_tutorial_upsert-anypoint-flow-connector-add.png?alt=media" alt="" width="375">
{% endstep %}
{% endstepper %}

### Configure the components

#### File connector configuration

A **File** connector placed at the beginning of a flow is set to behave like an inbound endpoint.

To configure the **File** connector:

{% stepper %}
{% step %}
Select the **File** connector on the **Canvas**.
{% endstep %}

{% step %}
In the **General** section, specify the path to the CSV file. We are using *src/main/resources/input* in this example.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-43dbdd98292c83c8a7239e4eac7e522668f3311f%2Fdocs_tutorial_upsert-anypoint-file-config-path.png?alt=media" alt="" width="75%">

{% hint style="success" %}
To create the input folder:

1. In the **Package Explorer**, right click the resources folder and select **New Folder**.
2. Type the name of the folder you wish to create and click Finish.
   {% endhint %}
   {% endstep %}

{% step %}
In the **General** section click the green **+** sign to create a new **Connector Configuration**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-cfc26e6704cbfffd1d63d0b5e976b998a4c09dde%2Fdocs_tutorial_upsert-anypoint-file-config-connector.png?alt=media" alt="" width="75%">
{% endstep %}

{% step %}
In the **Global Element Properties** window, click **OK**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-84d06ad6af55da7b5df37f29dcbf366179ca6c3d%2Fdocs_tutorial_upsert-anypoint-file-config-connector-properties.png?alt=media" alt="" width="303">
{% endstep %}

{% step %}
In the **Metadata** section, click **Add metadata** and then click the edit icon to add a new metadata type.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-c44ef918c4fdfa0f95bc1f571628fc575009d1c6%2Fdocs_tutorial_upsert-anypoint-file-config-metadata.png?alt=media" alt="" width="375">
{% endstep %}

{% step %}
In the **Select metadata type** window, click **Add**.
{% endstep %}

{% step %}
Type the name of the metadata type and click **Create type**. We are using *Asset\_by\_name* in this example.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-ff8304beb876739d15787cfdd381f48f77b414eb%2Fdocs_tutorial_upsert-anypoint-file-config-metadata-type.png?alt=media" alt="" width="365">
{% endstep %}

{% step %}
From the **Type** drop-down select **CSV**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-1a9df75a26b03a1b4a8ac189431f74f2dac855bb%2Fdocs_tutorial_upsert-anypoint-file-config-metadata-csv.png?alt=media" alt="" width="365">
{% endstep %}

{% step %}
In the **Sample File** field, select the file you have created.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-3eb7ab990a2271de7954e185754a245ec988c8a0%2Fdocs_tutorial_upsert-anypoint-file-config-metadata-csv-type.png?alt=media" alt="" width="365">

<i class="fa-chevrons-right">:chevrons-right:</i> The metadata is retrieved from the file.
{% endstep %}

{% step %}
In the **Select metadata type** window, click **Select**.
{% endstep %}
{% endstepper %}

#### CollibraDGC connector configuration

The CollibraDGC connector is used to establish a direct connection to the Collibra Data Governance Center.

To configure the CollibraDGC connector:

{% stepper %}
{% step %}
Select the CollibraDGC connector on the **Canvas**.
{% endstep %}

{% step %}
In the **General** section click the green **+** sign to create a new **Connector Configuration**.
{% endstep %}

{% step %}
In the **Global Element Properties** window, in the **General** tab, enter the connection details of your Collibra DGC environment:

* **Username**
* **Password**
* **Base Application Url**

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-ad38be1ef4a9364e3de8615e1e613fc2b765a864%2Fdocs_tutorial_upsert-anypoint-dgc-config-connector.png?alt=media" alt="" width="303">

{% hint style="success" %}
Use the **Test Connection...** feature to test the connection to Collibra DGC.
{% endhint %}
{% endstep %}

{% step %}
Click **OK** to save the configuration.
{% endstep %}

{% step %}
In the **General** section, from the **Operation** drop-down list select **Upsert assets by name**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-f3608d2c13e777f27f510ba8e9152ed75d69b7f8%2Fdocs_tutorial_upsert-anypoint-dgc-config-operation.png?alt=media" alt="" width="75%">
{% endstep %}

{% step %}
From the **Asset Type Id** drop-down list, select **Acronym**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-26f6b9f54ddc127ea62beee2187d6bbbf39d35ba%2Fdocs_tutorial_upsert-anypoint-dgc-config-type.png?alt=media" alt="" width="75%">
{% endstep %}
{% endstepper %}

The **Transform Message** component takes the output of the **File** connector and transforms it into the expected input for the CollibraDGC connector. Since you configured both connectors, their metadata is already loaded into the **Transform Message** component. To configure it, drag the fields from the input section over the corespondent fields in the output section:

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-4390b5a5faf2a607feeab915e4b0575950827ed1%2Fdocs_tutorial_upsert-anypoint-transformer-config.png?alt=media" alt="" width="75%">

<i class="fa-chevrons-right">:chevrons-right:</i> As you create the connections, the **Transform Message** component builds the required code.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-c613085c99293c60706f78bc80460cb3fff18eca%2Fdocs_tutorial_upsert-anypoint-transformer-code.png?alt=media" alt="" width="364">

Save your project.

### Run the application

To start the application in your development environment, right-click on the **Canvas** and select **Run project csv\_upsert**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-4e399cc23f5fcaaeabe6e1e4acc47a33ed717ab2%2Fdocs_tutorial_upsert-anypoint-project-run.png?alt=media" alt="" width="25%">

<i class="fa-chevrons-right">:chevrons-right:</i> If there are no errors, the console will show that the application has started.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-7f96b055a7b583fd0425ac2e318320d26de25474%2Fdocs_tutorial_upsert-anypoint-project-deployed.png?alt=media" alt="" width="75%">

To perform the upsert operation, copy the CSV file to the *src/main/resources/input* folder.

{% hint style="success" %}
You can use the **Package Explorer** to paste or drop the file.
{% endhint %}

If there are no errors, the contents of the CSV file are upserted to Collibra DGC, in the Business Glossary application, under the **All Business Assets** view.

![](https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-826d69ae2326f1fa50e8ee2d0300b6c51ba44bae%2Fdocs_tutorial_upsert-dgc-assets.png?alt=media)

In case of errors, check the console output as it contains details about what went wrong.

To stop the application, right click on the **Canvas** and select **Stop project csv\_upsert**.

<img src="https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-85ec1b716c4b4d211f3c9e2a92dbcbfca064b3c2%2Fdocs_tutorial_upsert-anypoint-project-stop.png?alt=media" alt="" width="25%">

### Next Steps

Add a definition to the CSV file, reload the CSV metadata and map the new column to the corresponding field fort the CollibraDGC connector input.

```
Name,Definition,Domain,Domain Type,Community
FAQ,Frequently Asked Questions,New Business Terms,Glossary,Data Governance Council
DGC,Data Governance Center,New Business Terms,Glossary,Data Governance Council
GDPR,General Data Protection Regulation,New Business Terms,Glossary,Data Governance Council
```

![](https://2912049260-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8DYDBo6jzvQ73H6CZj6k%2Fuploads%2Fgit-blob-31a148b7df57f0fe1f669b225fb8c34ea509a157%2Fdocs_tutorial_upsert-dgc-assets-definition.png?alt=media)
