Tag approval

In this tutorial you learn how to create a workflow that notifies a technical steward when a tag is added to an asset and allows the steward to add comments and set an attribute.

This scenario was part of the interactive Data Citizens - Under the Hood technical day 2019.

Prerequisites

  • Eclipse IDE.
  • Flowable Eclipse Designer plugin.
  • Access to a Collibra environment as a user with the Sysadmin global role or a global role that has at least the Workflow Administration global permission.
  • Access to the Collibra Console for troubleshooting.

For more installation details and to get familiar with the Eclipse IDE interface, see Getting Started with workflows.

Create the workflow

Open Eclipse IDE and create a new Flowable project and diagram:

  • Use Tag Approval for the project name.
  • Use PIITagApproval.bpmn for the diagram name.

    • For more information on how to create a Flowable project and diagram, see the Getting started with workflows and the Hello World! tutorials.

Add a pool and lanes

  1. Add a pool and configure the pool properties:
    • General:
      • Id: tagApprovalPool
      • Name: Tag Approval
    • Process:
      • Id: piiTagApprovalWorkflow

        The process ID must be unique at the Collibra environment level.

        Uploading a workflow with the same process ID as an existing workflow in Collibra replaces the existing workflow.

      • Name: PII Tag Approval

        The name is displayed as the workflow name in Collibra.

        Collibra does not accept two workflows with the same display name.

      • Namespace: http://www.collibra.com/apiv2

        By referencing version 2 of the Collibra Java API, the workflow is validated against V2 methods and deprecated V1 methods trigger a method does not exist error when called.

      • Documentation: The contents of the field become the description of the workflow in Collibra.
  2. Configure the lane properties:
    • Id: systemLane
    • Name: System
  3. From the Containter section of the Palette, drag a second Lane to the pool and configure the lane properties:
    • Id: technicalStewardLane
    • Name: Technical Steward

Add start and end events

  1. From the Start event section of the Palette, drag a StartEvent to the System lane.
  2. In the Properties view, select the Form section.

    The start event form variables set values that are used throughout the workflow. You can change these variables from the worfklow definition page in Collibra Data Intelligence Cloud.

  3. Add a variable to set the role of the user who reviews and approves the tags:
    1. Click New.
    2. Enter the required information:
      • Id: approverUserExpression
      • Name: Candidate user expression for the approver
      • Type: string
      • Default: role(Technical Steward)
      • Readable: False
      • Writeable: True
      • Required: True

  4. Add a variable to determine which tag triggers the review process:
    1. Click New.
    2. Enter the required information:
      • Id: piiTagName
      • Name: Search term for tags
      • Type: string
      • Default: PII
      • Readable: False
      • Writeable: True
      • Required: True
  5. Add a variable to set the UUID of the PII attribute type:
    1. Click New.
    2. Enter the required information:
      • Id: piiAttributeTypeId
      • Name: The UUID of the PII attribute
      • Type: string
      • Default: 00000000-0000-0000-0001-000500000029
      • Readable: False
      • Writeable: True
      • Required: True
  6. From the End event section of the Palette, drag an EndEvent to the System lane.

Create a script task to search for tags

The script searches through the newly added tags for the term specified by the piiTagName variable. Depending on the result, it sets the isPII variable to either true or false. It also makes the results available in the Collibra logs.

  1. From the Task section of the Palette drag a ScriptTask to the System lane.
  2. In the General section, enter a Name: Get and filter tags.
  3. In the Main config section:
    • Select the groovy script language.
    • Add the Get and filter tags script: 
      loggerApi.info("Get & filter tags script started...")

//Variables
def tags = []
def piiTagName = execution.getVariable("piiTagName")
def isPII = false
def assetUuid = item.id 

tags = tagApi.getTagsByAssetId(assetUuid)
loggerApi.info("Number of tags found: " + tags.size())

//Loop over the tag list and check for PII tag
for(tag in tags){
	if(tag.getName().equals(piiTagName)){
		isPII = true
	}
}

loggerApi.info("isPII status: " + isPII)

//Set variables to be used in workflow
execution.setVariable("isPII",isPII)

loggerApi.info("Get & filter tags script ended.")
  4. Connect the start event to the script task with a SequenceFlow connection.

Route the workflow based on the tags

End or continue the workflow based on the value of the isPII variable.

  1. From the Gateway section of the Palette, drag an ExclusiveGateway to the System lane, after the Get and filter tags script task.
  2. Connect the script task to the exclusive gateway with a SequenceFlow connection.
  3. From the End event section of the Palette, drag an EndEvent to the System lane, above the exclusive gateway.
  4. Connect the exclusive gateway to the second end event with a SequenceFlow connection.
  5. Select the sequence flow.
  6. In the Properties view, select the General section and enter a Name: No PII Found.
  7. In the Properties view, select the Main config tab and enter the Condition: ${!isPII}.
  8. From the Task section of the Palette drag a UserTask to the Technical Steward lane.
  9. Connect the exclusive gateway to the user task in the Technical Steward lane with a SequenceFlow connection.
  10. Select the sequence flow.
  11. In the Properties view, select the General section and enter a Name: PII Found.
  12. In the Properties view, select the Main config tab and enter the Condition: ${isPII}.

Configure the technical steward task

If a PII tag is found, create a task for users with a Technical Steward role to approve or reject if the tagged column contains personal identifiable information and provide their justification.

  1. In the Technical Steward lane, select the user task.
  2. In the Properties view, select the General section and enter the Name: Approve PII Tag.
  3. In the Properties view, select the Main Config section and enter the Candidate user: ${approverUserExpression}.
  4. In the Properties view, select the Documentation section and enter:Check if the column with tag "${piiTagName}" contains personal identifiable information. If you approve, a PII attribute is added to the column.
  5. In the Properties view, select the Form section and add the following elements to a dialog box:
    • A task button to replace the default call to action and define the task:
      • Id: taskButton1
      • Name: Approve/Reject
      • Type: taskButton
      • Readable: True
      • Writeable: True
      • Required: False
    • A comment box to allow the technical steward to provide a justification:
      • Id: commentBox
      • Name: Reason
      • Type: textarea
      • Readable: True
      • Writeable: True
      • Required: True
    • A button to approve:
      • Id: approveButton
      • Name: Approve
      • Type: button
      • Readable: True
      • Writeable: True
      • Required: False
    • A button to reject:
      • Id: rejectButton
      • Name: Reject
      • Type: button
      • Readable: True
      • Writeable: True
      • Required: False

The dialog box of the user task looks like this in Collibra Data Intelligence Cloud:

Add a script to save the justification to the comments section

The script saves the technical steward justification as a comment to the asset and also adds it to the Collibra log:

  1. From the Task section of the Palette, drag a ScriptTask inside the System lane.
  2. In the Properties view, select the General section and enter the Name: Save Comment.
  3. In the Main config section:
    • Select the groovy script language.
    • Add the Add comment script:
      import com.collibra.dgc.core.api.dto.instance.comment.AddCommentRequest
import com.collibra.dgc.core.api.model.ResourceType

loggerApi.info("Add comment script started...")

//Variables
def commentBox = execution.getVariable("commentBox")
def commentBoxContent = commentBox.toString()
def assetUuid = item.id

loggerApi.info("Comment box content: " + commentBoxContent)

commentApi.addComment(AddCommentRequest.builder()
	.baseResourceId(assetUuid)
	.baseResourceType(ResourceType.Asset)
	.content(commentBoxContent)
	.build()
	)

loggerApi.info("Add comment script ended.")
  4. Connect the Approve PII Tag script to the Save Comment script with a SequenceFlow connection.

Route the workflow based on the steward decision

Based on the technical steward decision, end or continue the workflow:

  1. From the Gateway section of the Palette, drag an ExclusiveGateway to the System lane, after the Save comment script task.
  2. Connect the Save comment script task to the exclusive gateway with a SequenceFlow connection.
  3. From the End event section of the Palette, drag an EndEvent to the System lane, above the exclusive gateway.
  4. Connect the exclusive gateway to the third end event with a SequenceFlow connection.
  5. Select the sequence flow.
  6. In the Properties view, select the General section and enter a Name: Rejected.
  7. In the Properties view, select the Main Config section and enter the Condition: ${rejectButton}.
  8. From the Task section of the Palette drag a ScriptTask to the System lane, after the exclusive gateway.
  9. Connect the exclusive gateway to the script task with a SequenceFlow connection.
  10. Select the sequence flow.
  11. In the Properties view, select the General section and enter a Name: Approved.
  12. In the Properties view, select the Main config tab and enter the Condition: ${approveButton}.
  13. Connect the script task to the last end event with a SequenceFlow connection.

Add a script to set the PII attribute

When the technical steward approves that the tagged column contains personally identifiable information, set the Personally Identifiable Information asset attribute to true:

  1. Select the last added script task.
  2. In the Properties view, select the General section and enter the Name: Set PII Attribute.
  3. In the Main config section:
    • Select the groovy script language.
    • Add the Set PII Attribute script:
      import com.collibra.dgc.core.api.dto.instance.attribute.AddAttributeRequest

loggerApi.info("Set PII Attribute script started...")

//Variables
def assetUuid = item.id
def piiAttributeTypeId = execution.getVariable("piiAttributeTypeId")
def piiAttributeTypeUuid = string2Uuid(piiAttributeTypeId)

//Set PII Attribute
attributeApi.addAttribute(AddAttributeRequest.builder()
	.assetId(assetUuid)
	.typeId(piiAttributeTypeUuid)
	.value(true)
	.build()
)

loggerApi.info("Set PII Attribute script ended")

After the script runs, you see a green check mark in the Personally Identifiable Information section of the Asset in Collibra Data Intelligence Cloud:

Save your work.

Deploy your workflow

  1. Sign in to Collibra as a user with the Sysadmin global role or a global role that has at least the Workflow Administration global permission.
  2. On the main menu, click , and then click Settings.
    The Collibra settings page opens.
  3. Click Workflows.
    The Workflows settings page appears on the Definitions tab page.
  4. Click Upload a file and locate the BPMN file you have just created inside the Eclipse workspace folder.

    With the BPMN file selected in Eclipse, select Navigate → Show In → System Explorer. An explorer window will open, showing the exact file location. You can now drag and drop the file to the Upload a file section in your browser.

An Activity Queue window displays the upload progress.

Once the file is uploaded, you see your workflow in the Workflows Definitions table.

For more details, see the Deploy a workflow section of the Collibra Data Intelligence Cloud Administration Guide.

Automatically trigger the workflow when a tag is added to a column

To fine tune the workflow you must change the default configuration in Collibra Data Intelligence Cloud.

Start the workflow automatically when a tag is added to an asset.

  1. In Collibra Data Intelligence Cloud, go to Settings → Workflows → Definitions.
  2. Select the PII Tag Approval workflow.
  3. Go to the Start Event section.
  4. Select the Asset Tag Event.
  5. Click Save.

Restrict the workflow to the column asset type.

  1. In the workflow definition page, change the Applies To to Asset.
  2. In the Applies To section, click Add.
  3. Search for Column and click Save.

Testing & Troubleshooting

Your Collibra Data Intelligence Cloud must have at least one asset that is a column as you restricted the workflow to this asset type. If you need to use another asset type, remove the restriction from the Applies To section of the workflow definition page.

To see the results of the workflow, ensure any other asset type you choose has the Personally Identifiable Information attribute assigned. For more information about global assignments see the Assign a characteristic type to an asset type section of the Collibra Administration Guide.

To test the workflow, at least one user must have the role of Technical Steward. If there are no users with this role, the workflow does not complete and you get an error.

You can change the default approver role from the Variables section of the workflow definition page.

If the workflow fails no tags are added to the asset.

In case of errors, the workflow does not start, and an error message is displayed.

The logs contain more information about the error:

  1. Open Collibra Console.
    Collibra Console opens with the Infrastructure page.
  2. Click the name of the environment you deployed the workflow in to display its details.
  3. Select Data Governance Center.
  4. Select Logs → dgc.log

Look for errors related to Java or Groovy.

Next steps

You have created a workflow that scans Collibra Data Intelligence Cloud for PII tags and asks a technical steward to review the asset and add a PII attribute.

This scenario was part of the interactive Data Citizens - Under the Hood technical day 2019. To get the full Under the Hood experience, look for the next session and the next Data Citizens conference.

Additional resources

  • Read the Getting started with workflows Collibra Administration Guide section.
  • Read the Using workflows Collibra User Guide section.
  • Consult the java documentation: https://<your_dgc_environment_url>/docs/javav2/index.html.