Logging in workflows

Incorporating logging into your Collibra workflows provides valuable insights during development and execution. Logging is particularly beneficial for debugging script tasks. By strategically placing log statements at various stages of your scripts, you can:

  • Monitor progress: Track the execution flow and identify the current stage of a script.
  • Identify errors: Pinpoint the specific location where an error might be occurring.
  • Observe data values: Inspect the values of variables and data being processed at different points in the script.

This practice helps you diagnose issues quickly and understand the behavior of complex logic in your workflows.

General logging capabilities

Workflows use the Java loggerApi interface for writing messages to the dgc.log file, which is available in Collibra Console. The interface offers four methods, each corresponding to a specific severity level:

  • error(String message): Logs critical errors that prevent execution.
  • warn(String message): Logs potential issues that do not stop execution but may require attention.
  • info(String message): Logs general operational information and status updates.
  • debug(String message): Logs detailed, low-level information, typically used during active debugging.

By default, the dgc.log file is configured to capture log entries up to the INFO level. This means that messages logged with ERROR, WARN, and INFO are recorded. Log statements at the DEBUG level are not recorded by default. This configuration helps manage log volume, as DEBUG logs can be very verbose, especially in production environments.

Temporarily enabling DEBUG logging for enhanced debugging

When debugging a workflow script, you may need more detailed information than what is captured at the INFO level. You can temporarily elevate the logging level to DEBUG for loggerApi in Collibra Console. This allows you to use DEBUG statements in your workflow code without permanently increasing log volume.

Prerequisites

You have at least the ADMIN role in Collibra Console.

Steps

  1. Open Collibra Console.
    Collibra Console opens with the Infrastructure page.
  2. In the tab pane, expand an environment to show its services.
  3. Select the Data Governance Center service.
  4. Select Logs → Cogwheel icon Settings → Add logger.

    The Add logger dialog boxappears.

  5. Enter the required information:
    1. Logger name: com.collibra.dgc.core.api.component.LoggerApiImpl
    2. Logger level: DEBUG
  6. Click Add logger to apply the changes and close the dialog box.

Important considerations and best practices

  • Impact: Changing the log level affects all workflows and services using this logger. A significant increase in logged information may occur.
  • Revert changes: After completing your debugging session, revert the log level to the default INFO setting or remove the custom logger entry. To do this, go to the same Logs settings page and either remove (Trash icon) the entry for com.collibra.dgc.core.api.component.LoggerApiImpl or change (Edit icon) the associated level back to INFO.
  • Environment: You should temporarily enable DEBUG logging only in non-production environments. This practice ensures you maintain control over log volume and avoid impacting users or system performance in a production environment.
  • Permanent DEBUG statements: You can include DEBUG-level log statements in your workflow code permanently. These statements are only captured when the logger level is set to DEBUG in the Collibra Console, ensuring your code remains debug-ready without causing excessive logging under normal operations.

Implementing logging in your workflows

You can integrate logging directly into your workflow logic in several ways:

  1. In script tasks.

    Script tasks are a common place to embed logging. You can add log statements at any point within your script and you can include multiple loggerApi calls in a single script task to trace the execution path and variable states.

    loggerApi.info("Starting script task script_1. Value for variable myVariable is: " + execution.getVariable("myVariable"));
  2. Via an execution listener.

    If you need to log information when the workflow reaches or completes a specific step, a separate script task solely for logging that might be inefficient. Instead, you can incorporate an execution listener directly into most process elements.

    To configure an execution listener for logging, edit the process in Workflow Designer:

    1. On the canvas, select the desired process element, for example a user task or service task.
    2. In the properties bar, go to Advanced → Listeners → Execution listeners.
    3. In the Execution listeners dialog box, click Add icon to add an execution listener.
    4. Configure the listener by defining the event that triggers the listener and the script or expression to execute.
      • Event: Start
      • Type: Expression
      • Value: ${loggerApi.debug("Sample log line")}

    This approach enables granular logging tied to specific workflow events without the overhead of additional, dedicated script tasks.

Conclusion

By leveraging loggerApi, you can monitor and debug workflows in Collibra with greater precision. Follow best practices, such as enabling DEBUG logging only in non-production environments and reverting settings after debugging. Properly implemented logging not only enhances development efficiency but also improves the maintainability of your workflows.