Profiling

Configuration and execution of database profiling.

Profile a Database asset

post

Triggers the profiling job for a specific Database asset.

This API executes the profiling as an asynchronous job and returns the job id of the triggered job in the response. To monitor the status of a job, use the Jobs resource of the REST Core API: GET /jobs/{jobId}.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

databaseIdstring · uuidRequired

The ID of the Database asset.

Responses

202

The profiling job has started.

application/json

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

post/databases/{databaseId}/profile

POST /rest/catalogDatabase/v1/databases/{databaseId}/profile HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "createdBy": "4d250cc5-e583-4640-9874-b93d82c7a6cb",
  "createdOn": 1475503010320,
  "lastModifiedBy": "a073ff90-e7bc-4b35-ba90-c4d475f642fe",
  "lastModifiedOn": 1476703764163,
  "system": true,
  "resourceType": "View",
  "name": "Test name",
  "type": "IMPORT",
  "userId": "123e4567-e89b-12d3-a456-426614174000",
  "progressPercentage": 50,
  "startDate": 1488016800,
  "endDate": 1488021800,
  "state": "COMPLETED",
  "message": "Import finished in 12s."
}

List profiling configurations

get

Returns the profiling configurations defined for the given criteria.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Query parameters

databaseIdstring · uuidOptional

The ID of the Database asset.

offsetinteger · int32Optional

The index of the first result to retrieve.

If not set (offset = 0), results will be retrieved starting from row 0.

Default: 0

limitinteger · int32 · max: 500Optional

The maximum number of results to retrieve.

If not set, the default limit (limit = 0) will be used. The maximum value for this parameter is 500.

Default: 0

Responses

200

The profiling configurations defined for the given criteria.

application/json

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

get/profilingConfigurations

GET /rest/catalogDatabase/v1/profilingConfigurations HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "results": [
    {
      "databaseId": "123e4567-e89b-12d3-a456-426614174000",
      "profilingType": "sample",
      "sampleSize": 20000,
      "runAfterMetadataSynchronization": true,
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX",
      "id": "123e4567-e89b-12d3-a456-426614174000"
    }
  ]
}

Add profiling configuration

post

Creates a profiling configuration for a specific Database asset. This profiling configuration provides a default configuration that applies to all schemas in the Database asset. You can define only a single default profiling configuration for a specific Database asset.

To define schema-specific profiling configurations, use the /schemaProfilingConfigurations endpoints.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Body

databaseIdstring · uuidRequired

The ID of the Database asset.

profilingTypestring · enumOptional

full: Select to profile based on all synchronized metadata.
sample: Select to profile based on a sample of the synchronized metadata. When you select Partial scan, you can define the maximum number of rows that you want to use for profiling (as sampleSize). By default, the maximum number of rows is 20000.

Default: samplePossible values:

sampleSizeinteger · max: 1000000Optional

If profilingType is defined as sample, this attribute defines how many rows are used for profiling. The minimum value is 100.

Default: 20000

runAfterMetadataSynchronizationbooleanOptional

If set to true, it will automatically create a data profile every time the metadata synchronization process of one or more schemas finishes.

This may take a long time. You can also add a schedule to profile at regular intervals.

Default: falseExample: true

excludeTypesstringOptional

A comma-separated list of table types to be excluded during profiling.

Default: ""Example: VIEW, MATERIALIZED VIEW, INDEX

Responses

201

The profiling configuration has been created.

application/json

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

post/profilingConfigurations

POST /rest/catalogDatabase/v1/profilingConfigurations HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 184

{
  "databaseId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingType": "sample",
  "sampleSize": 20000,
  "runAfterMetadataSynchronization": true,
  "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
}

{
  "databaseId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingType": "sample",
  "sampleSize": 20000,
  "runAfterMetadataSynchronization": true,
  "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX",
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

Retrieve a profiling configuration

get

Returns a profiling configuration.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

profilingConfigurationIdstring · uuidRequired

The ID of the profiling configuration.

Responses

200

The profiling configuration.

application/json

databaseIdstring · uuidRequired

The ID of the Database asset.

profilingTypestring · enumOptional

full: Select to profile based on all synchronized metadata.
sample: Select to profile based on a sample of the synchronized metadata. When you select Partial scan, you can define the maximum number of rows that you want to use for profiling (as sampleSize). By default, the maximum number of rows is 20000.

Default: samplePossible values:

sampleSizeinteger · max: 1000000Optional

If profilingType is defined as sample, this attribute defines how many rows are used for profiling. The minimum value is 100.

Default: 20000

runAfterMetadataSynchronizationbooleanOptional

If set to true, it will automatically create a data profile every time the metadata synchronization process of one or more schemas finishes.

This may take a long time. You can also add a schedule to profile at regular intervals.

Default: falseExample: true

excludeTypesstringOptional

A comma-separated list of table types to be excluded during profiling.

Default: ""Example: VIEW, MATERIALIZED VIEW, INDEX

idstring · uuidRead-onlyRequired

The ID of the profiling configuration.

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

get/profilingConfigurations/{profilingConfigurationId}

GET /rest/catalogDatabase/v1/profilingConfigurations/{profilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "databaseId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingType": "sample",
  "sampleSize": 20000,
  "runAfterMetadataSynchronization": true,
  "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX",
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

Delete profiling configuration

delete

Deletes a profiling configuration.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

profilingConfigurationIdstring · uuidRequired

The ID of the profiling configuration.

Responses

204

The profiling configuration has been deleted.

No content

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

delete/profilingConfigurations/{profilingConfigurationId}

DELETE /rest/catalogDatabase/v1/profilingConfigurations/{profilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Accept: */*

No content

Update profiling configuration

patch

Updates a profiling configuration.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

profilingConfigurationIdstring · uuidRequired

The ID of the profiling configuration.

Body

profilingTypestring · enumOptional

full: Select to profile based on all synchronized metadata.
sample: Select to profile based on a sample of the synchronized metadata. When you select Partial scan, you can define the maximum number of rows that you want to use for profiling (as sampleSize). By default, the maximum number of rows is 20000.

Possible values:

sampleSizeinteger · max: 1000000Optional

If profilingType is defined as sample, this attribute defines how many rows are used for profiling. The minimum value is 100.

runAfterMetadataSynchronizationbooleanOptional

If set to true, it will automatically create a data profile every time the metadata synchronization process of one or more schemas finishes.

This may take a long time. You can also add a schedule to profile at regular intervals.

Example: true

excludeTypesstringOptional

A comma-separated list of table types to be excluded during profiling.

Default: ""Example: VIEW, MATERIALIZED VIEW, INDEX

Responses

200

The profiling configuration has been updated.

application/json

databaseIdstring · uuidRequired

The ID of the Database asset.

profilingTypestring · enumOptional

full: Select to profile based on all synchronized metadata.
sample: Select to profile based on a sample of the synchronized metadata. When you select Partial scan, you can define the maximum number of rows that you want to use for profiling (as sampleSize). By default, the maximum number of rows is 20000.

Default: samplePossible values:

sampleSizeinteger · max: 1000000Optional

If profilingType is defined as sample, this attribute defines how many rows are used for profiling. The minimum value is 100.

Default: 20000

runAfterMetadataSynchronizationbooleanOptional

If set to true, it will automatically create a data profile every time the metadata synchronization process of one or more schemas finishes.

This may take a long time. You can also add a schedule to profile at regular intervals.

Default: falseExample: true

excludeTypesstringOptional

A comma-separated list of table types to be excluded during profiling.

Default: ""Example: VIEW, MATERIALIZED VIEW, INDEX

idstring · uuidRead-onlyRequired

The ID of the profiling configuration.

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

patch/profilingConfigurations/{profilingConfigurationId}

PATCH /rest/catalogDatabase/v1/profilingConfigurations/{profilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 126

{
  "profilingType": "full",
  "sampleSize": 1,
  "runAfterMetadataSynchronization": true,
  "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
}

{
  "databaseId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingType": "sample",
  "sampleSize": 20000,
  "runAfterMetadataSynchronization": true,
  "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX",
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

List schema profiling configurations

get

Returns the schema-specific profiling configurations defined for the given criteria. You can apply a filter on the schema connection or database to return only the configurations defined for a given schema or database.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Query parameters

schemaConnectionIdstring · uuidOptional

The ID of the schema connection.

databaseIdstring · uuidOptional

The ID of the Database asset.

offsetinteger · int32Optional

The index of the first result to retrieve.

If not set (offset = 0), results will be retrieved starting from row 0.

Default: 0

limitinteger · int32 · max: 500Optional

The maximum number of results to retrieve.

If not set, the default limit (limit = 0) will be used. The maximum value for this parameter is 500.

Default: 0

Responses

200

The schema profiling configurations defined for the given database and (optionally) schema.

application/json

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

get/schemaProfilingConfigurations

GET /rest/catalogDatabase/v1/schemaProfilingConfigurations HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "results": [
    {
      "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
      "profilingRules": [
        {
          "profilingType": "sample",
          "sampleSize": 20000,
          "include": "customers_*",
          "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
          "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
        }
      ],
      "id": "123e4567-e89b-12d3-a456-426614174000"
    }
  ]
}

Add schema profiling configuration

post

Creates a profiling configuration for a specific schema.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Body

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

Responses

201

The schema profiling configuration has been created.

application/json

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

post/schemaProfilingConfigurations

POST /rest/catalogDatabase/v1/schemaProfilingConfigurations HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 304

{
  "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingRules": [
    {
      "profilingType": "sample",
      "sampleSize": 20000,
      "include": "customers_*",
      "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
    }
  ]
}

{
  "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingRules": [
    {
      "profilingType": "sample",
      "sampleSize": 20000,
      "include": "customers_*",
      "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
    }
  ],
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

Add multiple schema profiling configurations

post

Add multiple schema profiling configurations.

This operation is executed in a single transaction, that creates all the configurations. In case the operation fails, none of the configurations are created.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Bodyobject · AddSchemaProfilingConfigurationRequest[]

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

Responses

201

The schema profiling configurations have been created succesfully. The created profiling configurations are included in the response.

application/json

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

idstring · uuidRead-onlyRequired

The ID of schema profiling configuration.

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

post/schemaProfilingConfigurations/batch

POST /rest/catalogDatabase/v1/schemaProfilingConfigurations/batch HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 306

[
  {
    "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
    "profilingRules": [
      {
        "profilingType": "sample",
        "sampleSize": 20000,
        "include": "customers_*",
        "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
        "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
      }
    ]
  }
]

[
  {
    "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
    "profilingRules": [
      {
        "profilingType": "sample",
        "sampleSize": 20000,
        "include": "customers_*",
        "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
        "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
      }
    ],
    "id": "123e4567-e89b-12d3-a456-426614174000"
  }
]

Retrieve schema profiling configuration

get

Returns a specific schema profiling configuration.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

schemaProfilingConfigurationIdstring · uuidRequired

The ID of the schema profiling configuration.

Responses

200

The schema profiling configuration identified by the given identifier.

application/json

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

idstring · uuidRead-onlyRequired

The ID of schema profiling configuration.

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

get/schemaProfilingConfigurations/{schemaProfilingConfigurationId}

GET /rest/catalogDatabase/v1/schemaProfilingConfigurations/{schemaProfilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingRules": [
    {
      "profilingType": "sample",
      "sampleSize": 20000,
      "include": "customers_*",
      "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
    }
  ],
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

Delete schema profiling configuration

delete

Deletes a schema profiling configuration.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

schemaProfilingConfigurationIdstring · uuidRequired

The ID of the schema profiling configuration.

Responses

204

The schema profiling configuration has been deleted successfully.

No content

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

delete/schemaProfilingConfigurations/{schemaProfilingConfigurationId}

DELETE /rest/catalogDatabase/v1/schemaProfilingConfigurations/{schemaProfilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Accept: */*

No content

Update schema profiling configuration

patch

Updates a specific profiling configuration.

You can only update some attributes of the schema profiling configuration via this API. The attributes that are not provided in the request are not modified.

Authorizations

AuthorizationstringRequired

Collibra REST API authentication using Basic Authentication.

Path parameters

schemaProfilingConfigurationIdstring · uuidRequired

The ID of the schema profiling configuration.

Body

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

Responses

200

The profiling configuration has been updated.

application/json

schemaConnectionIdstring · uuidRequired

The ID of the schema connection.

idstring · uuidRead-onlyRequired

The ID of schema profiling configuration.

400

The request is not valid.

application/json

401

The client is not authenticated.

application/json

403

The user lacks permissions.

application/json

404

Resource not found.

application/json

patch/schemaProfilingConfigurations/{schemaProfilingConfigurationId}

PATCH /rest/catalogDatabase/v1/schemaProfilingConfigurations/{schemaProfilingConfigurationId} HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 304

{
  "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingRules": [
    {
      "profilingType": "sample",
      "sampleSize": 20000,
      "include": "customers_*",
      "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
    }
  ]
}

{
  "schemaConnectionId": "123e4567-e89b-12d3-a456-426614174000",
  "profilingRules": [
    {
      "profilingType": "sample",
      "sampleSize": 20000,
      "include": "customers_*",
      "exclude": "customers_very_big_one_causing_troubles, customers_another_one_I_dont_want_to_profile, temp_*",
      "excludeTypes": "VIEW, MATERIALIZED VIEW, INDEX"
    }
  ],
  "id": "123e4567-e89b-12d3-a456-426614174000"
}

PreviousMetadata NextDatabase Connection

Last updated 23 days ago

Was this helpful?