S3

AWS S3 synchronization operations

Retrieve a capability

get

Returns the capability of the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

200

Id of attached Edge capability

application/json

string · uuidOptional

404

No capability attached

get/filesystem/{fileSystemType}/{assetId}/capability

HTTPcURLJavaScriptPython

GET /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/capability HTTP/1.1
Authorization: Basic username:password
Accept: */*

123e4567-e89b-12d3-a456-426614174000

Add a capability

put

Adds a capability to the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

string · uuidOptional

Responses

204

Capability attached

No content

put/filesystem/{fileSystemType}/{assetId}/capability

HTTPcURLJavaScriptPython

PUT /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/capability HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 38

"123e4567-e89b-12d3-a456-426614174000"

204

Capability attached

No content

Remove a capability

delete

Removes a capability from the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

204

Capability detached

No content

delete/filesystem/{fileSystemType}/{assetId}/capability

HTTPcURLJavaScriptPython

DELETE /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/capability HTTP/1.1
Authorization: Basic username:password
Accept: */*

204

Capability detached

No content

Find filesystem assets that have the given Edge capability attached

get

Authorizations

basicAuth

AuthorizationstringRequired

Query parameters

capabilityIdstring · uuidOptional

Responses

200

Filesystem asset ids

application/json

string · uuid[]Optional

get/filesystem

HTTPcURLJavaScriptPython

GET /rest/catalog/1.0/filesystem HTTP/1.1
Authorization: Basic username:password
Accept: */*

200

Filesystem asset ids

[
  "123e4567-e89b-12d3-a456-426614174000"
]

List crawlers

get

Returns a list of crawlers that are defined for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

200

Success

application/json

crawlerIdstring · uuidOptional

namestringOptional

fileSystemIdstring · uuidOptional

domainIdstring · uuidOptional

customClassifiersstring[]Optional

tableLevelinteger · int32Optional

targetsstringOptional

fileGroupPatternstringOptional

get/filesystem/{fileSystemType}/{assetId}/crawlers

HTTPcURLJavaScriptPython

GET /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/crawlers HTTP/1.1
Authorization: Basic username:password
Accept: */*

200

Success

[
  {
    "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
    "name": "text",
    "includePaths": [
      {
        "path": "text",
        "excludePatterns": [
          "text"
        ]
      }
    ],
    "fileSystemId": "123e4567-e89b-12d3-a456-426614174000",
    "domainId": "123e4567-e89b-12d3-a456-426614174000",
    "customClassifiers": [
      "text"
    ],
    "tableLevel": 1,
    "targets": "text",
    "fileGroupPattern": "text"
  }
]

Add a crawler

post

Adds a new crawler definition to the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

namestring · min: 1 · max: 2147483647Required

fileSystemAssetIdstring · uuidRequired

domainIdstring · uuidRequired

assetTypeIdstring · uuidRequired

customClassifiersstring[]Optional

tableLevelinteger · int32 · min: 1 · max: 100Optional

targetsstringOptional

fileGroupPatternstringOptional

Responses

200

Success

application/json

crawlerIdstring · uuidOptional

namestringOptional

fileSystemIdstring · uuidOptional

domainIdstring · uuidOptional

customClassifiersstring[]Optional

tableLevelinteger · int32Optional

targetsstringOptional

fileGroupPatternstringOptional

post/filesystem/{fileSystemType}/{assetId}/crawlers

HTTPcURLJavaScriptPython

POST /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/crawlers HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 324

{
  "name": "text",
  "includePaths": [
    {
      "path": "text",
      "excludePatterns": [
        "text"
      ]
    }
  ],
  "fileSystemAssetId": "123e4567-e89b-12d3-a456-426614174000",
  "domainId": "123e4567-e89b-12d3-a456-426614174000",
  "assetTypeId": "123e4567-e89b-12d3-a456-426614174000",
  "customClassifiers": [
    "text"
  ],
  "tableLevel": 1,
  "targets": "text",
  "fileGroupPattern": "text"
}

200

Success

{
  "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "includePaths": [
    {
      "path": "text",
      "excludePatterns": [
        "text"
      ]
    }
  ],
  "fileSystemId": "123e4567-e89b-12d3-a456-426614174000",
  "domainId": "123e4567-e89b-12d3-a456-426614174000",
  "customClassifiers": [
    "text"
  ],
  "tableLevel": 1,
  "targets": "text",
  "fileGroupPattern": "text"
}

Update a crawler

put

Updates a crawler definition for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

namestring · min: 1 · max: 2147483647Required

crawlerIdstring · uuidRequired

domainIdstring · uuidRequired

assetTypeIdstring · uuidRequired

customClassifiersstring[]Optional

tableLevelinteger · int32 · min: 1 · max: 100Optional

targetsstringOptional

fileGroupPatternstringOptional

Responses

200

Success

application/json

crawlerIdstring · uuidOptional

namestringOptional

fileSystemIdstring · uuidOptional

domainIdstring · uuidOptional

customClassifiersstring[]Optional

tableLevelinteger · int32Optional

targetsstringOptional

fileGroupPatternstringOptional

put/filesystem/{fileSystemType}/{assetId}/crawlers

HTTPcURLJavaScriptPython

PUT /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/crawlers HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 316

{
  "name": "text",
  "includePaths": [
    {
      "path": "text",
      "excludePatterns": [
        "text"
      ]
    }
  ],
  "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
  "domainId": "123e4567-e89b-12d3-a456-426614174000",
  "assetTypeId": "123e4567-e89b-12d3-a456-426614174000",
  "customClassifiers": [
    "text"
  ],
  "tableLevel": 1,
  "targets": "text",
  "fileGroupPattern": "text"
}

200

Success

{
  "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "includePaths": [
    {
      "path": "text",
      "excludePatterns": [
        "text"
      ]
    }
  ],
  "fileSystemId": "123e4567-e89b-12d3-a456-426614174000",
  "domainId": "123e4567-e89b-12d3-a456-426614174000",
  "customClassifiers": [
    "text"
  ],
  "tableLevel": 1,
  "targets": "text",
  "fileGroupPattern": "text"
}

Delete a crawler

delete

Deletes a crawler definition from the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

crawlerIdstring · uuidRequired

fileSystemAssetIdstring · uuidRequired

assetTypeIdstring · uuidRequired

Responses

204

Crawler deleted

No content

delete/filesystem/{fileSystemType}/{assetId}/crawlers

HTTPcURLJavaScriptPython

DELETE /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/crawlers HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 164

{
  "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
  "fileSystemAssetId": "123e4567-e89b-12d3-a456-426614174000",
  "assetTypeId": "123e4567-e89b-12d3-a456-426614174000"
}

204

Crawler deleted

No content

Retrieve a synchronization schedule

get

Returns the synchronization schedule for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

200

cloud synchronization schedule

application/json

idinteger · int64Optional

cronExpressionstringOptional

cronTimeZonestringOptional

lastRunTimeStampinteger · int64Optional

nextRunDateLongValueinteger · int64Optional

cronJsonstringOptional

workflowstringOptional

get/filesystem/{fileSystemType}/{assetId}/schedule

HTTPcURLJavaScriptPython

GET /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/schedule HTTP/1.1
Authorization: Basic username:password
Accept: */*

200

cloud synchronization schedule

{
  "id": 1,
  "cronExpression": "text",
  "cronTimeZone": "text",
  "lastRunTimeStamp": 1,
  "nextRunDateLongValue": 1,
  "cronJson": "text",
  "workflow": "text"
}

Add a synchronization schedule

post

Adds a synchronization schedule for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

fileSystemAssetIdstring · uuidRequired

cronExpressionstringRequired

cronTimeZonestringRequired

cronJsonstringOptional

Responses

200

Cloud synchronization schedule added

application/json

idinteger · int64Optional

cronExpressionstringOptional

cronTimeZonestringOptional

lastRunTimeStampinteger · int64Optional

nextRunDateLongValueinteger · int64Optional

cronJsonstringOptional

workflowstringOptional

post/filesystem/{fileSystemType}/{assetId}/schedule

HTTPcURLJavaScriptPython

POST /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/schedule HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 124

{
  "fileSystemAssetId": "123e4567-e89b-12d3-a456-426614174000",
  "cronExpression": "text",
  "cronTimeZone": "text",
  "cronJson": "text"
}

200

Cloud synchronization schedule added

{
  "id": 1,
  "cronExpression": "text",
  "cronTimeZone": "text",
  "lastRunTimeStamp": 1,
  "nextRunDateLongValue": 1,
  "cronJson": "text",
  "workflow": "text"
}

Update a synchronization schedule

put

Updates a synchronization schedule for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Body

application/json

fileSystemAssetIdstring · uuidRequired

cronExpressionstringRequired

cronTimeZonestringRequired

cronJsonstringOptional

Responses

200

S3 synchronization schedule updated

application/json

idinteger · int64Optional

cronExpressionstringOptional

cronTimeZonestringOptional

lastRunTimeStampinteger · int64Optional

nextRunDateLongValueinteger · int64Optional

cronJsonstringOptional

workflowstringOptional

201

New S3 synchronization schedule created

application/json

put/filesystem/{fileSystemType}/{assetId}/schedule

HTTPcURLJavaScriptPython

PUT /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/schedule HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 124

{
  "fileSystemAssetId": "123e4567-e89b-12d3-a456-426614174000",
  "cronExpression": "text",
  "cronTimeZone": "text",
  "cronJson": "text"
}

{
  "id": 1,
  "cronExpression": "text",
  "cronTimeZone": "text",
  "lastRunTimeStamp": 1,
  "nextRunDateLongValue": 1,
  "cronJson": "text",
  "workflow": "text"
}

Delete a synchronization schedule

delete

Deletes a synchronization schedule for the file system with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

204

Cloud synchronization schedule deleted

No content

delete/filesystem/{fileSystemType}/{assetId}/schedule

HTTPcURLJavaScriptPython

DELETE /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/schedule HTTP/1.1
Authorization: Basic username:password
Accept: */*

204

Cloud synchronization schedule deleted

No content

Synchronize a file system

post

Starts an asynchronous Edge synchronization job for the file system or capability with the specified ID.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Query parameters

cloudIngestionJobIdstring · uuidOptional

Responses

202

Cloud synchronization job successfully started

application/json

Represents a job. Job is a single atomic task that is to be performed asynchronously

idstring · uuidRequired

The id of the represented object (entity).

createdBystring · uuidOptional

The id of the user that created this resource.

Example: 4d250cc5-e583-4640-9874-b93d82c7a6cb

createdOninteger · int64Optional

The timestamp (in UTC time standard) of the creation of this resource.

Example: 1475503010320

lastModifiedBystring · uuidOptional

The id of the user who modified this resource the last time.

Example: a073ff90-e7bc-4b35-ba90-c4d475f642fe

lastModifiedOninteger · int64Optional

The timestamp (in UTC time standard) of the last modification of this resource.

Example: 1476703764163

systembooleanOptional

Whether this is a system resource or not.

resourceTypestring · enumRequiredDeprecated

The type of the resource, e.g. [Community, Asset, Domain, Attribute, Relation, WorkflowInstance, ...]. This property is deprecated and will be removed in the future. In order to stay backwards compatible and to enable introduction of new types of resources, where necessary, a resource discriminator property (of type String) has been added to specific subtypes of Resource, e.g. assignedResourceDiscriminator on AssignedResource, or attributeDiscriminator on Attribute. For new types of resource that didn't exist before this change, the resourceType will be BaseDataType.

Possible values:

namestringOptional

The name of the resource.

Example: Test name

typestringOptional

The type of the job.

Example: IMPORT

userIdstring · uuidOptional

The ID of the user that initiated this job.

visibilityinteger · int32Optional

The visibility of the job.

progressPercentagenumber · doubleOptional

The progress percentage of the job.

Example: 50

cancelablebooleanOptional

Whether this job is cancelable or not. If set to false it will not be possible to cancel the job once submitted.

startDateinteger · int64Optional

The start date of the job.

Example: 1488016800

endDateinteger · int64Optional

The end date of the job.

Example: 1488021800

statestring · enumOptional

The state of the job.

Example: COMPLETEDPossible values:

resultstring · enumOptional

The result of the job.

Example: SUCCESSPossible values:

messagestringOptional

The message of the job.

Example: Import finished in 12s.

post/filesystem/{fileSystemType}/{assetId}/synchronize

HTTPcURLJavaScriptPython

POST /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/synchronize HTTP/1.1
Authorization: Basic username:password
Accept: */*

202

Cloud synchronization job successfully started

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "createdBy": "4d250cc5-e583-4640-9874-b93d82c7a6cb",
  "createdOn": 1475503010320,
  "lastModifiedBy": "a073ff90-e7bc-4b35-ba90-c4d475f642fe",
  "lastModifiedOn": 1476703764163,
  "system": true,
  "name": "Test name",
  "type": "IMPORT",
  "userId": "123e4567-e89b-12d3-a456-426614174000",
  "visibility": 1,
  "progressPercentage": 50,
  "cancelable": true,
  "startDate": 1488016800,
  "endDate": 1488021800,
  "state": "COMPLETED",
  "result": "SUCCESS",
  "message": "Import finished in 12s."
}

Schedule canceling of cloud synchronization

delete

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

fileSystemTypestring · enumRequiredPossible values:

assetIdstring · uuidRequired

Responses

404

No job to cancel

No content

delete/filesystem/{fileSystemType}/{assetId}/synchronize

HTTPcURLJavaScriptPython

DELETE /rest/catalog/1.0/filesystem/{fileSystemType}/{assetId}/synchronize HTTP/1.1
Authorization: Basic username:password
Accept: */*

404

No job to cancel

No content

Get the custom classifiers.

get

Get the custom classifiers for S3 Glue crawler.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

assetIdstring · uuidRequired

crawlerIdstring · uuidRequired

Responses

200

Content of custom classifiers

application/json

stringOptional

get/filesystem/s3/{assetId}/crawlers/{crawlerId}

HTTPcURLJavaScriptPython

GET /rest/catalog/1.0/filesystem/s3/{assetId}/crawlers/{crawlerId} HTTP/1.1
Authorization: Basic username:password
Accept: */*

200

Content of custom classifiers

text

Set the custom classifiers.

patch

Set the custom classifiers for S3 Glue crawler.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

assetIdstring · uuidRequired

crawlerIdstring · uuidRequired

Body

application/json

crawlerIdstring · uuidRequired

customClassifierNamesstring[]Required

Responses

204

Custom classifiers has been successfully set

No content

patch/filesystem/s3/{assetId}/crawlers/{crawlerId}

HTTPcURLJavaScriptPython

PATCH /rest/catalog/1.0/filesystem/s3/{assetId}/crawlers/{crawlerId} HTTP/1.1
Authorization: Basic username:password
Content-Type: application/json
Accept: */*
Content-Length: 85

{
  "crawlerId": "123e4567-e89b-12d3-a456-426614174000",
  "customClassifierNames": [
    "text"
  ]
}

204

Custom classifiers has been successfully set

No content

Asynchronously synchronizes an S3 File System.

post

Starts a Jobserver based synchronization job for the give S3 File System.

Authorizations

basicAuth

AuthorizationstringRequired

Path parameters

assetIdstring · uuidRequired

S3 File System identifier

Responses

202

S3 synchronization job successfully started

application/json

Represents a job. Job is a single atomic task that is to be performed asynchronously

idstring · uuidRequired

The id of the represented object (entity).

createdBystring · uuidOptional

The id of the user that created this resource.

Example: 4d250cc5-e583-4640-9874-b93d82c7a6cb

createdOninteger · int64Optional

The timestamp (in UTC time standard) of the creation of this resource.

Example: 1475503010320

lastModifiedBystring · uuidOptional

The id of the user who modified this resource the last time.

Example: a073ff90-e7bc-4b35-ba90-c4d475f642fe

lastModifiedOninteger · int64Optional

The timestamp (in UTC time standard) of the last modification of this resource.

Example: 1476703764163

systembooleanOptional

Whether this is a system resource or not.

resourceTypestring · enumRequiredDeprecated

The type of the resource, e.g. [Community, Asset, Domain, Attribute, Relation, WorkflowInstance, ...]. This property is deprecated and will be removed in the future. In order to stay backwards compatible and to enable introduction of new types of resources, where necessary, a resource discriminator property (of type String) has been added to specific subtypes of Resource, e.g. assignedResourceDiscriminator on AssignedResource, or attributeDiscriminator on Attribute. For new types of resource that didn't exist before this change, the resourceType will be BaseDataType.

Possible values:

namestringOptional

The name of the resource.

Example: Test name

typestringOptional

The type of the job.

Example: IMPORT

userIdstring · uuidOptional

The ID of the user that initiated this job.

visibilityinteger · int32Optional

The visibility of the job.

progressPercentagenumber · doubleOptional

The progress percentage of the job.

Example: 50

cancelablebooleanOptional

Whether this job is cancelable or not. If set to false it will not be possible to cancel the job once submitted.

startDateinteger · int64Optional

The start date of the job.

Example: 1488016800

endDateinteger · int64Optional

The end date of the job.

Example: 1488021800

statestring · enumOptional

The state of the job.

Example: COMPLETEDPossible values:

resultstring · enumOptional

The result of the job.

Example: SUCCESSPossible values:

messagestringOptional

The message of the job.

Example: Import finished in 12s.

409

Other synchronization for this s3FileSystemId is currently active

post/filesystem/s3/{assetId}/synchronize/jobserver

HTTPcURLJavaScriptPython

POST /rest/catalog/1.0/filesystem/s3/{assetId}/synchronize/jobserver HTTP/1.1
Authorization: Basic username:password
Accept: */*

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "createdBy": "4d250cc5-e583-4640-9874-b93d82c7a6cb",
  "createdOn": 1475503010320,
  "lastModifiedBy": "a073ff90-e7bc-4b35-ba90-c4d475f642fe",
  "lastModifiedOn": 1476703764163,
  "system": true,
  "name": "Test name",
  "type": "IMPORT",
  "userId": "123e4567-e89b-12d3-a456-426614174000",
  "visibility": 1,
  "progressPercentage": 50,
  "cancelable": true,
  "startDate": 1488016800,
  "endDate": 1488021800,
  "state": "COMPLETED",
  "result": "SUCCESS",
  "message": "Import finished in 12s."
}