Overview

Cloudera data lake is a service for launching and managing data lake clusters powered by Cloudera Runtime.

Version information

Version : 0.9.111

License information

License : Apache 2.0
Terms of service : https://www.cloudera.com/legal/commercial-terms-and-conditions.html

URI scheme

Schemes : HTTPS

Consumes

  • application/json

Produces

  • application/json

Paths

Create backup of datalake.

POST /api/v1/datalake/backupDatalake

Description

Takes a backup of all the data in the datalake.

Parameters

Type Name Schema

Body

input
required

BackupDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

BackupDatalakeResponse

default

The default response on an error.

Error

Check the status of a datalake backup operation performed.

POST /api/v1/datalake/backupDatalakeStatus

Description

Identifies the backup operation based on the inputs provided and gets the status. BackupName and BackupId are mutually exclusive. Only one of them can be provided.

Parameters

Type Name Schema

Body

input
required

BackupDatalakeStatusRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

BackupDatalakeStatusResponse

default

The default response on an error.

Error

Cancels the specified ongoing backup operation.

POST /api/v1/datalake/cancelBackup

Description

Cancels a currently running backup operation. The operation must be in a STARTED or IN_PROGRESS state. Pending sub-operations will be ignored and marked as cancelled. The operation does not wait for the currently running sub-operation(s) to complete.

Parameters

Type Name Schema

Body

input
required

CancelBackupRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CancelBackupResponse

default

The default response on an error.

Error

Cancel running Datalake diagnostics collections

POST /api/v1/datalake/cancelDatalakeDiagnostics

Description

Cancel running Datalake diagnostics collection

Parameters

Type Name Schema

Body

input
required

CancelDatalakeDiagnosticsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CancelDatalakeDiagnosticsResponse

default

The default response on an error.

Error

Start DataLake Cloudera Manager based diagnostics collection

POST /api/v1/datalake/collectCmDiagnostics

Description

Start Datalake Cloudera Manager based diagnostics collection

Parameters

Type Name Schema

Body

input
required

CollectCmDiagnosticsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CollectCmDiagnosticsResponse

default

The default response on an error.

Error

Start DataLake diagnostics collection

POST /api/v1/datalake/collectDatalakeDiagnostics

Description

Start Datalake diagnostics collection

Parameters

Type Name Schema

Body

input
required

CollectDatalakeDiagnosticsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CollectDatalakeDiagnosticsResponse

default

The default response on an error.

Error

Creates an AWS datalake.

POST /api/v1/datalake/createAWSDatalake

Description

Creates an AWS datalake.

Parameters

Type Name Schema

Body

input
required

CreateAWSDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CreateAWSDatalakeResponse

default

The default response on an error.

Error

Creates a Data Lake on AWS GovCloud.

POST /api/v1/datalake/createAWSGovCloudDatalake

Description

Creates a Data Lake on AWS GovCloud.

Parameters

Type Name Schema

Body

input
required

CreateAWSGovCloudDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CreateAWSGovCloudDatalakeResponse

default

The default response on an error.

Error

Creates an Azure datalake.

POST /api/v1/datalake/createAzureDatalake

Description

Creates an Azure datalake.

Parameters

Type Name Schema

Body

input
required

CreateAzureDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CreateAzureDatalakeResponse

default

The default response on an error.

Error

Creates an GCP Data Lake.

POST /api/v1/datalake/createGCPDatalake

Description

Creates an GCP Data Lake.

Parameters

Type Name Schema

Body

input
required

CreateGCPDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

CreateGCPDatalakeResponse

default

The default response on an error.

Error

Deletes a datalake.

POST /api/v1/datalake/deleteDatalake

Description

Deletes a datalake.

Parameters

Type Name Schema

Body

input
required

DeleteDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

DeleteDatalakeResponse

default

The default response on an error.

Error

Gets external database server details.

POST /api/v1/datalake/describeDatabaseServer

Description

Gets external database server details for Data Lake by cluster CRN

Parameters

Type Name Schema

Body

input
required

DescribeDatabaseServerRequest

Responses

HTTP Code Description Schema

200

Database server details for the requested resource.

DescribeDatabaseServerResponse

default

The default response on an error.

Error

Describes a datalake.

POST /api/v1/datalake/describeDatalake

Description

Describes a datalake.

Parameters

Type Name Schema

Body

input
required

DescribeDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

DescribeDatalakeResponse

default

The default response on an error.

Error

Get cluster host status.

POST /api/v1/datalake/getClusterHostStatus

Description

Gets the status of the hosts in a cluster.

Parameters

Type Name Schema

Body

input
required

GetClusterHostStatusRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

GetClusterHostStatusResponse

default

The default response on an error.

Error

Get cluster service status.

POST /api/v1/datalake/getClusterServiceStatus

Description

Gets the status of the services in a cluster.

Parameters

Type Name Schema

Body

input
required

GetClusterServiceStatusRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

GetClusterServiceStatusResponse

default

The default response on an error.

Error

Gather Cloudera Manager roles that can be used for filtering in CM based diagnostics collection.

POST /api/v1/datalake/getCmRoles

Description

Gather Cloudera Manager roles that can be used for filtering in CM based diagnostics collection.

Parameters

Type Name Schema

Body

input
required

GetCmRolesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

GetCmRolesResponse

default

The default response on an error.

Error

Gather log descriptors that are used for diagnostics collection.

POST /api/v1/datalake/getDatalakeLogDescriptors

Description

Gather log descriptors that are used for diagnostics collection.

Parameters

Type Name Schema

Body

input
required

GetDatalakeLogDescriptorsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

GetDatalakeLogDescriptorsResponse

default

The default response on an error.

Error

List all the backup operations that were performed on the datalake.

POST /api/v1/datalake/listDatalakeBackups

Description

List all the backup operations that were performed on the datalake.

Parameters

Type Name Schema

Body

input
required

ListDatalakeBackupsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

ListDatalakeBackupsResponse

default

The default response on an error.

Error

List recent Datalake diagnostics collections

POST /api/v1/datalake/listDatalakeDiagnostics

Description

List recent Datalake diagnostics collection

Parameters

Type Name Schema

Body

input
required

ListDatalakeDiagnosticsRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

ListDatalakeDiagnosticsResponse

default

The default response on an error.

Error

Lists datalakes.

POST /api/v1/datalake/listDatalakes

Description

Lists datalakes.

Parameters

Type Name Schema

Body

input
required

ListDatalakesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

ListDatalakesResponse

default

The default response on an error.

Error

Lists the datalake versions.

POST /api/v1/datalake/listRuntimes

Description

Lists the available datalake runtime versions.

Parameters

Type Name Schema

Body

input
required

ListRuntimesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

ListRuntimesResponse

default

The default response on an error.

Error

Prepares the Data Lake cluster for upgrade.

POST /api/v1/datalake/prepareDatalakeUpgrade

Description

In order to reduce the chance of upgrade failures, we’re introducing a preparation phase for runtime upgrades. During this phase, we’re running all validations and downloading the required parcels for the machines. You can track the progress of the parcel preparation on the Cloudera Manager UI or you can check on the Management Console as well.

Parameters

Type Name Schema

Body

input
required

PrepareDatalakeUpgradeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

PrepareDatalakeUpgradeResponse

default

The default response on an error.

Error

Recover data lake to the original version after a failed upgrade.

POST /api/v1/datalake/recoverDatalake

Description

Recover data lake to the original version after a failed upgrade.

Parameters

Type Name Schema

Body

input
required

RecoverDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid recover data lake request.

RecoverDatalakeResponse

default

The default response on an error.

Error

Renew certificate on Datalake cluster by name or CRN

POST /api/v1/datalake/renewCertificate

Description

Deprecated, please use renew-public-certificate command instead. Renew certificate on Datalake cluster by name or CRN.

Parameters

Type Name Schema

Body

input
required

RenewCertificateRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RenewCertificateResponse

default

The default response on an error.

Error

Renews public certificate on Datalake cluster by name or CRN.

POST /api/v1/datalake/renewPublicCertificate

Description

Renews public certificate on Datalake cluster by name or CRN.

Parameters

Type Name Schema

Body

input
required

RenewPublicCertificateRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RenewPublicCertificateResponse

default

The default response on an error.

Error

Repairs a datalake.

POST /api/v1/datalake/repairDatalake

Description

Repairs a datalake.

Parameters

Type Name Schema

Body

input
required

RepairDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RepairDatalakeResponse

default

The default response on an error.

Error

Replaces recipes for the given instance groups.

POST /api/v1/datalake/replaceRecipes

Description

Replaces recipes for the given instance groups.

Parameters

Type Name Schema

Body

input
required

ReplaceRecipesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

ReplaceRecipesResponse

default

The default response on an error.

Error

Resizes the given datalake to the given target size.

POST /api/v1/datalake/resizeDatalake

Description

Resizes the datalake to the given size.

Parameters

Type Name Schema

Body

input
required

ResizeDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid resize datalake request.

ResizeDatalakeResponse

default

The default response on an error.

Error

Restore the datalake from backup taken.

POST /api/v1/datalake/restoreDatalake

Description

Restore the datalake from a backup that was taken. Backup to be used for restore is identified based on the information provided in the restore request. BackupId and BackupName are mutually exclusive. Only one of them can be provided. If both are provided, BackupId takes precedence.

Parameters

Type Name Schema

Body

input
required

RestoreDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RestoreDatalakeResponse

default

The default response on an error.

Error

Check the status of datalake restore operation.

POST /api/v1/datalake/restoreDatalakeStatus

Description

Identifies the restore operation based on the inputs provided and gets the status. RestoreId and BackupName are mutually exclusive. Only one of them can be provided.

Parameters

Type Name Schema

Body

input
required

RestoreDatalakeStatusRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RestoreDatalakeStatusResponse

default

The default response on an error.

Error

Retry last failed operation on a datalake.

POST /api/v1/datalake/retryDatalake

Description

Retry last failed operation on a datalake.

Parameters

Type Name Schema

Body

input
required

RetryDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RetryDatalakeResponse

default

The default response on an error.

Error

Rotate autotls certificates on the datalake’s hosts

POST /api/v1/datalake/rotateAutoTlsCertificates

Description

Deprecated, please use rotate-private-certificates command instead. Rotate autotls certificates on the datalake’s hosts

Parameters

Type Name Schema

Body

input
required

RotateAutoTlsCertificatesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RotateAutoTlsCertificatesResponse

default

The default response on an error.

Error

Rotates private certificates on the datalake’s hosts.

POST /api/v1/datalake/rotatePrivateCertificates

Description

Rotates private certificates on the datalake’s hosts.

Parameters

Type Name Schema

Body

input
required

RotatePrivateCertificatesRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RotatePrivateCertificatesResponse

default

The default response on an error.

Error

Rotate SaltStack user password on DataLake instances.

POST /api/v1/datalake/rotateSaltPassword

Description

Rotate SaltStack user password on DataLake instances.

Parameters

Type Name Schema

Body

input
required

RotateSaltPasswordRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

RotateSaltPasswordResponse

default

The default response on an error.

Error

Horizontally scale the Data Lake nodes, to add additional compute nodes to the cluster.

POST /api/v1/datalake/scaleHorizontally

Description

Horizontally scale the target host group in the Data Lake cluster. The possible option for the instance group are raz_scale_out, hms_scale_out, kafka_scale_out, solr_scale_out, storage_scale_out

Parameters

Type Name Schema

Body

input
required

ScaleHorizontallyRequest

Responses

HTTP Code Description Schema

200

Successful response to a valid Data Lake horizontal scaling request.

ScaleHorizontallyResponse

default

The default response on an error.

Error

Sets a catalog for a DataLake.

POST /api/v1/datalake/setCatalog

Description

Sets a catalog for a DataLake.

Parameters

Type Name Schema

Body

input
required

SetCatalogRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

SetCatalogResponse

default

The default response on an error.

Error

Upgrades the database of the Data Lake cluster.

POST /api/v1/datalake/startDatabaseUpgrade

Description

This command initiates the upgrade of the database of the Data Lake cluster.

Parameters

Type Name Schema

Body

input
required

StartDatabaseUpgradeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

StartDatabaseUpgradeResponse

default

The default response on an error.

Error

Start Datalake

POST /api/v1/datalake/startDatalake

Description

Deprecated, please use 'cdp environments start-environment' instead.

Parameters

Type Name Schema

Body

input
required

StartDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

StartDatalakeResponse

default

The default response on an error.

Error

Initiates the vertical scaling on Data Lake.

POST /api/v1/datalake/startDatalakeVerticalScaling

Description

Initiates the vertical scaling on Data Lake. You have the option to specify 'modifyDisks' or 'instanceTemplate' as parameter.

Parameters

Type Name Schema

Body

input
required

StartDatalakeVerticalScalingRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

StartDatalakeVerticalScalingResponse

default

The default response on an error.

Error

Stop Datalake

POST /api/v1/datalake/stopDatalake

Description

Deprecated, please use 'cdp environments stop-environment' instead.

Parameters

Type Name Schema

Body

input
required

StopDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

StopDatalakeResponse

default

The default response on an error.

Error

Syncs component versions from CM after a failed upgrade.

POST /api/v1/datalake/syncComponentVersionsFromCm

Description

Syncs component versions from CM after a failed upgrade.

Parameters

Type Name Schema

Body

input
required

SyncComponentVersionsFromCmRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid sync datahub CM component versions request.

SyncComponentVersionsFromCmResponse

default

The default response on an error.

Error

Run orchestrator engine state update on the Data Lake cluster.

POST /api/v1/datalake/updateOrchestratorState

Description

Run orchestrator engine state update on the Data Lake cluster.

Parameters

Type Name Schema

Body

input
required

UpdateOrchestratorStateRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

UpdateOrchestratorStateResponse

default

The default response on an error.

Error

OS or data platform upgrade for the SDX cluster.

POST /api/v1/datalake/upgradeDatalake

Description

OS or data platform upgrade for the SDX cluster. You have the option to either specify one of 'imageId', 'runtime' or 'lockComponents' or both 'imageId' and 'lockComponents' or none of the parameters.

Parameters

Type Name Schema

Body

input
required

UpgradeDatalakeRequest

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

UpgradeDatalakeResponse

default

The default response on an error.

Error

Validates AWS cloud storage.

POST /api/v1/datalake/validateAwsCloudStorage

Description

Validates AWS cloud storage for Data Lake.

Parameters

Type Name Schema

Body

input
required

ValidateAwsCloudStorageRequest

Responses

HTTP Code Description Schema

200

AWS cloud storage validation result for Data Lake.

ValidateAwsCloudStorageResponse

default

The default response on an error.

Error

Validates Azure cloud storage.

POST /api/v1/datalake/validateAzureCloudStorage

Description

Validates Azure cloud storage for Data Lake.

Parameters

Type Name Schema

Body

input
required

ValidateAzureCloudStorageRequest

Responses

HTTP Code Description Schema

200

Azure cloud storage validation result for Data Lake.

ValidateAzureCloudStorageResponse

default

The default response on an error.

Error

Definitions

AWSConfiguration

AWS configuration.

Name Description Schema

instanceProfile
optional

The instance profile used for the ID Broker instance.

string

AWSConfigurationRequest

Request object for AWS configuration.

Name Description Schema

instanceProfile
required

The ARN of an IAM instance profile.

string

storageBucketLocation
required

The location of the S3 bucket to be used as storage. The location has to start with s3a:// followed by the bucket name.

string

AdminOperationsBackupRestoreState

The state of Cloudera Manager admin operations.

Name Description Schema

precheckStoragePermission
required

Validate storage permissions before running a backup/restore.

BackupRestoreOperationStatus

rangerAuditCollectionValidation
required

Run the ranger audit collection validation in the backup precheck.

BackupRestoreOperationStatus

startServices
required

The status of the start services operation this is triggered after the backup/restore is complete.

BackupRestoreOperationStatus

stopServices
required

The status of the stop services operation that is triggered before the backup/restore is started.

BackupRestoreOperationStatus

AttachedVolumeDetail

The attached volume configuration.

Name Description Schema

count
optional

The number of volumes.

integer (int32)

size
optional

The size of each volume in GB.

integer (int32)

volumeType
optional

The type of volumes.

string

AzureConfiguration

Azure configuration.

Name Description Schema

managedIdentity
optional

The managed identity used for the ID Broker instance.

string

AzureConfigurationRequest

Request object for Azure configuration.

Name Description Schema

managedIdentity
required

The managed identity to use. The assumer should have Virtual Machine Contributor and Managed Identity Operator roles on subscription level.

string

storageLocation
required

The storage location to use. The location has to be in the following format abfs://filesystem@storage-account-name.dfs.core.windows.net. The filesystem must already exist and the storage account must be StorageV2.

string

BackupDatalakeRequest

Request object to perform a backup of datalake.

Name Description Schema

backupLocation
optional

Location where the back-up has to be stored. For example s3a://Location/of/the/backup.

string

backupName
optional

The name of the backup.

string

closeDbConnections
optional

Close the database connections while performing backup. Default is true.
Default : true

boolean

datalakeName
required

The name of the datalake.

string

skipAtlasIndexes
optional

Skips the backup of the Atlas indexes. If this option or –skipAtlasMetadata is not provided, Atlas indexes are backed up by default. Redundant if –skipAtlasMetadata is included.

boolean

skipAtlasMetadata
optional

Skips the backup of the Atlas metadata. If this option is not provided, the Atlas metadata is backed up by default.

boolean

skipRangerAudits
optional

Skips the backup of the Ranger audits. If this option is not provided, Ranger audits are backed up by default.

boolean

skipRangerHmsMetadata
optional

Skips the backup of the databases backing HMS/Ranger services. If this option is not provided, the HMS/Ranger services are backed up by default.

boolean

skipValidation
optional

Skips the validation steps that run prior to the backup. If this option is not provided, the validations are performed by default.

boolean

validationOnly
optional

Runs only the validation steps and then returns. If this option is not provided, the backup is performed as normal by default.

boolean

BackupDatalakeResponse

Response object to a request made for backup of datalake.

Name Description Schema

accountId
required

Account where the datalake exists.

string

backupId
required

Unique identifier for the backup requested.

string

backupLocation
required

Location of the backup to be used to perform restore.

string

backupName
optional

Name of the backup.

string

endTime
required

Time when the backup operation ended.

string

failureReason
optional

Reason for the failure.

string

internalState
required

Provides the details of the internal state where the backup operation stands.

string

operationStates
required

Provides the details of the internal state of each operation.

InternalBackupRestoreState

runtimeVersion
optional

The runtime version of the datalake when the backup was taken.

string

startTime
required

Time when the backup operation started.

string

status
required

Provide the current status.

string

userCrn
required

Crn of the user who triggered this operation.

string

BackupDatalakeStatusRequest

Request object to get the status of datalake backup. Returns the status of the latest backup that matches the provided input.

Name Description Schema

backupId
optional

Unique identifier of the backup performed. When provided, the status request will get the status entry that has the backupid provided.

string

backupName
optional

The name of the backup. When provided, the status request will get the status of the latest backup performed with the given backup name on the given datalake.

string

datalakeName
required

The name of the datalake. When backupName and backupId are not provided, status request will get the status of the latest backup operation performed on the given datalake.

string

BackupDatalakeStatusResponse

Response object to get the status of datalake backup status request.

Name Description Schema

accountId
required

Account where the datalake exists.

string

backupId
required

Unique identifier for the backup requested.

string

backupLocation
required

Location of the backup to be used to perform restore.

string

backupName
optional

Name of the backup.

string

endTime
required

Time when the backup operation ended.

string

failureReason
optional

Reason for the failure.

string

internalState
required

Provides the details of the internal state where the backup operation stands.

string

operationStates
required

Provides the details of the internal state of each operation.

InternalBackupRestoreState

runtimeVersion
optional

The runtime version of the datalake when the backup was taken.

string

startTime
required

Time when the backup operation started.

string

status
required

Provide the current status.

string

totalPredictedDurationInMinutes
optional

Predicted total duration for the backup process, in minutes.

string

userCrn
required

Crn of the user who triggered this operation.

string

BackupRestoreOperationStatus

Contains the status and failure reason of an operation.

Name Description Schema

durationInMinutes
optional

The duration of each operation, in minutes.

string

failureReason
optional

The failure reason if the operation was not successful.

string

predictedDurationInMinutes
optional

Predicted duration of the operation, in minutes.

string

status
required

The status of the backup or restore operation.

string

CancelBackupRequest

The request to cancel a backup operation.

Name Description Schema

backupId
required

Backup-id that identifies the backup to be cancelled.

string

force
optional

Will mark all unfinished operations as cancelled without waiting for their result and mark backup operation as finished (cancelled, successful or failed) immediately.
Default : false

boolean

CancelBackupResponse

Response to the cancel backup request.

Name Description Schema

result
required

Result of the cancel backup request. It can contain a description of the current process state or guidelines to a customer on the next steps.

string

CancelDatalakeDiagnosticsRequest

Request object for cancel running Datalake diagnostics collections.

Name Description Schema

crn
required

CRN of the Datalake cluster.

string

CancelDatalakeDiagnosticsResponse

Response object for cancel running Datalake diagnostics collections.

Type : object

ClouderaManagerDetails

Cloudera Manager details.

Name Description Schema

clouderaManagerRepositoryURL
required

Cloudera Manager repository URL.

string

clouderaManagerServerURL
optional

Cloudera Manager server URL.

string

version
required

Cloudera Manager version.

string

CollectCmDiagnosticsRequest

Request object for collecting DataLake diagnostics.

Name Description Schema

bundleSizeLimit
optional

Diagnostics bundle size limit in MB.

integer (int32)

caseNumber
optional

Optional support case number in case of SUPPORT destination, otherwise only act as additional data.

string

crn
required

CRN of the Datalake cluster.

string

description
optional

Additional information / title for the diagnostics collection.

string

destination
required

Destination of the diagnostics collection (Support, Own cloud storage, Engineering or collect only on the nodes)

enum (SUPPORT, CLOUD_STORAGE, LOCAL)

endDate
optional

Restrict collected logs and metrics (until the provided date timestamp).

string (date-time)

monitorMetricsCollection
optional

Flag to enable collection of metrics for chart display in CM based diagnostics collection.
Default : false

boolean

roles
optional

Array of roles for which to get logs and metrics. If set, this restricts the roles for log and metrics collection.

< string > array

startDate
optional

Restrict collected logs and metrics (from the provided date timestamp).

string (date-time)

storageValidation
optional

Enable/disable node level storage validation (can be disabled for example, if you have too many hosts and do not want to do too much parallel writes to s3/abfs)
Default : false

boolean

updatePackage
optional

If enabled, required package (cdp-telemetry) will be upgraded or installed on the nodes. (useful if package is not installed or needs to be upgraded) Network is required for this operation.
Default : false

boolean

CollectCmDiagnosticsResponse

Response object for collecting DataLake diagnostics.

Type : object

CollectDatalakeDiagnosticsRequest

Request object for collecting DataLake diagnostics.

Name Description Schema

additionalLogs
optional

Array of log descriptors that should be additionally collected during diagnostics collection.

< DatalakeVmLogRequest > array

caseNumber
optional

Optional support case number in case of SUPPORT destination, otherwise only act as additional data.

string

crn
required

CRN of the Datalake cluster.

string

description
required

Additional information / title for the diagnostics collection.

string

destination
required

Destination of the diagnostics collection (Support, Own cloud storage, Engineering or collect only on the nodes)

enum (SUPPORT, CLOUD_STORAGE, ENG, LOCAL)

endDate
optional

Date timestamp - collect files only for diagnostics that has lower created timestamp value than this.

string (date-time)

excludeHosts
optional

Array of host names (fqdn or IP address), collection will not run on the excluded hosts.

< string > array

hostGroups
optional

Array of host groups, collection will run only on the dedicated hosts that belongs to these host groups.

< string > array

hosts
optional

Array of host names (fqdn), collection will run only on the dedicated hosts.

< string > array

includeNginxReport
optional

Include Nginx report generated by GoAccess (if available).

boolean

includeSaltLogs
optional

Include salt minion/master/api system logs in the diagnostics collection.

boolean

includeSarOutput
optional

Include SAR (System Activity Report) generated outputs in the diagnostics collection (if available).

boolean

labels
optional

Array of labels that can filter logs that are collected during diagnostics collection.

< string > array

skipUnresponsiveHosts
optional

Skip unhealthy hosts from the diagnostics collection.

boolean

startDate
optional

Date timestamp - collect files only for diagnostics that has higher last modified timestamp value than this.

string (date-time)

storageValidation
optional

Enable/disable node level storage validation (can be disabled for example, if you have too many hosts and do not want to do too much parallel writes to s3/abfs)
Default : false

boolean

updatePackage
optional

If enabled, required package (cdp-telemetry) will be upgraded or installed on the nodes. (useful if package is not installed or needs to be upgraded) Network is required for this operation.
Default : false

boolean

CollectDatalakeDiagnosticsResponse

Response object for collecting DataLake diagnostics.

Type : object

CreateAWSDatalakeRequest

Request object for create AWS datalake request.

Name Description Schema

cloudProviderConfiguration
required

AWS configuration.

AWSConfigurationRequest

datalakeName
required

The datalake name. This name must be unique, must have between 5 and 100 characters, and must contain only lowercase letters, numbers and hyphens. Names are case-sensitive.
Length : 5 - 100

string

enableRangerRaz
optional

Whether to enable Ranger RAZ for the datalake. Defaults to not being enabled.

boolean

enableRangerRms
optional

Whether to enable Ranger RMS for the datalake. Defaults to not being enabled.

boolean

environmentName
required

The environment name or CRN.

string

image
optional

The image to use for the datalake. This must not be set if the runtime parameter is provided.

ImageRequest

javaVersion
optional

Configure the major version of Java on the cluster.

integer (int32)

multiAz
optional

Controls if the datalake is deployed in a multi-availability zone way.

boolean

recipes
optional

Additional recipes that will be attached on the datalake instances (by instance groups, most common ones are like 'master' or 'idbroker').

< InstanceGroupRecipeRequest > array

runtime
optional

Cloudera Runtime version.

string

scale
optional

The scale of the datalake. Allowed values are "LIGHT_DUTY" or "MEDIUM_DUTY_HA". Defaults to "LIGHT_DUTY" if not set.

DatalakeScaleType

tags
optional

Tags to be added to Data Lake related resources.

< DatalakeResourceTagRequest > array

CreateAWSDatalakeResponse

Response object for create AWS datalake request.

Name Description Schema

datalake
required

The datalake.

Datalake

CreateAWSGovCloudDatalakeRequest

Request object for createing AWS Data Lake request on GovCloud.

Name Description Schema

cloudProviderConfiguration
required

AWS configuration.

AWSConfigurationRequest

datalakeName
required

The Data Lake name. This name must be unique, must have between 5 and 100 characters, and must contain only lowercase letters, numbers and hyphens. Names are case-sensitive.
Length : 5 - 100

string

enableRangerRaz
optional

Whether to enable Ranger RAZ for the datalake. Defaults to not being enabled.

boolean

environmentName
required

The environment name or CRN.

string

image
optional

The image to use for the datalake. This must not be set if the runtime parameter is provided.

ImageRequest

javaVersion
optional

Configure the major version of Java on the cluster.

integer (int32)

multiAz
optional

Controls if the datalake is deployed in a multi-availability zone way.

boolean

recipes
optional

Additional recipes that will be attached on the datalake instances (by instance groups, most common ones are like 'master' or 'idbroker').

< InstanceGroupRecipeRequest > array

runtime
optional

Cloudera Runtime version.

string

scale
optional

The scale of the datalake. Allowed values are "LIGHT_DUTY" or "MEDIUM_DUTY_HA". Defaults to "LIGHT_DUTY" if not set.

DatalakeScaleType

tags
optional

Tags to be added to Data Lake related resources.

< DatalakeResourceTagRequest > array

CreateAWSGovCloudDatalakeResponse

Response object for creating AWS Data Lake request on GovCloud.

Name Description Schema

datalake
required

The Data Lake.

Datalake

CreateAzureDatalakeRequest

Request object for create Azure datalake request.

Name Description Schema

cloudProviderConfiguration
required

Azure configuration.

AzureConfigurationRequest

databaseType
optional

The type of the azure database. FLEXIBLE_SERVER is the next generation managed PostgreSQL service in Azure that provides maximum flexibility over your database, built-in cost-optimizations. SINGLE_SERVER is a fully managed database service with minimal requirements for customizations of the database.

enum (FLEXIBLE_SERVER, SINGLE_SERVER)

datalakeName
required

The datalake name. This name must be unique, must have between 5 and 100 characters, and must contain only lowercase letters, numbers and hyphens. Names are case-sensitive.
Length : 5 - 100

string

enableRangerRaz
optional

Whether to enable Ranger RAZ for the datalake. Defaults to not being enabled.

boolean

environmentName
required

The environment name or CRN.

string

flexibleServerDelegatedSubnetId
optional

This argument allows you to specify the subnet ID for the subnet within which you want to configure your Azure Flexible Server.

string

image
optional

The image to use for the datalake. This must not be set if the runtime parameter is provided.

ImageRequest

javaVersion
optional

Configure the major version of Java on the cluster.

integer (int32)

loadBalancerSku
optional

The SKU for the datalake load balancer. Allowed values are "BASIC", "STANDARD", or "NONE".

DatalakeLoadBalancerSkuType

multiAz
optional

Creates CDP datalake distributed across multiple availability zones in an Azure region.
Default : false

boolean

recipes
optional

Additional recipes that will be attached on the datalake instances (by instance groups, most common ones are like 'master' or 'idbroker').

< InstanceGroupRecipeRequest > array

runtime
optional

Cloudera Runtime version.

string

scale
optional

The scale of the datalake. Allowed values are "LIGHT_DUTY" or "MEDIUM_DUTY_HA". Defaults to "LIGHT_DUTY" if not set.

DatalakeScaleType

tags
optional

Tags to be added to Data Lake related resources.

< DatalakeResourceTagRequest > array

CreateAzureDatalakeResponse

Response object for create Azure datalake request.

Name Description Schema

datalake
required

The datalake.

Datalake

CreateGCPDatalakeRequest

Request object for create GCP Data Lake request.

Name Description Schema

cloudProviderConfiguration
required

GCP configuration.

GCPConfigurationRequest

datalakeName
required

The datalake name. This name must be unique, must have between 5 and 100 characters, and must contain only lowercase letters, numbers and hyphens. Names are case-sensitive.
Length : 5 - 100

string

environmentName
required

The environment name or CRN.

string

image
optional

The image to use for the datalake. This must not be set if the runtime parameter is provided.

ImageRequest

javaVersion
optional

Configure the major version of Java on the cluster.

integer (int32)

recipes
optional

Additional recipes that will be attached on the datalake instances (by instance groups, most common ones are like 'master' or 'idbroker').

< InstanceGroupRecipeRequest > array

runtime
optional

Cloudera Runtime version.

string

scale
optional

The scale of the datalake. Allowed values are "LIGHT_DUTY" or "MEDIUM_DUTY_HA". Defaults to "LIGHT_DUTY" if not set.

DatalakeScaleType

tags
optional

Tags that can be attached to GCP Data Lake resources. Please refer to Google documentation for the rules https://cloud.google.com/compute/docs/labeling-resources#label_format.

< DatalakeResourceGCPTagRequest > array

CreateGCPDatalakeResponse

Response object for create GCP Data Lake request.

Name Description Schema

datalake
required

The datalake.

Datalake

DatabaseBackupRestoreState

The state of the database backup/restore operation.

Name Description Schema

database
required

The status of the database backup/restore.

BackupRestoreOperationStatus

DatabaseServerSslConfig

SSL configuration for the database server

Name Description Schema

sslCertificateType
optional

SSL certificate type

enum (CLOUD_PROVIDER_OWNED, BRING_YOUR_OWN, NONE)

sslCertificates
optional

Set of SSL certificates for the actual database server

< string > array

sslMode
optional

SSL enforcement mode for the actual database server

enum (ENABLED, DISABLED)

Datalake

Information about a datalake.

Name Description Schema

certificateExpirationState
optional

Indicates the certificate status on the cluster.

enum (VALID, HOST_CERT_EXPIRING)

creationDate
optional

The date when the datalake was created.

string (date-time)

crn
required

The CRN of the datalake.

string

datalakeName
required

The name of the datalake.

string

enableRangerRaz
optional

Whether Ranger RAZ is enabled for the datalake.

boolean

environmentCrn
optional

The CRN of the environment.

string

multiAz
optional

Flag which marks that the datalake is deployed in a multi-availability zone way or not.

boolean

status
optional

The status of the datalake.

string

statusReason
optional

The reason for the status of the datalake.

string

DatalakeBackup

Backup object with details of backup performed.

Name Description Schema

accountId
required

Account where the datalake exists.

string

backupId
required

Request to perform a from a specific status for a specific backup. Else, status of the last backup performed will be returned.

string

backupLocation
required

Location of the backup to be used to perform restore.

string

backupName
required

Name of the backup.

string

endTime
required

Time when the backup operation ended.

string

failureReason
required

Reason for the failure.

string

internalState
required

Provides the details of the internal state where the backup operation stands.

string

startTime
required

Time when the backup operation started.

string

status
required

Provide the current status.

string

userCrn
required

Crn of the user who triggered this operation.

string

DatalakeDetails

Details about a datalake

Name Description Schema

awsConfiguration
optional

The AWS configuration.

AWSConfiguration

azureConfiguration
optional

The Azure configuration.

AzureConfiguration

cloudPlatform
optional

The cloud platform.

string

cloudStorageBaseLocation
optional

The base location for the cloud storage used for the datalake.

string

clouderaManager
optional

The Cloudera Manager details.

ClouderaManagerDetails

creationDate
optional

The date when the datalake was created.

string (date-time)

credentialCrn
optional

The CRN of credentials.

string

crn
required

The CRN of the datalake.

string

datalakeName
required

The name of the datalake.

string

enableRangerRaz
optional

Whether Ranger RAZ is enabled for the datalake.

boolean

endpoints
optional

The exposed service api endpoints.

Endpoints

environmentCrn
optional

The CRN of the environment.

string

gcpConfiguration
optional

The GCP configuration.

GCPConfiguration

instanceGroups
optional

The instance details.

< InstanceGroup > array

productVersions
optional

The product versions.

< ProductVersion > array

region
optional

The region of the datalake.

string

shape
optional

The shape of the datalake (either LIGHT_DUTY or MEDIUM_DUTY_HA).

DatalakeScaleType

status
optional

The status of the datalake.

string

statusReason
optional

The reason for the status of the datalake.

string

DatalakeDiagnosticsCollectionDetailsResponse

Response object for diagnostic collection collection details.

Name Description Schema

accountId
optional

Account Id that was used for the diagnostics collection.

string

case
optional

Case number for the diagnostics collection.

string

clusterVersion
optional

Version of the cluster that was used for the diagnostics collection.

string

description
optional

Description of the diagnostics collection.

string

destination
optional

Destination type of the diagnostics collection.

string

output
optional

Output destination of the diagnostics collection.

string

resourceCrn
optional

Crn of the cluster.

string

DatalakeDiagnosticsCollectionResponse

Response object for diagnostic collection flow details.

Name Description Schema

collectionDetails
optional

Additional details about the diagnostics collection.

DatalakeDiagnosticsCollectionDetailsResponse

created
optional

Creation date of the diagnostics collection flow.

string (date-time)

flowId
optional

Flow ID of the diagnostics collection flow.

string

flowState
optional

Current state of the diagnostics collection flow.

string

progressPercentage
optional

Progress percentage of the diagnostics collection flow (maximum value if finished).

integer (int32)

status
optional

Status of the diagnostics collection flow.

enum (RUNNING, FAILED, FINISHED, CANCELLED)

DatalakeInstanceStatus

The status of the instance.

Type : enum (REQUESTED, FAILED, CREATED, ORCHESTRATION_FAILED, SERVICES_RUNNING, SERVICES_HEALTHY, SERVICES_UNHEALTHY, WAITING_FOR_REPAIR, STOPPED, DELETED_ON_PROVIDER_SIDE, DELETED_BY_PROVIDER, DELETE_REQUESTED, DECOMMISSIONED, DECOMMISION_FAILED, TERMINATED)

DatalakeInstanceType

The type of the instance.

Type : enum (GATEWAY, GATEWAY_PRIMARY, CORE)

DatalakeLoadBalancerSkuType

Represents the Azure load balancer SKU type. The current default is BASIC. To disable the load balancer, use type NONE.

Type : enum (BASIC, STANDARD, NONE)

DatalakeResourceGCPTagRequest

A label that can be attached to GCP Data Lake resources. Please refer to Google documentation for the rules https://cloud.google.com/compute/docs/labeling-resources#label_format.

Name Description Schema

key
required

The key of tag.

string

value
required

The value of the tag.

string

DatalakeResourceTagRequest

Tag for a datalake resource.

Name Description Schema

key
required

The key of tag.

string

value
required

The value of the tag.

string

DatalakeScaleType

Represents the available datalake scales. Defaults to LIGHT_DUTY if not set.

Type : enum (LIGHT_DUTY, MEDIUM_DUTY_HA, ENTERPRISE)

DatalakeVmLogRequest

Log descriptor, contains a path and label pair. Used for diagnostics collections.

Name Description Schema

label
required

Label that will be used to identify a log descriptor. (will be used as a folder inside logs folder)

string

path
required

Path of the log file(s) that needs to be collected. (Can be glob wildcard)

string

DatalakeVmLogResponse

Log descriptor, contains a path and label pair. Used for diagnostics collections.

Name Description Schema

label
optional

Label that will be used to identify a log descriptor. (will be used as a folder inside logs folder)

string

path
optional

Path of the log file(s) that needs to be collected. (Can be glob wildcard)

string

DeleteDatalakeRequest

Request object for delete datalake request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake to be deleted.

string

force
optional

Whether the datalake should be force deleted. This option can be used when cluster deletion fails. This removes the entry from Cloudera Datalake service. Any lingering resources have to be deleted from the cloud provider manually. The default is false.

boolean

DeleteDatalakeResponse

Response object for delete datalake request.

Type : object

DescribeDatabaseServerRequest

Request object for obtaining Database Server details.

Name Description Schema

clusterCrn
optional

Stack CRN of the Data Lake

string

DescribeDatabaseServerResponse

Response object for obtaining Database Server details.

Name Description Schema

clusterCrn
optional

CRN of the cluster of the database server

string

creationDate
optional

Creation date of the database server

string (date-time)

crn
optional

CRN of the database server

string

databaseVendor
optional

Name of the database vendor (MYSQL, POSTGRES, …)

string

databaseVendorDisplayName
optional

Display name of the database vendor (MySQL, PostgreSQL, …)

string

description
optional

Description of the database server

string

environmentCrn
optional

CRN of the environment of the database server

string

host
optional

Host of the database server

string

name
optional

Name of the database server

string

port
optional

Port of the database server

integer (int32)

resourceStatus
optional

Ownership status of the database server

enum (UNKNOWN, SERVICE_MANAGED, USER_MANAGED)

sslConfig
optional

SSL configuration of the database server

DatabaseServerSslConfig

status
optional

Status of the database server stack

enum (REQUESTED, CREATE_IN_PROGRESS, AVAILABLE, UPDATE_IN_PROGRESS, UPDATE_REQUESTED, UPDATE_FAILED, CREATE_FAILED, ENABLE_SECURITY_FAILED, DELETE_REQUESTED, PRE_DELETE_IN_PROGRESS, DELETE_IN_PROGRESS, DELETE_FAILED, DELETE_COMPLETED, STOPPED, STOP_REQUESTED, START_REQUESTED, STOP_IN_PROGRESS, START_IN_PROGRESS, START_FAILED, STOP_FAILED, WAIT_FOR_SYNC, MAINTENANCE_MODE_ENABLED, UNKNOWN)

statusReason
optional

Additional status information about the database server stack

string

DescribeDatalakeRequest

Request object for describe datalake request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

DescribeDatalakeResponse

Response object for describe datalake request.

Name Description Schema

datalake
required

The datalake.

DatalakeDetails

Endpoint

Object which holds the exposed endpoint.

Name Description Schema

displayName
required

The more consumable name of the exposed service.

string

knoxService
required

The related knox entry.

string

mode
required

The SSO mode of the given service.

string

open
required

The access status of the given endpoint. Whether its open or not.

boolean

serviceName
required

The name of the exposed service

string

serviceUrl
required

The server url for the given exposed service’s API.

string

Endpoints

Object which holds the exposed endpoints for the given cluster.

Name Description Schema

endpoints
required

The exposed API endpoints.

< Endpoint > array

Error

An object returned on an error.

Name Description Schema

code
optional

The error code.

string

message
optional

The error message.

string

GCPConfiguration

GCP configuration.

Name Description Schema

serviceAccountEmail
optional

The email id of the service account used for the ID Broker instance.

string

GCPConfigurationRequest

Request object for GCP configuration.

Name Description Schema

serviceAccountEmail
required

Email id of the service account to be associated with the datalake IdBroker instance. This service account should have "token.creator" role for one or more storage accounts that has access to storage.

string

storageLocation
required

The location of the GCS bucket to be used as storage. The location has to start with gs:// followed by the bucket name.

string

GetClusterHostStatusRequest

Request object to get host status.

Name Description Schema

clusterName
required

The name or CRN of the cluster.

string

GetClusterHostStatusResponse

Response object for getting host status.

Name Description Schema

hosts
required

The cluster hosts status.

< HostStatus > array

GetClusterServiceStatusRequest

Request object to get service status.

Name Description Schema

clusterName
required

The name or CRN of the cluster.

string

GetClusterServiceStatusResponse

Response object to get service status.

Name Description Schema

services
required

The cluster services health.

< ServiceStatus > array

GetCmRolesRequest

Request object for obtaining Cloudera Manger roles. (that can be used for filtering in CM based diagnostics collection)

Name Description Schema

crn
required

CRN of the Datalake cluster.

string

GetCmRolesResponse

Response object for obtaining Cloudera Manger roles. (that can be used for filtering in CM based diagnostics collection)

Name Description Schema

roles
optional

Array of CM roles. (useful for CM based diagnostics collection filtering)

< string > array

GetDatalakeLogDescriptorsRequest

Request object for obtaining log descriptors. (that are used for diagnostics collection)

Type : object

GetDatalakeLogDescriptorsResponse

Response object for obtaining log descriptors. (useful for diagnostics collection filtering)

Name Description Schema

logs
optional

Array of log descriptors. (useful for diagnostics collection filtering)

< DatalakeVmLogResponse > array

HbaseBackupRestoreState

The state of each HBase backup/restore operation.

Name Description Schema

atlasEntityAuditEventTable
required

The status of the ATLAS_ENTITY_AUDIT_EVENTS_TABLE backup/restore.

BackupRestoreOperationStatus

atlasJanusTable
required

The status of the ATLAS_JANUS_TABLE backup/restore.

BackupRestoreOperationStatus

HealthCheck

The result of a health check.

Name Description Schema

name
optional

The name of service health check.

string

summary
optional

The service health check summary.

string

HostStatus

Information about cluster host status.

Name Description Schema

healthSummary
optional

The host health summary.

string

hostid
optional

Unique identifier of the cluster host given by Cloudera Manager.

string

hostname
optional

The cluster hostname.

string

ImageComponentVersions

Versions of the image components

Name Description Schema

cdp
optional

CDP version

string

cdpGBN
optional

CDP GBN

string

cm
optional

Version of CM

string

cmGBN
optional

GBN of CM

string

os
optional

OS name

string

osPatchLevel
optional

OS patch level

string

ImageInfo

Basic information about an image.

Name Description Schema

componentVersions
optional

Version of the image components

ImageComponentVersions

created
optional

Image creation timestamp

integer (int64)

imageCatalogName
optional

Name of the image catalog, the image is from.

string

imageId
optional

The id of the image.

string

imageName
optional

The name of the image.

string

ImageRequest

The image request for the datalake. When the 'runtime' parameter is set, only the 'os' parameter can be provided. Otherwise, you can use 'catalog name' and/or 'id' for selecting an image.

Name Description Schema

catalogName
optional

The name of the custom image catalog to use, defaulting to 'cdp-default' if not present.
Default : "cdp-default"

string

id
optional

The image ID from the catalog. The corresponding image will be used for the created cluster machines.

string

os
optional

The OS of the image used for cluster instances.

string

Instance

Object which holds some details of an instance for the given cluster.

Name Description Schema

attachedVolumes
optional

List of volumes attached to this instance.

< AttachedVolumeDetail > array

availabilityZone
optional

The availability zone of the instance.

string

clouderaManagerServer
optional

Whether the instance has Cloudera Manager deployed or not.

boolean

discoveryFQDN
optional

The FQDN of the instance.

string

id
required

The ID of the given instance.

string

instanceGroup
optional

The name of the instance group this instance belongs to.

string

instanceStatus
optional

The status of the instance.

DatalakeInstanceStatus

instanceTypeVal
optional

The instance type.

DatalakeInstanceType

instanceVmType
optional

The VM type of the instance. Supported values depend on the cloud platform.

string

privateIp
optional

The private IP of the given instance.

string

publicIp
optional

The public IP of the given instance.

string

rackId
optional

The rack ID of the instance in Cloudera Manager.

string

sshPort
optional

The SSH port for the instance.

integer (int32)

state
required

The actual state of the instance.

string

statusReason
optional

The reason for the current status of this instance.

string

subnetId
optional

The subnet ID of the instance.

string

InstanceGroup

The type of the instance group which also contains the actual instance(s)

Name Description Schema

availabilityZones
optional

List of availability zones that this instance group is associated with.

< string > array

instances
required

List of instances in this instance group.

< Instance > array

name
required

The name of the instance group.

string

recipes
optional

The recipe names for the Datalake cluster.

< string > array

InstanceGroupRecipeRequest

Represents an instance group - recipe pair (by names) request.

Name Description Schema

instanceGroupName
required

A valid instance/host group name. This name must be unique.

string

recipeNames
optional

Names of the recipes (empty is valid).

< string > array

InstanceGroupRecipeResponse

Represents an instance group - recipe pair (by names) response.

Name Description Schema

instanceGroupName
required

A valid instance/host group name. This name must be unique.

string

recipeNames
optional

Names of the recipes (empty is valid).

< string > array

InstanceTemplate

Instance template that specifies the core information for the vertical scale.

Name Description Schema

instanceType
optional

The type of the instance.

string

InternalBackupRestoreState

Object representing the state of each service running a backup.

Name Description Schema

adminOperations
required

Admin operations executed via Cloudera Manager for starting and stopping services.

AdminOperationsBackupRestoreState

database
required

Postgres operations to backup or restore the data lake metadata.

DatabaseBackupRestoreState

hbase
required

Hbase operations to backup or restore Atlas metadata tables.

HbaseBackupRestoreState

solr
required

Solr operations to backup up the edge, fulltext, ranger_audit, and vertex tables, or to delete/restore those tables.

SolrBackupRestoreState

ListDatalakeBackupsRequest

Request object to get the list of backup operations performed on a datalake.

Name Description Schema

datalakeName
required

The name of the datalake.

string

ListDatalakeBackupsResponse

Response object with the list of backup operations performed on a datalake.

Name Description Schema

backups
optional

Backup object with details of backup performed.

< DatalakeBackup > array

ListDatalakeDiagnosticsRequest

Request object for listing recent Datalake diagnostics collections.

Name Description Schema

crn
required

CRN of the Datalake cluster.

string

ListDatalakeDiagnosticsResponse

Response object for listing recent Datalake diagnostics collections.

Name Description Schema

collections
optional

description.

< DatalakeDiagnosticsCollectionResponse > array

ListDatalakesRequest

Request object for list datalakes request.

Name Description Schema

datalakeName
optional

The name or CRN of the datalake for which details are requested.

string

environmentName
optional

The name or CRN of the environment for which the datalakes will be listed.

string

ListDatalakesResponse

Response object for list datalakes request.

Name Description Schema

datalakes
required

The datalakes.

< Datalake > array

ListRuntimesRequest

Request object for list datalake runtime versions.

Type : object

ListRuntimesResponse

Response object for list datalake runtime versions.

Name Description Schema

versions
required

The list of datalake runtime versions.

< Runtime > array

PrepareDatalakeUpgradeRequest

Request object to prepare Data Lake upgrade. This command indicates the upgrade preparation for a specific image or a selected runtime version. Important to note that the imageId or the runtime parameter must be present in the request.

Name Description Schema

datalake
required

The name or CRN of the Data Lake cluster.

string

imageId
optional

The ID of an image to upgrade to

string

runtime
optional

The runtime version to upgrade to

string

PrepareDatalakeUpgradeResponse

Response object for prepare Data Lake upgrade request.

Name Description Schema

current
optional

Information about the current image

ImageInfo

reason
optional

The reason why upgrade is not possible

string

upgradeCandidates
optional

List of images and components to upgrade to

< ImageInfo > array

ProductVersion

Product version.

Name Description Schema

name
required

The name of the product.

string

version
required

The version of the product.

string

RecoverDatalakeRequest

Datalake recover request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

recoveryType
optional

The type of the recovery. The default value is RECOVER_WITHOUT_DATA. The recovery always runs with RECOVER_WITH_DATA if the on resize failure.

enum (RECOVER_WITH_DATA, RECOVER_WITHOUT_DATA)

RecoverDatalakeResponse

Datalake recover response.

Type : object

RenewCertificateRequest

Request object for renew datalake certificate request, deprecated.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

RenewCertificateResponse

Response object for renew certificate request, deprecated.

Type : object

RenewPublicCertificateRequest

Request object to renew the public certificate for Datalake.

Name Description Schema

datalake
required

The name or CRN of the datalake.

string

RenewPublicCertificateResponse

Response object for renew public certificate request.

Type : object

RepairDatalakeRequest

Request object for repair datalake request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

instanceGroupName
optional

The instance group where the failed instances will be repaired.

string

instanceGroupNames
optional

List of instance groups where the failed instances will be repaired.

< string > array

instances
optional

List of instances.

RepairInstancesRequest

RepairDatalakeResponse

Response object for repair datalake request.

Type : object

RepairInstancesRequest

Details for repair instances.

Name Description Schema

instanceIds
optional

List of instance ids.

< string > array

ReplaceRecipesRequest

The request for replacing recipes.

Name Description Schema

datalake
required

The name or CRN of the datalake.

string

instanceGroupRecipes
required

The list of instance group and recipe name pairs.

< InstanceGroupRecipeRequest > array

ReplaceRecipesResponse

The response for replacing recipes.

Name Description Schema

attachedRecipes
optional

The list of recipes, which will be attached to the cluster.

< InstanceGroupRecipeResponse > array

detachedRecipes
optional

The list of recipes, which will be detached from the cluster.

< InstanceGroupRecipeResponse > array

ResizeDatalakeRequest

Datalake resize request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

multiAz
optional

Whether to deploy a new datalake in a multi-availability zone way.
Default : false

boolean

targetSize
required

The target size for the datalake. The resize target size can be MEDIUM_DUTY or ENTERPRISE. If the runtime version >= 7.2.17 target size is ENTERPRISE. If not, the target size is MEDIUM_DUTY.

enum (MEDIUM_DUTY_HA, ENTERPRISE)

ResizeDatalakeResponse

Datalake resize response.

Name Description Schema

cloudbreakVersion
optional

The Cloudbreak version used to create the data lake.

string

datalakeCrn
optional

The CRN of the datalake. This will remain the same for the datalake created by the resize process.

string

datalakeName
optional

The name of the datalake. This will remain the same for the datalake created by the resize process.

string

environmentCrn
optional

The CRN of the environment the datalake is attached to. This will remain the same for the datalake created by the resize process.

string

runtime
optional

The runtime of the datalake. This will remain the same for the datalake created by the resize process.

string

status
optional

The current status of the datalake.

string

statusReason
optional

The reason for the current status of the datalake.

string

RestoreDatalakeRequest

Request to restore datalake from backup. Restore does not restore the database by default.

Name Description Schema

backupId
optional

The ID of the backup to be used to perform a restore. The ID could refer to backup of any datalake in the same account. This is the only way to restore from a backup that was taken on a datalake with a different name. This field is required if the –backup-location-override field is used.

string

backupLocationOverride
optional

Backup location. When provided, will be used to lookup the backup. If provided, the –backup-id parameter is required.

string

backupName
optional

The name of the backup. When provided, the restore will be performed using the latest successful backup whose name matches the parameter, and that was taken from the datalake that is being restored.

string

datalakeName
required

The name of the datalake to be restored. When backupId is not provided, the most recent successful backup on datalake with the provided name would be used.

string

includeDatabase
optional

DEPRECATED - The database is included in the restore by default. To skip it, use the –skip-ranger-hms-metadata flag.

boolean

skipAtlasIndexes
optional

Skips the restore of the Atlas indexes. If this option or –skipAtlasMetadata is not provided, then by default the Atlas indexes will be restored if the backup used includes the Atlas indexes. Redundant if –skipAtlasMetadata is included.

boolean

skipAtlasMetadata
optional

Skips the restore of the Atlas metadata. If this option is not provided, then by default the Atlas metadata will be restored if the backup used includes the Atlas metadata.

boolean

skipRangerAudits
optional

Skips the restore of the Ranger audits. If this option is not provided, then by default the Ranger audits will be restored if the backup used includes the Ranger audits.

boolean

skipRangerHmsMetadata
optional

Skips the restore of the databases backing HMS/Ranger services. If this option is not provided, then by default the Atlas lineage will be restored if the backup used includes the Atlas lineage information.

boolean

skipValidation
optional

Skips the validation steps that run prior to the restore. If this option is not provided, the validations are performed by default.

boolean

validationOnly
optional

Runs only the validation steps and then returns. If this option is not provided, the restore is performed as normal by default.

boolean

RestoreDatalakeResponse

Response object to a request to perform restore of datalake.

Name Description Schema

accountId
required

Account where the datalake exists.

string

backupId
required

Unique identifier of the backup used to perform restore.

string

backupLocation
required

Location of the backup to be used to perform restore.

string

backupName
optional

Name of the backup.

string

endTime
required

Time when the backup operation ended.

string

failureReason
optional

Reason for the failure.

string

internalState
required

Provides the details of the internal state where the restore operation stands.

string

operationStates
required

Provides the details of the internal state of each operation.

InternalBackupRestoreState

restoreId
required

Unique identifier of the restore operation performed.

string

runtimeVersion
optional

The runtime version of the datalake when the restore was initiated.

string

startTime
required

Time when the backup operation started.

string

status
required

Provide the current status.

string

userCrn
required

Crn of the user who triggered this operation.

string

RestoreDatalakeStatusRequest

Request object to get the status of a restore operation.

Name Description Schema

datalakeName
required

The name of the Data Lake for which the most recent restore status will be retrieved.

string

restoreId
optional

Unique identifier of the restore operation performed.

string

RestoreDatalakeStatusResponse

Response object with information on last restore operation on a datalake.

Name Description Schema

accountId
required

Account where the datalake exists.

string

backupId
required

Unique identifier of the backup used to perform restore.

string

backupLocation
required

Location of the backup to be used to perform restore.

string

backupName
optional

Name of the backup.

string

endTime
required

Time when the backup operation ended.

string

failureReason
optional

Reason for the failure.

string

internalState
required

Provides the details of the internal state where the restore operation stands.

string

operationStates
required

Provides the details of the internal state of each operation.

InternalBackupRestoreState

restoreId
required

Unique identifier of the restore operation performed.

string

runtimeVersion
optional

The runtime version of the datalake when the restore was initiated.

string

startTime
required

Time when the backup operation started.

string

status
required

Provide the current status.

string

userCrn
required

Crn of the user who triggered this operation.

string

RetryDatalakeRequest

Request object for retry datalake request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake to be retry on.

string

RetryDatalakeResponse

Response object for retry datalake request.

Type : object

RotateAutoTlsCertificatesRequest

Request object to rotate autotls certificates on datalake’s hosts, deprecated.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

RotateAutoTlsCertificatesResponse

Response object to rotate autotls certificates on datalake’s hosts, deprecated.

Type : object

RotatePrivateCertificatesRequest

Request object to rotate private certificates on datalake’s hosts.

Name Description Schema

datalake
required

The name or CRN of the datalake.

string

RotatePrivateCertificatesResponse

Response object to rotate private certificates on datalake’s hosts.

Type : object

RotateSaltPasswordRequest

Request object for rotating SaltStack user password on DataLake instances.

Name Description Schema

datalake
required

The name or CRN of the datalake.

string

RotateSaltPasswordResponse

Response object for rotating SaltStack user password on DataLake instances.

Type : object

Runtime

An advertised datalake runtime version.

Name Description Schema

defaultRuntimeVersion
required

Whether it is a default runtime version or not.

boolean

runtimeVersion
required

The actual version number.

string

ScaleHorizontallyRequest

Data Lake horizontal scaling request.

Name Description Schema

datalakeName
required

The name of the Data Lake

string

instanceGroupDesiredCount
required

The target number of the scaling operation. If the desired count is less than the actual node count the scaling action will be a downscale operation.
Minimum value : 0
Maximum value : 40

integer (int32)

instanceGroupName
required

The target host group name to be scaled.

enum (solr_scale_out, hms_scale_out, raz_scale_out, kafka_scale_out, storage_scale_out)

ScaleHorizontallyResponse

Data Lake horizontal scaling response

Name Description Schema

datalakeName
optional

The name of the datalake.

string

ServiceStatus

Information about a cluster service.

Name Description Schema

healthChecks
optional

The service health checks.

< HealthCheck > array

healthSummary
optional

The service health summary.

string

state
optional

The service state.

string

type
optional

The service type.

string

SetCatalogRequest

The request object to set catalog for a DataLake.

Name Description Schema

catalogName
required

The name of the catalog to be used.

string

datalake
required

The name or CRN of the affected DataLake.

string

SetCatalogResponse

The response object to set catalog for a DataLake request.

Type : object

SolrBackupRestoreState

The state of each Solr backup/restore operation.

Name Description Schema

edgeIndexCollection
required

The status of the EDGE_INDEX_COLLECTION backup/restore.

BackupRestoreOperationStatus

edgeIndexCollectionDelete
optional

The status of the EDGE_INDEX_COLLECTION deletion, which is done before a restore.

BackupRestoreOperationStatus

fulltextIndexCollection
required

The status of the FULLTEXT_INDEX_COLLECTION backup/restore.

BackupRestoreOperationStatus

fulltextIndexCollectionDelete
optional

The status of the FULLTEXT_INDEX_COLLECTION deletion, which is done before a restore.

BackupRestoreOperationStatus

rangerAuditsCollection
required

The status of the RANGER_AUDITS_COLLECTION backup/restore.

BackupRestoreOperationStatus

rangerAuditsCollectionDelete
optional

The status of the RANGER_AUDITS_COLLECTION deletion, which is done before a restore.

BackupRestoreOperationStatus

vertexIndexCollection
required

The status of the VERTEX_INDEX_COLLECTION backup/restore.

BackupRestoreOperationStatus

vertexIndexCollectionDelete
optional

The status of the VERTEX_INDEX_COLLECTION deletion, which is done before a restore.

BackupRestoreOperationStatus

StartDatabaseUpgradeRequest

Request object for Data Lake database upgrade.

Name Description Schema

datalake
required

The name or CRN of the Data Lake.

string

force
optional

Start the database upgrade flow even if the source and target versions are the same. Can be used to reinitiate an upgrade after a failure.

boolean

targetVersion
optional

The database engine major version to upgrade to.

enum (VERSION_11, VERSION_14)

StartDatabaseUpgradeResponse

Response object to Data Lake database upgrade.

Name Description Schema

targetVersion
optional

The database engine major version to upgrade to.

string

StartDatalakeRequest

Request object to start a datalake, deprecated.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

StartDatalakeResponse

Response object of starting a datalake, deprecated.

Type : object

StartDatalakeVerticalScalingRequest

The request object for Data Lake vertical scaling.

Name Description Schema

datalake
required

The name or CRN of the Data Lake.

string

group
required

The target group that requested vertical scaling.

string

instanceTemplate
optional

Instance template that specifies the core information for the vertical scale.

InstanceTemplate

StartDatalakeVerticalScalingResponse

The response object for Data Lake vertical scaling.

Name Description Schema

result
optional

The result of the operation.

string

StopDatalakeRequest

Request object to stop a datalake, deprecated.

Name Description Schema

datalakeName
required

The name or CRN of the datalakeName.

string

StopDatalakeResponse

Response object of stopping a datalake, deprecated.

Type : object

SyncComponentVersionsFromCmRequest

Datalake sync CM component versions request.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

SyncComponentVersionsFromCmResponse

Datalake sync CM component versions response.

Type : object

UpdateOrchestratorStateRequest

Request object for running orchestrator engine state update on the Data Lake cluster.

Name Description Schema

clusterCrn
required

The CRN of the Data Lake cluster.

string

UpdateOrchestratorStateResponse

Response object for running orchestrator engine state update on the Data Lake cluster.

Type : object

UpgradeDatalakeRequest

Request object to upgrade datalake.

Name Description Schema

datalakeName
required

The name or CRN of the datalake.

string

dryRun
optional

Checks the eligibility of an image to upgrade

boolean

imageId
optional

The id of an image to upgrade to

string

lockComponents
optional

Perform an os upgrade only

boolean

rollingUpgradeEnabled
optional

Enables the ability to perform rolling runtime upgrade.

boolean

runtime
optional

The runtime version to upgrade to

string

showAvailableImages
optional

Returns the list of images that are eligible for the upgrade

boolean

showLatestAvailableImagePerRuntime
optional

Returns the latest image that is eligible for the upgrade for each runtime version with at least one available upgrade candidate

boolean

skipAtlasMetadata
optional

Skips the backup of the Atlas metadata. Redundant if –skip-backup is included. If this option is not provided, the Atlas metadata is backed up by default.

boolean

skipBackup
optional

If provided, will skip the backup flow for the upgrade process.

boolean

skipBackupValidation
optional

Skips the validation steps that run prior to the backup. Redundant if –skip-backup is included. If this option is not provided, the validations are performed by default.

boolean

skipRangerAudits
optional

Skips the backup of the Ranger audits. Redundant if –skip-backup is included. If this option is not provided, Ranger audits are backed up by default.

boolean

skipRangerHmsMetadata
optional

Skips the backup of the databases backing HMS/Ranger services. Redundant if –skip-backup is included. If this option is not provided, the HMS/Ranger services are backed up by default.

boolean

UpgradeDatalakeResponse

Response object for upgrade datalake request.

Name Description Schema

current
optional

Information about the current image

ImageInfo

reason
optional

The reason why upgrade is not possible

string

upgradeCandidates
optional

List of images and components to upgrade to

< ImageInfo > array

ValidateAwsCloudStorageRequest

Request object for AWS cloud storage validation.

Name Description Schema

blueprintName
required

Name of the blueprint.

string

clusterName
required

Datalake cluster name.

string

credentialCrn
required

Credential CRN.

string

dataAccessRole
required

Data access role.

string

instanceProfile
required

AWS instance profile.

string

rangerAuditRole
required

Ranger audit role.

string

storageLocation
required

AWS S3 storage bucket base location.

string

ValidateAwsCloudStorageResponse

AWS cloud storage validation response.

Name Description Schema

error
optional

Error message if validation failed.

string

status
optional

Validation result. Allowed values are "OK", "ACCESS_DENIED", "ERROR", "RESOURCE_NOT_FOUND".

string

ValidateAzureCloudStorageRequest

Request object for Azure cloud storage validation.

Name Description Schema

accountKey
optional

Azure account key.

string

accountName
optional

Azure account name.

string

blueprintName
required

Name of the blueprint.

string

clusterName
required

Datalake cluster name.

string

credentialCrn
required

Credential CRN.

string

dataAccessRole
required

Data access role.

string

managedIdentity
optional

Azure managed identity.

string

rangerAuditRole
required

Ranger audit role.

string

secure
optional

Azure secure flag.

boolean

storageLocation
required

Azure storage base location.

string

ValidateAzureCloudStorageResponse

Azure cloud storage validation response.

Name Description Schema

error
optional

Error message if validation failed.

string

status
optional

Validation result. Allowed values are "OK", "ACCESS_DENIED", "ERROR", "RESOURCE_NOT_FOUND".

string

Copyright © 2024 Cloudera, Inc. All rights reserved.