Overview

Create and manage Cloudera Data Engineering Services.

Version information

Version : 0.9.128 (BETA)

License information

URI scheme

Schemes : HTTPS

Consumes

  • application/json

Produces

  • application/json

Paths

Cancel Cloudera Data Engineering(CDE) Service Backup

POST /api/v1/de/cancelBackup

Description

Cancel a currently running backup operation using the associated backup ID. The operation must be in a 'pending' state.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Cancel Backup command.

default

The default response on an error.

Backup Cloudera Data Engineering(CDE) Service

POST /api/v1/de/createBackup

Description

Initiates service backup and returns a backup ID, which can be used for log retrieval, restoration, deletion, and backup cancellation. Service backup includes the definition of the Service and its Virtual Clusters along with Virtual Clusters contents (jobs, resources, and credentials).

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Create Backup command.

default

The default response on an error.

Create Virtual Cluster

POST /api/v1/de/createVc

Description

Creates a virtual cluster and returns creation response.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Delete Cloudera Data Engineering(CDE) Service Backup

POST /api/v1/de/deleteBackup

Description

Delete a backup using the associated backup ID.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Delete Backup command.

default

The default response on an error.

Delete Virtual Cluster

POST /api/v1/de/deleteVc

Description

Deletes a virtual cluster and returns deletion response.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Describe Cloudera Data Engineering(CDE) Service Backup

POST /api/v1/de/describeBackup

Description

Describe a backup using the associated backup ID.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Describe Backup command.

default

The default response on an error.

Describe Cloudera Data Engineering(CDE) Service

POST /api/v1/de/describeService

Description

Returns details of CDE service.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Describe Virtual Cluster

POST /api/v1/de/describeVc

Description

Returns a description for the specified virtual cluster.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Disable Cloudera Data Engineering(CDE) Service

POST /api/v1/de/disableService

Description

Returns delete response.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Enable Cloudera Data Engineering(CDE) Service

POST /api/v1/de/enableService

Description

Returns enabling response.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Retrieve Cloudera Data Engineering(CDE) Service Backup Logs

POST /api/v1/de/getBackupLogs

Description

Retrieve backup logs using the associated backup ID.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Get Backup Logs command.

default

The default response on an error.

Get Kubeconfig for the CDE service

POST /api/v1/de/getKubeconfig

Description

Returns kubeconfig.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Retrieve Cloudera Data Engineering(CDE) Service Initialization Logs

POST /api/v1/de/getServiceInitLogs

Description

Retrieve service initialization logs, if any.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Get Service Init Logs command.

default

The default response on an error.

Get CDE Service Upgrade Status.

POST /api/v1/de/getUpgradeStatus

Description

The current status of the CDE Service upgrade. If all the steps are completed allStepsCompleted would be true. After a particular step triggered by upgrade-service is completed nextStep would point to the next step to take.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

List Cloudera Data Engineering(CDE) Service Backups

POST /api/v1/de/listBackups

Description

List all service backups.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for List Backups command.

default

The default response on an error.

List Cloudera Data Engineering(CDE) Services

POST /api/v1/de/listServices

Description

Returns list of cde services.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

List Virtual Clusters

POST /api/v1/de/listVcs

Description

Returns a list of virtual clusters for a given CDE service.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Restore Cloudera Data Engineering(CDE) Service

POST /api/v1/de/restoreService

Description

Initiates the service restoration process and returns the ID of the service being restored.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Response object for Restore Service command.

default

The default response on an error.

Update Cloudera Data Engineering (CDE) Service

POST /api/v1/de/updateService

Description

Returns update response consisting of operation ID that can be used to log update events.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Update Cloudera Data Engineering (CDE) Virtual Cluster

POST /api/v1/de/updateVc

Description

Updates Cloudera Data Engineering (CDE) Virtual Cluster and returns details of updated CDE service.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Trigger a particular step (prepare, backup, upgrade, resume) of the upgrade framework.

POST /api/v1/de/upgradeService

Description

Trigger a particular step (prepare, backup, upgrade, resume) of the multi-step upgrade process. Use get-upgrade-status to know what is the next step to take.

Parameters

Type Name Schema

Body

input
required

Responses

HTTP Code Description Schema

200

Expected response to a valid request.

default

The default response on an error.

Definitions

AccessControlResponse

Response object containing access control information for a Virtual Cluster.

Name Description Schema

fullAccessGroups
optional

Groups with full access.

< string > array

fullAccessUsers
optional

Users with full access.

< string > array

users
optional

Workload usernames of CDP users granted access to the Virtual Cluster.

< string > array

viewOnlyGroups
optional

Groups with view only access.

< string > array

viewOnlyUsers
optional

Users with view only access.

< string > array

AllPurposeInstanceGroupDetailsResponse

Instance group details for the All purpose Tier.

Name Description Schema

initial_instances
optional

Initial number of Instances when the service is enabled.

string

initial_spot_instances
optional

Initial number of spot Instances when the service is enabled.

string

instance_type
optional

Instance type of the cluster for CDE Service.

string

max_instances
optional

Maximum number of Instances for the CDE Service.

string

max_spot_instances
optional

Maximum number of Spot Instances for the CDE Service.

string

min_instances
optional

Minimum number of Instances for the CDE Service.

string

min_spot_instances
optional

Minimum number of Spot instances for the CDE Service.

string

root_vol_size
optional

EBS volume size in GB.

string

Backup

Service backup object.

Name Description Schema

archiveBasePath
optional

The path to the base location where the backups are stored.

string

archiveLocation
optional

Archive location under the base path.

string

cloudPlatform
required

The cloud platform where a backed-up CDE service was enabled.

string

created
required

The time when the backup was created.

string

creator
required

The workload name of the user who created this backup.

string

creatorCrn
required

CRN of of the user who created this backup.

string

description
optional

Backup description.

string

environmentCrn
required

CRN of CDP environment with which a backed-up CDE service was associated.

string

environmentName
required

Name of CDP environment with which a backed-up CDE service was associated.

string

errorMessage
optional

Error message, only set when there is an error.

string

id
required

Backup ID.

integer (int64)

serviceID
required

Service ID of a backed-up CDE service.

string

serviceName
required

Name of a backed-up CDE service.

string

serviceVersion
required

Version of a backed-up CDE service.

string

status
required

Backup status.

string

BackupContentOptions

Backup virtual cluster content options.

Name Description Schema

credentialFilter
optional

Filter credentials for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the credential fields (e.g. name, type, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)test-credential'. This will only take effect if credentials are backed up, as controlled by the 'includeCredentials' option.

< string > array

includeCredentialSecrets
optional

Whether to backup credential secrets. Secrets are obfuscated while storing. By default, credential secrets are backed up.
Default : true

boolean

includeCredentials
optional

Whether to backup credentials. By default, credentials are backed up.
Default : true

boolean

includeJobResources
optional

Whether to backup job resources. By default, job resources are backed up.
Default : true

boolean

includeJobRuns
optional

Whether to backup job runs. By default, job runs are backed up.
Default : true

boolean

includeJobs
optional

Whether to backup jobs. By default, jobs are backed up.
Default : true

boolean

includeResourceCredentials
optional

Whether to backup resource credentials. By default, resource credentials are backed up.
Default : true

boolean

includeResources
optional

Whether to backup resources. By default, resources are backed up.
Default : true

boolean

jobFilter
optional

Filter jobs for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the job fields (e.g. name, type, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)SparkPi'. This will only take effect if jobs are backed up, as controlled by the 'includeJobs' option.

< string > array

resourceFilter
optional

Filter resources for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the resource fields (e.g. name, type, status, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)test-resource'. This will only take effect if resources are backed up, as controlled by the 'includeResources' option.

< string > array

validateArchive
optional

Whether to validate backup archive. By default, backup archive is not validated.
Default : false

boolean

CancelBackupRequest

Request object for Cancel Backup command.

Name Description Schema

backupID
required

The ID of the backup to cancel.

integer (int64)

CancelBackupResponse

Response object for Cancel Backup command.

Type : object

ChartValueOverridesRequest

Object containing chart value overrides for enabling a CDE service or creation of virtual clusters.

Name Description Schema

chartName
optional

Name of the chart that has to be overridden, for eg- "dex-app", "dex-base".

string

overrides
optional

Space separated key-value pairs for overriding chart values. The key and the value must be separated using a colon(:) For eg- "airflow.enabled:true safari.enabled:true".

string

ChartValueOverridesResponse

Response object containing chart value overrides.

Name Description Schema

chartName
optional

Name of the chart that has to be overridden, for eg- "dex-app", "dex-base".

string

overrides
optional

Space separated key value-pairs for overriding chart values. The key and the value must be separated using a colon(:) For eg- "airflow.enabled:true safari.enabled:true".

string

CreateBackupRequest

Request object for Create Backup command.

Name Description Schema

backupPath
optional

Backup file relative path. The path should not include file name, and be relative to the service backup location. The service backup location can be found in the service description obtained via the 'describe-service' command.

string

backupVcContentOptions
optional

Backup virtual cluster content options.

description
optional

Backup description.

string

force
optional

Whether to ignore validation errors and proceed with the backup forcefully. By default, the backup operation is not forced.
Default : false

boolean

includeVc
optional

Whether to backup service virtual clusters. By default, virtual clusters are backed up.
Default : true

boolean

serviceId
required

Service ID of the service to backup.

string

timeout
optional

Backup operation timeout in seconds.

integer (int32)

CreateBackupResponse

Response object for Create Backup command.

Name Description Schema

backupID
required

Backup ID, which can be used for log retrieval, restoration, deletion, and backup cancellation.

integer (int64)

CreateVcRequest

Request object for CreateVc method.

Name Description Schema

aclUsers
optional

Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster.

string

chartValueOverrides
optional

Chart overrides for creating a virtual cluster.

clusterId
required

Cluster id of the CDE service where virtual cluster has to be created.

string

cpuRequests
required

Cpu requests for autoscaling.

string

fullAccessGroups
optional

Groups with full access.

< string > array

fullAccessUsers
optional

Users with full access.

< string > array

gpuRequests
optional

GPU requests for autoscaling.

string

guaranteedCpuRequests
optional

Guaranteed CPU requests for an Elastic Virtual Cluster.

string

guaranteedGpuRequests
optional

Guaranteed GPU requests for an Elastic Virtual Cluster.

string

guaranteedMemoryRequests
optional

Guaranteed Memory requests for an Elastic Virtual Cluster - eg. 30Gi.

string

memoryRequests
required

Memory requests for autoscaling - eg. 30Gi.

string

name
required

Name of the virtual cluster.

string

runtimeSpotComponent
optional

Used to describe where the Driver and the Executors would run. By default the Driver would run on on-demand instances and the Executors on spot instances. Setting it to ALL will run both the Driver and the Executors on spot instances whereas setting it to NONE should run both the Driver and the Executor on on-demand instances. Currently applicable for aws services only. Use this option only on services with spot instances enabled.

enum (ALL, NONE)

sessionTimeout
optional

Set a default timeout for your sessions. The default option is 8 hours. This option can be overridden when creating a new session.

string

smtpConfigs
optional

SMTP Configurations for Airflow Email Alerts.

sparkConfigs
optional

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

< string, string > map

sparkVersion
optional

Spark version for the virtual cluster. Currently supported Spark versions are SPARK2(deprecated), SPARK3, SPARK3_3 and SPARK3_5. This feature is only supported in CDE-1.7.0 and later. SPARK3_3 is supported in CDE-1.19 and later. SPARK3_5 is supported in CDE-1.21 and later.

enum (SPARK2, SPARK2_4, SPARK3, SPARK3_0, SPARK3_1, SPARK3_2, SPARK3_3, SPARK3_5)

vcTier
optional

Tier of the virtual cluster. Currently supported tiers are CORE and ALLP. CORE tiered virtual cluster enables operational deployment via batch jobs. ALLP virtual clusters are all-purpose virtual clusters supporting both operational batch jobs and interactive sessions. This feature is only supported in CDE-1.19.0 and beyond.

enum (ALLP, CORE)

viewOnlyGroups
optional

Groups with view only access.

< string > array

viewOnlyUsers
optional

Users with view only access.

< string > array

CreateVcResponse

Response object for CreateVc method.

Name Description Schema

Vc
optional

Created Virtual Cluster

CustomAzureFilesConfigs

Override Azure Files Configs.

Name Description Schema

azureFilesFQDN
optional

Azure File Share’s server address. Defaults to '<storageaccount>.file.core.windows.net'.

string

resourceGroup
required

Resource Group of the Storage Account.

string

storageAccountName
required

Azure Storage Account of the File Share.

string

DeleteBackupRequest

Request object for Delete Backup command.

Name Description Schema

backupID
required

The ID of the backup to delete.

integer (int64)

DeleteBackupResponse

Response object for Delete Backup command.

Type : object

DeleteVcRequest

Request object for DeleteVc method.

Name Description Schema

clusterId
required

Cluster id of the CDE service where virtual cluster was enabled.

string

vcId
required

Virtual cluster id

string

DeleteVcResponse

Response object for DeleteVc method.

Name Description Schema

status
optional

status of virtual cluster deletion.

string

DescribeBackupRequest

Request object for Describe Backup command.

Name Description Schema

backupID
required

The ID of the backup to describe.

integer (int64)

DescribeBackupResponse

Response object for Describe Backup command.

Name Description Schema

backup
required

Service backup.

DescribeServiceRequest

Request object for DescribeService method.

Name Description Schema

clusterId
required

Cluster id of the service to be described.

string

DescribeServiceResponse

Response object for DescribeService method.

Name Description Schema

service
optional

Description of the service

DescribeVcRequest

DescribeVc request object.

Name Description Schema

clusterId
required

Cluster id of the CDE service where vc was created.

string

vcId
required

Virtual cluster id.

string

DescribeVcResponse

DescribeVc response object.

Name Description Schema

vc
optional

Description of the virtual cluster.

DisableServiceRequest

DisableService request object.

Name Description Schema

clusterId
required

cluster Id of the service to disable.

string

force
optional

Force disable cde service.
Default : false

boolean

DisableServiceResponse

DisableService response object.

Name Description Schema

status
optional

Status of deletion

string

EnableServiceRequest

Request object for Enable Service method.

Name Description Schema

allPurposeInitialInstances
optional

Initial Instances when the service is enabled for the All Purpose Instance Group.

integer (int32)

allPurposeInitialSpotInstances
optional

Initial spot Instances when the service is enabled for the All Purpose Instance Group.

integer (int32)

allPurposeInstanceType
optional

Instance type of the cluster for CDE Service for the All Purpose Instance Group.

string

allPurposeMaximumInstances
optional

Maximum Instances for the CDE Service for the All Purpose Instance Group.

integer (int32)

allPurposeMaximumSpotInstances
optional

Maximum Spot Instances for the CDE Service for the All Purpose Instance Group.

integer (int32)

allPurposeMinimumInstances
optional

Minimum Instances for the CDE Service for the All Purpose Instance Group.

integer (int32)

allPurposeMinimumSpotInstances
optional

Minimum Spot instances for the CDE Service for the All Purpose Instance Group.

integer (int32)

allPurposeRootVolumeSize
optional

EBS volume size in GB for the All Purpose Instance Group.

integer (int32)

azureDatabasePrivateDNSZoneId
optional

Resource ID of the custom private DNS zone associated with the Azure database.

string

azureFilesharePrivateDNSZoneId
optional

Resource ID of custom private DNS zone for Azure storage account.

string

chartValueOverrides
optional

Chart overrides for enabling a service.

cpuRequests
optional

CPU Requests for the entire CDE service quota.

string

customAzureFilesConfigs
optional

CDE uses a default public File Shares storage provisioned by AKS. Enable this option to use your own public/private File Shares.

deployPreviousVersion
optional

If set to "true", the previous version of the CDE service will be deployed.
Default : false

boolean

enablePrivateNetwork
optional

Create a fully private CDE instance on either Amazon or Azure. This includes services such as Kubernetes, MySQL, etc. For Azure, this will also enable virtual network (VNet) access via private endpoints and private link.

boolean

enablePublicEndpoint
optional

Creates a CDE endpoint (Load Balancer) in a publicly accessible subnet. If set false, the endpoint will be created in a private subnet and you will need to setup access to the endpoint manually in your cloud account.
Default : false

boolean

enableWorkloadAnalytics
optional

If set false, diagnostic information about job and query execution is sent to Cloudera Workload Manager. Anonymization can be configured under Environments / Shared Resources / Telemetry. Refer documentation for more info at https://docs.cloudera.com/workload-manager/cloud/index.html.
Default : false

boolean

env
required

CDP environment where cde service should be enabled.

string

gpuRequests
optional

GPU requests for the entire CDE service quota.

string

initialInstances
optional

Initial Instances when the service is enabled.

integer (int32)

initialSpotInstances
optional

Initial spot Instances when the service is enabled.

integer (int32)

instanceType
required

Instance type of the cluster for CDE Service.

string

loadbalancerAllowlist
optional

List of CIDRs that would be allowed to access the load balancer.

< string > array

maximumInstances
required

Maximum Instances for the CDE Service.

integer (int32)

maximumSpotInstances
optional

Maximum Spot Instances for the CDE Service.

integer (int32)

memoryRequests
optional

Memory requests for the entire CDE service quota, eg. 100Gi.

string

minimumInstances
required

Minimum Instances for the CDE Service.

integer (int32)

minimumSpotInstances
optional

Minimum Spot instances for the CDE Service.

integer (int32)

name
required

Name of the CDE Service.

string

networkOutboundType
optional

Network outbound type. Currently 'udr' is the only supported.

enum (UDR)

nfsStorageClass
optional

NFS Storage class to override the default storage class in private cloud.

string

resourcePool
optional

Resource Pool for the CDE service.

string

rootVolumeSize
optional

EBS volume size in GB.

integer (int32)

skipValidation
optional

Skip Validation check.
Default : false

boolean

subnets
optional

List of Subnet IDs of CDP subnets to use for the kubernetes worker node.

< string > array

tags
optional

User defined labels that tag all provisioned cloud resources.

< string, string > map

useSsd
optional

Instance local storage (SSD) would be used for the workload filesystem (Example - spark local directory). In case the workload requires more space than what’s available in the instance storage, please use an instance type with sufficient instance local storage or choose an instance type without SSD and configure the EBS volume size. Currently supported only for aws services.

boolean

whitelistIps
optional

List of CIDRs that would be allowed to access kubernetes master API server.

< string > array

EnableServiceResponse

Response object for EnableService request.

Name Description Schema

service
optional

Service details of the created service.

Error

An object returned on an error.

Name Description Schema

code
optional

The error code.

string

message
optional

The error message.

string

GetBackupLogsRequest

Request object for Get Backup Logs command.

Name Description Schema

backupID
required

The ID of the backup.

integer (int64)

pageSize
optional

The size of each page.
Minimum value : 1
Maximum value : 1000

integer (int32)

startingToken
optional

A token to specify where to start paginating. This is the nextToken from a previously truncated response.

string

GetBackupLogsResponse

Response object for the Get Backup Logs command.

Name Description Schema

logs
required

Contains the logs of the backup operation.

< string > array

nextToken
optional

The token to use when requesting the next set of results. If there are no additional results, the string is empty.

string

GetKubeconfigRequest

GetKubeconfig request object.

Name Description Schema

clusterId
required

cluster id of the cde service.

string

GetKubeconfigResponse

GetKubeconfig response object.

Name Description Schema

kubeconfig
optional

kubeconfig for the cluster.

string

GetServiceInitLogsRequest

Request object for Get Service Init Logs command.

Name Description Schema

pageSize
optional

The size of each page.
Minimum value : 1
Maximum value : 500

integer (int32)

serviceID
required

The service ID of the service.

string

startingToken
optional

A token to specify where to start paginating. This is the nextToken from a previously truncated response.

string

GetServiceInitLogsResponse

Response object for the Get Service Init Logs command.

Name Description Schema

logs
required

Contains the logs of the service initialization operation.

< string > array

nextToken
optional

The token to use when requesting the next set of results. If there are no additional results, the string is empty.

string

GetUpgradeStatusRequest

Request object for GetUpgradeStatus method.

Name Description Schema

clusterId
required

Cluster ID of the service.

string

GetUpgradeStatusResponse

Response object for GetUpgradeStatus request.

Name Description Schema

allStepsCompleted
optional

Whether all the steps of the multi-step upgrade process completed.

boolean

nextStep
optional

The next step to take in multi-step upgrade process. Currently supported steps are prepare, backup, upgrade and resume.

enum (prepare, backup, upgrade, resume)

upgradeStatusMessage
optional

The status of the upgrade.

string

ListBackupsRequest

Request object for List Backups command.

Name Description Schema

filter
optional

Filter backups using 'fieldname(operator)argument' syntax. Here 'fieldname' is one of ('id', 'clusterID', 'cdeVersion', 'environment', 'environmentCrn', 'created', 'status') and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'status(eq)failed'.

< string > array

order
optional

Whether ordering is ascending ('ASC') or descending ('DESC'), default 'DESC'.

orderBy
optional

The 'fieldname' to order by, default "id". Here 'fieldname' is one of ('id', 'clusterID', 'cdeVersion', 'environment', 'environmentCrn', 'created', 'status').
Default : "id"

string

pageSize
optional

The size of each page.
Minimum value : 1
Maximum value : 500

integer (int32)

startingToken
optional

A token to specify where to start paginating. This is the nextToken from a previously truncated response.

string

ListBackupsResponse

Response object for List Backups command.

Name Description Schema

backups
required

The backups.

< Backup > array

nextToken
optional

The token to use when requesting the next set of results. If there are no additional results, the string is empty.

string

ListServicesRequest

Request object for ListServices method.

Name Description Schema

removeDeleted
optional

Filter out deleted CDE services from the list.
Default : false

boolean

ListServicesResponse

Response object for ListServices method.

Name Description Schema

services
optional

List of services.

< ServiceSummary > array

ListVcsRequest

ListVcs request object.

Name Description Schema

clusterId
required

Cluster id of the service.

string

ListVcsResponse

ListVcs response object.

Name Description Schema

vcs
optional

List of virtual clusters.

< VcSummary > array

Order

Order.

Type : enum (ASC, DESC)

RestoreServiceRequest

Request object for Restore Service command.

Name Description Schema

backupID
required

The ID of the backup to restore from.

integer (int64)

environmentCrn
required

CRN of the CDP environment with which a restored CDE service will be associated. Currently, CDE service can be restored only to the same CDP environment, with which a backed-up service was associated.

string

serviceId
optional

Specify the restored service ID. If not specified, a new service ID will be generated.

string

serviceName
optional

Specify the restored service name. If not specified, the service name from the backup will be reused.

string

RestoreServiceResponse

Response object for Restore Service command.

Name Description Schema

serviceID
required

The ID of the service being restored.

string

ServiceDescription

Detailed description of a CDE service.

Name Description Schema

backupLocation
optional

Base location for the service backup archives.

string

chartValueOverrides
optional

Chart overrides for the Virtual Cluster.

cloudPlatform
optional

The cloud platform where the CDE service is enabled.

string

clusterFqdn
optional

FQDN of the CDE service.

string

clusterId
required

Cluster Id of the CDE Service.

string

creatorCrn
optional

CRN of the creator.

string

creatorEmail
optional

Email address of the creator of the CDE service.

string

dataLakeAtlasUIEndpoint
optional

Endpoint of Data Lake Atlas.

string

dataLakeFileSystems
optional

The Data lake file system.

string

enablingTime
optional

Timestamp of service enabling.

string

environmentCrn
optional

CRN of the environment.

string

environmentName
required

CDP Environment Name.

string

loadbalancerAllowlist
optional

Comma-separated CIDRs that would be allowed to access the load balancer.

string

logLocation
optional

Location for the log files of jobs.

string

name
required

Name of the CDE Service.

string

networkOutboundType
optional

Network outbound type. Currently 'udr' is the only supported.

string

previousVersionDeployed
optional

The "true" value indicates that the previous version of the CDE service was requested to be deployed.

boolean

privateClusterEnabled
optional

If true, the CDE service was created with fully private Azure services (AKS, MySQL, etc.).

boolean

publicEndpointEnabled
optional

If true, the CDE endpoint was created in a publicly accessible subnet.

boolean

resources
optional

Resources details of CDE Service.

ssdUsed
optional

If true, SSD would have been be used for workload filesystem.

boolean

status
optional

Status of the CDE service.

string

subnets
optional

List of Subnet IDs of the CDP subnets used by the kubernetes worker node.

string

tenantId
optional

CDP tenant ID.

string

whitelistIps
optional

List of CIDRs that would be allowed to access kubernetes master API server.

string

workloadAnalyticsEnabled
optional

If true, diagnostic information about job and query execution is sent to Cloudera Workload Manager.

boolean

ServiceResources

Object to store resources for a CDE service.

Name Description Schema

allPurposeInstanceGroupDetails
optional

Resource details for the nodes used in All Purpose Virtual Clusters.

cpuRequests
optional

CPU Requests for the entire CDE service quota.

string

gpuRequests
optional

GPU requests for the entire CDE service quota.

string

initial_instances
optional

Initial instances for the CDE service.

string

initial_spot_instances
optional

Initial Spot Instances for the CDE Service.

string

instance_type
optional

Instance type of the CDE service.

string

max_instances
optional

Maximum instances for the CDE service.

string

max_spot_instances
optional

Maximum Number of Spot instances.

string

memoryRequests
optional

Memory requests for the entire CDE service quota, eg. 100Gi.

string

min_instances
optional

Minimum Instances for the CDE service.

string

min_spot_instances
optional

Minimum number of spot instances for the CDE service.

string

resourcePool
optional

Resource Pool for the CDE service.

string

root_vol_size
optional

Root Volume Size.

string

ServiceSummary

Summary of a CDE service.

Name Description Schema

clusterId
required

Cluster Id of the CDE Service.

string

creatorEmail
optional

Email Address of the CDE creator.

string

enablingTime
optional

Timestamp of service enabling.

string

environmentName
required

CDP Environment Name.

string

name
required

Name of the CDE Service.

string

status
required

Status of the CDE service.

string

SmtpConfigRequest

SMTP config request object.

Name Description Schema

email
required

Sender’s email address.

string

host
required

SMTP host.

string

password
optional

SMTP password.

string

port
optional

SMTP port.
Default : 25

integer (int32)

ssl
optional

Use SSL to secure the connection to the email server.
Default : false

boolean

startTls
optional

Use SMTP STARTTLS command to encrypt the mail.
Default : true

boolean

username
optional

SMTP username.

string

SmtpConfigResponse

SMTP config response object

Name Description Schema

email
required

Sender’s email address.

string

host
required

SMTP host.

string

port
optional

SMTP port.
Default : 25

integer (int32)

ssl
optional

Use SSL to secure the connection to the email server.
Default : false

boolean

startTls
optional

Use SMTP STARTTLS command to encrypt the mail.
Default : true

boolean

username
optional

SMTP username.

string

UpdateServiceRequest

Request object for UpdateService method.

Name Description Schema

allPurposeMaximumInstances
optional

Maximum number of instances for the CDE service for the All Purpose Instance Group.

integer (int32)

allPurposeMaximumSpotInstances
optional

Maximum number of spot instances for the CDE service for the All Purpose Instance Group.

integer (int32)

allPurposeMinimumInstances
optional

Minimum number of instances for the CDE service for the All Purpose Instance Group.

integer (int32)

allPurposeMinimumSpotInstances
optional

Minimum number of spot instances for the CDE service for the All Purpose Instance Group.

integer (int32)

clusterId
required

Cluster ID of the service to update.

string

cpuRequests
optional

CPU Requests for the entire CDE service quota.

string

gpuRequests
optional

GPU requests for the entire CDE service quota.

string

loadbalancerAllowlist
optional

List of CIDRs that would be allowed to access the load balancer.

< string > array

maximumInstances
optional

Maximum number of instances for the CDE service.

integer (int32)

maximumSpotInstances
optional

Maximum number of spot instances for the CDE service.

integer (int32)

memoryRequests
optional

Memory requests for the entire CDE service quota, eg. 100Gi.

string

minimumInstances
optional

Minimum number of instances for the CDE service.

integer (int32)

minimumSpotInstances
optional

Minimum number of spot instances for the CDE service.

integer (int32)

whitelistIps
optional

List of CIDRs that would be allowed to access kubernetes master API server.

< string > array

UpdateServiceResponse

Response object for UpdateService request.

Name Description Schema

operationId
optional

Deprecated. The operation ID of the update request.

string

status
optional

Status of update.

string

UpdateVcRequest

Request object for updateVc method.

Name Description Schema

aclUsers
optional

Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster.

string

clusterId
required

Cluster ID of the CDE service where Virtual Cluster was enabled.

string

discardSparkConfigs
optional

Discard the Spark configs inside a VC.
Default : false

boolean

fullAccessGroups
optional

Groups with full access.

< string > array

fullAccessUsers
optional

Users with full access.

< string > array

sparkConfigs
optional

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

< string, string > map

vcId
required

Virtual Cluster ID

string

viewOnlyGroups
optional

Groups with view only access.

< string > array

viewOnlyUsers
optional

Users with view only access.

< string > array

UpdateVcResponse

Response object for updateVc method.

Name Description Schema

Vc
optional

Updated Virtual Cluster.

UpgradeServiceRequest

Request object for upgrading a service.

Name Description Schema

clusterId
required

Cluster ID of the service to upgrade.

string

step
required

The maintenance step to be performed. Currently supported steps are prepare, backup, upgrade, cancel and resume.

enum (prepare, backup, upgrade, cancel, resume)

UpgradeServiceResponse

Response object for UpgradeService request.

Name Description Schema

message
optional

The status message of upgrade operation.
Default : "upgrade step triggered successfully"

string

VcDescription

CDE virtual cluster summary.

Name Description Schema

VcUiUrl
optional

URL of the CDE Virtual Cluster UI.

string

accessControl
optional

Access control object for the Virtual Cluster.

chartValueOverrides
optional

Chart overrides for the Virtual Cluster.

clusterId
required

Cluster ID of the CDE service that contains the Virtual Cluster.

string

creationTime
optional

Time of creation of the virtual Cluster.

string

creatorEmail
optional

Email address of the creator of Virtual Cluster.

string

creatorID
optional

ID of the creator of Virtual Cluster.

string

creatorName
optional

Name of the creator of the Virtual Cluster.

string

historyServerUrl
optional

Spark History Server URL for the Virtual Cluster.

string

livyServerUrl
optional

Livy Server URL for the Virtual Cluster.

string

resources
optional

Resources details of CDE Service.

safariUrl
optional

Safari URL for the Virtual Cluster.

string

sessionTimeout
optional

Default timeout for Sessions.

string

smtpConfig
optional

SMTP Configurations for Airflow Email Alerts.

sparkConfigs
optional

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

< string, string > map

sparkVersion
optional

Spark version for the virtual cluster.

string

status
optional

Status of the Virtual Cluster.

string

vcApiUrl
optional

Url for the Virtual Cluster APIs.

string

vcId
required

Virtual Cluster ID.

string

vcName
required

Name of the CDE Virtual Cluster.

string

vcTier
optional

Tier of the Virtual Cluster.

string

VcResources

Object to store resources for a CDE service

Name Description Schema

actualCpuRequests
optional

Actual CPU request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

string

actualMemoryRequests
optional

Actual Memory request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

string

cpuRequests
optional

The CPU requests for VC for running spark jobs.

string

gpuRequests
optional

GPU requests for autoscaling.

string

guaranteedCpuRequests
optional

Guaranteed CPU requests for an Elastic Virtual Cluster.

string

guaranteedGpuRequests
optional

Guaranteed GPU requests for an Elastic Virtual Cluster.

string

guaranteedMemoryRequests
optional

Guaranteed Memory requests for an Elastic Virtual Cluster - eg. 30Gi.

string

memRequests
optional

The Memory requests for VC for running spark jobs.

string

VcSummary

CDE virtual cluster summary.

Name Description Schema

VcUiUrl
optional

Url for the Virtual Cluster UI.

string

clusterId
required

Cluster ID of the CDE service that contains the virtual cluster.

string

creatorEmail
optional

Email Address of the Creator.

string

creatorName
optional

Name of the Creator.

string

status
optional

Status of the Virtual Cluster.

string

vcId
required

Virtual Cluster ID.

string

vcName
required

Name of the CDE Virtual Cluster.

string


Copyright © 2024 Cloudera, Inc. All rights reserved.