Overview
Create and manage Cloudera Data Engineering Services.
Version information
Version : 0.9.128 (BETA)
License information
License : Apache 2.0
Terms of service : https://www.cloudera.com/legal/commercial-terms-and-conditions.html
URI scheme
Schemes : HTTPS
Consumes
-
application/json
Produces
-
application/json
Paths
Cancel Cloudera Data Engineering(CDE) Service Backup
POST /api/v1/de/cancelBackup
Description
Cancel a currently running backup operation using the associated backup ID. The operation must be in a 'pending' state.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Cancel Backup command. |
|
default |
The default response on an error. |
Backup Cloudera Data Engineering(CDE) Service
POST /api/v1/de/createBackup
Description
Initiates service backup and returns a backup ID, which can be used for log retrieval, restoration, deletion, and backup cancellation. Service backup includes the definition of the Service and its Virtual Clusters along with Virtual Clusters contents (jobs, resources, and credentials).
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Create Backup command. |
|
default |
The default response on an error. |
Create Virtual Cluster
POST /api/v1/de/createVc
Description
Creates a virtual cluster and returns creation response.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Delete Cloudera Data Engineering(CDE) Service Backup
POST /api/v1/de/deleteBackup
Description
Delete a backup using the associated backup ID.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Delete Backup command. |
|
default |
The default response on an error. |
Delete Virtual Cluster
POST /api/v1/de/deleteVc
Description
Deletes a virtual cluster and returns deletion response.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Describe Cloudera Data Engineering(CDE) Service Backup
POST /api/v1/de/describeBackup
Description
Describe a backup using the associated backup ID.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Describe Backup command. |
|
default |
The default response on an error. |
Describe Cloudera Data Engineering(CDE) Service
POST /api/v1/de/describeService
Description
Returns details of CDE service.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Describe Virtual Cluster
POST /api/v1/de/describeVc
Description
Returns a description for the specified virtual cluster.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Disable Cloudera Data Engineering(CDE) Service
POST /api/v1/de/disableService
Description
Returns delete response.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Enable Cloudera Data Engineering(CDE) Service
POST /api/v1/de/enableService
Description
Returns enabling response.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Retrieve Cloudera Data Engineering(CDE) Service Backup Logs
POST /api/v1/de/getBackupLogs
Description
Retrieve backup logs using the associated backup ID.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Get Backup Logs command. |
|
default |
The default response on an error. |
Get Kubeconfig for the CDE service
POST /api/v1/de/getKubeconfig
Description
Returns kubeconfig.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Retrieve Cloudera Data Engineering(CDE) Service Initialization Logs
POST /api/v1/de/getServiceInitLogs
Description
Retrieve service initialization logs, if any.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Get Service Init Logs command. |
|
default |
The default response on an error. |
Get CDE Service Upgrade Status.
POST /api/v1/de/getUpgradeStatus
Description
The current status of the CDE Service upgrade. If all the steps are completed allStepsCompleted would be true. After a particular step triggered by upgrade-service is completed nextStep would point to the next step to take.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
List Cloudera Data Engineering(CDE) Service Backups
POST /api/v1/de/listBackups
Description
List all service backups.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for List Backups command. |
|
default |
The default response on an error. |
List Cloudera Data Engineering(CDE) Services
POST /api/v1/de/listServices
Description
Returns list of cde services.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
List Virtual Clusters
POST /api/v1/de/listVcs
Description
Returns a list of virtual clusters for a given CDE service.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Restore Cloudera Data Engineering(CDE) Service
POST /api/v1/de/restoreService
Description
Initiates the service restoration process and returns the ID of the service being restored.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Response object for Restore Service command. |
|
default |
The default response on an error. |
Update Cloudera Data Engineering (CDE) Service
POST /api/v1/de/updateService
Description
Returns update response consisting of operation ID that can be used to log update events.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Update Cloudera Data Engineering (CDE) Virtual Cluster
POST /api/v1/de/updateVc
Description
Updates Cloudera Data Engineering (CDE) Virtual Cluster and returns details of updated CDE service.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Trigger a particular step (prepare, backup, upgrade, resume) of the upgrade framework.
POST /api/v1/de/upgradeService
Description
Trigger a particular step (prepare, backup, upgrade, resume) of the multi-step upgrade process. Use get-upgrade-status to know what is the next step to take.
Parameters
Type | Name | Schema |
---|---|---|
Body |
input |
Responses
HTTP Code | Description | Schema |
---|---|---|
200 |
Expected response to a valid request. |
|
default |
The default response on an error. |
Definitions
AccessControlResponse
Response object containing access control information for a Virtual Cluster.
Name | Description | Schema |
---|---|---|
fullAccessGroups |
Groups with full access. |
< string > array |
fullAccessUsers |
Users with full access. |
< string > array |
users |
Workload usernames of CDP users granted access to the Virtual Cluster. |
< string > array |
viewOnlyGroups |
Groups with view only access. |
< string > array |
viewOnlyUsers |
Users with view only access. |
< string > array |
AllPurposeInstanceGroupDetailsResponse
Instance group details for the All purpose Tier.
Name | Description | Schema |
---|---|---|
initial_instances |
Initial number of Instances when the service is enabled. |
string |
initial_spot_instances |
Initial number of spot Instances when the service is enabled. |
string |
instance_type |
Instance type of the cluster for CDE Service. |
string |
max_instances |
Maximum number of Instances for the CDE Service. |
string |
max_spot_instances |
Maximum number of Spot Instances for the CDE Service. |
string |
min_instances |
Minimum number of Instances for the CDE Service. |
string |
min_spot_instances |
Minimum number of Spot instances for the CDE Service. |
string |
root_vol_size |
EBS volume size in GB. |
string |
Backup
Service backup object.
Name | Description | Schema |
---|---|---|
archiveBasePath |
The path to the base location where the backups are stored. |
string |
archiveLocation |
Archive location under the base path. |
string |
cloudPlatform |
The cloud platform where a backed-up CDE service was enabled. |
string |
created |
The time when the backup was created. |
string |
creator |
The workload name of the user who created this backup. |
string |
creatorCrn |
CRN of of the user who created this backup. |
string |
description |
Backup description. |
string |
environmentCrn |
CRN of CDP environment with which a backed-up CDE service was associated. |
string |
environmentName |
Name of CDP environment with which a backed-up CDE service was associated. |
string |
errorMessage |
Error message, only set when there is an error. |
string |
id |
Backup ID. |
integer (int64) |
serviceID |
Service ID of a backed-up CDE service. |
string |
serviceName |
Name of a backed-up CDE service. |
string |
serviceVersion |
Version of a backed-up CDE service. |
string |
status |
Backup status. |
string |
BackupContentOptions
Backup virtual cluster content options.
Name | Description | Schema |
---|---|---|
credentialFilter |
Filter credentials for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the credential fields (e.g. name, type, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)test-credential'. This will only take effect if credentials are backed up, as controlled by the 'includeCredentials' option. |
< string > array |
includeCredentialSecrets |
Whether to backup credential secrets. Secrets are obfuscated while storing. By default, credential secrets are backed up. |
boolean |
includeCredentials |
Whether to backup credentials. By default, credentials are backed up. |
boolean |
includeJobResources |
Whether to backup job resources. By default, job resources are backed up. |
boolean |
includeJobRuns |
Whether to backup job runs. By default, job runs are backed up. |
boolean |
includeJobs |
Whether to backup jobs. By default, jobs are backed up. |
boolean |
includeResourceCredentials |
Whether to backup resource credentials. By default, resource credentials are backed up. |
boolean |
includeResources |
Whether to backup resources. By default, resources are backed up. |
boolean |
jobFilter |
Filter jobs for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the job fields (e.g. name, type, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)SparkPi'. This will only take effect if jobs are backed up, as controlled by the 'includeJobs' option. |
< string > array |
resourceFilter |
Filter resources for backup using 'fieldname(operator)argument' syntax. Here, 'fieldname' is one of the resource fields (e.g. name, type, status, etc.), and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'name(eq)test-resource'. This will only take effect if resources are backed up, as controlled by the 'includeResources' option. |
< string > array |
validateArchive |
Whether to validate backup archive. By default, backup archive is not validated. |
boolean |
CancelBackupRequest
Request object for Cancel Backup command.
Name | Description | Schema |
---|---|---|
backupID |
The ID of the backup to cancel. |
integer (int64) |
CancelBackupResponse
Response object for Cancel Backup command.
Type : object
ChartValueOverridesRequest
Object containing chart value overrides for enabling a CDE service or creation of virtual clusters.
Name | Description | Schema |
---|---|---|
chartName |
Name of the chart that has to be overridden, for eg- "dex-app", "dex-base". |
string |
overrides |
Space separated key-value pairs for overriding chart values. The key and the value must be separated using a colon(:) For eg- "airflow.enabled:true safari.enabled:true". |
string |
ChartValueOverridesResponse
Response object containing chart value overrides.
Name | Description | Schema |
---|---|---|
chartName |
Name of the chart that has to be overridden, for eg- "dex-app", "dex-base". |
string |
overrides |
Space separated key value-pairs for overriding chart values. The key and the value must be separated using a colon(:) For eg- "airflow.enabled:true safari.enabled:true". |
string |
CreateBackupRequest
Request object for Create Backup command.
Name | Description | Schema |
---|---|---|
backupPath |
Backup file relative path. The path should not include file name, and be relative to the service backup location. The service backup location can be found in the service description obtained via the 'describe-service' command. |
string |
backupVcContentOptions |
Backup virtual cluster content options. |
|
description |
Backup description. |
string |
force |
Whether to ignore validation errors and proceed with the backup forcefully. By default, the backup operation is not forced. |
boolean |
includeVc |
Whether to backup service virtual clusters. By default, virtual clusters are backed up. |
boolean |
serviceId |
Service ID of the service to backup. |
string |
timeout |
Backup operation timeout in seconds. |
integer (int32) |
CreateBackupResponse
Response object for Create Backup command.
Name | Description | Schema |
---|---|---|
backupID |
Backup ID, which can be used for log retrieval, restoration, deletion, and backup cancellation. |
integer (int64) |
CreateVcRequest
Request object for CreateVc method.
Name | Description | Schema |
---|---|---|
aclUsers |
Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster. |
string |
chartValueOverrides |
Chart overrides for creating a virtual cluster. |
< ChartValueOverridesRequest > array |
clusterId |
Cluster id of the CDE service where virtual cluster has to be created. |
string |
cpuRequests |
Cpu requests for autoscaling. |
string |
fullAccessGroups |
Groups with full access. |
< string > array |
fullAccessUsers |
Users with full access. |
< string > array |
gpuRequests |
GPU requests for autoscaling. |
string |
guaranteedCpuRequests |
Guaranteed CPU requests for an Elastic Virtual Cluster. |
string |
guaranteedGpuRequests |
Guaranteed GPU requests for an Elastic Virtual Cluster. |
string |
guaranteedMemoryRequests |
Guaranteed Memory requests for an Elastic Virtual Cluster - eg. 30Gi. |
string |
memoryRequests |
Memory requests for autoscaling - eg. 30Gi. |
string |
name |
Name of the virtual cluster. |
string |
runtimeSpotComponent |
Used to describe where the Driver and the Executors would run. By default the Driver would run on on-demand instances and the Executors on spot instances. Setting it to ALL will run both the Driver and the Executors on spot instances whereas setting it to NONE should run both the Driver and the Executor on on-demand instances. Currently applicable for aws services only. Use this option only on services with spot instances enabled. |
enum (ALL, NONE) |
sessionTimeout |
Set a default timeout for your sessions. The default option is 8 hours. This option can be overridden when creating a new session. |
string |
smtpConfigs |
SMTP Configurations for Airflow Email Alerts. |
|
sparkConfigs |
Spark configs that will be applied to all the spark jobs inside a virtual cluster. |
< string, string > map |
sparkVersion |
Spark version for the virtual cluster. Currently supported Spark versions are SPARK2(deprecated), SPARK3, SPARK3_3 and SPARK3_5. This feature is only supported in CDE-1.7.0 and later. SPARK3_3 is supported in CDE-1.19 and later. SPARK3_5 is supported in CDE-1.21 and later. |
enum (SPARK2, SPARK2_4, SPARK3, SPARK3_0, SPARK3_1, SPARK3_2, SPARK3_3, SPARK3_5) |
vcTier |
Tier of the virtual cluster. Currently supported tiers are CORE and ALLP. CORE tiered virtual cluster enables operational deployment via batch jobs. ALLP virtual clusters are all-purpose virtual clusters supporting both operational batch jobs and interactive sessions. This feature is only supported in CDE-1.19.0 and beyond. |
enum (ALLP, CORE) |
viewOnlyGroups |
Groups with view only access. |
< string > array |
viewOnlyUsers |
Users with view only access. |
< string > array |
CreateVcResponse
Response object for CreateVc method.
Name | Description | Schema |
---|---|---|
Vc |
Created Virtual Cluster |
CustomAzureFilesConfigs
Override Azure Files Configs.
Name | Description | Schema |
---|---|---|
azureFilesFQDN |
Azure File Share’s server address. Defaults to '<storageaccount>.file.core.windows.net'. |
string |
resourceGroup |
Resource Group of the Storage Account. |
string |
storageAccountName |
Azure Storage Account of the File Share. |
string |
DeleteBackupRequest
Request object for Delete Backup command.
Name | Description | Schema |
---|---|---|
backupID |
The ID of the backup to delete. |
integer (int64) |
DeleteBackupResponse
Response object for Delete Backup command.
Type : object
DeleteVcRequest
Request object for DeleteVc method.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster id of the CDE service where virtual cluster was enabled. |
string |
vcId |
Virtual cluster id |
string |
DeleteVcResponse
Response object for DeleteVc method.
Name | Description | Schema |
---|---|---|
status |
status of virtual cluster deletion. |
string |
DescribeBackupRequest
Request object for Describe Backup command.
Name | Description | Schema |
---|---|---|
backupID |
The ID of the backup to describe. |
integer (int64) |
DescribeBackupResponse
Response object for Describe Backup command.
Name | Description | Schema |
---|---|---|
backup |
Service backup. |
DescribeServiceRequest
Request object for DescribeService method.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster id of the service to be described. |
string |
DescribeServiceResponse
Response object for DescribeService method.
Name | Description | Schema |
---|---|---|
service |
Description of the service |
DescribeVcRequest
DescribeVc request object.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster id of the CDE service where vc was created. |
string |
vcId |
Virtual cluster id. |
string |
DescribeVcResponse
DescribeVc response object.
Name | Description | Schema |
---|---|---|
vc |
Description of the virtual cluster. |
DisableServiceRequest
DisableService request object.
Name | Description | Schema |
---|---|---|
clusterId |
cluster Id of the service to disable. |
string |
force |
Force disable cde service. |
boolean |
DisableServiceResponse
DisableService response object.
Name | Description | Schema |
---|---|---|
status |
Status of deletion |
string |
EnableServiceRequest
Request object for Enable Service method.
Name | Description | Schema |
---|---|---|
allPurposeInitialInstances |
Initial Instances when the service is enabled for the All Purpose Instance Group. |
integer (int32) |
allPurposeInitialSpotInstances |
Initial spot Instances when the service is enabled for the All Purpose Instance Group. |
integer (int32) |
allPurposeInstanceType |
Instance type of the cluster for CDE Service for the All Purpose Instance Group. |
string |
allPurposeMaximumInstances |
Maximum Instances for the CDE Service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMaximumSpotInstances |
Maximum Spot Instances for the CDE Service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMinimumInstances |
Minimum Instances for the CDE Service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMinimumSpotInstances |
Minimum Spot instances for the CDE Service for the All Purpose Instance Group. |
integer (int32) |
allPurposeRootVolumeSize |
EBS volume size in GB for the All Purpose Instance Group. |
integer (int32) |
azureDatabasePrivateDNSZoneId |
Resource ID of the custom private DNS zone associated with the Azure database. |
string |
azureFilesharePrivateDNSZoneId |
Resource ID of custom private DNS zone for Azure storage account. |
string |
chartValueOverrides |
Chart overrides for enabling a service. |
< ChartValueOverridesRequest > array |
cpuRequests |
CPU Requests for the entire CDE service quota. |
string |
customAzureFilesConfigs |
CDE uses a default public File Shares storage provisioned by AKS. Enable this option to use your own public/private File Shares. |
|
deployPreviousVersion |
If set to "true", the previous version of the CDE service will be deployed. |
boolean |
enablePrivateNetwork |
Create a fully private CDE instance on either Amazon or Azure. This includes services such as Kubernetes, MySQL, etc. For Azure, this will also enable virtual network (VNet) access via private endpoints and private link. |
boolean |
enablePublicEndpoint |
Creates a CDE endpoint (Load Balancer) in a publicly accessible subnet. If set false, the endpoint will be created in a private subnet and you will need to setup access to the endpoint manually in your cloud account. |
boolean |
enableWorkloadAnalytics |
If set false, diagnostic information about job and query execution is sent to Cloudera Workload Manager. Anonymization can be configured under Environments / Shared Resources / Telemetry. Refer documentation for more info at https://docs.cloudera.com/workload-manager/cloud/index.html. |
boolean |
env |
CDP environment where cde service should be enabled. |
string |
gpuRequests |
GPU requests for the entire CDE service quota. |
string |
initialInstances |
Initial Instances when the service is enabled. |
integer (int32) |
initialSpotInstances |
Initial spot Instances when the service is enabled. |
integer (int32) |
instanceType |
Instance type of the cluster for CDE Service. |
string |
loadbalancerAllowlist |
List of CIDRs that would be allowed to access the load balancer. |
< string > array |
maximumInstances |
Maximum Instances for the CDE Service. |
integer (int32) |
maximumSpotInstances |
Maximum Spot Instances for the CDE Service. |
integer (int32) |
memoryRequests |
Memory requests for the entire CDE service quota, eg. 100Gi. |
string |
minimumInstances |
Minimum Instances for the CDE Service. |
integer (int32) |
minimumSpotInstances |
Minimum Spot instances for the CDE Service. |
integer (int32) |
name |
Name of the CDE Service. |
string |
networkOutboundType |
Network outbound type. Currently 'udr' is the only supported. |
enum (UDR) |
nfsStorageClass |
NFS Storage class to override the default storage class in private cloud. |
string |
resourcePool |
Resource Pool for the CDE service. |
string |
rootVolumeSize |
EBS volume size in GB. |
integer (int32) |
skipValidation |
Skip Validation check. |
boolean |
subnets |
List of Subnet IDs of CDP subnets to use for the kubernetes worker node. |
< string > array |
tags |
User defined labels that tag all provisioned cloud resources. |
< string, string > map |
useSsd |
Instance local storage (SSD) would be used for the workload filesystem (Example - spark local directory). In case the workload requires more space than what’s available in the instance storage, please use an instance type with sufficient instance local storage or choose an instance type without SSD and configure the EBS volume size. Currently supported only for aws services. |
boolean |
whitelistIps |
List of CIDRs that would be allowed to access kubernetes master API server. |
< string > array |
EnableServiceResponse
Response object for EnableService request.
Name | Description | Schema |
---|---|---|
service |
Service details of the created service. |
Error
An object returned on an error.
Name | Description | Schema |
---|---|---|
code |
The error code. |
string |
message |
The error message. |
string |
GetBackupLogsRequest
Request object for Get Backup Logs command.
Name | Description | Schema |
---|---|---|
backupID |
The ID of the backup. |
integer (int64) |
pageSize |
The size of each page. |
integer (int32) |
startingToken |
A token to specify where to start paginating. This is the nextToken from a previously truncated response. |
string |
GetBackupLogsResponse
Response object for the Get Backup Logs command.
Name | Description | Schema |
---|---|---|
logs |
Contains the logs of the backup operation. |
< string > array |
nextToken |
The token to use when requesting the next set of results. If there are no additional results, the string is empty. |
string |
GetKubeconfigRequest
GetKubeconfig request object.
Name | Description | Schema |
---|---|---|
clusterId |
cluster id of the cde service. |
string |
GetKubeconfigResponse
GetKubeconfig response object.
Name | Description | Schema |
---|---|---|
kubeconfig |
kubeconfig for the cluster. |
string |
GetServiceInitLogsRequest
Request object for Get Service Init Logs command.
Name | Description | Schema |
---|---|---|
pageSize |
The size of each page. |
integer (int32) |
serviceID |
The service ID of the service. |
string |
startingToken |
A token to specify where to start paginating. This is the nextToken from a previously truncated response. |
string |
GetServiceInitLogsResponse
Response object for the Get Service Init Logs command.
Name | Description | Schema |
---|---|---|
logs |
Contains the logs of the service initialization operation. |
< string > array |
nextToken |
The token to use when requesting the next set of results. If there are no additional results, the string is empty. |
string |
GetUpgradeStatusRequest
Request object for GetUpgradeStatus method.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster ID of the service. |
string |
GetUpgradeStatusResponse
Response object for GetUpgradeStatus request.
Name | Description | Schema |
---|---|---|
allStepsCompleted |
Whether all the steps of the multi-step upgrade process completed. |
boolean |
nextStep |
The next step to take in multi-step upgrade process. Currently supported steps are prepare, backup, upgrade and resume. |
enum (prepare, backup, upgrade, resume) |
upgradeStatusMessage |
The status of the upgrade. |
string |
ListBackupsRequest
Request object for List Backups command.
Name | Description | Schema |
---|---|---|
filter |
Filter backups using 'fieldname(operator)argument' syntax. Here 'fieldname' is one of ('id', 'clusterID', 'cdeVersion', 'environment', 'environmentCrn', 'created', 'status') and 'operator' is one of ('eq', 'noteq', 'lte', 'lt', 'gte', 'gt', 'in', 'notin', 'like', 'rlike', 'is', 'isnot'). Multiple filters are ANDed. For example, 'status(eq)failed'. |
< string > array |
order |
Whether ordering is ascending ('ASC') or descending ('DESC'), default 'DESC'. |
|
orderBy |
The 'fieldname' to order by, default "id". Here 'fieldname' is one of ('id', 'clusterID', 'cdeVersion', 'environment', 'environmentCrn', 'created', 'status'). |
string |
pageSize |
The size of each page. |
integer (int32) |
startingToken |
A token to specify where to start paginating. This is the nextToken from a previously truncated response. |
string |
ListBackupsResponse
Response object for List Backups command.
Name | Description | Schema |
---|---|---|
backups |
The backups. |
< Backup > array |
nextToken |
The token to use when requesting the next set of results. If there are no additional results, the string is empty. |
string |
ListServicesRequest
Request object for ListServices method.
Name | Description | Schema |
---|---|---|
removeDeleted |
Filter out deleted CDE services from the list. |
boolean |
ListServicesResponse
Response object for ListServices method.
Name | Description | Schema |
---|---|---|
services |
List of services. |
< ServiceSummary > array |
ListVcsRequest
ListVcs request object.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster id of the service. |
string |
ListVcsResponse
ListVcs response object.
Name | Description | Schema |
---|---|---|
vcs |
List of virtual clusters. |
< VcSummary > array |
Order
Order.
Type : enum (ASC, DESC)
RestoreServiceRequest
Request object for Restore Service command.
Name | Description | Schema |
---|---|---|
backupID |
The ID of the backup to restore from. |
integer (int64) |
environmentCrn |
CRN of the CDP environment with which a restored CDE service will be associated. Currently, CDE service can be restored only to the same CDP environment, with which a backed-up service was associated. |
string |
serviceId |
Specify the restored service ID. If not specified, a new service ID will be generated. |
string |
serviceName |
Specify the restored service name. If not specified, the service name from the backup will be reused. |
string |
RestoreServiceResponse
Response object for Restore Service command.
Name | Description | Schema |
---|---|---|
serviceID |
The ID of the service being restored. |
string |
ServiceDescription
Detailed description of a CDE service.
Name | Description | Schema |
---|---|---|
backupLocation |
Base location for the service backup archives. |
string |
chartValueOverrides |
Chart overrides for the Virtual Cluster. |
< ChartValueOverridesResponse > array |
cloudPlatform |
The cloud platform where the CDE service is enabled. |
string |
clusterFqdn |
FQDN of the CDE service. |
string |
clusterId |
Cluster Id of the CDE Service. |
string |
creatorCrn |
CRN of the creator. |
string |
creatorEmail |
Email address of the creator of the CDE service. |
string |
dataLakeAtlasUIEndpoint |
Endpoint of Data Lake Atlas. |
string |
dataLakeFileSystems |
The Data lake file system. |
string |
enablingTime |
Timestamp of service enabling. |
string |
environmentCrn |
CRN of the environment. |
string |
environmentName |
CDP Environment Name. |
string |
loadbalancerAllowlist |
Comma-separated CIDRs that would be allowed to access the load balancer. |
string |
logLocation |
Location for the log files of jobs. |
string |
name |
Name of the CDE Service. |
string |
networkOutboundType |
Network outbound type. Currently 'udr' is the only supported. |
string |
previousVersionDeployed |
The "true" value indicates that the previous version of the CDE service was requested to be deployed. |
boolean |
privateClusterEnabled |
If true, the CDE service was created with fully private Azure services (AKS, MySQL, etc.). |
boolean |
publicEndpointEnabled |
If true, the CDE endpoint was created in a publicly accessible subnet. |
boolean |
resources |
Resources details of CDE Service. |
|
ssdUsed |
If true, SSD would have been be used for workload filesystem. |
boolean |
status |
Status of the CDE service. |
string |
subnets |
List of Subnet IDs of the CDP subnets used by the kubernetes worker node. |
string |
tenantId |
CDP tenant ID. |
string |
whitelistIps |
List of CIDRs that would be allowed to access kubernetes master API server. |
string |
workloadAnalyticsEnabled |
If true, diagnostic information about job and query execution is sent to Cloudera Workload Manager. |
boolean |
ServiceResources
Object to store resources for a CDE service.
Name | Description | Schema |
---|---|---|
allPurposeInstanceGroupDetails |
Resource details for the nodes used in All Purpose Virtual Clusters. |
|
cpuRequests |
CPU Requests for the entire CDE service quota. |
string |
gpuRequests |
GPU requests for the entire CDE service quota. |
string |
initial_instances |
Initial instances for the CDE service. |
string |
initial_spot_instances |
Initial Spot Instances for the CDE Service. |
string |
instance_type |
Instance type of the CDE service. |
string |
max_instances |
Maximum instances for the CDE service. |
string |
max_spot_instances |
Maximum Number of Spot instances. |
string |
memoryRequests |
Memory requests for the entire CDE service quota, eg. 100Gi. |
string |
min_instances |
Minimum Instances for the CDE service. |
string |
min_spot_instances |
Minimum number of spot instances for the CDE service. |
string |
resourcePool |
Resource Pool for the CDE service. |
string |
root_vol_size |
Root Volume Size. |
string |
ServiceSummary
Summary of a CDE service.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster Id of the CDE Service. |
string |
creatorEmail |
Email Address of the CDE creator. |
string |
enablingTime |
Timestamp of service enabling. |
string |
environmentName |
CDP Environment Name. |
string |
name |
Name of the CDE Service. |
string |
status |
Status of the CDE service. |
string |
SmtpConfigRequest
SMTP config request object.
Name | Description | Schema |
---|---|---|
email |
Sender’s email address. |
string |
host |
SMTP host. |
string |
password |
SMTP password. |
string |
port |
SMTP port. |
integer (int32) |
ssl |
Use SSL to secure the connection to the email server. |
boolean |
startTls |
Use SMTP STARTTLS command to encrypt the mail. |
boolean |
username |
SMTP username. |
string |
SmtpConfigResponse
SMTP config response object
Name | Description | Schema |
---|---|---|
email |
Sender’s email address. |
string |
host |
SMTP host. |
string |
port |
SMTP port. |
integer (int32) |
ssl |
Use SSL to secure the connection to the email server. |
boolean |
startTls |
Use SMTP STARTTLS command to encrypt the mail. |
boolean |
username |
SMTP username. |
string |
UpdateServiceRequest
Request object for UpdateService method.
Name | Description | Schema |
---|---|---|
allPurposeMaximumInstances |
Maximum number of instances for the CDE service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMaximumSpotInstances |
Maximum number of spot instances for the CDE service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMinimumInstances |
Minimum number of instances for the CDE service for the All Purpose Instance Group. |
integer (int32) |
allPurposeMinimumSpotInstances |
Minimum number of spot instances for the CDE service for the All Purpose Instance Group. |
integer (int32) |
clusterId |
Cluster ID of the service to update. |
string |
cpuRequests |
CPU Requests for the entire CDE service quota. |
string |
gpuRequests |
GPU requests for the entire CDE service quota. |
string |
loadbalancerAllowlist |
List of CIDRs that would be allowed to access the load balancer. |
< string > array |
maximumInstances |
Maximum number of instances for the CDE service. |
integer (int32) |
maximumSpotInstances |
Maximum number of spot instances for the CDE service. |
integer (int32) |
memoryRequests |
Memory requests for the entire CDE service quota, eg. 100Gi. |
string |
minimumInstances |
Minimum number of instances for the CDE service. |
integer (int32) |
minimumSpotInstances |
Minimum number of spot instances for the CDE service. |
integer (int32) |
whitelistIps |
List of CIDRs that would be allowed to access kubernetes master API server. |
< string > array |
UpdateServiceResponse
Response object for UpdateService request.
Name | Description | Schema |
---|---|---|
operationId |
Deprecated. The operation ID of the update request. |
string |
status |
Status of update. |
string |
UpdateVcRequest
Request object for updateVc method.
Name | Description | Schema |
---|---|---|
aclUsers |
Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster. |
string |
clusterId |
Cluster ID of the CDE service where Virtual Cluster was enabled. |
string |
discardSparkConfigs |
Discard the Spark configs inside a VC. |
boolean |
fullAccessGroups |
Groups with full access. |
< string > array |
fullAccessUsers |
Users with full access. |
< string > array |
sparkConfigs |
Spark configs that will be applied to all the spark jobs inside a virtual cluster. |
< string, string > map |
vcId |
Virtual Cluster ID |
string |
viewOnlyGroups |
Groups with view only access. |
< string > array |
viewOnlyUsers |
Users with view only access. |
< string > array |
UpdateVcResponse
Response object for updateVc method.
Name | Description | Schema |
---|---|---|
Vc |
Updated Virtual Cluster. |
UpgradeServiceRequest
Request object for upgrading a service.
Name | Description | Schema |
---|---|---|
clusterId |
Cluster ID of the service to upgrade. |
string |
step |
The maintenance step to be performed. Currently supported steps are prepare, backup, upgrade, cancel and resume. |
enum (prepare, backup, upgrade, cancel, resume) |
UpgradeServiceResponse
Response object for UpgradeService request.
Name | Description | Schema |
---|---|---|
message |
The status message of upgrade operation. |
string |
VcDescription
CDE virtual cluster summary.
Name | Description | Schema |
---|---|---|
VcUiUrl |
URL of the CDE Virtual Cluster UI. |
string |
accessControl |
Access control object for the Virtual Cluster. |
|
chartValueOverrides |
Chart overrides for the Virtual Cluster. |
< ChartValueOverridesResponse > array |
clusterId |
Cluster ID of the CDE service that contains the Virtual Cluster. |
string |
creationTime |
Time of creation of the virtual Cluster. |
string |
creatorEmail |
Email address of the creator of Virtual Cluster. |
string |
creatorID |
ID of the creator of Virtual Cluster. |
string |
creatorName |
Name of the creator of the Virtual Cluster. |
string |
historyServerUrl |
Spark History Server URL for the Virtual Cluster. |
string |
livyServerUrl |
Livy Server URL for the Virtual Cluster. |
string |
resources |
Resources details of CDE Service. |
|
safariUrl |
Safari URL for the Virtual Cluster. |
string |
sessionTimeout |
Default timeout for Sessions. |
string |
smtpConfig |
SMTP Configurations for Airflow Email Alerts. |
|
sparkConfigs |
Spark configs that will be applied to all the spark jobs inside a virtual cluster. |
< string, string > map |
sparkVersion |
Spark version for the virtual cluster. |
string |
status |
Status of the Virtual Cluster. |
string |
vcApiUrl |
Url for the Virtual Cluster APIs. |
string |
vcId |
Virtual Cluster ID. |
string |
vcName |
Name of the CDE Virtual Cluster. |
string |
vcTier |
Tier of the Virtual Cluster. |
string |
VcResources
Object to store resources for a CDE service
Name | Description | Schema |
---|---|---|
actualCpuRequests |
Actual CPU request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster. |
string |
actualMemoryRequests |
Actual Memory request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster. |
string |
cpuRequests |
The CPU requests for VC for running spark jobs. |
string |
gpuRequests |
GPU requests for autoscaling. |
string |
guaranteedCpuRequests |
Guaranteed CPU requests for an Elastic Virtual Cluster. |
string |
guaranteedGpuRequests |
Guaranteed GPU requests for an Elastic Virtual Cluster. |
string |
guaranteedMemoryRequests |
Guaranteed Memory requests for an Elastic Virtual Cluster - eg. 30Gi. |
string |
memRequests |
The Memory requests for VC for running spark jobs. |
string |
VcSummary
CDE virtual cluster summary.
Name | Description | Schema |
---|---|---|
VcUiUrl |
Url for the Virtual Cluster UI. |
string |
clusterId |
Cluster ID of the CDE service that contains the virtual cluster. |
string |
creatorEmail |
Email Address of the Creator. |
string |
creatorName |
Name of the Creator. |
string |
status |
Status of the Virtual Cluster. |
string |
vcId |
Virtual Cluster ID. |
string |
vcName |
Name of the CDE Virtual Cluster. |
string |