create-vc¶

Description¶

Creates a virtual cluster and returns creation response.

Synopsis¶

  create-vc
--name <value>
--cluster-id <value>
--cpu-requests <value>
--memory-requests <value>
[--chart-value-overrides <value>]
[--smtp-configs <value>]
[--runtime-spot-component <value>]
[--spark-version <value>]
[--vc-tier <value>]
[--session-timeout <value>]
[--acl-users <value>]
[--full-access-users <value>]
[--full-access-groups <value>]
[--view-only-users <value>]
[--view-only-groups <value>]
[--spark-configs <value>]
[--cli-input-json <value>]
[--generate-cli-skeleton]

Options¶

--name (string)

Name of the virtual cluster.

--cluster-id (string)

Cluster id of the CDE service where virtual cluster has to be created.

--cpu-requests (string)

Cpu requests for autoscaling.

--memory-requests (string)

Memory requests for autoscaling - eg. 30Gi.

--chart-value-overrides (array)

Chart overrides for creating a virtual cluster.

Shorthand Syntax:

chartName=string,overrides=string ... (separate items with spaces)

JSON Syntax:

[
  {
    "chartName": "string",
    "overrides": "string"
  }
  ...
]

--smtp-configs (object)

SMTP config request object.

email -> (string)

Sender’s email address.

host -> (string)

SMTP host.

username -> (string)

SMTP username.

password -> (string)

SMTP password.

port -> (integer)

SMTP port.

startTls -> (boolean)

Use SMTP STARTTLS command to encrypt the mail.

ssl -> (boolean)

Use SSL to secure the connection to the email server.

Shorthand Syntax:

email=string,host=string,username=string,password=string,port=integer,startTls=boolean,ssl=boolean

JSON Syntax:

{
  "email": "string",
  "host": "string",
  "username": "string",
  "password": "string",
  "port": integer,
  "startTls": true|false,
  "ssl": true|false
}

--runtime-spot-component (string)

Used to describe where the Driver and the Executors would run. By default the Driver would run on on-demand instances and the Executors on spot instances. Setting it to ALL will run both the Driver and the Executors on spot instances whereas setting it to NONE should run both the Driver and the Executor on on-demand instances. Currently applicable for aws services only. Use this option only on services with spot instances enabled.

Possible values:

• ALL

• NONE

--spark-version (string)

Spark version for the virtual cluster. Currently supported Spark versions are SPARK2(deprecated), SPARK3, SPARK3_3 and SPARK3_5. This feature is only supported in CDE-1.7.0 and later. SPARK3_3 is supported in CDE-1.19 and later. SPARK3_5 is supported in CDE-1.21 and later.

Possible values:

• SPARK2

• SPARK2_4

• SPARK3

• SPARK3_0

• SPARK3_1

• SPARK3_2

• SPARK3_3

• SPARK3_5

--vc-tier (string)

Tier of the virtual cluster. Currently supported tiers are CORE and ALLP. CORE tiered virtual cluster enables operational deployment via batch jobs. ALLP virtual clusters are all-purpose virtual clusters supporting both operational batch jobs and interactive sessions. This feature is only supported in CDE-1.19.0 and beyond.

Possible values:

• ALLP

• CORE

--session-timeout (string)

Set a default timeout for your sessions. The default option is 8 hours. This option can be overridden when creating a new session.

--acl-users (string)

Comma-separated Workload usernames of CDP users to be granted access to the Virtual Cluster.

--full-access-users (array)

Users with full access.

Syntax:

"string" "string" ...

--full-access-groups (array)

Groups with full access.

Syntax:

"string" "string" ...

--view-only-users (array)

Users with view only access.

Syntax:

"string" "string" ...

--view-only-groups (array)

Groups with view only access.

Syntax:

"string" "string" ...

--spark-configs (map)

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

Shorthand Syntax:

KeyName1=string,KeyName2=string

JSON Syntax:

{"string": "string"
  ...}

--cli-input-json (string)

Performs service operation based on the JSON string provided. The JSON string follows the format provided by --generate-cli-skeleton. If other arguments are provided on the command line, the CLI values will override the JSON-provided values.

--generate-cli-skeleton (boolean)

Prints a sample input JSON to standard output. Note the specified operation is not run if this argument is specified. The sample input can be used as an argument for --cli-input-json.

Output¶

Vc -> (object)

CDE virtual cluster summary.

vcId -> (string)

Virtual Cluster ID.

vcName -> (string)

Name of the CDE Virtual Cluster.

clusterId -> (string)

Cluster ID of the CDE service that contains the Virtual Cluster.

status -> (string)

Status of the Virtual Cluster.

resources -> (object)

Object to store resources for a CDE service

cpuRequests -> (string)

The CPU requests for VC for running spark jobs.

memRequests -> (string)

The Memory requests for VC for running spark jobs.

actualCpuRequests -> (string)

Actual CPU request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

actualMemoryRequests -> (string)

Actual Memory request for the VC. This accounts for other dex apps(eg. livy, airflow), that run in the virtual cluster.

creatorEmail -> (string)

Email address of the creator of Virtual Cluster.

creatorID -> (string)

ID of the creator of Virtual Cluster.

creatorName -> (string)

Name of the creator of the Virtual Cluster.

vcApiUrl -> (string)

Url for the Virtual Cluster APIs.

VcUiUrl -> (string)

URL of the CDE Virtual Cluster UI.

historyServerUrl -> (string)

Spark History Server URL for the Virtual Cluster.

livyServerUrl -> (string)

Livy Server URL for the Virtual Cluster.

safariUrl -> (string)

Safari URL for the Virtual Cluster.

creationTime -> (string)

Time of creation of the virtual Cluster.

sparkVersion -> (string)

Spark version for the virtual cluster.

chartValueOverrides -> (array)

Chart overrides for the Virtual Cluster.

item -> (object)

Response object containing chart value overrides.

chartName -> (string)

Name of the chart that has to be overridden, for eg- “dex-app”, “dex-base”.

overrides -> (string)

Space separated key value-pairs for overriding chart values. The key and the value must be separated using a colon(:) For eg- “airflow.enabled:true safari.enabled:true”.

accessControl -> (object)

Response object containing access control information for a Virtual Cluster.

users -> (array)

Workload usernames of CDP users granted access to the Virtual Cluster.

item -> (string)

fullAccessUsers -> (array)

Users with full access.

item -> (string)

fullAccessGroups -> (array)

Groups with full access.

item -> (string)

viewOnlyUsers -> (array)

Users with view only access.

item -> (string)

viewOnlyGroups -> (array)

Groups with view only access.

item -> (string)

smtpConfig -> (object)

SMTP config response object

email -> (string)

Sender’s email address.

host -> (string)

SMTP host.

username -> (string)

SMTP username.

port -> (integer)

SMTP port.

startTls -> (boolean)

Use SMTP STARTTLS command to encrypt the mail.

ssl -> (boolean)

Use SSL to secure the connection to the email server.

vcTier -> (string)

Tier of the Virtual Cluster.

sessionTimeout -> (string)

Default timeout for Sessions.

sparkConfigs -> (map)

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

key -> (string)

value -> (string)

Spark configs that will be applied to all the spark jobs inside a virtual cluster.

Form Factors¶

public, private