Cloudera Lakehouse Optimizer Service

isAssociated
optional

The flag to indicate if the table is associated with the policy. TRUE if associated else FALSE.

boolean

tableName
optional

The table name.

string

namespace
optional

The namespace name.

string

tables
optional

The list of tables associated for a namespace.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

operation
required

The operation to be performed. APPEND will add provided policies to set of existing associated policies, REMOVEALL will delete all existing associations for this table, REPLACEALL will replace existing associations with provided set of policies.
Default : "APPEND"

enum (APPEND, REMOVEALL, REPLACEALL)

policies
optional

The list of policy names.

< string > array

tableName
required

The name of the table.

string

namespace
optional

The namespace name.

string

tables
optional

The list of tables associated for a namespace.

< string > array

associations
optional

List of associations for each namespace.

< CreateAssociations > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

associations
optional

List of associations for each namespace.

< Associations > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

resources
required

The policy resources like script and arguments.

PolicyResource

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

policies
optional

The list of policy names to be associated.

< string > array

tableName
required

The name of the table.

string

policies
optional

The associated policy names.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

Policy name that needs to be deleted. The version is identified through the resource.

string

resourceScope
required

Represents the hierarchical level (<catalog>.<namespace>.<table>) at which a policy is defined and needs to be deleted. Give for catalog level and catalog.namespace. for namespace level.

string

deletedPolicies
optional

Number of policy versions deleted.

integer (int32)

namespace
optional

The namespace.

string

status
optional

The namespace status.
Default : "UNAVAILABLE"

enum (ERROR, SUCCESS, AUTO_PAUSED, USER_PAUSED, PAUSED, UNAVAILABLE, WARNING)

tables
optional

The table to policy associations for a namespace

< TablePolicyAssociation > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The name of the policy.

string

policyVersion
required

The policy version.

string

arguments
optional

The policy arguments.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

script
optional

The policy script.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

actions
optional

The list of actions that would be performed on the table if the policy is executed.

< string > array

details
optional

The detailed output of the dry run in a base64 encoded string.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

cloudPlatform
optional

The cloud platform of the environment, possible values are AZURE or AWS.

string

datahubCrn
optional

The CRN of the CLO Data Hub. If CLO is not enabled it will be null.

string

datahubName
optional

The name of the CLO Data Hub. If CLO is not enabled it will be null.

string

environmentCrn
optional

The CRN of the environment.

string

environmentName
optional

The name of the environment.

string

isCloEnabled
optional

The flag indicating if CLO is enabled. If enabled it is true else false.

boolean

status
optional

The CLO Data Hub status. If CLO is not enabled it will be null.

string

code
optional

The error code.

string

message
optional

The error message.

string

datahubCrn
required

The CRN of the Cloudera Lakehouse Optimizer Data Hub.

string

dryRun
optional

Flag to indicate if the execution is a dry run. If true, it will perform a dry run else actual execution.
Default : true

boolean

namespace
required

Namespace name.

string

policyName
required

Policy name.

string

tableName
required

Table name.

string

dryRunResult
optional

The dry run result. This field will be present only if the request had dryRun as true.

DryRunResult

executionResult
optional

The execution result. This field will be present only if the request had dryRun as false.

ExecutionResult

details
optional

The detailed output of the execution task metadata in a base64 encoded string.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

taskId
optional

The unique identifier for the maintenance task in UUID format.

string

isAssociated
optional

The flag to indicate if the namespace is associated with the policy. TRUE if associated else FALSE.

boolean

namespace
optional

The namespace.

string

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

associatedNamespaces
optional

The namespaces.

< GetAssociatedNamespacesObject > array

namespace
optional

The namespace.

string

tables
optional

The list of table names in the namespace and a flag to indicate if it’s associated.

< AssociatedTableObject > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

associatedTables
optional

The namespace wise table list.

< GetAssociatedTablesObject > array

datahubCrn
required

The CRN of the Cloudera Lakehouse Optimizer Data Hub.

string

directionAssociation
optional

The direct table to policy associations for each namespace.

< DirectAssociationDetails > array

starAssociation
optional

The policy to entire namespace associations for each catalog.

< StarAssociationDetails > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

associations
optional

List of associations for each namespace.

< Associations > array

environmentCrn
required

The CRN of the environment.

string

cloudPlatform
optional

The cloud platform of the environment, possible values are AZURE or AWS.

string

datahubCrn
optional

The CRN of the CLO Data Hub.

string

datahubName
optional

The name of the CLO Data Hub.

string

environmentCrn
optional

The CRN of the environment.

string

environmentName
optional

The name of the environment.

string

status
optional

The CLO Data Hub status.

string

environmentCrn
required

The CRN of the environment.

string

datahubCrn
required

The CRN of the CLO Data Hub.

string

datahubCrn
required

The CRN of the CLO Data Hub.

string

scope
optional

Health check scope. The supported values are "Full" and "Partial". The "Full" performs a deep health check along with livy session creation. The "Partial" performs a shallow health check which doesn’t involve session creation.
Default : "Partial"

enum (Full, Partial)

action
optional

Action performed. For example, READ or NOOP or CREATE.

string

result
required

Map containing the service wise health check result.

< string, HealthCheckResult > map

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespaces
optional

The list of namespace names.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace. Provide * to list policies across all namespaces.

string

policyNames
optional

all policy names for a namespace.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The name of the policy.

string

policyDetail
optional

Policy detail corresponding to a unique policy version.

policyDetail

versions
optional

List of all version IDs of the policy.

< string > array

arguments
optional

JSON containing the action specific arguments and other user defined arguments.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

associations
optional

The list containing table associations for each namespace.

< NamespaceAssociation > array

creationTime
optional

The policy creation time in epoch milliseconds.

integer (int64)

lastExecutionTime
optional

The recent policy execution epoch time in milliseconds.

integer (int64)

scope
optional

The scope of the policy. It can be Catalog or Namespace.

string

tableCount
optional

The count of associated tables.

integer (int32)

version
optional

The policy version which is a UUID.

string

datahubCrn
required

The CRN of the CLO Data Hub.

string

scripts
optional

List of scripts.

< Script > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The namespace name.

string

tables
optional

The list of table names.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

tableName
required

The name of the table.

string

policies
optional

List of all policies associated with a table.

< TablePoliciesObject > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The namespace.

string

policyName
required

The policy name.

string

status
required

The table status. It can be UNAVAILABLE, SUCCESS, ERROR, PAUSED. UNAVAILABLE if no tasks ran recently, SUCCESS if recent task got completed successfully, ERROR if recent task has failed, PAUSED if the table is in paused state.
Default : "UNAVAILABLE"

enum (UNAVAILABLE, SUCCESS, ERROR, PAUSED)

tableName
required

The table name.

string

pausedTable
optional

The paused table details.

PausedTable

recentTask
optional

The recent maintenance task that ran on the table.

Task

timezones
optional

The list of all supported timezones.

< TimezoneObject > array

details
optional

Map with additional service specific details.

< string, string > map

duration
optional

Health check duration in millis.

integer (int64)

error
optional

Exception during health checks.

string

isHealthy
optional

Service health status. The supported values are true or false. "true" represents the service is healthy and "false" represents the service is in an unhealthy state.

boolean

message
optional

Service health check message.

string

time
optional

Time when the health check was performed. It will be in milliseconds.

integer (int64)

pageSize
optional

The page size.
Minimum value : 1
Maximum value : 100

integer (int32)

pageToken
optional

The page token. If the token value is 'ALL' then it will list all existing environments irrespective of pageSize.

string

environments
optional

The list containing environment details.

< Environment > array

nextPageToken
optional

The page token for requesting the next page of results.

string

totalEnvironments
optional

This field represents the total number of environments.

integer (int32)

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
optional

The namespace name. If set, then it will list policies from this namespace. Omitting it, will list policies across all namespaces.

string

policies
optional

List of all policies for a particular namespace or across all namespaces.

< PolicyObject > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The namespace name for which table and it’s policies will be listed.

string

tables
optional

List of all tables for a particular namespace

< TableObject > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

Namespace name to filter tasks.

string

pageSize
optional

The number of tasks.
Default : 1
Minimum value : 1
Maximum value : 100

integer (int32)

pageToken
optional

A token to specify where to start paginating. This is the nextToken from a previously truncated response.

string

nextPageToken
optional

The token to use when requesting the next set of results. If not present, there are no additional results.

string

tasks
optional

The tasks list.

< TaskObject > array

namespace
optional

The name of the namespace.

string

status
optional

The namespace status. Possible values are ERROR, SUCCESS, PAUSED, AUTO_PAUSED, USER_PAUSED, UNAVAILABLE and WARNING. PAUSED indicated that namespace is paused for maintenance actions, AUTO_PAUSED indicates namespace is paused automatically due to errors, USER_PAUSED indicates paused by user, ERROR indicates that recent task has failed or a table is in paused state due to exceptions, WARNING if a table is paused by user, SUCCESS if there are no recent task failures or paused tables, UNAVAILABLE if no tasks have run yet.
Default : "UNAVAILABLE"

enum (ERROR, SUCCESS, AUTO_PAUSED, USER_PAUSED, PAUSED, UNAVAILABLE, WARNING)

tableAssociations
optional

The list of associated tables and their details for a namespace.

< TableAssociation > array

namespace
optional

The namespace.

string

policies
optional

The list of associated policies for the namespace.

< string > array

status
optional

The namespace status.
Default : "UNAVAILABLE"

enum (ERROR, SUCCESS, PAUSED, WARNING, UNAVAILABLE)

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

tableName
required

The name of the table.

string

pausedSince
optional

The paused time in ISO format.

string

reasons
optional

The list containing the reasons for table getting paused.

< string > array

details
optional

The paused details.

PausedDetails

message
optional

The paused message.

string

creationTime
optional

The policy creation time in epoch milliseconds.

integer (int64)

isDefault
optional

Flag to indicate if the policy is a default one provided by Cloudera. Policy is default if true else false.

boolean

lastExecutionTime
optional

The recent task execution time in epoch milliseconds.

integer (int64)

policyName
optional

The policy name.

string

status
optional

The policy status.

enum (ERROR, SUCCESS, PAUSED, UNAVAILABLE)

tableCount
optional

The number of tables associated with this policy.

integer (int32)

taskCount
optional

The number of tasks ran for this policy.

integer (int32)

arguments
optional

A base64 encoded version of policy arguments json.

string

script
optional

A base64 encoded jexl script.

string

description
optional

The script description.

string

name
optional

The script name.

string

uri
optional

The script uri which follows CLO uri format. EG, dlm://tps/hive/snapshot.

string

catalog
optional

The catalog.

string

namespaces
optional

The policy to namespace associations for a catalog.

< NamespacePolicyAssociation > array

status
optional

The catalog status.
Default : "UNAVAILABLE"

enum (ERROR, SUCCESS, PAUSED, WARNING, UNAVAILABLE)

lastExecutionTime
optional

The time in epoch milliseconds of recent CLO task that ran for the table.

integer (int64)

name
optional

The table name.

string

namespace
optional

The namespace name.

string

status
optional

The table status.

enum (ERROR, SUCCESS, PAUSED, UNAVAILABLE)

taskCount
optional

The count of CLO tasks ran for the table.

integer (int32)

version
optional

The policy version applicable to this table.

string

lastExecutionTime
optional

The last task execution time in epoch milliseconds.

integer (int64)

namespace
optional

The name of the namespace.

string

policies
optional

The list of associated policies.

< string > array

status
optional

The table status.

enum (ERROR, SUCCESS, AUTO_PAUSED, USER_PAUSED, PAUSED, UNAVAILABLE, WARNING)

tableName
optional

The name of the table.

string

lastExecutionTime
optional

The last task execution time in epoch milliseconds.

integer (int64)

policyName
optional

The name of the policy.

string

status
optional

The status of the policy on this table.

enum (ERROR, SUCCESS, AUTO_PAUSED, USER_PAUSED, PAUSED, UNAVAILABLE, WARNING)

taskCount
optional

The number of tasks ran for this policies on this table.

integer (int32)

lastExecutionEpoch
optional

The last executed maintenance task in epoch milliseconds.

integer (int64)

policies
optional

The list of associated policies for the table.

< string > array

status
optional

The table status.
Default : "UNAVAILABLE"

enum (ERROR, SUCCESS, USER_PAUSED, AUTO_PAUSED, PAUSED, UNAVAILABLE, WARNING)

tableName
optional

The table name.

string

actions
optional

The list of actions that ran with a policy on a table.

< TaskAction > array

lastExecutionTime
optional

The recent policy execution epoch time in milliseconds.

integer (int64)

policyName
optional

Policy name.

string

tableName
optional

Table name.

string

action
optional

Action that was performed on the table.

string

status
optional

Status of the action.

string

catalog
optional

The catalog name.

string

createdAt
optional

The task creation time in epoch milliseconds.

integer (int64)

details
optional

A base64 encoded version of task details json.
Pattern : "^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$"

string (byte)

namespace
optional

The namespace name.

string

policy
optional

The policy name.

string

status
optional

The task status. It can be INIT, SUBMITTED, COMPLETED, FAILED. Initially the task status would be INIT and transition to SUBMITTED after submitting to spark. Based on the spark execution it changed to COMPLETED, FAILED.
Default : "INIT"

enum (INIT, SUBMITTED, COMPLETED, FAILED)

table
optional

The table name.

string

taskId
optional

Unique identified for task in UUID format.

string

displayName
optional

The timezone display name.

string

id
optional

The timezone id.

string

offset
optional

The offset in milliseconds from UTC.

integer (int32)

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

tableName
required

The name of the table.

string

datahubCrn
required

The CRN of the CLO Data Hub.

string

namespace
required

The name of the namespace.

string

policyName
required

The name of the policy that is to be unsubscribed.

string

tableName
required

The name of the table.

string

policies
optional

The list of policy names which are associated with the table.

< string > array

datahubCrn
required

The CRN of the CLO Data Hub.

string

policyName
required

The policy name.

string

resources
required

The policy resources like script and arguments.

PolicyResource