Package cm_api :: Package endpoints :: Module services :: Class ApiService
[hide private]
[frames] | no frames]

Class ApiService

source code

         object --+    
                  |    
types.BaseApiObject --+
                      |
                     ApiService
Known Subclasses:

Instance Methods [hide private]
 
__init__(self, resource_root, name=None, type=None)
Initializes internal state and sets all known writable properties of the object to None.
source code
 
__str__(self)
Default implementation of __str__.
source code
 
_get_cluster_name(self) source code
 
_path(self)
Return the API path for this service.
source code
 
_cmd(self, cmd, data=None, params=None) source code
 
_role_cmd(self, cmd, roles) source code
 
_parse_svc_config(self, json_dic, view=None)
Parse a json-decoded ApiServiceConfig dictionary into a 2-tuple.
source code
 
get_commands(self, view=None)
Retrieve a list of running commands for this service.
source code
 
get_running_activities(self) source code
 
query_activities(self, query_str=None) source code
 
get_activity(self, job_id) source code
 
get_impala_queries(self, start_time, end_time, filter_str='', limit=100, offset=0)
Returns a list of queries that satisfy the filter
source code
 
cancel_impala_query(self, query_id)
Cancel the query.
source code
 
get_query_details(self, query_id, format='text')
Get the query details
source code
 
get_config(self, view=None)
Retrieve the service's configuration.
source code
 
update_config(self, svc_config, **rt_configs)
Update the service's configuration.
source code
 
create_role(self, role_name, role_type, host_id)
Create a role.
source code
 
delete_role(self, name)
Delete a role by name.
source code
 
get_role(self, name)
Lookup a role by name.
source code
 
get_all_roles(self, view=None)
Get all roles in the service.
source code
 
get_roles_by_type(self, role_type, view=None)
Get all roles of a certain type in a service.
source code
 
get_role_types(self)
Get a list of role types in a service.
source code
 
get_all_role_config_groups(self)
Get a list of role configuration groups in the service.
source code
 
get_role_config_group(self, name)
Get a role configuration group in the service by name.
source code
 
create_role_config_group(self, name, display_name, role_type)
Create a role config group.
source code
 
update_role_config_group(self, name, apigroup)
Update a role config group.
source code
 
delete_role_config_group(self, name)
Delete a role config group by name.
source code
 
get_metrics(self, from_time=None, to_time=None, metrics=None, view=None)
Retrieve metric readings for the service.
source code
 
start(self)
Start a service.
source code
 
stop(self)
Stop a service.
source code
 
restart(self)
Restart a service.
source code
 
start_roles(self, *role_names)
Start a list of roles.
source code
 
stop_roles(self, *role_names)
Stop a list of roles.
source code
 
restart_roles(self, *role_names)
Restart a list of roles.
source code
 
bootstrap_hdfs_stand_by(self, *role_names)
Bootstrap HDFS stand-by NameNodes.
source code
 
create_beeswax_warehouse(self)
DEPRECATED: use create_hive_warehouse on the Hive service.
source code
 
create_hbase_root(self)
Create the root directory of an HBase service.
source code
 
create_hdfs_tmp(self)
Create the /tmp directory in HDFS with appropriate ownership and permissions.
source code
 
refresh(self, *role_names)
Execute the "refresh" command on a set of roles.
source code
 
decommission(self, *role_names)
Decommission roles in a service.
source code
 
recommission(self, *role_names)
Recommission roles in a service.
source code
 
deploy_client_config(self, *role_names)
Deploys client configuration to the hosts where roles are running.
source code
 
disable_hdfs_auto_failover(self, nameservice)
Disable auto-failover for a highly available HDFS nameservice.
source code
 
disable_hdfs_ha(self, active_name, secondary_name, start_dependent_services=True, deploy_client_configs=True, disable_quorum_storage=False)
Disable high availability for an HDFS NameNode.
source code
 
enable_hdfs_auto_failover(self, nameservice, active_fc_name, standby_fc_name, zk_service)
Enable auto-failover for an HDFS nameservice.
source code
 
enable_hdfs_ha(self, active_name, active_shared_path, standby_name, standby_shared_path, nameservice, start_dependent_services=True, deploy_client_configs=True, enable_quorum_storage=False)
Enable high availability for an HDFS NameNode.
source code
 
enable_jt_ha(self, new_jt_host_id, force_init_znode=True, zk_service_name=None)
Enable high availability for a MR JobTracker.
source code
 
disable_jt_ha(self, active_name)
Disable high availability for a MR JobTracker active-standby pair.
source code
 
failover_hdfs(self, active_name, standby_name, force=False)
Initiate a failover of an HDFS NameNode HA pair.
source code
 
format_hdfs(self, *namenodes)
Format NameNode instances of an HDFS service.
source code
 
init_hdfs_auto_failover(self, *controllers)
Initialize HDFS failover controller metadata.
source code
 
init_hdfs_shared_dir(self, *namenodes)
Initialize a NameNode's shared edits directory.
source code
 
roll_edits_hdfs(self, nameservice=None)
Roll the edits of an HDFS NameNode or Nameservice.
source code
 
cleanup_zookeeper(self, *servers)
Cleanup a ZooKeeper service or roles.
source code
 
init_zookeeper(self, *servers)
Initialize a ZooKeeper service or roles.
source code
 
sync_hue_db(self, *servers)
Synchronize the Hue server's database.
source code
 
enter_maintenance_mode(self)
Put the service in maintenance mode.
source code
 
exit_maintenance_mode(self)
Take the service out of maintenance mode.
source code
 
rolling_restart(self, slave_batch_size=None, slave_fail_count_threshold=None, sleep_seconds=None, stale_configs_only=None, unupgraded_only=None, restart_role_types=None, restart_role_names=None)
Rolling restart the roles of a service.
source code
 
create_replication_schedule(self, start_time, end_time, interval_unit, interval, paused, arguments, alert_on_start=False, alert_on_success=False, alert_on_fail=False, alert_on_abort=False)
Create a new replication schedule for this service.
source code
 
get_replication_schedules(self)
Retrieve a list of replication schedules.
source code
 
get_replication_schedule(self, schedule_id)
Retrieve a single replication schedule.
source code
 
delete_replication_schedule(self, schedule_id)
Delete a replication schedule.
source code
 
update_replication_schedule(self, schedule_id, schedule)
Update a replication schedule.
source code
 
trigger_replication_schedule(self, schedule_id, dry_run=False)
Trigger replication immediately.
source code
 
install_oozie_sharelib(self)
Installs the Oozie ShareLib.
source code
 
create_hive_metastore_tables(self)
Creates the Hive metastore tables in the configured database, if it hasn't been done yet.
source code
 
create_hive_warehouse(self)
Creates the Hive warehouse directory in HDFS.
source code
 
create_hive_metastore_database(self)
Create the Hive Metastore Database.
source code
 
update_hive_metastore_namenodes(self)
Update Hive Metastore to point to a NameNode's Nameservice name instead of hostname.
source code

Inherited from types.BaseApiObject: __setattr__, to_json_dict

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __sizeof__, __subclasshook__

Class Methods [hide private]

Inherited from types.BaseApiObject: from_json_dict

Inherited from types.BaseApiObject (private): _get_attributes

Static Methods [hide private]

Inherited from types.BaseApiObject: init

Class Variables [hide private]
  _ATTRIBUTES = {'name': None, 'type': None, 'displayName': None...

Inherited from types.BaseApiObject (private): _WHITELIST

Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, resource_root, name=None, type=None)
(Constructor)

source code 

Initializes internal state and sets all known writable properties of the object to None. Then initializes the properties given in the provided attributes dictionary.

Parameters:
  • resource_root - API resource object.
  • attrs - optional dictionary of attributes to set. This should only contain r/w attributes.
Overrides: object.__init__
(inherited documentation)

__str__(self)
(Informal representation operator)

source code 

Default implementation of __str__. Uses the type name and the first attribute retrieved from the attribute map to create the string.

Overrides: object.__str__
(inherited documentation)

_path(self)

source code 

Return the API path for this service.

This method assumes that lack of a cluster reference means that the object refers to the Cloudera Management Services instance.

_parse_svc_config(self, json_dic, view=None)

source code 

Parse a json-decoded ApiServiceConfig dictionary into a 2-tuple.

Parameters:
  • json_dic - The json dictionary with the config data.
  • view - View to materialize.
Returns:
2-tuple (service config dictionary, role type configurations)

get_commands(self, view=None)

source code 

Retrieve a list of running commands for this service.

Parameters:
  • view - View to materialize ('full' or 'summary')
Returns:
A list of running commands.

get_impala_queries(self, start_time, end_time, filter_str='', limit=100, offset=0)

source code 

Returns a list of queries that satisfy the filter

Parameters:
  • start_time (datetime.datetime) - Queries must have ended after this time
  • end_time (datetime.datetime) - Queries must have started before this time
  • filter - A filter to apply to the queries. For example: 'user = root and queryDuration > 5s'
  • limit - The maximum number of results to return
  • offset - The offset into the return list

cancel_impala_query(self, query_id)

source code 

Cancel the query.

@return The warning message, if any.

get_query_details(self, query_id, format='text')

source code 

Get the query details

Parameters:
  • profile_format - The format of the response ('text' or 'thrift_encoded') @return The details text

get_config(self, view=None)

source code 

Retrieve the service's configuration.

Retrieves both the service configuration and role type configuration for each of the service's supported role types. The role type configurations are returned as a dictionary, whose keys are the role type name, and values are the respective configuration dictionaries.

The 'summary' view contains strings as the dictionary values. The full view contains ApiConfig instances as the values.

Parameters:
  • view - View to materialize ('full' or 'summary') @return 2-tuple (service config dictionary, role type configurations)

update_config(self, svc_config, **rt_configs)

source code 

Update the service's configuration.

@param svc_config Dictionary with service configuration to update. @param rt_configs Dict of role type configurations to update. @return 2-tuple (service config dictionary, role type configurations)

create_role(self, role_name, role_type, host_id)

source code 

Create a role.

Parameters:
  • role_name - Role name
  • role_type - Role type
  • host_id - ID of the host to assign the role to
Returns:
An ApiRole object

delete_role(self, name)

source code 

Delete a role by name.

@param name Role name @return The deleted ApiRole object

get_role(self, name)

source code 

Lookup a role by name.

Parameters:
  • name - Role name
Returns:
An ApiRole object

get_all_roles(self, view=None)

source code 

Get all roles in the service.

Parameters:
  • view - View to materialize ('full' or 'summary')
Returns:
A list of ApiRole objects.

get_roles_by_type(self, role_type, view=None)

source code 

Get all roles of a certain type in a service.

Parameters:
  • role_type - Role type
  • view - View to materialize ('full' or 'summary')
Returns:
A list of ApiRole objects.

get_role_types(self)

source code 

Get a list of role types in a service.

Returns:
A list of role types (strings)

get_all_role_config_groups(self)

source code 

Get a list of role configuration groups in the service.

Returns:
A list of ApiRoleConfigGroup objects.

Since: API v3

get_role_config_group(self, name)

source code 

Get a role configuration group in the service by name.

Parameters:
  • name - The name of the role config group.
Returns:
An ApiRoleConfigGroup object.

Since: API v3

create_role_config_group(self, name, display_name, role_type)

source code 

Create a role config group.

Parameters:
  • name - The name of the new group.
  • display_name - The display name of the new group.
  • role_type - The role type of the new group.
Returns:
New ApiRoleConfigGroup object.

Since: API v3

update_role_config_group(self, name, apigroup)

source code 

Update a role config group.

Parameters:
  • name - Role config group name.
  • apigroup - The updated role config group.
Returns:
The updated ApiRoleConfigGroup object.

Since: API v3

delete_role_config_group(self, name)

source code 

Delete a role config group by name.

Parameters:
  • name - Role config group name.
Returns:
The deleted ApiRoleConfigGroup object.

Since: API v3

get_metrics(self, from_time=None, to_time=None, metrics=None, view=None)

source code 

Retrieve metric readings for the service.

Parameters:
  • from_time - A datetime; start of the period to query (optional).
  • to_time - A datetime; end of the period to query (default = now).
  • metrics - List of metrics to query (default = all).
  • view - View to materialize ('full' or 'summary') @return List of metrics and their readings.

start(self)

source code 

Start a service.

@return Reference to the submitted command.

stop(self)

source code 

Stop a service.

@return Reference to the submitted command.

restart(self)

source code 

Restart a service.

@return Reference to the submitted command.

start_roles(self, *role_names)

source code 

Start a list of roles.

Parameters:
  • role_names - names of the roles to start.
Returns:
List of submitted commands.

stop_roles(self, *role_names)

source code 

Stop a list of roles.

Parameters:
  • role_names - names of the roles to stop.
Returns:
List of submitted commands.

restart_roles(self, *role_names)

source code 

Restart a list of roles.

Parameters:
  • role_names - names of the roles to restart.
Returns:
List of submitted commands.

bootstrap_hdfs_stand_by(self, *role_names)

source code 

Bootstrap HDFS stand-by NameNodes.

Initialize their state by syncing it with the respective HA partner.

Parameters:
  • role_names - NameNodes to bootstrap.
Returns:
List of submitted commands.

create_beeswax_warehouse(self)

source code 

DEPRECATED: use create_hive_warehouse on the Hive service. Deprecated since v3.

Create the Beeswax role's warehouse for a Hue service.

Returns:
Reference to the submitted command.

create_hbase_root(self)

source code 

Create the root directory of an HBase service.

@return Reference to the submitted command.

create_hdfs_tmp(self)

source code 

Create the /tmp directory in HDFS with appropriate ownership and permissions.

Returns:
Reference to the submitted command

Since: API v2

refresh(self, *role_names)

source code 

Execute the "refresh" command on a set of roles.

Returns:
Reference to the submitted command.

decommission(self, *role_names)

source code 

Decommission roles in a service.

@param role_names Names of the roles to decommission. @return Reference to the submitted command.

recommission(self, *role_names)

source code 

Recommission roles in a service.

@param role_names Names of the roles to recommission. @return Reference to the submitted command.

Since: API v2

deploy_client_config(self, *role_names)

source code 

Deploys client configuration to the hosts where roles are running.

Returns:
Reference to the submitted command.

disable_hdfs_auto_failover(self, nameservice)

source code 

Disable auto-failover for a highly available HDFS nameservice.

Parameters:
  • nameservice - Affected nameservice.
Returns:
Reference to the submitted command.

disable_hdfs_ha(self, active_name, secondary_name, start_dependent_services=True, deploy_client_configs=True, disable_quorum_storage=False)

source code 

Disable high availability for an HDFS NameNode.

Parameters:
  • active_name - Name of the NameNode to keep.
  • secondary_name - Name of (existing) SecondaryNameNode to link to remaining NameNode.
  • start_dependent_services - whether to re-start dependent services.
  • deploy_client_configs - whether to re-deploy client configurations.
  • disable_quorum_storage - whether to disable Quorum-based Storage. Available since API v2. Quorum-based Storage will be disabled for all nameservices that have Quorum-based Storage enabled.
Returns:
Reference to the submitted command.

enable_hdfs_auto_failover(self, nameservice, active_fc_name, standby_fc_name, zk_service)

source code 

Enable auto-failover for an HDFS nameservice.

Parameters:
  • nameservice - Nameservice for which to enable auto-failover.
  • active_fc_name - Name of failover controller to create for active node.
  • standby_fc_name - Name of failover controller to create for stand-by node.
  • zk_service - ZooKeeper service to use.
Returns:
Reference to the submitted command.

enable_hdfs_ha(self, active_name, active_shared_path, standby_name, standby_shared_path, nameservice, start_dependent_services=True, deploy_client_configs=True, enable_quorum_storage=False)

source code 

Enable high availability for an HDFS NameNode.

Parameters:
  • active_name - name of active NameNode.
  • active_shared_path - shared edits path for active NameNode. Ignored if Quorum-based Storage is being enabled.
  • standby_name - name of stand-by NameNode.
  • standby_shared_path - shared edits path for stand-by NameNode. Ignored if Quourm Journal is being enabled.
  • nameservice - nameservice for the HA pair.
  • start_dependent_services - whether to re-start dependent services.
  • deploy_client_configs - whether to re-deploy client configurations.
  • enable_quorum_storage - whether to enable Quorum-based Storage. Available since API v2. Quorum-based Storage will be enabled for all nameservices except those configured with NFS High Availability.
Returns:
Reference to the submitted command.

enable_jt_ha(self, new_jt_host_id, force_init_znode=True, zk_service_name=None)

source code 

Enable high availability for a MR JobTracker.

Parameters:
  • zk_service_name - Name of the ZooKeeper service to use for auto-failover. If MapReduce service depends on a ZooKeeper service then that ZooKeeper service will be used for auto-failover and in that case this parameter can be omitted.
  • new_jt_host_id - id of the host where the second JobTracker will be added.
  • force_init_znode - Initialize the ZNode used for auto-failover even if it already exists. This can happen if JobTracker HA was enabled before and then disabled. Disable operation doesn't delete this ZNode. Defaults to true.
Returns:
Reference to the submitted command.

disable_jt_ha(self, active_name)

source code 

Disable high availability for a MR JobTracker active-standby pair.

Parameters:
  • active_name - name of the JobTracker that will be active after the disable operation. The other JobTracker and Failover Controllers will be removed.
Returns:
Reference to the submitted command.

failover_hdfs(self, active_name, standby_name, force=False)

source code 

Initiate a failover of an HDFS NameNode HA pair.

This will make the given stand-by NameNode active, and vice-versa.

Parameters:
  • active_name - name of currently active NameNode.
  • standby_name - name of NameNode currently in stand-by.
  • force - whether to force failover.
Returns:
Reference to the submitted command.

format_hdfs(self, *namenodes)

source code 

Format NameNode instances of an HDFS service.

@param namenodes Name of NameNode instances to format. @return List of submitted commands.

init_hdfs_auto_failover(self, *controllers)

source code 

Initialize HDFS failover controller metadata.

Only one controller per nameservice needs to be initialized.

Parameters:
  • controllers - Name of failover controller instances to initialize.
Returns:
List of submitted commands.

init_hdfs_shared_dir(self, *namenodes)

source code 

Initialize a NameNode's shared edits directory.

@param namenodes Name of NameNode instances. @return List of submitted commands.

roll_edits_hdfs(self, nameservice=None)

source code 

Roll the edits of an HDFS NameNode or Nameservice.

Parameters:
  • nameservice - Nameservice whose edits should be rolled. Required only with a federated HDFS.
Returns:
Reference to the submitted command.

Since: API v3

cleanup_zookeeper(self, *servers)

source code 

Cleanup a ZooKeeper service or roles.

If no server role names are provided, the command applies to the whole service, and cleans up all the server roles that are currently running.

Parameters:
  • servers - ZK server role names (optional).
Returns:
Command reference (for service command) or list of command references (for role commands).

init_zookeeper(self, *servers)

source code 

Initialize a ZooKeeper service or roles.

If no server role names are provided, the command applies to the whole service, and initializes all the configured server roles.

Parameters:
  • servers - ZK server role names (optional).
Returns:
Command reference (for service command) or list of command references (for role commands).

sync_hue_db(self, *servers)

source code 

Synchronize the Hue server's database.

Returns:
List of submitted commands.

enter_maintenance_mode(self)

source code 

Put the service in maintenance mode.

Returns:
Reference to the completed command.

Since: API v2

exit_maintenance_mode(self)

source code 

Take the service out of maintenance mode.

Returns:
Reference to the completed command.

Since: API v2

rolling_restart(self, slave_batch_size=None, slave_fail_count_threshold=None, sleep_seconds=None, stale_configs_only=None, unupgraded_only=None, restart_role_types=None, restart_role_names=None)

source code 

Rolling restart the roles of a service. The sequence is:
1. Restart all the non-slave roles
2. If slaves are present restart them in batches of size specified
3. Perform any post-command needed after rolling restart

@param: slave_batch_size Number of slave roles to restart at a time
        Must be greater than 0. Default is 1.
        For HDFS, this number should be less than the replication factor (default 3)
        to ensure data availability during rolling restart.
@param: slave_fail_count_threshold The threshold for number of slave batches that
        are allowed to fail to restart before the entire command is considered failed.
        Must be >= 0. Default is 0.
@param: sleep_seconds Number of seconds to sleep between restarts of slave role batches.
        Must be >=0. Default is 0.
@param: stale_configs_only Restart roles with stale configs only. Default is false.
@param: unupgraded_only Restart roles that haven't been upgraded yet. Default is false.
@param: restart_role_types Role types to restart. If not specified, all startable roles are restarted.
@param: restart_role_names List of specific roles to restart.
        If none are specified, then all roles of specified role types are restarted.
@return: Reference to the submitted command.
@since: API v3

create_replication_schedule(self, start_time, end_time, interval_unit, interval, paused, arguments, alert_on_start=False, alert_on_success=False, alert_on_fail=False, alert_on_abort=False)

source code 

Create a new replication schedule for this service.

The replication argument type varies per service type. The following types are recognized:

  • HDFS: ApiHdfsReplicationArguments
  • Hive: ApiHiveReplicationArguments
Parameters:
  • start_time (datetime.datetime) - The time at which the schedule becomes active and first executes.
  • end_time (datetime.datetime) - The time at which the schedule will expire.
  • interval_unit (str) - The unit of time the `interval` represents. Ex. MINUTE, HOUR, DAY. See the server documentation for a full list of values.
  • interval (int) - The number of time units to wait until triggering the next replication.
  • paused (bool) - Should the schedule be paused? Useful for on-demand replication.
  • arguments - service type-specific arguments for the replication job.
  • alert_on_start - whether to generate alerts when the job is started.
  • alert_on_success - whether to generate alerts when the job succeeds.
  • alert_on_fail - whether to generate alerts when the job fails.
  • alert_on_abort - whether to generate alerts when the job is aborted.
Returns:
The newly created schedule.

Since: API v3

get_replication_schedules(self)

source code 

Retrieve a list of replication schedules.

Returns:
A list of replication schedules.

Since: API v3

get_replication_schedule(self, schedule_id)

source code 

Retrieve a single replication schedule.

Parameters:
  • schedule_id - The id of the schedule to retrieve.
Returns:
The requested schedule.

Since: API v3

delete_replication_schedule(self, schedule_id)

source code 

Delete a replication schedule.

Parameters:
  • schedule_id - The id of the schedule to delete.
Returns:
The deleted replication schedule.

Since: API v3

update_replication_schedule(self, schedule_id, schedule)

source code 

Update a replication schedule.

Parameters:
  • schedule_id - The id of the schedule to update.
  • schedule - The modified schedule.
Returns:
The updated replication schedule.

Since: API v3

trigger_replication_schedule(self, schedule_id, dry_run=False)

source code 

Trigger replication immediately. Start and end dates on the schedule will be ignored.

Parameters:
  • schedule_id - The id of the schedule to trigger.
  • dry_run - Whether to execute a dry run.
Returns:
The command corresponding to the replication job.

Since: API v3

install_oozie_sharelib(self)

source code 

Installs the Oozie ShareLib. Oozie must be stopped before running this command.

Returns:
Reference to the submitted command.

Since: API v3

create_hive_metastore_tables(self)

source code 

Creates the Hive metastore tables in the configured database, if it hasn't been done yet.

Returns:
Reference to the submitted command.

Since: API v3

create_hive_warehouse(self)

source code 

Creates the Hive warehouse directory in HDFS.

Returns:
Reference to the submitted command.

Since: API v3

create_hive_metastore_database(self)

source code 

Create the Hive Metastore Database. Only works with embedded postgresql database. This command should usually be followed by a call to hiveCreateMetastoreDatabaseTables.

Returns:
Reference to the submitted command.

Since: API v4

update_hive_metastore_namenodes(self)

source code 

Update Hive Metastore to point to a NameNode's Nameservice name instead of hostname. Only available when all Hive Metastore Servers are stopped and HDFS has High Availability.

Back up the Hive Metastore Database before running this command.

Returns:
Reference to the submitted command.

Since: API v4


Class Variable Details [hide private]

_ATTRIBUTES

Value:
{'name': None, 'type': None, 'displayName': None, 'serviceState': ROAt\
tr(), 'healthSummary': ROAttr(), 'healthChecks': ROAttr(), 'clusterRef\
': ROAttr(ApiClusterRef), 'configStale': ROAttr(), 'serviceUrl': ROAtt\
r(), 'maintenanceMode': ROAttr(), 'maintenanceOwners': ROAttr(),}