Llama Default Configuration

[Go back to Llama Documentation]

namevaluedescription
llama.am.server.thrift.address0.0.0.0:15000The address the LlamaAM server listen at. If 0.0.0.0 is specified as IP, the server will listen in all available network addresses. If the port is not specified, the default port is 15000. If the specified port is 0, an ephemeral port will be used, the port in use will be printed in the logs at startup.
llama.am.server.thrift.server.min.threads10 Minimum number of threads used by the LlamaAM server uses for serving client requests.
llama.am.server.thrift.server.max.threads50 Maximum number of threads used by the LlamaAM server uses for serving client requests. This should be to to the size of the cluster plus a buffer (thrift synch IO requires one thread per connection, we need one connection per impalad node).
llama.am.server.thrift.transport.timeout.ms60000 Socket time, in milliseconds, used LlamaAM server for all its server and client Thrift connections.
llama.am.server.thrift.client.notifier.queue.threshold10000 Threshold of the outstanding client notification queue size to start producing warnings. The queue will continue to queue notifications requests when above the threshold.
llama.am.server.thrift.client.notifier.threads10 Number of threads used to do client notifications.
llama.am.server.thrift.client.notifier.max.retries5 Maximum number of retries for a client notification. After the maximum number of client notification retries has been reached without success the client is considered lost and all its reservations are released. A successful client notification resets the retries count.
llama.am.server.thrift.client.notifier.retry.interval.ms5000 Client notification retry interval, in milliseconds.
llama.am.server.thrift.client.notifier.heartbeat.ms5000 Heartbeat interval (if no other notification happened), from LlamaAM server to clients.
llama.am.server.thrift.node.name.mapping.classcom.cloudera.llama.am.HostnameOnlyNodeMapper The NodeMapper implementation LlamaAM server uses to convert requested locations into Yarn Nodes. The default (and only implementation for production) drops the port number if present (Impala uses DataNode addresses to request a location, these addresses may contain the DataNode port number. The DataNode port number is meaningless and unknown to Yarn).
llama.am.server.thrift.admin.addresslocalhost:15002The admin address the LlamaAM server listen at. If 0.0.0.0 is specified as IP, the server will listen in all available network addresses. If the port is not specified, the default port is 15002. If the specified port is 0, an ephemeral port will be used, the port in use will be printed in the logs at startup. IMPORTANT: It is strongly recommended to use 'localhost' as the bind address when security is not enabled.
llama.am.server.thrift.http.address0.0.0.0:15001The address the LlamaAM server exposes its HTTP server for JMX and the Web UI. If 0.0.0.0 is specified as IP, the server will listen in all available network addresses. If the port is not specified, the default port is 15001. The HTTP JSON JMX servlet is exposed over HTTP at '/jmx', i.e.: http://localhost:15001/jmx If the specified port is 0, an ephemeral port will be used, the port in use will be printed in the logs at startup.
llama.am.server.thrift.loggers.servlet.read.onlytrue If the /loggers servlet is read only. If not in read only mode, the servlet can be used to change logger levels.
llama.am.server.thrift.securityfalse Indicates if security is enabled or not. If enabled, LlamaAM server uses Kerberos Thrift SASL for all server and client Thrift connections.
llama.am.server.thrift.security.QOPauth Indicates the quality of protection if security is enabled. Valid values are: 'auth' : authentication 'auth-int' : authentication and integrity 'auth-conf' : authentication, integrity and confidentiality
llama.am.server.thrift.kerberos.keytab.filellama.keytab The location of the LlamaAM server keytab. If the path is relative, the keytab file is looked up in LlamaAM configuration directory.
llama.am.server.thrift.kerberos.server.principal.namellama/localhost LlamaAM Kerberos principal name. 'localhost' must be replaced with the hostname specified in the service principal.
llama.am.server.thrift.kerberos.notification.principal.nameimpala Principal short name, without the service hostname, used for client notifications. The hostname provided in the client address at registration by the client will be used as service hostname. IMPORTANT: they client hostname address provided at registration must match the service name in the client's Kerberos principal.
llama.am.server.thrift.client.acl* ACL for Llama AM clients. The ACL is a comma-separated list of user and group names. The user and group list is separated by a blank. For e.g. "alice,bob users,wheel". A special value of "*" means all users are allowed.
llama.am.server.thrift.admin.acl* ACL for Llama AM admins. The ACL is a comma-separated list of user and group names. The user and group list is separated by a blank. For e.g. "alice,bob users,wheel". A special value of "*" means all users are allowed.
llama.am.cluster.idllama Identification of a Llama (or an Active/Standby pair). YARN resources are grouped by this ID. When multiple applications on a YARN cluster use their own Llamas, it is recommended that they use different values for this config. On Llama restart, all YARN applications corresponding to this cluster-id are killed.
llama.am.rm.connector.classcom.cloudera.llama.am.yarn.YarnRMConnector Backing LlamaAM implementation to use. Available for functional testing: MockRMLlamaAMConnector
llama.am.rm.connector.recycle.interval.mins1380 Yarn AM-RM protocol does not have a way of renewing long lasting tokens. So, Llama recycles the yarn connectors to avoid any failures due to invalid tokens. By default its set to 75% of the value yarn.resourcemanager.delegation.token.renew-interval in yarn-site.xml. It can be overridden here but it should always be less than the yarn configured value.
llama.am.core.queues Queues LlamaAM should connect to at start up. Unlike queues that are created in response to a reservation, these queues will never expire. IMPORTANT: all queue names should be canonical queue names, this is prefixed with "root.". This is required because Yarn normalizes queue names to their canonical name.
llama.am.gang.anti.deadlock.enabledtrue Enables Llama AM gang scheduling anti deadlock detection.
llama.am.gang.anti.deadlock.no.allocation.limit.ms30000 Interval of time without any new allocation that will trigger the Llama AM anti-deadlock logic.
llama.am.gang.anti.deadlock.backoff.percent30 Percentage of resources that will be backed off by the Llama AM anti-deadlock logic. Random reservations will be backed off until the percentage of backed off resources reaches this percentage.
llama.am.gang.anti.deadlock.backoff.min.delay.ms10000 Minimum amount of time the backed off reservations will be in 'backed off' state. The actual amount time is a random value between the minimum and the maximum.
llama.am.gang.anti.deadlock.backoff.max.delay.ms30000 Maximum amount of time the backed off reservations will be in 'backed off' state. The actual amount time is a random value between the minimum and the maximum.
llama.am.throttling.enabledtrue Global setting that indicates if Llama should throttle reservations.
llama.am.throttling.enabled.#QUEUE#true Per queue setting that indicates if Llama should throttle reservations for the #QUEUE# queue. If not set, the 'llama.am.throttling.enabled' is used.
llama.am.throttling.maximum.placed.reservations10000 Global maximum number of reservations per queue. Once this number of reservations is reached for a queue, reservations are queued up.
llama.am.throttling.maximum.queued.reservations0 Global maximum number of queued reservations per queue. Once this number of reservations is reached for a queue, new reservations are rejected.
llama.am.throttling.maximum.placed.reservations.#QUEUE#10000 Maximum number of reservations for #QUEUE# queue. Once this number of reservations is reached for the #QUEUE# queue, reservations are queued up. If this property is not set for the #QUEUE#, the global configuration property is used 'llama.am.throttling.maximum.placed.reservations'.
llama.am.throttling.maximum.queued.reservations.#QUEUE#0 Maximum number of queued reservations for #QUEUE# queue. Once this number of reservations is reached for the #QUEUE# queue, new reservations are rejected. If this property is not set for the #QUEUE#, the global configuration property is used 'llama.am.throttling.maximum.queued.reservations'.
llama.am.mock.nodesnode1,node2 List of nodes to offer.
llama.am.mock.queuesqueue1,queue2 List of queues to offer.
llama.am.mock.events.min.wait.ms1000 Minimum wait time, in milliseconds, for events to be delivered after reservation. Actual wait time is a random value.
llama.am.mock.events.max.wait.ms10000 Maximum wait time, in milliseconds, for events to be delivered after reservation. Actual wait time is a random value.
llama.am.cache.enabledtrue Global setting that indicates if Llama should cache allocated resources on release.
llama.am.cache.enabled.#QUEUE#true Per queue setting that indicates if Llama should cache allocated resources on release for the #QUEUE# queue. If not set, the 'llama.am.caching.enabled' is used.
llama.am.cache.eviction.run.interval.timeout.ms5000 Interval of time between eviction policy runs.
llama.am.cache.eviction.policy.classcom.cloudera.llama.am.cache.ResourceCache$TimeoutEvictionPolicy The eviction policy for cached resources. The TimeoutEvictionPolicy evicts resources that have been sitting in the cache for a period of time equal or greater than the timeout.
llama.am.cache.eviction.timeout.policy.idle.timeout.ms30000 TimeoutEvictionPolicy policy timeout for resources sitting in the cache.
llama.am.queue.expire.ms300000 Time in milliseconds after which Llama will discard its AM for a queue that has been empty of reservations. Does not apply to queues specified with the llama.am.core.queues property.
llama.am.resource.normalizing.enabledtrue Whether to break resource requests into smaller requests of standard size before the cache.
llama.am.resource.normalizing.standard.mbs1024 The standard size in MB to break requests into when normalizing is turned on.
llama.am.resource.normalizing.standard.vcores1 The standard size in vcores to break requests into when normalizing is turned on.
llama.am.hadoop.user.namellama User name use by Llama when interacting with Yarn.
llama.am.yarn.priority0 Application priority when creating application in Yarn Resource Manager. NOTE: currently YARN does not use the application priority for scheduling decisions.
llama.am.yarn.app.monitor.timeout.ms30000 Timeout, in milliseconds, for waiting the Application Master to start or to stop.
llama.am.yarn.app.monitor.polling.ms200 Polling interval, in milliseconds, to determine if the Application Master has started or stopped.
llama.am.yarn.app.heartbeat.interval.ms200 LlamaAM Application Master heartbeat interval, in milliseconds. On each heartbeat the Application Master submits new reservations to Yarn Resource Manager and gets updates from it.
llama.am.yarn.container.handler.queue.threshold10000 Threshold of the outstanding container requests queue size to Yarn Node Managers to start producing warnings. The queue will continue to queue container requests when above the threshold.
llama.am.yarn.container.handler.threads10 Number of threads used to do container requests to Yarn Node Managers.
llama.nm.server.thrift.address0.0.0.0:15100The address the Llama NM Auxiliary Service listen at. If 0.0.0.0 is specified as IP, the server will listen in all available network addresses. IMPORTANT: if security is enabled do not use 0.0.0.0, instead, use the exact same hostname used in the kerberos service principal of the LlamaNM auxiliary service (i.e. llama/HOSTNAME). If the port is not specified, the default port is 15100. If the specified port is 0, an ephemeral port will be used, the port in use will be printed in the logs at startup.
llama.nm.server.thrift.server.min.threads10 Minimum number of threads used by the LlamaNM auxiliary service uses for serving client requests.
llama.nm.server.thrift.server.max.threads50 Maximum number of threads used by the LlamaNM auxiliary service uses for serving client requests.
llama.nm.server.thrift.transport.timeout.ms3600000 Socket time, in milliseconds, used LlamaNM auxiliary service for all its server and client Thrift connections.
llama.nm.server.thrift.client.notifier.queue.threshold10000 Threshold of the outstanding client notification queue size to start producing warnings. The queue will continue to queue notifications requests when above the threshold.
llama.nm.server.thrift.client.notifier.threads10 Number of threads used to do client notifications.
llama.nm.server.thrift.client.notifier.max.retries5 Maximum number of retries for a client notification. After the maximum number of client notification retries has been reached without success the client is considered lost and all its reservations are released. A successful client notification resets the retries count.
llama.nm.server.thrift.client.notifier.retry.interval.ms5000 Client notification retry interval, in milliseconds.
llama.nm.server.thrift.client.notifier.heartbeat.ms5000 Heartbeat interval (if no other notification happened), from LlamaNM auxiliary service to clients.
llama.nm.server.thrift.securityfalse Indicates if security is enabled or not. If enabled, LlamaNM auxiliary service uses Kerberos Thrift SASL for all server and client Thrift connections.
llama.nm.server.thrift.kerberos.keytab.filellama.keytab The location of the Llama NM auxiliary service keytab. If the path is relative, the keytab file is looked up in the Node Manager configuration directory.
llama.nm.server.thrift.kerberos.server.principal.namellama/localhost Llama NM auxiliary service keytab Kerberos principal name. 'localhost' must be replaced with the hostname specified in the service principal.
llama.nm.server.thrift.kerberos.notification.principal.nameimpala Principal short name, without the service hostname, used for client notifications. The hostname provided in the client address at registration by the client will be used as service hostname. IMPORTANT: they client hostname address provided at registration must match the service name in the client's Kerberos principal.
llama.nm.server.thrift.client.acl* ACL for Llama NM clients. The ACL is a comma-separated list of user and group names. The user and group list is separated by a blank. For e.g. "alice,bob users,wheel". A special value of "*" means all users are allowed.
llama.nm.server.thrift.admin.acl* ACL for Llama NM admins. The ACL is a comma-separated list of user and group names. The user and group list is separated by a blank. For e.g. "alice,bob users,wheel". A special value of "*" means all users are allowed.
hadoop.security.group.mappingorg.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback Class for user to group mapping (get groups for a given user) for ACL. The default implementation, org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback, will determine if the Java Native Interface (JNI) is available. If JNI is available the implementation will use the API within hadoop to resolve a list of groups for a user. If JNI is not available then the shell implementation, ShellBasedUnixGroupsMapping, is used. This implementation shells out to the Linux/Unix environment with the bash -c groups command to resolve a list of groups for a user.
hadoop.security.groups.cache.secs300 This is the config controlling the validity of the entries in the cache containing the user->group mapping. When this duration has expired, then the implementation of the group mapping provider is invoked to get the groups of the user and then cached back.
llama.am.ha.enabledfalse Enable Llama HA. When enabled, Llama starts in Standby mode and participates in leader election. When elected leader, it transitions to Active. When HA is enabled, one is required to set llama.am.ha.zk-quorum.
llama.am.ha.zk-quorum The Zookeeper quorum to use for leader election and fencing when HA is enabled.
llama.am.ha.zk-base/llama The base znode to be used to store all HA-related information. If using the same Zookeeper quorum for multiple Llama clusters, make sure each Llama cluster has a separate base znode. Also, make sure the path is absolute, i.e., starts with "/".
llama.am.ha.zk-timeout-ms10000 The session timeout to be used for connections to the Zookeeper quorum.
llama.am.ha.zk-aclworld:anyone:rwcda ACLs to control access to the Zookeeper quorum used for HA.
llama.am.ha.zk-auth Auth information to go with llama.am.ha.zk-acl