| name | value | description |
| llama.am.server.thrift.address | 0.0.0.0:15000 | The address the LlamaAM server listen at.
If 0.0.0.0 is specified as IP, the server will listen in all available
network addresses. If the port is not specified, the default port is 15000.
If the specified port is 0, an ephemeral port will be used, the port in
use will be printed in the logs at startup.
|
| llama.am.server.thrift.server.min.threads | 10 |
Minimum number of threads used by the LlamaAM server uses for serving
client requests.
|
| llama.am.server.thrift.server.max.threads | 50 |
Maximum number of threads used by the LlamaAM server uses for serving
client requests. This should be to to the size of the cluster plus a
buffer (thrift synch IO requires one thread per connection, we need one
connection per impalad node).
|
| llama.am.server.thrift.transport.timeout.ms | 60000 |
Socket time, in milliseconds, used LlamaAM server for all its server and
client Thrift connections.
|
| llama.am.server.thrift.client.notifier.queue.threshold | 10000 |
Threshold of the outstanding client notification queue size to start
producing warnings. The queue will continue to queue notifications
requests when above the threshold.
|
| llama.am.server.thrift.client.notifier.threads | 10 |
Number of threads used to do client notifications.
|
| llama.am.server.thrift.client.notifier.max.retries | 5 |
Maximum number of retries for a client notification.
After the maximum number of client notification retries has been reached
without success the client is considered lost and all its reservations
are released.
A successful client notification resets the retries count.
|
| llama.am.server.thrift.client.notifier.retry.interval.ms | 5000 |
Client notification retry interval, in milliseconds.
|
| llama.am.server.thrift.client.notifier.heartbeat.ms | 5000 |
Heartbeat interval (if no other notification happened), from LlamaAM
server to clients.
|
| llama.am.server.thrift.node.name.mapping.class | com.cloudera.llama.am.HostnameOnlyNodeMapper |
The NodeMapper implementation LlamaAM server uses to convert requested
locations into Yarn Nodes.
The default (and only implementation for production) drops the port
number if present (Impala uses DataNode addresses to request a location,
these addresses may contain the DataNode port number. The DataNode port
number is meaningless and unknown to Yarn).
|
| llama.am.server.thrift.admin.address | localhost:15002 | The admin address the LlamaAM server listen at.
If 0.0.0.0 is specified as IP, the server will listen in all available
network addresses. If the port is not specified, the default port is
15002.
If the specified port is 0, an ephemeral port will be used, the port in
use will be printed in the logs at startup.
IMPORTANT: It is strongly recommended to use 'localhost' as the bind
address when security is not enabled.
|
| llama.am.server.thrift.http.address | 0.0.0.0:15001 | The address the LlamaAM server exposes its HTTP server for
JMX and the Web UI.
If 0.0.0.0 is specified as IP, the server will listen in all available
network addresses.
If the port is not specified, the default port is 15001.
The HTTP JSON JMX servlet is exposed over HTTP at '/jmx', i.e.:
http://localhost:15001/jmx
If the specified port is 0, an ephemeral port will be used, the port in
use will be printed in the logs at startup.
|
| llama.am.server.thrift.loggers.servlet.read.only | true |
If the /loggers servlet is read only.
If not in read only mode, the servlet can be used to change logger levels.
|
| llama.am.server.thrift.security | false |
Indicates if security is enabled or not. If enabled, LlamaAM server uses
Kerberos Thrift SASL for all server and client Thrift connections.
|
| llama.am.server.thrift.security.QOP | auth |
Indicates the quality of protection if security is enabled.
Valid values are:
'auth' : authentication
'auth-int' : authentication and integrity
'auth-conf' : authentication, integrity and confidentiality
|
| llama.am.server.thrift.kerberos.keytab.file | llama.keytab |
The location of the LlamaAM server keytab. If the path is relative,
the keytab file is looked up in LlamaAM configuration directory.
|
| llama.am.server.thrift.kerberos.server.principal.name | llama/localhost |
LlamaAM Kerberos principal name.
'localhost' must be replaced with the hostname specified in the service
principal.
|
| llama.am.server.thrift.kerberos.notification.principal.name | impala |
Principal short name, without the service hostname, used for client
notifications. The hostname provided in the client address at registration
by the client will be used as service hostname. IMPORTANT: they client
hostname address provided at registration must match the service name
in the client's Kerberos principal.
|
| llama.am.server.thrift.client.acl | * |
ACL for Llama AM clients.
The ACL is a comma-separated list of user and group names. The user and
group list is separated by a blank. For e.g. "alice,bob users,wheel".
A special value of "*" means all users are allowed.
|
| llama.am.server.thrift.admin.acl | * |
ACL for Llama AM admins.
The ACL is a comma-separated list of user and group names. The user and
group list is separated by a blank. For e.g. "alice,bob users,wheel".
A special value of "*" means all users are allowed.
|
| llama.am.cluster.id | llama |
Identification of a Llama (or an Active/Standby pair). YARN resources
are grouped by this ID. When multiple applications on a YARN cluster
use their own Llamas, it is recommended that they use different values
for this config. On Llama restart, all YARN applications corresponding
to this cluster-id are killed.
|
| llama.am.rm.connector.class | com.cloudera.llama.am.yarn.YarnRMConnector |
Backing LlamaAM implementation to use.
Available for functional testing:
MockRMLlamaAMConnector
|
| llama.am.rm.connector.recycle.interval.mins | 1380 |
Yarn AM-RM protocol does not have a way of renewing long lasting tokens.
So, Llama recycles the yarn connectors to avoid any failures due to
invalid tokens. By default its set to 75% of the value
yarn.resourcemanager.delegation.token.renew-interval in yarn-site.xml.
It can be overridden here but it should always be less than the yarn
configured value.
|
| llama.am.core.queues | |
Queues LlamaAM should connect to at start up. Unlike queues that are
created in response to a reservation, these queues will never expire.
IMPORTANT: all queue names should be canonical queue names, this is
prefixed with "root.". This is required because Yarn normalizes queue
names to their canonical name.
|
| llama.am.gang.anti.deadlock.enabled | true |
Enables Llama AM gang scheduling anti deadlock detection.
|
| llama.am.gang.anti.deadlock.no.allocation.limit.ms | 30000 |
Interval of time without any new allocation that will trigger the Llama AM
anti-deadlock logic.
|
| llama.am.gang.anti.deadlock.backoff.percent | 30 |
Percentage of resources that will be backed off by the Llama AM
anti-deadlock logic.
Random reservations will be backed off until the percentage of backed off
resources reaches this percentage.
|
| llama.am.gang.anti.deadlock.backoff.min.delay.ms | 10000 |
Minimum amount of time the backed off reservations will be in 'backed off'
state.
The actual amount time is a random value between the minimum and the
maximum.
|
| llama.am.gang.anti.deadlock.backoff.max.delay.ms | 30000 |
Maximum amount of time the backed off reservations will be in 'backed off'
state.
The actual amount time is a random value between the minimum and the
maximum.
|
| llama.am.throttling.enabled | true |
Global setting that indicates if Llama should throttle reservations.
|
| llama.am.throttling.enabled.#QUEUE# | true |
Per queue setting that indicates if Llama should throttle reservations for
the #QUEUE# queue. If not set, the 'llama.am.throttling.enabled' is used.
|
| llama.am.throttling.maximum.placed.reservations | 10000 |
Global maximum number of reservations per queue. Once this number of
reservations is reached for a queue, reservations are queued up.
|
| llama.am.throttling.maximum.queued.reservations | 0 |
Global maximum number of queued reservations per queue. Once this number
of reservations is reached for a queue, new reservations are rejected.
|
| llama.am.throttling.maximum.placed.reservations.#QUEUE# | 10000 |
Maximum number of reservations for #QUEUE# queue. Once this number of
reservations is reached for the #QUEUE# queue, reservations are queued up.
If this property is not set for the #QUEUE#, the global configuration
property is used 'llama.am.throttling.maximum.placed.reservations'.
|
| llama.am.throttling.maximum.queued.reservations.#QUEUE# | 0 |
Maximum number of queued reservations for #QUEUE# queue. Once this number
of reservations is reached for the #QUEUE# queue, new reservations are
rejected.
If this property is not set for the #QUEUE#, the global configuration
property is used 'llama.am.throttling.maximum.queued.reservations'.
|
| llama.am.mock.nodes | node1,node2 |
List of nodes to offer.
|
| llama.am.mock.queues | queue1,queue2 |
List of queues to offer.
|
| llama.am.mock.events.min.wait.ms | 1000 |
Minimum wait time, in milliseconds, for events to be delivered after
reservation. Actual wait time is a random value.
|
| llama.am.mock.events.max.wait.ms | 10000 |
Maximum wait time, in milliseconds, for events to be delivered after
reservation. Actual wait time is a random value.
|
| llama.am.cache.enabled | true |
Global setting that indicates if Llama should cache allocated resources on
release.
|
| llama.am.cache.enabled.#QUEUE# | true |
Per queue setting that indicates if Llama should cache allocated resources
on release for the #QUEUE# queue. If not set, the
'llama.am.caching.enabled' is used.
|
| llama.am.cache.eviction.run.interval.timeout.ms | 5000 |
Interval of time between eviction policy runs.
|
| llama.am.cache.eviction.policy.class | com.cloudera.llama.am.cache.ResourceCache$TimeoutEvictionPolicy |
The eviction policy for cached resources. The TimeoutEvictionPolicy
evicts resources that have been sitting in the cache for a period of time
equal or greater than the timeout.
|
| llama.am.cache.eviction.timeout.policy.idle.timeout.ms | 30000 |
TimeoutEvictionPolicy policy timeout for resources sitting in the cache.
|
| llama.am.queue.expire.ms | 300000 |
Time in milliseconds after which Llama will discard its AM for
a queue that has been empty of reservations. Does not apply to queues
specified with the llama.am.core.queues property.
|
| llama.am.resource.normalizing.enabled | true |
Whether to break resource requests into smaller requests of standard size
before the cache.
|
| llama.am.resource.normalizing.standard.mbs | 1024 |
The standard size in MB to break requests into when normalizing is turned
on.
|
| llama.am.resource.normalizing.standard.vcores | 1 |
The standard size in vcores to break requests into when normalizing is
turned on.
|
| llama.am.hadoop.user.name | llama |
User name use by Llama when interacting with Yarn.
|
| llama.am.yarn.priority | 0 |
Application priority when creating application in Yarn Resource Manager.
NOTE: currently YARN does not use the application priority for
scheduling decisions.
|
| llama.am.yarn.app.monitor.timeout.ms | 30000 |
Timeout, in milliseconds, for waiting the Application Master to start
or to stop.
|
| llama.am.yarn.app.monitor.polling.ms | 200 |
Polling interval, in milliseconds, to determine if the Application Master
has started or stopped.
|
| llama.am.yarn.app.heartbeat.interval.ms | 200 |
LlamaAM Application Master heartbeat interval, in milliseconds. On each
heartbeat the Application Master submits new reservations to Yarn Resource
Manager and gets updates from it.
|
| llama.am.yarn.container.handler.queue.threshold | 10000 |
Threshold of the outstanding container requests queue size to Yarn Node
Managers to start producing warnings. The queue will continue to queue
container requests when above the threshold.
|
| llama.am.yarn.container.handler.threads | 10 |
Number of threads used to do container requests to Yarn Node Managers.
|
| llama.nm.server.thrift.address | 0.0.0.0:15100 | The address the Llama NM Auxiliary Service listen at.
If 0.0.0.0 is specified as IP, the server will listen in all available
network addresses. IMPORTANT: if security is enabled do not use 0.0.0.0,
instead, use the exact same hostname used in the kerberos service
principal of the LlamaNM auxiliary service (i.e. llama/HOSTNAME).
If the port is not specified, the default port is 15100.
If the specified port is 0, an ephemeral port will be used, the port in
use will be printed in the logs at startup.
|
| llama.nm.server.thrift.server.min.threads | 10 |
Minimum number of threads used by the LlamaNM auxiliary service uses for
serving client requests.
|
| llama.nm.server.thrift.server.max.threads | 50 |
Maximum number of threads used by the LlamaNM auxiliary service uses for
serving client requests.
|
| llama.nm.server.thrift.transport.timeout.ms | 3600000 |
Socket time, in milliseconds, used LlamaNM auxiliary service for all its
server and client Thrift connections.
|
| llama.nm.server.thrift.client.notifier.queue.threshold | 10000 |
Threshold of the outstanding client notification queue size to start
producing warnings. The queue will continue to queue notifications
requests when above the threshold.
|
| llama.nm.server.thrift.client.notifier.threads | 10 |
Number of threads used to do client notifications.
|
| llama.nm.server.thrift.client.notifier.max.retries | 5 |
Maximum number of retries for a client notification.
After the maximum number of client notification retries has been reached
without success the client is considered lost and all its reservations
are released.
A successful client notification resets the retries count.
|
| llama.nm.server.thrift.client.notifier.retry.interval.ms | 5000 |
Client notification retry interval, in milliseconds.
|
| llama.nm.server.thrift.client.notifier.heartbeat.ms | 5000 |
Heartbeat interval (if no other notification happened), from LlamaNM
auxiliary service to clients.
|
| llama.nm.server.thrift.security | false |
Indicates if security is enabled or not. If enabled, LlamaNM auxiliary
service uses Kerberos Thrift SASL for all server and client Thrift
connections.
|
| llama.nm.server.thrift.kerberos.keytab.file | llama.keytab |
The location of the Llama NM auxiliary service keytab. If the path is
relative, the keytab file is looked up in the Node Manager configuration
directory.
|
| llama.nm.server.thrift.kerberos.server.principal.name | llama/localhost |
Llama NM auxiliary service keytab Kerberos principal name.
'localhost' must be replaced with the hostname specified in the service
principal.
|
| llama.nm.server.thrift.kerberos.notification.principal.name | impala |
Principal short name, without the service hostname, used for client
notifications. The hostname provided in the client address at registration
by the client will be used as service hostname. IMPORTANT: they client
hostname address provided at registration must match the service name
in the client's Kerberos principal.
|
| llama.nm.server.thrift.client.acl | * |
ACL for Llama NM clients.
The ACL is a comma-separated list of user and group names. The user and
group list is separated by a blank. For e.g. "alice,bob users,wheel".
A special value of "*" means all users are allowed.
|
| llama.nm.server.thrift.admin.acl | * |
ACL for Llama NM admins.
The ACL is a comma-separated list of user and group names. The user and
group list is separated by a blank. For e.g. "alice,bob users,wheel".
A special value of "*" means all users are allowed.
|
| hadoop.security.group.mapping | org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback |
Class for user to group mapping (get groups for a given user) for ACL.
The default implementation,
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback,
will determine if the Java Native Interface (JNI) is available. If JNI is
available the implementation will use the API within hadoop to resolve a
list of groups for a user. If JNI is not available then the shell
implementation, ShellBasedUnixGroupsMapping, is used. This implementation
shells out to the Linux/Unix environment with the
bash -c groups
command to resolve a list of groups for a user.
|
| hadoop.security.groups.cache.secs | 300 |
This is the config controlling the validity of the entries in the cache
containing the user->group mapping. When this duration has expired,
then the implementation of the group mapping provider is invoked to get
the groups of the user and then cached back.
|
| llama.am.ha.enabled | false |
Enable Llama HA. When enabled, Llama starts in
Standby mode and participates in leader election. When elected leader,
it transitions to Active. When HA is enabled, one is required to set
llama.am.ha.zk-quorum.
|
| llama.am.ha.zk-quorum | |
The Zookeeper quorum to use for leader election and fencing when HA is
enabled.
|
| llama.am.ha.zk-base | /llama |
The base znode to be used to store all HA-related information. If using
the same Zookeeper quorum for multiple Llama clusters, make sure each
Llama cluster has a separate base znode. Also, make sure the path is
absolute, i.e., starts with "/".
|
| llama.am.ha.zk-timeout-ms | 10000 |
The session timeout to be used for connections to the Zookeeper quorum.
|
| llama.am.ha.zk-acl | world:anyone:rwcda |
ACLs to control access to the Zookeeper quorum used for HA.
|
| llama.am.ha.zk-auth | |
Auth information to go with llama.am.ha.zk-acl
|