apiHdfsReplicationArguments

Replication arguments for HDFS.

JSON

property type description
sourceService sourceService (apiServiceRef) The service to replicate from.
sourcePath sourcePath (string) The path to replicate.
destinationPath destinationPath (string) The destination to replicate to.
mapreduceServiceName mapreduceServiceName (string) The mapreduce service to use for the replication job.
schedulerPoolName schedulerPoolName (string) Name of the scheduler pool to use when submitting the MapReduce job. Currently supports the capacity and fair schedulers. The option is ignored if a different scheduler is configured.
userName userName (string) The user which will execute the MapReduce job. Required if running with Kerberos enabled.
sourceUser sourceUser (string) The user which will perform operations on source cluster. Required if running with Kerberos enabled.
numMaps numMaps (int) The number of mappers to use for the mapreduce replication job.
dryRun dryRun (boolean) Whether to perform a dry run. Defaults to false.
bandwidthPerMap bandwidthPerMap (int) The maximum bandwidth (in MB) per mapper in the mapreduce replication job.
abortOnError abortOnError (boolean) Whether to abort on a replication failure. Defaults to false.
removeMissingFiles removeMissingFiles (boolean) Whether to delete destination files that are missing in source. Defaults to false.
preserveReplicationCount preserveReplicationCount (boolean) Whether to preserve the HDFS replication count. Defaults to false.
preserveBlockSize preserveBlockSize (boolean) Whether to preserve the HDFS block size. Defaults to false.
preservePermissions preservePermissions (boolean) Whether to preserve the HDFS owner, group and permissions. Defaults to false. Starting from V10, it also preserves ACLs. Defaults to null (no preserve). ACLs is preserved if both clusters enable ACL support, and replication ignores any ACL related failures.
logPath logPath (string) The HDFS path where the replication log files should be written to.
skipChecksumChecks skipChecksumChecks (boolean) Whether to skip checksum based file validation during replication. Defaults to false.
skipListingChecksumChecks skipListingChecksumChecks (boolean) Whether to skip checksum based file comparison during replication. Defaults to false.
skipTrash skipTrash (boolean) Whether to permanently delete destination files that are missing in source. Defaults to null.
replicationStrategy replicationStrategy (replicationStrategy) The strategy for distributing the file replication tasks among the mappers of the MR job associated with a replication. Default is ReplicationStrategy#STATIC.
preserveXAttrs preserveXAttrs (boolean) Whether to preserve XAttrs, default to false This is introduced in V10. To preserve XAttrs, both CDH versions should be >= 5.2. Replication fails if either cluster does not support XAttrs.
exclusionFilters array of exclusionFilters (string) Specify regular expression strings to match full paths of files and directories matching source paths and exclude them from the replication. Optional. Available since V11.
raiseSnapshotDiffFailures raiseSnapshotDiffFailures (boolean)