com.cloudera.api.model
Class ApiHdfsReplicationArguments

java.lang.Object
  com.cloudera.api.model.ApiHdfsReplicationArguments

Direct Known Subclasses:

ApiHdfsCloudReplicationArguments

public class ApiHdfsReplicationArguments
extends Object

Replication arguments for HDFS.

Nested Class Summary

static class

ApiHdfsReplicationArguments.ReplicationStrategy The strategy for distributing the file replication tasks among the mappers of the MR job associated with a replication.

Constructor Summary

ApiHdfsReplicationArguments()

ApiHdfsReplicationArguments(ApiServiceRef sourceService, String sourcePath, String destinationPath, String mapreduceServiceName, Integer numMaps, String userName)

Method Summary

boolean	equals(Object o)
boolean	getAbortOnError() Whether to abort on a replication failure.
Integer	getBandwidthPerMap() The maximum bandwidth (in MB) per mapper in the mapreduce replication job.
String	getDestinationPath() The destination to replicate to.
List<String>	getExclusionFilters() Specify regular expression strings to match full paths of files and directories matching source paths and exclude them from the replication.
String	getLogPath() The HDFS path where the replication log files should be written to.
String	getMapreduceServiceName() The mapreduce service to use for the replication job.
Integer	getNumMaps() The number of mappers to use for the mapreduce replication job.
boolean	getPreserveBlockSize() Whether to preserve the HDFS block size.
boolean	getPreservePermissions() Whether to preserve the HDFS owner, group and permissions.
boolean	getPreserveReplicationCount() Whether to preserve the HDFS replication count.
Boolean	getPreserveXAttrs() Whether to preserve XAttrs, default to false This is introduced in V10.
Boolean	getRaiseSnapshotDiffFailures()
boolean	getRemoveMissingFiles() Whether to delete destination files that are missing in source.
ApiHdfsReplicationArguments.ReplicationStrategy	getReplicationStrategy() The strategy for distributing the file replication tasks among the mappers of the MR job associated with a replication.
String	getSchedulerPoolName() Name of the scheduler pool to use when submitting the MapReduce job.
boolean	getSkipChecksumChecks() Whether to skip checksum based file validation during replication.
Boolean	getSkipListingChecksumChecks() Whether to skip checksum based file comparison during replication.
Boolean	getSkipTrash() Whether to permanently delete destination files that are missing in source.
String	getSourcePath() The path to replicate.
ApiServiceRef	getSourceService() The service to replicate from.
String	getSourceUser() The user which will perform operations on source cluster.
String	getUserName() The user which will execute the MapReduce job.
int	hashCode()
boolean	isDryRun() Whether to perform a dry run.
void	setAbortOnError(boolean abortOnError)
void	setBandwidthPerMap(Integer bandwidthPerMap)
void	setDestinationPath(String path)
void	setDryRun(boolean dryRun)
void	setExclusionFilters(List<String> exclusionFilters)
void	setLogPath(String logPath)
void	setMapreduceServiceName(String name)
void	setNumMaps(Integer numMaps)
void	setPreserveBlockSize(boolean preserveBlockSize)
void	setPreservePermissions(boolean preservePermissions)
void	setPreserveReplicationCount(boolean preserveReplicationCount)
void	setPreserveXAttrs(Boolean preserveXAttrs)
void	setRaiseSnapshotDiffFailures(Boolean raiseSnapshotDiffFailures)
void	setRemoveMissingFiles(boolean removeMissingFiles)
void	setReplicationStrategy(ApiHdfsReplicationArguments.ReplicationStrategy replicationStrategy)
void	setSchedulerPoolName(String schedulerPoolName)
void	setSkipChecksumChecks(boolean skipChecksumChecks)
void	setSkipListingChecksumChecks(Boolean skipListingChecksumChecks)
void	setSkipTrash(Boolean skipTrash)
void	setSourcePath(String path)
void	setSourceService(ApiServiceRef sourceService)
void	setSourceUser(String sourceUser)
void	setUserName(String userName)
String	toString()
protected com.google.common.base.Objects.ToStringHelper	toStringHelper()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ApiHdfsReplicationArguments

public ApiHdfsReplicationArguments()

ApiHdfsReplicationArguments

public ApiHdfsReplicationArguments(ApiServiceRef sourceService,
                                   String sourcePath,
                                   String destinationPath,
                                   String mapreduceServiceName,
                                   Integer numMaps,
                                   String userName)

Method Detail

getSourceService

public ApiServiceRef getSourceService()

The service to replicate from.

setSourceService

public void setSourceService(ApiServiceRef sourceService)

getSourcePath

public String getSourcePath()

The path to replicate.

setSourcePath

public void setSourcePath(String path)

getDestinationPath

public String getDestinationPath()

The destination to replicate to.

setDestinationPath

public void setDestinationPath(String path)

getMapreduceServiceName

public String getMapreduceServiceName()

The mapreduce service to use for the replication job.

setMapreduceServiceName

public void setMapreduceServiceName(String name)

getSchedulerPoolName

public String getSchedulerPoolName()

Name of the scheduler pool to use when submitting the MapReduce job. Currently supports the capacity and fair schedulers. The option is ignored if a different scheduler is configured.

setSchedulerPoolName

public void setSchedulerPoolName(String schedulerPoolName)

getUserName

public String getUserName()

The user which will execute the MapReduce job. Required if running with Kerberos enabled.

setUserName

public void setUserName(String userName)

getSourceUser

public String getSourceUser()

The user which will perform operations on source cluster. Required if running with Kerberos enabled.

setSourceUser

public void setSourceUser(String sourceUser)

getNumMaps

public Integer getNumMaps()

The number of mappers to use for the mapreduce replication job.

setNumMaps

public void setNumMaps(Integer numMaps)

isDryRun

public boolean isDryRun()

Whether to perform a dry run. Defaults to false.

setDryRun

public void setDryRun(boolean dryRun)

getBandwidthPerMap

public Integer getBandwidthPerMap()

The maximum bandwidth (in MB) per mapper in the mapreduce replication job.

setBandwidthPerMap

public void setBandwidthPerMap(Integer bandwidthPerMap)

getAbortOnError

public boolean getAbortOnError()

Whether to abort on a replication failure. Defaults to false.

setAbortOnError

public void setAbortOnError(boolean abortOnError)

getRemoveMissingFiles

public boolean getRemoveMissingFiles()

Whether to delete destination files that are missing in source. Defaults to false.

setRemoveMissingFiles

public void setRemoveMissingFiles(boolean removeMissingFiles)

getPreserveReplicationCount

public boolean getPreserveReplicationCount()

Whether to preserve the HDFS replication count. Defaults to false.

setPreserveReplicationCount

public void setPreserveReplicationCount(boolean preserveReplicationCount)

getPreserveBlockSize

public boolean getPreserveBlockSize()

Whether to preserve the HDFS block size. Defaults to false.

setPreserveBlockSize

public void setPreserveBlockSize(boolean preserveBlockSize)

getPreservePermissions

public boolean getPreservePermissions()

Whether to preserve the HDFS owner, group and permissions. Defaults to false. Starting from V10, it also preserves ACLs. Defaults to null (no preserve). ACLs is preserved if both clusters enable ACL support, and replication ignores any ACL related failures.

setPreservePermissions

public void setPreservePermissions(boolean preservePermissions)

getLogPath

public String getLogPath()

The HDFS path where the replication log files should be written to.

setLogPath

public void setLogPath(String logPath)

getSkipChecksumChecks

public boolean getSkipChecksumChecks()

Whether to skip checksum based file validation during replication. Defaults to false.

setSkipChecksumChecks

public void setSkipChecksumChecks(boolean skipChecksumChecks)

getSkipListingChecksumChecks

public Boolean getSkipListingChecksumChecks()

Whether to skip checksum based file comparison during replication. Defaults to false.

setSkipListingChecksumChecks

public void setSkipListingChecksumChecks(Boolean skipListingChecksumChecks)

getSkipTrash

public Boolean getSkipTrash()

Whether to permanently delete destination files that are missing in source. Defaults to null.

setSkipTrash

public void setSkipTrash(Boolean skipTrash)

getReplicationStrategy

public ApiHdfsReplicationArguments.ReplicationStrategy getReplicationStrategy()

The strategy for distributing the file replication tasks among the mappers of the MR job associated with a replication. Default is ApiHdfsReplicationArguments.ReplicationStrategy.STATIC.

setReplicationStrategy

public void setReplicationStrategy(ApiHdfsReplicationArguments.ReplicationStrategy replicationStrategy)

getPreserveXAttrs

public Boolean getPreserveXAttrs()

Whether to preserve XAttrs, default to false This is introduced in V10. To preserve XAttrs, both CDH versions should be >= 5.2. Replication fails if either cluster does not support XAttrs.

setPreserveXAttrs

public void setPreserveXAttrs(Boolean preserveXAttrs)

getExclusionFilters

public List<String> getExclusionFilters()

Specify regular expression strings to match full paths of files and directories matching source paths and exclude them from the replication. Optional. Available since V11.

Returns:

exclusion paths, if set; null if no exclusion paths are specified.

setExclusionFilters

public void setExclusionFilters(List<String> exclusionFilters)

getRaiseSnapshotDiffFailures

public Boolean getRaiseSnapshotDiffFailures()

setRaiseSnapshotDiffFailures

public void setRaiseSnapshotDiffFailures(Boolean raiseSnapshotDiffFailures)

toStringHelper

protected com.google.common.base.Objects.ToStringHelper toStringHelper()

toString

public String toString()

Overrides:

toString in class Object

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object