com.cloudera.crunch.io.impl
Class FileSourceImpl<T>

java.lang.Object
  com.cloudera.crunch.io.impl.FileSourceImpl<T>

All Implemented Interfaces:

Source<T>

Direct Known Subclasses:

AvroFileSource, FileTableSourceImpl, SeqFileSource, TextFileSource

public abstract class FileSourceImpl<T>
extends Object
implements Source<T>

Field Summary

protected Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat>	inputFormatClass
protected org.apache.hadoop.fs.Path	path
protected PType<T>	ptype

Constructor Summary

FileSourceImpl(org.apache.hadoop.fs.Path path, PType<T> ptype, Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat> inputFormatClass)

Method Summary

void	configureSource(org.apache.hadoop.mapreduce.Job job, int inputId) Configure the given job to use this source as an input.
boolean	equals(Object other)
long	getSize(org.apache.hadoop.conf.Configuration configuration) Returns the number of bytes in this Source.
PType<T>	getType() Returns the PType for this source.
int	hashCode()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

path

protected final org.apache.hadoop.fs.Path path

ptype

protected final PType<T> ptype

inputFormatClass

protected final Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat> inputFormatClass

Constructor Detail

FileSourceImpl

public FileSourceImpl(org.apache.hadoop.fs.Path path,
                      PType<T> ptype,
                      Class<? extends org.apache.hadoop.mapreduce.lib.input.FileInputFormat> inputFormatClass)

Method Detail

getType

public PType<T> getType()

Description copied from interface: Source

Returns the PType for this source.

Specified by:

getType in interface Source<T>

configureSource

public void configureSource(org.apache.hadoop.mapreduce.Job job,
                            int inputId)
                     throws IOException

Description copied from interface: Source

Configure the given job to use this source as an input.

Specified by:

configureSource in interface Source<T>

Parameters:

job - The job to configure

inputId - For a multi-input job, an identifier for this input to the job

Throws:

IOException

getSize

public long getSize(org.apache.hadoop.conf.Configuration configuration)

Description copied from interface: Source

Returns the number of bytes in this Source.

Specified by:

getSize in interface Source<T>

equals

public boolean equals(Object other)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object