|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.cloudera.crunch.DoFn<S,T> com.cloudera.crunch.MapFn<S,T>
public abstract class MapFn<S,T>
A DoFn
for the common case of emitting exactly one value
for each input record.
Constructor Summary | |
---|---|
MapFn()
|
Method Summary | |
---|---|
abstract T |
map(S input)
Maps the given input into an instance of the output type. |
void |
process(S input,
Emitter<T> emitter)
Processes the records from a PCollection . |
float |
scaleFactor()
Returns an estimate of how applying this function to a PCollection
will cause it to change in side. |
Methods inherited from class com.cloudera.crunch.DoFn |
---|
cleanup, configure, getConfiguration, getCounter, getCounter, getStatus, getTaskAttemptID, initialize, progress, setConfigurationForTest, setContext, setStatus |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public MapFn()
Method Detail |
---|
public abstract T map(S input)
public void process(S input, Emitter<T> emitter)
DoFn
PCollection
.
process
in class DoFn<S,T>
input
- The input recordemitter
- The emitter to send the output topublic float scaleFactor()
DoFn
PCollection
will cause it to change in side. The optimizer uses these estimates to
decide where to break up dependent MR jobs into separate Map and Reduce
phases in order to minimize I/O.
Subclasses of DoFn
that will substantially alter the size of the
resulting PCollection
should override this method.
scaleFactor
in class DoFn<S,T>
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |