Calculation and Turing-Completeness #48
Description
FIrst off- it's nice. I come from the Unix tradition of short&sweet. Of course,
I want a new feature :)
Hadoop workflow drivers all seem to believe in code&fire&forget: some allow
if/then/else branching and oozie supports expressions. I have not found any
that support conditionally starting over or jumping into the middle.
Many machine learning algorithms use repetitive calculations driving towards a
finishing condition (often a maximum error value). The Mahout project, which
focuses on machine learning on Map/Reduce, has a great many multistage
calculation jobs which have to be coded in Java entirely due to this.
Here is a very simple feature addition that would solve all of these: a
decision node that calls a Java object. One of the possible decisions would be:
branch X is a goto to another place in the workflow. This would hack the
generation numbers on the paths after the target node.
Original issue reported on code.google.com by lance.no...@gmail.com on 31 Jul 2011 at 11:27