Examples of org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver

org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver

s.apache.org/jira/browse/MAHOUT-376).

As of the time of this writing, I don't have benchmarks for this method in comparison to other methods. However, non-hadoop differentiating characteristics of this method are thought to be :

"faster" and precision is traded off in favor of speed. However, there's lever in terms of "oversampling parameter" p. Higher values of p produce better precision but are trading off speed (and minimum RAM requirement). This also means that this method is almost guaranteed to be less precise than Lanczos unless full rank SVD decomposition is sought.

"more scale" -- can presumably take on larger problems than Lanczos one (not confirmed by benchmark at this time)

Specifically in regards to this implementation, I think couple of other differentiating points are:

no need to specify input matrix height or width in command line, it is what it gets to be.

supports any Writable as DRM row keys and copies them to correspondent rows of U matrix;

can request U or V or U_σ=U* Σ^0.5 or V_σ=V* Σ^0.5 none of which would require pass over input A and these jobs are parallel map-only jobs.

This class is central public API for SSVD solver. The use pattern is as follows:

create the solver using constructor and supplying computation parameters.
set optional parameters thru setter methods.
call {@link #run()}.
{@link #getUPath()} (if computed) returns the path to the directorycontaining m x k U matrix file(s).
{@link #getVPath()} (if computed) returns the path to the directorycontaining n x k V matrix file(s).

      Path[] LPath = new Path[1];
      LPath[0] = L.getRowPath();


      Path SSVDout = new Path(outputCalc, "SSVD");


      SSVDSolver solveIt = new SSVDSolver(depConf, LPath, SSVDout, blockHeight, clusters, oversampling, numReducers);


      solveIt.setComputeV(false);
      solveIt.setComputeU(true);
      solveIt.setOverwrite(true);
      solveIt.setQ(poweriters);
      // solveIt.setBroadcast(false);
      solveIt.run();
      data = new Path(solveIt.getUPath());
    } else {
      // Perform eigen-decomposition using LanczosSolver
      // since some of the eigen-output is spurious and will be eliminated
      // upon verification, we have to aim to overshoot and then discard
      // unnecessary vectors later

      Path[] LPath = new Path[1];
      LPath[0] = L.getRowPath();


      Path SSVDout = new Path(outputCalc, "SSVD");


      SSVDSolver solveIt = new SSVDSolver(depConf, LPath, SSVDout, blockHeight, clusters, oversampling, numReducers);


      solveIt.setComputeV(false);
      solveIt.setComputeU(true);
      solveIt.setOverwrite(true);
      solveIt.setQ(poweriters);
      // solveIt.setBroadcast(false);
      solveIt.run();
      data = new Path(solveIt.getUPath());
    } else {
      // Perform eigen-decomposition using LanczosSolver
      // since some of the eigen-output is spurious and will be eliminated
      // upon verification, we have to aim to overshoot and then discard
      // unnecessary vectors later

      Path[] LPath = new Path[1];
      LPath[0] = L.getRowPath();


      Path SSVDout = new Path(outputCalc, "SSVD");


      SSVDSolver solveIt = new SSVDSolver(depConf, LPath, SSVDout, blockHeight, clusters, oversampling, numReducers);


      solveIt.setComputeV(false);
      solveIt.setComputeU(true);
      solveIt.setOverwrite(true);
      solveIt.setQ(poweriters);
      // solveIt.setBroadcast(false);
      solveIt.run();
      data = new Path(solveIt.getUPath());
    } else {
      // Perform eigen-decomposition using LanczosSolver
      // since some of the eigen-output is spurious and will be eliminated
      // upon verification, we have to aim to overshoot and then discard
      // unnecessary vectors later

Related Classes of org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.