Examples of org.apache.mahout.cf.taste.impl.model.file.FileDataModel

Package org.apache.mahout.cf.taste.impl.model.file

Examples of org.apache.mahout.cf.taste.impl.model.file.FileDataModel

org.apache.mahout.cf.taste.impl.model.file.FileDataModel

A {@link DataModel} backed by a comma-delimited file. This class typically expects a file where each linecontains a user ID, followed by item ID, followed by preferences value, separated by commas. You may also use tabs.

The preference value is assumed to be parseable as a double. The user IDs and item IDs are read parsed as longs.

This class will reload data from the data file when {@link #refresh(Collection)} is called, unless the filehas been reloaded very recently already.

This class will also look for update "delta" files in the same directory, with file names that start the same way (up to the first period). These files should have the same format, and provide updated data that supersedes what is in the main data file. This is a mechanism that allows an application to push updates to without re-copying the entire data file.

The line may contain a blank preference value (e.g. "123,456,"). This is interpreted to mean "delete preference", and is only useful in the context of an update delta file (see above). Note that if the line is empty or begins with '#' it will be ignored as a comment.

It is also acceptable for the lines to contain additional fields. Fields beyond the third will be ignored.

Finally, for application that have no notion of a preference value (that is, the user simply expresses a preference for an item, but no degree of preference), the caller can simply omit the third token in each line altogether -- for example, "123,456".

Note that it's all-or-nothing -- all of the items in the file must express no preference, or the all must. These cannot be mixed. Put another way there will always be the same number of delimiters on every line of the file!

This class is not intended for use with very large amounts of data (over, say, tens of millions of rows). For that, a JDBC-backed {@link DataModel} and a database are more appropriate.

It is possible and likely useful to subclass this class and customize its behavior to accommodate application-specific needs and input formats. See {@link #processLine(String,FastByIDMap,boolean)} and{@link #processLineWithoutID(String,FastByIDMap)}

  @Override
  protected void setup(Context context) {
    Configuration jobConf = context.getConfiguration();
    String dataModelFile = jobConf.get(DATA_MODEL_FILE);
    String recommenderClassName = jobConf.get(RECOMMENDER_CLASS_NAME);
    FileDataModel fileDataModel;
    try {
      Path dataModelPath = new Path(dataModelFile);
      FileSystem fs = FileSystem.get(dataModelPath.toUri(), jobConf);
      File tempDataFile = File.createTempFile("mahout-taste-hadoop", "txt");
      tempDataFile.deleteOnExit();
      fs.copyToLocalFile(dataModelPath, new Path(tempDataFile.getAbsolutePath()));
      fileDataModel = new FileDataModel(tempDataFile);
    } catch (IOException ioe) {
      throw new IllegalStateException(ioe);
    }
    try {
      Class<? extends Recommender> recommenderClass = Class.forName(recommenderClassName).asSubclass(

View Full Code Here

  private LoadEvaluationRunner() {
  }


  public static void main(String[] args) throws Exception {


    DataModel model = new FileDataModel(new File(args[0]));


    int howMany = 10;
    if (args.length > 1) {
      howMany = Integer.parseInt(args[1]);
    }

View Full Code Here


  @Override
  public void configure(JobConf jobConf) {
    String dataModelFile = jobConf.get(DATA_MODEL_FILE);
    String recommenderClassName = jobConf.get(RECOMMENDER_CLASS_NAME);
    FileDataModel fileDataModel;
    try {
      FileSystem fs = FileSystem.get(jobConf);
      File tempDataFile = File.createTempFile("mahout-taste-hadoop", "txt");
      tempDataFile.deleteOnExit();
      fs.copyToLocalFile(new Path(dataModelFile), new Path(tempDataFile.getAbsolutePath()));
      fileDataModel = new FileDataModel(tempDataFile);
    } catch (IOException ioe) {
      throw new RuntimeException(ioe);
    }
    try {
      Class<? extends Recommender> recommenderClass = Class.forName(recommenderClassName).asSubclass(Recommender.class);

View Full Code Here

    Path dataModelPath = new Path(dataModelFile);
    FileSystem fs = FileSystem.get(dataModelPath.toUri(), jobConf);
    File tempDataFile = File.createTempFile("mahout-taste-hadoop", "txt");
    tempDataFile.deleteOnExit();
    fs.copyToLocalFile(dataModelPath, new Path(tempDataFile.getAbsolutePath()));
    FileDataModel fileDataModel = new FileDataModel(tempDataFile);


    try {
      Class<? extends Recommender> recommenderClass = Class.forName(recommenderClassName).asSubclass(
        Recommender.class);
      Constructor<? extends Recommender> constructor = recommenderClass.getConstructor(DataModel.class);

View Full Code Here

  private LoadEvaluationRunner() {
  }


  public static void main(String[] args) throws Exception {


    DataModel model = new FileDataModel(new File(args[0]));


    int howMany = 10;
    if (args.length > 1) {
      howMany = Integer.parseInt(args[1]);
    }

View Full Code Here

  @Override
  protected void setup(Context context) {
    Configuration jobConf = context.getConfiguration();
    String dataModelFile = jobConf.get(DATA_MODEL_FILE);
    String recommenderClassName = jobConf.get(RECOMMENDER_CLASS_NAME);
    FileDataModel fileDataModel;
    try {
      Path dataModelPath = new Path(dataModelFile);
      FileSystem fs = FileSystem.get(dataModelPath.toUri(), jobConf);
      File tempDataFile = File.createTempFile("mahout-taste-hadoop", "txt");
      tempDataFile.deleteOnExit();
      fs.copyToLocalFile(dataModelPath, new Path(tempDataFile.getAbsolutePath()));
      fileDataModel = new FileDataModel(tempDataFile);
    } catch (IOException ioe) {
      throw new IllegalStateException(ioe);
    }
    try {
      Class<? extends Recommender> recommenderClass =

View Full Code Here

    Path dataModelPath = new Path(dataModelFile);
    FileSystem fs = FileSystem.get(dataModelPath.toUri(), jobConf);
    File tempDataFile = File.createTempFile("mahout-taste-hadoop", "txt");
    tempDataFile.deleteOnExit();
    fs.copyToLocalFile(dataModelPath, new Path(tempDataFile.getAbsolutePath()));
    FileDataModel fileDataModel = new FileDataModel(tempDataFile);


    try {
      Class<? extends Recommender> recommenderClass = Class.forName(recommenderClassName).asSubclass(
        Recommender.class);
      Constructor<? extends Recommender> constructor = recommenderClass.getConstructor(DataModel.class);

View Full Code Here

    predictor.setConf(conf);
    predictor.run(new String[] { "--output", outputDir.getAbsolutePath(), "--pairs", pairs.getAbsolutePath(),
        "--userFeatures", userFeatures.getAbsolutePath(), "--itemFeatures", itemFeatures.getAbsolutePath(),
        "--tempDir", tempDir.getAbsolutePath() });


    FileDataModel dataModel = new FileDataModel(new File(outputDir, "part-r-00000"));
    assertEquals(3, dataModel.getNumUsers());
    assertEquals(2, dataModel.getNumItems());
    assertEquals(2.45f, dataModel.getPreferenceValue(0, 0), EPSILON);
    assertEquals(-6.6f, dataModel.getPreferenceValue(2, 1), EPSILON);
    assertEquals(-0.61f, dataModel.getPreferenceValue(1, 0), EPSILON);
  }

View Full Code Here

    }
  }


  private static void recommend(String ratingsFile, int ... userIds)
      throws TasteException, IOException {
    DataModel model = new FileDataModel(new File(ratingsFile));


    UserSimilarity similarity = new PearsonCorrelationSimilarity(model);


    UserNeighborhood neighborhood =
        new NearestNUserNeighborhood(

View Full Code Here

    }
  }


  public static void evaluate(String ratingsFile)
      throws TasteException, IOException {
    DataModel model = new FileDataModel(new File(ratingsFile));
    RecommenderEvaluator evaluator =
        new AverageAbsoluteDifferenceRecommenderEvaluator();
    RecommenderBuilder recommenderBuilder = new MyRecommendBuilder();
    evaluator.evaluate(
        recommenderBuilder,

View Full Code Here

0 1 2

TOP

Related Classes of org.apache.mahout.cf.taste.impl.model.file.FileDataModel

com.manning.hip.ch9.MovieUserEvaluator

com.manning.hip.ch9.MovieUserRecommender

mia.recommender.ch02.EvaluatorIntro

mia.recommender.ch02.IREvaluatorIntro

mia.recommender.ch02.ItemBaseRecommender

mia.recommender.ch02.ItemBaseRecommenderEvaluation

mia.recommender.ch02.UserBaseRecommender

mia.recommender.ch02.UserBaseRecommenderEvaluation

mia.recommender.ch03.IREvaluatorBooleanPrefIntro1

mia.recommender.ch03.IREvaluatorBooleanPrefIntro2

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.