Examples of org.kitesdk.data.spi.DatasetRepository

org.kitesdk.data.spi.DatasetRepository

A logical repository (storage system) of {@link Dataset}s.

Implementations of {@code DatasetRepository} are storage systems that containzero or more {@link Dataset}s. A repository acts as a factory, as well as a registry, of datasets. You can {@link #create(String,String,DatasetDescriptor)} anew {@link Dataset} with a name and schema, or retrieve a handle to anexisting dataset, by name, by way of the {@link #load(String,String)} method. Whilenot expressly forbidden, most repositories are expected to support only a single concrete {@link Dataset} implementation.

No guarantees are made as to the durability, reliability, or availability of the underlying storage. That is, a {@code DatasetRepository} could be ondisk, in memory, or some combination. See the implementation class for details about the guarantees it provides.

Implementations of {@link DatasetRepository} are immutable.
@see Dataset @see DatasetDescriptor

        .build();
  }


  @Test
  public void testAbsolute() {
    DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:/tmp/data");
    repo.delete("ns", "test");
    repo.create("ns", "test", descriptor);


    Dataset<Record> ds = Datasets.<Record, Dataset<Record>>
        load("dataset:file:/tmp/data/ns/test", Record.class);


    Assert.assertNotNull("Should load dataset", ds);
    Assert.assertTrue(ds instanceof FileSystemDataset);
    Assert.assertEquals("Locations should match",
        URI.create("file:/tmp/data/ns/test"),
        ds.getDescriptor().getLocation());
    Assert.assertEquals("Descriptors should match",
        repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
    Assert.assertEquals("Should report correct namespace",
        "ns", ds.getNamespace());
    Assert.assertEquals("Should report correct name",
        "test", ds.getName());


    repo.delete("ns", "test");
  }

View Full Code Here

    repo.delete("ns", "test");
  }


  @Test
  public void testRelative() {
    DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:target/data");
    repo.delete("ns", "test");
    repo.create("ns", "test", descriptor);


    Dataset<Record> ds = Datasets.<Record, Dataset<Record>>
        load("dataset:file:target/data/ns/test", Record.class);


    Assert.assertNotNull("Should load dataset", ds);
    Assert.assertTrue(ds instanceof FileSystemDataset);
    Path cwd = localFS.makeQualified(new Path("."));
    Assert.assertEquals("Locations should match",
        new Path(cwd, "target/data/ns/test").toUri(), ds.getDescriptor().getLocation());
    Assert.assertEquals("Descriptors should match",
        repo.load("ns", "test").getDescriptor(), ds.getDescriptor());
    Assert.assertEquals("Should report correct namespace",
        "ns", ds.getNamespace());
    Assert.assertEquals("Should report correct name",
        "test", ds.getName());


    repo.delete("ns", "test");
  }

View Full Code Here

    repo.delete("ns", "test");
  }


  @Test
  public void testViewConstraints() {
    DatasetRepository repo = DatasetRepositories.repositoryFor("repo:file:/tmp/data");
    repo.delete("ns", "test");
    repo.create("ns", "test", descriptor);


    RefinableView<Record> v = Datasets.<Record, RefinableView<Record>>
        load("view:file:/tmp/data/ns/test?username=user", Record.class);


    Assert.assertNotNull("Should load view", v);
    Assert.assertTrue(v instanceof FileSystemView);
    Assert.assertEquals("Locations should match",
        URI.create("file:/tmp/data/ns/test"),
        v.getDataset().getDescriptor().getLocation());


    DatasetDescriptor loaded = repo.load("ns", "test").getDescriptor();
    Assert.assertEquals("Descriptors should match",
        loaded, v.getDataset().getDescriptor());
    Assert.assertEquals("Should report correct namespace",
        "ns", v.getDataset().getNamespace());
    Assert.assertEquals("Should report correct name",
        "test", v.getDataset().getName());


    Constraints withUser = new Constraints(loaded.getSchema())
        .with("username", new Utf8("user"));
    Assert.assertEquals("Constraints should be username=user",
        withUser, ((FileSystemView) v).getConstraints());


    repo.delete("ns", "test");
  }

View Full Code Here

0 1 2 3 4 5 6 7

TOP

Related Classes of org.kitesdk.data.spi.DatasetRepository

org.kitesdk.cli.commands.DeleteDatasetCommand

org.kitesdk.cli.commands.TestCopyCommandCluster

org.kitesdk.cli.commands.TestCopyCommandLocal

org.kitesdk.cli.commands.TestCreateDatasetCommandCluster

org.kitesdk.cli.commands.TestTransformCommandCluster

org.kitesdk.cli.commands.TestTransformCommandLocal

org.kitesdk.data.Datasets

org.kitesdk.data.hbase.TestHBaseDatasetURIs

org.kitesdk.data.hbase.TestHBaseRepositoryURIs

org.kitesdk.data.mapreduce.DatasetKeyOutputFormat

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.