Examples of org.apache.drill.exec.store.dfs.FileSelection

org.apache.drill.exec.store.dfs.FileSelection
Jackson serializable description of a file selection. Maintains an internal set of file statuses. However, also serializes out as a list of Strings. All accessing methods first regenerate the FileStatus objects if they are not available. This allows internal movement of FileStatus and the ability to serialize if need be.

      @JsonProperty("selectionRoot") String selectionRoot
      ) throws IOException, ExecutionSetupException {


    this.formatPlugin = (EasyFormatPlugin<?>) engineRegistry.getFormatPlugin(storageConfig, formatConfig);
    Preconditions.checkNotNull(formatPlugin, "Unable to load format plugin for provided format config.");
    this.selection = new FileSelection(files, true);
    try{
      BlockMapBuilder b = new BlockMapBuilder(formatPlugin.getFileSystem().getUnderlying(), formatPlugin.getContext().getBits());
      this.chunks = b.generateFileWork(selection.getFileStatusList(formatPlugin.getFileSystem()), formatPlugin.isBlockSplittable());
      this.endpointAffinities = AffinityCreator.getAffinityMap(chunks);
    }catch(IOException e){

View Full Code Here

      @JsonProperty("format") FormatPluginConfig formatConfig, //
      @JacksonInject StoragePluginRegistry engineRegistry, //
      @JsonProperty("columns") List<SchemaPath> columns,
      @JsonProperty("selectionRoot") String selectionRoot
      ) throws IOException, ExecutionSetupException {
        this(new FileSelection(files, true),
            (EasyFormatPlugin<?>)engineRegistry.getFormatPlugin(storageConfig, formatConfig),
            columns,
            selectionRoot);
  }

View Full Code Here

        }
      }
    }


    if (newFiles.size() > 0) {
      FileSelection newFileSelection = new FileSelection(newFiles, origSelection.getSelection().selectionRoot, true);
      FormatSelection newFormatSelection = new FormatSelection(origSelection.getFormat(), newFileSelection);
      return newFormatSelection;
    }


    return origSelection;

View Full Code Here

      @JsonProperty("format") FormatPluginConfig formatConfig, //
      @JacksonInject StoragePluginRegistry engineRegistry, //
      @JsonProperty("columns") List<SchemaPath> columns,
      @JsonProperty("selectionRoot") String selectionRoot
      ) throws IOException, ExecutionSetupException {
        this(new FileSelection(files, true),
            (EasyFormatPlugin<?>)engineRegistry.getFormatPlugin(storageConfig, formatConfig),
            columns,
            selectionRoot);
  }

View Full Code Here

TOP

Related Classes of org.apache.drill.exec.store.dfs.FileSelection

org.apache.drill.exec.planner.logical.DrillPushPartitionFilterIntoScan

org.apache.drill.exec.store.dfs.easy.EasyGroupScan

org.apache.hadoop.fs.FileStatus

org.apache.hadoop.fs.Path

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.