Examples of com.facebook.presto.hive.util.BoundedExecutor

com.facebook.presto.hive.util.BoundedExecutor
THIS CLASS IS COPIED FROM com.facebook.presto.util.BoundedExecutor
Guarantees that no more than maxThreads will be used to execute tasks submitted through execute().
There are a few interesting properties: - Multiple BoundedExecutors over a single coreExecutor will have fair sharing of the coreExecutor threads proportional to their relative maxThread counts, but can use less if not as active. - Tasks submitted to a BoundedExecutor is guaranteed to have those tasks handed to threads in that order. - Will not encounter starvation

    }


    public ConnectorSplitSource get()
    {
        // Each iterator has its own bounded executor and can be independently suspended
        final SuspendingExecutor suspendingExecutor = new SuspendingExecutor(new BoundedExecutor(executor, maxThreads));
        final HiveSplitSource splitSource = new HiveSplitSource(connectorId, maxOutstandingSplits, suspendingExecutor);


        FutureTask<?> producer = new FutureTask<>(new Runnable()
        {
            @Override

View Full Code Here

                metastore,
                namenodeStats,
                hdfsEnvironment,
                directoryLister,
                DateTimeZone.forTimeZone(hiveClientConfig.getTimeZone()),
                new BoundedExecutor(executorService, hiveClientConfig.getMaxGlobalSplitIteratorThreads()),
                hiveClientConfig.getMaxSplitSize(),
                hiveClientConfig.getMaxOutstandingSplits(),
                hiveClientConfig.getMaxSplitIteratorThreads(),
                hiveClientConfig.getMinPartitionBatchSize(),
                hiveClientConfig.getMaxPartitionBatchSize(),

View Full Code Here

0 1 2 3

TOP

Related Classes of com.facebook.presto.hive.util.BoundedExecutor

com.facebook.presto.hive.HiveClient

com.facebook.presto.hive.HiveSplitIterable

com.facebook.presto.hive.HiveSplitSourceProvider

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.