Examples of org.apache.hive.service.server.ThreadFactoryWithGarbageCleanup

org.apache.hive.service.server.ThreadFactoryWithGarbageCleanup
A ThreadFactory for constructing new HiveServer2 threads that lets you plug in custom cleanup code to be called before this thread is GC-ed. Currently cleans up the following: 1. ThreadLocal RawStore object: In case of an embedded metastore, HiveServer2 threads (foreground & background) end up caching a ThreadLocal RawStore object. The ThreadLocal RawStore object has an instance of PersistenceManagerFactory & PersistenceManager. The PersistenceManagerFactory keeps a cache of PersistenceManager objects, which are only removed when PersistenceManager#close method is called. HiveServer2 uses ExecutorService for managing thread pools for foreground & background threads. ExecutorService unfortunately does not provide any hooks to be called, when a thread from the pool is terminated. As a solution, we're using this ThreadFactory to keep a cache of RawStore objects per thread. And we are doing clean shutdown in the finalizer for each thread.

    // Threads terminate when they are idle for more than the keepAliveTime
    // A bounded blocking queue is used to queue incoming operations, if #operations > poolSize
    String threadPoolName = "HiveServer2-Background-Pool";
    backgroundOperationPool = new ThreadPoolExecutor(poolSize, poolSize,
        keepAliveTime, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(poolQueueSize),
        new ThreadFactoryWithGarbageCleanup(threadPoolName));
    backgroundOperationPool.allowCoreThreadTimeOut(true);


    checkInterval = HiveConf.getTimeVar(
        hiveConf, ConfVars.HIVE_SERVER2_SESSION_CHECK_INTERVAL, TimeUnit.MILLISECONDS);
    sessionTimeout = HiveConf.getTimeVar(

View Full Code Here

    try {
      // Server thread pool
      String threadPoolName = "HiveServer2-Handler-Pool";
      ExecutorService executorService = new ThreadPoolExecutor(minWorkerThreads, maxWorkerThreads,
          workerKeepAliveTime, TimeUnit.SECONDS, new SynchronousQueue<Runnable>(),
          new ThreadFactoryWithGarbageCleanup(threadPoolName));


      // Thrift configs
      hiveAuthFactory = new HiveAuthFactory(hiveConf);
      TTransportFactory transportFactory = hiveAuthFactory.getAuthTransFactory();
      TProcessorFactory processorFactory = hiveAuthFactory.getAuthProcFactory(this);

View Full Code Here


      // Server thread pool
      String threadPoolName = "HiveServer2-HttpHandler-Pool";
      ExecutorService executorService = new ThreadPoolExecutor(minWorkerThreads, maxWorkerThreads,
          workerKeepAliveTime, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(),
          new ThreadFactoryWithGarbageCleanup(threadPoolName));
      ExecutorThreadPool threadPool = new ExecutorThreadPool(executorService);
      httpServer.setThreadPool(threadPool);


      // Connector configs
      SelectChannelConnector connector = new SelectChannelConnector();

View Full Code Here

TOP

Related Classes of org.apache.hive.service.server.ThreadFactoryWithGarbageCleanup

org.apache.hive.service.cli.session.SessionManager

org.apache.hive.service.cli.thrift.ThriftBinaryCLIService

org.apache.hive.service.cli.thrift.ThriftHttpCLIService

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.