Examples of com.streamreduce.util.WebHDFSClient

com.streamreduce.util.WebHDFSClient
This class provides a client interface to WebHDFS instances.

Author: Nick Heudecker

Created: 7/3/12 7:50 AM

                        awsClient.createBucket(outboundConfiguration);
                    } catch (IllegalStateException e) { //thrown when a bucket name is already taken
                        throw new InvalidOutboundConfigurationException(e.getMessage(), e);
                    }
                } else if (outboundConfiguration.getProtocol().equals("webhdfs")) {
                    externalClient = new WebHDFSClient(outboundConfiguration);
                    externalClient.validateConnection();
                }
            }
        } finally {
            if (externalClient != null) {

View Full Code Here

            }
        }
    }


    private void sendPayloadToWebHDFS(String filepath, byte[] payload)  throws IOException {
        WebHDFSClient webHDFSClient = new WebHDFSClient(outboundConfiguration);
        createDestinationDirectory(webHDFSClient, filepath.substring(0,filepath.lastIndexOf("/")));
        webHDFSClient.createFile(filepath, payload);
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("Message sent to WebHDFS destination at path: " + filepath);
        }
    }

View Full Code Here

                .dataTypes(OutboundDataType.RAW, OutboundDataType.EVENT, OutboundDataType.RAW,
                        OutboundDataType.PROCESSED)
                .originatingConnection(testConnection)
                .build();
        outboundClient = new WebHDFSOutboundClient(outboundConfiguration);
        webHDFSClient = new WebHDFSClient(outboundConfiguration);
        webHDFSClient.mkdirs("");
    }

View Full Code Here

TOP

Related Classes of com.streamreduce.util.WebHDFSClient

com.streamreduce.client.outbound.WebHDFSOutboundClient

com.streamreduce.client.outbound.WebHDFSOutboundClientTestIT

com.streamreduce.core.service.ConnectionServiceImpl

net.sf.json.JSONObject

org.apache.http.client.methods.HttpDelete

org.apache.http.client.methods.HttpEntityEnclosingRequestBase

org.apache.http.client.methods.HttpGet

org.apache.http.client.methods.HttpPost

org.apache.http.client.methods.HttpPut

org.apache.http.entity.ByteArrayEntity

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.