Examples of org.apache.commons.lang3.text.StrTokenizer

org.apache.commons.lang3.text.StrTokenizer
Tokenizes a string based based on delimiters (separators) and supporting quoting and ignored character concepts.
This class can split a String into many smaller strings. It aims to do a similar job to {@link java.util.StringTokenizer StringTokenizer}, however it offers much more control and flexibility including implementing the ListIterator interface. By default, it is set up like StringTokenizer.
The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.
Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.
Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.
At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.
Empty tokens may be removed or returned as null.
```
 "a,b,c"         - Three tokens "a","b","c"   (comma delimiter) " a, b , c "    - Three tokens "a","b","c"   (default CSV processing trims whitespace) "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched) 
```
This tokenizer has the following properties and options:

Property Type Default

delim CharSetMatcher { \t\n\r\f}

quote NoneMatcher {}

ignore NoneMatcher {}

emptyTokenAsNull boolean false

ignoreEmptyTokens boolean true

@since 2.2 @version $Id: StrTokenizer.java 1153241 2011-08-02 18:49:52Z ggregory $

Property	Type	Default
delim	CharSetMatcher	{ \t\n\r\f}
quote	NoneMatcher	{}
ignore	NoneMatcher	{}
emptyTokenAsNull	boolean	false
ignoreEmptyTokens	boolean	true

     *
     * @return the {@code StrSubstitutor} used by this object
     */
    private StrSubstitutor initSubstitutor()
    {
        return new StrSubstitutor(new StrLookup<Object>()
        {
            @Override
            public String lookup(String key)
            {
                Object result = resolve(key);

View Full Code Here


        String[] values;
        if (separator == null) {
            values = new String[]{valueString};
        } else {
            StrTokenizer tokenizer = new StrTokenizer(valueString, separator);
            values = tokenizer.getTokenArray();
        }


        String[] result = new String[values.length + 1];
        result[0] = key;
        System.arraycopy(values, 0, result, 1, values.length);

View Full Code Here

        }
        storeLine(values);
    }


    public void parseTokenized(String line) {
        StrTokenizer tokenizer = new StrTokenizer(line, separator);
        tokenizer.setIgnoreEmptyTokens(ignoreEmptyTokens);
        tokenizer.setQuoteChar(quoteChar);
        String[] tokens = tokenizer.getTokenArray();
        storeLine(tokens);
    }

View Full Code Here

      logger.trace("Buffer: {}, cursor: {}", buffer, cursor);
      logger.trace("Candidates {}", candidates);
    }
    if (StringUtils.isNotBlank(buffer)) {
      // User is typing a command
      StrTokenizer strTokenizer = new StrTokenizer(buffer);
      String action = strTokenizer.next();
      Collection<String> arguments = argumentMap.get(action);
      if (arguments != null) {
        if (logger.isTraceEnabled()) {
          logger.trace("Arguments found for {}, Tokens: {}", action, strTokenizer.getTokenList());
          logger.trace("Arguments for {}: {}", action, arguments);
        }
        List<String> args = new ArrayList<String>(arguments);
        List<Completer> completers = new ArrayList<Completer>();
        for (String token : strTokenizer.getTokenList()) {
          boolean argContains = arguments.contains(token);
          if (token.startsWith("-") && !argContains) {
            continue;
          }
          if (argContains) {

View Full Code Here

    return success;
  }


  @Override
  protected int executeCommand(String line) {
    String[] tokens = new StrTokenizer(line).getTokenArray();
    String action = tokens[0];
    String[] actionArgs = Arrays.copyOfRange(tokens, 1, tokens.length);
    if (logger.isDebugEnabled()) {
      logger.debug("Executing command action: {}, Tokens: {}", action, tokens.length);
    }

View Full Code Here

    return success;
  }


  @Override
  protected int executeCommand(String line) {
    String[] tokens = new StrTokenizer(line).getTokenArray();
    String action = tokens[0];
    String[] actionArgs = Arrays.copyOfRange(tokens, 1, tokens.length);
    if (logger.isDebugEnabled()) {
      logger.debug("Executing command action: {}, Tokens: {}", action, tokens.length);
    }

View Full Code Here

                fieldResult.add(makeAction((String) arrActionResult.get(x), prefixCount + splitCampiVal[x].replaceAll("/\\*", ""), builder));
                // appo=prefix+prefix_count+element.getText().replaceAll("/\\*","");
              } else if (StringUtils.defaultString(splitCampiVal[x]).contains("/*/")) {
                String appoSt = splitCampiVal[x];
                int count = 0;
                StrTokenizer strTokenizerGenericField = new StrTokenizer(genericField, StrMatcher.charSetMatcher("[]"));
                while (strTokenizerGenericField.hasNext()) {
                  String tokenStr2 = strTokenizerGenericField.nextToken();
                  try {
                    Integer.parseInt(tokenStr2);
                    appoSt = appoSt.replaceFirst("/\\*", "[" + tokenStr2 + "]");
                  } catch (Exception e) {
                    // TODO: handle exception

View Full Code Here


    return returnValue;
  }


  private static void analizeAction(String fieldName, StrBuilder bufFieldName, ArrayList arrActionResult) {
    StrTokenizer strTokenizer = new StrTokenizer(fieldName, ",");
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      try {
        String parseAct = tokenStr.substring(0, tokenStr.indexOf(":"));
        if (parseAct.equals("name")) {
          bufFieldName.replaceAll("name:", "");// (0,fieldName.substring(fieldName.indexOf(":")+1));
          arrActionResult.add("name");

View Full Code Here

    }
  }


  public static String scriptingResolver(String genericField, String fieldValue, XMLBuilder builder) {
    StringBuffer stringBuffer = new StringBuffer();
    StrTokenizer strTokenizer = new StrTokenizer(fieldValue, StrMatcher.charSetMatcher("{}"));
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      ArrayList arrayList = getFieldName(genericField, tokenStr, builder, "");
      if (arrayList.size() > 0) {
        for (int x = 0; x < arrayList.size(); x++) {
          stringBuffer.append(StringEscapeUtils.escapeEcmaScript((String) arrayList.get(x)));
          if (arrayList.size() - 1 != x) {

View Full Code Here

    return stringBuffer.toString();
  }


  public static String scriptingResolver(String genericField, String fieldValue, XMLBuilder builder, String prefixCount) {
    StringBuffer stringBuffer = new StringBuffer();
    StrTokenizer strTokenizer = new StrTokenizer(fieldValue, StrMatcher.charSetMatcher("{}"));
    while (strTokenizer.hasNext()) {
      String tokenStr = strTokenizer.nextToken();
      ArrayList arrayList = getFieldName(genericField, tokenStr, builder, prefixCount);
      if (arrayList.size() > 0) {
        for (int x = 0; x < arrayList.size(); x++) {
          stringBuffer.append(StringEscapeUtils.escapeEcmaScript((String) arrayList.get(x)));
          if (arrayList.size() - 1 != x) {

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

Related Classes of org.apache.commons.lang3.text.StrTokenizer

au.org.intersect.samifier.domain.PeptideSearchResult

co.jirm.orm.writer.SqlWriterStrategy

com.crawljax.forms.FormInput

com.lassitercg.faces.components.event.SheetUpdate

com.lassitercg.faces.components.sheet.BadUpdate

com.lassitercg.faces.components.sheet.Sheet$RowColIndex

com.softwareloop.tstconfig.Config

hudson.plugins.git.GitSCM

hudson.plugins.git.opt.PreBuildMergeOptions

jp.vmi.selenium.selenese.DriverIndependentTest

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.