Examples of CSVTokenizer

com.bbn.openmap.util.CSVTokenizer
Tokenizer for comma separated values files, at least as generated by excel.
token() returns the next token, which can be either:
- null, indicating an empty field.
- a Double, indicating a numeric field.
- a String, indicating an alphanumeric field.
- the NEWLINE object, indicating the end of a record.
- the EOF object, test with isEOF(), indicating the end of file.
```
 
```
com.cloudera.cdk.morphline.shaded.com.googlecode.jcsv.fastreader.CSVTokenizer
The CSVTokenizer specifies the behaviour how to parse each line into a List of Strings.
org.jfree.report.util.CSVTokenizer
The csv tokenizer class allows an application to break a Comma Separated Value format into tokens. The tokenization method is much simpler than the one used by the StringTokenizer class. The CSVTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.
The set of separator (the characters that separate tokens) may be specified either at creation time or on a per-token basis.
An instance of CSVTokenizer behaves in one of two ways, depending on whether it was created with the returnSeparators flag having the value true or false:
- If the flag is false, delimiter characters serve to separate tokens. A token is a maximal sequence of consecutive characters that are not separator.
- If the flag is true, delimiter characters are themselves considered to be tokens. A token is thus either one delimiter character, or a maximal sequence of consecutive characters that are not separator.
A CSVTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.
A token is returned by taking a substring of the string that was used to create the CSVTokenizer object.
The following is one example of the use of the tokenizer. The code:
```
 CSVTokenizer csvt = new CSVTokenizer("this,is,a,test"); while (csvt.hasMoreTokens()) { println(csvt.nextToken()); } 
```
prints the following output:
```
 this is a test 
```
@author abupon
org.jugile.util.CsvTokenizer
"Thus saith the Lord GOD; When I shall have gathered the house of Israel from the people among whom they are scattered, and shall be sanctified in them in the sight of the heathen, then shall they dwell in their land that I have given to my servant Jacob." (Ezekiel 28:25) ========== Simple utility for using csv data files. For some purposes simple and flat file with comma separated value rows is more convinient way to write and read the data than corresponding xml or json structure. @author jukka.rahkonen@iki.fi
org.kitesdk.morphline.shaded.com.googlecode.jcsv.fastreader.CSVTokenizer
The CSVTokenizer specifies the behaviour how to parse each line into a List of Strings.
org.pentaho.reporting.libraries.base.util.CSVTokenizer
The csv tokenizer class allows an application to break a Comma Separated Value format into tokens. The tokenization method is much simpler than the one used by the StringTokenizer class. The CSVTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.
The set of separator (the characters that separate tokens) may be specified either at creation time or on a per-token basis.
An instance of CSVTokenizer behaves in one of two ways, depending on whether it was created with the returnSeparators flag having the value true or false:
- If the flag is false, delimiter characters serve to separate tokens. A token is a maximal sequence of consecutive characters that are not separator.
- If the flag is true, delimiter characters are themselves considered to be tokens. A token is thus either one delimiter character, or a maximal sequence of consecutive characters that are not separator.
A CSVTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.
A token is returned by taking a substring of the string that was used to create the CSVTokenizer object.
The following is one example of the use of the tokenizer. The code:
```
 CSVTokenizer csvt = new CSVTokenizer("this,is,a,test"); while (csvt.hasMoreTokens()) { println(csvt.nextToken()); } 
```
prints the following output:
```
 this is a test 
```
@author abupon

Examples of com.bbn.openmap.util.CSVTokenizer

            URL csvURL = PropUtils.getResourceOrFileOrURL(null, locationFile);
            if (csvURL == null) {


            }
            streamReader = new BufferedReader(new InputStreamReader(csvURL.openStream()));
            CSVTokenizer csvt = new CSVTokenizer(streamReader);


            token = csvt.token();


            while (!csvt.isEOF(token)) {
                int i = 0;


                Debug.message("csvlocation",
                        "CSVLocationHandler| Starting a line | have"
                                + (readHeader ? " " : "n't ") + "read header");


                while (!csvt.isNewline(token) && !csvt.isEOF(token)) {


                    if (readHeader) {
                        tokenHandler.handleToken(token, i);
                    }


                    token = csvt.token();
                    // For some reason, the check above doesn't always
                    // work
                    if (csvt.isEOF(token)) {
                        break;
                    }
                    i++;
                }


                if (!readHeader) {
                    readHeader = true;
                } else {
                    lineCount++;
                    tokenHandler.createAndAddObjectFromTokens(qt);
                }
                token = csvt.token();
            }
        } catch (java.io.IOException ioe) {
            throw new com.bbn.openmap.util.HandleError(ioe);
        } catch (ArrayIndexOutOfBoundsException aioobe) {
            throw new com.bbn.openmap.util.HandleError(aioobe);

View Full Code Here

Examples of com.bbn.openmap.util.CSVTokenizer

            // This lets the property be specified as a file name
            // even if it's not specified as file:/<name> in
            // the properties file.
            URL csvURL = infoUrl;
            streamReader = new BufferedReader(new InputStreamReader(csvURL.openStream()));
            CSVTokenizer csvt = new CSVTokenizer(streamReader, readNumbersAsStrings);
            int count = 0;
            token = csvt.token();
            while (!csvt.isEOF(token)) {
                count++;


                Vector rec_line = new Vector();
                while (!csvt.isNewline(token)) {
                    rec_line.addElement(token);
                    token = csvt.token();
                    if (csvt.isEOF(token))
                        break;
                }


                //  Don't add the header record, because we don't care
                //  about it.
                if (header_read) {
                    records.addElement(rec_line);
                } else if (headersExist) {
                    headerRecord = rec_line;
                    header_read = true;
                }


                if (Debug.debugging("csv")) {
                    Debug.output("CSVFile.read: " + rec_line);
                }


                token = csvt.token();
            }
        } catch (java.io.IOException ioe) {
            throw new com.bbn.openmap.util.HandleError(ioe);
        } catch (ArrayIndexOutOfBoundsException aioobe) {
            throw new com.bbn.openmap.util.HandleError(aioobe);

View Full Code Here

Examples of com.cloudera.cdk.morphline.shaded.com.googlecode.jcsv.fastreader.CSVTokenizer

    assertEquals(Arrays.asList("", "", "x"), split("||x", '|', false));
  }
  
  private List<String> split(String line, char separator, boolean isQuoted) throws IOException {
    Record record = new Record();
    CSVTokenizer tokenizer;
    if (isQuoted) {
      tokenizer = new QuotedCSVTokenizer(separator, false, new ArrayList(), '"');
      tokenizer.tokenizeLine(line, new BufferedReader(new StringReader("")), record);      
    } else {
      tokenizer = new SimpleCSVTokenizer(separator, false, new ArrayList());
      tokenizer.tokenizeLine(line, null, record);
    }
    List results = new ArrayList();
    for (int i = 0; i < record.getFields().asMap().size(); i++) {
      assertEquals(1, record.get("column" + i).size());
      results.add(record.getFirstValue("column" + i));

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer


  protected String lookupVariable (final String entity)
  {
    // first, split the entity into separate strings (separator is '.').


    final CSVTokenizer tokenizer = new CSVTokenizer(entity, ".");
    if (tokenizer.hasMoreTokens())
    {
      final String name = tokenizer.nextToken();
      final Object base = performInitialLookup(name);
      try
      {
        if (tokenizer.hasMoreTokens())
        {
          return continueLookupVariable(tokenizer, base);
        }
        else
        {

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

   * @return a property value.
   * @throws BeanException if there was an error during the conversion.
   */
  public Object toPropertyValue (final String s) throws BeanException
  {
    final CSVTokenizer tokenizer = new CSVTokenizer(s);
    final ArrayList elements = new ArrayList();
    while (tokenizer.hasMoreTokens())
    {
      final String token = tokenizer.nextToken();
      elements.add(elementConverter.toPropertyValue(token));
    }
    final Object retval =
            Array.newInstance(elementType, elements.size());
    for (int i = 0; i < elements.size(); i++)

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

  }


  private String[] splitLine (final String line)
  {
    final ArrayList row = new ArrayList();
    final CSVTokenizer tokenizer = new CSVTokenizer(line, getSeparator());
    while (tokenizer.hasMoreElements())
    {
      row.add(tokenizer.nextElement());
    }
    return (String[]) row.toArray(new String[row.size()]);
  }

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

      final String bootModules =
              getGlobalConfig().getConfigProperty
              ("org.jfree.report.boot.Modules");
      if (bootModules != null)
      {
        final CSVTokenizer csvToken = new CSVTokenizer(bootModules, ",");
        while (csvToken.hasMoreTokens())
        {
          final String token = csvToken.nextToken();
          getPackageManager().load(token);
        }
      }
    }
    catch (SecurityException se)

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

    {
      throw new ReportDataFactoryException("Malformed query: " + query);
    }
    final String parameterText =
            query.substring(parameterStartIdx + 1, parameterEndIdx);
    final CSVTokenizer tokenizer = new CSVTokenizer(parameterText);
    final int size = tokenizer.countTokens();
    final String[] parameterNames = new String[size];
    int i = 0;
    while (tokenizer.hasMoreTokens())
    {
      parameterNames[i] = tokenizer.nextToken();
      i += 1;
    }
    return parameterNames;
  }

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

      final String bootModules =
              getGlobalConfig().getConfigProperty
              ("org.jfree.report.boot.Modules");
      if (bootModules != null)
      {
        final CSVTokenizer csvToken = new CSVTokenizer(bootModules, ",");
        while (csvToken.hasMoreTokens())
        {
          final String token = csvToken.nextToken();
          getPackageManager().load(token);
        }
      }
    }
    catch (SecurityException se)

View Full Code Here

Examples of org.jfree.report.util.CSVTokenizer

    {
      throw new ReportDataFactoryException("Malformed query: " + query);
    }
    final String parameterText =
            query.substring(parameterStartIdx + 1, parameterEndIdx);
    final CSVTokenizer tokenizer = new CSVTokenizer(parameterText);
    final int size = tokenizer.countTokens();
    final String[] parameterNames = new String[size];
    int i = 0;
    while (tokenizer.hasMoreTokens())
    {
      parameterNames[i] = tokenizer.nextToken();
      i += 1;
    }
    return parameterNames;
  }

View Full Code Here

0 1 2 3 4 5

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.