Examples of java.io.StreamTokenizer.ordinaryChar()

java.io.StreamTokenizer.ordinaryChar()
Specifies that the character argument is "ordinary" in this tokenizer. It removes any special significance the character has as a comment character, word component, string delimiter, white space, or number character. When such a character is encountered by the parser, the parser treats it as a single-character token and sets ttype field to the character value.
Making a line terminator character "ordinary" may interfere with the ability of a StreamTokenizer to count lines. The lineno method may no longer reflect the presence of such terminator characters in its line count. @param ch the character. @see java.io.StreamTokenizer#ttype

        // We can't use quoteChar here because it does backslash
        // substitution, so "c:\ptII" ends up as "c:ptII"
        // Substituting forward slashes for backward slashes seems like
        // overkill.
        // streamTokenizer.quoteChar('"');
        streamTokenizer.ordinaryChar('"');


        streamTokenizer.eolIsSignificant(true);


        // Current token
        String token = "";

View Full Code Here

    StreamTokenizer tokenizer =
      new StreamTokenizer(new StringReader(rangeString));
    tokenizer.resetSyntax();         
    tokenizer.whitespaceChars(0, ' ');    
    tokenizer.wordChars(' '+1,'\u00FF');
    tokenizer.ordinaryChar('[');
    tokenizer.ordinaryChar('(');
    tokenizer.ordinaryChar(',');
    tokenizer.ordinaryChar(']');
    tokenizer.ordinaryChar(')');

View Full Code Here

      new StreamTokenizer(new StringReader(rangeString));
    tokenizer.resetSyntax();         
    tokenizer.whitespaceChars(0, ' ');    
    tokenizer.wordChars(' '+1,'\u00FF');
    tokenizer.ordinaryChar('[');
    tokenizer.ordinaryChar('(');
    tokenizer.ordinaryChar(',');
    tokenizer.ordinaryChar(']');
    tokenizer.ordinaryChar(')');


    try {

View Full Code Here

    tokenizer.resetSyntax();         
    tokenizer.whitespaceChars(0, ' ');    
    tokenizer.wordChars(' '+1,'\u00FF');
    tokenizer.ordinaryChar('[');
    tokenizer.ordinaryChar('(');
    tokenizer.ordinaryChar(',');
    tokenizer.ordinaryChar(']');
    tokenizer.ordinaryChar(')');


    try {

View Full Code Here

    tokenizer.whitespaceChars(0, ' ');    
    tokenizer.wordChars(' '+1,'\u00FF');
    tokenizer.ordinaryChar('[');
    tokenizer.ordinaryChar('(');
    tokenizer.ordinaryChar(',');
    tokenizer.ordinaryChar(']');
    tokenizer.ordinaryChar(')');


    try {


      // get opening brace

View Full Code Here

    tokenizer.wordChars(' '+1,'\u00FF');
    tokenizer.ordinaryChar('[');
    tokenizer.ordinaryChar('(');
    tokenizer.ordinaryChar(',');
    tokenizer.ordinaryChar(']');
    tokenizer.ordinaryChar(')');


    try {


      // get opening brace
      tokenizer.nextToken();

View Full Code Here

   
    StreamTokenizer st = new StreamTokenizer(br);
    initTokenizer(st);
    readStructure(st);
    
    st.ordinaryChar(',');
    st.ordinaryChar('\t');
    
    m_cumulativeStructure = new FastVector(m_structure.numAttributes());
    for (int i = 0; i < m_structure.numAttributes(); i++) {
      m_cumulativeStructure.addElement(new Hashtable());

View Full Code Here

    StreamTokenizer st = new StreamTokenizer(br);
    initTokenizer(st);
    readStructure(st);
    
    st.ordinaryChar(',');
    st.ordinaryChar('\t');
    
    m_cumulativeStructure = new FastVector(m_structure.numAttributes());
    for (int i = 0; i < m_structure.numAttributes(); i++) {
      m_cumulativeStructure.addElement(new Hashtable());
    }

View Full Code Here

  st.wordChars('_', '_');
  st.wordChars('$', '$');
  st.wordChars(128 + 32, 255);
  st.whitespaceChars(0, ' ');
  st.lowerCaseMode(false);
  st.ordinaryChar('/');
  st.slashSlashComments(false);
  st.slashStarComments(false);
  return st;
    }

View Full Code Here

        StreamTokenizer t = new StreamTokenizer(new InputStreamReader(
                new FileInputStream(f)));


        t.commentChar('#');


        t.ordinaryChar(TT_START_SECTION);
        t.ordinaryChar(TT_END_SECTION);
        t.ordinaryChar(TT_EQAUL);


        t.wordChars('_', '_');
        t.wordChars(':', ':');

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.