Examples of java.io.StreamTokenizer.eolIsSignificant()

java.io.StreamTokenizer.eolIsSignificant()
Determines whether or not ends of line are treated as tokens. If the flag argument is true, this tokenizer treats end of lines as tokens; the nextToken method returns TT_EOL and also sets the ttype field to this value when an end of line is read.
A line is a sequence of characters ending with either a carriage-return character ('\r') or a newline character ('\n'). In addition, a carriage-return character followed immediately by a newline character is treated as a single end-of-line token.
If the flag is false, end-of-line characters are treated as white space and serve only to separate tokens. @param flag true indicates that end-of-line charactersare separate tokens; false indicates that end-of-line characters are white space. @see java.io.StreamTokenizer#nextToken() @see java.io.StreamTokenizer#ttype @see java.io.StreamTokenizer#TT_EOL


  private static StreamTokenizer createTokenizer(String input)
  {
    StreamTokenizer tokenizer = new StreamTokenizer(new StringReader(input));


    tokenizer.eolIsSignificant(false);
    tokenizer.lowerCaseMode(false);
    tokenizer.quoteChar('\"');
    tokenizer.slashSlashComments(false);
    tokenizer.slashStarComments(false);

View Full Code Here

        // Substituting forward slashes for backward slashes seems like
        // overkill.
        // streamTokenizer.quoteChar('"');
        streamTokenizer.ordinaryChar('"');


        streamTokenizer.eolIsSignificant(true);


        // Current token
        String token = "";


        // Single character token, usually a -

View Full Code Here

    // These words should all be numbers, for Double.valueOf to parse.


    tokenizer.resetSyntax();
    tokenizer.wordChars(0, 255);
    tokenizer.whitespaceChars(0, ' ');
    tokenizer.eolIsSignificant(true);
    java.util.Vector v = new java.util.Vector();


    // Ignore initial empty lines
    while (tokenizer.nextToken() == StreamTokenizer.TT_EOL)
      ;

View Full Code Here

    tokenizer = new StreamTokenizer(reader);


    initialize();


    tokenizer.commentChar('%');
    tokenizer.eolIsSignificant(true);
    while (StreamTokenizer.TT_EOF != 
     (currentToken = tokenizer.nextToken())) {


      // Skip empty lines 
      if (currentToken == StreamTokenizer.TT_EOL) {

View Full Code Here

        }


        // load the passwords
        FileInputStream in = new FileInputStream(passFile);
        StreamTokenizer stok = new StreamTokenizer(new InputStreamReader(in));
        stok.eolIsSignificant(false);
        //stok.wordChars('0', '0' + 16);
        passwordMap = new HashMap<String, byte[]>();
        while (stok.nextToken() != StreamTokenizer.TT_EOF) {
            String name = stok.sval;
            if (stok.nextToken() == StreamTokenizer.TT_EOF) {

View Full Code Here

        try {
            if (fontsDotDir.canRead()) {
                fr = new FileReader(fontsDotDir);
                BufferedReader br = new BufferedReader(fr, 8192);
                StreamTokenizer st = new StreamTokenizer(br);
                st.eolIsSignificant(true);
                int ttype = st.nextToken();
                if (ttype == StreamTokenizer.TT_NUMBER) {
                    int numEntries = (int)st.nval;
                    ttype = st.nextToken();
                    if (ttype == StreamTokenizer.TT_EOL) {

View Full Code Here

        tokens.resetSyntax();
        tokens.wordChars(0, 255);
        tokens.whitespaceChars(0, ' ');
        tokens.commentChar('#');
        tokens.quoteChar('"');
        tokens.eolIsSignificant(true);


        tokens.nextToken();
        while (tokens.ttype != StreamTokenizer.TT_EOF) {
            switch (tokens.ttype) {
                case StreamTokenizer.TT_WORD:

View Full Code Here


        StreamTokenizer st = new StreamTokenizer(source);


        st.slashSlashComments(true);
        st.slashStarComments(true);
        st.eolIsSignificant(false);
        st.quoteChar('"');
        st.wordChars('$', '$');
        st.wordChars('@', '@');
        st.wordChars('_', '_');
        st.wordChars('-', '-');

View Full Code Here


    if ( search_text != null ) {
      Reader r = new StringReader(search_text);
      StreamTokenizer input = new StreamTokenizer(r);
      input.resetSyntax();
      input.eolIsSignificant(true);
      input.wordChars('a', 'z');
      input.wordChars('A', 'Z');
      input.wordChars('0', '9');
      input.wordChars('%', '%');
      input.wordChars('*', '*');

View Full Code Here

        // Recognize quotes with single- and double-quotes.
        st.quoteChar('"');
        st.quoteChar('\'');


        // Recognize eol's.
        st.eolIsSignificant(true);
        
        // List of line # / token pairs.
        LinkedList<Pair<Integer, String>> lineNoTokenPairList = new LinkedList<Pair<Integer, String>>();
        
        try {

View Full Code Here

0 1 2 3 4 5 6 7 8 9

TOP

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.