TokenList class is used to store a list of tokens. This provides an add method which can be used to store an offset and length of a token within the buffer. Once the tokens have been added to they can be examined, in the order they were added, using the provided list method. This has a scalable capacity.
| |
| |