public class StringTokenizer extends Object implements ListIterator<String>, Cloneable
This class can split a String into many smaller strings. It aims to do a similar job to
StringTokenizer
, however it offers much more control and flexibility including
implementing the ListIterator
interface. By default, it is set up like StringTokenizer
.
The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.
Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.
Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.
At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.
Empty tokens may be removed or returned as null.
"a,b,c" - Three tokens "a","b","c" (comma delimiter) " a, b , c " - Three tokens "a","b","c" (default CSV processing trims whitespace) "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched)
Property | Type | Default |
---|---|---|
delim | CharSetMatcher | { \t\n\r\f} |
quote | NoneMatcher | {} |
ignore | NoneMatcher | {} |
emptyTokenAsNull | boolean | false |
ignoreEmptyTokens | boolean | true |
Constructor and Description |
---|
StringTokenizer()
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer, but with no text to
tokenize.
|
StringTokenizer(char[] input)
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer.
|
StringTokenizer(char[] input,
char delim)
Constructs a tokenizer splitting on the specified character.
|
StringTokenizer(char[] input,
char delim,
char quote)
Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified
quote character.
|
StringTokenizer(char[] input,
String delim)
Constructs a tokenizer splitting on the specified string.
|
StringTokenizer(char[] input,
StringMatcher delim)
Constructs a tokenizer splitting using the specified delimiter matcher.
|
StringTokenizer(char[] input,
StringMatcher delim,
StringMatcher quote)
Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified
quote matcher.
|
StringTokenizer(String input)
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer.
|
StringTokenizer(String input,
char delim)
Constructs a tokenizer splitting on the specified delimiter character.
|
StringTokenizer(String input,
char delim,
char quote)
Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified
quote character.
|
StringTokenizer(String input,
String delim)
Constructs a tokenizer splitting on the specified delimiter string.
|
StringTokenizer(String input,
StringMatcher delim)
Constructs a tokenizer splitting using the specified delimiter matcher.
|
StringTokenizer(String input,
StringMatcher delim,
StringMatcher quote)
Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified
quote matcher.
|
Modifier and Type | Method and Description |
---|---|
void |
add(String obj)
Unsupported ListIterator operation.
|
Object |
clone()
Creates a new instance of this Tokenizer.
|
String |
getContent()
Gets the String content that the tokenizer is parsing.
|
static StringTokenizer |
getCSVInstance()
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
static StringTokenizer |
getCSVInstance(char[] input)
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
static StringTokenizer |
getCSVInstance(String input)
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
StringMatcher |
getDelimiterMatcher()
Gets the field delimiter matcher.
|
StringMatcher |
getIgnoredMatcher()
Gets the ignored character matcher.
|
StringMatcher |
getQuoteMatcher()
Gets the quote matcher currently in use.
|
String[] |
getTokenArray()
Gets a copy of the full token list as an independent modifiable array.
|
List<String> |
getTokenList()
Gets a copy of the full token list as an independent modifiable list.
|
StringMatcher |
getTrimmerMatcher()
Gets the trimmer character matcher.
|
static StringTokenizer |
getTSVInstance()
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
static StringTokenizer |
getTSVInstance(char[] input)
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
static StringTokenizer |
getTSVInstance(String input)
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
boolean |
hasNext()
Tests whether there are any more tokens.
|
boolean |
hasPrevious()
Tests whether there are any previous tokens that can be iterated to.
|
boolean |
isEmptyTokenAsNull()
Tests whether the tokenizer currently returns empty tokens as null.
|
boolean |
isIgnoreEmptyTokens()
Tests whether the tokenizer currently ignores empty tokens.
|
String |
next()
Gets the next token.
|
int |
nextIndex()
Gets the index of the next token to return.
|
String |
nextToken()
Gets the next token from the String.
|
String |
previous()
Gets the token previous to the last returned token.
|
int |
previousIndex()
Gets the index of the previous token.
|
String |
previousToken()
Gets the previous token from the String.
|
void |
remove()
Throws
UnsupportedOperationException for this unsupported ListIterator operation. |
StringTokenizer |
reset()
Resets this tokenizer, forgetting all parsing and iteration already completed.
|
StringTokenizer |
reset(char[] input)
Resets this tokenizer, giving it a new input string to parse.
|
StringTokenizer |
reset(String input)
Resets this tokenizer, giving it a new input string to parse.
|
void |
set(String obj)
Throws
UnsupportedOperationException for this unsupported ListIterator operation. |
StringTokenizer |
setDelimiterChar(char delim)
Sets the field delimiter character.
|
StringTokenizer |
setDelimiterMatcher(StringMatcher delim)
Sets the field delimiter matcher.
|
StringTokenizer |
setDelimiterString(String delim)
Sets the field delimiter string.
|
StringTokenizer |
setEmptyTokenAsNull(boolean emptyAsNull)
Sets whether the tokenizer should return empty tokens as null.
|
StringTokenizer |
setIgnoredChar(char ignored)
Sets the character to ignore.
|
StringTokenizer |
setIgnoredMatcher(StringMatcher ignored)
Sets the matcher for characters to ignore.
|
StringTokenizer |
setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
Sets whether the tokenizer should ignore and not return empty tokens.
|
StringTokenizer |
setQuoteChar(char quote)
Sets the quote character to use.
|
StringTokenizer |
setQuoteMatcher(StringMatcher quote)
Set the quote matcher to use.
|
StringTokenizer |
setTrimmerMatcher(StringMatcher trimmer)
Sets the matcher for characters to trim.
|
int |
size()
Gets the number of tokens found in the String.
|
protected List<String> |
tokenize(char[] srcChars,
int offset,
int count)
Internal method to performs the tokenization.
|
String |
toString()
Gets the String content that the tokenizer is parsing.
|
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
forEachRemaining
public StringTokenizer()
This constructor is normally used with reset(String)
.
public StringTokenizer(char[] input)
input
- the string which is to be parsed, not clonedpublic StringTokenizer(char[] input, char delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterpublic StringTokenizer(char[] input, char delim, char quote)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterquote
- the field quoted string characterpublic StringTokenizer(char[] input, String delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter stringpublic StringTokenizer(char[] input, StringMatcher delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter matcherpublic StringTokenizer(char[] input, StringMatcher delim, StringMatcher quote)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterquote
- the field quoted string characterpublic StringTokenizer(String input)
input
- the string which is to be parsedpublic StringTokenizer(String input, char delim)
input
- the string which is to be parseddelim
- the field delimiter characterpublic StringTokenizer(String input, char delim, char quote)
input
- the string which is to be parseddelim
- the field delimiter characterquote
- the field quoted string characterpublic StringTokenizer(String input, String delim)
input
- the string which is to be parseddelim
- the field delimiter stringpublic StringTokenizer(String input, StringMatcher delim)
input
- the string which is to be parseddelim
- the field delimiter matcherpublic StringTokenizer(String input, StringMatcher delim, StringMatcher quote)
input
- the string which is to be parseddelim
- the field delimiter matcherquote
- the field quoted string matcherpublic static StringTokenizer getCSVInstance()
You must call a "reset" method to set the string which you want to parse.
public static StringTokenizer getCSVInstance(char[] input)
input
- the text to parsepublic static StringTokenizer getCSVInstance(String input)
input
- the text to parsepublic static StringTokenizer getTSVInstance()
You must call a "reset" method to set the string which you want to parse.
public static StringTokenizer getTSVInstance(char[] input)
input
- the string to parsepublic static StringTokenizer getTSVInstance(String input)
input
- the string to parsepublic void add(String obj)
add
in interface ListIterator<String>
obj
- this parameter ignored.UnsupportedOperationException
- alwayspublic Object clone()
CloneNotSupportedException
is caught, return null
.public String getContent()
public StringMatcher getDelimiterMatcher()
public StringMatcher getIgnoredMatcher()
These characters are ignored when parsing the String, unless they are within a quoted region. The default value is not to ignore anything.
public StringMatcher getQuoteMatcher()
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data. The default value is '"' (double quote).
public String[] getTokenArray()
public List<String> getTokenList()
public StringMatcher getTrimmerMatcher()
These characters are trimmed off on each side of the delimiter until the token or quote is found. The default value is not to trim anything.
public boolean hasNext()
public boolean hasPrevious()
hasPrevious
in interface ListIterator<String>
public boolean isEmptyTokenAsNull()
public boolean isIgnoreEmptyTokens()
public String next()
next
in interface Iterator<String>
next
in interface ListIterator<String>
NoSuchElementException
- if there are no more elementspublic int nextIndex()
nextIndex
in interface ListIterator<String>
public String nextToken()
next()
except it returns null rather than throwing
NoSuchElementException
when no tokens remain.public String previous()
previous
in interface ListIterator<String>
public int previousIndex()
previousIndex
in interface ListIterator<String>
public String previousToken()
public void remove()
UnsupportedOperationException
for this unsupported ListIterator operation.remove
in interface Iterator<String>
remove
in interface ListIterator<String>
UnsupportedOperationException
- alwayspublic StringTokenizer reset()
This method allows the same tokenizer to be reused for the same String.
public StringTokenizer reset(char[] input)
input
- the new character array to tokenize, not cloned, null sets no text to parsepublic StringTokenizer reset(String input)
input
- the new string to tokenize, null sets no text to parsepublic void set(String obj)
UnsupportedOperationException
for this unsupported ListIterator operation.set
in interface ListIterator<String>
obj
- this parameter ignored.UnsupportedOperationException
- alwayspublic StringTokenizer setDelimiterChar(char delim)
delim
- the delimiter character to usepublic StringTokenizer setDelimiterMatcher(StringMatcher delim)
The delimiter is used to separate one token from another.
delim
- the delimiter matcher to usepublic StringTokenizer setDelimiterString(String delim)
delim
- the delimiter string to usepublic StringTokenizer setEmptyTokenAsNull(boolean emptyAsNull)
emptyAsNull
- whether empty tokens are returned as nullpublic StringTokenizer setIgnoredChar(char ignored)
This character is ignored when parsing the String, unless it is within a quoted region.
ignored
- the ignored character to usepublic StringTokenizer setIgnoredMatcher(StringMatcher ignored)
These characters are ignored when parsing the String, unless they are within a quoted region.
ignored
- the ignored matcher to use, null ignoredpublic StringTokenizer setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
ignoreEmptyTokens
- whether empty tokens are not returnedpublic StringTokenizer setQuoteChar(char quote)
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.
quote
- the quote character to usepublic StringTokenizer setQuoteMatcher(StringMatcher quote)
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.
quote
- the quote matcher to use, null ignoredpublic StringTokenizer setTrimmerMatcher(StringMatcher trimmer)
These characters are trimmed off on each side of the delimiter until the token or quote is found.
trimmer
- the trimmer matcher to use, null ignoredpublic int size()
protected List<String> tokenize(char[] srcChars, int offset, int count)
Most users of this class do not need to call this method. This method will be called automatically by other (public) methods when required.
This method exists to allow subclasses to add code before or after the tokenization. For example, a subclass could alter the character array, offset or count to be parsed, or call the tokenizer multiple times on multiple strings. It is also be possible to filter the results.
StrTokenizer
will always pass a zero offset and a count equal to the length of the array to this
method, however a subclass may pass other values, or even an entirely different array.
srcChars
- the character array being tokenized, may be nulloffset
- the start position within the character array, must be validcount
- the number of characters to tokenize, must be validCopyright © 2014–2022 The Apache Software Foundation. All rights reserved.