Skip to content

Java StringTokenizer Tutorial with Examples

Java StringTokenizer tutorial with examples will help you understand how to use StringTokenizer in Java in an easy way. StringTokenizer class in Java is used to tokenize or split the string into tokens.

The StringTokenizer class is contained in the util package, so you need to import the java.util.StringTokenizer class in order to use it.

How to create an object of StringTokenizer class?

The StringTokenizer class provides several constructors using which we can create an object of the class.

This constructor creates a new StringTokenizer object for the specified string argument. Since there are no delimiters specified, the default set of delimiters, i.e. ” \t\n\r\f”, are used (a space character, a tab character, a newline character, a carriage return, and a form feed character). These delimiters will not be returned as tokens.

There is a two-argument constructor that accepts the string as well as the delimiter you want to use instead of the default set of delimiters to tokenize the string.

This constructor creates an object of the StringTokenizer object that will use the provided delimiter to tokenize the string. The specified delimiters will not be returned as tokens.

There is also a third constructor that accepts 3 arguments, the string to tokenize, the delimiter to use, and whether to return delimiters as tokens.

If the boolean flag is true, the specified delimiter will also be returned as tokens.

How to generate the string tokens using StringTokenizer?

Once the StringTokenizer object is created, we can use the hasMoreTokens and nextToken methods to get the tokens.

The hasMoreTokens method returns true if there are more tokens available, false otherwise.

The nextToken method returns the next token from the string tokenizer object. It also increases the current position after the token that is returned by this method.

This method throws “java.util.NoSuchElementException” exception in case there are no more tokens in the tokenizer object. Always make sure to call the hasMoreTokens method before calling the nextToken method to avoid this exception.

Here is an example code to tokenize the string.


The StringTokenizer class also provides hasMoreElements and nextElement methods that are identical to the hasMoreTokens and nextToken methods respectively. The StringTokenizer class provides these identical methods just so that it can implement the Enumeration interface.

You can also use these methods instead of the hasMoreTokens and nextToken methods as given below.


How to get the delimiters as tokens?

To return delimiters as tokens, we need to use a three-argument constructor while creating the tokenizer object as given below.


How to get the total number of tokens?

The countTokens method returns the total number of tokens from the current position. Unlike the hasMoreTokens method, this method does not advance the current position.


Please note that the countTokens does not always return the total number of tokens. In the above example since we have not called nextToken or nextElement method, the current position is at the start of the string. That is why it returned the total number of tokens.

As and when we get the next tokens from the tokenizer object, the countTokens method returns the total number of tokens remaining in the tokenizer object from the current position. See below given example to understand it.


How to specify multiple delimiters?

We can specify the multiple delimiters while creating the StringTokenizer object as given below. Any of the given delimiters will be used to create tokens of the string content.


As you can see from the output, tokens were generated using all of the delimiters we specified in the constructor.

How to change the delimiter in between iterating?

We can use the overloaded nextToken method to change the delimiter.

This method sets the default delimiter to the delimiter we provided. This delimiter will also be used for the subsequent nextToken method calls.


StringTokenizer Javadoc

Please let me know if you liked the Java StringTokenizer tutorial with examples in the comments section below.

About the author

Leave a Reply

Your email address will not be published.