Skip to content

Java StringTokenizer – Keep Delimiters

Java StringTokenizer – Keep Delimiters example shows how to keep delimiters while tokenizing the string content using the StringTokenizer object.

How to keep delimiters while using the StringTokenizer object?

In some cases, we want to get the delimiters as tokens as well while using the StringTokenizer object to tokenizing the string content. When we create the StringTokenizer object using the one or two-argument constructors, it does not return the delimiters as tokens. Please see the below-given code example to understand that.


As you can see from the output, the delimiter “,” is not returned back as tokens. In order to keep the delimiters as well, we need to use the below-given constructor while creating the StringTokenizer object.

If we pass true as the third argument, i.e. returnDelimiters, the StringTokenizer object also returns the delimiters as tokens when we iterate through the tokens.

See below code example showing that behavior.


By default, the StringTokenizer object does not return the empty or blank tokens. In that particular case, getting delimiters as tokens allows us to determine if the token is empty or blank. (If we get two consecutive delimiter tokens, it means that the token is empty or blank).

If you want to learn more about the tokenizer, please visit the Java StringTokenizer tutorial.

Please let me know your views in the comments section below.

About the author

Leave a Reply

Your email address will not be published.