Java StringTokenizer – Keep Delimiters

Java StringTokenizer – Keep Delimiters example shows how to keep delimiters while tokenizing the string content using the StringTokenizer object.

How to keep delimiters while using the StringTokenizer object?

In some cases, we want to get the delimiters as tokens as well while using the StringTokenizer object to tokenizing the string content. When we create the StringTokenizer object using the one or two-argument constructors, it does not return the delimiters as tokens. Please see the below-given code example to understand that.

Output

As you can see from the output, the delimiter “,” is not returned back as tokens. In order to keep the delimiters as well, we need to use the below-given constructor while creating the StringTokenizer object.

If we pass true as the third argument, i.e. returnDelimiters, the StringTokenizer object also returns the delimiters as tokens when we iterate through the tokens.

See below code example showing that behavior.

Output

By default, the StringTokenizer object does not return the empty or blank tokens. In that particular case, getting delimiters as tokens allows us to determine if the token is empty or blank. (If we get two consecutive delimiter tokens, it means that the token is empty or blank).

If you want to learn more about the tokenizer, please visit the Java StringTokenizer tutorial.

Please let me know your views in the comments section below.

About the author

RahimV

RahimV

My name is RahimV and I have over 16 years of experience in designing and developing Java applications. Over the years I have worked with many fortune 500 companies as an eCommerce Architect. My goal is to provide high quality but simple to understand Java tutorials and examples for free. If you like my website, follow me on Facebook and Twitter.

Add Comment

Your email address will not be published. Required fields are marked *