Java StringTokenizer – Using RegEx Pattern

Java StringTokenizer – Using RegEx Pattern example shows what happens when we use regex pattern as delimiter to tokenize the string.

Can we use a regex pattern as delimiter to tokenize the string?

The StringTokenizer class in Java is used to tokenize the string content. We can provide the set of delimiters using which we want to generate the tokens from the given string content.

One way to do this is to create a StringTokenizer object using the below given constructor.

This constructor creates a new StringTokenizer object that will tokenize the string content based on the set of delimiters we passed. Once the object is created, we can get the tokens from the string using hasMoreTokens and nextToken methods as given in below example.

Output

In the above example, I used a simple space character to create tokens from the string. However, many a time we need more complex approach to generate tokens, for example, using a regex pattern.

However, the StringTokenizer does not support regex pattern. Providing a regex pattern as delimiters does not work as given below.

Output

As you can see from the output, the word boundary “\\b” regex pattern did not work and tokens were not generated.

How to generate tokens based on regex pattern?

The StringTokenizer does not support the regex pattern, but you can use the String split method to do the same. The only difference is, the split method returns an array containing all tokens.

Here is the same example using the split method.

Output

If you want to tokenize the string using a regex pattern, just use the split method instead of the StringTokenizer class.

If you want to learn more about the string tokenizer, please visit the Java StringTokenizer Tutorial.

Please let me know your views in the comments section below.

About the author

RahimV

RahimV

My name is RahimV and I have over 16 years of experience in designing and developing Java applications. Over the years I have worked with many fortune 500 companies as an eCommerce Architect. My goal is to provide high quality but simple to understand Java tutorials and examples for free. If you like my website, follow me on Facebook and Twitter.

Add Comment

Your email address will not be published. Required fields are marked *