Java StringTokenizer - Get Array of Tokens

Java StringTokenizer get an array of tokens example shows how to get string tokens in a String array from the StringTokenizer object.

How to get an array of string tokens from the StringTokenizer?

The StringTokenizer class can tokenize the string data into tokens using the delimiters we provide. Once these tokens are generated from the string, we can iterate through the tokens using the nextToken or nextElement method.

These methods return the next token from the string from the current position. There are also methods, namely hasMoreTokens and hasMoreElements, that returns true if there are more tokens in the string content.

However, the StringTokenizer class does not have any method that returns tokens in the form of a String array. If we want, we need to do that manually by iterating the tokens using the provided methods.

In the code given below, I have created a method that iterates through the string tokens using hasMoreTokens and nextToken methods. It then adds each token element to the String array.

package com.javacodeexamples.stringtokenizerexamples;

import java.util.StringTokenizer;

public class StringTokenizerToArray {

public static void main(String[] args) {

String[] tokenArray = getTokensArray("Java StringTokenizer to Array", " ");

if(tokenArray != null) {

for(int i = 0 ; i < tokenArray.length ; i++) {

System.out.println( "Index: " + i + ", token: " + tokenArray[i] );

}

private static String[] getTokensArray(String strData, String strDelimiters) {

String[] strTokenArray = null;

try {

if( strData == null || strDelimiters == null )

return strTokenArray;

//create tokenizer object

StringTokenizer st = new StringTokenizer(strData, strDelimiters);

* Create the array having same size as total

* number of tokens

strTokenArray = new String[ st.countTokens() ];

//iterate through all the tokens

int count = 0;

while( st.hasMoreTokens() ) {

//add to an array

strTokenArray[count++] = st.nextToken();

}

}catch(Exception e) {

e.printStackTrace();

}

return strTokenArray;

}

Output

Index: 0, token: Java

Index: 1, token: StringTokenizer

Index: 2, token: to

Index: 3, token: Array

I have used the countTokens method of the StringTokenizer class to get the total number of tokens available in the string. We need to create the String array that can fit that many elements.

Important Note:

Except for educational purposes, it is suggested to use the Java String split method instead of the StringTokenizer class to generate string tokens. The StringTokenizer class is there just for backward compatibility purposes.

If you want to learn more about the tokenizer, please visit the Java StringTokenizer tutorial.

Please let me know your views in the comments section below.