strtok() — Tokenize string

Standards

Standards / Extensions	C or C++	Dependencies
ISO C XPG4 XPG4.2 C99 Single UNIX Specification, Version 3	both

Format

#include <string.h>

char *strtok(char * __restrict__string1, const char * __restrict__string2);

General description

Breaks a character string, pointed to by string1, into a sequence of tokens. The tokens are separated from one another by the characters in the string pointed to by string2.

The token starts with the first character not in the string pointed to by string2. If such a character is not found, there are no tokens in the string. strtok() returns a NULL pointer. The token ends with the first character contained in the string pointed to by string2. If such a character is not found, the token ends at the terminating NULL character. Subsequent calls to strtok() will return the NULL pointer. If such a character is found, then it is overwritten by a NULL character, which terminates the token.

If the next call to strtok() specifies a NULL pointer for string1, the tokenization resumes at the first character following the found and overwritten character from the previous call. For example:

/* Here are two calls  */
strtok(string," ")
strtok(NULL," ")


/* Here is the string they are processing */
                   abc defg hij
    first call finds  ↑
                       ↑ second call starts

Returned value

The first time strtok() is called, it returns a pointer to the first token in string1. In later calls with the same token string, strtok() returns a pointer to the next token in the string. A NULL pointer is returned when there are no more tokens. All tokens are NULL-terminated.

Example

CELEBS54

⁄* CELEBS54
 *
 * strtok() example:
 * 
 * This example parses tokens separated by commas, blanks and semicolons,
 * from a string until no tokens are left.  As the string is parsed,
 * pointers to the the following tokens are returned by strtok(),
 * and these tokens are written to stdout:
 *
 *   a
 *   string
 *   of
 *   tokens
 *
 * The final call to strtok() returns NULL indicating that 
 * there are no more tokens.
 *
 * Note that as the string is tokenized, it will be overwritten.
 *
 *⁄

#include <stdio.h>
#include <string.h>

int main(void)
{
   char *token, string[] = "a string, of,; ;;;,tokens\0,after null terminator";

   token = strtok(string, ", ;");
   do
   {
      printf("token: \"%s\"\n", token);
   }
   while (token = strtok(NULL, ", ;"));

}

Output

token: "a string"
token: " of"
token: " "
token: "tokens"