strtok function and multithreading - c

Question

I read a lot of stuff about strtok(char* s1, char* s2) and its implementation. However, I still can not understand what makes it a dangerous function to use in multi-threaded program. Can somebody please give me an example of a multi-threaded program and explain the issue there? Please not that I am looking for an example that shows me where the problem arises.
ps: strtok(char* s1, char* s2) is part of the C standard library.

Answer 1

In the first call to strtok, you supply the string and the delimiters. In subsequent calls, the first parameter is NULL, and you just supply the delimiters. strtok remembers the string that you passed in.
In a multithreaded environment, this is dangerous because many threads may be calling strtok with different strings. It will only remember the last one and return the wrong result.

Answer 2

Here is a concrete example:
Suppose first that your program is multi-threaded, and in one thread of execution, the following code runs:
char str1[] = "split.me.up";
// call this line A
char *word1 = strtok(str1, "."); // returns "split", sets str1[5] = '\0'
// ...
// call this line B
char *word2 = strtok(NULL, "."); // we hope to get back "me"
And in another thread, the following code runs:
char str2[] = "multi;token;string";
// call this line C
char *token1 = strtok(str2, ";"); // returns "multi", sets str2[5] = '\0'
// ...
// call this line D
char *token2 = strtok(NULL, ";"); // we hope to get back "token"
The point is, we don't really know what will be in word2 and token2:
If the commands are run in the order (A), (B), (C), (D), then we will get what we want.
But if, say, the commands run in the order (A), (C), (B), (D), then command (B) will search for a . delimeter in "token;string"! This is because the NULL first argument to command (B) tells strtok to continue searching in the last non-NULL search string it was passed, and because command (C) has already run, strtok will use str2.
Then command (B) will return token;string, at the same time setting the new starting character of a search to the NUL terminator at the end of str2. Then the command (D) will think it is searching an empty string, because it will begin its search at str2's NUL terminator, and so will return NULL as well.
Even if you place commands (A) and (B) right next to each other, and commands (C) and (D) right next to each other, there is no guarantee that (B) will be executed right after (A) before either (C) or (D), etc.
If you create some sort of mutex or alternate guard to protect the use of the strtok function, and only call strtok from a thread which has obtained a lock on said mutex, then strtok is safe to use. However, it is probably better just to use the thread-safe strtok_r as others have said.
Edit: There is one more issue, that nobody else has mentioned, namely that strtok modifies and potentially uses global (or static, whatever) variables, and does so in a probably-not-thread-safe way, so even if you don't rely on repeating calls to strtok to get successive "tokens" from the same string, it may not be safe to use it in a multi-threaded environment without guards, etc.

Answer 3

To explain in simple terms, Whenever they name it THREAD safe, they literally mean, it is not just your thread, other thread too can modify it! It is like a cake been shared with 5 friends concurrently. The results are unpredictable who consumed the cake, or who altered it!
Every call to the strtok() function, returns a refrence to a NULL terminated string and it uses a static buffer during parsing. Any subsequent call to the function will refer to that buffer only, and it gets altered.! It is independent of who called it, and thats is the reason for it is not thread safe.
Other hand strtok_r() using a additional 3rd argument called saveptr(we need to specify it) which is probably used to hold that reference for subsequent calls. Thus is no more system specific but in developer control.
An example:( from a book of Steven robbins, unix system programming)
An incorrect use of strtok to determine the average number of words per line.
#include <string.h>
#define LINE_DELIMITERS "\n"
#define WORD_DELIMITERS " "
static int wordcount(char *s) {
int count = 1;
if (strtok(s, WORD_DELIMITERS) == NULL)
return 0;
while (strtok(NULL, WORD_DELIMITERS) != NULL)
count++;
return count;
}
double wordaverage(char *s) { /* return average size of words in s */
int linecount = 1;
char *nextline;
int words;
nextline = strtok(s, LINE_DELIMITERS);
if (nextline == NULL)
return 0.0;
words = wordcount(nextline);
while ((nextline = strtok(NULL, LINE_DELIMITERS)) != NULL) {
words += wordcount(nextline);
linecount++;
}
return (double)words/linecount;
}
The wordaverage function determines the average number of words per line by using strtok to find the next line. The function then calls wordcount to count the number of words on this line. Unfortunately, wordcount also uses strtok, this time to parse the words on the line. Each of these functions by itself would be correct if the other one did not call strtok. The wordaverage function works correctly for the first line, but when wordaverage calls strtok to parse the second line, the internal state information kept by strtok has been reset by wordcount.