How to fix 'strtok destroy original string' in c - c

Question

I'm writing a compiler for assembler, and I need to do parsing to the text I get from a file, without making any changes in the original String. The function I used to copy the String was strcpyto a buffer, and to cut the String was strtok to cut the buffer.
Everything works perfect, but once I try to cut the original String after using the function addressingConstantIndex , I get null.
I tried to change the type of the buffer to a pointer of Character, it didn't really work. I guess the main problem it's in the way I copy the original String to the buffer.
int main(){
char desti[MAXCHAR];
char *temp;
char *token;
temp = "mov LIST[5] , r4";
strcpy(desti,temp);
printf("\ndest is : %s\n", desti);
token = strtok(desti," ");
printf("\nthe Token in Main is : %s \n", token);
token = strtok(NULL, ",");
printf("\nthe Token in Main is : %s\n", token);
printf("\nThe value is %d \n ",addressingConstantIndex(token));
token = strtok(NULL, " ,");
printf("\nthe Token in Main is : %s\n", token);
return 0;
}
int addressingConstantIndex(char * str) {
char buf[43];
char *token;
int ans;
strcpy(buf, str);
token = strtok(buf, "[");
printf("The string is %s\n",str);
if (token != NULL)
{
printf("the token is %s\n", token);
token = strtok(NULL, "]");
printf("the token is %s\n", token);
if(isOnlyNumber(token))
{
token = strtok(NULL, " ");
if (checkIfSpaces(token,0) == ERROR)
{
printf("ERROR: Extra characters after last bracket %s \n", str);
ans = ERROR;
} else
ans = OK;
} else {
printf("ERROR: Unknown string - %s - its not a macro & not a number.\n", token);
ans = ERROR;
}
} else {
printf("ERROR: %s , its not a LABEL", token);
ans = ERROR;
}
return ans;
}
int isOnlyNumber(char *str) {
int i, isNumber;
i = 0;
isNumber = 1;
if (!isdigit(str[i]) && !(str[i] == '-' || str[i] == '+'))
{
isNumber = ERROR;
}
i++;
while (i < strlen(str) && isNumber == 1)
{
if (!(isdigit(str[i]))) {
if (isspace(str[i]))
isNumber = checkIfSpaces(str, i);
else
isNumber = ERROR;
}
i++;
}
return isNumber;
}
int checkIfSpaces(char *str, int index) {
int i;
if (str == NULL)
{
return OK;
} else {
for (i = index; i < strlen(str); i++)
{
if (!isspace(str[i])) return ERROR;
}
}
return OK;
}
The expecting result:
dest is : mov LIST[5] , r4
the Token in Main is : mov
the Token in Main is : LIST[5]
The string is LIST[5]
the token is LIST
the token is 5
The value is 1
the Token in Main is : r4
The real result:
dest is : mov LIST[5] , r4
the Token in Main is : mov
the Token in Main is : LIST[5]
The string is LIST[5]
the token is LIST
the token is 5
The value is 1
the Token in Main is : (null)
The difference It's in the last row of the result.

Answer 1

The problem is that strtok() maintains a single static pointer to the current string location. So in addressingConstantIndex(), you start processing the local buf, so that when you return to main() it is no longer parsing desti but the now out of scope buf from addressingConstantIndex().
The simplest change to your existing code would be to use strtok_r() (or strtok_s() on Windows):
char* context = 0 ;
token = strtok_r( desti, " ", &context ) ;
printf("\nthe Token in Main is : %s \n", token);
token = strtok_r( NULL, ",", &context ) ;
...
Then similarly in addressingConstantIndex():
char* context = 0 ;
token = strtok_r(buf, "[", &context);
...

Answer 2

strtok replaces the separator in the string with a NULL on each call. Your code is finding the 'LIST[5]' token at the main level, at which point it has replaced the ',' with a NULL.
In addressingConstantIndex, strtok is reset with a new string and parses correctly (though your function is typed void instead of int).
At at the main level again, strtok is not being reset, so it is continuing to parse the string used in addressingConstantIndex.
To fix this you need to reset strtok again to continue. However you can't just call it with strtok(desti,",") as the desti has all the separators set to NULL from previous calls.
A quick solution is to copy the token to feed into addressingConstantIndex at the main level, and complete the main level before parsing at the next level.
int main(){
char desti[MAXCHAR];
char *temp;
char *token;
temp = "mov LIST[5] , r4";
strcpy(desti,temp);
printf("\ndest is : %s\n", desti);
token = strtok(desti," ");
printf("\nMnemonic : %s \n", token);
token = strtok(NULL, ",");
printf("\nLIst bit: %s\n", token);
char buf[80]; //Save the token
strcpy(buf, token);
token = strtok(NULL, " ,"); //Finish this level of processing
printf("\nRegister: %s\n", token);
//Continue at the next level with copy
printf("\nThe value is %d \n ",addressingConstantIndex(buf));
return 0;
}
Though a strtok_r solution might suit your needs better going forward