Strtok - reading empty string at end of line - c

Question

In my code below I use strtok to parse a line of code from a file that looks like:
1023.89,863.19 1001.05,861.94 996.44,945.67 1019.28,946.92 1023.89,863.19
As the file can have lines of different lengths I don't use fscanf. The code below works of except for one small glitch. It loops around one time too many and reads in a long empty string " " before looping again recognizing the null token "" and exiting the while loop. I don't know why this could be.
Any help would be greatly appreciated.
fgets(line, sizeof(line), some_file);
while ((line != OPC_NIL) {
token = strtok(line, "\t"); //Pull the string apart into tokens using the commas
input = op_prg_list_create();
while (token != NULL) {
test_token = strdup(token);
if (op_prg_list_size(input) == 0)
op_prg_list_insert(input,test_token,OPC_LISTPOS_HEAD);
else
op_prg_list_insert(input,test_token,OPC_LISTPOS_TAIL);
token = strtok (NULL, "\t");
}
fgets(line, sizeof(line), some_file);
}

Answer 1

You must use the correct list of delimiters. Your code contradicts comments:
token = strtok(line, "\t"); //Pull the string apart into tokens using the commas
If you want to separate tokens by commas, use "," instead of "\t". In addition, you certainly don't want the tokens to contain the newline character \n (which appears at the end of each line read from file by fgets). So add the newline character to the list of delimiters:
token = strtok(line, ",\n"); //Pull the string apart into tokens using the commas
...
token = strtok (NULL, ",\n");
You might want to add the space character to the list of delimiters too (is 863.19 1001.05 a single token or two tokens? Do you want to remove spaces at end of line?).

Answer 2

Your use of sizeof(line) tells me that line is a fixed size array living on the stack. In this case, (line != OPC_NIL) will never be false. However, fgets() will return NULL when the end of file is reached or some other error occurs. Your outer while loop should be rewritten as:
while(fgets(line, sizeof(line), some_file)) {
...
}
Your input file likely also has a newline character at the end of the last input line resulting in a single blank line at the end. This is the difference between this:
1023.89,863.19 1001.05,861.94 996.44,945.67 1019.28,946.92 1023.89,863.19↵
<blank line>
and this:
1023.89,863.19 1001.05,861.94 996.44,945.67 1019.28,946.92 1023.89,863.19
The first thing you should do in the while loop is check that the string is actually in the format you expect. If it's not then break:
while(fgets(line, sizeof(line), some_file)) {
if(strlen(line) == 0) // or other checks such as "contains tab characters"
break;
...
}