Why do these two methods of counting words differ significantly? - c

Question

I wrote a program that allows a user to find the number of instances of a word or collection of words in any text file. The user can enter something like this in the command line:
$ ./wordCount Mars TripToMars.txt
to search for the number of instances of the word "Mars" in the book Trip To Mars, or
$ ./wordCount -f collectionOfSearchWords.txt TripToMars.txt
to search for the number of instances of several words on individual lines in collectionOfSearchWords.txt.
To ensure that the program was correct, I used the grep commands:
$ grep -o 'Mars' TripToMars.txt | wc -w
and
$ grep -o -w 'Mars' TripToMars.txt | wc -w
The first command finds the number of instances of the word anywhere, which would include terms like "Marsa", "Marseen", "Marses", etc., while the second command finds only instances of "Mars" as a standalone word, which would include trailing punctuation such as "Mars.", "Mars!", "Mars?", etc.
Both grep commands return 49 as the number of instances of "Mars" in the book.
When I use the code in the while loop below (for simplicity, I'm only including the relevant code), the program returns 49. Awesome!
FILE *textToSearch;
char *readMode = "r";
int count;
char nextWord[100];
char d;
textToSearch = fopen(argVector[argCount-1], readMode);
if (textToSearch == NULL) {
fprintf(stderr, "Cannot open %s to be searched\n", argVector[argCount-1]);
return 1;
} else {
while (fscanf(textToSearch, "%*[^a-zA-Z]"), fscanf(textToSearch, "%80[a-zA-Z]", nextWord) > 0) {
// increment the counter if the word is a match
if (strcmp(nextWord, argVector[word]) == 0) {
count++;
}
}
}
But when I substitute this while loop for the previous one, the program returns 17.
while(1) {
d = fscanf(textToSearch, "%s", nextWord);
if (d == EOF) break;
// increment the counter if the word is a match
if (strcmp(nextWord, argVector[word]) == 0) {
count++;
}
}
So, what's the big difference between
while (fscanf(textToSearch, "%*[^a-zA-Z]"), fscanf(textToSearch, "%80[a-zA-Z]", nextWord) > 0) {}
and
while(1) {
d = fscanf(textToSearch, "%s", nextWord);
if (d == EOF) break;
}
?
EDIT:
I added this code:
if (strcmp(nextWordDict, nextWord) == 0 ||
strcmp(nextWordDict, strcat(nextWord, ".")) == 0 ||
strcmp(nextWordDict, strcat(nextWord, "?")) == 0 ||
strcmp(nextWordDict, strcat(nextWord, "!")) == 0 ||
strcmp(nextWordDict, strcat(nextWord, ",")) == 0) {
count++;
}
to the code producing 17 for Mars to try and account for cases with trailing punctuation, and there was no change. Still 17.
EDIT2:
As John Bollinger correctly points out below, this code does nothing because the string buffered into nextWord would already have the trailing punctuation, and the code would only be adding more. This is faulty thinking on my part.

Answer 1

You are incorrect when you say that the command ...
$ grep -o -w 'Mars' TripToMars.txt | wc -w
... "finds only instances of 'Mars' as a standalone word", or at least that statement is misleading in context. The command finds instances of "Mars" that are not part of a larger word, where a "word" is defined as a contiguous string of letters, digits, and/or underscores. In particular, it will match "Mars" where it is followed by a punctuation mark, which conflicts with what you seem to be claiming.
But what is the difference between your two scanning approaches? Well, this ...
while (fscanf(textToSearch, "%*[^a-zA-Z]"),
fscanf(textToSearch, "%80[a-zA-Z]", nextWord) > 0) { /* ... */ }
... scans zero or more characters that are not Latin letters, ignoring whether any are matched and whether any input error occurs, then scans a contiguous sequence of up to 80 Latin letters, recording that sequence in the nextWord buffer.
On the other hand, this ...
while(1) {
d = fscanf(textToSearch, "%s", nextWord);
if (d == EOF) break;
}
... ignores leading whitespace, then scans the next contiguous string of non-whitespace into nextWord.
The two differ significantly in what they do with characters that are neither Latin letters nor whitespace: the former ignores them, whereas the latter includes them in nextWord. When you then compare nextWord with the string "Mars", the latter misses on
Going to Mars.
and
The name "Mars"
and
Is there water on Mars?
because the adjacent punctuation is included in the comparison. Your text is quite likely to have many constructions similar to those, and your grep commands do not demonstrate otherwise.