Loop seems to terminate before reaching the null character in C - c

Question

Below is a section of a Tokenizer I built. The user types a string they wish to tokenize, that string is stored into a char array, and a null character ('\0') is placed as soon as the string ends. That section of the code seems to work fine after having tested it a few times.
The problem I'm getting occurs later on in the code when I make an array (tokenArray) of arrays (newToken). I use functions to get number of tokens and token length.
I entered the string "testing pencil calculator." I then store each token into an array. The problem is when I go to print the contents of the array, the loop that I have printing stops before it should.
Here's a sample input/output. My comments (not in code) noted by
$testing pencil calculator //string entered
complete index: 0 //index of the entire array, not the tokenized array
token length: 7 //length of 1st token "testing"
pointer: 0xbf953860
tokenIndex: 0 //index of the token array (array of arrays)
while loop iterations: 4 //number of times the while loop where i print is iterated. should be 7
test //the results of printing the first token
complete index: 8
token length: 6 //next token is "pencil"
tokenIndex: 1
while loop iterations: 5 //should be 6
penci //stops printing at penci
complete index: 15
token length: 10 //final token is "calculator"
pointer: 0xbf953862
tokenIndex: 2
while loop iterations: 5 //should be 10
calcu //stops printing at calcu
for the life of me, I simply cannot figure out why the while loop is exiting before it is supposed to. I doubt this is the only problem with my methodology, but until I can figure this out, I can't address other bugs.
Below is a section of my code that is responsible for this:
from main:
completeString[inputsize] = '\0';
char tokenArray[numTokens+1];
tokenArray[numTokens] = '\0';
putTokensInArray(tokenArray, completeString);
method where I'm getting errors:
char ** putTokensInArray(char tokenArray[], char * completeString){
int completeIndex = 0;
int tokenIndex = 0;
while(tokenArray[tokenIndex] != '\0'){
int tokenLength = tokenSize(completeString, completeIndex);
char newToken [tokenLength+1];
newToken[tokenLength] = '\0';
tokenArray[tokenIndex] = *newToken;
printf("\ncomplete index: %d", completeIndex);
printf("\ntoken length: %d", tokenLength);
printf("\ntokenIndex: %d\n", tokenIndex);
int i = 0;
while(newToken[i] != '\0'){
newToken[i] = completeString[i + completeIndex];
i++;
}
completeIndex += (tokenLength+1);
printf("while loop iterations: %d\n", i);
for(int j = 0; newToken[j] != '\0'; j++){
printf("%c", newToken[j]);
}
tokenIndex++;
tokenLength = 0;
}//big while loop
}//putTokensInArray Method
I have tried several things but just cannot get the grasp of it. I'm new to C, so it's entirely possible I'm making pointer mistakes or accessing memory I shouldn't be; on that note, how would I implement a malloc() and free()? I've been doing reading on that and seems to work, but I'm unable to implement those functions.

Answer 1

You are passing an uninitialized character array to your function putTokensInArray. Later in that function, in the while loop condition you are checking for \0 for every element starting from 0. However since the array is uninitialized, those characters could be ANY characters. There could be a \0 before the numTokens+1th element.
To fix this problem, pass the length of the character array i.e. numTokens to the putTokensInArray as an additional argument. Then in your while loop, do the following condition check instead:
while(tokenIndex < numTokens){