Null termination of string not working it seems - c

In these two versions:
//VERSION 1
char *c=malloc(10);
c[0]='h';
c[1]='i';
c[2]='\0';
c[3]='l';
printf("%s\n",c);
I am getting the expected result i.e. hi is being printed.
Now in this one:
//VERSION 2
char *c;
size_t siz=8;
c=malloc(sizeof(char)*(siz+1)); //char size is 1 byte on system
getline(&c,&siz,stdin);
c[siz]='\0';
printf("%s\n",c);
On inputting the value 'hello world' the output is 'hello world' - I was expecting that it won't print anything after reading 9th byte (it is set to \0).
Why is there difference in the two?
Is it happening because pointer c in version 2 is made to point to stdin and `\0' modification doesn't work that way in a stream? If yes then why is compiler now issuing any warning or error?

As you yourself noted in the comments, getline will check the pointer and size arguments to see if it needs to reallocate (or allocate) the buffer in the event the line from the stream exceeds the given buffer's size (a NULL buffer of size 0 being a plain allocation instead of a reallocation). When this happens, both the pointer and the size arguments are changed to match the new buffer (remember, you passed in pointers to the buffer pointer and size arguments, not just the arguments themselves, references not values).
So, in your example, after allocating a buffer of size 9 chars (9 bytes in your case); your c pointer is set to some memory with at least 9 bytes available and siz is still 8. However, after typing a line longer than 8 characters (including the new line) like "hello world\n", the buffer is reallocated to fit the whole string "hello world\n\0", ie 13 bytes, AND the size arguments is changed to 13. So, when getline returns, c points to this new buffer and siz is 13. You don't need to add a null termination since getline does it for you (assuming it succeeds). What you are doing is then setting c[13] to '\0' which luckily for you didn't trigger any exceptions as you are accessing past the end of the buffer (making the string "hello world\n\0\0").
For the results you're looking for, keep the original size aside, like in a macro:
#define SIZE 8
char* c;
size_t siz = SIZE;
c = malloc(sizeof(char) * (siz +1));
getline(&c, &siz, stdin); // if you type something longer than 8 bytes including new line, it will trigger the realloc and siz will be changed
c[SIZE] = '\0'; // prematurely end the string at 8 bytes
printed("%s\n", c); // now you'll get shorter strings, noting siz will still keep the full length for you

Related

How does realloc treat null bytes in strings?

Relatively new C programmer here. I am reviewing the following code for a tutorial for a side project I am working on to practice C. The point of the abuf struct is to create a string that can be appended to. Here is the code:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
typedef struct abuf {
char* str;
unsigned int size;
} abuf;
void abAppend(abuf *ab, const char *s, int len) {
char *new = realloc(ab->str, ab->size + len);
if (new == NULL) return;
memcpy(&new[ab->size], s, len);
ab->str = new;
ab->size += len;
}
int main(void) {
abuf ab = {
NULL,
0
};
char *s = "Hello";
abAppend(&ab, s, 5);
abAppend(&ab, ", world", 7);
return 0;
}
Everything compiles and my tests (redacted for simplicity) show that the string "Hello" is stored in ab's str pointer, and then "Hello, world" after the second call to abAppend. However, something about this code confuses me. On the initial call to abAppend, the str pointer is null, so realloc, according to its man page, should behave like malloc and allocate 5 bytes of space to store the string. But the string "Hello" also contains the terminating null byte, \0. This should be the sixth and final byte of the string, if I understand this correctly. Isn't this null byte lost if we store "Hello\0" in a malloc-ed container large enough only to store "Hello"?
On the second call to abAppend, we concatenate ", world" to str. The realloc will enlarge str to 12 bytes, but the 13th byte, \0, is not accounted for. And yet, everything works, and if I test for the null byte with a loop like for (int i = 0; ab.str[i] != '\0'; i++), the loop works fine and increments i 12 times (0 thru 11), and stops, meaning it encountered the null byte on the 13th iteration. What I don't get is why does it encounter the null byte, if we don't allocate space for it?
I tried to break this code by doing weird combinations of strings, to no avail. I also tried to allocate an extra byte in each call to abAppend and changed the function a little to account for the extra space, and it performed the exact same as this version. How the null byte gets processed is eluding me.
How does realloc treat null bytes in strings?
The behavior of realloc is not affected by the contents of the memory it manages.
But the string "Hello" also contains the terminating null byte, \0. This should be the sixth and final byte of the string,…
The characters are copied with memcpy(&new[ab->size], s, len);, where len is 5. memcpy copies characters without regard to whether there is a terminating null byte. Given length of 5, it copies 5 bytes. It does not append a terminating null character to those.
The realloc will enlarge str to 12 bytes, but the 13th byte, \0, is not accounted for.
On the second called to abAppend, 7 more bytes are copied with memcpy, after the first 5 bytes. memcpy is given a length of 7 and copies only 7 bytes.
… it encountered the null byte on the 13th iteration.
When you tested ab.str[12], you exceeded the rules for which the C standard defines the behavior. ab.str[12] is outside the allocated memory. It is possible it contained a null byte solely because nothing else in your process had used that memory for another purpose, and that is why your loop stopped. If you attempted this in the middle of a larger program that had done previous work, that byte might have contained a different value, and your test might have gone awry in a variety of ways.
You're correct that you only initially allocated space for the characters in the string "Hello" but not the terminating null byte, and that the second call only added enough bytes for the characters in tge string ", world" with no null terminating byte.
So what you have is an array of characters but not a string since it's not null terminated. If you then attempt to read past the allocated bytes, you trigger undefined behavior, and one of the ways UB can manifest itself is that things appear to work properly.
So you got "lucky" that things happened to work as if you allocated space for the null byte and set it.

Can `snprintf()` read out of bounds when a string argument is not null-terminated?

I have the following piece of code, which a colleague claims may contain an out-of-bounds read, which I do not agree with. Could you help settle this argument and explain why?
char *test_filename = malloc(Size + 1);
sprintf(test_filename, "");
if (Size > 0 && Data)
snprintf(test_filename, Size + 1, "%s", Data);
where Data is a non-null-terminated string of type const uint8_t *Data and Size is the size of Data, i.e., number of bytes in Data, of type size_t.
It may read out-of-bounds because the format string is %s, perhaps?
Your colleague is correct. Perhaps unintuitively, snprintf(test_filename, Size + 1, "%s", Data) is guaranteed to read bytes starting at Data until a 0 byte is encountered, in your case typically resulting in an out-of-bounds read.
It will only write Size of these bytes to test_filename and null terminate them, respecting the size limit of the destination; but it will continue to read on. The reason for that is a design choice which enables the caller to determine the needed destination size for dynamic allocation before anything is actually written: snprintf() returns the number of bytes which would be written if the destination had infinite space. This feature is supposed to be used with a destination size of 0 (and potentially a null pointer as the destination). This functionality is useful for arguments which are not strings: With numbers etc. the size of the output is difficult to predict (e.g. locale dependent) and best left to the function at run time.
At the same time the return value indicates whether the output was truncated: If it is greater or equal to the size parameter, not all of the input was used in the output. In your case, what was left out were the bytes starting a Data[Size] and ending with the first 0 byte, or a segmentation fault ;-).
Suggestion for a fix: First of all it is unclear why you would use the printf family to print a string; simply copy it. And then Andrew has a point in his comments that since Datais not null terminated it is not really a string (even if all bytes are printable); so don't start fiddling with strcpy and friends but simply memcpy() the bytes, and null terminate manually.
Oh, and the preceding sprintf(test_filename, ""); does not serve any discernible purpose. If you want to write a null byte to *Data, simply do so; but since you are not using strcat, which would rely on a terminated destination string to extend, it is quite unnecessary.
from the MAN page for snprintf()
The functions snprintf() and vsnprintf() write at most size bytes (including the terminating null byte ('\0')) to str.
Note the at most size bytes
This means that snprintf() will stop transferring bytes after the parameter Size bytes are transferred.
this statement;
sprintf(test_filename, "");
is completely unneeded and has no effect on the operation of the second call to snprintf()
If you want to result in a 'proper' string, suggest:
char *test_filename = calloc( sizeof( char ), Size + 1);
if (Size > 0 && Data)
snprintf(test_filename, Size, "%s", Data);
however, the function: snprintf() keeps reading until a NUL byte is encountered. This can create problems, upto and including a seg fault event.
The function: memcpy() is made for this kind of job. Suggest replacing the call to snprintf() with
memcpy( test_filename, Data, Size );

What happens when I write `char str[80];`?

What happens behind the scenes when I write: char str[80];?
I notice that I can now set str = "hello"; and also str = "hello world"; right afterwards. First time strlen(str) is 5, and second time it is 11;
But why? I thought that after str = "hello";, the char at index 5 becomes null (str[5] becomes '\0'). Doesn't that mean that str's size is now 6 and I shouldn't be able to set it to "hello world"?
And if not, then how does strlen and sizeof calculate the correct values every time?
I think you're getting confused between two different concepts: the allocated length of the array (how much total space is available), and the logical length of the string (how much space is being used).
When you write char str[80], you're getting storage space for 80 characters. You might not end up using all of that space, but regardless of what string you try storing in it, you're always going to have 80 slots into which you can place characters.
If you store the string "hello" into str, then the first six characters of str will be set to h, e, l, l, o, and a null terminating character. This doesn't change the allocated length, though - you still have 74 other slots that you can work with. If you then change it to "hello, world", you're using an extra seven characters, which fits just fine because you easily have enough allocated space to hold things. You've just changed the logical length, how much of that space is being used for meaningful data, but not the allocated length, how much space there is available.
Think of it this way. When you say char str[80], you're buying a plot of land that's, say, 80 acres. If you then put "hello" into it, you're using six acres of that available 80 acres. The rest of the land is still yours - you can build whatever you'd like there - so if you decide to tear everything down and build a longer string that uses up more acres of land, that's fine. No one is going to object.
The strlen function gives back the logical length of the string - how many characters are in the string that you're storing. It works by counting up characters until it finds a null terminator indicating the logical end of the string. The sizeof operator returns the allocated length of the array, how many slots you have. It works at compile-time and doesn't care what the array contents are.
When you declare a variable as char str[80], space for an 80 character array is allocated on the stack. This memory will be automatically released when that particular stack frame is out of scope.
When you assign it to the string literal "hello", it is copying each character into the array, then putting a null terminator at the end of the string (str[5] == '\0'). String length and array size are two different things, which is why you can reassign it to "hello world". String length is simply how many consecutive characters there are before the null terminator. If you instead declared str as char str[5], you would indeed cause a crash when you tried to reassign it to "hello world". It may be helpful to view a simple implementation of strlen:
size_t strlen(const char *str)
{
size_t return_val = 0;
while (str[return_val] != '\0') return_val++;
return return_val;
}
Of course, if there is no null terminating character, the above naive implementation will crash.
I am assuming that you are working in C. When you compile "char str[80];" basically a 80 character long space is allocated for you. sizeof(str) should always tell you that it is an 80 byte long chunk of memory. strlen(str) will count the non-zero characters starting at str[0]. This is why "Hello" is 5 and "Hello world".
I would suggest that you learn to use functions like strnlen, strncpy, strncmp, snprintf ..., this way you can prevent reading/writing beyond the end the array, for example: strnlen(str,sizeof(str)).
Also start working through online tutorials and find an introductory C/C++ book to learn from.
When you declare an array like char str[80]; 80 chars of space are reserved on the stack for you, but they are not initialized - they get whatever was already in memory at the time. It's your job as the programmer to initialize the array.
strlen does something along these lines:
int strlen(char *s)
{
int len = 0;
while(*s++) len++;
return len;
}
In other words, it returns the length of a null-terminated string in a character array, even if the length is less than the size of the total array.
sizeof returns the size of a type or expression. If your array is 80 chars long, and a char is a byte long, it will return 80, even if none of the values in the array have been initialized. If you had an array of 5 ints, and an int was 4 bytes long, sizeof would produce 20.

Weird behavior of printf() calls after usage of itoa() function

I am brushing up my C skills.I tried the following code for learning the usage of itoa() function:
#include<stdio.h>
#include<stdlib.h>
void main(){
int x = 9;
char str[] = "ankush";
char c[] = "";
printf("%s printed on line %d\n",str,__LINE__);
itoa(x,c,10);
printf(c);
printf("\n %s \n",str); //this statement is printing nothing
printf("the current line is %d",__LINE__);
}
and i got the following output:
ankush printed on line 10
9
//here nothing is printed
the current line is 14
The thing is that if i comment the statement itoa(x,c,10); from the code i get the above mentioned statement printed and got the following output:
ankush printed on 10 line
ankush //so i got it printed
the current line is 14
Is this a behavior of itoa() or i am doing something wrong.
Regards.
As folks pointed out in the comments, the size of the array represented by the variable c is 1. Since C requires strings have a NULL terminator, you can only store a string of length 0 in c. However, when you call itoa, it has no idea that the buffer you're handing it is only 1 character long, so it will happily keep writing out digits into memory after c (which is likely to be memory that contains str).
To fix this, declare c to be of a size large enough to handle the string you plan to put into it, plus 1 for the NULL terminator. The largest value a 32-bit int can hold is 10 digits long, so you can use char c[11].
To further explain the memory overwriting situation above, let's consider that c and str are allocated in contiguous regions on the stack (since they are local variables). So c might occupy memory address 1000 (because it is a zero character string plus a NULL terminator), and str would occupy memory address 1001 through 1008 (because it has 6 characters, plus the NULL terminator). When you try to write the string "9" into c, the digit 9 is put into memory address 1000 and the NULL terminator is put in memory address 1001. Since 1001 is the first address of str, str now represents a zero-length string (NULL terminator before any other characters). That's why you are getting the blank.
c must be a buffer long enough to hold your number.
Write
char c[20] ;
instead of
char c[] = "";

Null character in fread() and strncpy() in c

I am new programmer in general and I have start working now with c. I am trying to decode the IDEv3 mp3 tag and I came across with a variety of problems. While I was using the fread() and strncpy() commands I have noticed that both need to have the \n character as the end reference point. (Maybe I am wrong this is only an observation)
When I am printing the output they produce a non readable character. As a solution to overcome the problem I am using fread() for 4 Bytes instead of 3 in order to produce (8)\n characters (whole Byte), and a second step I am using strncpy() with 3 Bytes to an allocated memory which then I am using for printing. In theory when I am using fread() I should not encounter this problem.
A sample of code:
#include <stdio.h>
#include <stdlib.h>
typedef struct{
unsigned char header_id[3]; /* Unsigned character 3 Bytes (24 bits) */
}mp3_Header;
int main (int argc, char *argv[]) {
mp3_Header first;
unsigned char memory[4];
FILE *file = fopen( name.mp3 , "rb" );
if ( (size_t) fread( (void *) memory , (size_t) 4 , (size_t) 1 , (FILE *) file) !=1 ) {
printf("Could not read the file\n");
exit (0);
} /* End of if condition */
strncpy( (char *) first.header_id , (char *) memory , (size_t) 3);
printf ("This is the header_ID: %s\n", first.header_id);
fclose(file);
} /* End of main */
return 0;
Your observation with '\n' terminating strings isn't correct. Strings, in C, need to be terminated by a 0 byte (\0). However, some functions like fgets(), which are supposed to read lines from a file, take the \n at the end of the line as a terminator.
The problem with your code is that fread() ready binary data, and doesn't try to interpret that data as a string, which means it won't put the \0 at the end. But string functions, like strcpy, need this 0 byte to recognize the end of the string. strncpy stops after copying the \0 as well, but it won't ever put more bytes into the receiving string to prevent a buffer overflow. So it will copy your 3 bytes, but it won't put a \0 to the end of the string, as it would do if the string was shorter than the length argument.
So what you should do is declare header_id with one MORE element that what you actually need, and after the strcpy, set this extra element to \0. Like this:
strncpy( first.header_id , memory , 3);
first.header_id[3] = '\0';
Remember the 3 header bytes will go to array elements 0..2, so element 3 needs the terminator. Of course, you need to declare header_id[4] to have space for the extra \0.
Also note i omitted the type casts - you don't need them if your types are correct anyway. Passing an array to a function will pass a pointer to the 1st element anyway, so there's no need to cast the array header_id to a pointer in strncpy( (char *) first.header_id , (char *) memory , (size_t) 3);.
There are 2 correct ways of handling the header. I'm assuming the MP3 file has a IDV3 tag, so the file starts with "TAG" or "TAG+". So the part you want to read has 4 bytes.
a) You think of char *memory being a C "string", and first.header_id as well. Then do it this way (omitted everything else to show the important parts):
typedef struct{
unsigned char header_id[5];
} mp3_Header;
char memory[5];
fread(memory, 4, 1, file);
memory[4]='\0';
strncpy(first.header_id, memory, 5)
After the fread, your memory looks like this:
0 1 2 3 4
+----+----+----+----+----+
| T | A | G | + | ? |
+----+----+----+----+----+
The 5th byte, at index 4, is not defined, because you read only 4 bytes. If you use a string function on this string (for example printf("%s\n", memory)); the function doesn't know where to stop, because there is no terminating \0, and printf will continue to output garbage until the next \0 it finds somewhere in your computer's RAM. That's why you do memory[4]='\0' next so it looks like this:
0 1 2 3 4
+----+----+----+----+----+
| T | A | G | + | \0 |
+----+----+----+----+----+
Now, you can use strncpy to copy these 5 bytes to first.header_id. Note you need to copy 5 bytes, not just 4, you want the \0 copied as well.
(In this case, you could use strcpy (without n) as well - it stops at the first \0 it encounters. But these days, to prevent buffer overflows, people seem to agree on not using strcpy at all; instead, always use strncpy and explicitly state the length of the receiving string).
b) You treat memory as binary data, copy the binary data to the header, and then turn the binary data into a string:
typedef struct{
unsigned char header_id[5];
} mp3_Header;
char memory[4];
fread(memory, 4, 1, file);
memcpy(first.header_id, memory, 4)
first.header_id[4]='\0';
In this case, there is never a \0 at the end of memory. So it's sufficient to use a 4-byte-array now. In this case (copying binary data), you don't use strcpy, you use memcpy instead. This copies just the 4 bytes. But now, first.header_id has no end marker, so you have to assign it explicitly. Try drawing images like i did above if it isn't 100% clear to you.
But always remember: if you use operators like '+', you do NOT work upon the string. You work on the single characters. The only way, in C, to work on a string as a whole, is using the str* functions.
Yes, C strings always end in the null (0x00) character. It's the programmer's responsibility to understand that, and code appropriately.
For example, if your header_id will be up to a 3-printable-character string, you need to allocate 4 characters in that array to allow for the trailing null. (And you need to make sure that null is actually present.) Otherwise, printf won't know when to stop, and will keep printing until it finds a 0 byte.
When you copy binary data between buffers you should use appropriate function for the job, like memcpy(). Because you are dealing with binary data you must know exactly the length of the buffer as there is no null characters to indicate the end of data.
To make it a string simply allocate length+1 buffer and set the last byte to '\0' and voila, you have a string. However.. it is possible that there is already a null character in the binary data you copied so you should do some sanity checks before trusting it to really be a string you wanted. Something like \001 might be invalid id for mp3 format.. but it might be a broken file, you never know what you are dealing with.

Resources