I am trying to make this dynamic reallocation work in a portable fashion.
My program accepts a line of text from a user and appends it to a buffer. If the length of text in the buffer is 20 or more, it removes the first 20 characters and moves any characters after that to the start of the buffer.
I have this code which works clean on Linux but when I run it on windows it emits garbage. Does anyone know why/how to make this portable only using malloc. IE not using string.h(strcpy) str... anything but len.
c17 only - no broken stucts(not portable). here is my code. Compiles with no errors gcc 7.3, mingw 7.3. I replace gets and puts with safer functions and I still get garbage on windows. I assume this is a formatting issue...
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <malloc.h>
void wbuff (message)
char *message;
{
FILE *f = fopen("file.txt", "w");
fprintf(f, "%s", message);
fclose(f);
}
char *rean (message)
char *message;
{
/* performs (write) on buffer, trims lefover, then restores */
char buf[80] = "";
puts("enter a line");
gets(buf);
int bln = strlen( buf );
int mln = strlen( message );
int nln = bln + mln;
printf("new length %d\n", nln);
message = realloc(message, nln);
memmove(message + mln, buf, bln);
/* MISTAKE IS HERE?! */
if( nln >= 20 ) {
int exl = nln -20; // leftover length
char *lo = realloc(NULL, exl); // leftover placeholder
memmove(lo, message+20, exl); // copy leftover
wbuff(message); // write clear buff
message = realloc(NULL, nln);
message = realloc(NULL, exl); // resize buffer
memmove(message, lo, exl); // restore leftover
}
return message;
}
void main (void)
{
char *message = "";
message = realloc(NULL, 0);
while ( 1 == 1 ) {
message = rean( message );
puts(message);
}
return;
}
In C, strings are a sequence of characters terminated by a null byte. You have a number of off-by-one errors here mostly related to not accounting for that fact, as well as memory leaks.
When you first set message in main:
char *message = "";
message = realloc(NULL, 0);
message either is NULL to points to 0 bytes of memory. When you call then call rean the first time:
int mln = strlen( message );
You're either attempting to dereference a NULL pointer to read past the end of allocated memory. You want to allocate at least 1 byte to start and set that byte to 0 so you have an empty string:
char *message = realloc(NULL, 1);
message[0] = '\0';
Then later when you copy the buffer into the message:
message = realloc(message, nln);
memmove(message + mln, buf, bln);
You don't allocate enough space for the terminating null byte, nor do you copy it, so you don't really have a string. When you then try to print it, either puts or printf reads past the end of the allocated memory. You need to allocate 1 extra byte and copy 1 extra byte:
message = realloc(message, nln + 1); // allocate 1 extra byte for the null terminator
memmove(message + mln, buf, bln + 1); // copy 1 extra byte
There are similar issues when you recopy anything past 20 characters:
int exl = nln -20; // leftover length
char *lo = realloc(NULL, exl); // leftover placeholder
memmove(lo, message+20, exl); // copy leftover
wbuff(message); // write clear buff
message = realloc(NULL, nln);
message = realloc(NULL, exl); // resize buffer
memmove(message, lo, exl); // restore leftover
lines 2-3: You don't allocate space for the terminating null byte for lo nor do you copy it.
line 5: You leak the memory previously held by message in the first realloc by passing assigning to message while using NULL as the first argument
line 6-7: You leak the memory allocated in line 5 by doing the same thing. Also, you again don't allocate space for the null byte, nor do you copy it on the next line.
As before, allocate 1 extra byte for each allocation and move 1 extra byte to account for the null terminator. Also, free lo at the end of the block, remove the extra realloc for message, and pass the prior value of message to realloc so you don't leak memory:
int exl = nln -20;
char *lo = realloc(NULL, exl + 1); // allocate 1 extra byte
memmove(lo, message+20, exl + 1); // copy 1 extra byte
wbuff(message);
// remove extra realloc
message = realloc(message, exl + 1); // pass in old message, allocate 1 extra
memmove(message, lo, exl + 1); // copy 1 extra byte
free(lo); // free leftover
These issues of reading and writing past the end of allocated memory all invoke undefined behavior, which explains why you see different results on different operating systems.
As far as conforming code goes, use fgets intead of gets:
fgets(line, sizeof(line), stdin);
This function will include a newline in line if there's space for it, so be sure to remove it if that's the case.
Also change main to return int, and remove #include <malloc.h> since the malloc family of function is defined to reside in stdlib.h.
Had you used strcpy and strcat instead of memmove, you wouldn't have had to account for copying the null terminating byte as those functions do that for you. You would still however need to account for that when allocating memory. There's also no conflict between strcpy, malloc, and realloc, as they are all part of the standard and work together properly. Using them together is no problem. If they aren't working properly for you then you aren't using them correctly.
After applying my updates, you can replace this:
memmove(message + mln, buf, bln + 1);
With this:
strcat(message, buf);
And replace this:
memmove(lo, message+20, exl + 1); // copy leftover
...
memmove(message, lo, exl + 1); // restore leftover
With this:
strcpy(lo, message+20);
...
strcpy(message, lo);
And it will still work properly and be conforming.
Related
I got a problem when reading SSL response that causes a segmentation fault. I read the response into a buffer, then append it to a malloced string and memory reset it to 0 till the response is fully read, but when I try this in a multi threaded program, after some operations it gives me segmentation fault. When I remove strcat it doesn't give me segmentation fault even if I run it for hours.
Example:
char* response = malloc(10000);
char buffer[10000] = { 0 };
while(SSL_read(ssl,buf,sizeof(buffer)) > 0){
strcat(response,buffer);
memset(buffer,0,sizeof(buffer));
}
Errors
free(): invalid next size (normal)
malloc_consolidate(): invalid chunk
I made sure of freeing both of SSL and CTX and close socket and free the malloced string.
There are a few problems with your code:
You are not dealing with C strings, you are dealing with arbitrary byte sequences. SSL_read() reads bytes, not C strings, and you cannot treat them as strings. What you read cannot be assumed to be NUL-terminated (\0), so you should not use strcat, strlen or other similar functions that operate on strings. Zeroing out the entire buffers just to make sure there is a terminator makes little to no sense, as the terminator could very well be found in the middle of the data.
You are reading data continuously in a loop into a fixed size buffer. Your code will overflow the destination buffer (response) very easily.
Not an error, but there isn't really any need for an intermediate buffer to begin with. You are needlessly copying stuff around two times (one with SSL_read() and one with strcat) when you can read directly into response instead. On top of that, the memset() to clear the contents of buffer also adds a third scan of the data, slowing things down even more.
Again, not an error, but SSL_read() returns int and uses that to return the read size. You are not really using it, but you should, as you need to keep track of how much space is left on the buffer. You would be however much better off using size_t to avoid unwanted problems with signed math and possible overflows. You can use SSL_read_ex() for this purpose.
Here's a snippet of code that does what you want in a more robust way:
#define CHUNK_SIZE 10000
unsigned char *response = NULL;
size_t size = 0;
size_t space_left = 0;
size_t total_read = 0;
size_t n;
while (1) {
// Allocate more memory if needed.
if (space_left < CHUNK_SIZE) {
unsigned char *tmp = realloc(response, size + CHUNK_SIZE);
if (!tmp) {
// Handle realloc error
break;
}
response = tmp;
size += CHUNK_SIZE;
space_left += CHUNK_SIZE;
}
if (SSL_read_ex(ssl, response + total_read, space_left, &n)) {
total_read += n;
space_left -= n;
} else {
// Handle error
break;
}
}
You never initialized response(). The arguments to strcat() have to be null-terminated strings.
You should also subtract 1 from the size of the buffer when calling SSL_read(), to ensure there will always be room for its null terminator.
char* response = malloc(10000);
response[0] = '\0';
char buffer[10000] = { 0 };
while(SSL_read(ssl,buf,sizeof(buffer)-1) > 0){
strcat(response,buffer);
memset(buffer,0,sizeof(buffer));
}
In some network code I need to encode packet structures in a buffer to send(2) over a socket but memcpy'ing the encoded buffers into a bigger buffer seems problematic.
Here's a small code example illustrating what I am doing:
char tbuf[] = "test";
char *buf = malloc(300);
memset(buf, '\0', 300);
int bytes_to_copy = 300;
int bytes_copied = 0;
while (bytes_copied < bytes_to_copy) {
memcpy(buf + bytes_copied, tbuf, sizeof(tbuf));
bytes_copied += sizeof(tbuf);
}
/* free, return */
This should append "test" 60 times into buf but buf ends up only containing one "test". Am I using memcpy wrong?
You are repeatedly copying the null terminator at the end of test. sizeof tbuf is 5.
So all the C standard library functions will ignore all the other concatenants.
The solution: copy one less byte in the memcpy, and be sure to add a null terminator to the final string.
I'm coding a program that takes some files as parameters and prints all lines reversed. The problem is that I get unexpected results:
If I apply it to a file containing the following lines
one
two
three
four
I get the expected result, but if the file contains
september
november
december
It returns
rebmetpes
rebmevons
rebmeceds
And I don't understand why it adds a "s" at the end
Here is my code
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
void reverse(char *word);
int main(int argc, char *argv[], char*envp[]) {
/* No arguments */
if (argc == 1) {
return (0);
}
FILE *fp;
int i;
for (i = 1; i < argc; i++) {
fp = fopen(argv[i],"r"); // read mode
if( fp == NULL )
{
fprintf(stderr, "Error, no file");
}
else
{
char line [2048];
/*read line and reverse it. the function reverse it prints it*/
while ( fgets(line, sizeof line, fp) != NULL )
reverse(line);
}
fclose(fp);
}
return (0);
}
void reverse(char *word)
{
char *aux;
aux = word;
/* Store the length of the word passed as parameter */
int longitud;
longitud = (int) strlen(aux);
/* Allocate memory enough ??? */
char *res = malloc( longitud * sizeof(char) );
int i;
/in this loop i copy the string reversed into a new one
for (i = 0; i < longitud-1; i++)
{
res[i] = word[longitud - 2 - i];
}
fprintf(stdout, "%s\n", res);
free(res);
}
(NOTE: some code has been deleted for clarity but it should compile)
You forget to terminate your string with \0 character. In reversing the string \0 becomes your first character of reversed string. First allocate memory for one more character than you allocated
char *res = malloc( longitud * sizeof(char) + 1);
And the try this
for (i = 0; i < longitud-1; i++)
{
res[i] = word[longitud - 2 - i];
}
res[i] = '\0'; // Terminating string with '\0'
I think I know the problem, and it's a bit of a weird issue.
Strings in C are zero terminated. This means that the string "Hi!" in memory is actually represented as 'H','i','!','\0'. The way strlen etc then know the length of the string is by counting the number of characters, starting from the first character, before the zero terminator. Similarly, when printing a string, fprintf will print all the characters until it hits the zero terminator.
The problem is, your reverse function never bothers to set the zero terminator at the end, which it needs to since you're copying characters into the buffer character by character. This means it runs off the end of your allocated res buffer, and into undefined memory, which just happened to be zero when you hit it (malloc makes no promises of the contents of the buffer you allocate, just that it's big enough). You should get different behaviour on Windows, since I believe that in debug mode, malloc initialises all buffers to 0xcccccccc.
So, what's happening is you copy september, reversed, into res. This works as you see, because it just so happens that there's a zero at the end.
You then free res, then malloc it again. Again, by chance (and because of some smartness in malloc) you get the same buffer back, which already contains "rebmetpes". You then put "november" in, reversed, which is slightly shorter, hence your buffer now contains "rebmevons".
So, the fix? Allocate another character too, this will hold your zero terminator (char *res = malloc( longitud * sizeof(char) + 1);). After you reverse the string, set the zero terminator at the end of the string (res[longitud] = '\0';).
there are two errors there, the first one is that you need one char more allocated (all chars for the string + 1 for the terminator)
char *res = malloc( (longitud+1) * sizeof(char) );
The second one is that you have to terminate the string:
res[longitud]='\0';
You can terminate the string before entering in the loop because you know already the size of the destination string.
Note that using calloc instead of malloc you will not need to terminate the string as the memory gets alreay zero-initialised
Thanks, it solved my problem. I read something about the "\0" in strings but wasn't very clear, which is now after reading all the answers (all are pretty good). Thank you all for the help.
Seems to be a basic question but I would rather ask this to clear up than spend many more days on this.I am trying to copy data in a buffer which I receive(recv call) which will be then pushed to a file. I want to use memcpy to continuously append/add data to the buffer until the size of buffer is not enough to hold more data where I than use the realloc. The code is as below.
int vl_packetSize = PAD_SIZE + (int)p_size - 1; // PAD_SIZE is the size of char array sent
//p_size is the size of data to be recv. Same data size is used by send
int p_currentSize = MAX_PROTO_BUFFER_SIZE;
int vl_newPacketSize = p_currentSize;
char *vl_data = (char *)malloc(vl_packetSize);
memset((char *)vl_data,'\0',vl_packetSize);
/* Allocate memory to the buffer */
vlBuffer = (char *)malloc(p_currentSize);
memset((char *)vlBuffer,'\0',p_currentSize);
char *vlBufferCopy = vlBuffer;
if(vlBuffer==NULL)
return ERR_NO_MEM;
/* The sender first sends a padding data of size PAD_SIZE followed by actual data. I want to ignore the pad hence do vl_data+PAD_SIZE on memcpy */
if((p_currentSize - vl_llLen) < (vl_packetSize-PAD_SIZE)){
vl_newPacketSize +=vl_newPacketSize;
char *vlTempBuffer = (char *)realloc(vlBufferCopy,(size_t)vl_newPacketSize);
if(vlTempBuffer == NULL){
if(debug > 1)
fprintf(stdout,"Realloc failed:%s...Control Thread\n\n",fn_strerror_r(errno,err_buff));
free((void *)vlBufferCopy);
free((void *)vl_data);
return ERR_NO_MEM;
}
vlBufferCopy = vlTempBuffer;
vl_bytesIns = vl_llLen;
vl_llLen = 0;
vlBuffer = vlBufferCopy+vl_bytesIns;
fprintf(stdout,"Buffer val after realloc:%s\n\n",vlBufferCopy);
}
memcpy(vlBuffer,vl_data+PAD_SIZE,vl_packetSize-PAD_SIZE);
/*
fprintf(stdout,"Buffer val before increment:%s\n\n",vlBuffer);
fprintf(stdout,"vl_data length:%d\n\n",strlen(vl_data+PAD_SIZE));
fprintf(stdout,"vlBuffer length:%d\n\n",strlen(vlBuffer));
*/
vlBuffer+=(vl_packetSize-PAD_SIZE);
vl_llLen += (vl_packetSize-PAD_SIZE);
vl_ifNotFlush = 1;
//fprintf(stdout,"Buffer val just before realloc:%s\n\n",vlBufferCopy);
}
Problem: Whan ever I fputs the data into the file later on. Only the first data recv/added to buffer is gets into the file.
Also when I print the value of vlBufferCopy(which points to first location of data returned by malloc or realloc) I get the same result.
If I decrease the size by 1, I see entire data in the file, but it somehow misses the new line character and hence the data is
not inserted in the proper format in the file.
I know it is because of trailing '\0' but some how reducing the size by 1
(vlBuffer+=(vl_packetSize-PAD_SIZE-1);)
misses the new line character. fputs while putting the data removes the trailing null character
Please let me know what I am missing here to check or in the logic
(Note: I tried using strcat:
strcat(vlBuffer,vl_data+PAD_SIZE);
but I wanted to use memcpy as it is faster and also it can be used for any kind of buffer and not only character pointer
Thanks
strcat and memcpy are very different functions.
I suggest you read the documentation of each.
Mainly, there are two differences:
1. memcpy copies data where you tell it to. strcat finds the end of the string, and copies there.
2. memcpy copies the number of bytes you request. strcat copies until the terminating null.
If you're dealing with packets of arbitrary contents, you have no use for strcat, or other string functions.
You need to write to the file in a binary-safe way. Check how to use fwrite instead of fputs. fwrite will copy all the buffer, even if there's a zero in the middle of it.
const char *mybuff= "Test1\0Test2";
const int mybuff_len = 11;
size_t copied = fwrite(mybuff, mybuff_len, 1, output_file);
this code
int
main (void)
{
int i;
char pmbbuf[4];
wchar_t *pwchello = L"1234567890123456789012345678901234567890";
i = wcstombs (pmbbuf, pwchello, wcslen(pwchello)* MB_CUR_MAX + 1);
printf("%d\n", MB_CUR_MAX);
printf (" Characters converted: %u\n", i);
printf (" Multibyte character: %s\n\n", pmbbuf);
return 0;
}
and the strange thing is that it compiles with no warnings.
when I run ./a.out it printed
1
Characters converted: 40
Multibyte character: 1234(
Segmentation fault
Any ideas for the seg fault?
TIA,
cateof
You encounter buffer overrun because you don't null-terminate the buffer after conversion and the buffer size is also not enough to hold the result.
You could allocate memory dynamically since you don't know in advance how much memory is required:
int i;
char pmbbuf*;
wchar_t *pwchello = L"1234567890123456789012345678901234567890";
// this will not write anything, but return the number of bytes in the result
i = wcstombs (0, pwchello, wcslen(pwchello)* MB_CUR_MAX + 1);
//allocate memory - +1 byte for the trailing null, checking for null pointer returned omitted (though needed)
pmbbuf = malloc( i + 1 );
i = wcstombs (pmbbuf, pwchello, wcslen(pwchello)* MB_CUR_MAX + 1);
//put the trailing null
pmbbuf[i] = 0;
//whatever you want to do with the string - print, e-mail, fax, etc.
// don't forget to free memory
free( pmbbuf );
//defensive - to avoid misuse of the pointer
pmbbuf = 0;
You're trying to put a string that's definitely longer than 4 chars into a char array that can hold 4 characters. As you don't specify '4' as the maximum size, the conversion will write into memory that it doesn't own or that might be used by other variables, house keeping data like function return values on the stack or similar. This will lead to the seg fault as you're overwriting data that was pushed on the stack (stacks grow top-down) before you called wcstombs.