In some network code I need to encode packet structures in a buffer to send(2) over a socket but memcpy'ing the encoded buffers into a bigger buffer seems problematic.
Here's a small code example illustrating what I am doing:
char tbuf[] = "test";
char *buf = malloc(300);
memset(buf, '\0', 300);
int bytes_to_copy = 300;
int bytes_copied = 0;
while (bytes_copied < bytes_to_copy) {
memcpy(buf + bytes_copied, tbuf, sizeof(tbuf));
bytes_copied += sizeof(tbuf);
}
/* free, return */
This should append "test" 60 times into buf but buf ends up only containing one "test". Am I using memcpy wrong?
You are repeatedly copying the null terminator at the end of test. sizeof tbuf is 5.
So all the C standard library functions will ignore all the other concatenants.
The solution: copy one less byte in the memcpy, and be sure to add a null terminator to the final string.
Related
I am trying to make this dynamic reallocation work in a portable fashion.
My program accepts a line of text from a user and appends it to a buffer. If the length of text in the buffer is 20 or more, it removes the first 20 characters and moves any characters after that to the start of the buffer.
I have this code which works clean on Linux but when I run it on windows it emits garbage. Does anyone know why/how to make this portable only using malloc. IE not using string.h(strcpy) str... anything but len.
c17 only - no broken stucts(not portable). here is my code. Compiles with no errors gcc 7.3, mingw 7.3. I replace gets and puts with safer functions and I still get garbage on windows. I assume this is a formatting issue...
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <malloc.h>
void wbuff (message)
char *message;
{
FILE *f = fopen("file.txt", "w");
fprintf(f, "%s", message);
fclose(f);
}
char *rean (message)
char *message;
{
/* performs (write) on buffer, trims lefover, then restores */
char buf[80] = "";
puts("enter a line");
gets(buf);
int bln = strlen( buf );
int mln = strlen( message );
int nln = bln + mln;
printf("new length %d\n", nln);
message = realloc(message, nln);
memmove(message + mln, buf, bln);
/* MISTAKE IS HERE?! */
if( nln >= 20 ) {
int exl = nln -20; // leftover length
char *lo = realloc(NULL, exl); // leftover placeholder
memmove(lo, message+20, exl); // copy leftover
wbuff(message); // write clear buff
message = realloc(NULL, nln);
message = realloc(NULL, exl); // resize buffer
memmove(message, lo, exl); // restore leftover
}
return message;
}
void main (void)
{
char *message = "";
message = realloc(NULL, 0);
while ( 1 == 1 ) {
message = rean( message );
puts(message);
}
return;
}
In C, strings are a sequence of characters terminated by a null byte. You have a number of off-by-one errors here mostly related to not accounting for that fact, as well as memory leaks.
When you first set message in main:
char *message = "";
message = realloc(NULL, 0);
message either is NULL to points to 0 bytes of memory. When you call then call rean the first time:
int mln = strlen( message );
You're either attempting to dereference a NULL pointer to read past the end of allocated memory. You want to allocate at least 1 byte to start and set that byte to 0 so you have an empty string:
char *message = realloc(NULL, 1);
message[0] = '\0';
Then later when you copy the buffer into the message:
message = realloc(message, nln);
memmove(message + mln, buf, bln);
You don't allocate enough space for the terminating null byte, nor do you copy it, so you don't really have a string. When you then try to print it, either puts or printf reads past the end of the allocated memory. You need to allocate 1 extra byte and copy 1 extra byte:
message = realloc(message, nln + 1); // allocate 1 extra byte for the null terminator
memmove(message + mln, buf, bln + 1); // copy 1 extra byte
There are similar issues when you recopy anything past 20 characters:
int exl = nln -20; // leftover length
char *lo = realloc(NULL, exl); // leftover placeholder
memmove(lo, message+20, exl); // copy leftover
wbuff(message); // write clear buff
message = realloc(NULL, nln);
message = realloc(NULL, exl); // resize buffer
memmove(message, lo, exl); // restore leftover
lines 2-3: You don't allocate space for the terminating null byte for lo nor do you copy it.
line 5: You leak the memory previously held by message in the first realloc by passing assigning to message while using NULL as the first argument
line 6-7: You leak the memory allocated in line 5 by doing the same thing. Also, you again don't allocate space for the null byte, nor do you copy it on the next line.
As before, allocate 1 extra byte for each allocation and move 1 extra byte to account for the null terminator. Also, free lo at the end of the block, remove the extra realloc for message, and pass the prior value of message to realloc so you don't leak memory:
int exl = nln -20;
char *lo = realloc(NULL, exl + 1); // allocate 1 extra byte
memmove(lo, message+20, exl + 1); // copy 1 extra byte
wbuff(message);
// remove extra realloc
message = realloc(message, exl + 1); // pass in old message, allocate 1 extra
memmove(message, lo, exl + 1); // copy 1 extra byte
free(lo); // free leftover
These issues of reading and writing past the end of allocated memory all invoke undefined behavior, which explains why you see different results on different operating systems.
As far as conforming code goes, use fgets intead of gets:
fgets(line, sizeof(line), stdin);
This function will include a newline in line if there's space for it, so be sure to remove it if that's the case.
Also change main to return int, and remove #include <malloc.h> since the malloc family of function is defined to reside in stdlib.h.
Had you used strcpy and strcat instead of memmove, you wouldn't have had to account for copying the null terminating byte as those functions do that for you. You would still however need to account for that when allocating memory. There's also no conflict between strcpy, malloc, and realloc, as they are all part of the standard and work together properly. Using them together is no problem. If they aren't working properly for you then you aren't using them correctly.
After applying my updates, you can replace this:
memmove(message + mln, buf, bln + 1);
With this:
strcat(message, buf);
And replace this:
memmove(lo, message+20, exl + 1); // copy leftover
...
memmove(message, lo, exl + 1); // restore leftover
With this:
strcpy(lo, message+20);
...
strcpy(message, lo);
And it will still work properly and be conforming.
I have a program that reads the content of a file and saves it into buf. After reading the content it is supposed to copy two by two chars to an array. This code works fine if I'm not trying to read from a file but if I try to read it from a file the printf from buffer prints the two chars that I want but adds weird characters. I've confirmed and it's saving correctly into buf, no weird characters there. I can't figure out what's wrong... Here's the code:
char *buffer = (char*)malloc(2*sizeof(char));
char *dst = buffer;
char *src = buf;
char *end = buf + strlen(buf);
char *baby = '\0';
while (src<= end)
{
strncpy(dst, src, 2);
src+= 2;
printf("%s\n", buffer);
}
(char*)malloc(2*sizeof(char)); change to malloc(3*sizeof*buffer); You need an additional byte to store the terminating null character which is used to indicate the end-of-string. Aslo, do not cast the return value of malloc(). Thanks to unwind
In your case, with strncpy(), you have supplied n as 2, which is not having any scope to store the terminating null byte. without the trminating null, printf() won't be knowing where to stop. Now, with 3 bytes of memory, you can use strcpy() to copy the string properly
strncpy() will not add the terminating null itself, in case the n is equal to the size of supplied buffer, thus becoming very very unreliable (unlike strcpy()). You need to take care of it programmatically.
check the man page for strncpy() and strcpy() here.
I'm coding a program that takes some files as parameters and prints all lines reversed. The problem is that I get unexpected results:
If I apply it to a file containing the following lines
one
two
three
four
I get the expected result, but if the file contains
september
november
december
It returns
rebmetpes
rebmevons
rebmeceds
And I don't understand why it adds a "s" at the end
Here is my code
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
void reverse(char *word);
int main(int argc, char *argv[], char*envp[]) {
/* No arguments */
if (argc == 1) {
return (0);
}
FILE *fp;
int i;
for (i = 1; i < argc; i++) {
fp = fopen(argv[i],"r"); // read mode
if( fp == NULL )
{
fprintf(stderr, "Error, no file");
}
else
{
char line [2048];
/*read line and reverse it. the function reverse it prints it*/
while ( fgets(line, sizeof line, fp) != NULL )
reverse(line);
}
fclose(fp);
}
return (0);
}
void reverse(char *word)
{
char *aux;
aux = word;
/* Store the length of the word passed as parameter */
int longitud;
longitud = (int) strlen(aux);
/* Allocate memory enough ??? */
char *res = malloc( longitud * sizeof(char) );
int i;
/in this loop i copy the string reversed into a new one
for (i = 0; i < longitud-1; i++)
{
res[i] = word[longitud - 2 - i];
}
fprintf(stdout, "%s\n", res);
free(res);
}
(NOTE: some code has been deleted for clarity but it should compile)
You forget to terminate your string with \0 character. In reversing the string \0 becomes your first character of reversed string. First allocate memory for one more character than you allocated
char *res = malloc( longitud * sizeof(char) + 1);
And the try this
for (i = 0; i < longitud-1; i++)
{
res[i] = word[longitud - 2 - i];
}
res[i] = '\0'; // Terminating string with '\0'
I think I know the problem, and it's a bit of a weird issue.
Strings in C are zero terminated. This means that the string "Hi!" in memory is actually represented as 'H','i','!','\0'. The way strlen etc then know the length of the string is by counting the number of characters, starting from the first character, before the zero terminator. Similarly, when printing a string, fprintf will print all the characters until it hits the zero terminator.
The problem is, your reverse function never bothers to set the zero terminator at the end, which it needs to since you're copying characters into the buffer character by character. This means it runs off the end of your allocated res buffer, and into undefined memory, which just happened to be zero when you hit it (malloc makes no promises of the contents of the buffer you allocate, just that it's big enough). You should get different behaviour on Windows, since I believe that in debug mode, malloc initialises all buffers to 0xcccccccc.
So, what's happening is you copy september, reversed, into res. This works as you see, because it just so happens that there's a zero at the end.
You then free res, then malloc it again. Again, by chance (and because of some smartness in malloc) you get the same buffer back, which already contains "rebmetpes". You then put "november" in, reversed, which is slightly shorter, hence your buffer now contains "rebmevons".
So, the fix? Allocate another character too, this will hold your zero terminator (char *res = malloc( longitud * sizeof(char) + 1);). After you reverse the string, set the zero terminator at the end of the string (res[longitud] = '\0';).
there are two errors there, the first one is that you need one char more allocated (all chars for the string + 1 for the terminator)
char *res = malloc( (longitud+1) * sizeof(char) );
The second one is that you have to terminate the string:
res[longitud]='\0';
You can terminate the string before entering in the loop because you know already the size of the destination string.
Note that using calloc instead of malloc you will not need to terminate the string as the memory gets alreay zero-initialised
Thanks, it solved my problem. I read something about the "\0" in strings but wasn't very clear, which is now after reading all the answers (all are pretty good). Thank you all for the help.
How do I remove 20 spaces from the beginning of a buffer and 20 at the end?
I can run a for loop starting at 20 and -20 at the end total but how do I copy each char by char to another buffer or is there a simple padding command?
Thanks for all the help so far now works but just displays a about 20 charactors i.e the MFI-40576-021 section from the dump it does now skip the 20 crap bytes at the start.
ReadFile(h,buffer,0x224,&read,NULL);
buffer[read] = '\0';
char newbuff[1000];
memcpy(newbuff,buffer+20,sizeof(buffer)-40);
Create a new buffer of size s-40, do a memcpy from the source+20 to the destination for size s.
int read; //assumed
char buffer[0x224];
ReadFile(h,buffer,0x224,&read,NULL);
if(read-40 <=0) return;
char *newvar = (char*)malloc(read - 40 + 1); // Fails with void to char error
strncpy(newvar, buffer+20, read - 40);
newvar[read-40] = '\0';
If altering the original buffer is acceptable, it may be easiest to use memmove:
char* string = /* insert code here to obtain the string */;
size_t len = strlen(string) - 40; // Number of bytes to move
memmove(string, string + 20, len); // Move string down by 20 bytes
memset(&string[len], 0, 40); // Clear out the remaining buffer
This general procedure will also work if the data in your buffer is not a string. You'll just need to replace the strlen call with whatever code is needed to obtain the original buffer's length.
Use strncpy:
char *string = // however you obtain the string
char *new = malloc(strlen(string) - 40 + 1);
strncpy(new, string + 20, strlen(string) - 40);
Seems to be a basic question but I would rather ask this to clear up than spend many more days on this.I am trying to copy data in a buffer which I receive(recv call) which will be then pushed to a file. I want to use memcpy to continuously append/add data to the buffer until the size of buffer is not enough to hold more data where I than use the realloc. The code is as below.
int vl_packetSize = PAD_SIZE + (int)p_size - 1; // PAD_SIZE is the size of char array sent
//p_size is the size of data to be recv. Same data size is used by send
int p_currentSize = MAX_PROTO_BUFFER_SIZE;
int vl_newPacketSize = p_currentSize;
char *vl_data = (char *)malloc(vl_packetSize);
memset((char *)vl_data,'\0',vl_packetSize);
/* Allocate memory to the buffer */
vlBuffer = (char *)malloc(p_currentSize);
memset((char *)vlBuffer,'\0',p_currentSize);
char *vlBufferCopy = vlBuffer;
if(vlBuffer==NULL)
return ERR_NO_MEM;
/* The sender first sends a padding data of size PAD_SIZE followed by actual data. I want to ignore the pad hence do vl_data+PAD_SIZE on memcpy */
if((p_currentSize - vl_llLen) < (vl_packetSize-PAD_SIZE)){
vl_newPacketSize +=vl_newPacketSize;
char *vlTempBuffer = (char *)realloc(vlBufferCopy,(size_t)vl_newPacketSize);
if(vlTempBuffer == NULL){
if(debug > 1)
fprintf(stdout,"Realloc failed:%s...Control Thread\n\n",fn_strerror_r(errno,err_buff));
free((void *)vlBufferCopy);
free((void *)vl_data);
return ERR_NO_MEM;
}
vlBufferCopy = vlTempBuffer;
vl_bytesIns = vl_llLen;
vl_llLen = 0;
vlBuffer = vlBufferCopy+vl_bytesIns;
fprintf(stdout,"Buffer val after realloc:%s\n\n",vlBufferCopy);
}
memcpy(vlBuffer,vl_data+PAD_SIZE,vl_packetSize-PAD_SIZE);
/*
fprintf(stdout,"Buffer val before increment:%s\n\n",vlBuffer);
fprintf(stdout,"vl_data length:%d\n\n",strlen(vl_data+PAD_SIZE));
fprintf(stdout,"vlBuffer length:%d\n\n",strlen(vlBuffer));
*/
vlBuffer+=(vl_packetSize-PAD_SIZE);
vl_llLen += (vl_packetSize-PAD_SIZE);
vl_ifNotFlush = 1;
//fprintf(stdout,"Buffer val just before realloc:%s\n\n",vlBufferCopy);
}
Problem: Whan ever I fputs the data into the file later on. Only the first data recv/added to buffer is gets into the file.
Also when I print the value of vlBufferCopy(which points to first location of data returned by malloc or realloc) I get the same result.
If I decrease the size by 1, I see entire data in the file, but it somehow misses the new line character and hence the data is
not inserted in the proper format in the file.
I know it is because of trailing '\0' but some how reducing the size by 1
(vlBuffer+=(vl_packetSize-PAD_SIZE-1);)
misses the new line character. fputs while putting the data removes the trailing null character
Please let me know what I am missing here to check or in the logic
(Note: I tried using strcat:
strcat(vlBuffer,vl_data+PAD_SIZE);
but I wanted to use memcpy as it is faster and also it can be used for any kind of buffer and not only character pointer
Thanks
strcat and memcpy are very different functions.
I suggest you read the documentation of each.
Mainly, there are two differences:
1. memcpy copies data where you tell it to. strcat finds the end of the string, and copies there.
2. memcpy copies the number of bytes you request. strcat copies until the terminating null.
If you're dealing with packets of arbitrary contents, you have no use for strcat, or other string functions.
You need to write to the file in a binary-safe way. Check how to use fwrite instead of fputs. fwrite will copy all the buffer, even if there's a zero in the middle of it.
const char *mybuff= "Test1\0Test2";
const int mybuff_len = 11;
size_t copied = fwrite(mybuff, mybuff_len, 1, output_file);