I got a problem when reading SSL response that causes a segmentation fault. I read the response into a buffer, then append it to a malloced string and memory reset it to 0 till the response is fully read, but when I try this in a multi threaded program, after some operations it gives me segmentation fault. When I remove strcat it doesn't give me segmentation fault even if I run it for hours.
Example:
char* response = malloc(10000);
char buffer[10000] = { 0 };
while(SSL_read(ssl,buf,sizeof(buffer)) > 0){
strcat(response,buffer);
memset(buffer,0,sizeof(buffer));
}
Errors
free(): invalid next size (normal)
malloc_consolidate(): invalid chunk
I made sure of freeing both of SSL and CTX and close socket and free the malloced string.
There are a few problems with your code:
You are not dealing with C strings, you are dealing with arbitrary byte sequences. SSL_read() reads bytes, not C strings, and you cannot treat them as strings. What you read cannot be assumed to be NUL-terminated (\0), so you should not use strcat, strlen or other similar functions that operate on strings. Zeroing out the entire buffers just to make sure there is a terminator makes little to no sense, as the terminator could very well be found in the middle of the data.
You are reading data continuously in a loop into a fixed size buffer. Your code will overflow the destination buffer (response) very easily.
Not an error, but there isn't really any need for an intermediate buffer to begin with. You are needlessly copying stuff around two times (one with SSL_read() and one with strcat) when you can read directly into response instead. On top of that, the memset() to clear the contents of buffer also adds a third scan of the data, slowing things down even more.
Again, not an error, but SSL_read() returns int and uses that to return the read size. You are not really using it, but you should, as you need to keep track of how much space is left on the buffer. You would be however much better off using size_t to avoid unwanted problems with signed math and possible overflows. You can use SSL_read_ex() for this purpose.
Here's a snippet of code that does what you want in a more robust way:
#define CHUNK_SIZE 10000
unsigned char *response = NULL;
size_t size = 0;
size_t space_left = 0;
size_t total_read = 0;
size_t n;
while (1) {
// Allocate more memory if needed.
if (space_left < CHUNK_SIZE) {
unsigned char *tmp = realloc(response, size + CHUNK_SIZE);
if (!tmp) {
// Handle realloc error
break;
}
response = tmp;
size += CHUNK_SIZE;
space_left += CHUNK_SIZE;
}
if (SSL_read_ex(ssl, response + total_read, space_left, &n)) {
total_read += n;
space_left -= n;
} else {
// Handle error
break;
}
}
You never initialized response(). The arguments to strcat() have to be null-terminated strings.
You should also subtract 1 from the size of the buffer when calling SSL_read(), to ensure there will always be room for its null terminator.
char* response = malloc(10000);
response[0] = '\0';
char buffer[10000] = { 0 };
while(SSL_read(ssl,buf,sizeof(buffer)-1) > 0){
strcat(response,buffer);
memset(buffer,0,sizeof(buffer));
}
Related
I'm posting a continuation of my question from this thread.
I'm trying to create a string that begins with a '!' and adds 6 values read from a sensor (separated by commas) and then sends it over a serial port. A sample output would be: "!5,5,5,5,5,5" or "!34,34,34,34,34,34".
My code is mostly working; I'm able to send the value of one analog sensor across my serial port, !215!215!215 for example, but when I un-comment the for loop code below I see nothing across my serial port and the program seems, for lack of a better word, useless.
There seems to be a runtime error occuring in my for loop but I can't determine where it happens. Why does my code below successfully send serial data for one analog sensor without using the for loop, and send nothing when using the for loop? How can I tweak my code to achieve my desired output?
char* convertIntToString(uint8_t integerValue, char* str){
utoa(integerValue, str, 10);
return str;
}
char* concat(char *s1, char *s2)
{
char *result = malloc(strlen(s1)+strlen(s2)+1);
strcpy(result, s1);
strcat(result, s2);
return result;
}
int main(void){
uint8_t analogValue;
char *outputStr = malloc(1);
while (1) {
outputStr = realloc(outputStr, 2);
strcpy(outputStr, "!");
analogValue = ReadADC(0);
char str[4];
outputStr = concat(outputStr, convertIntToString(analogValue, str));
//RUNTIME ERROR IN THIS LOOP
for(int i = 0; i < 5; i++){
char* newStr = concat(outputStr, ",");
// free the old memory before using the new memory
free(outputStr);
outputStr = newStr;
newStr = concat(outputStr, convertIntToString(analogValue, str));
// free the old memory before using the new memory
free(outputStr);
outputStr = newStr;
}
CDC_Device_SendString(&VirtualSerial_CDC_Interface, outputStr); //send sring over serial port
free(outputStr);
}
}
Expanded from the comment above and comments in the previous question.
If you are able to calculate the maximum size of a "packet", then you can avoid dynamic memory all together and just use a fixed buffer size. The calculation doesn't even have to be 100% accurate, as long as it errs on the side of "too big".
e.g.: 5 instances of 5 numbers with a max of 3 digits separated by commas: 5 * 5 * 4 (3 digits + a comma). Not 100% right because the 5th group doesn't need a comma, so you are over estimating by one (or is that 5?). Just be aware of the possible cumulative effect of this if you have multiple "known errors".
So assuming you can estimate the max size, perhaps "encode" it via #defines - perhaps even fixing some of the "known errors").
So now you have char buffer[KNOWN_UPPER_BOUND], as long as you initialize it correctly (e.g. buffer[0] = '\0';, you can just keep appending to it via strcat(). If you were talking big numbers, you could keep track of the last index to avoid repeated scans through the string looking for the end.
e.g. (using globals for simplicity)
char buffer[KNOWN_UPPER_BOUND];
int last_index=0;
addString(char* str)
{
int len = strlen(str);
if (last_index + len > KNOWN_UPPER_BOUND)
{
/* error handling */
}
else
{
strcat(buffer[last_index], str);
last_index += n;
}
}
So what were some of the issues with the dynamic code?
Potential for leaks (much like the errors I mentioned in the calculation - ok (by which I mean 'not overly harmful in a small program') if you leak 2 bytes once, not so good when you put it in a loop and leak 2 bytes over and over again)
Speed issues - malloc is out of you control, it could be very slow. A lot of small allocations can fragment memory which may mean later on when you want a bigger block you can't get one.
Lots of copying and re-copying of data. Your concat is an example here - each concat does a malloc and copies both strings. Every time you call it.
You could still use dynamic memory to hold the final string, but build up each "component" in a fixed size buffer.
What if you move the declaration of char* newStr outside the loop.
Declaring the newStr as char array will be better than pointer to avoid leakage. something like char newStr[50]
I'm trying to use the getdelim function to read an entire text file's contents into a string.
Here is the code I am using:
ssize_t bytesRead = getdelim(&buffer, 0, '\0', fp);
This is failing however, with strerror(errno) saying "Error: Invalid Argument"
I've looked at all the documentation I could and just can't get it working, I've tried getline which does work but I'd like to get this function working preferably.
buffer is NULL initialised as well so it doesn't seem to be that
fp is also not reporting any errors and the file opens perfectly
EDIT: My implementation is based on an answer from this stackoverflow question Easiest way to get file's contents in C
Kervate, please enable compiler warnings (-Wall for gcc), and heed them. They are helpful; why not accept all the help you can get?
As pointed out by WhozCraig and n.m. in comments to your original question, the getdelim() man page shows the correct usage.
If you wanted to read records delimited by the NUL character, you could use
FILE *input; /* Or, say, stdin */
char *buffer = NULL;
size_t size = 0;
ssize_t length;
while (1) {
length = getdelim(&buffer, &size, '\0', input);
if (length == (ssize_t)-1)
break;
/* buffer has length chars, including the trailing '\0' */
}
free(buffer);
buffer = NULL;
size = 0;
if (ferror(input) || !feof(input)) {
/* Error reading input, or some other reason
* that caused an early break out of the loop. */
}
If you want to read the contents of a file into a single character array, then getdelim() is the wrong function.
Instead, use realloc() to dynamically allocate and grow the buffer, appending to it using fread(). To get you started -- this is not complete! -- consider the following code:
FILE *input; /* Handle to the file to read, assumed already open */
char *buffer = NULL;
size_t size = 0;
size_t used = 0;
size_t more;
while (1) {
/* Grow buffer when less than 500 bytes of space. */
if (used + 500 >= size) {
size_t new_size = used + 30000; /* Allocate 30000 bytes more. */
char *new_buffer;
new_buffer = realloc(buffer, new_size);
if (!new_buffer) {
free(buffer); /* Old buffer still exists; release it. */
buffer = NULL;
size = 0;
used = 0;
fprintf(stderr, "Not enough memory to read file.\n");
exit(EXIT_FAILURE);
}
buffer = new_buffer;
size = new_size;
}
/* Try reading more data, as much as fits in buffer. */
more = fread(buffer + used, 1, size - used, input);
if (more == 0)
break; /* Could be end of file, could be error */
used += more;
}
Note that the buffer in this latter snippet is not a string. There is no terminating NUL character, so it's just an array of chars. In fact, if the file contains binary data, the array may contain lots of NULs (\0, zero bytes). Assuming there was no error and all of the file was read (you need to check for that, see the former example), buffer contains used chars read from the file, with enough space allocated for size. If used > 0, then size > used. If used == 0, then size may or may not be zero.
If you want to turn buffer into a string, you need to decide what to do with the possibly embedded \0 bytes -- I recommend either convert to e.g. spaces or tabs, or move the data to skip them altogether --, and add the string-terminating \0 at end to make it a valid string.
EDIT: It has been proven in the comments that defining the length instead should produce the same results and would not use any significant extra data. If you are looking for a way to send data between machines running your program(s), sending the length is better than reading until a terminating character. BonzaiThePenguin has some very good points you should look at.
But for educational purposes: I never found good example code that does this for standard C sockets that handles situations where the data is not all received in one packet, or multiple separate messages are contained within one packet. Simply calling recv repeatedly will not work in all cases.
This is one of those questions where I've answered it myself below, but I'm not 100% confident in my response.
It isn't 'dangerous to allow the client to specify the size of the message it is sending'. Most of the protocols in the word do that, including HTTP and SSL. It's only dangerous when implementations don't bounds-check messages properly.
The fatal flaw with your suggestion is that it doesn't work for binary data: you have to introduce an escape character so that the terminating character can appear within a message, and then of course you also need to escape the escape. All this adds processing and data copying at both ends.
Here is what I came up with. I cannot guarantee that this is perfect because I am not a professional, so if there are any mistakes, I (and anyone else looking for help) would greatly appreciate it if someone would point them out.
Context: socket is the socket, buffer is the array that stores all network input, line is the array that stores just one message extracted from buffer (which is what the rest of your program uses), length is the length of both inputted arrays, and recvLength is a pointer to an integer stored outside of the function that is meant to be 0 initially and should not be freed or modified by anything else. That is, it should persist across multiple calls to this function on the same socket. This function returns the length of the data outputted in the line array.
size_t recv_line(int socket, char* buffer, char* line, size_t length, size_t* recvLength){ //receives until '\4' (EOT character) or '\0' (null character)
size_t readHead = 0;
size_t lineIndex = 0;
char currentChar = 0;
while (1){
for (; readHead < *recvLength; readHead = readHead + 1){
currentChar = buffer[readHead];
if (currentChar=='\4' || currentChar=='\0'){ //replace with the end character(s) of your choice
if (DEBUG) printf("Received message===\n%s\n===of length %ld\n", line, lineIndex+1);
memcpy(buffer, buffer + readHead + 1, length-(readHead)); //shift the buffer down
*recvLength -= (readHead + 1); //without the +1, I had an "off by 1" error before!
return lineIndex+1; //success
}
if (readHead >= length){
if (DEBUG) printf("Client tried to overflow the input buffer. Disconnecting client.\n");
*recvLength = 0;
return 0;
}
line[lineIndex] = currentChar;
lineIndex++;
}
*recvLength = recv(socket, buffer + readHead, length, 0);
}
printf("Unknown error in recv_line!\n");
return 0;
}
Simple example usage:
int function_listening_to_network_input(int socket){
char netBuffer[2048];
char lineBuffer[2048];
size_t recvLength = 0;
while (1==1){
size_t length = recv_line(socket, netBuffer, lineBuffer, 2048, &recvLength);
// handle it…
}
return 0;
}
Note that this does not always leave line as a null-terminated string. If you want it to, it's easy to modify.
Seems to be a basic question but I would rather ask this to clear up than spend many more days on this.I am trying to copy data in a buffer which I receive(recv call) which will be then pushed to a file. I want to use memcpy to continuously append/add data to the buffer until the size of buffer is not enough to hold more data where I than use the realloc. The code is as below.
int vl_packetSize = PAD_SIZE + (int)p_size - 1; // PAD_SIZE is the size of char array sent
//p_size is the size of data to be recv. Same data size is used by send
int p_currentSize = MAX_PROTO_BUFFER_SIZE;
int vl_newPacketSize = p_currentSize;
char *vl_data = (char *)malloc(vl_packetSize);
memset((char *)vl_data,'\0',vl_packetSize);
/* Allocate memory to the buffer */
vlBuffer = (char *)malloc(p_currentSize);
memset((char *)vlBuffer,'\0',p_currentSize);
char *vlBufferCopy = vlBuffer;
if(vlBuffer==NULL)
return ERR_NO_MEM;
/* The sender first sends a padding data of size PAD_SIZE followed by actual data. I want to ignore the pad hence do vl_data+PAD_SIZE on memcpy */
if((p_currentSize - vl_llLen) < (vl_packetSize-PAD_SIZE)){
vl_newPacketSize +=vl_newPacketSize;
char *vlTempBuffer = (char *)realloc(vlBufferCopy,(size_t)vl_newPacketSize);
if(vlTempBuffer == NULL){
if(debug > 1)
fprintf(stdout,"Realloc failed:%s...Control Thread\n\n",fn_strerror_r(errno,err_buff));
free((void *)vlBufferCopy);
free((void *)vl_data);
return ERR_NO_MEM;
}
vlBufferCopy = vlTempBuffer;
vl_bytesIns = vl_llLen;
vl_llLen = 0;
vlBuffer = vlBufferCopy+vl_bytesIns;
fprintf(stdout,"Buffer val after realloc:%s\n\n",vlBufferCopy);
}
memcpy(vlBuffer,vl_data+PAD_SIZE,vl_packetSize-PAD_SIZE);
/*
fprintf(stdout,"Buffer val before increment:%s\n\n",vlBuffer);
fprintf(stdout,"vl_data length:%d\n\n",strlen(vl_data+PAD_SIZE));
fprintf(stdout,"vlBuffer length:%d\n\n",strlen(vlBuffer));
*/
vlBuffer+=(vl_packetSize-PAD_SIZE);
vl_llLen += (vl_packetSize-PAD_SIZE);
vl_ifNotFlush = 1;
//fprintf(stdout,"Buffer val just before realloc:%s\n\n",vlBufferCopy);
}
Problem: Whan ever I fputs the data into the file later on. Only the first data recv/added to buffer is gets into the file.
Also when I print the value of vlBufferCopy(which points to first location of data returned by malloc or realloc) I get the same result.
If I decrease the size by 1, I see entire data in the file, but it somehow misses the new line character and hence the data is
not inserted in the proper format in the file.
I know it is because of trailing '\0' but some how reducing the size by 1
(vlBuffer+=(vl_packetSize-PAD_SIZE-1);)
misses the new line character. fputs while putting the data removes the trailing null character
Please let me know what I am missing here to check or in the logic
(Note: I tried using strcat:
strcat(vlBuffer,vl_data+PAD_SIZE);
but I wanted to use memcpy as it is faster and also it can be used for any kind of buffer and not only character pointer
Thanks
strcat and memcpy are very different functions.
I suggest you read the documentation of each.
Mainly, there are two differences:
1. memcpy copies data where you tell it to. strcat finds the end of the string, and copies there.
2. memcpy copies the number of bytes you request. strcat copies until the terminating null.
If you're dealing with packets of arbitrary contents, you have no use for strcat, or other string functions.
You need to write to the file in a binary-safe way. Check how to use fwrite instead of fputs. fwrite will copy all the buffer, even if there's a zero in the middle of it.
const char *mybuff= "Test1\0Test2";
const int mybuff_len = 11;
size_t copied = fwrite(mybuff, mybuff_len, 1, output_file);
I am trying to make a simple client and server. Right now I am able to prints the contents of a file out to the screen. I would now like to store every line i read from the buffer into an array. I have attempted this but for some reason it always just adds the last line received from the buffer. Can anyone point out where I have gone wrong
int getFile (char path[256], int fd)
{
char buffer[256];
char bufferCopy[256];
char arguments[1000][1000];
int total = 0;
char * ptr;
while(read(fd, buffer, 256) != NULL)
{
char * temp;
strcpy(arguments[total], buffer);
total++;
}
for(int i = 0; i < total; i++)
{
printf("\n %s", arguments[i]);
}
}
Your read call doesn't read lines, it reads up to 256 bytes from fd. read also doesn't know anything about null terminators so there is no guarantee that buffer will hold a string (i.e. have a null terminator) and hence no guarantee that strcpy will stop copying at a sensible place. You're almost certainly scribbling all over your stack and once you do that, all bets are off and you can't expect anything sensible to happen.
If you want to read lines then you might want to switch to fgets or keep using read and figure out where the EOLs are yourself.