I am having an issue with recv. I wrote a function that fills a structure with data, and the length (in bytes) of that data which is read from a socket.
For testing I am just printing the data to stdout byte by byte based on the total amount of bytes read by recv. For some reason the number of bytes being read seems to be correct sometimes and incorrect other times depending on what site I am querying. For example the following code works as intended on some sites:
data->data_sz = 0;
while((i = recv(sock, data->data + data->data_sz, CHUNKSIZE, 0)) > 0)
{
data->data_sz += i;
if(databff - data->data_sz < CHUNKSIZE)
{
databff *= 2;
if(!(tmp = realloc(data->data, databff)))
{
free(data->data);
(void) WSACleanup();
return 0;
}
data->data = tmp;
}
}
i = strsbstr(data->data, "\r\n\r\n") + 4; //i = the position of the first char after header info
if(i >= 0)
{
data->data_sz = data->data_sz - i; //data->data_sz = number of bytes without header info
memmove(data->data, data->data + i, data->data_sz);
if(!(tmp = realloc(data->data, data->data_sz)))
{
free(data->data);
(void)WSACleanup();
return 0;
}
data->data = tmp;
}
else
{
free(data->data);
(void) WSACleanup();
return 0;
}
return 1;
}
To print the data to stdout I just use a for loop:
//t_html->data_sz points to my data->data_sz structure
//t_html->data points to my data->data structure
for(i = 0; i <= t_html->data_sz; i++) (void)fputc((int)t_html->data[i], stdout);
The above code works for some sites but fails on others (for example when querying http://www.google.com I expect the final characters to be </html> but I get </html>l).
Basically my problem is that data->data_sz (the amount of bytes received) is not being calculated correctly, which makes it impossible to correctly use gathered data. I am really at a loss of what to do right now.
EDIT:
here is the strsbstr function which is called in the above code:
int strsbstr(const char *str, const char *sbstr)
{
char *sbstrlc;
if(!(strcmp(str, sbstr))) return 0;
if(!(sbstrlc = strstr(str, sbstr))) return -1;
return (int) (sbstrlc - str);
}
recv(sock, data->data + data->data_sz, CHUNKSIZE, 0) is potentially a problem. Why? Because you may not have CHUNKSIZE room left in your buffer. You have databff - data->data_sz left, actually (assuming data is allocated to a size of databff). It all depends on the initial values of databff and CHUNKSIZE, which I can't see and figure I'd point this out just in case.
Data is not guaranteed to be NULL-character terminated. Your printing loop says i <= t_html->data_sz; which is wrong. It should be i < t_html->data_sz;. If you use <=, you're accessing one past your buffer, which is likely why you get a weird character sometimes, and sometimes not.
Related
I have a python tcp server that accepts connections and generates a random string of length between (0,1M) characters, on the other side I have a c client that needs to listen on that socket and read the string and convert it into a single char of the same length as the string returned by the server
int receiver(int soc_desc, char * buffer)
{
char *arr = (char *)malloc(sizeof(char));
unsigned int received , total_received;
while (1)
{
memset(arr, 0, MAX); // clear the buffer
if ( received = recv(soc_desc, arr , MAX, 0) < 0)
{
break;
}
else
{
total_received += received;
}
}
printf("%s\n",arr);
return received;
}
// soc_desc is the socket descriptor
// buffer is the buffer that will hold the final output
The only way that I can think of is using malloc to read chunks of the data returned from the server but I am having bad time trying to figure it out and I need to convert the array of char pointers into a single char when the client is done receiving data from the server
Reassembling network data, particularly from TCP, can get tricky. The following code is untested and surely doesn't account for all contingencies, but hopefully is down the right path of what you need to do.
ssize_t receiver(int soc_desc, char * buffer)
{
// Whats the buffer argument used for?
// what you had before only allocated space for 1 char. That's not what you want
// This allocates for MAX+1 chars (I'm doing +1 for a NUL terminator)
char *arr = malloc(MAX+1);
// if MAX is small enough, you could do
// char arr[MAX+1];
// 0 buffer. You could use calloc instead of malloc + memset
memset(arr, 0, MAX+1);
// initialize total_received to 0
ssize_t received , total_received = 0;
size_t spaceLeftInBuf = MAX;
while (1)
{
// don't memset here, you'll erase the data you received last iteration
// write data to arr+total_receieved. This way you won't overwrite what
// you received the last iteration
received = recv(soc_desc, arr+total_received, spaceLeftInBuf, 0);
if (received < 0)
{
// there was an error
perror("recv failed: ");
// do something with the data already received? Ok, break and
// print what we've got
break;
}
else if (received == 0)
{
// socket closed gracefully, suppose we can break again and print
// what we've got
break;
else
{
// update counters
total_received += received;
spaceLeftInBuf -= received;
// is our buffer full? This may not be the right check, you need to
// decide when to process the data
// total_received better not ever be > MAX...
if (total_received >= MAX)
{
// "process" the data by printing it
printf("%s\n", arr);
// reset
total_received = 0;
spaceLeftInBuf = MAX;
// not particularly necessary to reset this to all 0s, but should
// make sure printing goes smoothly if we break out of this loop
memset(arr, 0, MAX); // arr[MAX] should already be '\0' from above
}
}
}
printf("%s\n",arr);
return received;
}
See Do I cast the result of malloc?
I found a way to do it but this is not tested enough and for sure will cause memory issues
char *arr = malloc(sizeof(char));
char tmp_buff[MAX];
memset(arr,0,MAX);
while (recv(soc_desc, tmp_buff , MAX, 0) > 0 )
{
strcat(arr , tmp_buff);
printf("Size : %ld arr : %s\n",strlen(tmp_buff),tmp_buff);
}
this is my code:
#include <stdlib.h>
#include <stdio.h>
int sendMessage(uint8_t *pui8MsgData, int messageLength, uint32_t ui32ObjID)
{
int total = 0;
int bytesleft = messageLength;
int n;
int chunkSize;
while (bytesleft)
{
chunkSize = bytesleft > sizeof(uint8_t)*8 ? sizeof(uint8_t)*8 : bytesleft;
uint8_t *buffer = (uint8_t *)malloc(sizeof(uint8_t) * chunkSize);
if(buffer == NULL)
{
printf("Memory allocation failed");
return 0;
}
memcpy(buffer, pui8MsgData, sizeof(uint8_t) * chunkSize);
n = send(buffer, chunkSize, ui32ObjID);
total += n;
bytesleft -= n;
}
return 1;
}
but for some reason, the malloc always return NULL.. what could be wrong? or how to get the error which is returned by malloc?
It is not possible to to tell you what is wrong here with 100% certainty; there's too little information.
However, the malloc() seems pointless, and you never free() it. This is a memory leak, which might explain why you run out of memory, causing malloc() to return NULL. Seems plausible to me.
Just pass the data directly to send(), no need to allocate a new buffer and copy data around.
Edit: also, you never update pui8MsgData so you're processing the first bytes of the message over and over.
So, to summarize, the loop should be something like:
while (bytesleft)
{
const chunkSize = bytesLeft > 8 ? 8 : bytesLeft;
const ssize_t n = send(ui32ObjID, pui8MsgData + total, chunkSize);
if (n < 0)
{
fprintf(stderr, "Send() failed\n");
return 0;
}
total += n;
bytesLeft -= n;
}
This fixes the problem by removing malloc(). I also swapped the arguments to send(), assuming ui32ObjID is a valid file descriptor.
You are using buffer as the first argument to send(). But the send() function expects a file descriptor, not some uint8_t * so send() will likely return -1. This results in an ever increasing value for bytesleft, and thus an infinite loop with infinite memory allocations, eventually returning NULL.
my task is get from server data of an file with HTTP protocol
but problem is that i dont know the size of the content so i iterate through read func to get data from socked but it iterates every time only once i dont know really why
int res, len, total, boolk = 0, p=0;
while ((res = read(client_socket,bufferOut,4095)) > 0)
{
bufferOut[res]= '\0';
if(p==0)
{
buffer = calloc(strlen(bufferOut)+1,sizeof(char));
}
else
{
buffer = realloc(buffer,strlen(buffer)*sizeof(char)+strlen(bufferOut)+1*sizeof(char));
}
strcat(buffer,bufferOut);
if(isEnough(bufferOut)!=0 && boolk == 0)
{
index = getIndex(buffer);
kk = getCode(buffer);
len = getLen(buffer);
boolk = 1;
}
if(strlen(bufferOut)>=len+index && boolk == 1)
{
break;
}
p++;
}
function isEnough only looks if full http header arrived getIndex get lenght of header and getLen get length of that file so i wanted to iterate until buffer is same lenght as header file which should be
You're working much too hard.
int
read_until_enough(int client_socket, char **buf_out, size_t *buflen_out)
{
char *rdbuf = xmalloc(INITIAL_BUFFER_SIZE);
size_t buflen = 0, bufalloc = INITIAL_BUFFER_SIZE;
for (;;) {
ssize_t n = recv(client_socket, rdbuf + buflen, bufalloc - buflen);
if (n <= 0) goto recv_failure;
if (is_enough(rdbuf, buflen)) break;
buflen += n;
if (buflen == bufalloc) {
bufalloc *= 2;
rdbuf = xrealloc(rdbuf, bufalloc);
}
}
*rdbuf_out = rdbuf;
*buflen_out = buflen;
return 0;
recv_failure:;
int save_errno = errno;
free(rdbuf);
errno = save_errno;
*rdbuf_out = 0;
*buflen_out = 0;
return -1;
}
Please note that is_enough now takes the size of the received data as an argument, caller gets both the received data and its size, and the buffer is NOT nul-terminated. HTTP is a binary-transparent protocol; you MUST NOTrfc2119 assume that you never receive nul bytes.
Please also note that is_enough is stateless and is expected to do all of the parsing. This is better separation of concerns. Worry about the performance cost of re-parsing the HTTP headers each time around the loop only after you have everything working.
Finally, the functions xmalloc and xrealloc are wrappers around malloc and realloc that either succeed or crash the program. I am using them here to avoid cluttering up the example with error-recovery logic for memory allocation. You will have to decide whether they are appropriate for whatever you're doing.
Need help in getting the following to work.
I have a multiple producer threads (each writing say 100 bytes of data) to ringbuffer.
And one single reader(consumer) thread ,reads 100 bytes at a time and writes to stdout.(Finally i want to write to files based on the data)
With this implementation ,I get the data read from ring buffer wrong sometimes. see below
Since the ringbuffer size is small it becomes full and some part of data is loss.This is not my current problem.
** Questions:
On printing the data thats read from ringbuffer ,some data gets
interchanged !!I'm unable to find the bug.
Is the logic/approach correct ? (or) Is there a
better way to do this
ringbuffer.h
#define RING_BUFFER_SIZE 500
struct ringbuffer
{
char *buffer;
int wr_pointer;
int rd_pointer;
int size;
int fill_count;
};
ringbuffer.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "ringbuffer.h"
int init_ringbuffer(char *rbuffer, struct ringbuffer *rb, size_t size)
{
rb->buffer = rbuffer;
rb->size = size;
rb->rd_pointer = 0;
rb->wr_pointer = 0;
rb->fill_count = 0;
return 0;
}
int rb_get_free_space (struct ringbuffer *rb)
{
return (rb->size - rb->fill_count);
}
int rb_write (struct ringbuffer *rb, unsigned char * buf, int len)
{
int availableSpace;
int i;
availableSpace = rb_get_free_space(rb);
printf("In Write AVAIL SPC=%d\n",availableSpace);
/* Check if Ring Buffer is FULL */
if(len > availableSpace)
{
printf("NO SPACE TO WRITE - RETURN\n");
return -1;
}
i = rb->wr_pointer;
if(i == rb->size) //At the end of Buffer
{
i = 0;
}
else if (i + len > rb->size)
{
memcpy(rb->buffer + i, buf, rb->size - i);
buf += rb->size - i;
len = len - (rb->size - i);
rb->fill_count += len;
i = 0;
}
memcpy(rb->buffer + i, buf, len);
rb->wr_pointer = i + len;
rb->fill_count += len;
printf("w...rb->write=%tx\n", rb->wr_pointer );
printf("w...rb->read=%tx\n", rb->rd_pointer );
printf("w...rb->fill_count=%d\n", rb->fill_count );
return 0;
}
int rb_read (struct ringbuffer *rb, unsigned char * buf, int max)
{
int i;
printf("In Read,Current DATA size in RB=%d\n",rb->fill_count);
/* Check if Ring Buffer is EMPTY */
if(max > rb->fill_count)
{
printf("In Read, RB EMPTY - RETURN\n");
return -1;
}
i = rb->rd_pointer;
if (i == rb->size)
{
i = 0;
}
else if(i + max > rb->size)
{
memcpy(buf, rb->buffer + i, rb->size - i);
buf += rb->size - i;
max = max - (rb->size - i);
rb->fill_count -= max;
i = 0;
}
memcpy(buf, rb->buffer + i, max);
rb->rd_pointer = i + max;
rb->fill_count -= max;
printf("r...rb->write=%tx\n", rb->wr_pointer );
printf("r...rb->read=%tx\n", rb->rd_pointer );
printf("DATA READ ---> %s\n",(char *)buf);
printf("r...rb->fill_count=%d\n", rb->fill_count );
return 0;
}
At the producer you also need to wait on conditional variable for the has empty space condition. The both conditional variables should be signaled unconditionally, i.e. when a consumer removes an element from the ring buffer it should signal the producers; when a producer put something in the buffer it should signal the consumers.
Also, I would move this waiting/signaling logic into rb_read and rb_write implementations, so your ring buffer is a 'complete to use solution' for the rest of your program.
As to your questions --
1. I can't find that bug either -- in fact, I've tried your code and don't see that behavior.
2. You ask if this is logic/approach correct -- well, as far as it goes, this does implement a kind of ring buffer. Your test case happens to have an integer multiple of the size, and the record size is constant, so that's not the best test.
In trying your code, I found that there is a lot of thread starvation -- the 1st producer thread to run (the last created) hits things really hard, trying and failing after the 1st 5 times to stuff things into the buffer, not giving the consumer thread a chance to run (or even start). Then, when the consumer thread starts, it stays cranking for quite some time before it releases the cpu, and the next producer thread finally starts. That's how it works on my machine -- it will be different on different machines, I'm sure.
It's too bad that your current code doesn't have a way to end -- creating files of 10's or 100's of MB ... hard to wade through.
(Probably a bit later for the author, but if anyone else searches for a "multiple producers single consumer")
I think the fundamental problem in that implementation is what rb_write modifies a global state (rb->fill_count and other rb->XX) w/o doing any synchronization between multiple writers.
For alternative ideas check the: http://www.linuxjournal.com/content/lock-free-multi-producer-multi-consumer-queue-ring-buffer.
I am new to C programming and I am getting confused with the pointer math. I have an array of characters of size 32. It is my understanding that this means that the array is also 32 bytes since a character variable is 1 byte big therefore 32 characters * 1 byte = 32 bytes. The problem is when having a function that has a void pointer that is pointing to an array of characters as described before. I believe that the code segment
for (count = 0; count < size; count++)
*((int*) raw_sk + count) = 0
should set all of the slots in the raw_sk buffer should be set to 0. However, when I run the program, I get a segmentation fault. I thought that it could be possibly be the fact that I am adding count to the address. I thought that if I were to add one to an address I would be moving to the next slot in the array. Can someone please point out where I am going wrong? The function I am using is below.
Thanks!
void
write_skfile (const char *skfname, void *raw_sk, size_t raw_sklen)
{
int fdsk = 0;
char *s = NULL;
int status = 0;
int count = 0;
int size = (raw_sklen);
/* armor the raw symmetric key in raw_sk using armor64 */
s = armor64(raw_sk, raw_sklen);
/* now let's write the armored symmetric key to skfname */
if ((fdsk = open (skfname, O_WRONLY|O_TRUNC|O_CREAT, 0600)) == -1) {
perror (getprogname ());
/*scrubs the armored buffer*/
for(count = 0; count < armor64len(s); count++)
s[count] = '0';
free (s);
/* scrub the buffer that's holding the key before exiting */
for (count = 0; count < size; count++)
*((int*)raw_sk + count) = 0;
exit (-1);
}
else {
status = write (fdsk, s, strlen (s));
if (status != -1) {
status = write (fdsk, "\n", 1);
}
for (count = 0; (size_t)count < 22; count++)
*((int*)raw_sk + count) = 0;
free (s);
close (fdsk);
/* do not scrub the key buffer under normal circumstances
(it's up to the caller) */
if (status == -1) {
printf ("%s: trouble writing symmetric key to file %s\n",
getprogname (), skfname);
perror (getprogname ());
/* scrub the buffer that's holding the key before exiting */
/* scrub the buffer that's holding the key before exiting MY CODE
for (count = 0; count < size; count++)
*((int*)raw_sk + count) = 0;*/
exit (-1);
}
}
}
You are incrementing the pointer by the size of an int. That is wrong. If you want to zero out the array you increment by the size of a char. Better yet, just use memset.
Your loop iterates over size*sizeof(int) bytes in total (where most probably sizeof(int)==4), but the array is only size bytes large. Hence, segmentation fault.
I think you meant to do
*((char*) raw_sk + count) = 0
since I assume raw_sk is pointing to char array
pointer arithmatic works by moving the memory address by size of type so in this case you want char