I am trying to write a program that will read up to N bytes at a time from a file, store that in a buffer, and send that buffer to a server. This will repeat until all bytes have been read.
My problem is that it successfully sends the first N bytes, but for the remaining X bytes it overwrites up to X bytes of the original N bytes and sends that along to the server, so that the total bytes received at the end of the program are 2*N instead of N+X.
For example if I have a message and reads N bytes (in this example reads the first three lines):
1
2
3
4
The message received by the server is:
1
2
3
4
2
3
Any help as to why this is happening is appreciated!
Relevant client code:
// send data
ssize_t remaining;
char buffer[SEND_BUFFER_SIZE];
int count = 1;
// do this if there are still bytes to read
while(read(0, buffer, sizeof(buffer)) > 0) {
printf("%s\n", buffer);
// count = read(0, buffer, sizeof(buffer));
send(sock, buffer, sizeof(buffer), 0);
// char buffer[SEND_BUFFER_SIZE];
}
// close socket
close(sock);
Relevant server code:
// receive message
char buffer[RECV_BUFFER_SIZE];
int recv_bytes = RECV_BUFFER_SIZE;
int i = 1;
int count;
// while (1 != 2) {
while(recv_bytes != 1) {
recv_bytes = recv(sock, buffer, RECV_BUFFER_SIZE, 0);
fwrite(buffer, recv_bytes, 1, stdout);
fflush(stdout);
if (recv_bytes < RECV_BUFFER_SIZE) {
break;
}
}
Related
Hi I am trying to implement TCP socket stream. I have a buffer of size 12, and I am reading into this buffer in a loop.
I expected the read operation to add onto the buffer with each read call until the buffer is full, but instead, it starts from the beginning of the buffer, overwriting what is already in there on each iteration.
How can I append to the buffer with each read, so that the previous bytes are preserved?
void *th_read(void *arg)
{
reader_arg_t *rArg = (reader_arg_t *)arg;
int buf_size = 12;
int bytes_recevied, rv;
char buf[buf_size];
while (1)
{
rv = 0;
bytes_recevied = rv;
memset(buf, 0, buf_size);
socket_context_t ctx;
tsq_dequeue(rArg->reader_queue, &ctx);
printf("reading from socket: %i\n", ctx.sockfd);
while (1)
{
rv = read(ctx.sockfd, buf, buf_size - bytes_recevied);
if (rv < 1)
break;
bytes_recevied += rv;
printf("bytes_recevied: %i\nbuf: %s\n", bytes_recevied, buf);
}
perror("read");
close(ctx.sockfd);
printf("%s\n", buf);
}
}
When I connect with telnet and write 2 times separates by pressing enter, I get this output. Writing hello the first time and the digit 1 the second time.
reading from socket: 5
bytes_recevied: 7
buf: hello
bytes_recevied: 10
buf: 1
lo
I want the buffer to contain hello1 instead of 1\rlo.
I found a question that showed the zero byte thing but its not useful for my use case unless I would maybe use a second buffer and add on each read the stuff from the first buffer until the zero byte.
reading buffer from socket
You should send new position of buffer into read function.
rv = read(ctx.sockfd, buf + bytes_received, buf_size - bytes_recevied);
I wrote function for read and write for a tcp program . I output in server side but I can't get read on client side . my code
read function :
int read_data (int sd , char **data_buf)
{
int in_length,length,size,bytesread;
char *temp_buf;
size = read(sd,&in_length,sizeof(in_length));/*send entire length of data*/
if( 0 > size )
{
printf("Error on reading from socket\n");
exit(1);
}
length = ntohl(in_length);
printf("Total length coming : %d\n",length);
*data_buf =(char *)malloc((length+1)*sizeof(char));
temp_buf =(char *)malloc((length+1)*sizeof(char));
while(length> 0)
{
bytesread = read(sd,temp_buf,4);
strcat(*data_buf,temp_buf);
temp_buf = temp_buf + bytesread;
length = length - bytesread;
}
return 1;
}
and my write functions as :
int write_data (int sd , char *buffer)
{
int length,len_buff,bytesread,size;
len_buff = strlen(buffer);/*total length of string*/
printf("string == %s\n",buffer);
length = htonl(len_buff);/*convert to host to n/w*/
printf("Total length send =%d\n",len_buff);
size = write(sd,&length,sizeof(length));/*write total size to server */
if( 0 > size)
{
printf("error\n");
exit(0);
}
while(length > 0)
{
bytesread = write(sd,buffer,4);/*write 4 bytes to server*/
buffer = buffer + bytesread;
length = length - bytesread;
}
return 1;
}
client program :
///.............code for socket and connections.................//
ret = write_data(sd,user_string);/*write entire datas to server*/
value_from_server = read_data(sd,&data_buf);
server side program :
value_from_client = read_data(connfd,&data_buf);
printf("the value from client : %s\n",data_buf);
index = string_function(data_buf,&store_buf);
printf("after string process : %s\n",store_buf);
write_data(connfd,store_buf);
printf("i am waiting for next string\n");
connfd is the new socket for communication with client . reading and writing function work perfectly on server side . writing function work on client side . but reading from server not work in client program . ant mistake on my code ?
bytesread = read(sd,temp_buf,4);
Why read 4 bytes always inside the loop? You should be reading the remaining number of bytes to be read. The socket is blocking and hence will be stuck if the server is done sending but client still tries reading 4 bytes to arrive in the last iteration.
Have print statements inside the loop to know the bytes read in each iteration and see if client is blocked with read
Your code has several logic errors.
size = read(sd,&in_length,sizeof(in_length));/*send entire length of data*/
if( 0 > size )
{
printf("Error on reading from socket\n");
exit(1);
}
length = ntohl(in_length);
Here you are assuming you read four bytes, rather than fewer, or end of stream. You must check for end of stream (zero return value), and you must loop until you get the four bytes.
while(length> 0)
{
bytesread = read(sd,temp_buf,4);
strcat(*data_buf,temp_buf);
temp_buf = temp_buf + bytesread;
length = length - bytesread;
}
Here again you are ignoring the possibility of end of stream or an error. It should be:
while ((bytesread = read(sd,temp_buf, length)) > 0)
{
temp_buf += bytes_read;
length -= bytesread;
}
if (bytesread < 0)
{
perror("read 2");
}
else if (length > 0)
{
// end of stream before all expected bytes were received ...
}
else
{
// The OK case
}
Your sending code is suboptimal:
while(length > 0)
{
bytesread = write(sd,buffer,4);/*write 4 bytes to server*/
buffer = buffer + bytesread;
length = length - bytesread;
}
There's no point in chunking into 4-byte writes. It should be:
while (length > 0)
{
bytesread = write(sd, buffer, length);
buffer = buffer + bytesread;
length = length - bytesread;
}
and of course the misnamed bytesread variable should be called byteswritten. In fact you can rely on this loop only executing once. Again it should be followed by a test of byteswritten == -1 to check for errors.
Your functions have logic errors in them.
The reading loop is reading exactly 4 bytes on each iteration. If the length of the data being read is not an even multiple of 4, read() will block on the last iteration waiting for data that does not arrive. The reading loop is also assuming that read() returns a null-terminated buffer, but that is not the case, so strcat() will attempt to copy data from surrounding memory and will either copy garbage or crash with a segfault. Also, the reading function is not null-terminating the data buffer it returns to the caller, but the caller assumes it is null-terminated.
The writing loop is writing exactly 4 bytes on each iteration. If the length of the data is not an even multiple of 4, write() will attempt to write data from surrounding memory on the last iteration, and will either send garbage or crash with a segfault.
You are also not doing adequate error handling in either function.
Try something more like this instead:
void read_raw_bytes (int sd, void *data, int length)
{
int bytes_read;
char *data_ptr;
data_ptr = (char*) data;
while( length > 0 )
{
bytes_read = read(sd, data_ptr, length);
if( bytes_read < 0 )
{
printf("Error on reading from socket\n");
exit(1);
}
if( bytes_read == 0 )
{
printf("Disconnected while reading from socket\n");
exit(1);
}
data_ptr += bytes_read;
length -= bytes_read;
}
}
void write_raw_bytes (int sd, void *data, int length)
{
int bytes_sent;
char *data_ptr;
data_ptr = (char*) data;
while( length > 0 )
{
bytes_sent = write(sd, data_ptr, length);
if( bytes_sent < 0 )
{
printf("Error on writing to socket\n");
exit(0);
}
data_ptr += bytes_sent;
length -= bytes_sent;
}
}
int read_data (int sd, char **data_buf)
{
int length;
read_raw_bytes (sd, &length, sizeof(length)); /*send entire length of data*/
length = ntohl(length);
printf("Total length coming : %d\n", length);
*data_buf = (char *) malloc((length+1)*sizeof(char));
if (*data_buf == NULL)
{
printf("Error on allocating memory\n");
exit(1);
}
read_raw_bytes (sd, *data_buf, length);
(*data_buf)[length] = 0;
return 1;
}
int write_data (int sd, char *buffer)
{
int length, len_buff;
len_buff = strlen(buffer); /*total length of string*/
printf("string == %s\n", buffer);
printf("Total length send =%d\n", len_buff);
length = htonl(len_buff); /*convert to host to n/w*/
write_raw_bytes (sd, &length, sizeof(length)); /*write total size to server */
write_raw_bytes (sd, buffer, len_buff);
return 1;
}
My code is too long to post all here so i'm going to sum up what's wrong.
In a server part i'm sending on a socket 3 things :
A message
The content of a file
Another message
In a client part i'm receiving these things but :
This first is to print on terminal
The second to write in a new file
The last to print on the terminal too
But my client is stuck on a read and i really don't know why. I'm on the problem for hour so if someone can help me, it will be very great !
edit : Basically, i think my problem is that i don't know what to write on the server to stop the read on the client.. Is it \n, \0.. ?
Here's the 2 part of code :
server
void send_content(t_server *s, FILE *fd, int rfd)
{
int len;
char *buff;
write(s->socket, "150 File status okay;" \
"about to open data connection.\n\0", strlen("150 File status okay;about to open data connection.\n\0"));
fseek(fd, 0, SEEK_END);
len = ftell(fd);
buff = malloc(len * sizeof(char));
read(rfd, buff, len);
write(s->socket, buff, len);
write(s->socket, "\n\0", strlen("\n\0"));
write(s->socket, "226 Closing data connection.\n\0", strlen("226 Closing data connection.\n\0"));
free(buff);
}
client
void getfile(t_client *c, char **tab)
{
int ret;
int fd;
int z;
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
if (strlen(buff) < 25)
return ;
fd = creat(tab[1], S_IRUSR | S_IWUSR);
while (z == 0 && (ret = read(c->fd, buff, 4096)) > 0)
{
if (ret < 4096)
z = -1;
write(fd, buff, strlen(buff));
memset(buff, '\0', 4096);
}
read(c->fd, buff, 4096); // Stuck here
write(1, buff, strlen(buff));
close(fd);
}
Like noted you need a read function like this to make sure you receive
specified number of bytes(this function will loop till it receives number of bytes it was told to). Just use this receivall method instead of read everywhere.
With files you typically first send the file length, and then receive the file.
I did something similar while ago, hope it will help you a bit. This is the client side, which tries to receive first file length from the server, then the file:
/* create file */
FILE * ptrMyFile = fopen(&filenames[i][0],"wb");
if(NULL == ptrMyFile)
{
printf("Unable to open file \n");
return 1;
}
int size = 0;
int t = 4;
/* first receive file size from server */
/* NOTE: error checking is omitted from code, nevertheless users should stil do some error checking when using this code */
readall(sockfd, (unsigned char*) &size, &t);
/* how many 256 byte chunks are there? */
int div = size / 256;
/* loop to receive each chunk. */
for(int k = 0; k < div; k++)
{
int chunk_size = 256;
/* make sure we receive 256 bytes */
readall(sockfd, buffer, &chunk_size);
/* write to file */
fwrite(buffer, chunk_size, 1, ptrMyFile);
}
/* read the final chunk. */
int whatsleft = size - 256 * div;
readall(sockfd, buffer, &whatsleft);
/* write */
fwrite(buffer, whatsleft, 1, ptrMyFile);
/* close file */
fclose(ptrMyFile);
I leave the server part to you.
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
You should be saving the return value of the call to read(), in order to find out how many bytes you just received. You may have to make several calls to read() in order to get the entire message. It's wrong to use strlen() to find out how many bytes were received, because the buffer contents are uninitialized, and the first chunk of the message could be cut off anywhere, so you can't count on it being null-terminated.
I'm trying to implement a working HTTP Client-Server application just to make practice with network programming.
The 2 programs have to follow this basic algorithm:
CLIENT - send a GET request
SERVER - send "+OK\r\n"
SERVER - send file size in bytes
SERVER - send file
CLIENT - send ACK
I'm having a lot of troubles in the reading part, probably because i perform some dirty read on the stream.
These are the 2 reading function that i'm using:
/* Reads a line from stream socket s to buffer ptr
The line is stored in ptr including the final '\n'
At most maxlen chasracters are read*/
int readline (SOCKET s, char *ptr, size_t maxlen)
{
size_t n;
ssize_t nread;
char c;
for (n=1; n<maxlen; n++)
{
nread=recv(s, &c, 1, 0);
if (nread == 1)
{
*ptr++ = c;
if (c == '\n')
break;
}
else if (nread == 0) /* connection closed by party */
{
*ptr = 0;
return (n-1);
}
else /* error */
return (-1);
}
*ptr = 0;
return (n);
}
and:
int readNumber(SOCKET s, long *num, int maxRead)
{
size_t n;
ssize_t nread;
int totRead;
long number=0;
for (n=1; n<maxRead+1; n++)
{
nread=recv(s, &number, sizeof(number), 0);
if (nread == sizeof(number))
{
totRead+=nread;
*num = number;
}
else if (nread == 0) /* connection closed by party */
{
*num = 0;
return (n-1);
}
else /* error */
{
printf("nread = %d\n", nread);
return (-1);
}
}
return (totRead);
}
this is the snippet of the main where i receive the +OK message and then the file size:
memset(rbuf,0,sizeof(rbuf)); //rbuf is the buffer where is store the read
printf("waiting for response...\n");
result = readline(s, rbuf, sizeof(rbuf)); //reading function is above
printf("Byte read(okMsg) = %d\n", result);
if (result <= 0)
//ERROR MANAGEMENT
{
printf("Read error/Connection closed\n");
closesocket(s);
SockCleanup();
exit(1);
}
else
{
long fileLength=0;
unsigned char *fBuf;
//RECEIVE OK
if(!strcmp(rbuf,"+OK\r\n"))
{
puts("+OK\n");
//RECEIVE FILE LEN
int nw = readNumber(s, &fileLength, 1); //reading function is above
printf("Byte read(fDim) = %d\n", nw);
printf("File is %ld bytes long\n", fileLength);
if(nw >0)
{
// RECEIVE FILE
}
}
}
When i send the "+OK\r\n" string the server tells me that it sends 8 bytes, but when i read i find the '\0' char only after 6 bytes.
By the way it reads correctly the message, but when i try to read the file size (that is a long) it gives me back a wrong number.
My opinion is that the stream buffer is dirty, and that i'm reading 2 bytes that are not part of the file size, but i'm not understanding why this happens.
Please ask me more info if i'm not clear enough.
SOLVED:
Thank you all for your answers!!!
You put me in the right mindset to understand what was wrong.
Look like the problem was this declaration in the server:
char *okMsg = "+OK\r\n";
instead of
char okMsg[] = "+OK\r\n";
that lead me to an undefined behavior.
long number=0;
for (n=1; n<maxRead+1; n++)
{
nread=recv(s, &number, sizeof(number), 0);
You forgot to design and implement a protocol to carry the data between your server and your client. Because TCP provides a stream of bytes, your protocol should be defined as a stream of bytes.
How many bytes convey this number? Is "however many bytes a 'long' happens to occupy on my platform" a good answer? What's the semantic meaning of the first byte? Is "whatever the first byte of a 'long' happens to mean on my platform" a good answer?
A good answer would be, "The size shall be conveyed as a 4-byte unsigned integer in little-endian byte order". Then make absolutely sure your code sends and receives in that format.
I'm having some problems with two programs I wrote, a server and a client. To keep it easy and chronological, I first wrote the server and tested it with both telnet and netcat and everything works fine (except for a difference in the return value of read() / recv(), as it looks like the normal telnet program adds an additional char at the end of the string to be sent, but anyway...).
Now I have written the client program too, but I am not getting all the data which was correctly received by the other two clients, in particular the row[i] strings I get from the MySQL query. Things change when I introduce a usleet() call after each send() function and all data is received correctly.
Now I was thinking about an incompatible buffer size problem (?), but after playing for a while and checking the dimensions, I was not able to find out anything.
You will find the code below, please if you have any advice don't hesitate to tell me.
Tnx
/* CLIENT CODE */
#define BUFSIZE 1000
...
void *send_handler(void *);
...
int main(int argc, char *argv[]) {
char buf[BUFSIZE];
...
socket stuff...
...
connect
...
/* receive string from server - working */
bzero(buf, BUFSIZE);
n = read(sockfd, buf, BUFSIZE);
if (n < 0)
error("ERROR reading from socket");
buf[n] = '\0';
printf("%s", buf);
...
/* send username to server - working */
bzero(buf, BUFSIZE);
fgets(buf, BUFSIZE, stdin);
n = write(sockfd, buf, strlen(buf));
if (n < 0)
error("ERROR writing to socket");
...
/* start receiving handler */
if( pthread_create( &thread_id , NULL , send_handler , (void*) &sockfd) < 0) {
perror("could not create thread");
return 1;
}
/* main thread for reading data */
while(1) {
bzero(buf, BUFSIZE);
n = read(sockfd, buf, BUFSIZE);
if (n < 0)
error("ERROR reading from socket");
buf[n] = '\0';
printf("%s", buf);
}
close(sockfd);
return 0;
}
void *send_handler(void *socket_desc) {
//Get the socket descriptor
int sock = *(int*)socket_desc;
char buf[BUFSIZE];
int n;
while (1) {
bzero(buf, BUFSIZE);
fgets(buf, BUFSIZE, stdin);
n = write(sock, buf, strlen(buf));
if (n < 0)
error("ERROR writing to socket");
}
}
/* SERVER CODE */
void *connection_handler(void *);
int main(int argc , char *argv[]) {
...
/* socket variables */
...
pthread_t thread_id;
...
socket stuff...
...
while((client_sock = accept(socket_desc, (struct sockaddr *)&client_addr, (socklen_t*)&client_len))) {
if( pthread_create( &thread_id , NULL , connection_handler , (void*) &client_sock) < 0) {
perror("could not create thread");
return 1;
}
}
return 0;
}
void *connection_handler(void *socket_desc) {
//Get the socket descriptor
int sock = *(int*)socket_desc;
...
/* mysql variables */
char cmd[1000];
...
MYSQL_RES *result;
MYSQL_ROW row;
MYSQL *con;
...
/* connection variables */
int read_size, i;
char *message;
char client_message[2000];
char buffer[1000];
...
//clear the buffers
memset(client_message, '\0', 2000);
...
snprintf(cmd, 999, "SELECT field1, field2, field3, field4 FROM files WHERE key='%s' ORDER BY id DESC", var);
if (mysql_query(con, cmd)) {
error checks...
}
result = mysql_store_result(con);
if (result == NULL) {
error checks...
}
else {
num_rows = mysql_num_rows(result);
}
if (num_rows == 0) {
message = "Nothing found\n";
send(sock , message , strlen(message), 0);
}
else {
num_fields = mysql_num_fields(result);
num_rows = mysql_num_rows(result);
snprintf(buffer, 999, "Number of rows: %d\n", num_rows);
send(sock , buffer , sizeof(buffer), 0);
//usleep(10000); // commented, but necessary to work properly...
memset(buffer, '\0', sizeof(buffer));
while ((row = mysql_fetch_row(result))) {
for(i = 0; i < num_fields; i++) {
snprintf(buffer, 999, "%s\t", row[i] ? row[i] : "NULL");
send(sock , buffer , sizeof(buffer), 0);
//usleep(10000);
memset(buffer, '\0', sizeof(buffer));
}
message = "\n";
send(sock , message , strlen(message), 0);
//usleep(10000);
}
message = "\n";
send(sock , message , strlen(message), 0);
//usleep(10000);
mysql_free_result(result);
}
...
}
EDIT: I changed
printf("%s", buf);
with
printf("Bytes read: %d\n", n);
in the client code and I obtained following output:
with commented usleep():
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 36
Bytes read: 1000
Bytes read: 31
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 2
(17 lines)
with usleep(0) slowing down the send flow (correct output obtained):
Bytes read: 1000
Bytes read: 33
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1
Bytes read: 1
Bytes read: 1000
Bytes read: 31
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1000
Bytes read: 1
Bytes read: 1
(21 lines)
Any hint?
SOLVED: just by replacing
sizeof(buffer);
with
strlen(buffer);
in the server part and everything works fine even without usleep(), the output is correct/complete.
Thanks anyway.
You can't assume that a single read reads an entire message. There are no messages in TCP, only bytes, and any given read may return as few as one byte, or the result of several writes at the peer all at once. You have to loop and parse.
You have a race condition in your code, you are passing a pointer to a local variable to the thread function. Since a filedescriptor (int) is not larger than a void pointer (I'm pretty sure that is guaranteed, but add an assertion nonetheless), you can also convert the descriptor value to a pointer instead of passing the address of the local filedescriptor:
int s = accept(...);
if(int e = pthread_create(.., &connection_handler, (void*)s, ..))
error(..);
BTW: pthread_create returns zero on success and otherwise an error code, which is not negative. But it's very unlikely that your code failed at that point.