I have a problem with receiving the correct values on the server-side of my client-server program.
header file included in both server and client:
#define CHUNK_SIZE 1024
#define ARR_LEN 3
client:
int uids[ARR_LEN] = {1994, 2423, 1222};
unsigned int uidlen = 0;
char uidbuffer[CHUNK_SIZE] = {0};
for(int i = 0; i < ARLL; i++)
{
uidlen = strlen(uids[i])+1;
snprintf(uidbuffer, uidlen, "%s", uids[i]);
if(send(socket, uidbuffer, strlen(uidbuffer), 0) < 0)
DIE("Write Error");
if(recv(socket, uidbuffer, sizeof(uidbuffer), 0) < 0)
DIE("Acknowledge Error");
memset(uidbuffer, 0, sizeof(uidbuffer));
}
server:
char uid_buff[CHUNK_SIZE];
for(int i = 0; i < ARR_LEN; i++)
{
memset(uid_buff, 0, sizeof(uid_buff));
// receiving the UID and storing it directly
if(recv(client_sock, uid_buff, sizeof(uid_buff), 0) < 0)
DIE("Receive Error");
printf("buffer content: %s\n", uid_buff);
uid_str = uid_buff;
uids[i] = (uid_t)strtol(uid_str, (char **)NULL, 10);
if(send(client_sock, uid_buff, sizeof(uid_buff), 0) < 0)
DIE("Acknowledge Error");
}
These are only parts of my program. I tried to only include the relevant parts. The output is this:
buffer content: 1994
buffer content: 24231222
buffer content:
While I want it to be:
buffer content: 1994
buffer content: 2423
buffer content: 1222
What could be the problem? I know that it's not so easy and that server–client communication is carried out in a stream of bytes rather than messages, but I want to mimmic that functionality by acknowledging every received "message". Could you please clue me what to do? I'm getting desperate.
You will need a protocol.
For example, you define that each message in your application has following format:
xx | message
this means first two bytes (mind the endianness) you receive indicate the length of the message that follows. Now you should first receive first two bytes - check the length - and then receive exactly that number of bytes. After that you know you have successfully received that message. And then you can proceed to other messages (which could/should have similar format: length + message itself).
Example:
Say you want to send three messages:
char s1[]="message1";
char s2[]="message2";
char s3[]="message3";
//You do this(client side):
int x1 = strlen(s1); // length of message1
int intsize = 4; // just size of integer -we'll need in next call
sendall(socket, &x1, &intsize); // send length of first message
sendall(socket, s1, &x1); // Now send the message
//On server:
int x = 0;
int y = 4; //size of integer most probably
receiveall(socket,&x,&y);//get length first; y=4 because that is size of integer
receiveall(socket, buffer, &x); // now we know how many bytes to expect - x - so request that number of bytes only
you can repeat this logic for other messages too.
Finally, you want to use such functions (here) instead of send and receive (because send and receive might not send/receive the number of bytes you tell it to):
int sendall(int s, char *buf, int *len)
{
int total = 0; // how many bytes we've sent
int bytesleft = *len; // how many we have left to send
int n;
while(total < *len) {
n = send(s, buf+total, bytesleft, 0);
if (n == -1) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually sent here
return n==-1?-1:0; // return -1 on failure, 0 on success
}
you will need a similar receiveall function.
You do not show the code that establishes the connection.
If you are using UDP sockets, each message is stand alone, you will receive a separate message for each message sent, but not necessarily in the same order.
If you are using TCP or Unix sockets, data may be received in chunks of different sizes than those used for sending. Only the sequence of bytes is preserved, not the chunk sizes. You need to specify a protocol: unless you specify a fixed number of bytes per message, you need to send some sort of separator to allow the server to tell where each message ends.
Related
I'm trying to send a jpg image from a client process to a server via sockets. The image contains binary data so I want to do it on a low-level programming basis using reads and writes. I'm also sending the image data in iterations of 100 bytes.
This is the code I've done, which is not sending the image identically as I want:
CLIENT
void send_image(char *path, char *filename, int socket) {
int fd = open(path, O_RDONLY); //I open the file of the image.jpg
int n = 1;
while (n > 0) {
char img_data[100];
n = read(fd, img_data, 100); //sending 100 bytes of image each iteration till n=0 (end of file)
if (!n) break;
int sending = 1;
write(socket, &sending, sizeof(int)); //Tell the client the image still has data to send
write(socket, img_data, strlen(img_data));
usleep(250);
}
sending = 0; //Tell the server the image has been fully sent
write(socket, &sending, sizeof(int));
close(fd);
}
SERVER
void receiving_image(char *path) {
int receiving = 0;
int j=0;
char *image_data = NULL; //Variable to store all the image data
read(socket, &receiving, sizeof(int)); //Reads that the client is going to send an image
while (receiving) {
char data[100]; //Variable that stores partial data (100 bytes) of an image on each iteration
read(socket, data, 100);
image_data = realloc(image_data, (j + strlen(data)) * sizeof(char)); //Readjust the size of the main image data.
for (int i=0; i<(int) strlen(data); i++) {
image_data[j] = data[i]; //copy the partial data of the image to the main variable of the image
j++;
}
j = (int) strlen(image_data);
read(socket, &receiving, sizeof(int)); //Read if the image is still sending
}
image_to_directory(path, image_data); //Copy image to directory
}
This compiles and runs fine, but when I check the directory on the server side where the image has been stored, I can see it's not the same image as the client has sent (I confirmed via md5sum and hashes are not equal).
Is there something I am missing?
You shouldn't be using strlen for your binary data length calculations. It is intended only for terminated strings (thus the name). You also have highly-ill-advised naked calls to read/write, which is a recipe for disaster when sending data over sockets.
You never seem to be sending more than 100 bytes at a time, which is helpful in this case to develop a more solid protocol. Consider this:
First octet is a uint8_t byte count N, and will be in 0..100.
Following the byte count, N bytes are transferred.
Repeat 1-2 until no more bytes remain.
Notify the server of EOF by sending a single zero-octet
An example of this sender code is shown here.
void send_image(const char *path, int socket)
{
int fd = open(path, O_RDONLY); //I open the file of the image.jpg
if (fd == -1)
return;
ssize_t n = 0;
do
{
// note the first octet will prefix the length
uint8_t img_data[101];
n = read(fd, img_data+1, 100);
if (n > 0)
{
// you never know just how many bytes are going to
// be sent, so setup the frame, but then ensure even
// piecewise deliver can succeed.
img_data[0] = (uint8_t)n;
ssize_t sent = 0;
size_t pos = 0;
do
{
sent = write(socket, img_data+pos, (n+1)-pos);
if (sent < 0)
break;
pos += sent;
} while ( pos < (n+1) && sent > 0);
}
} while (n > 0);
uint8_t done = 0;
write(socket, &done, sizeof done); // not much we can do if this fails
close(fd);
}
I make no claims the above code will even compile, but the concept should be fairly obvious. That's it, however. Obviously there is more that could/should be done (checksums, restart options, etc.), but that's the basic premise.
The server side can do something similar, which I leave as an exercise for you. The point of all of this is to utilize the return values from your read/write calls. They're there for a reason. If you find yourself coding a "naked" read or write (where you don't gather the result of function and utilize it in some way), chances are you've done something horribly wrong.
i am haveing trouble transferring int array over socket in c.
what is the correct use of htonl()?
lets say i have :
int arra[3]={6000,7000,8000};
and socket called new_socket
how do i transfer it correctly to the other end of socket?
client is reading it by the following code:
char buf[BUFLEN] = ""; // buffer for recv() calls
for (i = 0; i < nbytes; i += INTLEN) {
int file_port = ntohl(*(int *)&buf[i]);
int i, wp1, ret;
for (i = 0; i < 3; i++) {
wp1 = htonl(arr[i]);
ret = write(sockfd, wp1, sizeof(int));
//error check based on ret
}
this works. TCP is a stream, one write or multiple writes doesn't matter, you can read it all in one go or read until you have necessary number of bytes and do paste the entire program in either side and the output you are getting
I am trying to implement a web server in C. I am sending a response of a request using the send command in a loop.
buf_len = 1;
while (buf_len > 0)
{
buf_len = read(fh, out_buf, BUF_SIZE);
if (buf_len > 0)
{
send(client_s, out_buf, buf_len, 0);
}
}
Should I terminate the last character of by char out_buf[BUF_SIZE] with '\0'.
Also when I am trying to send a png file with the same code, Sometimes I receive the entire image and sometimes only half of it. It loads fully when I refresh the page.
For send function, see man page
RETURN VALUE
On success, these calls return the number of characters sent. On error, -1 is returned, and errno is set appropriately.
So it doesn't guarantee that buf_len bytes are sent out, you can check the return value.
As Hardy said above, send does not guarantee that buf_len bytes are sent out. So one of many ways to do it is
buf_len = 1;
while (buf_len > 0) {
buf_len = read(fh, out_buf, BUF_SIZE);
if(buf_len < 0) {
perror("read");
exit(1);
}
int sent = 0;
int count = buf_len;
while(count) {
int n = send(client_s, out_buf + sent, count, 0);
count -= n;
sent += n;
}
}
How is out_buf defined?
I currently have a client app that works but it is single threaded.
my packets look like this: < len_of_data>|< data>"
"|" is used as a separator for my data.
< len_of_data> is always 4 digits long followed.
< data> looks like: |< transaction id>|< command>|< buflen>|< buf>|< checksum>|
my code to create the packets is:
_snprintf_s(data_buffer, WS_MAX_DATA_PACKET_SIZE,
WS_MAX_DATA_PACKET_SIZE - 1,
"%s%d%s%d%s%d%s%s%s%d%s",
WS_PACKET_SEP, pkt->transaction_id,
WS_PACKET_SEP, pkt->command,
WS_PACKET_SEP, pkt->bufsize,
WS_PACKET_SEP, pkt->buf,
WS_PACKET_SEP, pkt->checksum, WS_PACKET_SEP);
buf_len = strlen(data_buffer);
_snprintf_s(send_buffer, WS_MAX_DATA_PACKET_SIZE,
WS_MAX_DATA_PACKET_SIZE - 1, "%04d%s%s",
buf_len, WS_PACKET_SEP, data_buffer);
buf_len = strlen(send_buffer);
// Send buffer
bytes_sent = send(ConnectSocket, send_buffer, buf_len, 0);
The client thread sends a command to the server, then calls a GetIncomingPackets() function. In GetIncomingPackets(), I call recv() to get 5 bytes, this should be the len of the rest of packet, I parse these 5 bytes and verify that they match my expected format. Then I convert the first 4 bytes to an integer, x. Then I call recv() again to get x bytes more and then parse those out into my packet structure.
The problem happens when I add another thread to do the same thing (send and receive commands).
I start my app and fire 2 threads and send them to send different commands and wait for responses. When the threads call GetIncomingPackets(), the data I am getting back is invalid. The first 5 bytes I am expecting are missing sometimes, and I just get the following 5 bytes, therefore I am unable to get my < len_of_data > packet.
I even added a critical section block between the 2 recv() calls in my GetIncomingPackets() so the treads dont interrupt each other while getting a full packet.
Without some extra code for error checking, this how the function looks like
#define WS_SIZE_OF_LEN_PACKET 5
bool GetIncomingPackets(SOCKET sd, dev_sim_packet_t *pkt )
{
char len_str_buf[WS_SIZE_OF_LEN_PACKET + 1] = {0}; // + 1 for NULL char
char data_buf[WS_MAX_DATA_PACKET_SIZE + 1] = {0};
int ret = 0;
int data_len = 0;
EnterCriticalSection( &recv_critical_section );
nReadBytes = WS_RecvAll(sd, len_str_buf, WS_SIZE_OF_LEN_PACKET );
ret = WS_VerifyLenPacket(len_str_buf);
// Convert data packet lenght string received to int
data_len = WS_ConvertNumberFromString(len_str_buf, WS_SIZE_OF_LEN_PACKET );
// Get data from packet
nReadBytes = WS_RecvAll(sd, data_buf, data_len);
LeaveCriticalSection( &recv_critical_section );
ret = ParseMessager(data_buf, data_len, pkt);
}
My question is, what could be causing this problem, and how could I fix it? Or is there better ways to do what i am trying to do. The reason that I'm trying to make it multi-threaded is because my app will communicate with 2 other sources, and I want to have a thread to handle each request that comes in from either source.
thanks in advance and feel free to ask any questions if I didn't explain something well.
Here's the code for WS_RecvAll(). The buffer is a static buffer declared in GetIncomingPackets() like this:
char data_buf[WS_MAX_DATA_PACKET_SIZE + 1] = {0}; // + 1 for NULL char
int WS_RecvAll(SOCKET socket_handle, char* buffer, int size)
{
int ret = 0;
int read = 0;
int i = 0;
char err_buf[100] = {0};
while(size)
{
ret = recv(socket_handle, &buffer[read], size, 0);
if (ret == SOCKET_ERROR)
{
printf("***ERROR***: recv failed, error = %d\n", WSAGetLastError());
return WS_ERROR_RECV_FAILED;
}
if (ret == 0) {
break;
}
read += ret;
size -= ret;
}
return read;
}
It's very difficult to debug MT problems, particularly at one remove, but if you are using astatic buffer, should not:
LeaveCriticalSection( &recv_critical_section );
ret = ParseMessager(data_buf, data_len, pkt);
be:
ret = ParseMessager(data_buf, data_len, pkt);
LeaveCriticalSection( &recv_critical_section );
And why use a static buffer in any case?
Im curious to know whether you have used the same socked descriptor in both the threads to connect to the server.
I'm trying to receive a single packet at a time from the server, since packets are going too fast, and each is of undefined size, calling recv() with number of bytes to read will read the first packet and maybe a part of the second packet. Since each packet is NULL terminated, I thought reading byte by byte until a NULL byte is received.
int recvLen = 0;
char TB;
char recvBuffer[1024];
while (recv(Socket, &TB, 1, 0) > 0 && TB != 0 && recvLen < 1024)
{
recvBuffer[recvLen] = TB;
recvLen++;
}
I don't think this method is efficient at all. If the server sent 1024 bytes, recv() will be called 1024 times.
Is there any other method to recv() until a NULL char is received, or some better method than this one I'm using?
EDIT:
i added the packet size infront of the data sent from the server, but now, if a false packet or even sometimes for no reason, packets gets messed up and no correct data is received. here is my code
#define UPLOAD_LEN 2755
int PacketSize, recvLen;
char Size[4];
char recvBuffer[UPLOAD_LEN+1];
while(1)
{
if(recv(Socket,Size,4,0)>0)
{
Size[4] = '\0';
PacketSize = atoi(Size);
if (PacketSize > UPLOAD_LEN || PacketSize <= 0) continue;
recvLen = recv(Socket, recvBuffer, PacketSize, 0);
} else recvLen = -1;
if (recvLen > 0)
{
recvBuffer[recvLen] = '\0';
ProcessData(recvBuffer);
}
else
{
closesocket(Socket);
}
}
I have never understood why communications protocols never support the one use case programmers expect to be able to do: exchange arbitrarily sized blobs with sends and recv's aligned on boundaries.
So theres no real shortcut here. You need to keep a persistent buffer that holds any data left over from the previous call to recv. Keep adding data to the end as you receive it, and return up to the terminating zero each time you find one. You'll probably have at least a partial following packet, so move that to the start of the buffer to serve as your initial state on the next call.
Create a buffer and extract your protocol messages from that. If the buffer does not contain a complete message, then recv() until it does. Here's a simple C implementation to buffer a socket (lightly tested, compiles on MS VS2008):
#include <winsock2.h>
#include <string.h>
typedef struct buffsock {
SOCKET s;
char* buf;
size_t maxlen;
size_t curlen;
} buffsock_t;
void buffsock_init(buffsock_t* bs,SOCKET s,size_t maxlen)
{
bs->s = s;
bs->buf = malloc(maxlen);
bs->maxlen = maxlen;
bs->curlen = 0;
}
void buffsock_free(buffsock_t* bs)
{
free(bs->buf);
bs->buf = NULL;
bs->maxlen = 0;
bs->curlen = 0;
bs->s = INVALID_SOCKET;
}
/* Attempt to fill internal buffer.
* Returns 0 if socket closed.
* Returns number of additional bytes in buffer otherwise.
*/
int buffsock_fill(buffsock_t* bs)
{
int bytes;
bytes = recv(bs->s,bs->buf + bs->curlen,bs->maxlen - bs->curlen,0);
if(bytes == SOCKET_ERROR)
return -1;
bs->curlen += bytes;
return bytes;
}
/* Return up to <bytes> from buffered socket.
* If return value 0 socket was closed.
* If return value >0 and <bytes socket received partial message.
*/
int buffsock_bytes(buffsock_t* bs,size_t bytes,void* msg)
{
while(bs->curlen < bytes)
{
int result;
result = buffsock_fill(bs);
if(result == -1)
return -1; /* error on socket */
if(result == 0)
break;
}
if(bytes > bs->curlen)
bytes = bs->curlen;
memcpy(msg,bs->buf,bytes);
bs->curlen -= bytes;
memmove(bs->buf,bs->buf + bytes,bs->curlen);
return bytes;
}
/* Implmementation of a protocol with two big-endian bytes indicating
* msg size followed by <size> bytes of message.
* Returns -1 if error on socket.
* Returns -2 if partial message recv'd (shouldn't happen as long as
* internal buffer is bigger than max message size).
* Returns -3 if user buffer not big enough to hold message.
* Returns size of message otherwise.
*/
int get_protocol_message(buffsock_t* bs,void* msg,size_t maxlen)
{
int bytes;
u_short len;
bytes = buffsock_bytes(bs,sizeof(u_short),&len);
if(bytes == 0)
return 0; /* socket closed, no more messages */
if(bytes == -1)
return -1; /* error on socket */
if(bytes < sizeof(u_short))
return -2; /* partial message */
len = ntohs(len);
if(len > maxlen)
return -3; /* message exceeds user buffer */
bytes = buffsock_bytes(bs,len,msg);
if(bytes < len)
return -2; /* partial message */
return bytes;
}
Use it like this:
int len;
char msg[256];
buffsock_t bs;
/* open a socket */
buffsock_init(&bs,sock,1024);
len = get_protocol_message(&bs,msg,sizeof(msg));
The key is TCP/IP has no concept of message boundaries, so recv() can return 1 to number of bytes requested. The received buffer could contain multiple or even partial messages.
This code just appends received data into a buffer. The protocol requests bytes from the buffer, and the buffer is filled from the socket. as bytes are removed the remaining buffered data is shifted to the beginning of the buffer.
In this case, two bytes are requested, converted to a length, then the remaining bytes are requested. If a request can't be satisfied, more data is recv'd.
Hope this helps.
There are several ways that you could do this.
Option #1: Before sending out any information, send out an int at the front of your packet which contains the size of the packet. Read this int, and then allocate a buffer which is the length of the int that you just received. Then you can recv() the entire packet at one time.
Option #2: Read in 1024 bytes at a time. recv() will give you back the number of bytes read. You can then use strlen() to figure out if you have more than one packet in your buffer. It would probably make the most sense to make this recursive(assuming that you could have several packets in 1024 bytes); so that you split the packets based on NULL bytes.