I'm trying to send a jpg image from a client process to a server via sockets. The image contains binary data so I want to do it on a low-level programming basis using reads and writes. I'm also sending the image data in iterations of 100 bytes.
This is the code I've done, which is not sending the image identically as I want:
CLIENT
void send_image(char *path, char *filename, int socket) {
int fd = open(path, O_RDONLY); //I open the file of the image.jpg
int n = 1;
while (n > 0) {
char img_data[100];
n = read(fd, img_data, 100); //sending 100 bytes of image each iteration till n=0 (end of file)
if (!n) break;
int sending = 1;
write(socket, &sending, sizeof(int)); //Tell the client the image still has data to send
write(socket, img_data, strlen(img_data));
usleep(250);
}
sending = 0; //Tell the server the image has been fully sent
write(socket, &sending, sizeof(int));
close(fd);
}
SERVER
void receiving_image(char *path) {
int receiving = 0;
int j=0;
char *image_data = NULL; //Variable to store all the image data
read(socket, &receiving, sizeof(int)); //Reads that the client is going to send an image
while (receiving) {
char data[100]; //Variable that stores partial data (100 bytes) of an image on each iteration
read(socket, data, 100);
image_data = realloc(image_data, (j + strlen(data)) * sizeof(char)); //Readjust the size of the main image data.
for (int i=0; i<(int) strlen(data); i++) {
image_data[j] = data[i]; //copy the partial data of the image to the main variable of the image
j++;
}
j = (int) strlen(image_data);
read(socket, &receiving, sizeof(int)); //Read if the image is still sending
}
image_to_directory(path, image_data); //Copy image to directory
}
This compiles and runs fine, but when I check the directory on the server side where the image has been stored, I can see it's not the same image as the client has sent (I confirmed via md5sum and hashes are not equal).
Is there something I am missing?
You shouldn't be using strlen for your binary data length calculations. It is intended only for terminated strings (thus the name). You also have highly-ill-advised naked calls to read/write, which is a recipe for disaster when sending data over sockets.
You never seem to be sending more than 100 bytes at a time, which is helpful in this case to develop a more solid protocol. Consider this:
First octet is a uint8_t byte count N, and will be in 0..100.
Following the byte count, N bytes are transferred.
Repeat 1-2 until no more bytes remain.
Notify the server of EOF by sending a single zero-octet
An example of this sender code is shown here.
void send_image(const char *path, int socket)
{
int fd = open(path, O_RDONLY); //I open the file of the image.jpg
if (fd == -1)
return;
ssize_t n = 0;
do
{
// note the first octet will prefix the length
uint8_t img_data[101];
n = read(fd, img_data+1, 100);
if (n > 0)
{
// you never know just how many bytes are going to
// be sent, so setup the frame, but then ensure even
// piecewise deliver can succeed.
img_data[0] = (uint8_t)n;
ssize_t sent = 0;
size_t pos = 0;
do
{
sent = write(socket, img_data+pos, (n+1)-pos);
if (sent < 0)
break;
pos += sent;
} while ( pos < (n+1) && sent > 0);
}
} while (n > 0);
uint8_t done = 0;
write(socket, &done, sizeof done); // not much we can do if this fails
close(fd);
}
I make no claims the above code will even compile, but the concept should be fairly obvious. That's it, however. Obviously there is more that could/should be done (checksums, restart options, etc.), but that's the basic premise.
The server side can do something similar, which I leave as an exercise for you. The point of all of this is to utilize the return values from your read/write calls. They're there for a reason. If you find yourself coding a "naked" read or write (where you don't gather the result of function and utilize it in some way), chances are you've done something horribly wrong.
Related
Summary: I am working on a multithreaded application (both the proxy and cache are multithreaded). I am passing a request the client (client is a black box, cannot see the code) makes to the proxy, and the proxy connects with a cache to query for the file. The proxy and cache share a message queue for the requests, and a POSIX shared memory channel for the data. If the request path is not found in the cache, then I post to the shared memory channel a -1. If it is found, I send back a concatenated string (size of data chunk|data) thru the shared memory channel instead. The proxy determines the status from the shared memory channel and sends the response back to the client, followed by the data if it exists.
The problem: I send back the perfect amount of data and have no issues, client and proxy filesizes and chunks sent/received match. I can also print the data and see that it APPEARS to be the same in both the proxy and cache (the characters are really weird). But when I go to open the jpg which the client wrote from the data it received, it is corrupted. I have scoured this code for days, but cannot find the reason.
I have a handler function for a proxy I made. It is ran infinitely inside the server, and waits until there is a request to serve:
ssize_t proxy_worker_handler(custom_struct *ctx, const char *path, void *arg)
{
int segment_to_lock_into = *(int *)arg % GLOB_NSEGMENTS;
size_t filesize = 0;
mq_send(COMMAND_CHANNEL_ID, path, BUFSIZE, 1);
data_channel current_channel = GLOB_DATA_CHANNEL_SEGMENTS[segment_to_lock_into];
// Take proxy ownership of this segment for now
sem_wait(current_channel.proxy_sem);
// Wait until we can read, writer sends the signal
sem_wait(current_channel.reader_sem);
// Now we can read, see if file was found and send proper header
if (strncmp(current_channel.data, "-1", strlen("-1")) == 0)
{
gfs_sendheader(ctx, GF_FILE_NOT_FOUND, 0);
sem_post(current_channel.writer_sem);
}
else
{
// Get the filesize
filesize = (size_t)atoi(current_channel.data);
gfs_sendheader(ctx, GF_OK, filesize);
sem_post(current_channel.writer_sem);
// Now receive data
int curr_received = 0;
int delimiter, data_len;
int padding_size = 10;
char *data_str;
char data_len_str[padding_size];
while (curr_received < (int)filesize)
{
// Wait to be able to read
sem_wait(current_channel.reader_sem);
data_str = "";
memset(data_len_str, 0, padding_size);
// Break up the memory segment, to get size and string sent
data_str = strstr(current_channel.data, "|") + 1;
delimiter = (data_str - 1) - current_channel.data;
for (int i = 0; i < delimiter; i++)
data_len_str[i] = current_channel.data[i];
data_len = atoi(data_len_str);
printf("%s", data_str);
// Use the above info to send the data
curr_received += data_len;
gfs_send(ctx, data_str, (size_t)data_len);
sem_post(current_channel.writer_sem);
}
printf("%d of %zu\n", curr_received, filesize);
}
// Free up the segment for proxy workers waiting
sem_post(current_channel.proxy_sem);
return filesize;
}
I also have a cache worker, which is called infinitely as well but handles itself:
void *cache_worker(void *arg)
{
char *path;
int fd;
struct stat st;
char filesize[21];
size_t batch_read, start_point;
int segment_to_lock_into = *(int *)arg % GLOB_NSEGMENTS;
free(arg);
data_channel current_channel = GLOB_DATA_CHANNEL_SEGMENTS[segment_to_lock_into];
while (1)
{
// Receive the request path from the boss
pthread_mutex_lock(&queue_lock);
while (steque_isempty(&queue) == 1)
pthread_cond_wait(&queue_cond, &queue_lock);
path = steque_pop(&queue);
pthread_mutex_unlock(&queue_lock);
fd = simplecache_get(path);
sleep(cache_delay);
// Because strdup uses malloc
free(path);
// Take cache ownership of this segment for now
sem_wait(current_channel.cache_sem);
// Now, see if the file exists
if (fd == CACHE_FAILURE)
{
sem_wait(current_channel.writer_sem);
// Write and then increment the read semaphore, so the proxy knows to read it
memset(current_channel.data, 0, GLOB_SEGSIZE);
memcpy(current_channel.data, "-1", strlen("-1"));
sem_post(current_channel.reader_sem);
close(fd);
}
else
{
// Tell the reader the size of the file first
sem_wait(current_channel.writer_sem);
fstat(fd, &st);
snprintf(filesize, 21, "%ld", st.st_size);
memset(current_channel.data, 0, GLOB_SEGSIZE);
memcpy(current_channel.data, filesize, 21);
sem_post(current_channel.reader_sem);
// Open the file and send the data
int padding_size = 10;
char data[GLOB_SEGSIZE];
char to_send[GLOB_SEGSIZE];
batch_read = 0, start_point = 0;
while ((int)(batch_read = pread(fd, data, (size_t)(GLOB_SEGSIZE - padding_size), start_point)) > 0)
{
sem_wait(current_channel.writer_sem);
memset(to_send, 0, GLOB_SEGSIZE);
memset(current_channel.data, 0, GLOB_SEGSIZE);
snprintf(to_send, GLOB_SEGSIZE, "%zu|%s", batch_read, data);
memcpy(current_channel.data, to_send, (size_t)GLOB_SEGSIZE);
start_point += batch_read;
printf("%s", data);
sem_post(current_channel.reader_sem);
}
printf("%zu of %s\n", start_point, filesize);
}
// Free up the segment for cache workers waiting
sem_post(current_channel.cache_sem);
}
return NULL;
}
I am trying to write and read Integer value into/from C socket. Sometimes ntohs() return very big values like 55000 , 32000 etc...Though client is always sending value <1500. If I run the program it happens after 10-15 minutes...Sometimes after 20-30 minutes.
Can you please check below code and tell me
Why this line getting printed ?
printf("Garbage value - ntohs problem ..Exiting... ");
// write exactly n byte
inline int write_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = write(fd, buf, left)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
// send exactly n byte
inline int send_n(int fd, char *buf, int n) {
int nwrite, left = n;
int totalwrite = 0;
while (totalwrite != n) {
if ((nwrite = send(fd, buf, left, MSG_NOSIGNAL)) <= 0) {
break;
} else {
totalwrite = totalwrite + nwrite;
left -= nwrite;
buf += nwrite;
}
}
if (totalwrite == 0)
return nwrite;
return totalwrite;
}
uint16_t nread, len, plength, nsend;
int MTU = 1500;
char buffer[2000];
// Server receive ( Linux 64 bit)
while (1) {
// read packet length
nread = read_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nread <=0) {
break;
}
len = ntohs(plength);
if (len <=0 || len > 1500 ) {
**printf("Garbage value - ntohs problem ..Exiting... "); // WHY ?**
break;
}
// read packat data
nread = read_n(SOCKFD, buffer, len);
if (nread != len) {
break;
}
}
//---------------------
// CLIENT send ( Android 5 )
while (1) {
nread = read(tunfd, buffer, MTU);
if (nread <= 0 || nread > 1500) { // always <=1500
break;
}
plength = htons(nread);
// send packet lenght
nsend = send_n(TCP_SOCKFD, (char *) &plength, sizeof(plength));
if (nsend != sizeof(plength)) {
break;
}
// send packet data
nsend = send_n(TCP_SOCKFD, buffer, nread);
if (nsend != nread) {
break;
}
}
Thank you
We cannot tell you with certainty what's happening because you cannot provide a verifiable example. Additionally, you've not presented the implementation of read_n(), but supposing that it follows the same model as write_n() and send_n(), we can nevertheless perform some analysis.
Each of the data transfer functions returns a short count in the event that data transfer is interrupted by an error. The client code watches for this, and breaks out of its loop if it detects it. Well and good. The server code does not do this when reading plength, however. Since plength, as a uint16_t, is two bytes in size, a partial read is possible and would go unnoticed by your server code.
In your example, plength is modified only via the one read_n() call presented. Network byte order is big-endian, so the most-significant byte is read first. It is possible that the combination of that byte with the stale one left over from the previous read would represent a number exceeding 1500. For example, if a 221(0x00dd)-byte packet is followed by a 1280(0x0500)-byte packet, and a partial read occurs on the second packet size, then the combined result will be 1501(0x05dd).
I don't presently see any reason to think that the client sends data different in nature than you think it does, and I don't presently see any other way that your server code could give the appearance of receiving different data than the client sends, especially since client and server each abort at the first recognized sign of trouble.
Do note, however, that this code could still be made more robust. In particular, consider that read(), write(), and send() can fail even when there is no problem with the underlying socket or data transfer request. In particular, they can fail with EINTR if the call is interrupted by a signal, and if the socket is in non-blocking mode then they can fail with EAGAIN. There may be others. It does not seem useful to operate your socket in non-blocking mode, but you might indeed want to watch for EINTR and resume reading after receiving it.
I would also suggest that, at least during development, you emit more data about the nature of the error. Call perror(), for example, and afterward print the bad data. You might even consider logging data sent and received.
I have a problem with receiving the correct values on the server-side of my client-server program.
header file included in both server and client:
#define CHUNK_SIZE 1024
#define ARR_LEN 3
client:
int uids[ARR_LEN] = {1994, 2423, 1222};
unsigned int uidlen = 0;
char uidbuffer[CHUNK_SIZE] = {0};
for(int i = 0; i < ARLL; i++)
{
uidlen = strlen(uids[i])+1;
snprintf(uidbuffer, uidlen, "%s", uids[i]);
if(send(socket, uidbuffer, strlen(uidbuffer), 0) < 0)
DIE("Write Error");
if(recv(socket, uidbuffer, sizeof(uidbuffer), 0) < 0)
DIE("Acknowledge Error");
memset(uidbuffer, 0, sizeof(uidbuffer));
}
server:
char uid_buff[CHUNK_SIZE];
for(int i = 0; i < ARR_LEN; i++)
{
memset(uid_buff, 0, sizeof(uid_buff));
// receiving the UID and storing it directly
if(recv(client_sock, uid_buff, sizeof(uid_buff), 0) < 0)
DIE("Receive Error");
printf("buffer content: %s\n", uid_buff);
uid_str = uid_buff;
uids[i] = (uid_t)strtol(uid_str, (char **)NULL, 10);
if(send(client_sock, uid_buff, sizeof(uid_buff), 0) < 0)
DIE("Acknowledge Error");
}
These are only parts of my program. I tried to only include the relevant parts. The output is this:
buffer content: 1994
buffer content: 24231222
buffer content:
While I want it to be:
buffer content: 1994
buffer content: 2423
buffer content: 1222
What could be the problem? I know that it's not so easy and that server–client communication is carried out in a stream of bytes rather than messages, but I want to mimmic that functionality by acknowledging every received "message". Could you please clue me what to do? I'm getting desperate.
You will need a protocol.
For example, you define that each message in your application has following format:
xx | message
this means first two bytes (mind the endianness) you receive indicate the length of the message that follows. Now you should first receive first two bytes - check the length - and then receive exactly that number of bytes. After that you know you have successfully received that message. And then you can proceed to other messages (which could/should have similar format: length + message itself).
Example:
Say you want to send three messages:
char s1[]="message1";
char s2[]="message2";
char s3[]="message3";
//You do this(client side):
int x1 = strlen(s1); // length of message1
int intsize = 4; // just size of integer -we'll need in next call
sendall(socket, &x1, &intsize); // send length of first message
sendall(socket, s1, &x1); // Now send the message
//On server:
int x = 0;
int y = 4; //size of integer most probably
receiveall(socket,&x,&y);//get length first; y=4 because that is size of integer
receiveall(socket, buffer, &x); // now we know how many bytes to expect - x - so request that number of bytes only
you can repeat this logic for other messages too.
Finally, you want to use such functions (here) instead of send and receive (because send and receive might not send/receive the number of bytes you tell it to):
int sendall(int s, char *buf, int *len)
{
int total = 0; // how many bytes we've sent
int bytesleft = *len; // how many we have left to send
int n;
while(total < *len) {
n = send(s, buf+total, bytesleft, 0);
if (n == -1) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually sent here
return n==-1?-1:0; // return -1 on failure, 0 on success
}
you will need a similar receiveall function.
You do not show the code that establishes the connection.
If you are using UDP sockets, each message is stand alone, you will receive a separate message for each message sent, but not necessarily in the same order.
If you are using TCP or Unix sockets, data may be received in chunks of different sizes than those used for sending. Only the sequence of bytes is preserved, not the chunk sizes. You need to specify a protocol: unless you specify a fixed number of bytes per message, you need to send some sort of separator to allow the server to tell where each message ends.
I'm writing a small and simple server (in C language for Linux stations).
A client requests a file to my server, my server asks this file to another server which sends it to my server.
My server should NOT receive ALL the file before sending it to the client BUT must send the bytes of the file so as they arrive.
This is an exercise in school so I can not dissociate myself from this requirement.
I have implemented the function explained below. The problem is that the client receives a non-deterministic number of bytes and NEVER the entire file.
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
size_t n;
ssize_t nread;
ssize_t nwritten;
char c;
for (n=1; n<file_size; n++)
{
nread=recv(socketa, &c, 1, 0);
if (nread == 1)
{
nwritten = send(socketb,&c,1,0);
}
else if (nread == 0)
{
*buffer = 0;
return (-1); /* Errore */
}
else
return (-1); /* Errore */
}
}
*buffer = 0;
return (n);
}
Someone could kindly tell me where I'm wrong?
Is it an stupid idea to change the values SO_SNDBUF and SO_RCVBUF on both the server and the client?
Assuming the file_size is the total number of bytes you want to send, then your for loop will only send file_size - 1 bytes. In other words, you are off by one. Start from 0 instead to fix this:
for (n=0; n<file_size; n++)
{ //..
You capture the return value of send(), but you do not check to see if it was successful or not.
You are treating a 0 return value from recv() the same as an error. Since you do not show what you do after returning -1 from your function, I don't know if this may be contributing to your problem or not.
Certain errors on send() and recv() are "soft", in that you are allowed to retry the operation for those particular errors. One such error is EINTR, but check the documentation on your system to see if there are others.
In order to optimize performance and simplify your code, you can use splice()+pipes. Sendfile enables you to "forward" data between file descriptors, without the copy to user space.
Are you sure you have copied the correct code? That part as it is would not compile, there is a } in the last else which don't match with a corresponding {.
Also, how you get to know the file size? if it's send thru the socket as an integer, bear in mind the possible byte order of the source and destination machines.
Anyway, you are reading one byte at a time, you should improve it this way:
EDIT: use buffer and not the extra buff[2048];
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
ssize_t nread;
ssize_t nwritten;
ssize_t bLeft=file_size;
while (bLeft > 0)
{
nread=recv(socketa, buffer, bleft, 0);
if (nread > 0)
{
nwritten = send(socketb, buffer, nread, 0);
bLeft -= nread;
buffer+=nread;
}
else if (nread == 0)
{
// I think this could raise a memory exception, read below
*buffer = 0;
return (-1); /* Errore */
}
else
{
return (-1); /* Errore */
}
}
// If buffer is allocated with file_size bytes this one will raise a memory exception
// *buffer = 0;
return (file_size-bLeft);
}
I made an app in C for Linux environment. This app works, but after
some time (5 hours more or less) its performance is erratic.
This is the background. I have some remote computers (400 more or
less) that send its data through sockets (my app) every 5 minutes. All
process is a loop and I need that works all day. The app first send
data (if exists) and then receive data (also, if exists). My app works
like a Server side with some arguments or it works like a Client side
with other arguments. The app working as Server mode, after the listening state,
only waits for client connections. Once that a client it's connected,
the App in Client mode sends a message to indicates that will be sent
data, then sends a new message with the size and name concatenated to
server side (always sends 64 bytes, so my app writes 64 bytes and in
the other side reads 64 bytes), then sends data (the file). The Server
reads the message of size and name of file to receive (of 64 bytes),
splits the message, storing the size in a variable and file name in
other, and then after received the file data. compares the size stored
in the variable with the size of data readed, if all is ok, then the
server sends a message and stores the data in a new file with the name
that was it before received. So on untill the last file will be sent
it.
When my app was in state of development, I noticed that after some
hours the messages they become corrupted, I mean that the messages are
incomplete or have more data, probably data of the next ones. So the
file data too. So, how can i send the file data through the same
socket?? Note: I have a buffer to send and receive messages, and
other buffer for send and receive file data.
And other question:
Original code:
int copydata(int readfd, int writefd, int offset, int bytes) {
char buffer[4096];
int crbytes = 0, cwbytes = 0, trbytes = 0, twbytes = 0;
if (offset) {
if (lseek(readfd, offset, SEEK_CUR) < 0) {
return -1;
}
}
while ((crbytes = read(readfd, buffer, bytes)) > 0) {
if (crbytes < 0) {
return -1;
}
trbytes += crbytes;
//printf("dbgmsg::Readed data <%dB> | Total readed data <%dB>\n", crbytes, trbytes);
while (crbytes > 0) {
cwbytes = write(writefd, buffer, crbytes);
if (cwbytes < 0) {
return -1;
}
twbytes += cwbytes;
crbytes -= cwbytes;
//printf("dbgmsg::Written data <%dB> | Total written data <%dB>\n", cwbytes, twbytes);
}
}
return twbytes;
}
This code is used to send and receive file data. The side that sends
data uses the file descriptor of the file that we want to send (to
read) <readfd> and writes on the file descriptor of the socket (to
write) <writefd>. The side that receives data, use <readfd> to read
from the socket file descriptor and <writefd> to write in the file
descriptor from the file where we want to write the data.
If are sent other messages before use this function, both, client and
server will be stuck in the end of the inner "while loop", in other
words, the client sends all file data, and the server receives all
file data (in this point, the received data is complete, how i know?
Because i can open the the received file). No errors, only one, "no message of
desired type". If i not sent messages before this function all works
fine.
To skip this little problem, i modify the code, passing a file size as
argument too and write between two whiles one if structure.
int copydata(int readfd, int filesz, int writefd, int offset, int bytes) {
char buffer[4096];
int crbytes = 0, cwbytes = 0, trbytes = 0, twbytes = 0;
if (offset) {
if (lseek(readfd, offset, SEEK_CUR) < 0) {
return -1;
}
}
while ((crbytes = read(readfd, buffer, bytes)) > 0) {
if (crbytes < 0) {
return -1;
}
trbytes += crbytes;
//printf("dbgmsg::Readed data <%dB> | Total readed data <%dB>\n", crbytes, trbytes);
while (crbytes > 0) {
cwbytes = write(writefd, buffer, crbytes);
if (cwbytes < 0) {
return -1;
}
twbytes += cwbytes;
crbytes -= cwbytes;
//printf("dbgmsg::Written data <%dB> | Total written data <%dB>\n", cwbytes, twbytes);
}
if (twbytes == filesz) { break; }
}
return twbytes;
}
Thanks for advance, and sorry for my english!!
If the program encounters an "incomplete" write(), your inner loop resends the beginning of the buffer.
while (crbytes > 0) {
cwbytes = write(writefd, buffer, crbytes);
if (cwbytes < 0) {
return -1;
}
twbytes += cwbytes;
crbytes -= cwbytes;
//printf("dbgmsg::Written data <%dB> | Total written data <%dB>\n", cwbytes, twbytes);
}
You could change it to something like:
for ( sentbytes=0; sentbytes < crbytes; sentbytes += cwbytes ) {
cwbytes = write(writefd, buffer+sentbytes, crbytes - sentbytes);
if (cwbytes < 0) { return -1; }
twbytes += cwbytes;
}