I would like to read from stdin until I reach end of file and then write that to a socket.
When I do this I don't want to read all of the data and then after reading, write to the socket.
It should be: read a byte of data, write that to the socket, read another byte, write that to the socket, etc. until I reach end of file.
Here is what I have so far:
char *buffer[65535];
while (read(0, buffer, 65535) > 0) {
write(sock, buffer, 65535);
}
sock is the socket file descriptor. I just used 65535 because somewhere I read that it was the max you can read() in.
How does this look?
How does this look?
Answer: a little buggy.
Just because you request 64KiB data doesn't mean you're going to get 64KiB of data. You need to record how much data you did receive, and only write what you read. Hence (noting the change of type for buffer mentioned by WhozCraig in his comment):
char buffer[65536];
int nbytes;
while ((nbytes = read(0, buffer, sizeof(buffer))) > 0)
{
if (write(sock, buffer, nbytes) != nbytes)
{
…process short write…
}
}
There are several ways to process a short write. One is to capture the number of bytes actually written, and keep looping until all the data read has been written. The loop below would replace all of the if (write(…)…) code.
Warning: untested code!
int nleft = nbytes;
int nwritten;
char *start = buffer;
while (nleft > 0 && (nwritten = write(sock, start, nleft)) > 0)
{
nleft -= nwritten;
start += nwritten;
}
if (nwritten < 0)
…process error…
Or you can simply report an error and abandon the writing process, which is much simpler but not as resilient.
Related
Extremely new to socket programming and C in general. I am trying to write a basic program to send and receive data between two machines. I understand that recv will not get all your data at once -- you essentially have to loop it until it has read the whole message.
In lieu of just setting a limit on both machines, I have created a simple Message struct on the client side:
struct Message {
size_t length;
char contents[1024 - sizeof(size_t)];
} message;
message.length = sizeof(struct Message);
message.contents = information_i_want_to_send;
When it arrives at the server, I have recv read into a buffer: received = recv(ioSock, &buffer, 1024, 0) (Which coincidentally is the same size as my Message struct -- but assuming it wasn't...).
I then extract Message.length from the buffer like this:
size_t messagelength;
messagelength = *((size_t *) &buffer);
Then I loop recv into the buffer while received < messagelength.
This works, but I can't help feeling it's really ugly and it feels hacky. (Especially if the first recv call reads less than sizeof(size_t) or the machines are different bit architectures, in which case the size_t cast won't work..). Is there a better way to do this?
You have a fixed-size message, so you can use something like this:
#include <errno.h>
#include <limits.h>
// Returns the number of bytes read.
// EOF was reached if the number of bytes read is less than requested.
// On error, returns -1 and sets errno.
ssize_t recv_fixed_amount(int sockfd, char *buf, size_t size) {
if (size > SSIZE_MAX) {
errno = EINVAL;
return -1;
}
ssize_t bytes_read = 0;
while (size > 0) {
ssize_t rv = recv(sockfd, buf, size, 0);
if (rv < 0)
return -1;
if (rv == 0)
return bytes_read;
size -= rv;
bytes_read += rv;
buf += rv;
}
return bytes_read;
}
It would be used something like this:
typedef struct {
uint32_t length;
char contents[1020];
} Message;
Message message;
ssize_t bytes_read = recv_fixed_amount(sockfd, &(message.length), sizeof(message.length));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(message.length)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
bytes_read = recv_fixed_amount(sockfd, &(message.content), sizeof(message.content));
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != msg_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
size_t is not going to be the same everywhere, so I switched to a uint32_t.
I read the fields independently because the padding within the struct can vary between implementations. They would need to be sent that way as well.
The receiver is populating message.length with the information from the stream, but doesn't actually use it.
A malicious or buggy sender could provide a value for message.length that's too large and crash the receiver (or worse) if it doesn't validate it. Same goes for contents. It might not be NUL-terminated if that's expected.
But what if the length wasn't fixed? Then the sender would need to somehow communicate how much the reader needs to read. A common approach is a length prefix.
typedef struct {
uint32_t length;
char contents[];
} Message;
uint32_t contents_size;
ssize_t bytes_read = recv_fixed_amount(sockfd, &contents_size, sizeof(contents_size));
if (bytes_read == 0) {
printf("EOF reached\n");
exit(EXIT_SUCCESS);
}
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != sizeof(contents_size)) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Message *message = malloc(sizeof(Message)+contents_size);
if (!message) {
perror("malloc");
exit(EXIT_FAILURE);
}
message->length = contents_size;
bytes_read = recv_fixed_amount(sockfd, &(message->contents), contents_size);
if (bytes_read < 0) {
perror("recv");
exit(EXIT_FAILURE);
}
if (bytes_read != contents_size) {
fprintf(stderr, "recv: Premature EOF.\n");
exit(EXIT_FAILURE);
}
Notes:
message->length contains the size of message->contents instead of the size of the structure. This is far more useful.
Another approach is to use a sentinel value. This is a value that tells the reader the message is over. This is what the NUL that terminates C strings is. This is more complicated because you don't know how much to read in advance. Reading byte-by-byte is too expensive, so one normally uses a buffer.
while (1) {
extend_buffer_if_necessary();
recv_into_buffer();
while (buffer_contains_a_sentinel()) {
// This also shifts the remainder of the buffer's contents.
extract_contents_of_buffer_up_to_sentinel();
process_extracted_message();
}
}
The advantage of using a sentinel value is that one doesn't need to know the length of the message in advance (so the sender can start sending it before it's fully created.)
The disadvantage is the same as for C strings: The message can't contain the sentinel value unless some form of escaping mechanism is used. Between this and the complexity of the reader, you can see why a length prefix is usually preferred over a sentinel value. :)
Finally, there's a better solution than sentinel values for large messages that you want to start sending before they are fully created: A sequence of length-prefixed chunks. One keeps reading chunks until a chunk of size 0 is encountered, signaling the end.
HTTP supports both length-prefixed messages (in the form of Content-Length: <length> header) and this approach (in the form of the Transfer-Encoding: chunked header).
There are Two ways to do that...
1.)
Use Binary Synchronous protocol. (Use of STX - Start of Text and ETX - End of Text ) for identification of the Text start and end.
2.)
Attach the number of bytes of data being sent at the start of Data. The socket will read those number of bytes and will get the number of bytes to be received from the socket. Then read all data and get the amount of data required.
Hmm... Seems tough...?? Let me give you an example.
Actual Data need to be sent: ABCDEFGHIJ
New Data format : 0010ABCDEFGHIJ
Data required in server side: ABCDE
recv function will read the first 4 bytes to get the number of bytes of actual data(In loop untill it gets 4 bytes):
int received1= recv(ioSock, recvbuf, 4, 0);
As per the above case, 'recvbuf' will be 0010 converted to an integer will give value as '10' which can be stored in some integer variable. So we have :
int toReadVal = 10
Now all we need is to read these 10 digits in next recv call :
int received= recv(ioSock, recvbuf1, toReadVal, 0);
Finally, we get the value of recvbuf1 as ABCDEFGHIG. Now you can truncate the value as per your requirement.
I am writing an application to read data from /dev/ttyUSB0.
I found it necessary to call sleep before calling read in the while loop so that I get the entire line at once. Otherwise, sometimes I get part of the line and the rest in the next iteration.
Do I have to package my data with a header containing the length of the string being sent over? Or is there a better way?
while(1) {
usleep(10000);
unsigned char buf[80];
int rdlen;
ioctl(fd, FIONREAD, &rdlen);
if (rdlen > 0) {
rdlen = read(fd, buf, rdlen);
}
if (rdlen > 0) {
...
}
The better way is to simply deal with receiving partial lines, have your code reading the data figure out when you have a complete line if that is important.
something like (not tested at all):
char buffer[1000];
size_t offset = 0;
while(1 ){
int len = read(fd, buffer+offset,(unsigned)(sizeof(buffer)-offset));
if(!strchr(buffer+offset, '\n')){
/* no end-of-line */
offset +=len;
}
else
{
/* deal with complete line */
offset = 0;
}
}
My code is too long to post all here so i'm going to sum up what's wrong.
In a server part i'm sending on a socket 3 things :
A message
The content of a file
Another message
In a client part i'm receiving these things but :
This first is to print on terminal
The second to write in a new file
The last to print on the terminal too
But my client is stuck on a read and i really don't know why. I'm on the problem for hour so if someone can help me, it will be very great !
edit : Basically, i think my problem is that i don't know what to write on the server to stop the read on the client.. Is it \n, \0.. ?
Here's the 2 part of code :
server
void send_content(t_server *s, FILE *fd, int rfd)
{
int len;
char *buff;
write(s->socket, "150 File status okay;" \
"about to open data connection.\n\0", strlen("150 File status okay;about to open data connection.\n\0"));
fseek(fd, 0, SEEK_END);
len = ftell(fd);
buff = malloc(len * sizeof(char));
read(rfd, buff, len);
write(s->socket, buff, len);
write(s->socket, "\n\0", strlen("\n\0"));
write(s->socket, "226 Closing data connection.\n\0", strlen("226 Closing data connection.\n\0"));
free(buff);
}
client
void getfile(t_client *c, char **tab)
{
int ret;
int fd;
int z;
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
if (strlen(buff) < 25)
return ;
fd = creat(tab[1], S_IRUSR | S_IWUSR);
while (z == 0 && (ret = read(c->fd, buff, 4096)) > 0)
{
if (ret < 4096)
z = -1;
write(fd, buff, strlen(buff));
memset(buff, '\0', 4096);
}
read(c->fd, buff, 4096); // Stuck here
write(1, buff, strlen(buff));
close(fd);
}
Like noted you need a read function like this to make sure you receive
specified number of bytes(this function will loop till it receives number of bytes it was told to). Just use this receivall method instead of read everywhere.
With files you typically first send the file length, and then receive the file.
I did something similar while ago, hope it will help you a bit. This is the client side, which tries to receive first file length from the server, then the file:
/* create file */
FILE * ptrMyFile = fopen(&filenames[i][0],"wb");
if(NULL == ptrMyFile)
{
printf("Unable to open file \n");
return 1;
}
int size = 0;
int t = 4;
/* first receive file size from server */
/* NOTE: error checking is omitted from code, nevertheless users should stil do some error checking when using this code */
readall(sockfd, (unsigned char*) &size, &t);
/* how many 256 byte chunks are there? */
int div = size / 256;
/* loop to receive each chunk. */
for(int k = 0; k < div; k++)
{
int chunk_size = 256;
/* make sure we receive 256 bytes */
readall(sockfd, buffer, &chunk_size);
/* write to file */
fwrite(buffer, chunk_size, 1, ptrMyFile);
}
/* read the final chunk. */
int whatsleft = size - 256 * div;
readall(sockfd, buffer, &whatsleft);
/* write */
fwrite(buffer, whatsleft, 1, ptrMyFile);
/* close file */
fclose(ptrMyFile);
I leave the server part to you.
char buff[4096];
z = 0;
read(c->fd, buff, 4096);
write(1, buff, strlen(buff));
You should be saving the return value of the call to read(), in order to find out how many bytes you just received. You may have to make several calls to read() in order to get the entire message. It's wrong to use strlen() to find out how many bytes were received, because the buffer contents are uninitialized, and the first chunk of the message could be cut off anywhere, so you can't count on it being null-terminated.
I'm writing a small and simple server (in C language for Linux stations).
A client requests a file to my server, my server asks this file to another server which sends it to my server.
My server should NOT receive ALL the file before sending it to the client BUT must send the bytes of the file so as they arrive.
This is an exercise in school so I can not dissociate myself from this requirement.
I have implemented the function explained below. The problem is that the client receives a non-deterministic number of bytes and NEVER the entire file.
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
size_t n;
ssize_t nread;
ssize_t nwritten;
char c;
for (n=1; n<file_size; n++)
{
nread=recv(socketa, &c, 1, 0);
if (nread == 1)
{
nwritten = send(socketb,&c,1,0);
}
else if (nread == 0)
{
*buffer = 0;
return (-1); /* Errore */
}
else
return (-1); /* Errore */
}
}
*buffer = 0;
return (n);
}
Someone could kindly tell me where I'm wrong?
Is it an stupid idea to change the values SO_SNDBUF and SO_RCVBUF on both the server and the client?
Assuming the file_size is the total number of bytes you want to send, then your for loop will only send file_size - 1 bytes. In other words, you are off by one. Start from 0 instead to fix this:
for (n=0; n<file_size; n++)
{ //..
You capture the return value of send(), but you do not check to see if it was successful or not.
You are treating a 0 return value from recv() the same as an error. Since you do not show what you do after returning -1 from your function, I don't know if this may be contributing to your problem or not.
Certain errors on send() and recv() are "soft", in that you are allowed to retry the operation for those particular errors. One such error is EINTR, but check the documentation on your system to see if there are others.
In order to optimize performance and simplify your code, you can use splice()+pipes. Sendfile enables you to "forward" data between file descriptors, without the copy to user space.
Are you sure you have copied the correct code? That part as it is would not compile, there is a } in the last else which don't match with a corresponding {.
Also, how you get to know the file size? if it's send thru the socket as an integer, bear in mind the possible byte order of the source and destination machines.
Anyway, you are reading one byte at a time, you should improve it this way:
EDIT: use buffer and not the extra buff[2048];
int Recv_and_send_file (int socketa, int socketb, char *buffer, size_t file_size){
ssize_t nread;
ssize_t nwritten;
ssize_t bLeft=file_size;
while (bLeft > 0)
{
nread=recv(socketa, buffer, bleft, 0);
if (nread > 0)
{
nwritten = send(socketb, buffer, nread, 0);
bLeft -= nread;
buffer+=nread;
}
else if (nread == 0)
{
// I think this could raise a memory exception, read below
*buffer = 0;
return (-1); /* Errore */
}
else
{
return (-1); /* Errore */
}
}
// If buffer is allocated with file_size bytes this one will raise a memory exception
// *buffer = 0;
return (file_size-bLeft);
}
I've got some code I'm writing that is expecting messages from a Matlab program via a named pipe, e.g., "/tmp/named_pipe_0". I can get pipes mkfifo and opened find, but when the C program goes to read() from the pipe, instead of the expected 5004 bytes, I'll get short values like 4096, 904, 5000, 4096, etc. I've already verified that Matlab is supposedly sending the correct 5004 bytes (at least, it's told to), so I'm wondering what the cause is. Anyone run across something like this before?
Matt
This is expected, a read on a pipe/socket/named pipe gives you back the data as soon as something is available.
If you need to read 5004 byte, you'd do it in a loop that appends you your own buffer up till you get that many bytes (or an error or eof occurs)
e.g.
size_t readn(int fd, void *buf, ssize_t len)
{
ssize_t tot = 0;
unsigned char *p = buf;
while (tot != len) {
ssize_t r = read(fd, p + tot, len - tot);
if (r == 0) //premature end of reading
break;
else if (r == -1) //error
return -1
tot += r;
}
return tot;
}
...
char buf[5004];
if (readn(pipe_fd, buf, sizeof buf) != sizeof buf) {
// something went bad
} else {
//got all the 5004 bytes
}