When we use read() function with file descriptor(not socket), it has byte-offset like this.
read(file descriptor, message1, 20);
read(file descriptor, message2, 40);
After number1 command, the file descriptor's byte offset is 20 as it says.
Then, number 2 command read the file at the byte offset 20, and the file descriptor's final offset is 60 now.
So, now my question is about socket. Let me explain you situations.
On server-client model, client sends server 100 bytes(lets assum it's string data). But, unfortunately, server receive data sequentially 10, 20, 70(not 100 bytes at once.)
So, because of this situation, server source code is like this.
51 while(recv_len<100)
52 {
53 recv_count=read(cli_sock, message[recv_len], 1024);
54
55 if(recv_count==-1)
56 error_handling("read() error");
57
58 recv_len+=recv_count;
59 }
So client writes 10 bytes data one of 100 bytes string to server, at first.
Anyway, server execute read() function, and reads 1024 bytes. So, the server takes 10 bytes. But, its byte-offset is 1024 now.
So, after client's second data(20 bytes data), server must read at byte offset 1024. Then, how can server reads 20 bytes which is sent from client? I mean originally, server must read 20 bytes at byte offset 10(not 1024), because 10 bytes data had sent before 20bytes. But, server is at byte offset 1024. How this can be possible?
This
recv_count=read(cli_sock, message[recv_len], 1024);
will not do what you think.
It should be :
recv_count=read(cli_sock, message + recv_len, 1024 - recv_len);
or
recv_count=read(cli_sock, &message[recv_len], 1024 - recv_len);
read the man page and if don't understand something, don't hesitate to ask for precision (even thought that blue112 already explain it all).
You've got it wrong. The third parameter of a read() call isn't an offset, it represents the number of bytes you want to read, at max.
The actual number read is returned by the read() call.
Look here : http://man7.org/linux/man-pages/man2/read.2.html
Also, you should pass a pointer to read() calls.
So message[recv_len] isn't gonna do. You're looking for message + recv_len
Related
I am creating a server/client TCP in C.
The idea is for the server to send a relatively large amount of information. However, the buffer in the client has a size of only 512 (I don't want to increase this size), and obviously, the information sent by the server is larger than this. Let's imagine 812 bytes.
What I want to do is, in the client, read 512 bytes, print them on the client's console, and then read the remaining bytes, and print them as well.
Here's what should happen:
1) Create server, and block in the read() system call (waiting for the client to write something);
2) Create the client, and write something in the socket, and then blocks on read(), waiting for the server to respond;
3) The server's read() call returns, and now server has to send that large amount of data, using the following code (after creating a new process):
dup2(new_socketfd, STDOUT_FILENO); // Redirect to socket
execlp("/application", "application", NULL); // Application that prints the information to send to the client
Let's imagine "application" printed 812 bytes of data to the socket.
4) Now the client has to read 812 bytes, with a buffer size of 512. That's my problem.
How can I approach this problem? I was wondering if I could make a loop, and read until there's nothing to read, 512 by 512 bytes. But as soon as there's nothing to read, client will block on read().
Any ideas?
recv will block when there is no data in the stream. Any data extracted from the stream, the length is returned from recv.
You can write a simple function to extract the full data just by using an offset variable and checking the return value.
A simple function like this will do.
ssize_t readfull(int descriptor,char* buffer, ssize_t sizetoread){
ssize_t offset = 0;
while (offset <sizetoread) {
ssize_t read = recv(descriptor,buffer+offset,sizetoread-offset,0);
if(read < 1){
return offset;
}
offset+=read;
}
return offset;
}
Also servers typically send some kind of EOF when the data is finished. Either the server might first send the length of the message to be read which is a constant size either four or eight bytes, then it sends the data so you know ahead of time how much to read. Or, in the case of HTTP for example, there is the content-length field as well as the '\r\n' delimeters.
Realistically there is no way to know how much data the server has available to send you, it's impractical. The server has to tell you how much data there is through some kind of indicator.
Since you're writing the server yourself, you can first send a four byte message which can be an int value of how much data the client should read.
So your server can look like this:
int sizetosend = arbitrarysize;
send(descriptor,(char*)&sizetosend,sizeof(int),0);
send(descriptor,buffer,sizetosend,0);
Then on your client side, read four bytes then the buffer.
int sizetoread = 0;
ssize_t read = recv(descriptor,(char*)&sizetoread,sizeof(int),0);
if(read < 4)
return;
//Now just follow the code I posted above
I have a client server program in which i read the file from the server to the client in chunks, over TCP. However, not all chunks being sent are of the size I have given. Some of them are less than the requested size or even 1.
This is my server side code for reading from the file and writing to socket
rd=read(filed, buffer,128);
write(socketd,buffer,128);
This true for only non text files. On using a txt file, it works perfectly, but fails for pdfs, ppts, images. Is there any way to read these non text files in chunks?
Sample of size in bytes being read in each iteration for a pdf file
128
128
128
128
116
128
128
36
You should only write the amount that was actually read from the file. If the size of the file isn't a multiple of 128, then the last read can't return 128 bytes, it just returns what's left in the file. Use rd as the amount to write to the socket.
while ((rd = read(filed, buffer, 128)) > 0) {
write(socketd, buffer, rd);
}
Issue resolved. I was using strlen(buffer) which gave those numbers. The integer returned by read() was 128, as desired.
As the title says, is there a limit to the number of bytes that can be written at once on a connection-oriented socket?
If I want to send a buffer of, for example, 1024 bytes, can I use a
write(tcp_socket, buffer, 1024);
or should I use multiple write() calls with a lower amount of bytes for each one?
write() does not guarantee that all bytes will be written so multiple calls to write() are required. From man write:
The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setrlimit(2)), or the call was interrupted by a signal handler after having written less than count bytes. (See also pipe(7).)
write() returns the number of bytes written so a running total of the bytes written must be maintained and used as an index into buffer and to calculate the number of remaining bytes to be written:
ssize_t total_bytes_written = 0;
while (total_bytes_written != 1024)
{
assert(total_bytes_written < 1024);
ssize_t bytes_written = write(tcp_socket,
&buffer[total_bytes_written],
1024 - total_bytes_written);
if (bytes_written == -1)
{
/* Report failure and exit. */
break;
}
total_bytes_written += bytes_written;
}
From my experience, it is better to stay in the 1024 byte limit
There's no inherent limit. TCP/IP will fragment and reassemble packets as required. Your system may impose (a possibly tunable) upper limit, but it is likely to be in the multi-MB range. See your manpage for setsockopt() and always check write()'s return value.
The actual amount you can write will depend on the type of socket. In general, you need to check the return value to see how many bytes were actually written. The number of bytes written can vary depend on whether the socket is in blocking mode or not.
Also, if the socket is blocking, you may not want to wait for all the data to be written in one go. You may want to write some at a time in order to be able to something else in between write operations.
as you can see write socket the maximum buffer size is 1048576 bytes.
I'm using C to implement a client server application. The client sends info to the server and the server uses it to send information back. I'm currently in the process of writing the code to handle the receiving of data to ensure all of it is, in fact, received.
The issue I'm having is best explained after showing some code:
int totalRead = 0;
char *pos = pBuffer;
while(totalRead < 6){
if(int byteCount = read(hSocket, pos, BUFFER_SIZE - (pos-pBuffer)>0)){
printf("Read %d bytes from client\n", byteCount);
pos += byteCount;
totalRead += byteCount;
}else return -1;
}
The code above runs on the server side and will print out "Read 1 bytes from client" 6 times and the program will continue working fine. I've hard-coded 6 here knowing I'm writing 6 bytes from the client side but I'll make my protocol require the first byte sent to be the length of rest of the buffer.
int byteCount = read(hSocket, pBuffer, BUFFER_SIZE);
printf("Read %d bytes from client", byteCount);
The code above, used in place of the first code segment, will print "Read 6 bytes from client" and continue working fine but it doesn't guarantee I've received every byte if only 5 were read for instance.
Can anyone explain to me why this is happening and a possible solution? I guess the first method ensures all bytes are being delivered but it seems inefficient reading one byte at a time...
Oh and this is taking place in a forked child process and I'm using tcp/ip.
Note: My goal is to implement the first code segment successfully so I can ensure I'm reading all bytes, I'm having trouble implementing it correctly.
Basically the right way to do this is a hybrid of your two code snippets. Do the first one, but don't just read one byte at a time; ask for all the bytes you're expecting. But look at bytesRead, and if it's less than you expected, adjust your destination pointer, adjust your expected number read, and call read() again. This is just how it works: sometimes the data you're expecting is split across packets and isn't all available at the same time.
Reading your comment below and looking at your code, I was puzzled, because yeah, that is what you're trying to do. But then I looked very closely at your code:
read(hSocket, pos, BUFFER_SIZE - (pos-pBuffer)>0)){
^
|
THIS ---------|
That "> 0" is inside the parentheses enclosing read's arguments, not outside; that means it's part of the arguments! In fact, your last argument is interpreted as
(BUFFER_SIZE - (pos-pBuffer)) > 0
which is 1, until the end, when it becomes 0.
Your code isn't quite right. read and write may not read or write the total amount of data you've requested they should. Instead, you should advance the read or write pointer after each call and count how many bytes you have left to submit with in the transmission.
If you get back negative one from either read or write you've gotten an error. A zero indicates the transmission completed (there were no more bytes to send) and any number above zero indicates how many bytes were sent in the last call to read or write respectively.
The recv() library function man page mention that:
It returns the number of bytes received. It normally returns any data available, up to the requested amount, rather than waiting for receipt of the full amount requested.
If we are using blocking recv() call and requested for 100 bytes:
recv(sockDesc, buffer, size, 0); /* Where size is 100. */
and only 50 bytes are send by the server then this recv() is blocked until 100 bytes are available or it will return receiving 50 bytes.
The scenario could be that:
server crashes after sendign only 50 bytes
bad protocol design where server is only sending 50 bytes while client is expecting 100 and server is also waiting for client's reply (i.e. socket close connection has not been initiated by server in which recv will return)
I am interested on Linux / Solaris platform. I don't have the development environment to check it out myself.
recv will return when there is data in the internal buffers to return. It will not wait until there is 100 bytes if you request 100 bytes.
If you're sending 100 byte "messages", remember that TCP does not provide messages, it is just a stream. If you're dealing with application messages, you need to handle that at the application layer as TCP will not do it.
There are many, many conditions where a send() call of 100 bytes might not be read fully on the other end with only one recv call when calling recv(..., 100); here's just a few examples:
The sending TCP stack decided to bundle together 15 write calls, and the MTU happened to be 1460, which - depending on timing of the arrived data might cause the clients first 14 calls to fetch 100 bytes and the 15. call to fetch 60 bytes - the last 40 bytes will come the next time you call recv() . (But if you call recv with a buffer of 100 , you might get the last 40 bytes of the prior application "message" and the first 60 bytes of the next message)
The sender buffers are full, maybe the reader is slow, or the network is congested. At some point, data might get through and while emptying the buffers the last chunk of data wasn't a multiple of 100.
The receiver buffers are full, while your app recv() that data, the last chunk it pulls up is just partial since the whole 100 bytes of that message didn't fit the buffers.
Many of these scenarios are rather hard to test, especially on a lan where you might not have a lot of congestion or packet loss - things might differ as you ramp up and down the speed at which messages are sent/produced.
Anyway. If you want to read 100 bytes from a socket, use something like
int
readn(int f, void *av, int n)
{
char *a;
int m, t;
a = av;
t = 0;
while(t < n){
m = read(f, a+t, n-t);
if(m <= 0){
if(t == 0)
return m;
break;
}
t += m;
}
return t;
}
...
if(readn(mysocket,buffer,BUFFER_SZ) != BUFFER_SZ) {
//something really bad is going on.
}
The behavior is determined by two things. The recv low water mark and whether or not you pass the MSG_WAITALL flag. If you pass this flag the call will block until the requested number of bytes are received, even if the server crashes. Other wise it returns as soon as at least SO_RCVLOWAT bytes are available in the socket's receive buffer.
SO_RCVLOWAT
Sets the minimum number of bytes to
process for socket input operations.
The default value for SO_RCVLOWAT is
1. If SO_RCVLOWAT is set to a larger value, blocking receive calls normally
wait until they have received the
smaller of the low water mark value or
the requested amount. (They may return
less than the low water mark if an
error occurs, a signal is caught, or
the type of data next in the receive
queue is different than that returned,
e.g. out of band data). This option
takes an int value. Note that not all
implementations allow this option to
be set.
If you read the quote precisely, the most common scenario is:
the socket is receiving data. That 100 bytes will take some time.
the recv() call is made.
If there are more than 0 bytes in the buffer, recv() returns what is available and does not wait.
While there are 0 bytes available it blocks and the granularity of the threading system determines how long that is.