Socket programming read() is reading all of my writes() - c

I have a client and a server. I have two read() in my client and two write() in my server code. The server sends data to the client on the first write(), the client reads and stores to a buffer but it doesn't stop reading, it keeps reading through the server's second write() because in my client i have it set up to read 255 in the stream(from my understanding). I put 255 because i don't know how long the data datasize for first write() is. How do i fix this?
Client:
n = read(sockfd,buffer,255);
if (n < 0)
error("ERROR reading from socket");
printf("%s\n",buffer);
n = read(sockfd,buffer,255);
if (n < 0)
error("ERROR reading from socket");
printf("%s\n",buffer);
Server:
n = write(newsockfd,datasize,strlen(datasize));
if (n < 0) error("ERROR writing to socket");
n = write(newsockfd,data,255);
if (n < 0) error("ERROR writing to socket");

What you are experiencing is how TCP works. If the server calls write() multiple times before the client calls read(), then read() can receive everything that was previously written, up to the maximum buffer size that you specify. TCP has no concept of message boundaries, like UDP does. There is nothing wrong with that. You just need to account for it, that's all.
If you need to know where one message ends and the next begins, then you simply need to frame your messages. There are a couple of different ways you can do that.
Send the data length before sending the actual data, so the client knows how much data to read, eg:
Server:
int datalen = ...; // # of bytes in data
int tmp = htonl(datalen);
n = write(newsockfd, (char*)&tmp, sizeof(tmp));
if (n < 0) error("ERROR writing to socket");
n = write(newsockfd, data, datalen);
if (n < 0) error("ERROR writing to socket");
Client:
int buflen;
n = read(sockfd, (char*)&buflen, sizeof(buflen));
if (n < 0) error("ERROR reading from socket");
buflen = ntohl(buflen);
n = read(sockfd, buffer, buflen);
if (n < 0) error("ERROR reading from socket");
else printf("%*.*s\n", n, n, buffer);
wrap the data with delimiters that do not appear in the actual data, then the client can keep reading and look for those delimiters. Use whatever delimiters make sense for your data (STX/ETX, line breaks, special reserved characters, etc):
Server:
char delim = '\x2';
n = write(newsockfd, &delim, 1);
if (n < 0) error("ERROR writing to socket");
n = write(newsockfd, data, datalen);
if (n < 0) error("ERROR writing to socket");
delim = '\x3';
n = write(newsockfd, &delim, 1);
if (n < 0) error("ERROR writing to socket");
Client:
char tmp;
do
{
n = read(sockfd, &tmp, 1);
if (n < 0) error("ERROR reading from socket");
if (tmp != '\x2')
continue;
buflen = 0;
do
{
n = read(sockfd, &tmp, 1);
if (n < 0) error("ERROR reading from socket");
if (tmp == '\x3')
break;
// TODO: if the buffer's capacity has been reached, either reallocate the buffer with a larger size, or fail the operation...
buffer[buflen] = tmp;
++buflen;
}
while (1);
printf("%*.*s\n", buflen, buflen, buffer);
break;
}
while (1);

You can't assume that one read will read exactly what was written by one write. TCP is a byte stream protocol. No message boundaries. read() can read as little as one byte and as much as the length you provide, depending on what data has arrived, which you can't control at either the sending or the receiving end. You also can't control whether TCP coalesces outgoing writes into one segment.
If you want messages, you have to implement them yourself, e.g. lines, length-word prefix, type-length-value, STX/ETX, XML, ...
NB When you get an error, don't just print a message of your own devising. Print the error. In this case, call 'perror()', or make up a formatted string with 'strerror'.

You've got the right idea about sending the length data before sending the actual data, but you're sending datasize in the wrong format. Sending it as an ascii string means the length of datasize will vary depending on the length of data:
For instance:
If data is 5 bytes in length, datasize will be "5".
If data is 100 bytes in length, datasize will be "100".
Unfortunately when it comes to serializing data, this just won't work, datasize must always take up the same number of bytes. You need to write this into the socket as an integer, and read it again at the other end as an integer. Then write this exact number of bytes of data into the socket and read this exact number of bytes of data at the other end:
For example:
#include <stdio.h>
#include <stdint.h>
#include <string.h>
void send(int sock)
{
const char* msg = "this is a message!";
uint16_t len = strlen(msg);
uint16_t networkLen = htons(len); // convert to network byte order
write(sock, &networkLen, sizeof(networkLen));
write(sock, msg, len);
}
void receive(int sock)
{
char msg[1024];
uint16_t networkLen;
read(sock, &networkLen, sizeof(networkLen));
uint16_t len = ntohs(networkLen); // convert back to host byte order
read(sock, msg, sizeof(msg) - 1);
msg[len] = '\0';
printf("%u %s\n", len, msg);
}
int main(int argc, char** argv)
{
int sockets[2];
pipe(sockets);
send(sockets[1]);
receive(sockets[0]);
}

Socket == stream of bytes, it doesn't make some packetizing etc. So if Server should send 2 packets to client, you need to do something so client can distinguish each of them. For instance, if you decide that server should send 2 packets by 255 bytes each, your client procedure which receives one packet would look like this:
int count = 0;
while (count < 255) {
n = read(sockfd, buffer + count, 255 - count);
if (n < 0) {
error("ERROR reading from socket");
printf("%s\n",buffer);
return;
}
count += n;
}
// here buffer has 255 bytes for the packet
The program has cycle, because you can receive any number in read result in between 0..255, or negative if the socket was closed.
you can do the same for second packet.
If your packets are different in size, then you have to tell client from server what the packet size is. You can send in first 2 bytes the length of your packet, and use the number instead of 255 constant in code above.

Related

How a client in Matlab can send binary data to server in C?

I want to send data from a client written in Matlab to a server written in C. When reading in the server, not all data is received in one read action and it is divided into two consecutive reads.
The Matlab code contains the code for a TCP server that receives some data, then does some processing and then writes (sends) the output of the processing over a TCP socket as a client to the server in C.
Here is the code for the client written in Matlab.
% A TCP server object defined for getting raw data and processing
% it. The object is called 'TCPServer'. It is not related to the
% problem.
% TCP client for sending the data to the C server
TCPClient = tcpip('192.168.1.2', 8080, 'NetworkRole', 'client');
set(TCPClient,'OutputBufferSize', 1464);
% 1464 is total number of bytes of the struct to be sent
% to the C server.
set(TCPClient,'Timeout', 10);
fopen(TCPClient);
while (1)
while(1) % Waits for incoming CSI data
nBytes = get(TCPServer,'BytesAvailable');
if nBytes >= bufferLen
disp('Data received');
break;
end
end
data = fread(TCPServer,nBytes,'uint8');
flushinput(TCPServer);
% Does some processing and generate 'spec' and 'doas'
% arrays to be sent to the C server
message = [typecast(spec, 'uint8') typecast(doas, 'uint8')];
fwrite(TCPClient, message);
end
Here is the code written in C for the server receiving the data from the client in Matlab.
#define SA struct sockaddr
struct doa_struct {
double spec[181], doa[2];
};
// Function that reads received data
void func(int sockfd)
{
struct doa_struct *doa_data;
unsigned char buff[1464];
int num_bytes;
// infinite loop receiving data
for (;;) {
bzero(buff, 1464);
// read the data from client and copy it in buffer
num_bytes = read(sockfd, buff, 1464);
// Get the buffer which contains the client contents
doa_data = (struct doa_struct *) buff;
printf("doa: %f\t%f\t%d\n", doa_data->doa[0], doa_data->doa[1], num_bytes);
}
}
// Driver function
int main()
{
int sockfd, connfd, len;
struct sockaddr_in servaddr, cli;
// socket create and verification
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd == -1) {
printf("socket creation failed...\n");
exit(0);
}
else
printf("Socket successfully created..\n");
bzero(&servaddr, sizeof(servaddr));
// assign IP, PORT
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port = htons(8080);
// Binding newly created socket to given IP and verification
if ((bind(sockfd, (SA*)&servaddr, sizeof(servaddr))) != 0) {
printf("socket bind failed...\n");
exit(0);
}
else
printf("Socket successfully binded..\n");
// Now server is ready to listen and verification
if ((listen(sockfd, 5)) != 0) {
printf("Listen failed...\n");
exit(0);
}
else
printf("Server listening..\n");
len = sizeof(cli);
// Accept the data packet from client and verification
connfd = accept(sockfd, (SA*)&cli, &len);
if (connfd < 0) {
printf("server acccept failed...\n");
exit(0);
}
else
printf("server acccept the client...\n");
// Function for chatting between client and server
func(connfd);
// After chatting close the socket
close(sockfd);
}
I receive the following result on the server.
doa: 0.000000 0.000000 1408
doa: 0.000000 0.000000 56
Each line printed has three values. The first two are the received data that should not be zero. The last number is the number of bytes received by read() function in the server. The sum of bytes received by the server in the two lines (two read() operations) is 1464 which is the number of bytes sent by the client in one fwrite() operation.
The correct output would be one line like the following:
doa: 25.000000 45.000000 1464
Two non-zero data values received by a transfer of 1464 bytes. I have also checked the client code. The client sends (writes) 1464 bytes by fwrite() operation in Matlab.
It's completely normal for a read() call to only return some of the data you're expecting. This happens because at a low level, the network stack breaks the overall data stream up into multiple fixed-size packets for transmission over the wire. From the read(2) man page:
RETURN VALUE ... It is not an error if this number is smaller than the number of bytes requested; this
may happen for example because fewer bytes are actually available right now (maybe
because we were close to end-of-file, or because we are reading from a pipe, or from a
terminal), or because read() was interrupted by a signal.
When receiving network data, you'll need to keep calling read() until you've received the expected number of bytes. For example, something like this (untested code):
void func(int sockfd)
{
struct doa_struct *doa_data;
int expected_bytes = 1464;
unsigned char buff[expected_bytes];
int num_bytes;
// Read a message
bzero(buff, 1464);
int bytes_read = 0;
while (bytes_read < expected_bytes) {
// read data into buff until we've read all the expected bytes
// NOTE: if the Matlab side sends a malformed message, this could
// hang in this loop forever...for real-world use, you'd need to
// account for those types of scenarios.
num_bytes = read(sockfd, buff + bytes_read, expected_bytes - bytes_read);
if (num_bytes <= 0) {
// handle error cases...
return;
}
bytes_read += num_bytes;
}
// Get the buffer which contains the client contents
doa_data = (struct doa_struct *) buff;
printf("doa: %f\t%f\t%d\n", doa_data->doa[0], doa_data->doa[1], num_bytes);
}
As for why your output is printing 0's instead of the expected values: it may be because of your printf format specifier. Your compiler may require "%lf" to print doubles, rather than "%f". Traditionally, the "%lf" was required for doubles, which are 64-bit values rather than the 32-bit float values. However, C99 and later compilers can blur this line -- see this related stack overflow question.

Socket Send And Receive Buffer

I tried to learn about the conception of Socket-Send(Receive)-Buffer.And I wrote these codes:
Client:
int client = socket(AF_INET, SOCK_STREAM, 0);
int s = getsockopt(client, SOL_SOCKET, SO_SNDBUF, &sendBuffSize, &len);
int status = connect(client, (struct sockaddr*)&addr, sizeof(struct sockaddr_in));
printf("The send buff size is : %d.\n", sendBuffSize);
char buf[100000];
int n, wn;
int fd = open("./1.txt", O_RDONLY);
while ((n = read(fd, buf, sizeof(buf))) > 0) {
wn = write(client, buf, n);
printf("Write %d bytes.\n", wn);
}
Server: I set the connected client as Non-block,and add this client into the epoll.Once the client sends data to the server, I put the main thread into sleep[ten seconds].
char buf[8192];
sleep(10);
int rn;
while ((rn = read(events[i].data.fd, buf, sizeof(buf))) > 0) {
printf("Read %d bytes.\n", rn);
}
The client send Buffer size is 16384 and the server receive Buffer size is 20000[setsockopt].
According to the book:The client calls the write function will block if the socket send buffer is full.
But I get the result[Client] :
Result
And the server :
Result
Questions:
Receive buffer size + Send buffer size < 100000; but why the write function do not block?
Why the server read 8192 + 6808 = 15000 bytes instead of read continuously 8192 bytes?
There is no evidence here that the client writes did not block. On the contrary, the fact that all the writes were 100,000 bytes except the last, when you ran out of input, shows that it must have blocked, to transfer all that data into a socket buffer that is smaller.
TCP segmentizes, and IP packetises, the data sent over the wire. You have no control over that process. In any case a read() can transfer any amount of bytes from 1 up to the count supplied, or zero upwards in non-blocking mode. It is a streaming protocol, not a messaging protocol. There is no other guarantee about how much any individual read() will return at a time.

How can I pass integer through the buffer?

I want to create a socket that can read some numbers rather than string so that I can perform some manipulations on them.
How can I send integer instead of string
This is my current program:
int main(int argc, char *argv[])
{
int sockfd, newsockfd, port, clilen;
struct sockaddr_in serv_addr, cli_addr;
if (argc < 2)
error("ERROR, no port provided\n");
port = atoi(argv[1]);
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0)
error("ERROR opening socket");
bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = INADDR_ANY;
serv_addr.sin_port = htons(port); //host to network
if (bind(sockfd, (struct sockaddr *)&serv_addr,sizeof(serv_addr)) < 0)
error("ERROR binding to socket");
listen(sockfd,2);
clilen = sizeof(cli_addr);
newsockfd = accept(sockfd,(struct sockaddr *)&cli_addr, &clilen);
int n;
void* buffer[256];
n = read(newsockfd,buffer,255);
if (n < 0) error("ERROR reading from socket");
printf("Message received:%d\n",d);
n = write(newsockfd,buffer, strlen((char *)buffer));
if (n < 0)
error("ERROR writing back to socket");
return 0;}
I tried this:
int n= 5;
write(newsockfd,&n, sizeof(int));
But on the client side (also a void* buffer), it doesn't seem to read anything. Can anyone help me how can I rectify this?
One solution could be to pass strings then convert to int etc. But is that the proper way?
The problem is two-fold:
The first problem is that the declaration void* buffer[256] declares buffer to be an array of 256 pointers to void, not really what's intended I guess. Instead if you want an array of 256 bytes just do char buffer[256].
The second problem is that on the receiving side you treat the data as a string, but it's not a string it's a binary data value.
The second problem can be solved in two ways: Either you convert the integer to a string, and send that string. Or you receive the raw data of the integer, and convert it to an int. The first solution involves sprintf, and the other solution involves either memcpy or pointers and casting.
For the first solution above, you could do something like this:
char temp[32];
snprintf(temp, sizeof temp, "%d", n);
write(newsockfd, temp, sizeof temp);
On the receiving end you read it as a string:
char buffer[32];
n = read(newsockfd, buffer, sizeof buffer);
if (n <= 0)
{
// Error or connection closed
}
// TODO: Since TCP is a streaming protocol, we could actually receive
// less than the number of bytes we asked to read, so we need
// to read in a loop
// Print the received data as a string
printf("Received '%s'\n", buffer); // Works because the sender sent with the terminator
// Convert to integer
int i = strtol(buffer, NULL, 10);
printf("Received %d\n", i);
For the second solution above, send the integer value as raw binary data as you already do, but when receiving this do something like
char buffer[32];
n = read(newsockfd, buffer, sizeof buffer);
if (n <= 0)
{
// Error or connection closed
}
// TODO: Since TCP is a streaming protocol, we could actually receive
// less than the number of bytes we asked to read, so we need
// to read in a loop
int i = *(int *) buffer;
printf("Received %d\n", i);
Note that this second solution, as it is written in this answer, relies that both the sender and the receiver have the same endianness, as well as the same size for int (which might not always be the case).
The safest way is generally to convert to a string on the sending side, and receive the string as, well, a string. I.e. the first solution.
I'd like to modify the lucid answer Joachim Pileborg provided.
int value, todo = sizeof(value);
char *p = (void*) &value;
while ( (n = read(newsockfd, p, todo)) < todo ) {
if( n == 0 ) { /* handle eof and break */ }
if( n < 0 ) { /* handle error and break */ }
todo -= n;
p += n
}
if( todo == 0 ) {
printf("Received %d\n", value);
}
You know how many bytes you're receiving, so there's no reason to read more. The loop deals with case he warns of: that read(2) may fail to read all 4 bytes (unlikely in this case, but not in general).
You can always get char * from void *, and pass any pointer to a function accepting void *. What you cannot do, portably, is
char buffer[32];
...
int i = *(int *) buffer;
because a character array is not guaranteed to have integer alignment. On some architectures, that provokes SIGBUS. If you read into character array, the safe way to interpret those bytes as an integer is
memcpy(&i, buffer, sizeof i);

How to solve: sending UDP packet using Sendto() got "message too long"

I want to use the sendto() API to send video and audio data through UDP packet. The sending buffer size I got using getsockopt() is 114688, however, sendto() returned -1 when the data packet less than 65536 not 114688. And the error message is Message too long.
When I used setsockopt() to adjust the sending buffer size as 200000, I used getsockopt() and found the sending buffer size was not 200000 but 262142. So I still got the same error when I sent data packet with a size bigger than 65536.
I am confused about this situation. I want to know what the reason is and how to solve this problem.
When I used FFMPEG library to send the video and audio packet, there is no error. So I am sure there is a solution for this problem and I missed something.
Is there anyone can help me about this problem? I really can not understand what the reason is.
The OS I used is ubuntu 11.04,I got the same results in ubuntu 11.10.
That is the code I used to create socket and configure the parameter:
unsigned char *output_buffer = (unsigned char*)av_malloc(IO_BUFFER_SIZE);
if (NULL == output_buffer) {
printf("Couldn't allocate input buffer.\n");
return NULL;
}
output_context_data_t *context_data = (output_context_data_t *)malloc(sizeof(output_context_data_t));
if (NULL == context_data) {
printf("Could not allocate output context data.\n");
av_free(output_buffer);
return NULL;
}
context_data->socket = socket(AF_INET, SOCK_DGRAM, 0);
if(context_data->socket < 0) {
printf("socket creating fail!\n");
return NULL;
}
context_data->socket_addr->sin_family = AF_INET;
context_data->socket_addr->sin_port = htons(output_port);
ret = inet_pton(AF_INET, output_ip, &(context_data->socket_addr->sin_addr));
if(0 == ret) {
printf("inet_pton fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, IPPROTO_IP, IP_MULTICAST_TTL,
&option_ttl, sizeof(int));
if(ret < 0) {
printf("ttl configuration fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(int));
if(ret < 0) {
printf("resue configuration fail!\n");
return NULL;
}
That is the code to send UDP packet:
int send_size = sendto(context_data->socket, buf, buf_size, 0,
(struct sockaddr *)context_data->socket_addr, sizeof(*context_data->socket_addr)));
//the video or audio data is in buf and its size is buf_size.
That is the code I used to get the sending buffer size:
int bufsize;
int size = sizeof(bufsize);
getsockopt(context_data->socket,SOL_SOCKET, SO_SNDBUF, &bufsize, &size);
That is the code I used to configure the sending buffer size:
tmp = 200000;
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_SNDBUF, &tmp, sizeof(tmp));
if(ret < 0) {
printf("sending buffer size configuration fail!\n");
return NULL;
}
You can not send messages (datagrams) larger than 2^16 65536 octets with UDP. The length field of a UDP packet is 16 bits. The buffer sizes you're requesting are not about the size for a packet, but how many octets the OS does buffer incoming and outgoing in total (spread over multiple packets). But a single packet can not get larger.
Per #datenwolf's answer, you simply can't send more than 64k in a single UDP datagram, as that limit is implicit in the two-byte length field in the protocol.
Furthermore, it's not actually a good idea to send even that much at once. You should limit your packets to the MTU on the path between the two ends (typically in the region of 1500 bytes or less) so that you don't get fragmentation in the IP layer.
Fragmentation is bad - ok?
Why not just call sendto several times, with an offset into the buffer?
int sendto_bigbuffer(int sock, const void *buffer, const size_t buflen, int flags,
const struct sockaddr *dest_addr, socklen_t addrlen)
{
size_t sendlen = MIN(buflen, 1024);
size_t remlen = buflen;
const void *curpos = buffer;
while (remlen > 0)
{
ssize_t len = sendto(sock, curpos, sendlen, flags, dest_addr, addrlen);
if (len == -1)
return -1;
curpos += len;
remlen -= len;
sendlen = MIN(remlen, 1024);
}
return buflen;
}
Something like the above function will send the buffer 1024 bytes at a time.

How can I read an arbitrarily large file to a TCP Socket in C?

I am just learning socket programming and I am trying to write an echo client that reads from stdin and writes to a socket and then reads the server response from the socket to stdout. The problem is that I don't know how long stdin will be or how long the server's response will be. The code I am trying to use is as follows (creating the socket and connecting to the server are left out):
length = BUF_SIZE;
while (length == BUF_SIZE) { // length will equal BUF_SIZE if buf is full, when length < BUF_SIZE we have reached an EOF
// Reads from STDIN to buf
if ((length = read(STDIN_FILENO, buf, BUF_SIZE)) < 0){
fprintf(stderr, "Error in reading from STDIN");
return 4;
}
// Writes from buf to the socket
if ((write(sock, buf, BUF_SIZE)) < 0){
fprintf(stderr, "Error writing to socket");
return 5;
}
}
if ((status = shutdown(sock, 1)) < 0){ // Shuts down socket from doing more receives
fprintf(stderr, "Error shutting down socket for writing");
return 6;
}
length = BUF_SIZE;
while (length == BUF_SIZE){
// Read from socket to buf
if ((length = read(sock, buf, BUF_SIZE)) < 0){
fprintf(stderr, "Error reading from socket");
return 7;
}
// Write from buf to STDOUT
if ((write(STDOUT_FILENO, buf, BUF_SIZE)) < 0){
fprintf(stderr, "Error writing to STDOUT");
return 8;
}
}
close(sock);
exit(0);
BUF_SIZE is defined as 100. When I run my program the program typically connects to the server and sends the proper message, but what it writes to stdout is either nothing or gibberish.
What am I doing wrong?
Your while loop is only going to work the first time through. read()/write() are only going to return the amount that they actually read/write which may well not be equal to BUF_SIZE. Let's say you read ten bytes from the socket and then you write a hundred to stdout - the last 90 are going to be garbage.
Something along these lines will get you somewhat closer to what you want.
while (1)
{
if ((length = read(STDIN_FILENO, buf, BUF_SIZE)) < 0)
{
fprintf(stderr, "Error in reading from STDIN");
return 4;
}
if ((write(sock, buf, length)) < 0)
{
fprintf(stderr, "Error writing to socket");
return 5;
}
if ((length = read(sock, buf, BUF_SIZE)) < 0)
{
fprintf(stderr, "Error reading from socket");
return 7;
}
if ((write(STDOUT_FILENO, buf, length)) < 0)
{
fprintf(stderr, "Error writing to STDOUT");
return 8;
}
}

Resources