I Want to measure how much time takes for each socket to transfer 100MB data file so I implement many of sockets type like TCP, some pipe's mmaps etc..
The process to measure is (on the client side )::= before I send the data I take time and after the data is sent I stop the time.
(On the Server side ) ::= before I receive the data I take time and after the file is written I take again time.
the problem is on the server side I never out from the while loop in UDP / UDS Dgram
so I never can measure the time it takes to transfer data between them.
For example here is half of my server the while part in UDP ipv6 server:
clock_t start = 0, end = 0;
FILE *fp;
char *filename = "new_data.txt";
fp = fopen(filename, "w");
/* now wait until we get a datagram */
printf("waiting for a datagram...\n");
clilen = sizeof(client_addr);
start = clock();
while (1) {
ssize_t bytes = recvfrom(sock, buffer, 1024, 0,
(struct sockaddr *) &client_addr,
&clilen);
if (bytes < 0) {
perror("recvfrom failed");
exit(4);
}
fprintf(fp, "%s", buffer);
bzero(buffer, 1024);
}
close(sock);
//fclose(fp);
end = clock();
double cpu_time_used = ((double) (end - start)) / CLOCKS_PER_SEC;
printf("Receive in %f seconds\n", cpu_time_used);
One way to solve it would be to have an extra byte in your buffer that is a flag that indicates if there is any more data to come after this packet. Clear the flag when there is more data, set the flag when there is not. Something like:
Client:
#define CHUNK_SIZE 1024
char buffer[CHUNK_SIZE + 1] = { 0 };
while ((num_bytes = fread(buffer + 1, 1, CHUNK_SIZE, fp)) > 0)
{
sendto(..., buffer, 1 + num_bytes, ...);
}
buffer[0] = 1;
sendto(sock, buffer, 1, ...);
Server:
// Same chunk size and buffer as above
while ((num_bytes = recvfrom(sock, buffer, CHUNK_SIZE + 1, ...)) > 0)
{
if (num_bytes > 1)
{
fwrite(buffer + 1, 1, num_bytes - 1, fp);
}
if (buffer[0] == 1)
break;
}
There are other ways, such as having a TCP control socket running at the same time, over which you send out-of-band messages to indicate when the transmission is finished, but that is much more complex.
Related
I'm trying to write a program which either reads from a socket or from stdin. If socket then print to stdout (to user), otherwise print to socket. So far I'm using poll to awake program when there's activity on either, my problem is that after the initial connection, poll always reports activity on the socket, even though client hasn't written anything else.
Is there any way to distinguish between "someone has connected to the socket" and "someone put a message on the channel"? Looking through man pages for poll, select and others I don't really see an appropriate flag.
If there is no appropriate flag, what would be a way to accomplish what I'm trying to do ?
What I currently have is something like below. When I enter something in stdin, poll returns and the loop determines the activity was on the socket (wrong).
edit: There's other issues at well but this is what I'm struggling with right now.
/* blocking accept for first and only connection */
newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
/* poll stdin and newsockfd */
struct pollfd fds[2];
fds[0].fd = STDIN_FILENO;
fds[0].events = POLLIN;
fds[1].fd = newsockfd;
fds[1].events = POLLIN;
ioctl(newsockfd, FIONBIO, (char *)&on); /* int on = 1 */
while (1) {
int rc = poll(fds, 2, -1);
if (rc <= 0)
exit(1);
for (int i = 0; i < 2; i++) {
if (fds[i].revents == 0)
continue;
if (fds[i].revents != POLLIN)
exit(1);
if (fds[i].fd = newsockfd) {
n = read(newsockfd, buffer, 255);
printf("read %d chars from newsockfd: %s\n", n, buffer);
} else if (fds[i].fd = STDIN_FILENO) {
read(STDIN_FILENO, buffer, 255);
n = dprintf(newsockfd, "%s", buffer);
printf("wrote %d chars to newsockfd: %s\n", n, buffer);
}
}
}
I tried to learn about the conception of Socket-Send(Receive)-Buffer.And I wrote these codes:
Client:
int client = socket(AF_INET, SOCK_STREAM, 0);
int s = getsockopt(client, SOL_SOCKET, SO_SNDBUF, &sendBuffSize, &len);
int status = connect(client, (struct sockaddr*)&addr, sizeof(struct sockaddr_in));
printf("The send buff size is : %d.\n", sendBuffSize);
char buf[100000];
int n, wn;
int fd = open("./1.txt", O_RDONLY);
while ((n = read(fd, buf, sizeof(buf))) > 0) {
wn = write(client, buf, n);
printf("Write %d bytes.\n", wn);
}
Server: I set the connected client as Non-block,and add this client into the epoll.Once the client sends data to the server, I put the main thread into sleep[ten seconds].
char buf[8192];
sleep(10);
int rn;
while ((rn = read(events[i].data.fd, buf, sizeof(buf))) > 0) {
printf("Read %d bytes.\n", rn);
}
The client send Buffer size is 16384 and the server receive Buffer size is 20000[setsockopt].
According to the book:The client calls the write function will block if the socket send buffer is full.
But I get the result[Client] :
Result
And the server :
Result
Questions:
Receive buffer size + Send buffer size < 100000; but why the write function do not block?
Why the server read 8192 + 6808 = 15000 bytes instead of read continuously 8192 bytes?
There is no evidence here that the client writes did not block. On the contrary, the fact that all the writes were 100,000 bytes except the last, when you ran out of input, shows that it must have blocked, to transfer all that data into a socket buffer that is smaller.
TCP segmentizes, and IP packetises, the data sent over the wire. You have no control over that process. In any case a read() can transfer any amount of bytes from 1 up to the count supplied, or zero upwards in non-blocking mode. It is a streaming protocol, not a messaging protocol. There is no other guarantee about how much any individual read() will return at a time.
I'm trying to read data from a UDP socket, but after reading the first 255 bytes, read() seems to drop the rest of the data on the socket and block until another data-gram comes in.
Here's the network code I'm using:
int sock;
struct sockaddr_in remote_addr, self_addr;
uint8_t network_init(uint16_t port)
{
memset((char *) &remote_addr, 0, sizeof(remote_addr));
remote_addr.sin_family = AF_INET;
remote_addr.sin_addr.s_addr = inet_addr("192.168.1.22");
remote_addr.sin_port = htons(3001);
memset((char *) &self_addr, 0, sizeof(self_addr));
self_addr.sin_family = AF_INET;
self_addr.sin_addr.s_addr = INADDR_ANY;
self_addr.sin_port = htons(3001);
if ((sock = socket(AF_INET, SOCK_DGRAM, 0)) < 0)
{
fprintf(stderr, "Could not create socket.");
return 1;
}
else if (bind(sock, (struct sockaddr *) &self_addr, sizeof(self_addr)) != 0)
{
fprintf(stderr, "Could not bind to socket.");
return 1;
}
return 0;
}
void network_send(uint8_t *data, uint8_t len)
{
sendto(sock, data, len, 0, (struct sockaddr *) &remote_addr, sizeof(remote_addr));
}
void read_data()
{
int len = 0;
ioctl(sock, FIONREAD, &len);
// We have data
if (len > 0)
{
char *buffer = (char *) malloc(256);
uint8_t buflen;
printf("==== %d | Data:\n", len);
while (len > 0)
{
buflen = min(255, len);
len = len - buflen;
buffer[buflen] = '\0';
printf("len: %d, buflen: %d,\n",len, buflen);
read(sock, buffer, buflen);
printf("%s\n", buffer);
}
printf("\n");
}
}
Here's the command I'm using to send data:
echo -n '12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567' | nc -u localhost 3001
And here's the output:
==== 257 | Data:
len: 2, buflen: 255,
123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345
len: 0, buflen: 2,
^C
Also, after performing this read, ioctl(sock, FIONREAD, &len); produces a length result of 0. My suspicion is that for some reason, read() is clearing out the rest of the data before it has a chance to be read, but I can't seem to find any reference to this behaviour in any documentation.
I'm developing on an Ubuntu linux machine (x86_64).
With UDP sockets, each call to read() reads a whole datagram out of the kernel. If the read buffer isn't big enough for the entire datagram, the rest of it will be discarded. It's not like a stream socket, where you can keep calling until you get everything.
Since FIONREAD tells you the number of bytes in the message, you should use that as the size to malloc() rather than using 256:
if (len > 0) {
char *buffer = malloc(len);
...
P.S. Do I cast the result of malloc?
I want to use the sendto() API to send video and audio data through UDP packet. The sending buffer size I got using getsockopt() is 114688, however, sendto() returned -1 when the data packet less than 65536 not 114688. And the error message is Message too long.
When I used setsockopt() to adjust the sending buffer size as 200000, I used getsockopt() and found the sending buffer size was not 200000 but 262142. So I still got the same error when I sent data packet with a size bigger than 65536.
I am confused about this situation. I want to know what the reason is and how to solve this problem.
When I used FFMPEG library to send the video and audio packet, there is no error. So I am sure there is a solution for this problem and I missed something.
Is there anyone can help me about this problem? I really can not understand what the reason is.
The OS I used is ubuntu 11.04,I got the same results in ubuntu 11.10.
That is the code I used to create socket and configure the parameter:
unsigned char *output_buffer = (unsigned char*)av_malloc(IO_BUFFER_SIZE);
if (NULL == output_buffer) {
printf("Couldn't allocate input buffer.\n");
return NULL;
}
output_context_data_t *context_data = (output_context_data_t *)malloc(sizeof(output_context_data_t));
if (NULL == context_data) {
printf("Could not allocate output context data.\n");
av_free(output_buffer);
return NULL;
}
context_data->socket = socket(AF_INET, SOCK_DGRAM, 0);
if(context_data->socket < 0) {
printf("socket creating fail!\n");
return NULL;
}
context_data->socket_addr->sin_family = AF_INET;
context_data->socket_addr->sin_port = htons(output_port);
ret = inet_pton(AF_INET, output_ip, &(context_data->socket_addr->sin_addr));
if(0 == ret) {
printf("inet_pton fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, IPPROTO_IP, IP_MULTICAST_TTL,
&option_ttl, sizeof(int));
if(ret < 0) {
printf("ttl configuration fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(int));
if(ret < 0) {
printf("resue configuration fail!\n");
return NULL;
}
That is the code to send UDP packet:
int send_size = sendto(context_data->socket, buf, buf_size, 0,
(struct sockaddr *)context_data->socket_addr, sizeof(*context_data->socket_addr)));
//the video or audio data is in buf and its size is buf_size.
That is the code I used to get the sending buffer size:
int bufsize;
int size = sizeof(bufsize);
getsockopt(context_data->socket,SOL_SOCKET, SO_SNDBUF, &bufsize, &size);
That is the code I used to configure the sending buffer size:
tmp = 200000;
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_SNDBUF, &tmp, sizeof(tmp));
if(ret < 0) {
printf("sending buffer size configuration fail!\n");
return NULL;
}
You can not send messages (datagrams) larger than 2^16 65536 octets with UDP. The length field of a UDP packet is 16 bits. The buffer sizes you're requesting are not about the size for a packet, but how many octets the OS does buffer incoming and outgoing in total (spread over multiple packets). But a single packet can not get larger.
Per #datenwolf's answer, you simply can't send more than 64k in a single UDP datagram, as that limit is implicit in the two-byte length field in the protocol.
Furthermore, it's not actually a good idea to send even that much at once. You should limit your packets to the MTU on the path between the two ends (typically in the region of 1500 bytes or less) so that you don't get fragmentation in the IP layer.
Fragmentation is bad - ok?
Why not just call sendto several times, with an offset into the buffer?
int sendto_bigbuffer(int sock, const void *buffer, const size_t buflen, int flags,
const struct sockaddr *dest_addr, socklen_t addrlen)
{
size_t sendlen = MIN(buflen, 1024);
size_t remlen = buflen;
const void *curpos = buffer;
while (remlen > 0)
{
ssize_t len = sendto(sock, curpos, sendlen, flags, dest_addr, addrlen);
if (len == -1)
return -1;
curpos += len;
remlen -= len;
sendlen = MIN(remlen, 1024);
}
return buflen;
}
Something like the above function will send the buffer 1024 bytes at a time.
Any ideas why when the server writes a socket while the client is waiting on select, select never finishes?
I am using c to communicate between sockets. My client connects to my server fine.
socket_desc=socket(AF_INET,SOCK_STREAM,0);//create the socket descriptor
client->address.sin_addr.s_addr = inet_addr(ipAddress);
client->address.sin_family = AF_INET;
client->address.sin_port = htons(port);
bind(socket_desc,&address,sizeof(address));
connect(socket_desc, &address, sizeof(address));
When I use recv to block and listen for data, everything works fine:
int bytesRead = 1;
while(bytesRead){
int bufsize=1024;
char *buffer=malloc(bufsize);
bytesRead = recv(socket_desc, buffer, bufsize, 0);
printf("CLIENT RECV: %s", buffer);
}
If I try to use select, it doesn't seem to read any data. If I add STDIN to the fd_set, I can force it to read from the socket, but select doesn't seem to get triggered from the socket_desc reading in data...?
int running = 1;
while(running){
/* wait for something to happen on the socket */
struct timeval selTimeout;
selTimeout.tv_sec = 2; /* timeout (secs.) */
selTimeout.tv_usec = 0; /* 0 microseconds */
fd_set readSet;
FD_ZERO(&readSet);
FD_SET(STDIN_FILENO, &readSet);//stdin manually trigger reading
FD_SET(socket_desc, &readSet);//tcp socket
int numReady = select(3, &readSet, NULL, NULL, &selTimeout);
//IT ONLY GETS PAST SELECT ON RETURN FROM THE KEYBOARD
if(numReady > 0){
char buffer[100] = {'\0'};
int bytesRead = read(socket_desc, &buffer, sizeof(buffer));
printf("bytesRead %i : %s", bytesRead, buffer);
if(bytesRead == 0){
running = FALSE;
printf("Shutdowning client.\n");
}
}
The first parameter to select should be the maximum socket id plus 1. So in your case, it should be
socket_desc+1
Can you try with that and see if it works?
The reason it only gets when you press a key on the keyboard is because stdin is 0, which would be within 0 - (3 - 1) range, which is what is checked. If you set the first parameter to socket_desc+1, then 0 - (socket_desc) range should be checked for ready sockets