I want to use the sendto() API to send video and audio data through UDP packet. The sending buffer size I got using getsockopt() is 114688, however, sendto() returned -1 when the data packet less than 65536 not 114688. And the error message is Message too long.
When I used setsockopt() to adjust the sending buffer size as 200000, I used getsockopt() and found the sending buffer size was not 200000 but 262142. So I still got the same error when I sent data packet with a size bigger than 65536.
I am confused about this situation. I want to know what the reason is and how to solve this problem.
When I used FFMPEG library to send the video and audio packet, there is no error. So I am sure there is a solution for this problem and I missed something.
Is there anyone can help me about this problem? I really can not understand what the reason is.
The OS I used is ubuntu 11.04,I got the same results in ubuntu 11.10.
That is the code I used to create socket and configure the parameter：
unsigned char *output_buffer = (unsigned char*)av_malloc(IO_BUFFER_SIZE);
if (NULL == output_buffer) {
printf("Couldn't allocate input buffer.\n");
return NULL;
}
output_context_data_t *context_data = (output_context_data_t *)malloc(sizeof(output_context_data_t));
if (NULL == context_data) {
printf("Could not allocate output context data.\n");
av_free(output_buffer);
return NULL;
}
context_data->socket = socket(AF_INET, SOCK_DGRAM, 0);
if(context_data->socket < 0) {
printf("socket creating fail!\n");
return NULL;
}
context_data->socket_addr->sin_family = AF_INET;
context_data->socket_addr->sin_port = htons(output_port);
ret = inet_pton(AF_INET, output_ip, &(context_data->socket_addr->sin_addr));
if(0 == ret) {
printf("inet_pton fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, IPPROTO_IP, IP_MULTICAST_TTL,
&option_ttl, sizeof(int));
if(ret < 0) {
printf("ttl configuration fail!\n");
return NULL;
}
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(int));
if(ret < 0) {
printf("resue configuration fail!\n");
return NULL;
}
That is the code to send UDP packet：
int send_size = sendto(context_data->socket, buf, buf_size, 0,
(struct sockaddr *)context_data->socket_addr, sizeof(*context_data->socket_addr)));
//the video or audio data is in buf and its size is buf_size.
That is the code I used to get the sending buffer size：
int bufsize;
int size = sizeof(bufsize);
getsockopt(context_data->socket,SOL_SOCKET, SO_SNDBUF, &bufsize, &size);
That is the code I used to configure the sending buffer size：
tmp = 200000;
ret = setsockopt(context_data->socket, SOL_SOCKET, SO_SNDBUF, &tmp, sizeof(tmp));
if(ret < 0) {
printf("sending buffer size configuration fail!\n");
return NULL;
}
You can not send messages (datagrams) larger than 2^16 65536 octets with UDP. The length field of a UDP packet is 16 bits. The buffer sizes you're requesting are not about the size for a packet, but how many octets the OS does buffer incoming and outgoing in total (spread over multiple packets). But a single packet can not get larger.
Per #datenwolf's answer, you simply can't send more than 64k in a single UDP datagram, as that limit is implicit in the two-byte length field in the protocol.
Furthermore, it's not actually a good idea to send even that much at once. You should limit your packets to the MTU on the path between the two ends (typically in the region of 1500 bytes or less) so that you don't get fragmentation in the IP layer.
Fragmentation is bad - ok?
Why not just call sendto several times, with an offset into the buffer?
int sendto_bigbuffer(int sock, const void *buffer, const size_t buflen, int flags,
const struct sockaddr *dest_addr, socklen_t addrlen)
{
size_t sendlen = MIN(buflen, 1024);
size_t remlen = buflen;
const void *curpos = buffer;
while (remlen > 0)
{
ssize_t len = sendto(sock, curpos, sendlen, flags, dest_addr, addrlen);
if (len == -1)
return -1;
curpos += len;
remlen -= len;
sendlen = MIN(remlen, 1024);
}
return buflen;
}
Something like the above function will send the buffer 1024 bytes at a time.
Related
I have trouble reading RTP packets from a multicast socket which is opened using
the following function:
int
open_multicast_socket
(const char *group_address,
uint16_t port)
{
assert(group_address != NULL);
int
s;
if (-1 != (s = socket(
AF_INET, SOCK_DGRAM, 0
)))
{
int
reuse = 1;
if (-1 != setsockopt(
s, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof reuse
))
{
struct sockaddr_in
sock_addr;
bzero(&sock_addr, sizeof sock_addr);
if (1 == inet_pton(
AF_INET, group_address, &sock_addr.sin_addr
))
{
sock_addr.sin_family = AF_INET;
sock_addr.sin_port = htons(port);
if (0 == bind(
s, (struct sockaddr*)&sock_addr, sizeof sock_addr
))
{
struct ip_mreq
mreq = {
.imr_multiaddr.s_addr = inet_addr(group_address),
.imr_interface.s_addr = htonl(INADDR_ANY)
};
if (0 == setsockopt(
s, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof mreq
))
{
//fcntl(s, F_SETFL, O_NONBLOCK);
return s;
} // setsockopt
else
{
perror("setsockopt");
close(s);
}
} // bind
else
{
perror("bind");
close(s);
}
} // inet_pton
else
{
perror("inet_pton");
close(s);
}
} // setsockopt
else
{
perror("setsockopt");
close(s);
}
} // socket
else
{
perror("socket");
}
return -1;
}
If I read RTP header plus payload in one read operation, I get the entire
packet. However, if I attempt to receive the RTP header first, then - a custom
header in the payload - the 2nd read always gets a next RTP header instead,
discarding all attached data. Because payload length may vary, the only way to
receive a whole packet, it seems, is to guess its max possible size.
I tried to get a number of available bytes before reading:
ioctl(sock, FIONREAD, &nbytes);
but it always returns 0.
Polling on the socket always fails, as if no data is available at all.
When non-blocking is enabled (i.e. fcntl(sock, F_SETFL, O_NONBLOCK);) - read
always fails (-1), so does recv(sock, buf, buf_len, MSG_DONTWAIT).
So is there a way to properly parse RTP packets via consequensive non-blocking
read calls?
Non-blocking is essential, because it should be possible to check whether a connection was lost and re-open the socket if necessary.
Unlike TCP which is a stream based protocol, UDP is a packet based protocol. This means that whenever you read from a UDP socket (multicast or not) you'll get exactly one UDP datagram. If your buffer isn't big enough to hold the entire datagram, the remaining data is essentially lost.
Make sure your buffer is big enough to hold a complete datagram. If your network supports jumbo frames end-to-end that means your buffer should be 9000 bytes, otherwise it should be 1500 bytes.
One should read the complete buffer from the socket and then parse them.
One can create a buffer of MTU size, read from the socket to this temp buffer and then parser the complete buffer and then take action.
One can use select() or poll() to check if the data is present in the socket. Read it when it is available.
I wrote the code in order to handle receiving UDP packets. The packets are all same length(120 bytes), and about 1,000 packets are coming in every second. Simply, my code is like this.
int sock = -1;
int flag = 0;
int nRead = 0;
#define LOCAL_BUFF_SIZE (8192)
char buff[LOCAL_BUFF_SIZE];
struct sockaddr_in sockAddr;
memset((void *)&sockAddr, 0x00, sizeof(struct sockaddr_in));
if((sock = socket(PF_INET, SOCK_DGRAM, 0)) < 0)
{
/* Print error and terminate */
}
/* Make it non-blocking */
flag = fcntl( sock, F_GETFL, 0 );
fcntl( sock, F_SETFL, flag | O_NONBLOCK );
sockAddr.sin_family = AF_INET;
sockAddr.sin_port = htons(portNum);
sockAddr.sin_addr.s_addr = INADDR_ANY;
if(bind(sock, (struct sockaddr *)&sockAddr, sizeof (sockAddr)) < 0)
{
/* Print error and terminate */
}
while(...)
{
nRead = recv(sock, buff, LOCAL_BUFF_SIZE, 0);
if(nBytes > 0)
{
/* Process the data */
}
else
{
/* If it's error, handle error */
}
}
When I wrote this code, I expect that recv() function returns every bytes in the UDP socket buffer at that moment, but, it seems that it only returns one packet(120 byte) every time even though there are more bytes in the buffer. So now I encountered with packet loss. I know that there are many other ways to solve this problem, but, for now reading all existent bytes in the UDP buffer at once is the easiest way for me. So, is there any way to read all bytes in the UDP buffer at once?
Thanks in advance
UDP is a message oriented protocol, therefore, you are getting single message in one recv operation. You can possible use recvmmsg() system call to receive multiple messages in a single call.
I have already written a code for receiving files from Android to C, but the problem is i am receiving data loss. There is a mismatch of bytes when the received text file is checked with the original text file. How to eradicate this? I have given the code for reference.`
#include<stdio.h>
#include<unistd.h>
#include<string.h>
#include<stdlib.h>
#include<sys/ioctl.h>
#include<sys/types.h>
#include<arpa/inet.h>
#include<sys/socket.h>
int receive_text(long int new_socket)
{
int buffersize = 0, recv_size = 0, size = 0, read_size, write_size;
char verify = '1';
int errno;
FILE *text;
char *pBuf;
//Find the size of the text
recv(new_socket, (char *)&size, sizeof(int), 0);
//Send our verification signal
//send(new_socket, &verify, sizeof(char), 0);
text = fopen("/home/sosdt009/Desktop/received.txt", "w");
if (text == NULL)
{
puts("Error has occurred. Text file could not be opened \n");
return -1;
}
//Loop while we have not received the entire file yet
while (recv_size < size)
{
ioctl(new_socket, FIONREAD, &buffersize);
//We check to see if there is data to be read from the socket
if (buffersize > 0)
{
pBuf = malloc(buffersize);
if (!pBuf)
{
fprintf(stderr, "Memory Error. Cannot allocate!\n");
exit(-1);
}
//memset(pBuf,0,buffersize);
read_size = recv(new_socket, pBuf, buffersize, 0);
if (read_size < 0)
{
printf("%s", strerror(errno));
}
//Write the currently read data into our text file
write_size = fwrite(pBuf, 1, buffersize, text);
free(pBuf);
printf("%d \n", write_size);
//Increment the total number of bytes read
recv_size += write_size;
printf(" %d \n", recv_size);
}
}
fclose(text);
return 1;
}
int main(int argc , char *argv[]) {
int socket_desc , new_socket, c, read_size, buffer = 0;
struct sockaddr_in server , client;
char *readin;
//Create socket
socket_desc = socket(AF_INET,SOCK_STREAM,0);
if (socket_desc == -1)
{
printf("Could not create socket:");
}
//Prepare the sockaddr_in structure
server.sin_family = AF_INET;
server.sin_addr.s_addr = INADDR_ANY;
server.sin_port = htons( 6777 );
//Bind
if( bind(socket_desc,(struct sockaddr *)&server , sizeof(server)) < 0)
{
puts("bind failed");
return 1;
}
puts("Bind completed");
//Listen
listen(socket_desc,3);
//Accept and incoming connection
puts("Waiting for incoming connections...");
c = sizeof(struct sockaddr_in);
if((new_socket = accept(socket_desc,(struct sockaddr *)&client,(socklen_t *)&c)) )
{
puts("Connection accepted");
}
fflush(stdout);
close(socket_desc);
if (new_socket<0)
{
perror("Accept Failed");
return 1;
}
while(1)
{
receive_text(new_socket);
}
close(socket_desc);
return 0;
}
Here
read_size = recv(new_socket, pBuf, buffersize, 0);
recv() tells you how much bytess it received for the current iteration, namly read_size bytes.
And here
write_size = fwrite(pBuf, 1, buffersize, text);
you ignore the number of bytes received, but always write buffersize.
Fix this by writing to the target file the actual received amount of data:
write_size = fwrite(pBuf, 1, read_size, text);
The call to recv() receiving the number of bytes to come fully misses any error checking.
When transfering binary data you also need to make sure not to stumble over
possible different widths of an integer value on sender and receiver
different Endianness on sender and receiver
To take care of the 1st possible issue above use a well defined data type in terms of bit-width. Here int32_t instead of int or probably the unsigned pedant: unint32_t
For the 2nd possible pitfall tranfer data in network byte order. To achieve this do not send the plain int (or uint32_t) but convert it to network byte-order (prior to sending it) via a call to htonl(). On the receiving side then convert it back to host-byte order using ntohl().
The case that the sender closes the socket during transmission is ignored but should be handled by testing the result of recv() against 0, which indicates the closure of the socket by the sender.
Also casting int c to socklen_t on the call to accept() is possibly provoking undefined behaviour.
To fix this instead of
int c;
define
socklen_t c;
Also2 the code close socket_desc twice and does not close new_socket at all.
Also3 doing
ioctl(new_socket, FIONREAD, &buffersize);
to test whether data is available does make sense, as the socket's buffer is filles asynchronously to the program by the kernel, so the value returned by the call to ioctl() might be outdated within a wink.
Just remove this call to ioctl() and define a fixed-size buffer to read into. recv() will block until data is available.
I'm trying to create a simple proxy server using BSD sockets, which listens on a port for a request and then passes that request on to another server, before sending the server's response back to the browser.
I am able to receive a REST request from the browser, using the code below:
void *buffer = malloc(512);
long length = 0;
while (1) {
void *tempBuffer = malloc(512);
long response = recv(acceptedDescriptor, tempBuffer, 512, 0);
if (response == 0 || response < 512) {
free(tempBuffer);
printf("Read %lu bytes\n", length);
break;
}
memcpy(buffer + length, tempBuffer, response);
free(tempBuffer);
length += response;
realloc(buffer, length + 512);
}
However, recv() should return 0 when the connection is closed by the peer (in this case the browser), but this is never the case. The only way I am able to detect whether or not the connection has closed is by checking if the response is less than the maximum amount requested from recv(), 512 bytes. This is sometimes problematic as some requests I see are incomplete.
If there is no more data to receive, recv() blocks and never returns, and setting the accepted descriptor to be non-blocking means that the read loop goes on forever, never exiting.
If I:
Set the listening socket descriptor to non-blocking, I get a EAGAIN error (resource temporarily unavailable) when I try to accept() the connection
Set the accepted socket descriptor to non-blocking, recv() never returns 0 and the loop continues on forever
Set them both to non-blocking, I get a 'bad file descriptor' error when trying to accept() the connection
Don't set either of them to non-blocking, the loop never exits because recv() never returns.
The socket itself is created as follows, but since it is able to detect a request, I can't see anything wrong with its initialisation:
int globalDescriptor = -1;
struct sockaddr_in localServerAddress;
...
int initSocket() {
globalDescriptor = socket(AF_INET, SOCK_STREAM, 0);
if (globalDescriptor < 0) {
perror("Socket Creation Error");
return 0;
}
localServerAddress.sin_family = AF_INET;
localServerAddress.sin_addr.s_addr = INADDR_ANY;
localServerAddress.sin_port = htons(8374);
memset(localServerAddress.sin_zero, 0, 8);
int res = 0;
setsockopt(globalDescriptor, SOL_SOCKET, SO_REUSEADDR, &res, sizeof(res));
//fcntl(globalDescriptor, F_SETFL, O_NONBLOCK);
return 1;
}
...
void startListening() {
int bindResult = bind(globalDescriptor, (struct sockaddr *)&localServerAddress, sizeof(localServerAddress));
if (bindResult < 0) {
close(globalDescriptor);
globalDescriptor = 0;
perror("Socket Bind Error");
exit(1);
}
listen(globalDescriptor, 1);
struct sockaddr_in clientAddress;
int clientAddressLength = sizeof(clientAddress);
while (1) {
memset(&clientAddress, 0, sizeof(clientAddress));
clientAddressLength = sizeof(clientAddress);
int acceptedDescriptor = accept(globalDescriptor, (struct sockaddr *)&clientAddress, (socklen_t *)&clientAddressLength);
//fcntl(acceptedDescriptor, F_SETFL, O_NONBLOCK);
if (acceptedDescriptor < 0) {
perror("Incoming Connection Error");
exit(1);
}
void *buffer = malloc(512);
long length = 0;
while (1) {
void *tempBuffer = malloc(512);
long response = recv(acceptedDescriptor, tempBuffer, 512, 0);
if (response == 0) {
free(tempBuffer);
printf("Read %lu bytes\n", length);
break;
}
memcpy(buffer + length, tempBuffer, response);
free(tempBuffer);
length += response;
realloc(buffer, length + 512);
}
executeRequest(buffer, length, acceptedDescriptor);
close(acceptedDescriptor);
free(buffer);
}
}
...
The startListening() function is called only if initSocket() returns 1:
int main(int argc, const char *argv[]) {
if (initSocket() == 1) {
startListening();
}
return 0;
}
I'm probably doing something stupid here, but I'd appreciate any information you may have about this problem and how I could fix it.
Since your REST request is a HTTP method, it has the well-defined HTTP Message Length, so you just have to recv() until the complete message has arrived.
Hi I have the following program to check the send buffer size for a UDP socket. However, I the return value is a bit confusing to me. I use the following simple app:
#include <sys/socket.h>
#include <stdio.h>
int main(int argc, char **argv)
{
int sockfd, sendbuff;
socklen_t optlen;
sockfd = socket(AF_INET, SOCK_DGRAM, 0);
if(sockfd == -1)
printf("Error");
int res = 0;
// Get buffer size
optlen = sizeof(sendbuff);
res = getsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &sendbuff, &optlen);
if(res == -1)
printf("Error getsockopt one");
else
printf("send buffer size = %d\n", sendbuff);
// Set buffer size
sendbuff = 98304;
printf("sets the send buffer to %d\n", sendbuff);
res = setsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &sendbuff, sizeof(sendbuff));
if(res == -1)
printf("Error setsockopt");
// Get buffer size
optlen = sizeof(sendbuff);
res = getsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &sendbuff, &optlen);
if(res == -1)
printf("Error getsockopt two");
else
printf("send buffer size = %d\n", sendbuff);
return 0;
}
The output on my machine is:
send buffer size = 129024
sets the send buffer to 98304
new send buffer size = 196608
Can anybody clarify what I'm doing wrong here or how to interpret the output?
You're not doing anything wrong. Linux doubles the value (within the kernel) when you set it, and returns the doubled value when you query it. man 7 socket says:
[...]
SO_SNDBUF
Sets or gets the maximum socket send buffer in bytes. The ker-
nel doubles this value (to allow space for bookkeeping overhead)
when it is set using setsockopt(), and this doubled value is
returned by getsockopt(). The default value is set by the
wmem_default sysctl and the maximum allowed value is set by the
wmem_max sysctl. The minimum (doubled) value for this option is
2048.
[...]
NOTES
Linux assumes that half of the send/receive buffer is used for internal
kernel structures; thus the sysctls are twice what can be observed on
the wire.
[...]