I am trying to understand the functionality of curl_multi_perform so that I can use it in my project. I found a sample code at https://gist.github.com/clemensg/4960504.
Below is the code where my doubt is:
for (i = 0; i < CNT; ++i) {
init(cm, i); // this is setting options in curl easy handles.
}
// I thought this statement will start transfer.
//A-- curl_multi_perform(cm, &still_running);
sleep(5); // put this to check when transfer starts.
do {
int numfds=0;
int res = curl_multi_wait(cm, NULL, 0, MAX_WAIT_MSECS, &numfds);
if(res != CURLM_OK) {
fprintf(stderr, "error: curl_multi_wait() returned %d\n", res);
return EXIT_FAILURE;
}
/*
if(!numfds) {
fprintf(stderr, "error: curl_multi_wait() numfds=%d\n", numfds);
return EXIT_FAILURE;
}
*/
//B-- curl_multi_perform(cm, &still_running);
} while(still_running);
My understanding is that when curl_multi_perform is called the transfer starts but in the above code the curl_multi_perform at label A does not start the transfer. I checked in the wireshark logs. I see the first log output when the control moves past the sleep() statement.
I even tried below code:
for (i = 0; i < CNT; ++i) {
init(cm, i); // this is setting options in curl easy handles.
curl_multi_perform(cm, &still_running);
sleep(5);
}
but, the result was the same. I didn't see any log in the wireshark while the control was in this loop, but, once I started seeing the logs in wireshark they were at 5 seconds interval.
Apart from these doubts, the other doubts I have are:
Why there are two curl_multi_perform at label A & B?
Can I call curl_multi_perform multiple times while adding handles
in between the calls?
Help appreciated.
Thanks
curl_multi_perform works in a non-blocking way. It'll do as much as it can without blocking and then it returns, expecting to get called again when needed. The first call is thus most likely to start resolving the name used in the URL, and then perhaps the second or third call starts the actual transfer or something. It is designed so that an application shouldn't have to care about which exact function call number that does it.
Then you keep calling it until all the transfers are completed.
I've tried to explain this concept in a chapter in the "everything curl" book: Driving with the "multi" interface
Related
LANG: C / ENV: Linux
I am developing a streaming engine, for now I am able to start, stop and pause the stream, but seeking is the operation that's giving me a lot of headache, I already asked a question here before and fixed some issues inside the code from the answers.
Using lseek() function, I am passing the open streaming file descriptor as first argument, plus I am using UDP for transmitting, something like the following code:
transport_fd = open(tsfile, O_RDONLY);
int offset = 1024;
off_t offsetIndicator;
if ((offsetIndicator=lseek(transport_fd, offset, SEEK_CUR))<0) printf("Error seeking\n");
Whenever I try to seek while streaming, the streaming stops and the pictures hangs.
Is there anything I should pay attention to?, i.e: like attempting to sleep() or nanosleep() after seeking into the file in order for the changes to take effect.
I couldn't find examples, papers or realted articles for best practices in such engines.
EDIT:
After testing, it seems like the file continued to stream but receiving devices on the network didn't catch the stream connection anymore, and calculating the time it took to finish after subtract seeking time, the stream seems to be finished normally.
CODE SNIPPET:
while (!completed)
{
while (/* Comparing conditions */ && !completed)
{
if (seekLck == 1) // seekLck is a semaphore to test seek signal from father process initiated by 0
{
int offset = 1024;
off_t offsetIndicator;
if ((offsetIndicator=lseek(transport_fd, offset, SEEK_CUR))<0)
printf("Error seeking\n");
nanosleep(&nano_sleep_packet, 0); //Try to sleep to see if it is still hanging, didn't work
seekLck = 0;
}
len = read(transport_fd, send_buf, packet_size);
if(len < 0) {
fprintf(stderr, "File read error \n");
completed = 1;
}
else if (len == 0)
{
fprintf(stderr, "Sent done\n");
completed = 1;
}
else
{
sent = sendto(sockfdstr, send_buf, len, 0, (struct sockaddr *)&addr, sizeof(struct sockaddr_in));
if(sent <= 0)
{
perror("send(): error ");
completed = 1;
}
}
}
nanosleep(&nano_sleep_packet, 0);
}
close(transport_fd);
close(sockfdstr);
free(send_buf);
printf("cleaning up\n");
return 0;
}
The main question was "Why isn't the file being streamed (Played) even when lseek() is working fine?"...
Actually nothing was wrong from the server side, but even though, the clients weren't able to continue streaming after losing the frame count (streaming ffmpeg frames, clients are getting the stream from a video scrambler).
What worked for me in this situation is getting the socket parameter and killing (in a clean way) the process that needs to be seeked in while holding the stream position, after that start a totally new stream from the seek position with the same socket parameter so it replaces the old one.
I hope this will help someone out there especially that there's no much documentation about those stuff.
Code Connected Volume 1 - page 47 have an example about how to receive multipart message:
while (1) {
zmq_msg_t message;
zmq_msg_init(&message);
zmq_msg_recv(&message, socket, 0);
// Process the message frame
...
zmq_msg_close(&message);
if (!zmq_msg_more(&message))
break;
}
Is this correct? Shouldn't we use zmq_msg_close() after zmq_msg_more()?
The API reference for zmq_msg_more() and zmq_msg_recv() (ZeroMQ 3.2.2 Stable) both contain examples showing that zmq_msg_close() is called after zmq_msg_more. As far as I know the API docs do not specifically state anything to contradict this, thus the example from Code Connected seems wrong. The documentation for zmq_msg_close() states that the actual memory release may be postponed by underlaying layers, implicating that the zmq_msg_more() operation may succeed but it still looks wrong to call it after closing the message.
Example from zmq_msg_more() API documentation (3.2.2) (edited slightly for readability):
zmq_msg_t part;
while (true)
{
// Create an empty ØMQ message to hold the message part
int rc = zmq_msg_init (&part);
assert (rc == 0);
// Block until a message is available to be received from socket
rc = zmq_recvmsg (socket, &part, 0);
assert (rc != -1);
if (zmq_msg_more (&part))
fprintf (stderr, "more\n");
else
{
fprintf (stderr, "end\n");
break;
}
zmq_msg_close (part);
}
However, looking in the ZeroMq Guide regarding Multi-Part Messages, that example actually checks for more messages after closing the message, but that is achieved by checking the socket using zmq_getsockopt(), without using any references to the message. I suspect the Code Connected examples simply used that example and changed from zmq_getsockopt() to zmq_msg_more() (probably incorrecly so).
Example from ZeroMq Guide (multi-part messages):
while (1)
{
zmq_msg_t message;
zmq_msg_init (&message);
zmq_msg_recv (&message, socket, 0);
// Process the message frame
zmq_msg_close (&message);
int more;
size_t more_size = sizeof (more);
zmq_getsockopt (socket, ZMQ_RCVMORE, &more, &more_size);
if (!more)
break; // Last message frame
}
Maybe this answer will help someone.
from "libzmq/doc/zmq_msg_more.txt"
The zmq_msg_more() function indicates whether this is part of a multi-part
message, and there are further parts to receive. This method can safely be
called after zmq_msg_close(). This method is identical to zmq_msg_get()
with an argument of ZMQ_MORE.
I have a problem which I don't really know how to solve. I have a program that multiplexes multiple connections. Those connections are receiving streaming data all at the same time. I had to configure non blocking sockets as the streams have different bitrates. Now what I actually did is keep those sockets in an array looping through them and detecting with a select if there is data to read and proceding to the next element in the array if not.
It works very well except for teh fact that teh CPU is always at 100%. Actually if at some point there is nothing to read from any socket it will still loop. I don't really know how it would be possible to block the loop whenever no data is available on any socket and just keep going when there is data. I think this may be the solution but I don't really see how I could do this. The program has to be very responsive though as it is a UDP stream recorder and if it blocks for too long, this will produce lags in the file.
I thank you a lot.
PS.: Just for info I am still learning so please don't blame me even if the solution may be obvious.
EDIT:
here's some pseudo code:
When a recording request comes in, I create a new connection and connect to the stream address. If it succeeds, I build my fdset using following function:
build_fdset()
{
int ii;
/* */
FD_ZERO(&fdset);
/* */
for (ii = 0; ii < max; ii++)
{
if (astRecorder[ii].bUsed != FALSE && astRecorder[ii].socket != INVALID_SOCKET)
{
FD_SET(astRecorder[ii].socket,&fdset);
/* */
if (astRecorder[ii].socket > maxSocket)
maxSocket = astRecorder[ii].socket;
}
}
}
Then the loop handling the connections:
main_loop()
{
struct timeval timeout;
/* */
timeout.tv_sec = 1;
timeout.tv_usec = 0;
/* */
for (;;)
{
memcpy(&fdset_cpy,&fdset,sizeof(fdset));
int ret = select((maxSocket + 1) , &fdset_cpy, NULL, NULL, &timeout);
if (iSelectRet <= 0)
continue;
else
{
int ii;
for(ii = 0; ii < max; ii++)
{
if ((recorders[ii].bUsed) && (FD_ISSET(recorders[ii].socket, &fdset_cpy)))
{
/* receive from socket */
/* handle received data */
}
}
}
}
}
PROBLEM: When I set timeout to timeout.tv_sec = 1 timeout.tv_usec = 0 everything works fine BUT i get 100% CPU usage! When I give NULL as timeout, the program blocks on the select() although there is data on the sockets.
SOLUTION:
Well I finally found the error! In the above code I set the timeout values only once before the main loop. Well the problem with that is that as for fdset, the timeout structure is modified by the select() function. So after the first correct timed out select, the timeout structure gets modified by the select() function and is set to 0. This results in 0 timeout, thus the problem that the next time the loop gets to the select function, the timeout given to the select is 0!!!
Thanks a lot still to those who tried to help! I apreciate it =)
The timeout of the select call can be NULL which means to wait forever.
You could also use sleep after you check all your streams to give up CPU at the end of your loop. This way you don't depend on a single stream to have incoming data sometime in the near future, at the risk of not servicing your other streams.
I'm having trouble receiving "large" files from a web server using C sockets; namely when these files (or so I suspect) are larger than the size of the buffer I'm using to receive them. If I attempt to ask (through a GET request) for a simple index.html that's not bigger than a few bytes, I get it fine, but anything else fails. I'm assuming that my lack of knowledge on select() or recv() is what's failing me. See here:
fd_set read_fd_set;
FD_ZERO(&read_fd_set);
FD_SET((unsigned int)socketId, &read_fd_set);
/* Initialize the timeout data structure. */
struct timeval timeout;
timeout.tv_sec = 2;
timeout.tv_usec = 0;
// Receives reply from the server
int headerReceived = 0;
do {
select(socketId+1, &read_fd_set, NULL, NULL, &timeout);
if (!(FD_ISSET(socketId, &read_fd_set))) {
break;
}
byteSize = recv(socketId, buffer, sizeof buffer, 0);
if (byteSize == 0 || (byteSize < BUFFER_SIZE && headerReceived)) {
break;
}
headerReceived = 1;
} while(1);
That's right, after sending the GET request to the web server, which I'm pretty sure the server is getting just fine, and GET requests from any other client (like any web browser) are working as intended.
Thanks in advance, any help is greatly appreciated.
if (byteSize == 0 || (byteSize < BUFFER_SIZE && headerReceived))
{
break;
}
headerReceived is set to true after the first read. It is entirely possible and likely subsequent recv()s will be less than BUFFER_SIZE. You are out of the read loop at that point. Recv() is going to return whatever number of bytes there are to read, not necessarily how many you request.
Also either stick with BUFFER_SIZE or sizeof(buffer). Mixing and matching is just asking for a bug somewhere down the road.
One thing that I spot is that you don't reinitialize the selection during the loop. This is probably why you get small files successfully; they are received in one go and the loop doesn't have to be iterated.
I suggest you put the:
FD_ZERO(&read_fd_set);
FD_SET((unsigned int)socketId, &read_fd_set);
timeout.tv_sec = 2;
timeout.tv_usec = 0;
inside the loop (before you invoke select), and it might just work.
You did not say what O/S you are using, but according to the POSIX spec:
Upon successful completion, the select() function may modify the
object pointed to by the timeout argument.
(And I believe Linux, for example, does precisely this.)
So it is very possible that later invocations of your loop have the timeout set to zero, which will cause select to return immediately with no descriptors ready.
I would suggest re-initializing the timeout structure immediately before calling select every time through the loop.
I will start by saying that I am a college student with little c++ experience. How many times have you heard that right? I am working with the test program testISO_TCP (simplified version) from the libnodave library. This program does a simple read of flag values and data blocks while it is connected to a seimens 300 PLC. The program doesn't bring up any errors per se. What I am trying to do is hopefully add some code to this program that will protect the reads from ever crashing. Let me explain a little better. Say for example I have a lot of reads implemented in the code. As of now there are only two reads. Eventually I will run this code with many more reads. Now, say that I am running the test program and for some reason I lose the connection to the PLC. I would like to have the program do one of two things: 1) Once the connection is lost, do a retry connect a certain amount of times and when it runs out of tries, exit. or 2) Somehow continue to read from the PLC until they are all done.
I hope this is enough information to get some help. I will post the code that I have been looking at for so long without any idea how to do this effectively. Thanks to all in advance.
#define PLAY_WITH_KEEPALIVE
#include <stdlib.h>
#include <stdio.h>
#include "nodavesimple.h"
#include "openSocket.h"
#ifdef PLAY_WITH_KEEPALIVE
#include <winsock.h>
#endif
int main(int argc, char **argv) {
int a,b,c,res, doRun, doStop, doRead, doreadFlag, useProtocol, useSlot;
#ifdef PLAY_WITH_KEEPALIVE
int opt;
#endif
float d;
daveInterface * di;
daveConnection * dc;
_daveOSserialType fds;
doRun=0;
doStop=0;
doRead=0;
doreadFlag=0;
useProtocol=daveProtoISOTCP;
useSlot=2;
fds.rfd=openSocket(102, argv[1]);
#ifdef PLAY_WITH_KEEPALIVE
errno=0;
opt=1;
//res=setsockopt(fds.rfd, SOL_SOCKET, SO_KEEPALIVE, &opt, 4);
//LOG3("setsockopt %s %d\n", strerror(errno),res);
#endif
fds.wfd=fds.rfd;
if (fds.rfd>0)
{
di =daveNewInterface(fds,"IF1",0, daveProtoISOTCP, daveSpeed187k);
daveSetTimeout(di,5000000);
dc =daveNewConnection(di,2,0, 2); // insert your rack and slot here
if (0==daveConnectPLC(dc))
{
printf("Connected.\n");
res=daveReadBytes(dc,daveFlags,0,0,16,NULL);
if (0==res)
{
a=daveGetU32(dc);
b=daveGetU32(dc);
c=daveGetU32(dc);
d=daveGetFloat(dc);
printf("FD0: %d\n",a);
printf("FD4: %d\n",b);
printf("FD8: %d\n",c);
printf("FD12: %f\n",d);
}//end 0==res
}//end daveConnectPLC
else
{
printf("Couldn't connect to PLC.\n Please make sure you use the -2 option with a CP243 but not with CPs 343 or 443.\n");
//closeSocket(fds.rfd);
//return -2;
}
}//end fds.rfd
fds.rfd=openSocket(102, argv[1]);
fds.wfd=fds.rfd;
if (fds.rfd>0)
{
di =daveNewInterface(fds,"IF1",0, daveProtoISOTCP, daveSpeed187k);
daveSetTimeout(di,5000000);
dc =daveNewConnection(di,2,0, 2); // insert your rack and slot here
if (0==daveConnectPLC(dc))
{
printf("Connected.\n");
res=daveReadBytes(dc,daveDB,1,0,64,NULL);
if (0==res)
{
a=daveGetU16(dc);
printf("DB1:DW0: %d\n",a);
a=daveGetU16(dc);
printf("DB1:DW1: %d\n...\n",a);
a=daveGetU16At(dc,62);
printf("DB1:DW32: %d\n",a);
}//end 0==res
return 0;
}//end daveConnectPLC
else
{
printf("Couldn't connect to PLC.\n Please make sure you use the -2 option with a CP243 but not with CPs 343 or 443.\n");
closeSocket(fds.rfd);
return -2;
}
}//end fds.rfd
else
{
printf("Couldn't open TCP port. \nPlease make sure a CP is connected and the IP address is ok. \n");
return -1;
}
}// end main
You have to check the return value of the daveReadBytes function.
If it is not zero, something went wrong and you can use the daveStrerror function to get a proper error message:
printf ("error: %s\n", daveStrerror(res));
After that it's up to you to decide to either simply retry the read or disconnect (with closeSocket(...)) and then create a new connection from the beginning. Check the documentation on what errorcodes there are. Some errors can't be resolved by retrying (e.g. because you try reading a data block that doesn't exist).
I have a loop that attempts to connect 3 times and exits gracefully if it fails
You may be able to write some other code to first check to see if the connection is up and also if the PLC is up.
Typically if you try to connect to an IP address that doesn't esond; it will hang there and tie up resources...
I am also new programmer.But want to say that. First we have to differentiate between the TCP/IP connection with the ethernet card ISO_TCP. The openSocket() function does the connection to remote IP adress in the given port/Service (102 ISO_TCP). When called next, the function daveNewInterface(), it will initialise the specific interface for doing a connection to the PLC. After this, the function daveNewConnection() tries to open a connection on a given MPI adress, and very important, the given rack and slot. If this function returns the value 0, it will call the daveConnectPLC() function to connect to the PLC. At this point it´s established a ethernet connection,and also the PLC Connection.
Now you can use all the function from the libnodave library for read or write data, stop or run the PLC and much more.
In the actually simplified TCP_ISO code there are no function to disconnect the adapter or close the connection with the PLC, in your code there is the closeSocket() function and also the function that returns -2. Find at what line the code breaks, introducing for example a log after every function to see the returns values.
All info for detecting loss of communication is in the documentation.