I've been writing code for a PUB-SUB ZeroMQ system (in C), with one publisher and multiple subscribers.
On the client code, I must subscribe to a topic, according to a filter, like so:
(...)
void *context = zmq_ctx_new ();
void *subscriber = zmq_socket (context, ZMQ_SUB);
int rc = zmq_connect (subscriber, "tcp://localhost:5555");
char *filter = (argc > 1)? argv [1]: "10001 "; // example of a filter
rc = zmq_setsockopt (subscriber, ZMQ_SUBSCRIBE, filter, strlen (filter));
(...)
However, looking at code examples of the publisher (seen in the documentation of ZeroMQ), I don't see any line where the topic is created:
(...)
void *context = zmq_ctx_new ();
void *publisher = zmq_socket (context, ZMQ_PUB);
int rc = zmq_bind (publisher, "tcp://*:5556");
while (1) {
(...)
char update [20];
sprintf (update, "%05d %d %d", zipcode, temperature, relhumidity);
s_send (publisher, update);
(...)
The message is simply sent and no topic is specified. How is this possible?
In python versions, I've seen, in the publisher code:
socket.send_string("%d %d" % (topic, number))
but I have not seen anything like this in C.
Publishers don't explicity set topics in ZMQ. Topics are implied by the message, starting from the first character. The python code example is just a simple string concatenation that adds the "topic" to the front of the message.
Say you publish a message:
Hello - I am a ZMQ message
subscribing to
H
Hello
Hello -
Will all receive your message.
Welcome to ZeroMQ Zen-of-Zero.
All bytes of the message are considered as The Topic and the subscriber-side specified "Topic-filtering" operates on trivial and plain byte-matching of the set of subscribed-to topics against the left-n-bytes of the message.
Those messages, that match any of the recorded Topic(s) are considered as messages to be delivered, those not matching any one are not ( except when an inverse matching option was set )
Related
Hello GStreamer community & fans,
I have a working pipeline that connects to multiple H.264 IP camera streams using multiple rtspsrc elements aggregated into a single pipeline for downstream video processing.
Intermittently & randomly, streams coming in from remote & slower connections will have problems, timeout, retry and go dead, leaving that stream with a black image when viewing the streams post processing. The other working streams continue to process normally. The rtspsrc elements are setup to retry the rtsp connection, and that seems to somewhat work, but for when it doesn't, I'm looking for a way to disconnect the stream entirely from the rtspsrc element and restart that particular stream without disrupting the other streams.
I haven't found any obvious examples or ways to accomplish this, so I've been tinkering with the rtspsrc element code itself using this public function to access the rtspsrc internals that handle connecting.
__attribute__ ((visibility ("default"))) GstRTSPResult my_gst_rtspsrc_conninfo_reconnect(GstRTSPSrc *, gboolean);
GstRTSPResult
my_gst_rtspsrc_conninfo_reconnect(GstRTSPSrc *src, gboolean async)
{
int retries = 0, port = 0;
char portrange_buff[32];
// gboolean manual_http;
GST_ELEMENT_WARNING(src, RESOURCE, READ, (NULL),
(">>>>>>>>>> Streamer: A camera closed the streaming connection. Trying to reconnect"));
gst_rtspsrc_set_state (src, GST_STATE_PAUSED);
gst_rtspsrc_set_state (src, GST_STATE_READY);
gst_rtspsrc_flush(src, TRUE, FALSE);
// manual_http = src->conninfo.connection->manual_http;
// src->conninfo.connection->manual_http = TRUE;
gst_rtsp_connection_set_http_mode(src->conninfo.connection, TRUE);
if (gst_rtsp_conninfo_close(src, &src->conninfo, TRUE) == GST_RTSP_OK)
{
memset(portrange_buff, 0, sizeof(portrange_buff));
g_object_get(G_OBJECT(src), "port-range", portrange_buff, NULL);
for (retries = 0; portrange_buff[retries] && isdigit(portrange_buff[retries]); retries++)
port = (port * 10) + ((int)(portrange_buff[retries]) + 48);
if (port != src->client_port_range.min)
GST_ELEMENT_WARNING(src, RESOURCE, READ, (NULL), (">>>>>>>>>> Streamer: port range start mismatch"));
GST_WARNING_OBJECT(src, ">>>>>>>>>> Streamer: old port.min: %d, old port.max: %d, old port-range: %s\n", (src->client_port_range.min), (src->client_port_range.max), (portrange_buff));
src->client_port_range.min += 6;
src->client_port_range.max += 6;
src->next_port_num = src->client_port_range.min;
memset(portrange_buff, 0, sizeof(portrange_buff));
sprintf(portrange_buff, "%d-%d", src->client_port_range.min, src->client_port_range.max);
g_object_set(G_OBJECT(src), "port-range", portrange_buff, NULL);
for (retries = 0; retries < 5 && gst_rtsp_conninfo_connect(src, &src->conninfo, async) != GST_RTSP_OK; retries++)
sleep(10);
}
if (retries < 5)
{
gst_rtspsrc_set_state(src, GST_STATE_PAUSED);
gst_rtspsrc_set_state(src, GST_STATE_PLAYING);
return GST_RTSP_OK;
}
else return GST_RTSP_ERROR;
}
I realize this is probably not best practice and I'm doing this to find a better way once I understand the internals better through this learning experience.
I appreciate any feedback anyone has to this problem.
-Doug
On the Eclipse Paho MQTT website, the developers provide a client example (http://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/pubsync.html) that does the following:
Create a client object with the specified parameters
Connect the client with the specified connection options
Publish a MQTT message
Disconnect the client
Destroy the client object
This works well if all you want is to publish one single message.
In my code, I have a function that contains pretty much the same code as in the aforementioned example, however, the function is called repeatedly from main() as I need to publish a large number of messages one after another. The problem is, if I use the code exactly as in the example, every time my function is called a new connection is created and shortly after destroyed. This happens again and again and again as long as the function is called repeatedly, causing a huge overhead.
Is there a way to check whether a client object has already been created, and if so, don't do it again but use the existing one?
In my understanding, the MQTTClient_isConnected() function is supposed to do that: https://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/_m_q_t_t_client_8h.html#ad9e40bdb7149ee3e5d075db7f51a735f
But if I try it like this, I get a Segmentation fault:
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, mqtt.addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
exit(EXIT_FAILURE);
}
}
[EDIT]
Here is a simple demo code that better illustrates what I'm trying to accomplish:
#include <stdio.h>
#include <MQTTClient.h>
MQTTClient client;
void publish_MQTT() {
MQTTClient_connectOptions conn_opts = MQTTClient_connectOptions_initializer;
MQTTClient_message pubmsg = MQTTClient_message_initializer;
MQTTClient_deliveryToken token;
char *payload = (char *)calloc(1024, sizeof(char));
strcpy(payload, "hello");
printf("DEBUG_BEFORE >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
fprintf(stderr, RED "\n==> Connection to MQTT Broker failed.\n" RESET_CL);
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
printf("DEBUG_AFTER >> MQTTClient_isConnected(client) = %d\n", MQTTClient_isConnected(client)); // DEBUG OUTPUT
pubmsg.payload = payload;
pubmsg.payloadlen = strlen(payload);
pubmsg.qos = QOS;
pubmsg.retained = 0;
MQTTClient_publishMessage(client, TOPIC, &pubmsg, &token);
MQTTClient_waitForCompletion(client, token, TIMEOUT);
//MQTTClient_disconnect(client, 10000);
//MQTTClient_destroy(&client);
free(payload);
}
int main(void) {
for (i=0; i<1000; i++) {
publish_MQTT();
}
return 0;
}
Please ignore the fact that the addr parameter is never specified (in my real code it is) or that it is pretty useless specifying a message in the publish_MQTT() function (in my real code, data is passed from main() to that function).
I figured it out: Apparently, there is absolutely nothing wrong with the example codes in the original posting.
It turns out I was appending the port of the MQTT server to the addr parameter again and again (in a section of the code not shown here as I didn't suspect the source of the error to be there), every time the publish_MQTT() function was called. This made the addr char string grow and eventually exceed the specified length, thus causing the SegFault.
This way everything works just as intended:
printf("\nADDR = %s\n\n", addr); // DEBUG OUTPUT
if (!MQTTClient_isConnected(client)) {
strcat(strcat(addr, ":"), pt); // This line needed to be placed here, not before that if block
MQTTClient_create(&client, addr, CLIENT_ID, MQTTCLIENT_PERSISTENCE_NONE, NULL);
conn_opts.keepAliveInterval = 20;
conn_opts.cleansession = 1;
conn_opts.username = TOKEN;
if (MQTTClient_connect(client, &conn_opts) != MQTTCLIENT_SUCCESS) {
printf("\n==> Connection to MQTT Broker failed.\n");
MQTTClient_destroy(&client);
free(payload);
exit(EXIT_FAILURE);
}
}
Probably you are setting up "clean session flag", what's mean: "
If the ClientId represents a Client already connected to the Server then the Server MUST disconnect the existing Client [MQTT-3.1.4-2]." (from mqtt standard). So you client is disconnected (the existing one).
Code from example seems to be reasobable. It looks like there is problem with passing function argument. For example if function needed address, and you are giving objects itself.
Morze from standard:
"3.2.2.2 Session Present
Position: bit 0 of the Connect Acknowledge Flags.
If the Server accepts a connection with CleanSession set to 1, the Server MUST set Session Present to 0 in the CONNACK packet in addition to setting a zero return code in the CONNACK packet [MQTT-3.2.2-1].
If the Server accepts a connection with CleanSession set to 0, the value set in Session Present depends on whether the Server already has stored Session state for the supplied client ID. If the Server has stored Session state, it MUST set Session Present to 1 in the CONNACK packet [MQTT-3.2.2-2]. If the Server does not have stored Session state, it MUST set Session Present to 0 in the CONNACK packet. This is in addition to setting a zero return code in the CONNACK packet".
Is it possible and if so how to archive it that one application using libmosquitto get messages from one broker and publish it to another?
Its pretty simple to just change topic in mosquito_publish function, but set of broker takes place in
mosquitto_connect(mosq, "localhost",1883 , 60);
Running mosquitto_connet second time
e.g.
mosquitto_connect(mosq, "mqtt.example.io",1883 , 60);
ends up connecting to last one.
I tried to create two mosquitto structs but I dont know how to inform second one about message form subbed channel in order to get info from it, change it and push to proper broker.
Something like this should do the job (I've not tested it though). You'll have to add error checking.
#include <stdio.h>
#include <mosquitto.h>
void on_connect1(struct mosquitto *mosq, void *obj, int result)
{
int rc = MOSQ_ERR_SUCCESS;
if(!result){
mosquitto_subscribe(mosq, NULL, "/v1/topic1", 0);
}else{
fprintf(stderr, "%s\n", mosquitto_connack_string(result));
}
}
void on_message1(struct mosquitto *mosq, void *obj, const struct mosquitto_message *message)
{
struct mosquitto *mosq2 = (struct mosquitto *)obj;
mosquitto_publish(mosq, NULL, "/v1/topic2", message->payloadlen, message->payload, message->qos, message->retain);
}
int main(int argc, char *argv[])
{
struct mosquitto *mosq1, *mosq2;
mosquitto_lib_init();
mosq2 = mosquitto_new(NULL, true, NULL);
mosq1 = mosquitto_new(NULL, true, mosq2);
mosquitto_connect_callback_set(mosq1, on_connect1);
mosquitto_message_callback_set(mosq1, on_message1);
mosquitto_connect(mosq2, "mqtt.example.io", 1883, 60);
mosquitto_connect(mosq1, "localhost", 1883, 60);
mosquitto_loop_start(mosq2);
mosquitto_loop_forever(mosq1, -1, 1);
mosquitto_destroy(mosq1);
mosquitto_destroy(mosq2);
mosquitto_lib_cleanup();
return 0;
}
Yes,
You need two totally separate instances of client. Which means two separate instances the mosquitto structure returned by the mosquitto_new() function. One for each broker.
At the moment you are re-using the same structure so it is only holding the details of the last call to mosquitto_connect()
This command line using mosquitto can do the job:
mosquitto_sub -h localhost -t '#' | { while [ 1 -lt 2 ] ; do read message ; if [[ "$message" != "$prev_message" ]]; then mosquitto_pub -h localhost -t "tenewtest" -m "$message" ; prev_message=$message ; fi ; done ; }
If you simply want to forward the messages, then there is a 'bridge' functionality in mosquitto broker. It makes it easier through configuration. It has a feature where you can specify the topics you want to forward & also authentication options. It is quite feature rich.
I'am trying to build a open source Cometd Library for C developers, to be able to run a client under barely any given device.
I manage to complete the handshake and data transfer with the longPolling technique, the next step is logically to give the option to the user to choose a Websocket connection.
To ease up the work I decided to use a library for that, I again fetched the internet and found two really relevant libraries:
The WSlay library, which seems to work well but doesn't run under OSX due to the epoll.h include which is not supported by apple.
The NoPoll library, which I think is the best one to use.
The problem is that I can't make it perform right, I always get a failed connections. After reading the full documentation I still don't understand why?
Here is my code:
int main(void)
{
cometd *cometd = cometd_new();
JsonNode *connect = json_node_new(JSON_NODE_OBJECT);
noPollCtx *ctx = nopoll_ctx_new(); //init the context of the lib
noPollMsg *msg; //variable to get messages
if (!ctx) //check if the context is well inited
puts("error ctx is nill");
/*After the new you can enable or disable the Websocket simply do this, enabled by defautl*/
cometd->config->webSockState = true;
cometd_configure(cometd, COMETDOPT_URL, "http://m.zpush.ovh:8080/str/strd");
cometd_configure(cometd, COMETDOPT_MAX_BACKOFF, 5000);
struct _cometd_ext* logger = cometd_ext_logger_new();
cometd_ext_add(&cometd->exts, logger);
cometd_connect(cometd);
connect = cometd_msg_connect_new(cometd);
cometd_transport_send(cometd, connect);
//everything until here is for the handshake wich proceed well and the server returns a successful header
//here i check if the user wants to Websocket upgrade his connection
if (cometd->config->webSockState == true)
{
// here i target the server
noPollConn *conn = nopoll_conn_new(ctx, "54.171.156.38" , "8080" , "m.zpush.ovh:8080" , "ws://m.zpush.ovh:8080/str/strd", NULL, "null");
if (!nopoll_conn_wait_until_connection_ready(conn, 5) )
// here i check if the connection is ready
{puts("nopoll_conn failed"); return (0);}
if (nopoll_conn_send_text (conn, "hello how are you doing, do we connect ?", 40) != 40)
{puts("send text just failed."); return(0);}
else
{
while (nopoll_true) //the loop to receive and send messages
{
msg = nopoll_conn_get_msg(conn);
if (msg)
break;
if (! nopoll_conn_is_ok (conn))
{
puts("connection failed during msg wait");
return nopoll_false;
}
}
}
}
cometd_subscribe(cometd, "/service/GmY-HuzW/6sd0/ls", handler);
cometd_subscribe(cometd, "service/GmY-HuzW/6sd0/updateMeta", handler);
cometd_subscribe(cometd, "/service/GmY-HuzW/6sd0/ls", handler);
cometd_subscribe(cometd, "/service/GmY-HuzW/6sd0/newFile", handler);
cometd_transport_send(cometd, cometd_ping_ls(cometd, "/service/GmY-HuzW/6sd0/ls"));
cometd_listen(cometd);
nopoll_ctx_unref(ctx);
return 0;
}
NoPoll implements a debug option, here is the result :
(proc 49413): (debug) nopoll_ctx.c:260 (nopoll_bool nopoll_ctx_register_conn(noPollCtx *, noPollConn *)) registered connection id 2, role: 0
(proc 49413): (debug) nopoll_conn.c:505 (noPollConn *__nopoll_conn_new_common(noPollCtx *, nopoll_bool, const char *, const char *, const char *, const char *, const char *, const char *)) Created noPoll conn-id=2 (ptr: 0x7f9d00501b60, context: 0x7f9d0041cdf0, socket: 3)
(proc 49413): (debug) nopoll_conn.c:284 (char *__nopoll_conn_get_client_init(noPollConn *)) Created Sec-WebSocket-Key nonce: Z0WLawAAAADGI3syAAAAAA==
(proc 49413): (debug) nopoll_conn.c:551 (noPollConn *__nopoll_conn_new_common(noPollCtx *, nopoll_bool, const char *, const char *, const char *, const char *, const char *, const char *)) Sending websocket client init: GET / HTTP/1.1
Host: 10.0.0.103
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: Z0WLawAAAADGI3syAAAAAA==
Origin: http://10.0.0.103
Sec-WebSocket-Version: 13
(proc 49413): (critical) nopoll_conn.c:643 (noPollConn *__nopoll_conn_new_common(noPollCtx *, nopoll_bool, const char *, const char *, const char *, const char *, const char *, const char *)) ***Failed to send websocket init message, error code was: 57 (2), closing session***
(proc 49413): (debug) nopoll_conn.c:1036 (void nopoll_conn_shutdown(noPollConn *)) shutting down connection id=2 (session: 3, role: client)
(proc 49413): (debug) nopoll_conn.c:651 (noPollConn *__nopoll_conn_new_common(noPollCtx *, nopoll_bool, const char *, const char *, const char *, const char *, const char *, const char *)) Web socket initial client handshake sent
-> for better reading : Failed to send websocket init message, error code was: 57 (2), closing session
I really hope some one already successfully used the library and could give me some hints or guide lines. Any help would be very much appreciated, and will give me the ability to finish this library.
The code is still a little bit messy but if you want a full sight of my code I can upload to GitHub.
--> Update :
I found a nasty trick to overcome this problem, i commented line N•845 nopoll_conn_shutdown(); in nopoll_conn.c method wich i think you have guessed it, belongs to the library. Then make install and no more error 57.
Working on your example, I found out that the remote server is closing the connection by reporting the following error:
INFO: conn=0x1238500, conn-id=2, nopoll_conn_is_ok (0x1238500)=1, nopoll_conn_is_ready (0x1238500)=1...sending content
ERROR: connection failed during msg wait, peer reported status=[1011] and reason=[java.text.ParseException: hello how are you doing, do we connect ?]
That is, you are sending unexpected application format for this WebSocket, which is causing the connection close. However, noPoll's support for close frames with body (which includes a status code and a reason) wasn't supported.
I've updated the library and now it is fully supported. You'll have to get a copy of the library from here (use SVN rev 227 at least).
https://dolphin.aspl.es/svn/publico/nopoll/trunk/
http://www.aspl.es/nopoll/downloads.html
Along with that, here is a working updated example that goes to the point I'm talking about plus the couple of functions that will allow you get status and reason reported by remote peer:
#include <nopoll.h>
int main (void) {
/* init the context of the lib */
noPollCtx * ctx = nopoll_ctx_new();
/* variable to get messages */
noPollMsg * msg;
noPollConn * conn;
/* nopoll_log_enable (ctx, nopoll_true);
nopoll_log_color_enable (ctx, nopoll_true); */
if (!ctx) {
puts("error ctx is nill");
return -1; /* do not continue */
}
/* here i target the server */
conn = nopoll_conn_new (ctx, "54.171.156.38" ,
"8080" ,
"m.zpush.ovh:8080" ,
"ws://m.zpush.ovh:8080/str/strd",
NULL, "null");
if (!nopoll_conn_wait_until_connection_ready (conn, 5) ) {
/* here i check if the connection is ready */
puts ("nopoll_conn failed");
return -1;
} /* end if */
printf ("INFO: conn=%p, conn-id=%d, nopoll_conn_is_ok (%p)=%d, nopoll_conn_is_ready (%p)=%d...sending content\n",
conn, nopoll_conn_get_id (conn), conn,
nopoll_conn_is_ok (conn), conn,
nopoll_conn_is_ready (conn));
if (nopoll_conn_send_text (conn, "hello how are you doing, do we connect ?", 40) != 40) {
puts("send text just failed.");
return - 1;
} /* end if */
while (nopoll_true) {
/* the loop to receive and send messages */
msg = nopoll_conn_get_msg(conn);
if (msg)
break;
if (! nopoll_conn_is_ok (conn)) {
printf ("ERROR: connection failed during msg wait, peer reported status=[%d] and reason=[%s]\n",
nopoll_conn_get_close_status (conn),
nopoll_conn_get_close_reason (conn));
return nopoll_false;
}
}
/* close connection always */
nopoll_conn_close (conn);
/* release context */
nopoll_ctx_unref (ctx);
return 0;
}
Best Regards,
I'm trying to develop simple RESTful api using FastCGI (and restcgi). When I tried to implement POST method I noticed that the input stream (representing request body) is wrong. I did a little test and looks like when I try to read the stream only every other character is received.
Body sent: name=john&surname=smith
Received: aejh&unm=mt
I've tried more clients just to make sure it's not the client messing with the data.
My code is:
int main(int argc, char* argv[]) {
// FastCGI initialization.
FCGX_Init();
FCGX_Request request;
FCGX_InitRequest(&request, 0, 0);
while (FCGX_Accept_r(&request) >= 0) {
// FastCGI request setup.
fcgi_streambuf fisbuf(request.in);
std::istream is(&fisbuf);
fcgi_streambuf fosbuf(request.out);
std::ostream os(&fosbuf);
std::string str;
is >> str;
std::cerr << str; // this way I can see it in apache error log
// restcgi code here
}
return 0;
}
I'm using fast_cgi module with apache (not sure if that makes any difference).
Any idea what am I doing wrong?
The problem is in fcgio.cpp
The fcgi_steambuf class is defined using char_type, but the int underflow() method downcasts its return value to (unsigned char), it should cast to (char_type).
I encountered this problem as well, on an unmodified Debian install.
I found that the problem went away if I supplied a buffer to the fcgi_streambuf constructor:
const size_t LEN = ... // whatever, it doesn't have to be big.
vector<char> v (LEN);
fcgi_streambuf buf (request.in, &v[0], v.size());
iostream in (&buf);
string s;
getline(in, s); // s now holds the correct data.
After finding no answer anywhere (not even FastCGI mailing list) I dumped the original fastcgi libraries and tried using fastcgi++ libraries instead. The problem disappeared. There are also other benefits - c++, more features, easier to use.
Use is.read() not is >> ...
Sample from restcgi documentation:
clen = strtol(clenstr, &clenstr, 10);
if (*clenstr)
{
cerr << "can't parse \"CONTENT_LENGTH="
<< FCGX_GetParam("CONTENT_LENGTH", request->envp)
<< "\"\n";
clen = STDIN_MAX;
}
// *always* put a cap on the amount of data that will be read
if (clen > STDIN_MAX) clen = STDIN_MAX;
*content = new char[clen];
is.read(*content, clen);
clen = is.gcount();