Can somebody please provide an example for sending (and if possible also receiving) messages using basic ZMQ socket from some external python script. I understand that we can use Agent objects and their vip.pubsub.publish and vip.pubsub.subscribe methods to do this, but I feel like having a whole agents objects, and letting them run in background adds unnecessary complexity to scripts that just need a simple way to access the VIP message bus.
Thank you.
While this is possible, it would require re-implementing the vip protocol. The best way to do this is to use the internal web service that is available through the platform itself.
Your agent will register with the master web service either a python callback or a static directory and a regular expression for routing.
Related
I want to implement a Unix daemon (let's call it myUnixd), and want the user to be able to interact with this daemon via the command line, for example:
myUnixd --help # will display help information
myUnixd --show # will show some data (the's deamon should be doing the work)
So my question is: How can I communicate with the daemon? I was thinking about Unix domain sockets. Can someone tell me the right way to do this?
Thanks.
Use Berkeley sockets. Specifically, you can create a "UNIX domain socket" (otherwise known as a "local domain socket," which will create what looks like a text file. Write to the text file to send text to the daemon, read from it to receive text from the daemon. You can implement this with a few function calls.
If you want something more advanced, you can also use DBus, which offers a more sophisticated interface, but which is more complicated to learn.
use tcp socket if you want to use telnet to communicate with your daemon.
One could also use Remote Procedure Call (RPC) for such client-server communication. There are different types of messages (protocols) that can be used together with it, one of them is JSON.
The JSON-RPC protocol is very well accepted for such tasks. You can find different tools and libraries to embed in your software. A quick search on google gives this C library. The advantage of such libraries is that from a JSON specification file, where you define all your remote function calls, it creates client and/or server stubs that you can just use in your code out of the box.
As a listener one can use sockets, as the other responses state, or just an embedded HTTP server like microhttpd (and libcurl for the client). There are plenty of examples out there to just reuse. HTTP allows you also to run your client behind a proxy.
So, i am assuming that i will need to use sockets(i am a newbie to C).
The program will be for Windows(in pure C). And i shall be using these examples
http://cs.baylor.edu/~donahoo/practical/CSockets/winsock.html
My question is, instead of the client program connecting via TCP, i want the server to accept connections via a web browser i.e via HTTP.
So if the server program is running you type http://yourip:port/?gettemps and the server responds, but how do i do it?
As you might have guessed, this program will be for monitoring temps, remotely, via a web browser. But not for the CPU, for the GPU using AMD's ADL library(so yeah, only AMD cards).
The simplest option that is supported by most web servers is CGI - Common Gateway Interface.
Microsoft, of cource, has their own way of running web apps - ISAPI.
HTTP is quite a big standard, you might want to use some library such as libcurl to handle the details for you.
If you decide to code it yourself, HTTP is running over TCP so you first need to open a TCP socket at the standard HTTP port 80. Then simply listen on the socket and parse the incoming HTTP data - a great summary is given here: http://www.jmarshall.com/easy/http/.
Web browsers sends http get request to the server via tcp. If you are writing a web server from scratch than, you will need to parse data from web browser. http get request are string like for example GET /images/logo.png HTTP/1.1. So tokenize that string as it comes through tcp and get the command.
As you received your commands to the server call appropriate functions to handle your request.
Here is an great example of simple http server. You might want to make server multi-threaded as you may have multiple simultaneous users.
If you have already set up your web server to run the app on the appropriate port you can use getenv("QUERY_STRING") to access the web equivalent of command line parameters.
It would be better to call your program directly rather than just using the server to access a single default program as your example does, thus you could use http://yourip:port/yourprogram?cmd=gettemps. In this example getenv("QUERY_STRING") would return 'cmd=gettemps'.
Having looked at several available http server libraries I have not yet found what I am looking for and am sure I can't be the first to have this set of requirements.
I need a library which presents an API which is 'pipelined'. Pipelining is used to describe an HTTP feature where multiple HTTP requests can be sent across a TCP link at a time without waiting for a response. I want a similar feature on the library API where my application can receive all of those request without having to send a response (I will respond but want the ability to process multiple requests at a time to reduce the impact of internal latency).
So the web server library will need to support the following flow
1) HTTP Client transmits http request 1
2) HTTP Client transmits http request 2 ...
3) Web Server Library receives request 1 and passes it to My Web Server App
4) My Web Server App receives request 1 and dispatches it to My System
5) Web Server receives request 2 and passes it to My Web Server App
6) My Web Server App receives request 2 and dispatches it to My System
7) My Web Server App receives response to request 1 from My System and passes it to Web Server
8) Web Server transmits HTTP response 1 to HTTP Client
9) My Web Server App receives response to request 2 from My System and
passes it to Web Server
10) Web Server transmits HTTP response 2 to HTTP Client
Hopefully this illustrates my requirement. There are
two key points to recognise. Responses to the Web Server Library are
asynchronous and there may be several HTTP requests passed to My Web
Server App with responses outstanding.
Additional requirements are
Embeddable into an existing 'C' application
Small footprint; I don't need all the functionality available in Apache etc.
Efficient; will need to support thousands of requests a second
Allows asynchronous responses to requests; their is a small latency to responses and given the required request throughput a synchronous architecture is not going to work for me.
Support persistent TCP connections
Support use with Server-Push Comet connections
Open Source / GPL
support for HTTPS
Portable across linux, windows; preferably more.
I will be very grateful for any recommendation
Best Regards
You could try libmicrohttp.
Use the Onion, Luke. This is lightweight and easy to use HTTP server library in C.
For future reference, that meets your requirement, take a look at libasyncd
I'm one of contributors.
Embeddable into an existing 'C' application
It's written in C.
Small footprint; I don't need all the functionality available in Apache etc.
Very compact.
Efficient; will need to support thousands of requests a second
It's libevent based framework. Can handle more than that.
Allows asynchronous responses to requests;
It's asynchronous. Also support pipelining.
Support persistent TCP connections
Sure, keep-alive.
Support use with Server-Push Comet connections
It's up to how you code your logic.
Open Source / GPL
under BSD license
support for HTTPS
Yes. it supports https with openssl.
Portable across linux, windows; preferably more.
Portable but not windows at this moment but it's portable to windows.
What you want is something that supports HTTP pipelining. You should make yourself familiar with that page if you are not already.
Yes, go for libmicrohttp. It has support for SSL etc and work in both Unix and Windows.
However, Christopher is right on the spot in his comment. If you have a startup time for each response, you are not going to gain much by pipelining. However, if you only have a significant response time to the first request, you may win something.
On the other hand, if each response has a startup time, you may gain a lot by not using pipelining, but create a new request for each object. Then each request can have its own thread, sucking up the startup costs in parallel. All responses will then be sent "at once" in the optimum case. libmicrohttp supports this mode of operation in its MHD_USE_THREAD_PER_CONNECTION thread model.
Following up on previous comments and updates...
You don't say how many concurrent connections you'll have but just "a TCP link".
If it's a single connection, then you'll be using HTTP pipelining as previously mentioned; so you would only need a handful of threads — rather than thousands — to process the requests at the head of the pipeline.
So you wouldn't need to have a thread for every request; just a small pool of workers for each connection.
Have you done any testing or implementation so far to show whether you actually do have problems with response latency for pipelined connections?
If your embedded device is powerful enough to cope with thousands of requests per second, including doing TLS setup, encryption and decryption, I would worry about premature optimisation at this level.
Howard,
Have you taken a look at lighthttpd? It meets all of your requirements except it isn't explicitly an embedded webserver. But it is open source and compiling it in to your application shouldn't be too hard. You can then write a custom plugin to handle your requests.
Can't believe no one has mentioned nginx. I've read large portions of the source-code and it is extremely modular. You could probably get the parts you need working pretty quickly.
uIP or lwip could work for you. I personally use uIP. It's good for a small number of clients and concurrent connections (or as you call it, "pipelining"). However, it's not as scalable or as fast at serving up content as lwip from what I've read. I went with simplicity and small size of uIP instead of the power of lwip, as my app usually only has 1 user.
I've found uIP pretty limited as the concurrent connections increase. However, I'm sure that's a limitation of my MAC receive buffers available and not uIP itself. I think lwip uses significantly more memory in some way to get around this. I just don't have enough ethernet RAM to support a ton of request packets coming in. That said, I can do background ajax polling with about a 15ms latency on a 56mhz processor.
http://www.sics.se/~adam/software.html
I've actually modified uIP in several ways. Adding a DHCP server and supporting multipart POST for file uploads are the big things.) Let me know if you have any questions.
I need to decode a packet sent using TIBCO-RV and pull fields out of the header and skip over the message body. I have not been able to any examples or documentation. Does anybody know of any open source applications that might do this or if there is a Wireshark dissector out there somewhere?
Maybe you should try applying for a license and getting the official documentation. According to Wikipedia:
TIBCO provides messaging APIs in C,
C++, Java, Visual BASIC , Perl and
.NET to receive data feeds on MS Excel
spreadsheets and other applications of
choice.
Failing that, you could perhaps dive into the TIBCO:RV Perl module.
The methods which TibcoRV implements reliable mutli-cast are propriety, but one would assume easy to reverse engineer. I don't believe any of the official documentation goes into detail on the packet level detail. It's quite easy to get the data out if you have the API.
Several things come to mind:
Is the client on your machine running? This is required in order to create the multicast subscription (unless you are using broadcast mode). Otherwise, you need to have some client subscribe to the multicast channel, or your switch shouldn't forward the traffic.
Generally, you will have a single rrd running locally. You have TCP traffic between the RRD and your app. You can use an app like socketsniff to view the traffic between the two.
The user, administrators and support staff need detailed runtime and monitoring information from a daemon developed in C.
In my case these information are e.g.
the current system health, like throughput (MB/s), already written data, ...
the current configuration
I would use JMX in the Java world and the procfs (or sysfs) interface for a kernel module. A log file doesn't seem to be the best way.
What is the best way for such a information interface for a C daemon?
I thought about opening a socket and implementing a bare-metal http or xmlrpc server, but that seems to be overkill. What are alternatives?
You can use a signal handler in your daemon that reacts to, say USR1, and dumps information to the screen/log/net. This way, you can just send the process a USR1 signal whenever you need the info.
You could listen on a UNIX-domain socket, and write regularly write the current status (say once a second) to anyone who connects to it. You don't need to implement a protocol like HTTP or XMLRPC - since the communication will be one-way just regularly write a single line of plain text containing the state.
If you are using a relational database anyway, create another table and fill it with the current status as frequent as necessary. If you don't have a relational database, write the status in a file, and implement some rotation scheme to avoid overwriting a file that somebody reads at that very moment.
Write to a file. Use a file locking protocol to force atomic reads and writes. Anything you agree on will work. There's probably a UUCP locking library floating around that you can use. In a previous life I found one for Linux. I've also implemented it from scratch. It's fairly trivial to do that too.
Check out the lockdev(3) library on Linux. It's for devices, but it may work for plain files too.
I like the socket idea best. There's no need to support HTTP or any RPC protocol. You can create a simple application specific protocol that returns requested information. If the server always returns the same info, then handling incoming requests is trivial, though the trivial approach may cause problems down the line if you ever want to expand on the possible queries. The main reason to use a pre-existing protocol is to leverage existing libraries and tools.
Speaking of leveraging, another option is to use SNMP and access the daemon as a managed component. If you need to query/manage the daemon remotely, this option has its advantages, but otherwise can turn out to be greater overkill than an HTTP server.