Suppose an embedded system which is running different set of softwares which should be communicated together asynchronously, i implemented the data communication mechanism using shared memory ( its fast and simple to use ).
In case of IPC in multiple softwares with different technologies, i was getting into the problem how to notify softwares like a bus mechanism. while there are good IPC/notify mechanism such as unix signal/eventfd/shared semaphores/unix sockets and ..., as i know all of them can be used as point-to-point notification system and i can't find any native solution for bus like notification system.
In BUS notification system, one can notify multiple slaves in single bus notification, rather than creating multiple notification objects for each slave and call notify for all of them.
I know there are already working systems such as D-BUS, but D-BUS is considered too complex for small embedded system and i am looking for native solutions.
Is there any simple, lightweight and native event notification system like D-BUS in linux/unix ?
I found that inotify could be used in such case, but i think is there any
other method exists which was designed for notification propose only?
EDIT:
I think multicast IPC (which was answered here is sightly different from publish/subscribe or BUS.
To be clear, i found that inotify could be used in this situation, suppose on file and multiple file watchers which resembles publish/subscribe IPC pattern, i want to know is there any other solution to this problem?!
Related
I'm doing an embedded-system project now. From my view, AT commands can be sent to a device to retrieve 4G information, dial and so on. On the other hand, we can also do that by calling APIs provided by the 4G vendor.
My question is what's the relationship between them? Is the API a wrapper for the AT commands?
TL;DR
Vendor's API (not only C, but also C++, Java or Python depending to the vendor and the modem model) can both be wrappers for AT commands and a wider, more powerful set of API were the user can port complex applications.
It depends on the vendor and on the model.
A jungle of modems produced by different vendors
It is impossible to define a general "rule" about API provided by a Cellular Module (not necessarily a 4G module).
First of all every vendor usually implements standard AT Commands (both Hayes commands and extended standard commands for cellular devices). In the same way every vendor has it's own implementation of the user application area where the customers can store their own application to control the modem's functionalities and to use them according to their own application requirements.
AT commands remain the interface to be used when the modem needs to be connected (and driven) by an external host. When the user application area is used, instead, a wider set of API is usually provided. They may include:
A library exporting a subset of the OS capabilities (threads management, events, semaphores, mutexes, SW timers, FS access and so on)
A library offering the capability to manage the specific HW of the device (GPIOs, SPI, I2C, ADC, DAC and so on)
A library offering a programmatical way to perform action, related to connectivity, that would normally be executed through AT commands (registration status check, PIN code insertion, PDP context activation, SMS management, TCP/UDP/TLS sockets)
The latter usually access a base layer involving all the functionalities provided by the modem. Usually this is the same layer invoked by the AT commands sent through modem's serial interface.
Sending AT commands... from the vendor's API?
Of course it often happens that the library mentioned above provides just a subset of the functionalities usually exported with the AT commands set so, in order to "fill the gap", a further set of APIs is usually exported as well:
A set of functions that allow the simulation of AT commands sent to the modem's serial port. Sending them and parsering the responses they send in the vitual internal serial/USB interface allow the user to port in the internal user application area the the application they previously run on an external host processor (with obvious BOM benefits).
As an example, please check Telit Appzone here and here. It was the inspiration of my answer because I know it very well.
I don't know why you name the title that there's a relationship between AT command and Linux-C API.
Regarding AT command, you can take a look at this wiki article for general information.
Each module has a specified AT command sets. Normally, the module manufacture just offers AT command set and what return values are.
Is API a wrapper of AT command?
If you can use the API provided by the manufacturer, then yes, it's a wrapper of the AT command handler.
My question is what's the relationship between them? Is the API a wrapper for the AT commands?
It is impossible to be sure without having any details of the device, but probably any C API for it wraps the AT command set, either by communicating with the device directly over an internal serial interface or by going through a device driver that uses AT commands to communicate with it.
However, it is at least conceivable, albeit unlikely, that the 4G device offers an alternative control path that the C API uses (definitely via a driver in this case).
I'm not quite sure what the point of the question is, though. If you are programming the device and its 4G component in C, and the manufacturer has provided a C API, then use it! If you are programming in some other language then at least consider using the C API, which you should be able to access from most other languages in some language-specific way. You should not expend effort on rolling your own without a compelling reason to reject the API already provided to you.
I plan to use the PRISM libraries for a project running on a PC that controls one or multiple instruments and visualizes and stores the data of the device(s) and lets the user enter some control data. The devices have various digital and analog sensors and actors. They can be of different type and intelligence. Most often they have no 'real' intelligence and all the control logic sits in the PC.
This 'intelligence' needs to be constantly reading the data from a device. The communication can be of various kind, like a COM port, TCP/IP socket, HTTP to a web interface, etc.
I am not sure what's the best solution for that 'intelligent logic'. Since it needs a continuous communication with the device, it needs to be separated from all the UI tasks. It will need some kind of state-machine in a background worker or thread to build the higher process logic.
Question: Should it be an instance per device registered in PRISM as a service with a reference to that background worker? Or should that background worker be created and linked to the ViewModel I need for each configured instrument to handle it's data to show and edit? Or is there another best solution?
I think this si more general architecture question than a specific PRISM one...
I've done something similar with other MVVM framework and my solution was based on a single listener (I had only TCP sockets to coomunicate with instruments) registerd as a service. In your application you can either have multiple queues or a single queue with multiple producers.
All messages from the devices were inserted in a concurrent queue and each ViewModel (one for each device) read from that queue.
Communication from ViewModel to device happened directly without going through an "output" queue.
The whole application was build on await/async pattern to decouple UI from communication. I was able to send and receive multiple commands and notifications from serval devices at the same time without any issue.
But again this is really a broad question and mine is a broad answer lot of things depend of how you have to interact with dvices. My solution balances complexity with flexibility, but a lot of other architectures are available.
I am writing a cross-platform library which emulates sockets behaviour, having additional functionality in the between (App->mylib->sockets).
I want it to be the most transparent possible for the programmer, so primitives like select and poll must work accordingly with this lib.
The problem is when data becomes available (for instance) in the real socket, it will have to go through a lot of processing, so if select points to the real socket fd, app will be blocked a lot of time. I want the select/poll to unblock only when data is ready to be consumed (after my lib has done all the processing).
So I came across this eventfd which allows me to do exactly what I want, i.e. to manipule select/poll behaviour on a given fd.
Since I am much more familiarized with Linux environment, I don't know what is the windows equivalent of eventfd. Tried to search but got no luck.
Note:
Other approach would be to use another socket connected with the interface, but that seems to be so much overhead. To make a system call with all data just because windows doesn't have (appears so) this functionality.
Or I could just implement my own select, reinventing the wheel. =/
There is none. eventfd is a Linux-specific feature -- it's not even available on other UNIXy operating systems, such as BSD and Mac OS X.
Yes, but it's ridiculous. You can make a Layered Service Provider (globally installed...) that fiddles with the system's network stack. You get to implement all the WinSock2 functions yourself, and forward most of them to the underlying TCP. This is often used by firewalls or antivirus programs to insert themselves into the stack and see what's going on.
In your case, you'd want to use an ioctl to turn on "special" behaviour for your application. Whenever the app tries to create a socket, it gets forwarded to your function, which in turn opens a real TCP socket (say). Instead of returning that HANDLE though, you use a WinSock function to create ask for a dummy handle from the kernel, and give that to the application instead. You do your stuff in a thread. Then, when the app calls WinSock functions on the dummy handle, they end up in your implementation of read, select, etc. You can decouple select notifications on the dummy handle from those on the actual handle. This lets you do things like, for example, transparently give an app a socket that wraps data each way in encryption, indistinguishably from the original socket. (Almost indistinguishably! You can call some LSP APIs on a handle to find out if there's actually and underlying handle you weren't given.)
Pretty heavy-weight, and monstrous in some ways. But, it's there... Hope that's a useful overview.
I'm writing a TCP server/client application on Windows, to become familiar with the Winsock API. I come from an UNIX background and would like to know which of these could be the best approach to implement the application:
First the specification
Must scale well on multiprocessor and single-processor systems.
No hardset limit of connections.
Application can both listen for connections, acting as server, and act as client.
Multi threaded.
First approach:
Non-blocking select-like socket for listening, in the 'server' thread.
for each client connecting we spawn a separate thread.
Second approach:
Blocking socket for listening, in the 'server' thread.
for each client connecting we spawn a separate thread.
Third approach:
Non-blocking select-like socket for listening, in the 'server' thread.
No separate thread for each incoming connection, the protocol would need state information kept across sessions I suppose.
I wonder what is the most efficient and scalable approach, and especially if it can work with a UDP socket too.
Note: I'm writing the application in plain and old C. No .NET nor C++ involved, C++ exceptions disabled too.
As Gary says, I/O Completion Ports are the most efficient way to manage multiple network connections in a non-blocking/async manner on Windows platforms.
With IOCP you get notified when your networking operations complete and you can process these completions with a small number of threads. You get to decide how many threads you allocate to process the completions and the kernel decides when to use the threads that you're providing. It uses them in a LIFO order, to reduce context switching, so that if you are only using the minimal number of threads required at any point and you're reusing the same threads rather than cycling through all of the threads that you have available for use.
The asynchronous nature of IOCP programming can be a little confusing to start with, but once you get the hang of it it's fairly straight forward.
I have some free IOCP server code which demonstrates the basics and provides some example servers that are pretty easy to build on. You can find the code here: http://www.serverframework.com/products---the-free-framework.html. That page also links to some articles that I wrote to explain the code.
Relating this to the detail of your question. You should be looking at a variation on your third approach. Use AcceptEx() to accept new connections, this can be used in an asynchronous manner and so you don't need a separate thread for connection acceptance and can use the threads that are also processing your overlapped/async read and write operations.
I've written an asynchronous client which does not use blocking sockets, so if you're interested in that approach, then take a look at my client: http://codesprout.blogspot.com/2011/04/asynchronous-http-client.html
It's an HTTP client, but I've shown very little HTTP protocol processing in there, it's all just .NET sockets. The server would work in a similar way: you can take advantage of the *Async methods such as AsseptAsync.
Under Windows, the best performances are achieved by using I/O completion calls.
This is because the lists and queuing mechanism is done in the kernel, far from the heavy user-mode overhead (which drags your code down if you dare to do the hard work yourself).
Unfortunately, Windows I/O completion calls need to allocate many threads to scale and this is quickly killing the performances (as compared to Linux epoll which can scale independently of the number of worker threads you decide to involve in the task).
Recently, I discovered http://gwan.com/ a Web server which came from Windows and was then ported under Linux. And their authors describe the problem in details on their forum.
I am working on a server application for an embedded ARM platform. The ARM board is connected to various digital IOs, ADCs, etc that the system will consistently poll. It is currently running a Linux kernel with the hardware interfaces developed as drivers. The idea is to have a client application which can connect to the embedded device and receive the sensory data as it is updated and issue commands to the device (shutdown sensor 1, restart sensor 2, etc). Assume the access to the sensory devices is done through typical ioctl.
Now my question relates to the design/architecture of this server application running on the embedded device. At first I was thinking to use something like libevent or libev, lightweight C event handling libraries. The application would prioritize the sensor polling event (and then send the information to the client after the polling is done) and process client commands as they are received (over a typical TCP socket). The server would typically have a single connection but may have up to a dozen or so, but not something like thousands of connections. Is this the best approach to designing something like this? Of the two event handling libraries I listed, is one better for embedded applications or are there any other alternatives?
The other approach under consideration is a multi-threaded application in which the sensor polling is done in a prioritized/blocking thread which reads the sensory data and each client connection is handled in separate thread. The sensory data is updated into some sort of buffer/data structure and the connection threads handle sending out the data to the client and processing client commands (I supposed you would still need an event loop of sort in these threads to monitor for incoming commands). Are there any libraries or typical packages used which facilitate designing an application like this or is this something you have to start from scratch?
How would you design what I am trying to accomplish?
I would use a unix domain socket -- and write the library myself, can't see any advantages to using libvent since the application is tied to linux, and libevent is also for hundreds of connections. You can do all of what you are trying to do with a single thread in your daemon. KISS.
You don't need a dedicated master thread for priority queues you just need to write your threads so that it always processes high priority events before anything else.
In terms of libraries, you will possibly benifit from Google's protocol buffers (for serialization and representing your protocol) -- however it only has first class supports for C++, and the over the wire (serialization) format does a bit of simple bit shifting to numeric data. I doubt it will add any serious overhead. However an alternative is ASN.1 (asn1c).
My suggestion would be a modified form of your 2nd proposal. I would create a server that has two threads. One thread polling the sensors, and another for ALL of your client connections. I have used in embedded devices (MIPS) boost::asio library with great results.
A single thread that handles all sockets connections asynchronously can usually handle the load easily (of course, it depends on how many clients you have). It would then serve the data it has on a shared buffer. To reduce the amount and complexity of mutexes, I would create two buffers, one 'active' and another 'inactive', and a flag to indicate the current active buffer. The polling thread would read data and put it in the inactive buffer. When it finished and had created a 'consistent' state, it would flip the flag and swap the active and inactive buffers. This could be done atomically and should therefore not require anything more complex than this.
This would all be very simple to set up since you would pretty much have only two threads that know nothing about the other.