I'm building a system consisting of many (> 100) equal nodes all connected via CAN bus.
The idea is that all nodes must have the same information, any node may generate an event and will broadcast it via CAN. For those events, the 8 byte payload provided by a CAN frame are enough and a broadcast will reach all nodes on a bus, so the requirement is met.
Now I also want to distribute firmware updates (or other files) to all nodes via CAN, obviously here I need some sort of fragmentation and the 8 bytes are a bit scarce.
Someone suggested CANopen to me to save me some work, but it appears that it only supports peer-to-peer mode with SDO block transfer and no broadcast.
Is there already a protocol that supports distributing files to all CAN nodes or do I have to come up with my own?
If so, what considerations should I take?
I haven't used CAN before.
In order to send bigger messages you can use the ISO TP layer. I have used a module in python that implements it and you can probably find libraries for other devices in other languates since it is quite common. To implement CANopen to send bigger than 8 bytes messages is overkill.
Yes, PDOs is used to process real time data, to transfer the same variables always, not an stream data protocol.
May be you can add a feedback PDO from the slaves to the server. I've worked with some nodes that when I wanted to enable them, I'd to send a enable and then wait that in a PDO from slave to master, the slave said it was enabled.
Or you can use SYNC.
Related
Let's assume I have m UDP streams uniquely identified by some id (e.g. RTP SSRC). I need to process them in n associated threads and association is 1-N, i.e. one UDP stream is processed by one or many threads.
What is the difference in kernel's networking stack performance if I:
Start m UDP servers each on different port. Each server processes one stream and pushes it's data to one or more associated threads.
Start just one server. All streams are handled by it's single port and this thread pushes each stream data next to one or more associated threads.
I think it comes down to the question: is it better to open one single port or many of them, where each will receive proportionally less data?
Is there possibility that single socket may be overwhelmed by the amount of incoming data? Or maybe socket, which is more logical thing in linux kernel than a physical thing, has not so much to do that the data itself, so there is no real difference?
What is the maximum bitrate the single UDP socket (with enlarged buffer) can handle?
I am sure I will best find the answer by browsing of kernel's networking code but maybe someone could give the answer straight away please. Thank you.
There is no easy answering this question because it all boils down to the processing speed of your threads and how you delegate the work among them.
If you think that the udp socket is going to be overwhelmed you can create a queue right behind the udp socket. This queue can grow as large as you allow it to grow. Of course you then use more memory.
What you will have then is a consumers/producer paradigm. One thread is putting things in the queue, other threads are taking from the queue.
If the processing of the threads is slower than the thread which is filling the queue, and this keeps going for a long time. Your queue is anyhow going to get overrun.
There are frameworks dedicated to the task of multimedia processing. You might want to take a look at gstreamer. http://gstreamer.freedesktop.org/documentation/
It has support for RTP and is basically a system that allows to create a pipeline of a datastream which is exactly what you are doing here.
You will find that gstreamer has ready made queues that allow to queue up some data somewhere in the pipeline. This anyhow proves to me that something like this is needed when you are processing at high speeds. Though I am not a gstreamer specialist. Gstreamer has building blocks so you can experiment with a pipeline and easily add queueing, remove it and compare the results of the overall pipeline. It does require some studying to get to know the api. It is written in C.
The more sockets you have, the more socket receive buffers you have, so the more space is available for incoming data.
This suggests that multiple socket may be the better option.
However datagrams can be lost anywhere, not just at the target host.
I'm writing a program on Linux to control about 1000 Patient Monitors at same time over UDP sockets. I've successfully written a library to parse and send messages to collect the data from a single patient monitor device. There are various scheduling constraints on the the device, listed below:-
Each device must constantly get an alive-request from computer client within max time-period of 300 milliseconds(may differ for different devices), otherwise connection is lost.
Computer client must send a poll-request to a device in order fetch the data within some time period. I'm polling for about 5 seconds of averaged data from patient monitor, therefore, I'm required to send poll-request in every 5 * 3 = 15 seconds. If I fail to send the request within 15 seconds time-frame, I looses the connection from device.
Now, I'm trying to extend my current program so that it is capable of handling about 1000+ devices at same time. Right now, my program can efficiently handle and parse response from just one device. In case of handling multiple devices, it is necessary to synchronize multiple responses from different device and serialize them and stream it over TCP socket, so that remote computers can also analyze the data. Well, that is not a problem because it is a well know multiple-producer and single consumer problem. My main concern is, what approach should I use in order to maintain alive-connection 1000+ devices.
After reading over Internet and browsing for similar questions on this website, I'm mainly considering two options:-
Use one thread per device. In order to control 1000+ device, I would end up in making 1000+ threads which does not look feasible to me.
Use multiplexing approach, selecting FD that requires attention and deal with it one at a time. I'm not sure how would I go about it and if multiplexing approach would be able to maintain alive-connection with all the devices considering above two constants.
I need some suggestions and advice on how to deal with this situation where you need to control 1000+ real-time-device over UDP sockets. Each device requires some alive-signal every 300 milliseconds (differ for different devices) and they require poll request in about 3 times the time interval mentioned during association phase. For example, patient monitors in ICU may require real-time (1 second averaged) data where as patient monitors in general wards may require 10-seconds averaged data, therefore, poll period for two devices would be 3*1(3 seconds) and 3*10 (30 seconds) respectively.
Thanks
Shivam Kalra
for the most part either approach is at least functionally capable of handling the functionality you describe, but by the sounds of things performance will be a crucial issue. From the figures you have provided it seems that the application could be CPU-buond.
A multithreaded approach has the advantage of using all of the available CPU cores on the machine, but multithreaded programs are notorious for being difficult to make reliable and robust.
You could also use the Apache's old tried-and-true forked-worker model - create, say, a separate process to handle a maximum of 100 devices. You could then need to write code to manage the mapping of connections to processes.
You could also use multiple hosts and some mechanism to distribute devices among them. This would have the advantage of making it easier to handle recovery situations. It sounds like your application could well be mission critical, and it may need to be architected so that if any one piece of hardware breaks then other hardware will take over automatically.
FTP is a pure TCP-connect protocol, and thus AFAIK "as fast as it gets" when considering TCP file transfer options.
However, there are some other products that do not run over TCP - examples are the commercial products BI.DAN-GUN, fasp and FileCatalyst. The latter product points out problems with pure TCP, and one can read more on Wikipedia, e.g. starting from Network Congestion.
What other alternatives are there? .. in particular Open Source ones? Also, one would think that this should be an RFC of sorts - a standard largish-file-transfer-specific protocol, probably running over UDP. Anyone know of such a protocol, or an initiative? (The Google SPDY is interesting, but doesn't directly address fast large-file-transfer)
Why do you think using TCP makes the transfer slower? TCP is usually able to use all available bandwidth. Using UDP instead is unlikely to be faster. In fact, if you tried to make a reliable UDP based file transfer, you'd likely end up implementing an inferior alternative to TCP - since you'd have to implement the reliability yourself.
What is problematic about FTP is that it performs multiple synchronous request-response commands for every file you transfer, and opens a new data connection for every file. This results in extremely inefficient transfers when a lot of smaller files are being transferred, because much of the time is spent waiting requests/responses and establishing data connections instead of actually transferring data.
A simple way to get around this issue is to pack the files/folders into an archive. While you can, of course, just make the archive, send it using FTP or similar, and unpack it on the other side, the time spent packing and unpacking may be unacceptable. You can avoid this delay by doing the packing and unpacking on-line. I'm not aware of any software that integrates such on-line packing/unpacking. You can, however, simply use the nc and tar programs in a pipeline (Linux, on Windows use Cygwin):
First run on the receiver:
nc -l -p 7000 | tar x -C <destination_folder>
This will make the receiver wait for a connection on port number 7000. Then run on the sender:
cd /some/folder
tar c ./* | nc -q0 <ip_address_of_receiver>:7000
This will make the sender connect to the receiver, starting the transfer. The sender will creating the tar archive, sending it to the receiver, which will be extracting it - all at the same time. If you need, you can reverse the roles of sender and receiver (by having the receiver connect to the sender).
This online-tar approach has none of the two performance issues of FTP; it doesn't perform any request-response commands, and uses only a single TCP connections.
However, note that this is not secure; anybody could connect to the receiver before our sender does, send him his own tar archive. If this is an issue, a VPN can be used, in combination with appropriate firewall rules.
EDIT: you mentioned packet loss as a problem with TCP performance, which is a significant problem, if the FileCatalyst page is to be believed. It is true that TCP may perform non-optimally with high packet loss links. This is because TCP usually reacts aggressively to packet loss, because it assumes loss is due to congestion; see Additive_increase/multiplicative_decrease. I'm not aware of any free/open source file transfer programs that would attempt to overcome this with custom protocols. You may however try out different TCP congestion avoidance algorithms. In particular, try Vegas, which does not use packet loss as a signal to reduce transmission rate.
There is a number of open source projects trying to tackle file transfer via UDP.
Take a look at UFTP, Tsunami or UDT, each project is at a different stage of development.
Depending on the bandwidth the latency and the pocket loss you are working with, each project will produce a different result. There is a blog article that tries to compare the 3 projects, here is the link http://www.filecatalyst.com/open-source-fast-file-transfers
Not open source, but Aspera's transfer speeds are worth checking out and are not affected by packet loss or network delay. You can see a chart here.
It also uses a proprietary protocol called fasp.
Consider using UDP based file transfer, have a look at JSCAPE MFT Server which implements a proprietary protocol known as AFTP (Accelerated File Transfer Protocol). Please review this link :
http://www.jscape.com/blog/bid/80668/UDP-File-Transfer-up-to-100x-Faster-than-TCP
Please keep in mind that JSCAPE's AFTP works optimally over long distance connections which have network latency. If there is no network latency AFTP will not perform any better than plain old regular FTP (over TCP).
Yes I do work for JSCAPE LLC.
I am writing a simple multi-drop RS485 protocol for serial communications within a distributed system. I am using an addressable model where slave devices are given a window of 20ms to respond. The master uC polls the connected devices for updates and they respond accordingly. I've employed checksums and take the necessary overrun precautions to ensure that connected devices will not respond to malformed messages. This method has proved effective in approximately 99% of situations, but I lose the packet if a new device is introduced during a communication session. Plugging in a new device "hot" will have negative effects on the signal being monitored by the slave devices, if only for an extremely short time. I'm on the software side of engineering, but how I can mitigate this situation without trying to recreate TCP? We use a polling model because it is fast and does the job well for our application, no need for RTOS functionality. I have an abundance of cycles on each cpu, think in basic terms.
Sending packets over the RS485 is not a reliable communication. You will have to handle the lost of packets anyway. Of course, you won't have to reinvent TCP. But you will have to detect lost packets by means of timeout monitoring and sequence numbers. In simple applications this can be done at application level, what keeps you far off from the complexity of TCP. When your polling model discards all packets with invalid checksum this might be integrated with less effort.
If you want to check for collisions, that can be caused by hot plugs or misbehaving devices there are probably some improvements. Some hardware allows to read back the own transmissing. If you find a difference between sent data and receive data, you can assume a collision and repeat the packet. This will also require a kind of sequence numbering.
Perhaps I've missed something in your question, but can't you just write the master so that if a response isn't seen from a device within the allowed time, it re-polls that device?
I am working on a server application for an embedded ARM platform. The ARM board is connected to various digital IOs, ADCs, etc that the system will consistently poll. It is currently running a Linux kernel with the hardware interfaces developed as drivers. The idea is to have a client application which can connect to the embedded device and receive the sensory data as it is updated and issue commands to the device (shutdown sensor 1, restart sensor 2, etc). Assume the access to the sensory devices is done through typical ioctl.
Now my question relates to the design/architecture of this server application running on the embedded device. At first I was thinking to use something like libevent or libev, lightweight C event handling libraries. The application would prioritize the sensor polling event (and then send the information to the client after the polling is done) and process client commands as they are received (over a typical TCP socket). The server would typically have a single connection but may have up to a dozen or so, but not something like thousands of connections. Is this the best approach to designing something like this? Of the two event handling libraries I listed, is one better for embedded applications or are there any other alternatives?
The other approach under consideration is a multi-threaded application in which the sensor polling is done in a prioritized/blocking thread which reads the sensory data and each client connection is handled in separate thread. The sensory data is updated into some sort of buffer/data structure and the connection threads handle sending out the data to the client and processing client commands (I supposed you would still need an event loop of sort in these threads to monitor for incoming commands). Are there any libraries or typical packages used which facilitate designing an application like this or is this something you have to start from scratch?
How would you design what I am trying to accomplish?
I would use a unix domain socket -- and write the library myself, can't see any advantages to using libvent since the application is tied to linux, and libevent is also for hundreds of connections. You can do all of what you are trying to do with a single thread in your daemon. KISS.
You don't need a dedicated master thread for priority queues you just need to write your threads so that it always processes high priority events before anything else.
In terms of libraries, you will possibly benifit from Google's protocol buffers (for serialization and representing your protocol) -- however it only has first class supports for C++, and the over the wire (serialization) format does a bit of simple bit shifting to numeric data. I doubt it will add any serious overhead. However an alternative is ASN.1 (asn1c).
My suggestion would be a modified form of your 2nd proposal. I would create a server that has two threads. One thread polling the sensors, and another for ALL of your client connections. I have used in embedded devices (MIPS) boost::asio library with great results.
A single thread that handles all sockets connections asynchronously can usually handle the load easily (of course, it depends on how many clients you have). It would then serve the data it has on a shared buffer. To reduce the amount and complexity of mutexes, I would create two buffers, one 'active' and another 'inactive', and a flag to indicate the current active buffer. The polling thread would read data and put it in the inactive buffer. When it finished and had created a 'consistent' state, it would flip the flag and swap the active and inactive buffers. This could be done atomically and should therefore not require anything more complex than this.
This would all be very simple to set up since you would pretty much have only two threads that know nothing about the other.