I have to send a command over serial and receive back an answer based on the command and do something based on the message received. I was told that I have to use callbacks as this is an asynchronous operation.
I have a 2 threads, one that can send messages and one that receives the messages.
Example:
//Thread 1
sendMessage("Initialize");
//Thread 2
while(1)
{
checkForMessages();
}
How can I write a function that is initialized for a specific message and handles the message recieved.
Example:
CommHandle(Command,MsgReceived)
{
if(command)
{
if(MsgReceived == ok)
...
if(MsgReceived == error)
...
}
}
I was told that I have to use callbacks as this is an asynchronous operation.
Not necessarily. There is something in Windows called "asynchronous I/O", this is to be regarded as an internal Windows term, which is synonymous with "overlapped I/O" (explanation here). When you are using overlapped I/O, you will get a callback when the transmission is finished. This is nice, since it reduces CPU load, but it is not really necessary if your program has nothing better to do while waiting. So it depends on the nature of your application.
But no matter the nature of your application, you should indeed handle all serial communication through threads, so that you won't cause the main GUI thread to freeze up in embarrassing ways.
Having one rx and one tx thread gives you a dilemma though: they are using the same port handle and they cannot freely access it, because that wouldn't be thread-safe. The solution is to either make one single super-thread handling all transmissions, or to protect the port handle through a mutex.
I'm not sure which method that is best, I have no recommendation. I have only used the "super-thread" one myself: one obvious advantage was that I could centralize WaitFor instructions like "kill thread", "port is open", "port is closed" at one place. But at the same time the code turned out rather complex.
How can I write a function that is initialized for a specific message and handles the message recieved.
Let your thread(s) shovel their received data into some buffers. A tx buffer and a rx buffer. Depending on your serial protocol and performance, you might have to use double buffers: one that is used for the current transmission and one that contains the most recently received data.
Then from main, pick up the data from the buffers. They need to be thread-safe. Once you have gotten that far, simply write a parser like you would with any form of data and take actions from there
Related
I have a very simple code, a data decomposition problem in which in a loop each process sends two large messages to the ranks before and after itself at each cycle. I run this code in a cluster of SMP nodes (AMD Magny cores, 32 core per node, 8 cores per socket). It's a while I'm in the process of optimizing this code. I have used pgprof and tau for profiling and it looks to me that the bottleneck is the communication. I have tried to overlap the communication with the computations in my code however it looks that the actual communication starts when the computations finish :(
I use persistent communication in ready mode (MPI_Rsend_init) and in between the MPI_Start_all and MPI_Wait_all bulk of the computation is done. The code looks like this:
void main(int argc, char *argv[])
{
some definitions;
some initializations;
MPI_Init(&argc, &argv);
MPI_Rsend_init( channel to the rank before );
MPI_Rsend_init( channel to the rank after );
MPI_Recv_init( channel to the rank before );
MPI_Recv_init( channel to the rank after );
for (timestep=0; temstep<Time; timestep++)
{
prepare data for send;
MPI_Start_all();
do computations;
MPI_Wait_all();
do work on the received data;
}
MPI_Finalize();
}
Unfortunately the actual data transfer does not start until the computations are done, I don't understand why. The network uses QDR InfiniBand Interconnect and mvapich2. each message size is 23MB (totally 46 MB message is sent). I tried to change the message passing to eager mode, since the memory in the system is large enough. I use the following flags in my job script:
MV2_SMP_EAGERSIZE=46M
MV2_CPU_BINDING_LEVEL=socket
MV2_CPU_BINDING_POLICY=bunch
Which gives me an improvement of about 8%, probably because of better placement of the ranks inside the SMP nodes however still the problem with communication remains. My question is why can't I effectively overlap the communications with the computations? Is there any flag that I should use and I'm missing it? I know something is wrong, but whatever I have done has not been enough.
By the order of ranks inside the SMP nodes the actual message sizes between the nodes is also 46MB (2x23MB) and the ranks are in a loop. Can you please help me? To see the flags that other users use I have checked /etc/mvapich2.conf however it is empty.
Is there any other method that I should use? do you think one sided communication gives better performance? I feel there is a flag or something that I'm not aware of.
Thanks alot.
There is something called progression of operations in MPI. The standard allows for non-blocking operations to only be progressed to completion once the proper testing/waiting call was made:
A nonblocking send start call initiates the send operation, but does not complete it. The send start call can return before the message was copied out of the send buffer. A separate send complete call is needed to complete the communication, i.e., to verify that the data has been copied out of the send buffer. With suitable hardware, the transfer of data out of the sender memory may proceed concurrently with computations done at the sender after the send was initiated and before it completed. Similarly, a nonblocking receive start call initiates the receive operation, but does not complete it. The call can return before a message is stored into the receive buffer. A separate receive complete call is needed to complete the receive operation and verify that the data has been received into the receive buffer. With suitable hardware, the transfer of data into the receiver memory may proceed concurrently with computations done after the receive was initiated and before it completed.
(words in bold are also bolded in the standard text; emphasis added by me)
Although this text comes from the section about non-blocking communication (ยง3.7 of MPI-3.0; the text is exactly the same in MPI-2.2), it also applies to persistent communication requests.
I haven't used MVAPICH2, but I am able to speak about how things are implemented in Open MPI. Whenever a non-blocking operation is initiated or a persistent communication request is started, the operation is added to a queue of pending operations and is then progressed in one of the two possible ways:
if Open MPI was compiled without an asynchronous progression thread, outstanding operations are progressed on each call to a send/receive or to some of the wait/test operations;
if Open MPI was compiled with an asynchronous progression thread, operations are progressed in the background even if no further communication calls are made.
The default behaviour is not to enable the asynchronous progression thread as doing so increases the latency of the operations somehow.
The MVAPICH site is unreachable at the moment from here, but earlier I saw a mention of asynchronous progress in the features list. Probably that's where you should start from - search for ways to enable it.
Also note that MV2_SMP_EAGERSIZE controls the shared memory protocol eager message size and does not affect the InfiniBand protocol, i.e. it can only improve the communication between processes that reside on the same cluster node.
By the way, there is no guarantee that the receive operations would be started before the ready send operations in the neighbouring ranks, so they might not function as expected as the ordering in time is very important there.
For MPICH, you can set MPICH_ASYNC_PROGRESS=1 environment variable when runing mpiexec/mpirun. This will spawn a background process which does "asynchronous progress" stuff.
MPICH_ASYNC_PROGRESS - Initiates a spare thread to provide
asynchronous progress. This improves progress semantics for
all MPI operations including point-to-point, collective,
one-sided operations and I/O. Setting this variable would
increase the thread-safety level to
MPI_THREAD_MULTIPLE. While this improves the progress
semantics, it might cause a small amount of performance
overhead for regular MPI operations.
from MPICH Environment Variables
I have tested on my cluster with MPICH-3.1.4, it worked! I believe MVAPICH will also work.
I am a bit unsure about this question, but I constantly running into troubles with my current design, and would be really greatful, if someone could show be a different approach to this.
My program writes async commands to a device via rs232, while constantly reading and reacting to received data.
This works all nice and neat, but during my init-phase, I have to send a bunch of commands, which would have to wait for a response, before letting the program countinue.
Now, this part I have to do in plain C, and all I could come up with was to use global vars and while loops. But I find this really not pretty.
Take this as an pseudo-code example:
OnReceive(data){
switch determineCommand(data)
case CMD1:
config.value1 = data.value1
case CMD2:
config.value2 = data.value2
default:
print data
}
DoCommandChain(){
Send(CMD1)
If("Send(CMD1)" got its response){
Send(CMD2)
}
}
Now, the problem is "If XY got its response" - because I don't want to use some vars to detect this and I can't rely on a return value for Send(CMD1), because this is just an indicator, that CMD1 was send - not there was something corresponding received.
I am asking here, because I would like to know what I could search for / read about, to solve this mess in a nice way.
Right now, my best idea for this would be to somehow setup a timer-guarded-function to monitor, if a specific response is received. Then, depending on the status of this response, go to the next Send or retry the last.
Like this:
Instead of `Send(CMD1)` -> `DoUntilResponse(Send(CMD1),Timeout,NumberOfRetries)`
DoUntilResponse(function,Timeout,NumberOfRetries){
registerExpectedResponse(CMD1, gotResponse) //awaiting some response for CMD1
for(i=0 ; i!=Numberofretries; i++){
if (Send(CMD1) == successfulSend ){
while(not timeout){
if gotResponse then break;
}
if gotResponse then break;
}
}
}
Edit:
Just to clarify: I am not worried about the serial-connection, or about how to fire the OnReceive function - this is all working already. What I can't get a clear idea of, is how to solve the above pseudo-code without the use of polling and preferably in pure C.
Assuming that write and read operations are executed in different threads, use the following algorithm:
Write thread.
Register expected response.
Send packet to the device
Wait for Received event with timeout. If event is set, continue. Timeout -
report error and exit (or make additional trial).
Read thread.
For every received packet:
Add it to the input buffer.
Analyze buffer. If it contains valid response, set "Received" event,
releasing the Write thread, clear input buffer, and continue reading.
If input buffer contains unrecognized data, report error, clear buffer
and continue reading.
If input buffer contains the beginning of expected response,
continue reading.
Threading and event notification is OS-specific. If your C++ compiler doesn't support multithreading, use portable library like Boost, or use OS-specific API. Notice that Read thread implements stream parsing logic, since serial communication is stream-oriented.
Edit.
"Register expected response" means: set some program variable(s) than mean "Command of type X is sent to device". According to this variable, Read thread expects to receive the packet, which should be sent by device according to communication protocol (application-specific). Another received packet, which may be generally valid, should be treated as error, because this is not response to the command just sent to device.
Another way: Set expected response size. In this case, once Read thread received expected amount of data, it sets Received event, leaving packet recognition task to the sender.
It is operating system specific. On Linux or Unix you should consider using a multiplexing syscall like poll(2) (or perhaps select(2) which I feel is obsolete, because it limits the maximal file descriptor number, see the C10K problem).
The C standard don't know about serial ports.
I need to write a Terminal to communicate with a COM - port and I need to be able to send commands from the COM-Port, as well as from the Console at the same time.
(I want to get access to a computer via two sensor nodes, that are communicating with each other wirelessly, so I still need a way to send something from the node to the Computer)
Now, I have implemented a Non Overlapped Serial Communication, but I am not sure, how to implement the "Send and Receive at the same time"-Part and I only have around 4 days to solve the problem. There is not really that much information out there, so I would welcome any pointers on how to implement that fastest or easiest way.
Overlapped I/O-Communication is not exactly very time friendly as far as I can see.
Is it possible to do this with multi-threading (only overlapped)?
I am guessing in that case I would have to read the buffer every few ms and make an own thread for the input?
Whether to use overlapped I/O or not isn't really the issue: overlapping just frees up some time for your program. I have written many programs like this and the conclusion is always to use a thread to handle all COM routines. Whether this thread calls overlapped or synchronous methods is less relevant, as long as the thread lies idle doing WaitForMultipleObjects().
The way I have written my most recent COM terminal is this (pseudo):
thread()
{
while not kill the thread event
{
WaitForMultipleObjects (open port, close port, kill the thread event)
if (open port)
{
send();
receive();
wait_for_send_and_receive();
}
}
}
send()
{
take COM_port mutex
if(there is something to send)
{
copy send_data to local_data, protect this with mutex
WriteFileEx(COM_port,
local_data,
size,
some_overlapped_struct_stuff);
handle errors
}
release COM_port mutex
}
receive()
{
take COM_port mutex
ReadFileEx(COM_port, ...);
handle errors
release COM_port mutex
}
wait_for_send_and_receive()
{
WaitForMultipleObjects (open port,
close port,
kill the thread event,
send done event from send callback routine (overlapped I/O),
receive done event from receive callback routine (overlapped I/O)
);
}
Naturally this is quite an over-simplification since you need various functions for open/close COM port, data shuffling etc. Several mutices are likely needed.
I'd share the real, working production code if it wasn't corporate property :( 4 days seems a bit optimistic, judging from my project log, it took me several months to develop a working COM port terminal to production quality level. The COM port driver alone is around 1k loc, with a lot of Win API calls all over.
I'm coding a part of little complex communication protocol to control multiple medical devices from single computer terminal. Computer terminal need to manage about 20 such devices. Every device uses same protocol fro communication called DEP. Now, I've created a loop that multiplexes within different devices to send the request and received the patient data associated with a particular device. So structure of this loop, in general, is something like this:
Begin Loop
Select Device i
if Device.Socket has Data
Strip Header
Copy Data on Queue
end if
rem_time = TIMEOUT - (CurrentTime - Device.Session.LastRequestTime)
if TIMEOUT <= 0
Send Re-association Request to Device
else
Sort Pending Request According to Time
Select First Request
Send the Request
Set Request Priority Least
end Select
end if
end Select
end Loop
I might have made some mistake in above pseudo-code, but I hope I've made myself clear about what this loop is trying to do. I've priority list structure that selects the device and pending request for that device, so that, all the requests and devices are selected at good optimal intervals.
I forgot to mention, above loop do not actually parse the received data, but it only strips off the header and put it in a queue. The data in queue is parsed in different thread and recorded in file or database.
I wish to add a feature so that other computers may also import the data and control the devices attached to computer terminal remotely. For this, I would need to create socket that would listen to commands in this INFINITE LOOP and send the data in different thread where PARSING is performed.
Now, my question to all the concurrency experts is that:
Is it a good design to use single socket for reading and writing in two different threads? Where each of the thread will be strictly involved in either reading or writing not both. Also, I believe socket is synchronized on process level, so do I need locks to synchronize the read and write over one socket from different threads?
There is nothing inherently wrong with having multiple threads handle a single socket; however, there are many good and bad designs based around this one very general idea. If you do not want to rediscover the problems as you code your application, I suggest you search around for designs that best fit your planned particular style of packet handling.
There is also nothing inherently wrong with having a single thread handle a single socket; however, if you put the logic handling on that thread, then you have selected a bad design, as then that thread cannot handle requests while it is "working" on the last reqeust.
In your particular code, you might have an issue. If your packets support fragmentation, or even if your algorithm gets a little ahead of the hardware due to timing issues, you might have just part of the packet "received" in the buffer. In that case, your algorithm will fail in two ways.
It will process a partial packet, one which has the first part of it's data.
It will mis-process the subsequent packet, as the information in the buffer will not start with a valid packet header.
Such failures are difficult to conceive and diagnose until they are encountered. Perhaps your library already buffers and splits messages, perhaps not.
In short, your design is not dictated by how many threads are accessing your socket: how many threads access your socket is dictated by your design.
I'm writing a program in linux to interface, through serial, with a piece of hardware. The device sends packets of approximately 30-40 bytes at about 10Hz. This software module will interface with others and communicate via IPC so it must perform a specific IPC sleep to allow it to receive messages that it's subscribed to when it isn't doing anything useful.
Currently my code looks something like:
while(1){
IPC_sleep(some_time);
read_serial();
process_serial_data();
}
The problem with this is that sometimes the read will be performed while only a fraction of the next packet is available at the serial port, which means that it isn't all read until next time around the loop. For the specific application it is preferable that the data is read as soon as it's available, and that the program doesn't block while reading.
What's the best solution to this problem?
The best solution is not to sleep ! What I mean is a good solution is probably to mix
the IPC event and the serial event. select is a good tool to do this. Then you have to find and IPC mechanism that is select compatible.
socket based IPC is select() able
pipe based IPC is select() able
posix message queue are also selectable
And then your loop looks like this
while(1) {
select(serial_fd | ipc_fd); //of course this is pseudo code
if(FD_ISSET(fd_set, serial_fd)) {
parse_serial(serial_fd, serial_context);
if(complete_serial_message)
process_serial_data(serial_context)
}
if(FD_ISSET(ipc_fd)) {
do_ipc();
}
}
read_serial is replaced with parse_serial, because if you spend all your time waiting for complete serial packet, then all the benefit of the select is lost. But from your question, it seems you are already doing that, since you mention getting serial data in two different loop.
With the proposed architecture you have good reactivity on both the IPC and the serial side. You read serial data as soon as they are available, but without stopping to process IPC.
Of course it assumes you can change the IPC mechanism. If you can't, perhaps you can make a "bridge process" that interface on one side with whatever IPC you are stuck with, and on the other side uses a select()able IPC to communicate with your serial code.
Store away what you got so far of the message in a buffer of some sort.
If you don't want to block while waiting for new data, use something like select() on the serial port to check that more data is available. If not, you can continue doing some processing or whatever needs to be done instead of blocking until there is data to fetch.
When the rest of the data arrives, add to the buffer and check if there is enough to comprise a complete message. If there is, process it and remove it from the buffer.
You must cache enough of a message to know whether or not it is a complete message or if you will have a complete valid message.
If it is not valid or won't be in an acceptable timeframe, then you toss it. Otherwise, you keep it and process it.
This is typically called implementing a parser for the device's protocol.
This is the algorithm (blocking) that is needed:
while(! complete_packet(p) && time_taken < timeout)
{
p += reading_device.read(); //only blocks for t << 1sec.
time_taken.update();
}
//now you have a complete packet or a timeout.
You can intersperse a callback if you like, or inject relevant portions in your processing loops.