I'm having one of those mental-block moments in trying to implement what should be a fairly simple routine, for serial IO.
The situation is that we have an embedded linux board (think Pi / Beagle) which communicates with another device on the UART using the standard Linux termios code.
The problem is we have two clashing requirements:
In the transmit direction, we want to block on the Linux messaging queue msgrcv() function until a message arrives for us to send.
In the receive direction, we need to wait/block for an incoming message (which can have a termination character for canonical mode operation).
Tx and Rx are asynchronous and not related to each other - either could want to happen at any time.
Polling would be a pain as it introduces an overhead in CPU cycles and delay in response.
One approach would be to split this into two threads, one handling the Tx and blocking on msgrcv(), and the other on the Rx and blocking on UART read() in canonical mode - but that would introduce the pain of setting up semaphores between the Tx & Rx processes and both having to repeatedly open & close the serial port, and the Rx thread would presumably end up having to poll the semaphore in case the Tx wanted control, putting us back to polling.
I would stress that I'm relatively new to all this Linux stuff, so am entirely ready to be shown the bleeding obvious solution/method/call/operation that I'm missing here.
Is there some way to be blocking on the UART Rx but still able to transmit on demand?
In the end I followed Martin James' suggestion, doing something like this:
fd = open(serial_port);
pthread_create(TxThread, fd);
pthread_create(RxThread, fd);
Linux seems entirely happy with this, both threads do their jobs with no problem.
It might be simpler to have a thread for the msgqueue, in combination with a pipe, so you can wait on the pipe and the serial port using select()/poll()/... in your serial tx/rx thread.
Fortunately you don't even have to do that. It turns out that on Linux (but not on all other unixes!) message queue IDs are also file descriptors, so you can just wait on the message queue and the serial port in one poll() call. You only have to check which one is active to figure out what you have to do.
See also Why is there no poll/select-like mechanism for message queues.
Related
I have an imx8 module running Linux on my PCB and i would like some tips or pointers on how to modify the UART driver to allow me to be able to detect the end of frame very quickly (less than 2ms) from my user space C application. The UART frame does not have any specific ending character or frame length. The standard VTIME of 100ms is much too long
I am reading from a Sim card, i have no control over the data, no control over the size or content of the data. I just need to detect the end of frame very quickly. The frame could be 3 bytes or 500. The SIM card reacts to data that it receives, typically I send it a couple of bytes and then it will respond a couple of ms later with an uninterrupted string of bytes of unknown length. I am using an iMX8MP
I thought about using the IDLE interrupt to detect the frame end. Turn it on when any byte is received and off once the idle interrupt fires. How can I propagate this signal back to user space? Or is there an existing method to do this?
Waiting for an "idle" is a poor way to do this.
Use termios to set raw mode with VTIME of 0 and VMIN of 1. This will allow the userspace app to get control as soon as a single byte arrives. See:
How to read serial with interrupt serial?
How do I use termios.h to configure a serial port to pass raw bytes?
How to open a tty device in noncanonical mode on Linux using .NET Core
But, you need a "protocol" of sorts, so you can know how much to read to get a complete packet. You prefix all data with a struct that has (e.g.) A type and a payload length. Then, you send "payload length" bytes. The receiver gets/reads that fixed length struct and then reads the payload which is "payload length" bytes long. This struct is always sent (in both directions).
See my answer: thread function doesn't terminate until Enter is pressed for a working example.
What you have/need is similar to doing socket programming using a stream socket except that the lower level is the UART rather than an actual socket.
My example code uses sockets, but if you change the low level to open your uart in raw mode (as above), it will be very similar.
UPDATE:
How quickly after the frame finished would i have the data at the application level? When I try to read my random length frames currently reading in 512 byte chunks, it will sometimes read all the frame in one go, other times it reads the frame broken up into chunks. –
Engo
In my link, in the last code block, there is an xrecv function. It shows how to read partial data that comes in chunks.
That is what you'll need to do.
Things missing from your post:
You didn't post which imx8 board/configuration you have. And, which SIM card you have (the protocols are card specific).
And, you didn't post your other code [or any code] that drives the device and illustrates the problem.
How much time must pass without receiving a byte before the [uart] device is "idle"? That is, (e.g.) the device sends 100 bytes and is then finished. How many byte times does one wait before considering the device to be "idle"?
What speed is the UART running at?
A thorough description of the device, its capabilities, and how you intend to use it.
A uart device doesn't have an "idle" interrupt. From some imx8 docs, the DMA device may have an "idle" interrupt and the uart can be driven by the DMA controller.
But, I looked at some of the linux kernel imx8 device drivers, and, AFAICT, the idle interrupt isn't supported.
I need to read everything in one go and get this data within a few hundred microseconds.
Based on the scheduling granularity, it may not be possible to guarantee that a process runs in a given amount of time.
It is possible to help this a bit. You can change the process to use the R/T scheduler (e.g. SCHED_FIFO). Also, you can use sched_setaffinity to lock the process to a given CPU core. There is a corresponding call to lock IRQ interrupts to a given CPU core.
I assume that the SIM card acts like a [passive] device (like a disk). That is, you send it a command, and it sends back a response or does a transfer.
Based on what command you give it, you should know how many bytes it will send back. Or, it should tell you how many optional bytes it will send (similar to the struct in my link).
The method you've described (e.g.) wait for idle, then "race" to get/process the data [for which you don't know the length] is fraught with problems.
Even if you could get it to work, it will be unreliable. At some point, system activity will be just high enough to delay wakeup of your process and you'll miss the window.
If you're reading data, why must you process the data within a fixed period of time (e.g. 100 us)? What happens if you don't? Does the device catch fire?
Without more specific information, there are probably other ways to do this.
I've programmed such systems before that relied on data races. They were unreliable. Either missing data. Or, for some motor control applications, device lockup. The remedy was to redesign things so that there was some positive/definitive way to communicate that was tolerant of delays.
Otherwise, I think you've "fallen in love" with "idle interrupt" idea, making this an XY problem: https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem
I want to write C code for an embedded system such that the data received at the RS232 port should be read continuously without creating a separate "RS232 TASK" for reading the data.
Can anyone help me with this?
I just need a basic approach for reading data without task creation.
Identify the function that tells you if some data was received. Commonly it returns a boolean value or the number of received bytes. (BTW, most protocols on RS232 allows 5 to 8 data bits per transmission.)
Use that function in a conditional block to call the next function that actually reads one or more received bytes. In case that nothing was received, this prevents your loop to block.
Example (without knowing how the functions are named in your case):
/* any task */ {
for (;;) /* or any other way of looping */ {
/* do some stuff, if needed */
if (areRs232DataAvailable()) {
uint8_t data = fetchRs232ReceivedByte();
/* handle received data */
}
/* do some stuff, if needed */
}
}
I would ask why you think reading data from a UART (which I assume is what you mean by "RS-232") requires a task at all? A solution will depend a great deal on your platform and environment and you have not specified other than FreeRTOS which does not provide any serial I/O support.
If your platform or device library already includes serial I/O, then you might use that, but at the very lowest level, the UART will have a status register with a "data available" bit, and a register or FIFO containing that data. You can simply poll the data availability, then read the data.
To avoid data loss while the processor is perhaps busy with other tasks, you would use either interrupts or DMA. At the very least the UART will be capable of generating an interrupt on receipt of a character. The interrupt handler would place the new data into a FIFO buffer (such as an RTOS message queue), and tasks that receive serial data simply read from the buffer asynchronously.
DMA works similarly, but you queue the data in response to the DMA interrupt. That will reduce the interrupt rate, but you have to deal with the possibility of a partially full DMA buffer waiting indefinitely. Also not all platforms necessarily support UART as a DMA source, or even DMA at all.
This may be a very basic question/design but I am struggling with the correct method to handle the system I am going to define here.
I have a system with a single client (PC) that will connect to an embedded Linux board (Raspberry Pi) via TCP/IP protocol. This will be a command/response system where the PC will ask for something and the raspberry PI will respond with the results.
For Example:
CMD => Read/Return ADC Channel X
RSP => ADC Channel X Data
For this type of system I have already defined a packet protocol that will allow for this interaction. My problem is how to handle this on the Raspberry PI. I envision having a single thread handling the TCP connection; placing incomming data into a thread safe queue and pulling outgoing data from a thread safe queue. Then the main thread would poll the queue periodically looking for data. When data is found the command would be processed and a response will be generated. All commands have a response.
The main thread will also be doing other time critical tasks (PID control loop) so it cannot wait for incoming or outgoing data.
My guess is this type of system is fairly common and there is probably a good approach to implementing this type of system. I am very new to Linux programming but I have been programming highly embedded systems (No OS) forever. Just struggling with the correct approach for this type of design.
Note I chose TCP/IP because it handles retying in case of failure. In my case every command has a response so UDP could be used if it makes the design easier/more flexible.
Any help is greatly appreciated.
I tend to avoid threads if I can and only use them if I have to because they make debugging the program harder. They turn a determinsitic problem into a non-deterministic one. So my initial approach would be to see if I can do this without a thread and still achieve concurrency. This is possible using select which will notify your main program when there is something on the socket that needs to be read. Then, when there is something on the socket, it can read the data, process it, and wait for the next event. Problems with this approach is if the computation on the received data takes longer than the acceptable time wanted to process the next element of data, you could end up with a backlog of unprocessed data on the socket. If this is going to happen then you can go ahead and run the receive loop in thread, and the work function in another thread, or fork a new process and deal with a copy of the data from the new process.
the ultra classic linux approach is to have a listener program that forks a new copy of itself for each new client. Linux even has a built in demon that does that for you (initd - although that might have changed with all the systemd stuff). Thats how sshd, telnetd, ftpd all work. No threads
I am writing a program that interfaces with a particular serial device. The serial device has two channels, and a hardware rx and tx buffer for each channel. Basically, at any given time , you can read/write to either channel on the device.
I am trying to read data from a channel, validate it (and perhaps use some of the data), and then transmit it. Reads are accomplished with iotctl calls to the device, while writes are accomplished with a call to the write() system call.
The main issue I have is with data throughput. I'd like to have an individual thread handle reading and writing for each channel (i.e., a read thread and write thread for each of the two channels). However, I have hit a snag. Everything on the device, from Linux's perspective is accessed via one single device, and I'm not sure that Linux notes that the device has multiple channels.
As a result, currently I open a single file descriptor to the device and perform my reads and writes serially. I'd like to go to the threaded approach, but I'm wondering if concurrent ioctl() and write() calls would cause issues. I understand that read() and write() and not thread safe, but I was wondering if there's any way around that (perhaps calling open() twice, one with read privileges, one with write privileges).
Thanks for your help. Also, I want to avoid having to write my own driver, but that may be an inevitable conclusion...
Also, as a side note, I'm particularly concerned that the device has extremely small hardware buffers. Is there any way to determine how much space the OS uses for a software buffer for data? That is, can I determine whether or not the OS has it's own buffer that is used to prevent overflow of the hardware buffer? The device in question is an I2C UART Bridge.
You can use semaphore to make a mutual exclusion between read/write thread
sem_t sync_rw;
/*init semaphore */
err=sem_init(&sync_rw,0,1); /* shared between thread and initialized with 1 */
if( err != 0 )
{
perror("cannot init semaphore \n");
return -1;
}
in thread write function you do this :
sem_wait(&sync_rw);
write(...)
sem_post(&sync_rw);
same for thread reader :
sem_wait(&sync_rw);
iotctl(...)
sem_post(&sync_rw);
finally :
sem_destroy(&sync_rw);
I'm writing a program in linux to interface, through serial, with a piece of hardware. The device sends packets of approximately 30-40 bytes at about 10Hz. This software module will interface with others and communicate via IPC so it must perform a specific IPC sleep to allow it to receive messages that it's subscribed to when it isn't doing anything useful.
Currently my code looks something like:
while(1){
IPC_sleep(some_time);
read_serial();
process_serial_data();
}
The problem with this is that sometimes the read will be performed while only a fraction of the next packet is available at the serial port, which means that it isn't all read until next time around the loop. For the specific application it is preferable that the data is read as soon as it's available, and that the program doesn't block while reading.
What's the best solution to this problem?
The best solution is not to sleep ! What I mean is a good solution is probably to mix
the IPC event and the serial event. select is a good tool to do this. Then you have to find and IPC mechanism that is select compatible.
socket based IPC is select() able
pipe based IPC is select() able
posix message queue are also selectable
And then your loop looks like this
while(1) {
select(serial_fd | ipc_fd); //of course this is pseudo code
if(FD_ISSET(fd_set, serial_fd)) {
parse_serial(serial_fd, serial_context);
if(complete_serial_message)
process_serial_data(serial_context)
}
if(FD_ISSET(ipc_fd)) {
do_ipc();
}
}
read_serial is replaced with parse_serial, because if you spend all your time waiting for complete serial packet, then all the benefit of the select is lost. But from your question, it seems you are already doing that, since you mention getting serial data in two different loop.
With the proposed architecture you have good reactivity on both the IPC and the serial side. You read serial data as soon as they are available, but without stopping to process IPC.
Of course it assumes you can change the IPC mechanism. If you can't, perhaps you can make a "bridge process" that interface on one side with whatever IPC you are stuck with, and on the other side uses a select()able IPC to communicate with your serial code.
Store away what you got so far of the message in a buffer of some sort.
If you don't want to block while waiting for new data, use something like select() on the serial port to check that more data is available. If not, you can continue doing some processing or whatever needs to be done instead of blocking until there is data to fetch.
When the rest of the data arrives, add to the buffer and check if there is enough to comprise a complete message. If there is, process it and remove it from the buffer.
You must cache enough of a message to know whether or not it is a complete message or if you will have a complete valid message.
If it is not valid or won't be in an acceptable timeframe, then you toss it. Otherwise, you keep it and process it.
This is typically called implementing a parser for the device's protocol.
This is the algorithm (blocking) that is needed:
while(! complete_packet(p) && time_taken < timeout)
{
p += reading_device.read(); //only blocks for t << 1sec.
time_taken.update();
}
//now you have a complete packet or a timeout.
You can intersperse a callback if you like, or inject relevant portions in your processing loops.