I have a trouble in reading datas from serial in Linux system. I am trying to connect a sensor with the linux system by using UART. I can read and write /deve/ttyS1. But the problem is that I dont want to poll the message from UART. Instead, I want to use the asynchronized mode to get the data. As the data comes, the call back function will enter a certain routine and run my code. The problem no is that the sensor sends me different packets and each one contains various bytes of data. They are coming every one second!
For example:
Time Sensor MyLinux
1s 50bytes
124bytes
2s 40bytes
174bytes
3s 60bytes
244bytes
My question is how to use the asynch serial programming so that in the call back function, those two packets can be read as two messages
say
50 bytes comes, the call back function can let me read 50 bytes
127 bytes comes, the call back function can let me read 127 bytes
Now, its like, 50 bytes comes, I can only read 27bytes, and the rest 23 are in the next message.
My setting for serial port in POSIX is:
/* now we setup the values in port's termios */
serial->tio.c_cflag=baudrate|databits|checkparity|stopbits|CLOCAL|CREAD;
serial->tio.c_iflag=IGNPAR;
serial->tio.c_oflag=0;
serial->tio.c_lflag=0;
serial->tio.c_cc[VMIN]=28;
serial->tio.c_cc[VTIME]=6;
/* we flush the port */
tcflush(serial->fd,TCOFLUSH);
tcflush(serial->fd,TCIFLUSH);
/* we send new config to the port */
tcsetattr(serial->fd,TCSANOW,&(serial->tio));
Try to set VMIN and VTIME at following values:
serial->tio.c_cc[VMIN]=0;
serial->tio.c_cc[VTIME]=1;
Then you'll come out of select() after reading a complete chunk of data from your sensor. If the number of bytes is less, than you've expected, you can set timeout for select and read the data once more appending to your current buffer. If you get data before the timeout, then you have your complete message.
Related
I have an imx8 module running Linux on my PCB and i would like some tips or pointers on how to modify the UART driver to allow me to be able to detect the end of frame very quickly (less than 2ms) from my user space C application. The UART frame does not have any specific ending character or frame length. The standard VTIME of 100ms is much too long
I am reading from a Sim card, i have no control over the data, no control over the size or content of the data. I just need to detect the end of frame very quickly. The frame could be 3 bytes or 500. The SIM card reacts to data that it receives, typically I send it a couple of bytes and then it will respond a couple of ms later with an uninterrupted string of bytes of unknown length. I am using an iMX8MP
I thought about using the IDLE interrupt to detect the frame end. Turn it on when any byte is received and off once the idle interrupt fires. How can I propagate this signal back to user space? Or is there an existing method to do this?
Waiting for an "idle" is a poor way to do this.
Use termios to set raw mode with VTIME of 0 and VMIN of 1. This will allow the userspace app to get control as soon as a single byte arrives. See:
How to read serial with interrupt serial?
How do I use termios.h to configure a serial port to pass raw bytes?
How to open a tty device in noncanonical mode on Linux using .NET Core
But, you need a "protocol" of sorts, so you can know how much to read to get a complete packet. You prefix all data with a struct that has (e.g.) A type and a payload length. Then, you send "payload length" bytes. The receiver gets/reads that fixed length struct and then reads the payload which is "payload length" bytes long. This struct is always sent (in both directions).
See my answer: thread function doesn't terminate until Enter is pressed for a working example.
What you have/need is similar to doing socket programming using a stream socket except that the lower level is the UART rather than an actual socket.
My example code uses sockets, but if you change the low level to open your uart in raw mode (as above), it will be very similar.
UPDATE:
How quickly after the frame finished would i have the data at the application level? When I try to read my random length frames currently reading in 512 byte chunks, it will sometimes read all the frame in one go, other times it reads the frame broken up into chunks. –
Engo
In my link, in the last code block, there is an xrecv function. It shows how to read partial data that comes in chunks.
That is what you'll need to do.
Things missing from your post:
You didn't post which imx8 board/configuration you have. And, which SIM card you have (the protocols are card specific).
And, you didn't post your other code [or any code] that drives the device and illustrates the problem.
How much time must pass without receiving a byte before the [uart] device is "idle"? That is, (e.g.) the device sends 100 bytes and is then finished. How many byte times does one wait before considering the device to be "idle"?
What speed is the UART running at?
A thorough description of the device, its capabilities, and how you intend to use it.
A uart device doesn't have an "idle" interrupt. From some imx8 docs, the DMA device may have an "idle" interrupt and the uart can be driven by the DMA controller.
But, I looked at some of the linux kernel imx8 device drivers, and, AFAICT, the idle interrupt isn't supported.
I need to read everything in one go and get this data within a few hundred microseconds.
Based on the scheduling granularity, it may not be possible to guarantee that a process runs in a given amount of time.
It is possible to help this a bit. You can change the process to use the R/T scheduler (e.g. SCHED_FIFO). Also, you can use sched_setaffinity to lock the process to a given CPU core. There is a corresponding call to lock IRQ interrupts to a given CPU core.
I assume that the SIM card acts like a [passive] device (like a disk). That is, you send it a command, and it sends back a response or does a transfer.
Based on what command you give it, you should know how many bytes it will send back. Or, it should tell you how many optional bytes it will send (similar to the struct in my link).
The method you've described (e.g.) wait for idle, then "race" to get/process the data [for which you don't know the length] is fraught with problems.
Even if you could get it to work, it will be unreliable. At some point, system activity will be just high enough to delay wakeup of your process and you'll miss the window.
If you're reading data, why must you process the data within a fixed period of time (e.g. 100 us)? What happens if you don't? Does the device catch fire?
Without more specific information, there are probably other ways to do this.
I've programmed such systems before that relied on data races. They were unreliable. Either missing data. Or, for some motor control applications, device lockup. The remedy was to redesign things so that there was some positive/definitive way to communicate that was tolerant of delays.
Otherwise, I think you've "fallen in love" with "idle interrupt" idea, making this an XY problem: https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem
I'm programming on a STM32 board and I'm confused on how to use my peripherals : polling, interrupt, DMA, DMA interrupt...
Actually, I coded an UART module which send basics data and it works in polling, interrupt and DMA mode.
But I'd like to be able to send and receive specific frames with variable lengths, for example:
[ START | LGTH | CMD_ID | DATA(LGTH) | CRC ]
I also have sensors and I'd like to interact received DATA in these UART frames with sensors.
So, what I don't understand is:
how to program the UART module to work in "frame" mode? (buffer? circular DMA? interrupt? where, when..)
when I'm able to send or receive frame with my UART, what is the best way to interact with sensors? (inside a timer interrupt? in a state machine ? with extern variable? ...)
Here is my Libraries tree
In future, the idea is to carry this application in freertos
Thank you!
Absolutelly in DMA when it is available.
You have one big (good solution is cyclic) buffer and you just write data from one side. If DMA does not already work, you start the DMA with your buffer.
If DMA works, you just write your data to buffer and you wait DMA transfer complete interrupt.
Later in this interrupt you increase read pointer of buffer (as you sent some data already) and check if any data available to send over DMA. Set memory address to DMA and number of bytes in buffer to send.
Again, when DMA TC IRQ happens, do process again.
There is no support for FRAME, but only in plain bytes. It means you have to "invent" your own frame protocol and use it in app.
Later, when you want to send that FRAME over UART, you have to:
Write start byte to buffer
Write other header bytes
Write actual data
Write stop bytes/CRC/whatever
Check if DMA does not work, if it does not, start it.
Normally, I use this frame concept:
[START, ADDRESS, CMD, LEN, DATA, CRC, STOP]
START: Start byte indicating start of frame
ADDRESS: Address of device when multiple devices are in use on bus
CMD: Command ID
LEN: 2 bytes for data length
DATA: Actual data in bytes of variable length
CRC: 2 bytes for CRC including: address, cmd, len, data
STOP: Stop byte indicating end of frame
This is how I do it in every project where necessary. This does not use CPU to send data, just sets DMA and starts transmission.
From app perspective, you just have to create send_send(data, len) function which will create frame and put it to buffer for transmission.
Buffer size must be big enough to fit your requirements:
How much data at particular time (is it continues or a lot of data at small time)
UART baudrate
For specific question, ask and maybe I can provide some code examples from my libraries as reference.
In this case, where you need to implement that protocol, I would probably use plain interrupts and, in the handler, use a byte-by-byte state-machine to parse the incoming bytes into a frame buffer.
Only when a complete, valid frame has been received is it necessary to signal some semaphore/event and requuest a scheduler run, otherwise, you can handle any protocol error as you require - maybe tx some 'error-repeat' message and reset the state-machine to await the nexx start-of-frame buyte.
If you use DMA for this, then the variable frame-length is going to be awkward and you STILL have to iterate the received data to validate you protocol:(
DMA doesn't sound like a good fit for this, to me...
EDIT: if no preemptive multitasker, then forget about all that semaphore gunge above:) Still, it's easier to check a boolean 'validFrameRx' flag than parse DMA block data.
How to read data on serial port in byte by byte fashion.
I have a source which sends out packets of varying packet size. I am reading the data in raw mode(non-canonical). When i set VMIN, i am able to get packet of that size or slightly larger.
for ex: If the received packet size is 46 bytes, and if i set VMIN to say '1'. I receive the data in 2 chunks(meaning 2 read calls were needed to get the complete data with one fetching first 32 and next fetching the rest 14 bytes).
If i set VMIN to 46, complete packet is fetched.
But the problem here is varying packet size. If the data packet size is more(say 70 bytes), it will mess up the buffer and following reads as it reads 60+ bytes in first read and rest in next read.
So i am thinking to read the data byte by byte and determine the end of the packet.
Does anyone know if it is do-able. Or any suggestion on how to read the complete data packet in one read operation.
UART setting:
Baud: 115200
No parity.
1 stop bit.
8N1.
No flow control.
Thanks in advance.
A good approach for for processing serial data is to Read chunks of data from port the into a buffer and then pull byte by byte from the buffer.
Serial port reading is affected by the timeout settings and incoming data flow, so the number of bytes per read are not guaranteed to be consistent. For example, if you knew that packets were always going to be 46 bytes, then you might think to set Vmin to 46 and expect to get 46 bytes per read. However, if the sending source sends multiple packets without delays between, then you might get all of one and part of another packet. If the sending source were to delay during the transmission of a packet for longer than the receiving port's timeout, then you would get fewer than Vmin bytes.
Be sure to code for the possibility of lost data. For example, let's say that packets start with and ends with . You start pulling data from the buffer and the first byte is , but 49 bytes later you encounter meaning a new packet, but without having seen the from the previous packet. There should of course also be a CRC for the packet, or at least a checksum.
Since you are reading data that is structured into packets of variable size, you should add a 2 byte header for each packet and set it to the packet size.
In the reader you would read 2 bytes first and then decide how many bytes to read to receive the whole packet.
I have studied many useful threads and some tutorials, but I'm still having some issues with something that should be very simple. For reference here are some threads that I've perused:
How to implement a timeout in read function call?
how to open, read, and write from serial port in C
At any rate, I have a bit of a problem. My code works fine if I receive data. If I don't, the read() function stalls and the only way to get out of my program is to use kill -9 (NOTE: I use signal handling to signal to the thread reading the serial data to terminate. This is not the culprit, the read() call still stalls even if I have removed my signal handling). What I'm trying to do is to have a read that blocks and reads a chunk at a time (therefore saving CPU usage), however if the read receives no data, I wan't it to timeout.
Here are the settings that I'm applying to the port:
struct termios serial_struct;
serial_struct.c_cflag = B115200 | CS8 | CLOCAL | CREAD;
serial_struct.c_iflag = IGNPAR;
serial_struct.c_oflag = 0;
serial_struct.c_lflag = 0;
serial_struct.c_cc[VTIME] = 1; // timeout after .1s that isn't working
serial_struct.c_cc[VMIN] = 64; // want to read a chunk of 64 bytes at a given time
I then set these settings with tcsetattr() and confirm that the port received the settings via tcgetattr(). I'm thinking that my settings may be conflicting, because my reads appear to be blocking and wait until 64 bytes are received, but do not do anything with regards to the timeout. I understand that I can use select() to deal with a timeout, but I'm hoping to avoid the multiple system calls.
As always, thanks in advance for the help.
From man 3 termios:
MIN > 0; TIME > 0: TIME specifies the limit for a timer in tenths of a second. Once an initial byte of input becomes available, the timer is restarted after each further byte is received. read(2) returns either when the lesser of the number of bytes requested or MIN byte have been read, or when the inter-byte timeout expires. Because the timer is only started after the initial byte becomes available, at least one byte will be read.
Note that the timer does not start until at least one byte of data is received. After receiving that first data byte, the read will timeout if there is ever a gap of TIME tenths of a second between receiving consecutive data bytes.
I am sending/receiving data over a serial line in Linux and I would like to find the delay between characters.
Modbus uses a 3.5 character delay to detect message frame boundaries. If there is more than a 1.5 character delay, the message frame is declared incomplete.
I'm writing a quick program in C which is basically
fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY | O_NONBLOCK);
// setup newtio
....
tcsetattr(fd, TCSANOW, &newtio);
for(;;) {
res = read(fs, buf, 1);
if (res > 0) {
// store time in milliseconds?
//do stuff
}
}
Is there some way of measuring the time here? Or do I need to look at retrieving data from the serial line in a different way?
I've also tried hooking into SIGIO to get a signal whenever there is data but I seem to get data 8 bytes at a time.
(yes, I know there exist some modbus libraries but I want to use this in other applications)
The simple answer is... you cannot (not without writing you own serial driver)!
If you are writing a MODBUS master there is some hope: You can either detect the end of a slave response by waiting any amount of time (provided its longer than 3.5 chars) without receiving anything (select(2) can help you here), or by parsing the response on the fly, as you read it (the second method wastes much less time). You must also be careful to wait at least 3.5 characters-time before staring to transmit a new request, after receiving the response to the previous request. "At least" is operative here! Waiting more doesn't matter. Waiting less does.
If you a writing a MODBUS slave then you' re out of luck. You simply cannot do it reliably from userspace Linux. You have to write you own serial driver.
BTW, this is not Linux's fault. This is due to the unbelievable stupidity of MODBUS's framing method.
MODbus is like a lot of old protocols and really hates modern hardware.
The reason you're getting 8 bytes at a time is :
Your PC has a (at least) 16 byte serial FIFO on receive and transmit, in the hardware. Most are 64byte or bigger.
It is possible to tell the uart device to time out and issue a received interrupt after a number of char times.
The Trigger Level is adjustable, but the low-level driver sets it "smartly". try low-latency mode using setserial)
You can fiddle with the code in the serial driver if you must. Google it (mature content warning) it is not pretty.
so the routine is as pseudocode
int actual=read (packet, timeout of 1.5 chars)
look at actual # of received bytes
if less than a packet, has issues, discard.
not great.
You can't use timeouts. On higher baud rates 3.5 character timeout means a few milliseconds, or even hundreds of microseconds. Such timeouts can't be handled in the Linux user space.
On the client side, it isn't a big deal since Modbus doesn't send asynchronous messages. So it's up to you not to send 2 consecutive messages within 3.5 character timeout.
On the server side, the problem is that if your clients have an extremely short response timeouts and Linux is too busy you can't write a bullet-proof framing solution. There is a chance that read() function will return more than one packet. Here is (a little contrived) example.
Client writes a packet to server. Timeout is let's say 20 ms.
Let's say that Linux is at the moment very busy, so kernel doesn't wake up your thread within next 50 ms.
After 20 ms client detects that it didn't receive any response so it sends another packet to server (maybe resent the previous one).
If Linux wakes up your reading thread after 50 ms, read() function can get 2 packets or even 1 and half depending to how many bytes were received by the serial port driver.
In my implementation I use a simple method that tries to parse bytes on-the-fly - first detecting the function code and then I try to read all remaining bytes for a specific function. If I get one and half packet I parse just the first one and remaining bytes are left in the buffer. If more bytes come within a short timeout I add them and try to parse, otherwise I discard them. It's not a perfect solution (for instance some sub-codes for function 8 doesn't have a fixed size) but since MODBUS RTU doesn't have any STX ETX characters, it's the best one I were able to figure out.
I think you are going about this the wrong way. There is a built in mechanism for ensuring that characters come in all together.
Basically, you are going to want to use ioctl() and set the VMIN and VTIME parameters appropriately. In this case, it seems like you'd want VMIN (minimum number of characters in a packet) to be 0 and VTIME (minimum amount of time allowed between characters to be 15 (they are tenths of seconds).
Some really basic example code:
struct termio t;
t.c_cc[ VMIN ] = 0;
t.c_cc[ VTIME ] = 15;
if (ioctl( fd, TCSETAW, &t ) == -1)
{
printf( msg, "ioctl(set) failed on port %s. Exiting...", yourPort);
exit( 1 );
}
Do this before your open() but before your read(). Here's a couple of links that I've found wildly helpful:
Serial Programming Guide
Understanding VMIN and VMAX
I hope that at least helps/points you in the right direction even if it isn't a perfect answer for your question.