I'm fairly new to VxWorks OS and hence wouldn't mind explanations in case I differ in my understanding of things under the hood when they differ from more traditional OSes like Linux and the likes. With that out of the way, let me begin my actual question.
I am trying to create a loop-back test for testing changes I made to the serial UART driver on the board. Since I do not want to use a cross cable to actually short two UART ports externally, I've connected both of those ports to my dev machine. One configured as an output port from the dev machine's perspective (and consequently as an Input port on the board) and the other an input port (an output port on the board). I am actually doing the loopback using a shared memory buffer which I am protecting using a mutex. So there are two threads on the board, one of which reads from the input port, copies data to the shared memory and the other reads from the memory and sends it over the output port.
And I am using regular open, read and write calls in my VxWorks application (by the way I think it is part of the application code as I call the functions from usrAppInit.c not withstanding the fact that I can even call driver routines from here! (Is it because of a flat memory model vis-a-vis Linux?? Anyhow).
Now I these ports on VxWorks have been opened in a non blocking mode and here's the code snippet which configures one of the ports:
if( (fdIn = open(portstrIn, O_RDONLY | O_NOCTTY, 0) ) == ERROR) {
return 1;
}
if(((status = ioctl(fdIn, FIOSETOPTIONS, OPT_RAW))) == ERROR)
{
return 1;
}
/* set the baud rate to 115200
*/
if((status = ioctl(fdIn, FIOBAUDRATE, 115200)) == ERROR)
{
return 1;
}
/* set the HW options
*/
if((status = ioctl(fdIn, SIO_HW_OPTS_SET, (CS8 | 0 | 0))) == ERROR)
{
return 1;
}
And similarly the output port is also configured. These two are part of two separate tasks spawned using taskSpawn and have the same priority of 100. However what I am annoyed by, is that when I write to the in port from my dev machine (using a python script), the read call on the board get's sort of staggered (I wonder if that's the right way to refer to it). It is most likely due to the short availability of hardware buffer space on the UART input buffer (or some such). This is usually not much of a problem if that is all I am doing.
However, as explained earlier, I am trying to copy the whole received character stream into a common memory buffer (guarded by a mutex of course) which is then read by another task and then re-transmitted over another serial port (sort of an in memory loopback if you will)
In lieu of that aforementioned staggering of the read calls, I thought of holding the mutex as long as there are characters to be read from the Inport and once there are no chars to be read, release the mutex and since this is VxWorks, do an explicit taskDelay(0) to schedule the next ready task (my other task). However since this is a blocking read, I am (as expected) stuck on the read call due to which my other thread never gets a chance to execute.
I did think of checking if the buffer was full and then doing the explicit task switch however if any of you have a better idea, I'm all ears.
Also just to see how this staggered read thing works from the perspective of the kernel, I timed it using a time(NULL) call just before and right after the read. So surprisingly, the very first chunk shows up a number, every other chunk after that (if it's a part of the same data block coming from the outside) shows 0. Could anyone explain that as well?
Keen to hear
I don't have 50 rep points for commenting, but without a loopback cable attached, the only way you can arrive at testing serial loopback behavior is to switch the uart into loopback mode. This often means making changes to the specific hardware part driver.
Related
I have an imx8 module running Linux on my PCB and i would like some tips or pointers on how to modify the UART driver to allow me to be able to detect the end of frame very quickly (less than 2ms) from my user space C application. The UART frame does not have any specific ending character or frame length. The standard VTIME of 100ms is much too long
I am reading from a Sim card, i have no control over the data, no control over the size or content of the data. I just need to detect the end of frame very quickly. The frame could be 3 bytes or 500. The SIM card reacts to data that it receives, typically I send it a couple of bytes and then it will respond a couple of ms later with an uninterrupted string of bytes of unknown length. I am using an iMX8MP
I thought about using the IDLE interrupt to detect the frame end. Turn it on when any byte is received and off once the idle interrupt fires. How can I propagate this signal back to user space? Or is there an existing method to do this?
Waiting for an "idle" is a poor way to do this.
Use termios to set raw mode with VTIME of 0 and VMIN of 1. This will allow the userspace app to get control as soon as a single byte arrives. See:
How to read serial with interrupt serial?
How do I use termios.h to configure a serial port to pass raw bytes?
How to open a tty device in noncanonical mode on Linux using .NET Core
But, you need a "protocol" of sorts, so you can know how much to read to get a complete packet. You prefix all data with a struct that has (e.g.) A type and a payload length. Then, you send "payload length" bytes. The receiver gets/reads that fixed length struct and then reads the payload which is "payload length" bytes long. This struct is always sent (in both directions).
See my answer: thread function doesn't terminate until Enter is pressed for a working example.
What you have/need is similar to doing socket programming using a stream socket except that the lower level is the UART rather than an actual socket.
My example code uses sockets, but if you change the low level to open your uart in raw mode (as above), it will be very similar.
UPDATE:
How quickly after the frame finished would i have the data at the application level? When I try to read my random length frames currently reading in 512 byte chunks, it will sometimes read all the frame in one go, other times it reads the frame broken up into chunks. –
Engo
In my link, in the last code block, there is an xrecv function. It shows how to read partial data that comes in chunks.
That is what you'll need to do.
Things missing from your post:
You didn't post which imx8 board/configuration you have. And, which SIM card you have (the protocols are card specific).
And, you didn't post your other code [or any code] that drives the device and illustrates the problem.
How much time must pass without receiving a byte before the [uart] device is "idle"? That is, (e.g.) the device sends 100 bytes and is then finished. How many byte times does one wait before considering the device to be "idle"?
What speed is the UART running at?
A thorough description of the device, its capabilities, and how you intend to use it.
A uart device doesn't have an "idle" interrupt. From some imx8 docs, the DMA device may have an "idle" interrupt and the uart can be driven by the DMA controller.
But, I looked at some of the linux kernel imx8 device drivers, and, AFAICT, the idle interrupt isn't supported.
I need to read everything in one go and get this data within a few hundred microseconds.
Based on the scheduling granularity, it may not be possible to guarantee that a process runs in a given amount of time.
It is possible to help this a bit. You can change the process to use the R/T scheduler (e.g. SCHED_FIFO). Also, you can use sched_setaffinity to lock the process to a given CPU core. There is a corresponding call to lock IRQ interrupts to a given CPU core.
I assume that the SIM card acts like a [passive] device (like a disk). That is, you send it a command, and it sends back a response or does a transfer.
Based on what command you give it, you should know how many bytes it will send back. Or, it should tell you how many optional bytes it will send (similar to the struct in my link).
The method you've described (e.g.) wait for idle, then "race" to get/process the data [for which you don't know the length] is fraught with problems.
Even if you could get it to work, it will be unreliable. At some point, system activity will be just high enough to delay wakeup of your process and you'll miss the window.
If you're reading data, why must you process the data within a fixed period of time (e.g. 100 us)? What happens if you don't? Does the device catch fire?
Without more specific information, there are probably other ways to do this.
I've programmed such systems before that relied on data races. They were unreliable. Either missing data. Or, for some motor control applications, device lockup. The remedy was to redesign things so that there was some positive/definitive way to communicate that was tolerant of delays.
Otherwise, I think you've "fallen in love" with "idle interrupt" idea, making this an XY problem: https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem
Not sure how to word the title, but what I'm trying to do is test my micro controller with my Linux PC to ensure data is correct. After hours of searching, I found out that the stty command can change how data is managed through the serial port and it turned out that by default if xon or xoff characters are received from the port, they don't get displayed. At first I thought my computer was too slow that I was losing characters at 57.6Kbps but that wasn't the case.
Back in the day when I was playing with the serial mouse in QuickBasic for DOS, I could use this command to start the serial port:
OPEN "com1:1200,n,7,1,op0" for binary as #1
So what I want to do now is create something simple in C that would allow me to open up the serial port in the rawest mode possible. I want it so that whatever data I give to it is sent to it unmodified. I also want to receive data unmodified. so if the controller decides to send a character the PC would recognize as a special control code, I still want to see the character, not have the PC go funny just because a character matches a control code.
One idea I thought of is to create a fork to the stty program and use nearly every (50+?) parameters added to the program making the requirement of program stack space a bit high.
Another idea I thought of is to do direct I/O with the port address itself (using inb and outb) but I'm not sure if the kernel would run those commands through anything else before the data reaches the port, but I'd rather use that as a last option in case I ever replace my computer and the serial port value changes (or becomes a serial port made through USB to serial converter hardware).
so rather than inb and outb and those variants (like inw), and without executing stty with specifying 50+ parameters in my program, is there a function in C I can use (without requiring a special library not included with a standard linux distribution) to force the serial port device as a raw device so I can do any I/O on it without the kernel modifying or dropping data?
I am writing a program that interfaces with a particular serial device. The serial device has two channels, and a hardware rx and tx buffer for each channel. Basically, at any given time , you can read/write to either channel on the device.
I am trying to read data from a channel, validate it (and perhaps use some of the data), and then transmit it. Reads are accomplished with iotctl calls to the device, while writes are accomplished with a call to the write() system call.
The main issue I have is with data throughput. I'd like to have an individual thread handle reading and writing for each channel (i.e., a read thread and write thread for each of the two channels). However, I have hit a snag. Everything on the device, from Linux's perspective is accessed via one single device, and I'm not sure that Linux notes that the device has multiple channels.
As a result, currently I open a single file descriptor to the device and perform my reads and writes serially. I'd like to go to the threaded approach, but I'm wondering if concurrent ioctl() and write() calls would cause issues. I understand that read() and write() and not thread safe, but I was wondering if there's any way around that (perhaps calling open() twice, one with read privileges, one with write privileges).
Thanks for your help. Also, I want to avoid having to write my own driver, but that may be an inevitable conclusion...
Also, as a side note, I'm particularly concerned that the device has extremely small hardware buffers. Is there any way to determine how much space the OS uses for a software buffer for data? That is, can I determine whether or not the OS has it's own buffer that is used to prevent overflow of the hardware buffer? The device in question is an I2C UART Bridge.
You can use semaphore to make a mutual exclusion between read/write thread
sem_t sync_rw;
/*init semaphore */
err=sem_init(&sync_rw,0,1); /* shared between thread and initialized with 1 */
if( err != 0 )
{
perror("cannot init semaphore \n");
return -1;
}
in thread write function you do this :
sem_wait(&sync_rw);
write(...)
sem_post(&sync_rw);
same for thread reader :
sem_wait(&sync_rw);
iotctl(...)
sem_post(&sync_rw);
finally :
sem_destroy(&sync_rw);
I'm writing a program in linux to interface, through serial, with a piece of hardware. The device sends packets of approximately 30-40 bytes at about 10Hz. This software module will interface with others and communicate via IPC so it must perform a specific IPC sleep to allow it to receive messages that it's subscribed to when it isn't doing anything useful.
Currently my code looks something like:
while(1){
IPC_sleep(some_time);
read_serial();
process_serial_data();
}
The problem with this is that sometimes the read will be performed while only a fraction of the next packet is available at the serial port, which means that it isn't all read until next time around the loop. For the specific application it is preferable that the data is read as soon as it's available, and that the program doesn't block while reading.
What's the best solution to this problem?
The best solution is not to sleep ! What I mean is a good solution is probably to mix
the IPC event and the serial event. select is a good tool to do this. Then you have to find and IPC mechanism that is select compatible.
socket based IPC is select() able
pipe based IPC is select() able
posix message queue are also selectable
And then your loop looks like this
while(1) {
select(serial_fd | ipc_fd); //of course this is pseudo code
if(FD_ISSET(fd_set, serial_fd)) {
parse_serial(serial_fd, serial_context);
if(complete_serial_message)
process_serial_data(serial_context)
}
if(FD_ISSET(ipc_fd)) {
do_ipc();
}
}
read_serial is replaced with parse_serial, because if you spend all your time waiting for complete serial packet, then all the benefit of the select is lost. But from your question, it seems you are already doing that, since you mention getting serial data in two different loop.
With the proposed architecture you have good reactivity on both the IPC and the serial side. You read serial data as soon as they are available, but without stopping to process IPC.
Of course it assumes you can change the IPC mechanism. If you can't, perhaps you can make a "bridge process" that interface on one side with whatever IPC you are stuck with, and on the other side uses a select()able IPC to communicate with your serial code.
Store away what you got so far of the message in a buffer of some sort.
If you don't want to block while waiting for new data, use something like select() on the serial port to check that more data is available. If not, you can continue doing some processing or whatever needs to be done instead of blocking until there is data to fetch.
When the rest of the data arrives, add to the buffer and check if there is enough to comprise a complete message. If there is, process it and remove it from the buffer.
You must cache enough of a message to know whether or not it is a complete message or if you will have a complete valid message.
If it is not valid or won't be in an acceptable timeframe, then you toss it. Otherwise, you keep it and process it.
This is typically called implementing a parser for the device's protocol.
This is the algorithm (blocking) that is needed:
while(! complete_packet(p) && time_taken < timeout)
{
p += reading_device.read(); //only blocks for t << 1sec.
time_taken.update();
}
//now you have a complete packet or a timeout.
You can intersperse a callback if you like, or inject relevant portions in your processing loops.
I am sending/receiving data over a serial line in Linux and I would like to find the delay between characters.
Modbus uses a 3.5 character delay to detect message frame boundaries. If there is more than a 1.5 character delay, the message frame is declared incomplete.
I'm writing a quick program in C which is basically
fd = open(MODEMDEVICE, O_RDWR | O_NOCTTY | O_NONBLOCK);
// setup newtio
....
tcsetattr(fd, TCSANOW, &newtio);
for(;;) {
res = read(fs, buf, 1);
if (res > 0) {
// store time in milliseconds?
//do stuff
}
}
Is there some way of measuring the time here? Or do I need to look at retrieving data from the serial line in a different way?
I've also tried hooking into SIGIO to get a signal whenever there is data but I seem to get data 8 bytes at a time.
(yes, I know there exist some modbus libraries but I want to use this in other applications)
The simple answer is... you cannot (not without writing you own serial driver)!
If you are writing a MODBUS master there is some hope: You can either detect the end of a slave response by waiting any amount of time (provided its longer than 3.5 chars) without receiving anything (select(2) can help you here), or by parsing the response on the fly, as you read it (the second method wastes much less time). You must also be careful to wait at least 3.5 characters-time before staring to transmit a new request, after receiving the response to the previous request. "At least" is operative here! Waiting more doesn't matter. Waiting less does.
If you a writing a MODBUS slave then you' re out of luck. You simply cannot do it reliably from userspace Linux. You have to write you own serial driver.
BTW, this is not Linux's fault. This is due to the unbelievable stupidity of MODBUS's framing method.
MODbus is like a lot of old protocols and really hates modern hardware.
The reason you're getting 8 bytes at a time is :
Your PC has a (at least) 16 byte serial FIFO on receive and transmit, in the hardware. Most are 64byte or bigger.
It is possible to tell the uart device to time out and issue a received interrupt after a number of char times.
The Trigger Level is adjustable, but the low-level driver sets it "smartly". try low-latency mode using setserial)
You can fiddle with the code in the serial driver if you must. Google it (mature content warning) it is not pretty.
so the routine is as pseudocode
int actual=read (packet, timeout of 1.5 chars)
look at actual # of received bytes
if less than a packet, has issues, discard.
not great.
You can't use timeouts. On higher baud rates 3.5 character timeout means a few milliseconds, or even hundreds of microseconds. Such timeouts can't be handled in the Linux user space.
On the client side, it isn't a big deal since Modbus doesn't send asynchronous messages. So it's up to you not to send 2 consecutive messages within 3.5 character timeout.
On the server side, the problem is that if your clients have an extremely short response timeouts and Linux is too busy you can't write a bullet-proof framing solution. There is a chance that read() function will return more than one packet. Here is (a little contrived) example.
Client writes a packet to server. Timeout is let's say 20 ms.
Let's say that Linux is at the moment very busy, so kernel doesn't wake up your thread within next 50 ms.
After 20 ms client detects that it didn't receive any response so it sends another packet to server (maybe resent the previous one).
If Linux wakes up your reading thread after 50 ms, read() function can get 2 packets or even 1 and half depending to how many bytes were received by the serial port driver.
In my implementation I use a simple method that tries to parse bytes on-the-fly - first detecting the function code and then I try to read all remaining bytes for a specific function. If I get one and half packet I parse just the first one and remaining bytes are left in the buffer. If more bytes come within a short timeout I add them and try to parse, otherwise I discard them. It's not a perfect solution (for instance some sub-codes for function 8 doesn't have a fixed size) but since MODBUS RTU doesn't have any STX ETX characters, it's the best one I were able to figure out.
I think you are going about this the wrong way. There is a built in mechanism for ensuring that characters come in all together.
Basically, you are going to want to use ioctl() and set the VMIN and VTIME parameters appropriately. In this case, it seems like you'd want VMIN (minimum number of characters in a packet) to be 0 and VTIME (minimum amount of time allowed between characters to be 15 (they are tenths of seconds).
Some really basic example code:
struct termio t;
t.c_cc[ VMIN ] = 0;
t.c_cc[ VTIME ] = 15;
if (ioctl( fd, TCSETAW, &t ) == -1)
{
printf( msg, "ioctl(set) failed on port %s. Exiting...", yourPort);
exit( 1 );
}
Do this before your open() but before your read(). Here's a couple of links that I've found wildly helpful:
Serial Programming Guide
Understanding VMIN and VMAX
I hope that at least helps/points you in the right direction even if it isn't a perfect answer for your question.