I am creating a software switch, as a school project. It's implemented in C using lpcap and working fine (despite some bugs) on my Ubuntu machine. However I have a Mac and it's not working there as it should.
When frame is captured using pcap_next_ex() number of captured frames is increased. For some reason during first few seconds (5 to 30) it doesn't increment number of frames, like no frames were received, BUT I CAN SEE those frames in Wireshark. How is this possible?
If interested here is my code.
https://github.com/Horkyze/Software-switch
For some reason during first few seconds (5 to 30) it doesn't increment number of frames, like no frames were received,
Or, rather, like no frames were passed from the capture mechanism to libpcap.
Given that you did not set a timeout, the default timeout is used. It happens to be 0; the behavior with a timeout is platform-dependent and undefined and, for systems that use BPF, such as OS X (and *BSD and Solaris 11), that behavior is "don't pass packets from the capture mechanism to userland until there's no room for the next packet in the kernel packet buffer", which means that the delay between the reception of a frame and its delivery to userland could be arbitrarily long.
Apple's pcap_set_timeout() man page is more emphatic about this (and I'm going to change the standard libpcap man page to say the same thing:
The behavior, if the timeout isn't specified, is undefined. We recom-
mend always setting the timeout to a non-zero value.
Given the "switch" in the name of your application, you probably don't want any timeout at all but instead want "immediate mode". In immediate mode, set with pcap_set_immediate_mode() rather than pcap_set_timeout(), packets are delivered to user mode as soon as they arrive.
This will also work on Ubuntu (including immediate mode if it's a new enough version of Ubuntu that it has a version of libpcap with immediate mode). Note that, on a Linux system with a version of the kernel new enough to implement TPACKET_V3 and a version of libpcap new enough to use TPACKET_V3, the behavior can be quite different from versions of Linux where either the kernel or libpcap doesn't do TPACKET_V3, so setting the timeout is a good idea on any OS.
Related
I have an imx8 module running Linux on my PCB and i would like some tips or pointers on how to modify the UART driver to allow me to be able to detect the end of frame very quickly (less than 2ms) from my user space C application. The UART frame does not have any specific ending character or frame length. The standard VTIME of 100ms is much too long
I am reading from a Sim card, i have no control over the data, no control over the size or content of the data. I just need to detect the end of frame very quickly. The frame could be 3 bytes or 500. The SIM card reacts to data that it receives, typically I send it a couple of bytes and then it will respond a couple of ms later with an uninterrupted string of bytes of unknown length. I am using an iMX8MP
I thought about using the IDLE interrupt to detect the frame end. Turn it on when any byte is received and off once the idle interrupt fires. How can I propagate this signal back to user space? Or is there an existing method to do this?
Waiting for an "idle" is a poor way to do this.
Use termios to set raw mode with VTIME of 0 and VMIN of 1. This will allow the userspace app to get control as soon as a single byte arrives. See:
How to read serial with interrupt serial?
How do I use termios.h to configure a serial port to pass raw bytes?
How to open a tty device in noncanonical mode on Linux using .NET Core
But, you need a "protocol" of sorts, so you can know how much to read to get a complete packet. You prefix all data with a struct that has (e.g.) A type and a payload length. Then, you send "payload length" bytes. The receiver gets/reads that fixed length struct and then reads the payload which is "payload length" bytes long. This struct is always sent (in both directions).
See my answer: thread function doesn't terminate until Enter is pressed for a working example.
What you have/need is similar to doing socket programming using a stream socket except that the lower level is the UART rather than an actual socket.
My example code uses sockets, but if you change the low level to open your uart in raw mode (as above), it will be very similar.
UPDATE:
How quickly after the frame finished would i have the data at the application level? When I try to read my random length frames currently reading in 512 byte chunks, it will sometimes read all the frame in one go, other times it reads the frame broken up into chunks. –
Engo
In my link, in the last code block, there is an xrecv function. It shows how to read partial data that comes in chunks.
That is what you'll need to do.
Things missing from your post:
You didn't post which imx8 board/configuration you have. And, which SIM card you have (the protocols are card specific).
And, you didn't post your other code [or any code] that drives the device and illustrates the problem.
How much time must pass without receiving a byte before the [uart] device is "idle"? That is, (e.g.) the device sends 100 bytes and is then finished. How many byte times does one wait before considering the device to be "idle"?
What speed is the UART running at?
A thorough description of the device, its capabilities, and how you intend to use it.
A uart device doesn't have an "idle" interrupt. From some imx8 docs, the DMA device may have an "idle" interrupt and the uart can be driven by the DMA controller.
But, I looked at some of the linux kernel imx8 device drivers, and, AFAICT, the idle interrupt isn't supported.
I need to read everything in one go and get this data within a few hundred microseconds.
Based on the scheduling granularity, it may not be possible to guarantee that a process runs in a given amount of time.
It is possible to help this a bit. You can change the process to use the R/T scheduler (e.g. SCHED_FIFO). Also, you can use sched_setaffinity to lock the process to a given CPU core. There is a corresponding call to lock IRQ interrupts to a given CPU core.
I assume that the SIM card acts like a [passive] device (like a disk). That is, you send it a command, and it sends back a response or does a transfer.
Based on what command you give it, you should know how many bytes it will send back. Or, it should tell you how many optional bytes it will send (similar to the struct in my link).
The method you've described (e.g.) wait for idle, then "race" to get/process the data [for which you don't know the length] is fraught with problems.
Even if you could get it to work, it will be unreliable. At some point, system activity will be just high enough to delay wakeup of your process and you'll miss the window.
If you're reading data, why must you process the data within a fixed period of time (e.g. 100 us)? What happens if you don't? Does the device catch fire?
Without more specific information, there are probably other ways to do this.
I've programmed such systems before that relied on data races. They were unreliable. Either missing data. Or, for some motor control applications, device lockup. The remedy was to redesign things so that there was some positive/definitive way to communicate that was tolerant of delays.
Otherwise, I think you've "fallen in love" with "idle interrupt" idea, making this an XY problem: https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem
Not sure how to word the title, but what I'm trying to do is test my micro controller with my Linux PC to ensure data is correct. After hours of searching, I found out that the stty command can change how data is managed through the serial port and it turned out that by default if xon or xoff characters are received from the port, they don't get displayed. At first I thought my computer was too slow that I was losing characters at 57.6Kbps but that wasn't the case.
Back in the day when I was playing with the serial mouse in QuickBasic for DOS, I could use this command to start the serial port:
OPEN "com1:1200,n,7,1,op0" for binary as #1
So what I want to do now is create something simple in C that would allow me to open up the serial port in the rawest mode possible. I want it so that whatever data I give to it is sent to it unmodified. I also want to receive data unmodified. so if the controller decides to send a character the PC would recognize as a special control code, I still want to see the character, not have the PC go funny just because a character matches a control code.
One idea I thought of is to create a fork to the stty program and use nearly every (50+?) parameters added to the program making the requirement of program stack space a bit high.
Another idea I thought of is to do direct I/O with the port address itself (using inb and outb) but I'm not sure if the kernel would run those commands through anything else before the data reaches the port, but I'd rather use that as a last option in case I ever replace my computer and the serial port value changes (or becomes a serial port made through USB to serial converter hardware).
so rather than inb and outb and those variants (like inw), and without executing stty with specifying 50+ parameters in my program, is there a function in C I can use (without requiring a special library not included with a standard linux distribution) to force the serial port device as a raw device so I can do any I/O on it without the kernel modifying or dropping data?
When using snd_pcm_writei() in non-blocking mode everything works perfect for a while but eventually the audio gets choppy. It sounds like the ring buffer pointers are getting out of sync (ie. sometimes I can tell that the audio is playing out of order). How long it takes for the problem to start it's hardware dependent. On a Gentoo box on real hardware it seldom happens, but on a buildroot system running on QEMU it happens after about 5 minutes. On both cases draining the pcm stream fixes the problem. I have verified that I'm writing the samples correctly by also writting them to a file and playing them with aplay.
Currently I'm setting avail_min to the period size (1024 frames) and calling snd_pcm_wait() before writting chunks of the period size. But I tried a number of different variations (different chunk sizes, checking avail myself and use pthread_cond_timedwait() instead of snd_pcm_wait(), etc). But the only thing that works fine is using blocking mode but I can not do that.
You can see the current source code here: https://bitbucket.org/frodzdev/mediabox/src/5a6471316c7ae481b329e7e0d4af1bb68a32e71d/src/audio.c?at=staging&fileviewer=file-view-default (it needs a little cleanup since I'm trying all kinds of things). The code that does the actual IO starts at line 375.
Edit:
I think I got a solution but I don't understand why it seems to work. It seems that it does not matter if I'm using non-blocking mode, the problem is when I wait to make sure there's room on the buffer (either through snd_pcm_wait(), pthread_cond_timedwait(), or usleep()).
The version that seems to work is here: https://bitbucket.org/frodzdev/mediabox/src/c3eb290087d9bbe0d5f37653a33a1ba88ef0628b/src/audio.c?fileviewer=file-view-default. I switched to blocking mode while still waiting before calling snd_pcm_writei() and it didn't made a difference. Then I added the call to snd_pcm_avail() before calling snd_pcm_status() on avbox_audiostream_gettime(). This function is called constantly by another thread to get the stream clock and it only uses snd_pcm_status() to get the timestamps. Now it seems to work (at least it is a lot less probable to happen) but I don't understand exactly why. I understand that snd_pcm_avail() will synchronize the pointers with the kernel but I don't really understand when it needs to be called and the difference between snd_pcm_state() et al and snd_pcm_status(). Does snd_pcm_status() also synchronize anything? It seems not because sometimes snd_pcm_status_get_state() will return RUNNING when snd_pcm_avail() returns -EPIPE. The ALSA documentation is really vague. Perhaps understanding these things will help me understand my problem?
Now, when I said that it seems to be working I mean that I cannot reproduce it on real hardware. It still happens on QEMU though way less often. But considering that on the next commit I switched to blocking mode without waiting (which I've used in the past and never had a problem with on real hardware) and it still happens in QEMU and also the fact that this is a common issue with QEMU I'm starting to think that I may have fixed the issue on my end and now it's just a QEMU problem. Is there any way to determine if the problem is a bug on my end that is easier to trigger on the emulator or if it's just an emulator problem?
Edit: I realize that I should fill the buffer before waiting but at this point my concern is not to prevent underruns but to make sure that my code can handle them when they happen. Besides the buffer is filling up after a few iterations. I confirmed this by outputing avail, buffer_size, etc before writing each packet and the numbers I get don't make perfect sense, they show an error of 1 or 2 periods about every 8th period. Also (and this is the main problem) I'm not detecting any underruns, the audio get choppy but all writes succeed. In fact, if the problem start happening and I trigger an underrun by overloading the CPU it will correct itself when the pcm is reset.
In line 505: You're using time as argument to malloc.
In line 568: Weren't you playing audio? In this case you should do wait only after you wrote the frames. Let's think ...
Audio device generates an interrupt when it terminates to process a period.
| period A | period B |
^ ^
irq irq
Before you start the pcm, audio device doesn't generate any interrupt. Notice here that you're waiting and you haven't started the pcm yet. You only starts it when you call snd_pcm_writei().
When you wait for audio data you'll be awake only when the current period has been fully processed -- in your first wait the first period wasn't even written -- so in a comfortable situation you should write the whole buffer, wait for the first interrupt, and then write the just processed period, and on and on.
Initially, buffer is empty:
| | |
write():
|############|############|
wait():
..............
When we wake up:
| |############|
write():
|############|############|
I found the problem is you're writing audio just before it be played, then sometimes it may arrive delayed in the buffer.
I'm writing a sniffer using libpcap. My problem is that there's a 7-10 second delay between calling pcap_loop() or pcap_next() and actually getting a packet(the callback function being called). However, if I use wireshark with the same filter on the same device, there is no such delay after I hit the "start" button. Why is there a delay in my program and is there a way to fix that?
I'm working on atheros wifi chips. The device is set to monitor mode using
airmon-ng start wlan0
I'm sure there're plenty of traffic to listen to, for I can see the packages in wireshark.
Thank you.
I'm using 10000
The to_ms argument to pcap_open_live() and pcap_set_timeout() is in milliseconds.
10000 milliseconds is 10 seconds.
Try using 1000, which is the value tcpdump uses - that'll reduce the delay to 1 second - or using 100, which is the value Wireshark uses - that'll reduce the delay to 1/10 second.
I read on a tutorial about this field: " on at least some platforms, this means that you may wait until a sufficient number of packets arrive before seeing any packets, so you should use a non-zero timeout"
The tutorial in question is the tcpdump.org "How to use libpcap" tutorial, and the passage in question was added in this CVS commit:
revision 1.8
date: 2005/08/27 23:58:39; author: guy; state: Exp; lines: +34 -31
Use a non-zero timeout in pcap_open_live(), so you don't wait for a
bufferful of packets before any are processed.
Correctly explain the difference between pcap_loop() and
pcap_dispatch().
In sniffex.c, don't print the payload if there isn't any.
so I'm familiar with it. :-)
I'd have to spend some time looking at the Linux kernel code (again) to see what effect a timeout value of 0 would have on newer kernels. However, when writing code that uses libpcap/WinPcap to do live captures, you should always act as if you're writing code for such a platform; your code will then be more portable to other platforms and will not break if the behavior of a zero timeout changes.
I'm using 3.1 Sarge, kernel 2.4.26 on a TS-7400 board running ARM 9 architecture.
I am using the POSIX library terminos and fcntl.
I am writing a program to communicate between 2 embedded devices over serial. The program uses the POSIX time out flag VTIME and works successfully Ubuntu 10.1 but it does not time out on the board. I need the program to try resending a command if there is no response after a certain time. I know the board is transmitting OK the first time but then the program locks up waiting for a response. I am running the serial in delay mode so it will wait in read() until at least 1 byte is received or .1 secs have passed as defined by VTIME.
What is the problem or if VTIME simply does not work in this kernel what is another way to accomplish this?
Investigate the select() system call. This will let you execute a read when there is something to actually read, instead of waiting for .1 seconds, hoping something will show up. If this is supposed to be a straight port of your code then this may not be and appropriate thing to do.
It is an alternative....