Serial communication buffer data out of order - c

I'm trying to implement a simple SLAM project with Arduino and C on Linux Mint 15.
The Arduino project is sending data to notebook via bluetooth (serial). The data is read by a C program. In Arduino serial, the data is shown correctly, but in the notebook, the received data is wrong. (In image, white is Arduino data. The terminal shows the 'received' data.)
I'm sending d080x096y099z035 (for example) and receiving 99z0356y0999z035 (out of order?).
So, I have some questions:
What can I do to make the read()command in C, read the data in the correct order and length? (order: d000x000y000z000, length = 16)
In the Arduino sending function, are there length differences using Serial.print(char buffer[]) and Serial.println(char buffer[])? (Like adding a '\n' or something else at the end of buffer?)
Should I use the delay() function in the Arduino code or in the C code?
In Arduino:
...
int buffer_size = 17;
char buffer[17];
//void setup()
void loop(){
//create the string resp = "d000x111y222z333"
...
resp.toCharArray(buffer, buffersize);
bluetooth.print(buffer);
delay(200);
}
In C program:
...
int fd = open("/dev/rfcomm4", O_RDONLY | O_NOCTTY | O_NDELAY);
printf("fd code %d\n", fd);
if (fd == -1)
{
gchar *msg = "open_port: Unable to open /dev/rfcomm4";
gtk_label_set_text(GTK_LABEL(label), msg);
perror("error: ");
}
char buffer[17];
int n;
printf("entering in loop...\n");
while (1)
{
n = read(fd, buffer, sizeof(buffer));
printf("%s\n", buffer);
}

Sorry I'm not an expert but just a few ideas you might check concering your questions:
to 1) I guess it might be a problem with encoding, as Python AFAIK expects files to be unicode. So try open (.... ,encoding='ascii') or whatever encoding you use
Please also pay attention that you might block the GTK mainthread, that causes heavy delays in your UI. So I recommend creating a own thread for reading the serial port and filling an internal buffer, that get's rendered by the GTK mainthread, if you send an update request:
http://www.pardon-sleeuwaegen.be/antoon/python/page0.html

Related

ESP32 Bluetooth Connection disconnects when finished with reading buffer

I'm programming an ESP32 to accept Bluetooth commands and send Bluetooth Data back to my phone using the Serial profile. For this I'm using the Arduino Espressif Bluetooth Serial Library. Whenever I send something to the ESP32 it processes it and then suddenly closes the Bluetooth Connection.
Up to know I have already tried various delays because I thought that maybe the processor was not keeping up with other stuff due to which it crashed.
However when monitoring using the Serial Connection via USB it still keeps sending status updates.
Other than that I could not really find a solution (also on the Internet).
As I'm pretty much a beginner I did not want to try and build my own Serial Bluetooth Library.
The ESP does not crash when sending it the data. It also keeps processing the data which was sent. I can see that as it sends the chars I sent it via Bluetooth using the Serial interface after having collected them.
The Connection cannot be rebuild after this incident, no matter how long I wait.
My Main function, containing the function call as well as a buffer to write the result to as I thought that maybe I was misusing that.
void loop() {
if (ESP_BT.available() > 0)
{
char *buffer = (char*) malloc(InputSize);
getCurrentMessage(ESP_BT, buffer, InputSize);
Serial.println(buffer);
strncpy(currentMessage, buffer, InputSize);
free(buffer);
}
if (millis() %2000 == 0){
Serial.println("Debug");
delay(1);
}
}
The function which gets called which should read the inputBuffer of the BluetoothSerial into my Buffer.
void getCurrentMessage(BluetoothSerial ESP_BT, char* receivedChars, int InputSize)
{
Serial.println("DEBUG: getCurrentMessageInit");
static byte ndx = 0;
char rc;
while (ESP_BT.available() > 0){
ESP_BT.println("DEBUG: Message Available");
Serial.println("DEBUG: Message Available");
rc = ESP_BT.read();
receivedChars[ndx] = rc;
ndx++;
delay(100);
if (ndx >= InputSize){
while(ESP_BT.available() > 0){
ESP_BT.read();
}
}
}
}
I would expect the Bluetooth Connection to keep working. This it does not do.
I also got the error Code "queue.c:1442 (xQueueGenericReceive)- assert failed!" When not using the delays and the ESP then rebooted.
This it does not do after I included the delays.
The problem was that I did not call my Bluetooth Object by reference.
Instead of giving my function the Bluetooth Object I should point to it:
void getCurrentMessage(BluetoothSerial* ESP_BT, char* receivedChars, int InputSize)
{
Serial.println("DEBUG: getCurrentMessageInit");
static byte ndx = 0;
char rc;
while (ESP_BT->available() > 0){
ESP_BT->println("DEBUG: Message Available");
Serial.println("DEBUG: Message Available");
rc = ESP_BT.read();
receivedChars[ndx] = rc;
ndx++;
delay(100);
if (ndx >= InputSize){
while(ESP_BT->available() > 0){
ESP_BT->read();
}
}
}
}

configfs USB gadget write() call hangs

I'm trying to emulate a HID device (a gamepad for PS3) from a Raspberry Pi Zero using configfs with kernel v4.19.40 (pretty much the same process as described here). Following the instructions in gadget_hid.txt, I was able to get a working prototype that is able to send button presses from the Pi Zero to a USB host (a PS3). However, when some state changes on the USB host (i.e. a certain game is started), the program freezes. strace says that it is stuck on a call to write() to the configfs HID file (/dev/hidgX, in my case /dev/hidg0). How could the USB host block the device from sending data? If the USB host is not ready to receive, won't it just ignore the data sent?
In debugging I found this patch which seemed to be related but I saw no change after applying it.
Here is a small code sample (the full version can be found in gadget_hid.txt) that produces the same problem:
int main(int argc, const char *argv[]) {
int fd = 0;
char report[8];
int to_send = 8;
fd_set rfds;
if ((fd = open("/dev/hidg0", O_RDWR, 0666)) == -1) {
perror("/dev/hidg0");
return 3;
}
while (1) {
FD_ZERO(&rfds);
FD_SET(fd, &rfds);
// Generate some data to send
fill_report(&report);
// Gets stuck here
write(fd, report, to_send);
}
close(fd);
return 0;
}

C program keeps sending same data on serial port

I'm working on a SOM board running Linux embedded for ARM, and I'm developing a C program to communicate with an external device through a serial (RS232) port. I am experiencing a strange behavior though. I'm also using another serial port of the board to communicate with the linux running on the board.
The software has a simple structure: is a text-only console-like program, with this as main menu:
Possible commands:
1 - 4: Select serial device (pump should be on 1)
m - pump op. mode configuration
r - reads from the serial device
w - writes to the serial device
>>>>>>>>>>>>>>Current device is /dev/ttymxc1
>>>>>>>>>>>>>>Enter input (q quits):
and a secondary menu (opened by the "m" option above)
SPEED:
r - rpm (sends 1M<CR>) //<CR> stands for carriage return
f - flow rate (sends 1N<CR>)
QUANTITY:
v - volume (sends 1H<CR>)
t - time (sends 1O<CR>)
DIRECTION:
c - clockwise (sends 1T<CR>)
a - c-clockwise (sends 1K<CR>)
>>>>>>>>>>>>>>Enter input (q quits):
Communication using the main menu options "r" and "w" works fine (thus removing any doubt I may have regarding serial settings like baud rate): "w" invokes a routine ("serial_write" below) that sends a single character input by the user, while "r" returns the data read as soon it arrives (using "serial_read" below). The character I send arrives correctly, and the answer is shown correctly on the console, no matter the times I repeat the "w" and "r" cycle.
The options in the secondary menu should behave in the same way: they simply invoke a routine ("sendSimpleCmd" below) that invokes "serial_write" with a costant char as argument (different for each option), and then it invokes "serial_read".
The problem is that this only works for the first option selected: after that, the program keeps sending data linked to the first option selected, no matter the option I choose. It keeps doing this until I go back to the main menu, then choose again "m": at this point the data sent is the one I expect, but the subsequent choices will be ignored until I go back to the main menu (or close the software, if that matters).
The strangest thing is that I receive the expected data on the same serial I'm using to communicate with board while on the "right" serial port I keep getting the first message. This is the text pasted from the console when I choose "a" as second option, after having chosen "f" as first option (comments added by me):
SPEED:
r - rpm (sends 1M<CR>)
f - flow rate (sends 1N<CR>)
QUANTITY:
v - volume (sends 1H<CR>)
t - time (sends 1O<CR>)
DIRECTION:
c - clockwise (sends 1T<CR>)
a - c-clockwise (sends 1K<CR>)
>>>>>>>>>>>>>>Enter input (q quits):
a //second option
1Knding 1M //mixup of data
wrote 4 characters on fs 4
serial_read: *
The mixup is made by the software output ("Sending data 1M") and the data that should be sent after choosing option "a" (1K). Since on the "right" port I get the same message over and over, while on the "wrong" port I get the right message, it seems that somehow the software autonomously changes port.
The question is:
Could this behavior be caused by my coding, or is bound to something else, like some kernel configuration? If more information is needed, just ask.
Thank you in advance
Serial_write
void serial_write(char text[], int length){
if (selectedDevice == 0){
printf("Select device first!\r\n");
return;
}
int n;
length = length +1 + 2;
char toBeSent[length];
strcat(toBeSent, PUMP_CMD_MSG_START); //header, "1"
strcat(toBeSent, text);
strcat(toBeSent, PUMP_CMD_MSG_END); //footer, "<CR>"
printf("Sending %s\r\n", toBeSent);
n = write (fd, toBeSent, length);
if (n<0){
printf("writing failed on /dev/ttymxc%i\r\n", selectedDevice);
} else {
printf("wrote %i characters on fs %i\r\n", n, fd);
}
}
Serial_read
int serial_read(char *buffer, int size){
int bytes = 0;
int n;
int i = 0;
char tmp_buffer[size];
while(1){
ioctl(fd, FIONREAD, &bytes);
if (bytes > 0){
break;
}
i++;
if(i> 1000){
printf("FIONREAD tries exceeded 1000, aborting read\r\n");
return;
}
usleep(1000);
}
n=read(fd, tmp_buffer, sizeof(tmp_buffer));
for(i=0;i<n;i++) {
buffer[i]=tmp_buffer[i];
}
printf("serial_read: %s\r\n", buffer);
return 0;
}
sendSimpleCmd
void sendSimpleCmd(char text[]){
int bufSize= 20;
char answer[bufSize];
serial_write(text,1);
if (serial_read(answer, bufSize) == 0) {
printf("Ricevuto da pompa \"%s\":", answer);
//handling of possible answers, doesn't do anything relevant since it always receives "*" as answer
if (strcmp(answer, PUMP_ANS_OK) == 0){ //PUMP_ANS_OK is "*"
printf("ok!\r\n");
} else if (strcmp(answer, PUMP_ANS_NOK) == 0){
printf("errore!\r\n");
} else {
printf("sconosciuto!\r\n");
}
} else {
// printf("read failed\r\n");
}
}
You should be initializing toBeSent's contents before using strcat.
Your compiler might be saving you by initializing the array with 0s rather than garbage, but if not, it could be causing buffer overflows. Theres no code protecting against such, so this could be the cause of unexpected program behavior. Without seeing the rest of your code and knowing some other details, it will be difficult to know what exactly the issue is. If this serves as an example of the rest of the code, then the solution is likely to revise the code to fix these issues.
Consider using safe string functions to help prevent buffer overflows.

(C Socket Programming) Seperate send() calls from server ending up in same client recv() buffer

I was wondering if anyone could shed any light as to why two seperate send() calls would end up in the same recv() buffer using the loopback address for testing yet once switched to two remote machines they would require two recv() calls instead? I have been looking at the wireshark captures yet cant seem to make any sense as to why this would be occuring. Perhaps someone could critique my code and tell me where im going wrong. The two incoming messages from the server is of an undetermined length to the client. By the way i'm using BSD sockets using C in Ubuntu.
In the example shown below im parsing the entire buffer to extract the two seperate messages from it which i'll admit isn't an ideal approach.
-------SERVER SIDE--------
// Send greeting string and receive again until end of stream
ssize_t numBytesSent = send(clntSocket, greeting, greetingStringLen, 0);
if (numBytesSent < 0)
DieWithSystemMessage("send() failed");
//-----------------------------Generate "RANDOM" Message -----------------------
srand(time(NULL)); //seed random number from system clock
size_t randomStringLen = rand() % (RANDOMMSGSIZE-3); //generates random num
// betweeen 0 and 296
char randomMsg [RANDOMMSGSIZE] = "";
// declare and initialize allowable characteer set for the
const char charSet[] = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
if (randomStringLen) {
--randomStringLen;
for (size_t i = 0; i < randomStringLen; i++) {
int p = rand() % (int) (sizeof charSet - 1);
randomMsg[i] = charSet[p];
}
randomStringLen = strlen(randomMsg);
printf("Random String Size Before newline: %d\n", (int)randomStringLen);
strcat(randomMsg,"\r\n");
}
randomStringLen = strlen(randomMsg);
printf("Random String: %s\n", randomMsg);
//-----------------------------Send "RANDOM" Message ---------------------------
// Send greeting string and receive again until end of stream
numBytesSent = send(clntSocket, randomMsg, randomStringLen, 0);
if (numBytesSent < 0)
DieWithSystemMessage("send() failed");
//------------------------------------------------------------------------------
------CLIENT SIDE-------
//----------------------------- Receive Server Greeting ---------------------------
char buffer[BUFSIZE] = ""; // I/O buffer
// Receive up to the buffer size (minus 1 to leave space for
// a null terminator) bytes from the sender
ssize_t numBytesRcvd = recv(sock, buffer, BUFSIZE - 1, 0);
if (numBytesRcvd < 0)
DieWithSystemMessage("recv() failed");
buffer[numBytesRcvd] = '\0'; //terminate the string after calling recv()
printf("Buffer contains: %s\n",buffer); // Print the buffer
//printf("numBytesRecv: %d\n",(int)numBytesRcvd); // Print the buffer
//------------------------ Extracts the random message from buffer ---------------------------
char *randomMsg = strstr(buffer, "\r\n"); // searches from first occurance of substring
char randomMessage [BUFSIZE] = "";
strcat(randomMessage, randomMsg+2);
int randomStringLen = strlen(randomMessage)-2;
printf("Random Message: %s\n",randomMessage); // Print the buffer
char byteSize [10];
sprintf(byteSize,"%d", randomStringLen);
printf("ByteSize = %s\n",byteSize);
//----------------------- Send the number for random bytes recieved -------------------------
size_t byteStringLen = strlen(byteSize); // Determine input length
numBytes = send(sock, byteSize, byteStringLen, 0);
if (numBytes < 0)
DieWithSystemMessage("send() failed");
else if (numBytes != byteStringLen)
DieWithUserMessage("send()", "sent unexpected number of bytes");
shutdown(sock,SHUT_WR); // further sends are disallowed yet recieves are still possible
//----------------------------------- Recieve Cookie ----------------------------------------
On Unix systems recv and send are just special cases of the read and write that accepts additional flags. (Windows also emulates this with Winsock).
You shouldn't assume that one recv corresponds to one send because that's generally isn't true (just like you can read a file in multiple parts, even if it was written in a single write). Instead you should start each "message" with a header that tells you how long the message is, if it's important to know what were the separate messages, or just read the stream like a normal file, if it's not important.
TCP is a byte-stream protocol, not a message protocol. There is no guarantee that what you write with a single send() will be received via a single recv(). If you need message boundaries you must implement them yourself, e.g. with a length-word prefix, a type-length-value protocol, or a self-describing protocol like XML.
You're experiencing a TCP congestion avoidance optimization commonly referred to as the Nagle algorithm (named after John Nagle, its inventor).
The purpose of this optimization is to reduce the number of small TCP segments circulating over a socket by combining them together into larger ones. When you write()/send() on a TCP socket, the kernel may not transmit your data immediately; instead it may buffer the data for a very short delay (typically a few tens of milliseconds), in case another request follows.
You may disable Nagle's algorithm on a per-socket basis, by setting the TCP_NODELAY option.
It is customary to disable Nagle in latency-sensitive applications (remote control applications, online games, etc..).

init, read and write for linux serial device with C

I'm working on a new project where I want to make a connection with an FTDI which is connected to my debian machine. I am intending to write the code with C, not C++. Here lies my problem. All the examples I find are incomplete or are made for a c++ compiler in stead of the GCC compiler.
The goal is to talk to my microcontroller which is connected to the FTDI. For debugging i want to start building a linux application which is able to:
initialize a serial connection on startup with ttyUSB1
send a character string
display character strings when they are received by the pc
save the communication to a .txt file
Is there any example code or tutorial to make this?
If I succeed I will defenetly place the code here so that new viewers can use it to!
Edit:
Like I said I would post the code if I had it, and this is what worked for me:
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <termios.h>
#define MODEM "/dev/ttyUSB0"
#define BAUDRATE B115200
int main(int argc,char** argv)
{
struct termios tio;
struct termios stdio;
struct termios old_stdio;
int tty_fd, flags;
unsigned char c='D';
tcgetattr(STDOUT_FILENO,&old_stdio);
printf("Please start with %s /dev/ttyS1 (for example)\n",argv[0]);
memset(&stdio,0,sizeof(stdio));
stdio.c_iflag=0;
stdio.c_oflag=0;
stdio.c_cflag=0;
stdio.c_lflag=0;
stdio.c_cc[VMIN]=1;
stdio.c_cc[VTIME]=0;
tcsetattr(STDOUT_FILENO,TCSANOW,&stdio);
tcsetattr(STDOUT_FILENO,TCSAFLUSH,&stdio);
fcntl(STDIN_FILENO, F_SETFL, O_NONBLOCK); // make the reads non-blocking
memset(&tio,0,sizeof(tio));
tio.c_iflag=0;
tio.c_oflag=0;
tio.c_cflag=CS8|CREAD|CLOCAL; // 8n1, see termios.h for more information
tio.c_lflag=0;
tio.c_cc[VMIN]=1;
tio.c_cc[VTIME]=5;
if((tty_fd = open(MODEM , O_RDWR | O_NONBLOCK)) == -1){
printf("Error while opening\n"); // Just if you want user interface error control
return -1;
}
cfsetospeed(&tio,BAUDRATE);
cfsetispeed(&tio,BAUDRATE); // baudrate is declarated above
tcsetattr(tty_fd,TCSANOW,&tio);
while (c!='q'){
if (read(tty_fd,&c,1)>0){
write(STDOUT_FILENO,&c,1); // if new data is available on the serial port, print it out
printf("\n");
}
if (read(STDIN_FILENO,&c,1)>0){
write(tty_fd,&c,1);//if new data is available on the console, send it to serial port
printf("\n");
}
}
close(tty_fd);
tcsetattr(STDOUT_FILENO,TCSANOW,&old_stdio);
return EXIT_SUCCESS;
}
Most of the code came from http://en.wikibooks.org/wiki/Serial_Programming/Serial_Linux but i also used a bit from the code posted below.
Handling with serial ports ( for linux OS ) :
- To open communication, you will need a descriptor which will be the handle for your serial port.
- Set the flags to control how the comunication will be.
- Write the command to this Handle ( make sure you're formatting the input correctly ).
- Get the answer. (make sure you're to read the amount of information you want )
- Close the handle.
It will seem like this:
int fd; // file descriptor
int flags; // communication flags
int rsl_len; // result size
char message[128]; // message to send, you can even dinamically alocate.
char result[128]; // result to read, same from above, thanks to #Lu
flags = O_RDWR | O_NOCTTY; // Read and write, and make the job control for portability
if ((fd = open("/dev/ttyUSB1", flags)) == -1 ) {
printf("Error while opening\n"); // Just if you want user interface error control
return -1;
}
// In this point your communication is already estabilished, lets send out something
strcpy(message, "Hello");
if (rsl_len = write(fd, message, strlen(message)) < 0 ) {
printf("Error while sending message\n"); // Again just in case
return -2;
}
if (rsl_len = read(fd, &result, sizeof(result)) < 0 ) {
printf("Error while reading return\n");
return -3;
}
close(fd);
Note that you have to care about what to write and what to read.
Some more flags can be used in case of parity control, stop bits, baud rate and more.
Since gcc is a C/C++ compiler, you don't need to limit yourself to pure C.
Sticking to pure C is OK if you enjoy writing lots of boilerplate code, and if you really know what you're doing. Many people use Unix APIs incorrectly, and a lot of example code out there is much too simplistic. Writing correct Unix C code is somewhat annoying, to say the least.
Personally, I'd suggest using not only C++, but also a higher-level application development framework like Qt. Qt 5 comes bundled with a QtSerialPort module that makes it easy to enumerate the serial ports, configure them, and get data into and out of them. Qt does not force you to use the gui modules, it can be a command line application, or a non-interactive server/daemon.
QtSerialPort is also usable from Qt 4, but it doesn't come bundled with Qt 4, you have to add it to your project. I suggest starting out with Qt 5, it's nicer to use with its C++11 support.
The code using Qt can be pretty simple, not much longer than your plain-English description. The below is a Qt console application using Qt 5 and C++11. It uses the core and serialport modules. It also handles the SIGINT signal so that the output file gets flushed before the process would terminate due to a ^C. I'm using QLocalSocket in place of raw Unix API to communicate from the Unix signal handler, the functionality is the same.
Only the code within main is strictly required, the rest is just to make it properly wrap things up when you hit ^C.
#include <QCoreApplication>
#include <QSerialPort>
#include <QFile>
#include <QTextStream>
#include <QLocalServer>
#include <QLocalSocket>
#include <cstdio>
#include <csignal>
QLocalSocket * xmit;
static void signalHandler(int)
{
xmit->write(" ");
xmit->flush();
}
static bool setupSignalHandler()
{
QLocalServer srv;
srv.listen("foobarbaz");
xmit = new QLocalSocket(qApp);
xmit->connectToServer(srv.serverName(), QIODevice::WriteOnly);
srv.waitForNewConnection();
QLocalSocket * receive = srv.nextPendingConnection();
receive->setParent(qApp);
qApp->connect(receive, &QLocalSocket::readyRead, &QCoreApplication::quit);
struct sigaction sig;
sig.sa_handler = signalHandler;
sigemptyset(&sig.sa_mask);
sig.sa_flags = SA_RESTART;
return ! sigaction(SIGINT, &sig, NULL);
}
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
setupSignalHandler();
QSerialPort port("ttyUSB1");
QFile file("file.txt");
QTextStream err(stderr, QIODevice::WriteOnly);
QTextStream out(stdout, QIODevice::WriteOnly);
if (!file.open(QIODevice::WriteOnly)) {
err << "Couldn't open the output file" << endl;
return 1;
}
if (!port.open(QIODevice::ReadWrite)) {
err << "Couldn't open the port" << endl;
return 2;
}
port.setBaudRate(9600);
QObject::connect(&port, &QSerialPort::readyRead, [&](){
QByteArray data = port.readAll();
out << data;
file.write(data);
});
out << "Use ^C to quit" << endl;
return a.exec();
}

Resources