How to handle buffering serial data - c

Question

I am trying to figure out a nice solution to reading serial data, and what to do when a read() is done but it contains an incomplete message.
The expected messages between devices have a defined start and end byte so its easy to see when a message starts and ends.
I can open a serial port fine and read from the serial port. But I am encountering the computer is reading faster than data coming through and I get an incomplete message.
For this example, lets say the message expected is
0x10 0xFF 0xFF 0xFF 0xFF 0x11
With 0x10 the start, 0x11 the end, and 0xFF is the data bytes
I am new to C so I may be missing something obvious,
My current solution
int main() {
/* Ommited serial port opening and checking*/
char read_buffer[80];
char message_buffer[80];
int message_buffer_index = 0;
int start_index = -1;
int end_index = -1;
int read_bytes;
read_bytes = read(serial_port, read_buffer, sizeof(read_buffer) - 1);
/* Now lets say read_bytes returns 3 and read buffer is {0x10, 0xFF, 0xFF} */
/* What should I do with the read_buffer? Currently appending to message buffer*/
memcpy(&message_buffer[message_buffer_index], &read_buffer[0], read_bytes);
/* Now check the message buffer for a full message */
for (int i = 0; i < 80; i++) {
if (message_buffer[i] = 0x10) {
start_index = i;
continue;
}
if (message_buffer[i] = 0x11) {
end_index = i;
}
if (start_index != -1 && end_index != -1) {
/* Found a message, do something with it, not super important here */
process_message();
/* Now how to erase the full message from the
buffer and push any non processed data to the
front? */
remove_message();
}
}
}
int process_message();
int remove_message();

Answer 1

To minimize the overhead of making many read() syscalls of small byte counts (e.g. the misguided solution of reading a byte at a time), use an intermediate buffer in your code.
The read() of the serial terminal should be in blocking mode to avoid a return code of zero bytes.
#define BLEN 1024
unsigned char rbuf[BLEN];
unsigned char *rp = &rbuf[BLEN];
int bufcnt = 0;
/* get a byte from intermediate buffer of serial terminal */
static unsigned char getbyte(void)
{
if ((rp - rbuf) >= bufcnt) {
/* buffer needs refill */
bufcnt = read(fd, rbuf, BLEN);
if (bufcnt <= 0) {
/* report error, then abort */
}
rp = rbuf;
}
return *rp++;
}
For proper termios initialization code for the serial terminal, see this answer. You should increase the VMIN parameter to something closer to the BLEN value or at least the length of longest expected message, and a VTIME of 1.
Now you can conveniently access the received data a byte at a time with minimal performance penalty.
#define MLEN 1024 /* choose appropriate value for message protocol */
int main()
{
unsigned char mesg[MLEN];
...
while (1) {
while (getbyte() != 0x10)
/* discard data until start found */ ;
length = 0;
while ((mesg[length] = getbyte()) != 0x11) {
/* accumulate data until end found */
length++;
}
/* process the message */
...
} /* loop for next message */
...
}
Note that your detection for a message frame is not robust.
If the data is binary and therefore can use the same values as these start and end bytes, then this parsing of the received data is prone to misaligned message frames.
See this answer for a description of a proper alogrithm.

Answer 2

You need circular buffer. Place data in the buffer and the process takes them when for example there is enough data or in any convenient moment.
Wikipedia has excellent article about it https://en.wikipedia.org/wiki/Circular_buffer

Answer 3

Better use stdio for reading, something like this:
FILE *fp = fdopen(serial_port, "r");
while (blabla) {
while (fgetc(fp) != 0x10)
; // wait until start
while ((c = fgetc(fp)) != 0x11)
message_buffer[message_buffer_index++] = c;
// here you have a complete message
}
Insert checks for EOF and errors if needed