remove characters from EOL - c

I read a buffer from a serial device. It returns me these results (2 lines every time)
Hello World.
My name is John.
Hello World.^M^JMy name
is Mike.
Hello World.^M^JMy name
is ^M^JERROR Peter.
These results are in Linux command line. ^M^J is the EOL and means \r\n in Windows. The first result is ok but the other two are terrible. Is there any way to check for ^M^J characters and remove them? Because I want these results:
Hello World.
My name is John.
Hello World.
My name is Mike.
Hello World.
My name is Peter.
With this code I read the buffer
char buff[150];
memset(buff, 0, sizeof(buff));
for (;;)
{
n=read(fd,buff,sizeof(buff));
printf("%s", buff);
}
UPDATE
I open and configure my device in this way
int open_port(void)
{
int fd; // file description for the serial port
fd = open("/dev/ttyAMA0", O_RDWR | O_NOCTTY | O_NDELAY);
if(fd == -1) // if open is unsucessful
{
//perror("open_port: Unable to open /dev/ttyAMA0 - ");
printf("open_port: Unable to open /dev/ttyAMA0. \n");
}
else
{
fcntl(fd, F_SETFL, 0);
printf("port is open.\n");
}
return(fd);
} //open_port
And configure port
int configure_port(int fd) // configure the port
{
struct termios port_settings; // structure to store the port settings in
cfsetispeed(&port_settings, B9600); // set baud rates
cfsetospeed(&port_settings, B9600);
port_settings.c_cflag &= ~PARENB; // set no parity, stop bits, data bits
port_settings.c_cflag &= ~CSTOPB;
port_settings.c_cflag &= ~CSIZE;
port_settings.c_cflag |= CS8;
tcsetattr(fd, TCSANOW, &port_settings); // apply the settings to the port
return(fd);
} //configure_port

The printf() is behaving in an interesting fashion when it sees a \r\n rather than a lone \n. It interprets the paired character end-of-line as not an end-of-line and so rather than doing the usual end-of-line-function, it shows you a ^M^J. Simple eliminating the \r will then give you the desired behavior.
char buff[150];
int n = read(fd,buff,sizeof(buff)); // buff is not NUL terminated
if (n < 0) {
// deal with I/O error
}
if (n == 0) {
// deal with end-of-file
}
else {
for (int i=0; i<n; i++) {
if (isprint(buff[i]) || (buff[i] == '\n')) {
putchar(buff[i]);
}
else if (buff[i] == '\r') {
; // drop it
}
else {
; // TBD deal with unexpected control codes and codes 127-255
}
}
}
Notes:
1) You filled buff before from the serial device using a read(). As serial devices are binary, the bytes read may include NUL bytes. Reading an array of bytes with sporadically intersperse NUL bytes in a buffer and treating it like a NUL terminated character string will lead to missed data.
2) A read() does not append a \0 byte to the end of the buffer it reads and may explain your "ERROR".
3) In general, you are reading a binary device and writing to a textual output. The incoming binary stream is likely ASCII text using \r\n as an end-of-line but your stdout wants to use \n as end-of-line. As long as the bytes are printable ASCII (codes 32-126), all works as expected when printing to stdout. But when you read a \0, \r, \n, paired \r\n, other control characters, communication errors, etc., you need to consider how you want that displayed.

Firstly, ^M^J are end of line, not end of file.
Secondly, read reads binary data from the specified file descriptor. It reads the number of characters you have specified till it reaches the end of the file, or gets an error. If you want to read lines at a time, read one byte at a time, or use some other line oriented I/O calls (sscanf, that sort of thing)

You can check out this question, which proposes a function for reading lines from a file and dealing with Windows' carriage return.

Open the file with O_TEXT
#include <fcntl.h>
fd = open("/dev/ttyAMA0", O_RDWR | O_NOCTTY | O_NDELAY | O_TEXT);

Related

Copy & paste in non-canonical mod

I'm creating a program to implement a linux shell
I've changed terminal mod into non-canonical
void ft_getch_prepare(void)
{
int ret;
struct termios new_opts;
ret = tcgetattr(STDIN_FILENO, &new_opts);
new_opts.c_lflag &= ~(ICANON | ECHO | ECHOE | ECHOK
| ECHONL | ECHOPRT | ECHOKE | ICRNL);
new_opts.c_cc[VMIN] = 1;
new_opts.c_cc[VTIME] = 1;
ret += tcsetattr(STDIN_FILENO, TCSANOW, &new_opts);
}
int ft_getch(void)
{
int c;
c = 0;
ft_getch_prepare();
read(0, &c, 4);
return (c);
}
but when I want to copy a string and paste it, it only show the first character of the copied string
For example, I want to paste this string "HELLO WORLD" into my terminal, but
it only shows the first character "H"
If I complete your program with
int main()
{
int i = ft_getch();
printf("%x\n", i);
}
I get
$ ./a.out
4c4c4548
when I try to paste HELLO WORLD which is what I expect. (48 is the hexadecimal code for H, 45 for E, 4C for L; it look reversed as I'm on a little endian architecture).
The ICRNL flag constant applies to c_iflag, not c_lflag. You are turning it off in the wrong place. It is unclear to me why you're turning it off at all, but if you want to do so, then you need to modify the correct flag set.
The ECHOE, ECHOL, ECHONL, ECHOPRT, and ECHOKE local-mode flags are only relevant in canonical mode, which you are turning off. It should not be harmful to turn these off, too, but it does make your code harder to read and follow than it needs to be.
With respect to
when I want to copy a string and paste it, it only show the first character of the copied string
, I suspect you're being bitten by the input timer and / or minimum character count properties of non-canonical mode. These are controlled by the c_cc[VTIME] and c_cc[VMIN] elements of the "special characters" array in your termios structure. If you are configuring a terminal that will support interactive input, or for which there may otherwise be unbounded-length pauses in input, then you need to turn off the timer and ensure that reads block properly by setting
new_opts.c_cc[VTIME] = 0;
new_opts.c_cc[VMIN] = 1;
. I am uncertain whether that will be sufficient for your purposes, however, in part because I cannot judge whether the manner in which you are reading the input contributes to the issue.
UPDATE:
Since you've now disclosed your input function, I can say that you indeed do have significant problems there if it is supposed to provide an interface equivalent to getc(). You are reading four bytes at a time instead of one, and you are not handling EOF or errors properly. Moreover, the multi-byte read introduces the possibility of short reads, which you do not detect or handle.
If you're trying to read a single character at a time, then do that. The return value of getc() is int instead of char not because it's appropriate to try to read ints from the stream but to provide for result values that are not valid chars -- specifically, EOF.
I decline to rewrite your code for you, but to emulate getc(), it needs to do this:
read a single char at a time
check the return value of read. If it is anything other than 1 (for a one-character read) then return EOF
otherwise, return the char read, converted to type unsigned char.

When exiting terminal rawmode my contents stays on the screen

I've used this small editor for the basis of my project that I'm doing: https://github.com/antirez/kilo
The editor uses the terminal in rawmode and writes using VT100 escape sequences, however when exiting the program the contents that were displayed, stay displayed.
Before exiting...
After exiting...
As you can see the prompt appears again but what was left of the editor stays there until written over.
// Low level terminal handling
void disable_raw_mode(int fd)
{
// dont bother checking the return value as its too late
if (Editor.rawmode) {
tcsetattr(fd, TCSAFLUSH, &orig_termios);
Editor.rawmode = 0;
}
}
void editor_at_exit(void)
{
disable_raw_mode(STDIN_FILENO);
}
int enable_raw_mode(int fd)
{
struct termios raw;
if(Editor.rawmode) return 0; //already enabled
if(!isatty(STDIN_FILENO)) goto fatal;
atexit(editor_at_exit);
if(tcgetattr(fd, &orig_termios) == -1) goto fatal;
raw = orig_termios; // modify the original mode
/* input modes: no break, no CR to NL, no parity check, no strip char,
* * no start/stop output control. */
raw.c_iflag &= ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON);
// output modes - disable post processing
raw.c_oflag &= ~(OPOST);
//control modes - set 8 bit chars
raw.c_cflag |= (CS8);
//local modes, choing off, canonical off, no extended functions, no signal chars (, etc)
raw.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG);
//control chars - set return condition: min number of bytes and a timer
raw.c_cc[VMIN] = 0; // return each byte, or zero for a timeout
raw.c_cc[VTIME] = 1; //100ms timeout
//put terminal in raw mode after flushing
if(tcsetattr(fd, TCSAFLUSH, &raw) < 0) goto fatal;
Editor.rawmode = 1;
return 0;
fatal:
errno = ENOTTY;
return -1;
}
From what I understand, when the program exits the atexit(editor_at_exit) function is called and in that function raw mode is disabled. What am I missing to clean the terminal back to what it was before the editor was opened. I'm not looking to just clear the whole terminal.
The functionality you are looking for is called "alternate screen buffer", which originated in xterm but is nowadays supported by most terminals.
Alternate screen buffer is designed to provide exactly this functionality for full-screen terminal programs. In normal operation, output gets added to the scrollback buffer (and most terminals let the user scroll back to previous lines). Switching to the alternate screen buffer the scrollback buffer is left alone, and alternate screen buffer output is not added to the scrollback buffer. When returning from alternate screen buffer, the original scrollback buffer state is restored. This is what full-screen applications like nano use.
To switch to the alternate screen buffer, I recommend writing (the C string)
"\033[?1049h\033[2J\033[H" (15 chars)
to the terminal. If the terminal emulator supports the alternate screen buffer, this changes to it, clearing it and moving the cursor to the upper left corner. If the terminal emulator does not support it, this will clear the screen and move the cursor to the upper left corner.
To return from the alternate screen buffer, I recommend writing (the C string)
"\033[2J\033[H\033[?1049l" (15 chars)
to the terminal. If the terminal emulator supports the alternate screen buffer, this first clears the alternate screen buffer, then returns to the original scrollback buffer (like e.g. nano does). If the terminal emulator does not support it, this will clear the screen and move the cursor to the upper left corner.
I recommend this pair ("\033[?1049h\033[2J\033[H" and "\033[2J\033[H\033[?1049l"), because it works in a reasonable fashion regardless of whether the terminal emulator supports the alternate screen buffer or not, not leaving the full-screen application state visible afterwards.
If standard input is a terminal, I also recommend using e.g.
int write_term(const char *p)
{
const char *q = p;
ssize_t n;
int retval = 0, saved_errno;
/* Nothing to write? */
if (!q || !*q)
return 0;
saved_errno = errno;
/* async-signal safe version of q = p + strlen(p) */
while (*q)
q++;
while (p < q) {
n = write(STDIN_FILENO, p, (size_t)(q - p));
if (n > 0) {
p += n;
} else
if (n != -1) {
retval = EIO;
break;
} else
if (errno != EINTR && errno != EAGAIN && errno != EWOULDBLOCK) {
retval = errno;
break;
}
}
errno = saved_errno;
return retval;
}
to write the strings to the terminal, because standard C I/O functions may not be able to write to standard input (which is intended for reading only, after all). The above function is extremely careful, ignoring signal delivery (and even busy-looping if necessary if standard input is nonblocking), and even keeping errno intact; it is also async-signal safe, which means it could be used in a signal handler safely (although I advise against changing terminal buffer mode or settings in a signal handler, as that gets quite complicated to do correctly).
(The OP's code might have a suitable low-level I/O function already implemented, but one was not shown in the question.)

C| how to check if my input buffer(stdin) is empty?

I want to know how to check if my input buffer (perhaps its called stdin) is empty or not.
I dont want the program to stop if the buffer is empty, and I dont want the input to necessarily end with \n, therefore just using scanf is not enough.
I tried searching on google and on this website but no answer was enough.
I tried using feof(stdin) like this:
int main()
{
char c,x;
int num;
scanf("%c",&c);
scanf("%c",&x);
num=feof(stdin);
printf("%d",num);
}
but all it did was printing 0 no matter the input. adding fflush(stdin) after the second scanf gave the same result.
other answers suggested using select and poll but I couldnt find any explanations for those functions.
Some other forum told me to use getchar() but I think they misunderstood my question.
if you suggest I use select/poll, could you please add an explanation about how to use those?
Here is the code for solving this:
fseek (stdin, 0, SEEK_END);
num = ftell (stdin);
fseek will put the pointer at the end of the stdin input buffer. ftell will return the size of file.
If you don't want to block on an empty stdin you should be able to fcntl it to O_NONBLOCK and treat it like any other non-blocking I/O. At that point a call to something like fgetc should return immediately, either with a value or EAGAIN if the stream is empty.
int ch = getc(stdin);
if (ch == EOF)
puts("stdin is empty");
else
ungetc(ch, stdin);
Try this, ungetc(ch, stdin); is added to eliminate the side effect.
You can use select() to handle the blocking issue and the man page select(2) has a decent example that polls stdin. That still doesn't address the problem of needing a line-delimiter ('\n'). This is actually due to the way the terminal handles input.
On Linux you can use termios,
#include <stdio.h>
#include <unistd.h>
#include <termios.h>
// immediate mode getchar().
static int getch_lower_(int block)
{
struct termios tc = {};
int status;
char rdbuf;
// retrieve initial settings.
if (tcgetattr(STDIN_FILENO, &tc) < 0)
perror("tcgetattr()");
// non-canonical mode; no echo.
tc.c_lflag &= ~(ICANON | ECHO);
tc.c_cc[VMIN] = block ? 1 : 0; // bytes until read unblocks.
tc.c_cc[VTIME] = 0; // timeout.
if (tcsetattr(STDIN_FILENO, TCSANOW, &tc) < 0)
perror("tcsetattr()");
// read char.
if ((status = read(STDIN_FILENO, &rdbuf, 1)) < 0)
perror("read()");
// restore initial settings.
tc.c_lflag |= (ICANON | ECHO);
if (tcsetattr(STDIN_FILENO, TCSADRAIN, &tc) < 0)
perror("tcsetattr()");
return (status > 0) ? rdbuf : EOF;
}
int getch(void)
{
return getch_lower_(1);
}
// return EOF if no input available.
int getch_noblock(void)
{
return getch_lower_(0);
}

Understanding read + write in c

char buf[1];
if (argc == 1) {
while (read(STDIN_FILENO, buf, 1) > 0) {
write(1, buf, sizeof(buf));
}
}
I have a few things I'd like to clarify about this snippet. We run it, ./exec_file Let's say we just press Enter. We move to the next line and read 1 byte '\n' then write it to stdout bringing us down one more line... simple enough. Now lets say we type h, then Enter. The program spits out h on the next line with an invisible '\n'.
Looking at the code after we type h it reads it into the buffer then writes it to stdout but somehow the program waits to spit it out on the next line till after I've pressed Enter..how?
Lastly, when we first hit the while loop wouldn't read initially return 0 since we haven't typed anything in initially??
stdin behaves a bit different than most other streams.
First, input is line buffered. That means that input isn't available until you press enter. this explains while the h won't appear until you press enter.
Since it is a stream it doesn't really have an end. Instead of failing when there is no data to read, the call will block until some data is available (or until the program receives a signal). A socket works the same way.
The blocking behaviour can be turned off using fcntl :
int fd = STDIN_FILENO;
int flags = fcntl(fd, F_GETFL, 0);
fcntl(fd, F_SETFL, flags | O_NONBLOCK);
The terminal is by default line buffered, because it is in canonical mode. From Linux manuals tcgetattr(3):
Canonical and noncanonical mode
The setting of the ICANON canon
flag in c_lflag determines whether the terminal is operating in
canonical mode (ICANON set) or noncanonical mode (ICANON unset).
By default, ICANON set.
In canonical mode:
Input is made available line by line. An input line is
available
when one of the line delimiters is typed (NL, EOL, EOL2; or EOF at
the start of line). Except in the case of EOF, the line delimiter
is included in the buffer returned by read(2).
Line editing is enabled (ERASE, KILL; and if the IEXTEN flag is
set:
WERASE, REPRINT, LNEXT). A read(2) returns at most one line of
input; if the read(2) requested fewer bytes than are available in
the current line of input, then only as many bytes as requested are
read, and the remaining characters will be available for a future
read(2).
You can switch off canonical mode on the terminal by calling tcgetattr with proper flags. First of all disable the canonical mode; then set the timeout to 0; set minimum read to 1 for blocking reads or 0 for non-blocking reads. Usually it is customary to also disable local echo, otherwise everything you type would still be automatically visible (and displayed twice in your program):
#include <stdio.h>
#include <unistd.h>
#include <termios.h>
int main() {
struct termios old_settings, new_settings;
int is_terminal;
// check whether the stdin is a terminal to begin with
if (is_terminal = isatty(STDIN_FILENO)) {
// get the old settings
tcgetattr(STDIN_FILENO, &old_settings);
new_settings = old_settings;
// disable canonical mode and echo
new_settings.c_lflag &= (~ICANON & ~ECHO);
// at least one character must be written before read returns
new_settings.c_cc[VMIN] = 1;
// no timeout
new_settings.c_cc[VTIME] = 0;
tcsetattr(STDIN_FILENO, TCSANOW, &new_settings);
}
while (read(STDIN_FILENO, buf, 1) > 0) {
// add this here so that you can verify that it is character by character,
// and not the local echo from the terminal
write(STDOUT_FILENO, ">", 1);
write(STDOUT_FILENO, buf, sizeof(buf));
}
// finally restore the old settings if it was a terminal
if (is_terminal) {
tcsetattr(STDIN_FILENO, TCSANOW, &old_settings);
}
return 0;
}
If you still want the blocking to happen, but want to read character by character, you can use termios to configure how the input will be given to your program. See the code below.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <termios.h>
int main()
{
char buf[1];
struct termios term, term_orig;
if (tcgetattr(0, &term_orig)) {
printf("tcgetattr failed\n");
exit(-1);
}
term = term_orig;
term.c_lflag &= ~ICANON;
term.c_lflag |= ECHO;
term.c_cc[VMIN] = 1;
term.c_cc[VTIME] = 0;
if (tcsetattr(0, TCSANOW, &term)) {
printf("tcsetattr failed\n");
exit(-1);
}
while (read(0, buf, 1) > 0) {
write(1, buf, sizeof(buf));
}
return 0;
}

Implementing a KeyPress Event in C

I have a infinite loop like the following one, and within this loop, I want to continuously check the keyboard to see if the escape key (ESC) has been pressed or not. If it is pressed, then the loop should be broken. How I can do this in C? (I am using gcc, and do access to pthreads as well in case this must be done via threads)
while(1){
//do something
//check for the ESC key
}
This is heavily system dependent. In Unix/Linux systems, the default terminal handler gathers lines and only notifies the program when a full line is available (after Enter is hit.) If you instead want keystrokes immediately, you need to put the terminal into non-canonical mode:
#include <termios.h>
struct termios info;
tcgetattr(0, &info); /* get current terminal attirbutes; 0 is the file descriptor for stdin */
info.c_lflag &= ~ICANON; /* disable canonical mode */
info.c_cc[VMIN] = 1; /* wait until at least one keystroke available */
info.c_cc[VTIME] = 0; /* no timeout */
tcsetattr(0, TCSANOW, &info); /* set immediately */
Once you've done that, you can use any calls that read from stdin and they will return keys without waiting for the end of the line. You can in addition set c_cc[VMIN] = 0 to cause it to not wait for keystrokes at all when you read from stdin.
If, however, you're reading stdin with stdio FILE related calls (getchar, etc), setting VMIN = 0 will make it think you've reached EOF whenever there are no keys available, so you'll have to call clearerr after that happens to try to read more characters. You can use a loop like:
int ch;
while((ch = getchar()) != 27 /* ascii ESC */) {
if (ch < 0) {
if (ferror(stdin)) { /* there was an error... */ }
clearerr(stdin);
/* do other stuff */
} else {
/* some key OTHER than ESC was hit, do something about it? */
}
}
After you're done, you probably want to be sure to set the terminal back into canonical mode, lest other programs (such as your shell) get confused:
tcgetattr(0, &info);
info.c_lflag |= ICANON;
tcsetattr(0, TCSANOW, &info);
There are also other things you can do with tcsetattr -- see then manual page for details. One thing that might suffice for your purposes is setting an alternative EOL character.
If the main job you're doing can be placed within this main loop, you could go for using STDIN in non-blocking mode. You still have a problem with the terminal which does line-buffering normally. You shall put the terminal to raw mode as well.
What about using Ctrl-C (interrupt)?
Non-blocking means that the read() system call always returns immediately even if there are no new bytes in the file. On Linux/Unix you can make STDIN nonblocking this way:
#include <unistd.h>
#include <fcntl.h>
fcntl(0, F_SETFL, O_NONBLOCK); /* 0 is the stdin file decriptor */
This is what you want:
#include <stdio.h>
#include <conio.h>
void main() {
int c;
while((c = getch()) != EOF )
if(c == 27) break;
/* 27 is the ASCII code for Esc */
}

Resources