I am creating a serial port application in which i am creating two threads one is WRITER THREAD which will write data to serial port and a READER THREAD which will read data from serial port.I know how to open, configure,read and write data on serial port but how to do it using threads.
I am using LINUX(ubuntu) and trying to open ttyS0 port programming in C.
The way I have done this in the past is to set up the port for asynchronous I/O using a VMIN of 0 and a VTIME of, say, 5 deciseconds. The purpose of this was to allow the thread to notice when it was time for the application to shut down, as it could try to read, time out, check for a quit flag, and then try to read some more.
Here is an example read function:
size_t myread(char *buf, size_t len) {
size_t total = 0;
while (len > 0) {
ssize_t bytes = read(fd, buf, len);
if (bytes == -1) {
if (errno != EAGAIN && errno != EINTR) {
// A real error, not something that trying again will fix
if (total > 0) {
return total;
}
else {
return -1;
}
}
}
else if (bytes == 0) {
// EOF
return total;
}
else {
total += bytes;
buf += bytes;
len -= bytes;
}
}
return total;
}
The write function would look as you would expect.
In your setup function, make sure to set:
struct termios tios;
...
tios.c_cflag &= ~ICANON;
tios.c_cc[VMIN] = 0;
tios.c_cc[VTIME] = 5; // You may want to tweak this; 5 = 1/2 second, 10 = 1 second, ...
...
Using of a serial port from 2 threads is simple, if only one thread reads and other thread only writes.
You should use one file descriptor for the serial port.
Open and initialize it in one thread by using normal open, tcsetattr, etc functions.
Then deliver the file descriptor to the other thread(s).
Now the reader thread can use read() function, and the writer can use write() function without any extra synchronization. You can also use select() in both threads.
Closing of the file descriptor needs attention, you should do it in one thread for avoiding problems.
Related
I am writing a little web server which involves epoll and multithread. For small and short http/1.1 requests and responses, it works as expected. But when working with large size file downloads, it is always interrupted by the timer I devised. I expire the timers with a fixed timeout value, but I also have a if statement to check if the response was sent successfully.
static void
_expire_timers(list_t *timers, long timeout)
{
httpconn_t *conn;
int sockfd;
node_t *timer;
long cur_time;
long stamp;
timer = list_first(timers);
if (timer) {
cur_time = mstime();
do {
stamp = list_node_stamp(timer);
conn = (httpconn_t *)list_node_data(timer);
if ((cur_time - stamp >= timeout) && httpconn_close(conn)) {
sockfd = httpconn_sockfd(conn);
DEBSI("[CONN] socket closed, server disconnected", sockfd);
close(sockfd);
list_del(timers, stamp);
}
timer = list_next(timers);
} while (timer);
}
}
I realized that in a non-blocking environment, the write() function might be interrupted during the request-response communication. I wonder how long write() can hold or how much data write() can send, so I can tweek the timout setting in my code.
This is the code which involves write(),
void
http_rep_get(int clifd, void *cache, char *path, void *req)
{
httpmsg_t *rep;
int len_msg;
char *bytes;
rep = _get_rep_msg((list_t *)cache, path, req);
bytes = msg_create_rep(rep, &len_msg);
/* send msg */
DEBSI("[REP] Sending reply msg...", clifd);
write(clifd, bytes, len_msg);
/* send body */
DEBSI("[REP] Sending body...", clifd);
write(clifd, msg_body_start(rep), msg_body_len(rep));
free(bytes);
msg_destroy(rep, 0);
}
And the following is the epoll loop I use to process the incoming requests,
do {
nevents = epoll_wait(epfd, events, MAXEVENTS, HTTP_KEEPALIVE_TIME);
if (nevents == -1) perror("epoll_wait()");
/* expire the timers */
_expire_timers(timers, HTTP_KEEPALIVE_TIME);
/* loop through events */
for (i = 0; i < nevents; i++) {
conn = (httpconn_t *)events[i].data.ptr;
sockfd = httpconn_sockfd(conn);
/* error case */
if ((events[i].events & EPOLLERR) || (events[i].events & EPOLLHUP) ||
(!(events[i].events & EPOLLIN))) {
perror("EPOLL ERR|HUP");
list_update(timers, conn, mstime());
break;
}
else if (sockfd == srvfd) {
_receive_conn(srvfd, epfd, cache, timers);
}
else {
/* client socket; read client data and process it */
thpool_add_task(taskpool, httpconn_task, conn);
}
}
} while (svc_running);
The http_rep_get() is executed by the threadpool handler httpconn_task(), HTTP_KEEPALIVE_TIME is the fixed timeout. The handler httpconn_task() will add a timer to the timers once a request arrives. Since the write() is executed in http_rep_get(), I think it might be interrupted by the timers. I guess I can change the way to write to the clients, but I need to make sure how much the write() can do.
If you are interested, you may browser my project to help me with this.
https://github.com/grassroot72/Maestro
Cheers,
Edward
Is there a size limit of write() for a socket fd?
It depends on what you mean by a limit.
As the comments explain, a write call may write fewer bytes than you ask it to. Furthermore, this is expected behavior if you perform a large write to a socket. However, there is no reliable way to determine (or predict) how many bytes will be written before you call write.
The correct way to deal with this is to check how many bytes were actually written each time, and use a loop for ensure that all bytes are written (or until you get a failure).
I wrote a chardevice that passes some messages received from the network to an user space application. The user space application has to both read the chardevice and send/receive messages via TCP sockets to other user-space applications. Both read and receiving should be blocking.
Since Libevent is able to handle multiple events at the same time, I thought registering an event for the file created by the chardevice and an event for a socket would just work, but I was wrong.
But a chardevice creates a "character special file", and libevent seems to not be able to block. If I implement a blocking mechanism inside the chardevice, i.e. mutex or semaphore, then the socket event blocks too, and the application cannot receive messages.
The user space application has to accept outside connections at any time.
Do you know how to make it work? Maybe also using another library, I just want a blocking behaviour for both socket and file reader.
Thank you in advance.
Update: Thanks to #Ahmed Masud for the help. This is what I've done
Kernel module chardevice:
Implement a poll function that waits until new data is available
struct file_operations fops = {
...
.read = kdev_read,
.poll = kdev_poll,
};
I have a global variable to handle if the user space has to stop, and a wait queue:
static working = 1;
static wait_queue_head_t access_wait;
This is the read function, I return -1 if there is an error in copy_to_user, > 0 if everything went well, and 0 if the module has to stop. used_buff is atomic since it handles the size of a buffer shared read by user application and written by kernel module.
ssize_t
kdev_read(struct file* filep, char* buffer, size_t len, loff_t* offset)
{
int error_count;
if (signal_pending(current) || !working) { // user called sigint
return 0;
}
atomic_dec(&used_buf);
size_t llen = sizeof(struct user_msg) + msg_buf[first_buf]->size;
error_count = copy_to_user(buffer, (char*)msg_buf[first_buf], llen);
if (error_count != 0) {
atomic_inc(&used_buf);
paxerr("send fewer characters to the user");
return error_count;
} else
first_buf = (first_buf + 1) % BUFFER_SIZE;
return llen;
}
When there is data to read, I simply increment used_buf and call wake_up_interruptible(&access_wait).
This is the poll function, I just wait until the used_buff is > 0
unsigned int
kdev_poll(struct file* file, poll_table* wait)
{
poll_wait(file, &access_wait, wait);
if (atomic_read(&used_buf) > 0)
return POLLIN | POLLRDNORM;
return 0;
}
Now, the problem here is that if I unload the module while the user space application is waiting, the latter will go into a blocked state and it won't be possible to stop it. That's why I wake up the application when the module is unloaded
void
kdevchar_exit(void)
{
working = 0;
atomic_inc(&used_buf); // increase buffer size to application is unlocked
wake_up_interruptible(&access_wait); // wake up application, but this time read will return 0 since working = 0;
... // unregister everything
}
User space application
Libevent by default uses polling, so simply create an event_base and a reader event.
base = event_base_new();
filep = open(fname, O_RDWR | O_NONBLOCK, 0);
evread = event_new(base, filep, EV_READ | EV_PERSIST,
on_read_file, base);
where on_read_file simply reads the file, no poll call is made (libevent handles that):
static void
on_read_file(evutil_socket_t fd, short event, void* arg)
{
struct event_base* base = arg;
int len = read(...);
if (len < 0)
return;
if (len == 0) {
printf("Stopped by kernel module\n");
event_base_loopbreak(base);
return;
}
... // handle message
}
The script file has over 6000 bytes which is copied into a buffer.The contents of the buffer are then written to the device connected to the serial port.However the write function only returns 4608 bytes whereas the buffer contains 6117 bytes.I'm unable to understand why this happens.
{
FILE *ptr;
long numbytes;
int i;
ptr=fopen("compass_script(1).4th","r");//Opening the script file
if(ptr==NULL)
return 1;
fseek(ptr,0,SEEK_END);
numbytes = ftell(ptr);//Number of bytes in the script
printf("number of bytes in the calibration script %ld\n",numbytes);
//Number of bytes in the script is 6117.
fseek(ptr,0,SEEK_SET);
char writebuffer[numbytes];//Creating a buffer to copy the file
if(writebuffer == NULL)
return 1;
int s=fread(writebuffer,sizeof(char),numbytes,ptr);
//Transferring contents into the buffer
perror("fread");
fclose(ptr);
fd = open("/dev/ttyUSB3",O_RDWR | O_NOCTTY | O_NONBLOCK);
//Opening serial port
speed_t baud=B115200;
struct termios serialset;//Setting a baud rate for communication
tcgetattr(fd,&serialset);
cfsetispeed(&serialset,baud);
cfsetospeed(&serialset,baud);
tcsetattr(fd,TCSANOW,&serialset);
long bytesw=0;
tcflush(fd,TCIFLUSH);
printf("\nnumbytes %ld",numbytes);
bytesw=write(fd,writebuffer,numbytes);
//Writing the script into the device connected to the serial port
printf("bytes written%ld\n",bytesw);//Only 4608 bytes are written
close (fd);
return 0;
}
Well, that's the specification. When you write to a file, your process normally is blocked until the whole data is written. And this means your process will run again only when all the data has been written to the disk buffers. This is not true for devices, as the device driver is the responsible of determining how much data is to be written in one pass. This means that, depending on the device driver, you'll get all data driven, only part of it, or even none at all. That simply depends on the device, and how the driver implements its control.
On the floor, device drivers normally have a limited amount of memory to fill buffers and are capable of a limited amount of data to be accepted. There are two policies here, the driver can block the process until more buffer space is available to process it, or it can return with a partial write only.
It's your program resposibility to accept a partial read and continue writing the rest of the buffer, or to pass back the problem to the client module and return only a partial write again. This approach is the most flexible one, and is the one implemented everywhere. Now you have a reason for your partial write, but the ball is on your roof, you have to decide what to do next.
Also, be careful, as you use long for the ftell() function call return value and int for the fwrite() function call... Although your amount of data is not huge and it's not probable that this values cannot be converted to long and int respectively, the return type of both calls is size_t and ssize_t resp. (like the speed_t type you use for the baudrate values) long can be 32bit and size_t a 64bit type.
The best thing you can do is to ensure the whole buffer is written by some code snippet like the next one:
char *p = buffer;
while (numbytes > 0) {
ssize_t n = write(fd, p, numbytes);
if (n < 0) {
perror("write");
/* driver signals some error */
return 1;
}
/* writing 0 bytes is weird, but possible, consider putting
* some code here to cope for that possibility. */
/* n >= 0 */
/* update pointer and numbytes */
p += n;
numbytes -= n;
}
/* if we get here, we have written all numbytes */
ANSWER
https://stackoverflow.com/a/12507520/962890
it was so trivial.. args! but lots of good information received. thanks to everyone.
EDIT
link to github: https://github.com/MarkusPfundstein/stream_lame_testing
ORIGINAL POST
i have some questions regarding IPC through pipelines. My goal is to receive MP3 data per TCP/IP stream, pipe it through LAME to decode it to wav, do some math and store it on disk (as a wav). I am using non blocking IO for the whole thing.
What irritates me a bit is that the tcp/ip read is way more fast than the pipe line trough lame. When i send a ~3 MB mp3 the file gets read on the client side in a couple of seconds. In the beginning, i can also write to the stdin of the lame process, than it stops writing, it reads the rest of the mp3 and if its finished i can write to lame again. 4096 bytes take approx 1 second (to write and read from lame). This is pretty slow, because i want to decode my wav min 128kbs.
The OS Is a debian 2.6 kernel on a this micro computer:
https://www.olimex.com/dev/imx233-olinuxino-maxi.html
65 MB RAM
400 MhZ
ulimit -n | grep pipe returns 512 x 8 , means 4096 which is ok. Its a 32 bit system.
The weird thing is that
my_process | lame --decode --mp3input - output.wav
goes very fast.
Here is my fork_lame code (which shall essentialy connect stout of my process to stdin of lame and visa versa)
static char * const k_lame_args[] = {
"--decode",
"--mp3input",
"-",
"-",
NULL
};
static int
fork_lame()
{
int outfd[2];
int infd[2];
int npid;
pipe(outfd); /* Where the parent is going to write to */
pipe(infd); /* From where parent is going to read */
npid = fork();
if (npid == 0) {
close(STDOUT_FILENO);
close(STDIN_FILENO);
dup2(outfd[0], STDIN_FILENO);
dup2(infd[1], STDOUT_FILENO);
close(outfd[0]); /* Not required for the child */
close(outfd[1]);
close(infd[0]);
close(infd[1]);
if (execv("/usr/local/bin/lame", k_lame_args) == -1) {
perror("execv");
return 1;
}
} else {
s_lame_pid = npid;
close(outfd[0]); /* These are being used by the child */
close(infd[1]);
s_lame_fds[WRITE] = outfd[1];
s_lame_fds[READ] = infd[0];
}
return 0;
}
This are the read and write functions. Please not that in write_lame_in. when i write to stderr instead of s_lame_fds[WRITE], the output is nearly immedieatly so its definitly the pipe through lame. But why ?
static int
read_lame_out()
{
char buffer[READ_SIZE];
memset(buffer, 0, sizeof(buffer));
int i;
int br = read(s_lame_fds[READ], buffer, sizeof(buffer) - 1);
fprintf(stderr, "read %d bytes from lame out\n", br);
return br;
}
static int
write_lame_in()
{
int bytes_written;
//bytes_written = write(2, s_data_buf, s_data_len);
bytes_written = write(s_lame_fds[WRITE], s_data_buf, s_data_len);
if (bytes_written > 0) {
//fprintf(stderr, "%d bytes written\n", bytes_written);
s_data_len -= bytes_written;
fprintf(stderr, "data_len write: %d\n", s_data_len);
memmove(s_data_buf, s_data_buf + bytes_written, s_data_len);
if (s_data_len == 0) {
fprintf(stderr, "finished\n");
}
}
return bytes_written;
}
static int
read_tcp_socket(struct connection_s *connection)
{
char buffer[READ_SIZE];
int bytes_read;
bytes_read = connection_read(connection, buffer, sizeof(buffer)-1);
if (bytes_read > 0) {
//fprintf(stderr, "read %d bytes\n", bytes_read);
if (s_data_len + bytes_read > sizeof(s_data_buf)) {
fprintf(stderr, "BUFFER OVERFLOW\n");
return -1;
} else {
memcpy(s_data_buf + s_data_len,
buffer,
bytes_read);
s_data_len += bytes_read;
}
fprintf(stderr, "data_len: %d\n", s_data_len);
}
return bytes_read;
}
The select stuff is pretty basic select logic. All blocks are non blocking of course.
Anyone any idea? I'd really appreciate any help ;-)
Oops! Did you check your LAME output?
Looking at your code, in particular
static char * const k_lame_args[] = {
"--decode",
"--mp3input",
"-",
"-",
NULL
};
and
if (execv("/usr/local/bin/lame", k_lame_args) == -1) {
means you are accidentally omitting the --decode flag as it will be argv[0] for LAME, instead of the first argument (argv[1]). You should use
static char * const k_lame_args[] = {
/* argv[0] */ "lame",
/* argv[1] */ "--decode",
/* argv[2] */ "--mp3input",
/* argv[3] */ "-",
/* argv[4] */ "-",
NULL
};
instead.
I think you are seeing a slowdown because you're accidentally recompressing the MP3 audio. (I noticed this just a minute ago, so haven't checked if LAME does that if you omit the --decode flag, but I believe it does.)
It is possible there is some sort of a blocking issue wrt. nonblocking pipes (not really being nonblocking), causing your end to block until LAME consumes the data.
Could you try an alternative approach? Use normal, blocking pipes, and a separate thread (using pthreads), which has the singular purpose of writing data from a circular buffer to LAME. Your main thread then keeps filling the circular buffer from your TCP/IP connection, and can easily also track and report buffer levels -- very useful during development and debugging. I've had much better success with blocking pipes and threads than nonblocking pipes, in general.
In Linux, threads really do not have that much of an overhead, so you should be comfortable in using them even on embedded architectures. The only trick you must master is specifying a sensible stack size for the worker thread -- in this case 16384 bytes is quite likely enough -- because only the initial stack given to the process will automatically grow and threads stacks are fixed an by default quite large.
Do you need example code?
Edited to add:
Your program receives data from the TCP/IP connection probably at a steady rate. However, LAME consumes the data in largeish chunks. In other words, the situation is like a car being towed, with the tow car jerking and stopping, with the towee jerking into it every time: both your process and LAME are most of the time waiting the other to receive/send more data.
First, those two close are not required (actually, you shouldn't do that), because the two dup2 which follow will do it automatically :
close(STDOUT_FILENO);
close(STDIN_FILENO);
I'm not very familiar with C design patterns and searching for the best solution for the following problem. I want to write a little chat client based on libpurple.
While running the program I want to be able to connect and disconnect several instant message accounts. The connect and disconnect calls should be passed over command line, but waiting for input with gets(); is no solution, because the program should run all the time getting new messages from the already connected instant message accounts.
You probably want to use poll (or select) for handling the events. So after establishing the connections, you have the file descriptors, and in addition you have the standard input, which also has a file descriptor from the OS (namely 0), and you can pass all those file descriptors to poll, which notifies you when there is incoming data on any of the file descriptors. Example code:
/* fd1, fd2 are sockets */
while(1) {
pollfd fds[3];
int ret;
fds[0].fd = fd1;
fds[1].fd = fd2;
fds[2].fd = STDIN_FILENO;
fds[0].events = POLLIN;
fds[1].events = POLLIN;
fds[2].events = POLLIN;
ret = poll(fds, 3, -1); /* poll() blocks, but you can set a timeout here */
if(ret < 0) {
perror("poll");
}
else if(ret == 0) {
printf("timeout\n");
}
else {
if(fds[0].revents & POLLIN) {
/* incoming data from fd1 */
}
if(fds[0].revents & (POLLERR | POLLNVAL)) {
/* error on fd1 */
}
if(fds[1].revents & POLLIN) {
/* incoming data from fd2 */
}
if(fds[1].revents & (POLLERR | POLLNVAL)) {
/* error on fd2 */
}
if(fds[2].revents & POLLIN) {
/* incoming data from stdin */
char buf[1024];
int bytes_read = read(STDIN_FILENO, buf, 1024);
/* handle input, which is stored in buf */
}
}
}
You didn't mention the OS. This works for POSIX (OS X, Linux, Windows with mingw). If you need to use the Win32 API, it'll look a bit different but the principle is the same.
Check out select(2). I'm not really sure how libpurple works, but if it allows notification via file-descriptor (like a file or socket), then select is your solution.
You could also try creating a seperate thread with pthread_create(3). That way it can block on gets (or whatever) while the rest of your program does it's thing.