I wrote a chardevice that passes some messages received from the network to an user space application. The user space application has to both read the chardevice and send/receive messages via TCP sockets to other user-space applications. Both read and receiving should be blocking.
Since Libevent is able to handle multiple events at the same time, I thought registering an event for the file created by the chardevice and an event for a socket would just work, but I was wrong.
But a chardevice creates a "character special file", and libevent seems to not be able to block. If I implement a blocking mechanism inside the chardevice, i.e. mutex or semaphore, then the socket event blocks too, and the application cannot receive messages.
The user space application has to accept outside connections at any time.
Do you know how to make it work? Maybe also using another library, I just want a blocking behaviour for both socket and file reader.
Thank you in advance.
Update: Thanks to #Ahmed Masud for the help. This is what I've done
Kernel module chardevice:
Implement a poll function that waits until new data is available
struct file_operations fops = {
...
.read = kdev_read,
.poll = kdev_poll,
};
I have a global variable to handle if the user space has to stop, and a wait queue:
static working = 1;
static wait_queue_head_t access_wait;
This is the read function, I return -1 if there is an error in copy_to_user, > 0 if everything went well, and 0 if the module has to stop. used_buff is atomic since it handles the size of a buffer shared read by user application and written by kernel module.
ssize_t
kdev_read(struct file* filep, char* buffer, size_t len, loff_t* offset)
{
int error_count;
if (signal_pending(current) || !working) { // user called sigint
return 0;
}
atomic_dec(&used_buf);
size_t llen = sizeof(struct user_msg) + msg_buf[first_buf]->size;
error_count = copy_to_user(buffer, (char*)msg_buf[first_buf], llen);
if (error_count != 0) {
atomic_inc(&used_buf);
paxerr("send fewer characters to the user");
return error_count;
} else
first_buf = (first_buf + 1) % BUFFER_SIZE;
return llen;
}
When there is data to read, I simply increment used_buf and call wake_up_interruptible(&access_wait).
This is the poll function, I just wait until the used_buff is > 0
unsigned int
kdev_poll(struct file* file, poll_table* wait)
{
poll_wait(file, &access_wait, wait);
if (atomic_read(&used_buf) > 0)
return POLLIN | POLLRDNORM;
return 0;
}
Now, the problem here is that if I unload the module while the user space application is waiting, the latter will go into a blocked state and it won't be possible to stop it. That's why I wake up the application when the module is unloaded
void
kdevchar_exit(void)
{
working = 0;
atomic_inc(&used_buf); // increase buffer size to application is unlocked
wake_up_interruptible(&access_wait); // wake up application, but this time read will return 0 since working = 0;
... // unregister everything
}
User space application
Libevent by default uses polling, so simply create an event_base and a reader event.
base = event_base_new();
filep = open(fname, O_RDWR | O_NONBLOCK, 0);
evread = event_new(base, filep, EV_READ | EV_PERSIST,
on_read_file, base);
where on_read_file simply reads the file, no poll call is made (libevent handles that):
static void
on_read_file(evutil_socket_t fd, short event, void* arg)
{
struct event_base* base = arg;
int len = read(...);
if (len < 0)
return;
if (len == 0) {
printf("Stopped by kernel module\n");
event_base_loopbreak(base);
return;
}
... // handle message
}
Related
I am writing a little web server which involves epoll and multithread. For small and short http/1.1 requests and responses, it works as expected. But when working with large size file downloads, it is always interrupted by the timer I devised. I expire the timers with a fixed timeout value, but I also have a if statement to check if the response was sent successfully.
static void
_expire_timers(list_t *timers, long timeout)
{
httpconn_t *conn;
int sockfd;
node_t *timer;
long cur_time;
long stamp;
timer = list_first(timers);
if (timer) {
cur_time = mstime();
do {
stamp = list_node_stamp(timer);
conn = (httpconn_t *)list_node_data(timer);
if ((cur_time - stamp >= timeout) && httpconn_close(conn)) {
sockfd = httpconn_sockfd(conn);
DEBSI("[CONN] socket closed, server disconnected", sockfd);
close(sockfd);
list_del(timers, stamp);
}
timer = list_next(timers);
} while (timer);
}
}
I realized that in a non-blocking environment, the write() function might be interrupted during the request-response communication. I wonder how long write() can hold or how much data write() can send, so I can tweek the timout setting in my code.
This is the code which involves write(),
void
http_rep_get(int clifd, void *cache, char *path, void *req)
{
httpmsg_t *rep;
int len_msg;
char *bytes;
rep = _get_rep_msg((list_t *)cache, path, req);
bytes = msg_create_rep(rep, &len_msg);
/* send msg */
DEBSI("[REP] Sending reply msg...", clifd);
write(clifd, bytes, len_msg);
/* send body */
DEBSI("[REP] Sending body...", clifd);
write(clifd, msg_body_start(rep), msg_body_len(rep));
free(bytes);
msg_destroy(rep, 0);
}
And the following is the epoll loop I use to process the incoming requests,
do {
nevents = epoll_wait(epfd, events, MAXEVENTS, HTTP_KEEPALIVE_TIME);
if (nevents == -1) perror("epoll_wait()");
/* expire the timers */
_expire_timers(timers, HTTP_KEEPALIVE_TIME);
/* loop through events */
for (i = 0; i < nevents; i++) {
conn = (httpconn_t *)events[i].data.ptr;
sockfd = httpconn_sockfd(conn);
/* error case */
if ((events[i].events & EPOLLERR) || (events[i].events & EPOLLHUP) ||
(!(events[i].events & EPOLLIN))) {
perror("EPOLL ERR|HUP");
list_update(timers, conn, mstime());
break;
}
else if (sockfd == srvfd) {
_receive_conn(srvfd, epfd, cache, timers);
}
else {
/* client socket; read client data and process it */
thpool_add_task(taskpool, httpconn_task, conn);
}
}
} while (svc_running);
The http_rep_get() is executed by the threadpool handler httpconn_task(), HTTP_KEEPALIVE_TIME is the fixed timeout. The handler httpconn_task() will add a timer to the timers once a request arrives. Since the write() is executed in http_rep_get(), I think it might be interrupted by the timers. I guess I can change the way to write to the clients, but I need to make sure how much the write() can do.
If you are interested, you may browser my project to help me with this.
https://github.com/grassroot72/Maestro
Cheers,
Edward
Is there a size limit of write() for a socket fd?
It depends on what you mean by a limit.
As the comments explain, a write call may write fewer bytes than you ask it to. Furthermore, this is expected behavior if you perform a large write to a socket. However, there is no reliable way to determine (or predict) how many bytes will be written before you call write.
The correct way to deal with this is to check how many bytes were actually written each time, and use a loop for ensure that all bytes are written (or until you get a failure).
I am developing a Linux module which I want to use to run my C program from kernel mode.
My problem here, in function read() of the module, I need to use a function named eval_keycode(), which is defined in my user space program.
When I try to compile my module, this error occurs :
error: implicit declaration of function ‘eval_keycode’
which is confirming my problem described above.
This is the read() function of my module :
ssize_t exer_read(struct file *pfile, char __user *buffer, size_t length, loff_t *offset) {
struct file *f = pfile->private_data;
enum { MAX_BUF_SIZE = 4096 };
size_t buf_size = 0;
char *buf = NULL;
ssize_t total = 0;
ssize_t rc = 0;
struct input_event *ev;
int yalv;
/* Allocate temporary buffer. */
if (length) {
buf_size = min_t(size_t, MAX_BUF_SIZE, length);
ev = kmalloc(buf_size, GFP_KERNEL);
if (ev == NULL) {
return -ENOMEM;
}
}
/* Read file to buffer in chunks. */
do {
size_t amount = min_t(size_t, length, buf_size);
rc = kernel_read(f, ev, amount, offset);
if (rc > 0) {
/* Have read some data from file. */
if (copy_to_user(buffer, ev, rc) != 0) {
/* Bad user memory! */
rc = -EFAULT;
} else {
/* Update totals. */
total += rc;
buffer += rc;
*offset += rc;
length -= rc;
for (yalv = 0; yalv < (int) (rc / sizeof(struct input_event)); yalv++) {
if (ev[yalv].type == EV_KEY) {
if (ev[yalv].value == 0)
eval_keycode(ev[yalv].code);
}
}
if (rc < amount) {
/* Didn't read the full amount, so terminate early. */
rc = 0;
}
}
}
}
while (rc > 0 && length > 0);
/* Free temporary buffer. */
kfree(buf);
if (total > 0) {
return total;
}
return rc;
}
This is my user space eval_keycode() defined function :
void eval_keycode(int code)
{
static int red_state = 0;
static int green_state = 0;
switch (code) {
case 260:
printf("BTN left pressed\n");
/* figure out red state */
red_state = red_state ? 0 : 1;
change_led_state(LED_PATH "/" red "/brightness", red_state);
break;
case BTN_RIGHT:
printf("BTN right pressed\n");
/* figure out green state */
green_state = green_state ? 0 : 1;
change_led_state(LED_PATH "/" green "/brightness", green_state);
break;
}
}
How can call the eval_keycode function from user space in order to solve this problem ?
Thank you.
You can, but it is a really bad idea. You need to establish a pointer to your user mode function, arrange for the process containing that function to be running (in the kernel) when you invoke it. That is a lot of work, and is fundamentally malware due to the security holes it creates. Additionally, in the mad dash to lock the door to the now empty barn in the wake of spectre et al, new layers of hackery are being deployed in newer CPUs to make this even harder.
A different approach:
In your original query, you are running this driver as a "tee"; that is, you take the input you receive from the device, give a copy to the caller, and call eval_keycode with each input. Eval_keycode doesn't modify the data, and the kernel module discards it afterwards. So Eval_keycode doesn't really need to be a function; or rather, there could be a user function:
void ProcessEvents(int fd) {
struct input_event ev;
while (read(fd, &ev, sizeof ev) == sizeof ev) {
eval_keycode(&ev);
}
}
if you could arrange for all the events to be fed into that fd. With this setup, your problem becomes more plumbing than kernel renovation. The user creates a pipe/socket/fifo/... and passes the write end to your kernel module (yay more ioctl()s). Your kernel module can then carefully use kernel_write() ( or vfs_write if you are stuck in the past ) to make these events available to the user handler. It wants to be careful about where its blocking points are.
You could extend this to work as a transform; that is where your driver transforms the events via a user mode handler; but at that point, you might really consider FUSE a better solution.
There is no traditional (in the way a library works) way to "call" a user space "function".
Your user space code should be running in its' own process (or another user space process), in which you would implement communications (through shared memory, interprocess calls [IPC], device files, interrupts..) where you handle the exchange of data, and act on the data (e.g. calling your eval_keycode function).
You basically want an upcall. You can find some explanation about that here, but it doesn't seem like Linux has an official upcall API.
However, as others have already mentioned, this isn't very good design. Upcalls are useful to servers implemented in the kernel.
If your exer_read() is only called for your own code (on your files for which you're implementing the driver), then perhaps inotify would be a better design.
If your exer_read() can be called for any file (e.g. you want any file write on the machine to change the LED state), then you want your userspace process containing eval_keycode() to poll some character device, and you want your module to write the code to this character device instead of calling eval_keycode().
If, however, change_led_state() is synchronous, and you actually need the read to block until it returns, then you are advised to reconsider your design... but that's a valid use case for upcalls.
Working on a Linux (Ubuntu) application. I need to read many files in a non-blocking fashion. Unfortunately epoll doesn't support real file descriptor (file descriptor from file), it does support file descriptor that's network socket. select does work on real file descriptors, but it has two drawbacks, 1) it's slow, linearly go through all the file descriptors that are set, 2) it's limited, it typically won't allow more than 1024 file descriptors.
I can change each file descriptors to be non-blocking and use non-blocking "read" to poll, but it's very expensive especially when there are a large number of file descriptors.
What are the options here?
Thanks.
Update 1
The use case here is to create some sort of file server, with many clients requesting for files, serve them in a non-blocking fashion. Due to network side implementation (not standard TCP/IP stack), can't use sendfile().
You could use multiple select calls combined with either threading or forking. This would reduce the number of FD_ISSET calls per select set.
Perhaps you can provide more details about your use-case. It sounds like you are using select to monitor file changes, which doesn't work as you would expect with regular files. Perhaps you are simply looking for flock
You could use Asynchronous IO on Linux. The relevant AIO manpages (all in section 3) appear to have quite a bit of information. I think that aio_read() would probably be the most useful for you.
Here's some code that I believe you should be able to adapt for your usage:
...
#define _GNU_SOURCE
#include <aio.h>
#include <unistd.h>
typedef struct {
struct aiocb *aio;
connection_data *conn;
} cb_data;
void callback (union sigval u) {
// recover file related data prior to freeing
cb_data data = u.sival_ptr;
int fd = data->aio->aio_fildes;
uint8_t *buffer = data->aio->aio_buf;
size_t len = data->aio->aio_nbytes;
free (data->aio);
// recover connection data pointer then free
connection_data *conn = data->conn;
free (data);
...
// finish handling request
...
return;
}
...
int main (int argc, char **argv) {
// initial setup
...
// setup aio for optimal performance
struct aioinit ainit = { 0 };
// online background threads
ainit.aio_threads = sysconf (_SC_NPROCESSORS_ONLN) * 4;
// use defaults if using few core system
ainit.aio_threads = (ainit.aio_threads > 20 ? ainit.aio_threads : 20)
// set num to the maximum number of likely simultaneous requests
ainit.aio_num = 4096;
ainit.aio_idle_time = 5;
aio_init (&ainit);
...
// handle incoming requests
int exit = 0;
while (!exit) {
...
// the [asynchronous] fun begins
struct aiocb *cb = calloc (1, sizeof (struct aiocb));
if (!cb)
// handle OOM error
cb->aio_fildes = file_fd;
cb->aio_offset = 0; // assuming you want to send the entire file
cb->aio_buf = malloc (file_len);
if (!cb->aio_buf)
// handle OOM error
cb->aio_nbytes = file_len;
// execute the callback in a separate thread
cb->aio_sigevent.sigev_notify = SIGEV_THREAD;
cb_data *data = malloc (sizeof (cb_data));
if (!data)
// handle OOM error
data->aio = cb; // so we can free() later
// whatever you need to finish handling the request
data->conn = connection_data;
cb->aio_sigevent.sigev_value.sival_ptr = data; // passed to callback
cb->aio_sigevent.sigev_notify_function = callback;
if ((err = aio_read (cb))) // and you're done!
// handle aio error
// move on to next connection
}
...
return 0;
}
This will result in you no longer having to wait on files being read in your main thread. Of course, you can create more performant systems using AIO, but those are naturally likely to be more complex and this should work for a basic use case.
I'm not very familiar with C design patterns and searching for the best solution for the following problem. I want to write a little chat client based on libpurple.
While running the program I want to be able to connect and disconnect several instant message accounts. The connect and disconnect calls should be passed over command line, but waiting for input with gets(); is no solution, because the program should run all the time getting new messages from the already connected instant message accounts.
You probably want to use poll (or select) for handling the events. So after establishing the connections, you have the file descriptors, and in addition you have the standard input, which also has a file descriptor from the OS (namely 0), and you can pass all those file descriptors to poll, which notifies you when there is incoming data on any of the file descriptors. Example code:
/* fd1, fd2 are sockets */
while(1) {
pollfd fds[3];
int ret;
fds[0].fd = fd1;
fds[1].fd = fd2;
fds[2].fd = STDIN_FILENO;
fds[0].events = POLLIN;
fds[1].events = POLLIN;
fds[2].events = POLLIN;
ret = poll(fds, 3, -1); /* poll() blocks, but you can set a timeout here */
if(ret < 0) {
perror("poll");
}
else if(ret == 0) {
printf("timeout\n");
}
else {
if(fds[0].revents & POLLIN) {
/* incoming data from fd1 */
}
if(fds[0].revents & (POLLERR | POLLNVAL)) {
/* error on fd1 */
}
if(fds[1].revents & POLLIN) {
/* incoming data from fd2 */
}
if(fds[1].revents & (POLLERR | POLLNVAL)) {
/* error on fd2 */
}
if(fds[2].revents & POLLIN) {
/* incoming data from stdin */
char buf[1024];
int bytes_read = read(STDIN_FILENO, buf, 1024);
/* handle input, which is stored in buf */
}
}
}
You didn't mention the OS. This works for POSIX (OS X, Linux, Windows with mingw). If you need to use the Win32 API, it'll look a bit different but the principle is the same.
Check out select(2). I'm not really sure how libpurple works, but if it allows notification via file-descriptor (like a file or socket), then select is your solution.
You could also try creating a seperate thread with pthread_create(3). That way it can block on gets (or whatever) while the rest of your program does it's thing.
I am creating a serial port application in which i am creating two threads one is WRITER THREAD which will write data to serial port and a READER THREAD which will read data from serial port.I know how to open, configure,read and write data on serial port but how to do it using threads.
I am using LINUX(ubuntu) and trying to open ttyS0 port programming in C.
The way I have done this in the past is to set up the port for asynchronous I/O using a VMIN of 0 and a VTIME of, say, 5 deciseconds. The purpose of this was to allow the thread to notice when it was time for the application to shut down, as it could try to read, time out, check for a quit flag, and then try to read some more.
Here is an example read function:
size_t myread(char *buf, size_t len) {
size_t total = 0;
while (len > 0) {
ssize_t bytes = read(fd, buf, len);
if (bytes == -1) {
if (errno != EAGAIN && errno != EINTR) {
// A real error, not something that trying again will fix
if (total > 0) {
return total;
}
else {
return -1;
}
}
}
else if (bytes == 0) {
// EOF
return total;
}
else {
total += bytes;
buf += bytes;
len -= bytes;
}
}
return total;
}
The write function would look as you would expect.
In your setup function, make sure to set:
struct termios tios;
...
tios.c_cflag &= ~ICANON;
tios.c_cc[VMIN] = 0;
tios.c_cc[VTIME] = 5; // You may want to tweak this; 5 = 1/2 second, 10 = 1 second, ...
...
Using of a serial port from 2 threads is simple, if only one thread reads and other thread only writes.
You should use one file descriptor for the serial port.
Open and initialize it in one thread by using normal open, tcsetattr, etc functions.
Then deliver the file descriptor to the other thread(s).
Now the reader thread can use read() function, and the writer can use write() function without any extra synchronization. You can also use select() in both threads.
Closing of the file descriptor needs attention, you should do it in one thread for avoiding problems.