I'm embedding a JVM into my webserver which has 4 worker threads that never die. The following code runs on each http request inside any one of 4 workers:
// normally I would do URL routing here first, but this is just a JNI test now
jclass cls;
jmethodID method;
jobjectArray args;
jclass stringClass;
jstring jstr;
(*jvm)->AttachCurrentThread (jvm, &env, NULL);
cls = (*env)->FindClass(env, "HelloWorldClass");
method = (*env)->GetStaticMethodID(env, cls, "main", "([Ljava/lang/String;)V");
jstr = (*env)->NewStringUTF(env, "Hello world!");
stringClass = (*env)->FindClass(env, "java/lang/String");
args = (*env)->NewObjectArray(env, 1, stringClass, jstr);
(*env)->CallStaticVoidMethod(env, cls, method, args);
When I step through with the debugger, it works. But when I put some load on it with weighttp benchmark, it randomly segfaults either on the FindClass() line or the CallSTaticVoidMethod() line. What could be the problem? I read through a lot of docs, I don't see how I would need to lock or free anything here.
This is pretty much the most basic JNI code that is possible, carried over from the official doc: http://java.sun.com/docs/books/jni/html/invoke.html
Looks like I have put JNIEnv* into global scope. While that itself shouldn't have caused a problem because it gets overwritten in each thread where it gets used, it appears as if JNI wants/needs to internally free it every time it's used. The wonders of API design!
Related
Goal: allow c extension to receive block/proc for delayed execution while retaining current execution context.
I have a method in c (exposed to ruby) that accepts a callback (via VALUE hash argument) or a block.
// For brevity, lets assume m_CBYO is setup to make a CBYO module available to ruby
extern VALUE m_CBYO;
VALUE CBYO_add_callback(VALUE callback)
{
if (rb_block_given_p()) {
callback = rb_block_proc();
}
if (NIL_P(callback)) {
rb_raise(rb_eArgError, "either a block or callback proc is required");
}
// method is called here to add the callback proc to rb_callbacks
}
rb_define_module_function(m_CBYO, "add_callback", CBYO_add_callback, 1);
I have a struct I'm using to store these with some extra data:
struct rb_callback
{
VALUE rb_cb;
unsigned long long lastcall;
struct rb_callback *next;
};
static struct rb_callback *rb_callbacks = NULL;
When it comes time (triggered by an epoll), I iterate over the callbacks and execute each callback:
rb_funcall(cb->rb_cb, rb_intern("call"), 0);
When this happens I am seeing that it successfully executes the ruby code in the callback, however, it is escaping the current execution context.
Example:
# From ruby including the above extension
CBYO.add_callback do
puts "Hey now."
end
loop do
puts "Waiting for signal..."
sleep 1
end
When a signal is received (via epoll) I will see the following:
$> Waiting for signal...
$> Waiting for signal...
$> Hey now.
$> // process hangs
$> // Another signal occurs
$> [BUG] vm_call_cfunc - cfp consistency error
Sometimes, I can get more than one signal to process before the bug surfaces again.
I found the answer while investigating a similar issue.
As it turns out, I too was trying to use native thread signals (with pthread_create) which are not supported with MRI.
TLDR; the Ruby VM is not currently (at the time of writing) thread safe. Check out this nice write-up on Ruby Threading for a better overall understanding of how to work within these confines.
You can use Ruby's native_thread_create(rb_thread_t *th) which will use pthread_create behind the scenes. There are some drawbacks that you can read about in the documentation above the method definition. You can then run the callback with Ruby's rb_thread_call_with_gvl method. Also, I haven't done it here, but it might be a good idea to create a wrapper method so you can use rb_protect to handle exceptions the callback may raise (otherwise they will be swallowed by the VM).
VALUE execute_callback(VALUE callback)
{
return rb_funcall(callback, rb_intern("call"), 0);
}
// execute the callback when the thread receives signal
rb_thread_call_with_gvl(execute_callback, data->callback);
I've been trying to look for questions on how to use g_spawn_sync() and they said that it is good to use when you want to execute a command in the terminal besides using pipes.
The only thing I can't figure out now is why the command cat /proc/cpuinfo doesn't work. error->message returns (No such file or directory)but if I use commands like ls or cat alone, it works. I also tried running cd /proc && cat cpuinfo but it gives me the same error.
I'm not an expert of glib but I read in the manual that I can use G_SPAWN_SEARCH_PATH so that it will check my PATH for the commands I can use without including the absolute path for the command.
I have the following code:
gchar *argv[] = { "cat /proc/cpuinfo", NULL };
char *output = NULL; // will contain command output
GError *error = NULL;
int exit_status = 0;
if (!g_spawn_sync(NULL, argv, NULL, G_SPAWN_SEARCH_PATH, NULL, NULL,
&output, NULL, &exit_status, &error))
{
printf("[getHardwareInfo] DEBUG: Error on g_spawn_sync %s.\n", error->message);
}
tl;dr: Do not use g_spawn_command_line_sync() unless you really know what you are doing.
Firstly, the actual problem you are hitting: John Szakmeister’s comment was correct: g_spawn_sync() takes an array of arguments, the first one of which is the path to the program to execute (or to look for in $PATH, if you’ve specified G_SPAWN_SEARCH_PATH). By passing the array { "cat /proc/cpuinfo", NULL }, you are saying that you want to run the program cat /proc/cpuinfo with no arguments, not the program cat with the argument /proc/cpuinfo.
However, there are many other problems here, and I think it’s important to mention them before people start cargo-culting this code, because they have security implications:
As LegalProgrammer says, why are you spawning cat when you could just call g_file_get_contents()?
Failing that, use GSubprocess instead of g_spawn_*(). It’s a more modern API, which allows you to monitor the lifecycle of the spawned process more easily, as well as getting streaming I/O in and out of the subprocess.
Do not ignore the warnings in the manual about the security implications of using g_spawn_command_line_sync(). There are several:
It will run the first matching program found in your $PATH, so if an attacker has control of your $PATH, or write access to any directory in that $PATH (such as ~/.local/bin), you will end up running an attacker-controlled program.
It’s a synchronous function, so will block on the subprocess completing, which could take unbounded time. Your program will be unresponsive for that time.
It returns the output in a single allocation, rather than as a stream, so if the subprocess returns many megabytes of output, you may hit allocation failures and abort.
The obvious next step from “g_spawn_command_line_sync() seems to do what I want” is “let’s use g_strdup_printf() to put together a command to run with it”, and then you have shell injection vulnerabilities, where an attacker who controls any of the parameters to that printf() can twist the entire shell command to execute their arbitrary code.
I'm answering my question here. After reading the manual again, I decided to use another function, g_spawn_command_line_sync, which is simpler to use than g_spawn_sync.
A simple version of g_spawn_sync() with little-used parameters removed, taking a command line instead of an argument vector. See g_spawn_sync() for full details. command_line will be parsed by g_shell_parse_argv(). Unlike g_spawn_sync(), the G_SPAWN_SEARCH_PATH flag is enabled. Note that G_SPAWN_SEARCH_PATH can have security implications, so consider using g_spawn_sync() directly if appropriate. Possible errors are those from g_spawn_sync() and those from g_shell_parse_argv().
Here is my new code:
char *output = NULL; // will contain command output
GError *error = NULL;
gint exit_status = 0;
if (!g_spawn_command_line_sync("cat /proc/cpuinfo", &output, NULL, &exit_status, &error))
{
printf("[getHardwareInfo] DEBUG: Error on g_spawn_command_line_sync %s.\n", error->message);
I will give a detailed exp of the program and lead to the issue regarding the use of netlink socket communication.
The last paragraph asks the actual question I need an answer for, so you might wanna start by peeking it first.
Disclaimer before I start:
- I have made an earlier search before asking here and did not find complete solution / alternative to my issue.
- I know how to initialize a module and insert it to kernel.
- I know to handle communication between module and user-space without using netlink sockets. Meaning using struct file_operations func pointers assignments to later be invoked by the module program whenever a user attempts to read/write etc. and answer to the user using copy_to_user / copy_from_user.
- This topic refers to Linux OS, Mint 17 dist.
- Language is C
Okay, so I am building a system with 3 components:
1. user.c : user application (user types commands here)
2. storage.c : storage device ('virtual' disk-on-key)
3. device.ko : kernel module (used as proxy between 1. and 2.)
The purpose of this system is to be able (as a user) to:
- Copy files to the virtual disk-on-key device (2) - like an "upload" from local directory that belongs to the user.
- Save files from the virtual device on local directory - like "download" from the device storage to the user directory.
Design:
Assuming programs (1),(2) are compiled and running + (3) has successfully inserted using the bash command ' sudo insmod device.ko ' , the following should work like this (simulation ofc):
Step 1 (in user.c) -> user types 'download file.txt'
Step 2 (in device.ko) -> the device recognizes the user have tried to 'write' to it (actually user just passing the string "download file.txt") and invokes the 'write' implementation of the method we set on struct file_operation earlier on module_init().
The device (kernel module) now passes the data (string with a command) to the storage.c application, expecting an answer to later be retrieved to the user.c application.
Step 3 (in storage.c) -> now, lets say this program performs a busy-wait loop of 'readmsg()' and that's how a request from module event is triggered and recognized, the storage device now recognizes that the module has sent a request (string with a command \ data). Now, the storage programs shall perform an implementation of some function 'X' to send the data requested using sendmsg() somewhere inside the function.
Now, here comes the issue.
Usually, on all of the examples I've looked on web, the communication between the kernel-module and a user-space (or the storage.c program in our case) using netlink is triggered by the user-space and not vice versa. Meaning that the sendmsg() function from the user-space invokes the 'request(struct sk_buff *skb)' method (which is set on the module_init() part as following:
struct netlink_kernel_cfg cfg = {
.input = request // when storage.c sends something, it invokes the request function
};
so when the storage.c performs something like:
sendmsg(sock_fd,&msg,0); // send a msg to the module
the module invokes and runs the:
static void request(struct sk_buff *skb) {
char *msg ="Hello from kernel";
msg_size=strlen(msg);
netlink_holder=(struct nlmsghdr*)skb->data;
printk(KERN_INFO "Netlink received msg payload:%s\n",(char*)nlmsg_data(netlink_holder));
pid = netlink_holder->nlmsg_pid; // pid of sending process
skb_out = nlmsg_new(msg_size,0);
if(!skb_out){
printk(KERN_ERR "Failed to allocate new skb\n");
return;
}
netlink_holder=nlmsg_put(skb_out,0,0,NLMSG_DONE,msg_size,0); // add a new netlink message to an skb. more info: http://elixir.free-electrons.com/linux/v3.2/source/include/net/netlink.h#L491
NETLINK_CB(skb_out).dst_group = 0; // not in multicast group
strncpy(nlmsg_data(netlink_holder),msg,msg_size); // assign data as char* (variable msg)
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage. more info: http://elixir.free-electrons.com/linux/latest/source/include/net/netlink.h#L598
if(result<0)
printk(KERN_INFO "Error while sending bak to user\n");
}
and from all that big chunk, the only thing that im interesting in is actually doing this:
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage.
BUT I can't use nlmsg_unicast() without having the strcut sk_buff* which is provided automatically for me whenever there's an invoke from storage.c !
To sum up everything:
How do I send a msg from the device.ko (kernel module) to the user-space withtout having to wait for request to invoke / rely on the provided strcut sk_buff parameter from the earlier shown 'request()' method ?
Hope this sums up the point.
Thanks.
The only question here is that you need the user-space program connected to kernel-space first to get the pid of your user-program.
After get the pid, you can manually construct the skb_out and send it out through netlink_unicast or nlmsg_unicast.
The pid is always needed, you can set it as static and let your user-space program connect to your device.ko to make a long-maintained link.
Although this question is asked at 2017, I believe OP has already found the answer :D
I'm writing a plugin for my statusbar to print MPD state, currently using the libmpdclient library. It has to be robust to properly handle lost connections in case MPD is restarted, but simple checking with mpd_connection_get_error on existing mpd_connection object does not work – it can detect the error only when the initial mpd_connection_new fails.
This is a simplified code I'm working with:
#include <stdio.h>
#include <unistd.h>
#include <mpd/client.h>
int main(void) {
struct mpd_connection* m_connection = NULL;
struct mpd_status* m_status = NULL;
char* m_state_str;
m_connection = mpd_connection_new(NULL, 0, 30000);
while (1) {
// this check works only on start up (i.e. when mpd_connection_new failed),
// not when the connection is lost later
if (mpd_connection_get_error(m_connection) != MPD_ERROR_SUCCESS) {
fprintf(stderr, "Could not connect to MPD: %s\n", mpd_connection_get_error_message(m_connection));
mpd_connection_free(m_connection);
m_connection = NULL;
}
m_status = mpd_run_status(m_connection);
if (mpd_status_get_state(m_status) == MPD_STATE_PLAY) {
m_state_str = "playing";
} else if (mpd_status_get_state(m_status) == MPD_STATE_STOP) {
m_state_str = "stopped";
} else if (mpd_status_get_state(m_status) == MPD_STATE_PAUSE) {
m_state_str = "paused";
} else {
m_state_str = "unknown";
}
printf("MPD state: %s\n", m_state_str);
sleep(1);
}
}
When MPD is stopped during the execution of the above program, it segfaults with:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fb2fd9557e0 in mpd_status_get_state () from /usr/lib/libmpdclient.so.2
The only way I can think of to make the program safe is to establish a new connection in every iteration, which I was hoping to avoid. But then what if the connection is lost between individual calls to libmpdclient functions? How often, and more importantly how exactly, should I check if the connection is still alive?
The only way I could find that really works (beyond reestablishing a connection with each run) is using the idle command. If mpd_recv_idle (or mpd_run_idle) returns 0, it is an error condition, and you can take that as a cue to free your connection and run from there.
It's not a perfect solution, but it does let you keep a live connection between runs, and it helps you avoid segfaults (though I don't think you can completely avoid them, because if you send a command and mpd is killed before you recv it, I'm pretty sure the library still segfaults). I'm not sure if there is a better solution. It would be fantastic if there was a reliable way to detect if your connection was still alive via the API, but I can't find anything of the sort. It doesn't seem like libmpdclient is well-built for very long-lived connections that have to deal with mpd instances that go up and down over time.
Another lower-level option is to use sockets to interact with MPD through its protocol directly, though in doing that you'd likely reimplement much of libmpdclient itself anyway.
EDIT: Unfortunately, the idle command does block until something happens, and can sit blocking for as long as a single audio track will last, so if you need your program to do other things in the interim, you have to find a way to implement it asynchronously or in another thread.
Assuming "conn" is a connection created with "mpd_connection_new":
if (mpd_connection_get_error(conn) == MPD_ERROR_CLOSED) {
// mpd_connection_get_error_message(conn)
// will return "Connection closed by the server"
}
You can run this check after almost any libmpdclient call, including "mpd_recv_idle" or (as per your example) "mpd_run_status".
I'm using libmpdclient 2.18, and this certainly works for me.
I have a complex cgi executable written in C, I configured in Apache2 and now it is running succesfully. How can I debug this program in the source code, such as set break points and inspect variables? Any tools like gdb or eclipse? Any tutorial of how to set up the debugging environment?
Thanks in advance!!
The CGI interface basically consists in passing the HTTP request to the executable's standard input and getting the response on the standard output. Therefore you can write test requests to files and manually execute your CGI without having to use Apache. The debugging can then be done with GDB :
gdb ./my_cgi
>> break some_func
>> run < my_req.txt
with my_req.txt containing the full request:
GET /some/func HTTP/1.0
Host: myhost
If you absolutely need the CGI to be run by Apache it may become tricky to attach GDB to the right process. You can for example configure Apache to have only one worker process, attach to it with gdb -p and use set follow-fork-mode child to make sure it switches to the CGI process when a request arrives.
I did this: in cgi main i added code to look for an existing file, like /var/tmp/flag. While existing, i run in a loop. Time enough to attach to cgi process via gdb. After then i delete /var/tmp/flag and from now I can debug my cgi code.
bool file_exists(const char *filename)
{
ifstream ifile(filename);
return ifile;
}
int cgiMain()
{
while (file_exists ("/var/tmp/flag"))
sleep (1);
...
your code
Unless FastCGI or SCGI is used, the CGI process is short-lived and you need to delay its exit to have enough time to attach the debugger while the process is still running. For casual debugging the easiest option is to simply use sleep() in an endless loop at the breakpoint location and exit the loop with the debugger once it is attached to the program.
Here's a small example CGI program:
#include <stdio.h>
#include <unistd.h>
void wait_for_gdb_to_attach() {
int is_waiting = 1;
while (is_waiting) {
sleep(1); // sleep for 1 second
}
}
int main(void) {
wait_for_gdb_to_attach();
printf("Content-Type: text/plain;charset=us-ascii\n\n");
printf("Hello!");
return 0;
}
Suppose it is compiled into cgi-debugging-example, this is how you would attach the debugger once the application enters the endless loop:
sudo cgdb cgi-debugging-example $(pgrep cgi-debugging)
Next you need to exit the infinite loop and wait_for_gdb_to_attach() function to reach the "breakpoint" in your application. The trick here is to step out of sleep functions until you reach wait_for_gdb_to_attach() and set the value of the variable is_waiting with the debugger so that while (is_waiting) exits:
(gdb) finish
Run till exit from 0x8a0920 __nanosleep_nocancel () at syscall-template.S:81
0x8a07d4 in __sleep (seconds=0) at sleep.c:137
(gdb) finish
Run till exit from 0x8a07d4 in __sleep (seconds=0) at sleep.c:137
wait_for_gdb_to_attach () at cgi-debugging-example.c:6
Value returned is $1 = 0
(gdb) set is_waiting = 0 # <<<<<< to exit while
(gdb) finish
Run till exit from wait_for_gdb_to_attach () cgi-debugging-example.c:6
main () at cgi-debugging-example.c:13
Once you are out of wait_for_gdb_to_attach(), you can continue debugging the program or let it run to completion.
Full example with detailed instructions here.
I'm not sure how to use gdb or other frontends in eclipse, but I just debugged my CGI program with gdb. I'd like to share something that other answers didn't mention, that CGIs usually need to read request meta-variables defined in RFC 3875#4.1 with getenv(3). Popular request variables in my mind are:
SCRIPT_NAME
QUERY_STRING
CONTENT_LENGTH
CONTENT_TYPE
REMOTE_ADDR
There variables are provided by http servers such as Apache. When debugging with gdb, we need to set these values by our own with set environment. In my case, there're only a few variables neededa(and the source code is very old, it still uses SCRIPT_URL instead of SCRIPT_NAME), so here's my example:
gdb cgi_name
set environment SCRIPT_URL /path/to/sub/cgis
set environment QUERY_STRING p1=v1&p2=v2
break foo.c:42
run
For me both solutions for debugging the CGI in gdb without web server presented above didn't work.
Maybe the second solution works for a GET Request.
I needed a combination of both, first setting the environment variables from rfc3875 (not sure if all of them are really neded).
Then I was able to pass only the params (not the compltete request) via STDIN from a file.
gdb cgi_name
set environment REQUEST_METHOD=POST
set environment CONTENT_LENGTH=1337
set environment CONTENT_TYPE=application/json
set environment SCRIPT_NAME=my.cgi
set environment REMOTE_ADDR=127.0.0.1
run < ./params.txt
With params.txt:
{"user":"admin","pass":"admin"}