I have a website where each webpage is compiled into a binary (I have 100 webpages, therefore I have 100 binaries). Apache's .htaccess contains the line "SetHandler cgi-script" which instructs apache to use CGI when a binary (webpage) is requested.
How can I modify this website to use FastCGI instead of CGI ?
Do I just have to include this header and use this while loop (FastCGI.com) in each of the 100 binaries and modify .htaccess to "SetHandler fastcgi-script" ?
#include "fcgi_stdio.h" // instead of stdio.h
while(FCGI_Accept() >= 0)
So how will FastCGI work exactly ? Apache will dispatch webpages using 1 persistent process for the entire website or will there be 1 persistent process for each of the 100 binaries ?
A FastCGI script is a network server that listens for connections in a loop. The web server forward requests to the FCGI server which sends back some dynamically generated content - all over a socket connection. Thus a FCGI script is faster than CGI as it is not re-spawned for each request.
I don't understand why you need 100 binaries for 100 pages. A single script is enough to generate content for 100 pages, based on some request parameter. The FCGI server should also scale pretty well for multiple connections as it is usually made to poll on the socket file descriptor. (Look at the code of the implementation to make sure of this).
To generate 100 pages you don't necessarily need 100 if statements. Consider this pseudo-code:
hash_table page_generators; // map page types to function objects (or function pointers)
page_generators["login_page"] = handle_login_page_fn;
page_generators["contact_page"] = handle_contact_page_fn;
// ... and so on
// request handler
page_type = request.get("page_type");
fn = page_generators[page_type];
if (fn == NULL)
return "<html><body>Invalid request</body></html>";
else
return fn(request);
Related
I have an application project running on Linux environment, which includes libuv and another third-party library, the third-party library provides APIs for starting a TCP connection to remote server (say xxx_connect()) and getting file descriptor of the active connection (say xxx_get_socket()) . So far I managed to get valid file descriptor from xxx_get_socket() after xxx_connect() completed successfully, and initialize uv_poll_t handle with that file descriptor in my program.
Currently I am working on reconnecting function, after reconnecting the same server (by running xxx_connect() again), xxx_get_socket() returns different file descriptor, that means it is necessary to update io_watcher.fd member of a uv_poll_t handle to receive data in the new active connection.
AFAIK uv_poll_init() internally invokes uv__io_check_fd() , uv__nonblock() and uv__io_init() , it seems possible to modify io_watcher.fd of a uv_poll_t handle without closing the handle and then initializing it again (see sample code below), which has extra latency. However I'm not sure if it is safe to do so, I don't know whether io_watcher.fd member of a uv_poll_t handle is referenced elsewhere in libuv (e.g. uv_run()) which makes thing more complex. Is my approach feasible or should I re-initialize the uv_poll_t handle in such case ? Appreciate any feedback.
Possible approach , simplified sample code :
int uv_poll_change_fd( uv_poll_t *handle, int new_fd ) {
if (uv__fd_exists(handle->loop, new_fd))
// ..... some code ....
err = uv__io_check_fd(handle->loop, new_fd);
if(err)
// ..... some code ....
err = uv__nonblock(new_fd, 1);
// ..... some code ....
handle->io_watcher.fd = new_fd;
}
while(byte_count != 0){
byte_count = SSL_read(conn,get_buffer,sizeof(get_buffer));
printf("%s",get_buffer);
write_to_file(get_buffer,html,byte_count); // func to write to file
}
I've been trying to write a http/https client using sockets and SSL in C. The task is to get the HTML file of the landing page of a given website into a file on my system. I've handled the HTTP redirections and I was able to read only a portion of the HTTP payload since I've only called recv/SSL_read once. When I put this in a while loop it reads a few more 16kb segments and the connection times out. Is there any other way I can obtain whole of the HTML file ? (Sorry if this question seems vague, I'll be glad to make edits according to your responses)
I will give a detailed exp of the program and lead to the issue regarding the use of netlink socket communication.
The last paragraph asks the actual question I need an answer for, so you might wanna start by peeking it first.
Disclaimer before I start:
- I have made an earlier search before asking here and did not find complete solution / alternative to my issue.
- I know how to initialize a module and insert it to kernel.
- I know to handle communication between module and user-space without using netlink sockets. Meaning using struct file_operations func pointers assignments to later be invoked by the module program whenever a user attempts to read/write etc. and answer to the user using copy_to_user / copy_from_user.
- This topic refers to Linux OS, Mint 17 dist.
- Language is C
Okay, so I am building a system with 3 components:
1. user.c : user application (user types commands here)
2. storage.c : storage device ('virtual' disk-on-key)
3. device.ko : kernel module (used as proxy between 1. and 2.)
The purpose of this system is to be able (as a user) to:
- Copy files to the virtual disk-on-key device (2) - like an "upload" from local directory that belongs to the user.
- Save files from the virtual device on local directory - like "download" from the device storage to the user directory.
Design:
Assuming programs (1),(2) are compiled and running + (3) has successfully inserted using the bash command ' sudo insmod device.ko ' , the following should work like this (simulation ofc):
Step 1 (in user.c) -> user types 'download file.txt'
Step 2 (in device.ko) -> the device recognizes the user have tried to 'write' to it (actually user just passing the string "download file.txt") and invokes the 'write' implementation of the method we set on struct file_operation earlier on module_init().
The device (kernel module) now passes the data (string with a command) to the storage.c application, expecting an answer to later be retrieved to the user.c application.
Step 3 (in storage.c) -> now, lets say this program performs a busy-wait loop of 'readmsg()' and that's how a request from module event is triggered and recognized, the storage device now recognizes that the module has sent a request (string with a command \ data). Now, the storage programs shall perform an implementation of some function 'X' to send the data requested using sendmsg() somewhere inside the function.
Now, here comes the issue.
Usually, on all of the examples I've looked on web, the communication between the kernel-module and a user-space (or the storage.c program in our case) using netlink is triggered by the user-space and not vice versa. Meaning that the sendmsg() function from the user-space invokes the 'request(struct sk_buff *skb)' method (which is set on the module_init() part as following:
struct netlink_kernel_cfg cfg = {
.input = request // when storage.c sends something, it invokes the request function
};
so when the storage.c performs something like:
sendmsg(sock_fd,&msg,0); // send a msg to the module
the module invokes and runs the:
static void request(struct sk_buff *skb) {
char *msg ="Hello from kernel";
msg_size=strlen(msg);
netlink_holder=(struct nlmsghdr*)skb->data;
printk(KERN_INFO "Netlink received msg payload:%s\n",(char*)nlmsg_data(netlink_holder));
pid = netlink_holder->nlmsg_pid; // pid of sending process
skb_out = nlmsg_new(msg_size,0);
if(!skb_out){
printk(KERN_ERR "Failed to allocate new skb\n");
return;
}
netlink_holder=nlmsg_put(skb_out,0,0,NLMSG_DONE,msg_size,0); // add a new netlink message to an skb. more info: http://elixir.free-electrons.com/linux/v3.2/source/include/net/netlink.h#L491
NETLINK_CB(skb_out).dst_group = 0; // not in multicast group
strncpy(nlmsg_data(netlink_holder),msg,msg_size); // assign data as char* (variable msg)
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage. more info: http://elixir.free-electrons.com/linux/latest/source/include/net/netlink.h#L598
if(result<0)
printk(KERN_INFO "Error while sending bak to user\n");
}
and from all that big chunk, the only thing that im interesting in is actually doing this:
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage.
BUT I can't use nlmsg_unicast() without having the strcut sk_buff* which is provided automatically for me whenever there's an invoke from storage.c !
To sum up everything:
How do I send a msg from the device.ko (kernel module) to the user-space withtout having to wait for request to invoke / rely on the provided strcut sk_buff parameter from the earlier shown 'request()' method ?
Hope this sums up the point.
Thanks.
The only question here is that you need the user-space program connected to kernel-space first to get the pid of your user-program.
After get the pid, you can manually construct the skb_out and send it out through netlink_unicast or nlmsg_unicast.
The pid is always needed, you can set it as static and let your user-space program connect to your device.ko to make a long-maintained link.
Although this question is asked at 2017, I believe OP has already found the answer :D
my program get events from remote systems, every event contains an timestamp.
I want to log this events to syslog using the event timestamp instead of systemtime.
Is there any way to send a custom header to syslog deamon ?
I'm using rsyslog on debian
EDIT:
The "events" are generated by some "bare-metal" devices.
My application is a gateway between a realtime-ethernet (EthernetPOWERLINK) and a normal network.
I want to save them in micro-second precision, because its important to know in wich sequence they are occoured.
So i need the exact timestamp created by the bare-metal devices.
I'like to put this events into syslog.
I did not found any lib (except syslog.h) to write into syslog).
I really need to build the packages myself and send them to rsyslog deamon ?
No, don't open that can of worms.
If you allow the sender to specify the timestamp, you allow an attacker to spoof the timestamps of events they wish to hide. That kind of defeats the entire purpose (security-wise) of using a separate machine for logging.
What you can do, however, is compare the current time and the timestamp, and include that at the start of every logged message, using something like
struct timespec now;
struct timespec timestamp;
double delta;
int priority = facility | level;
const char *const message;
delta = difftime(timestamp.tv_sec, now.tv_sec)
+ ((double)timestamp.tv_nsec - now.tv_nsec) / 1000000000.0;
syslog(priority, "[%+.0fs] %s\n", delta, message);
On a typically configured Linux machine, that should produce something similar to
Jan 18 08:01:02 hostname service: [-1s] Original message
assuming the message took at least half a second to arrive. If hostname has its clock running fast, the delta would be positive. Normally, the delta is zero. In the case of a very slow network, the delta is negative, since the original event happened in the past relative to the timestamp shown.
If you already have infrastructure in place to monitor the logged messages, you can have a daemon or a cron script read the log files, and generate new log files (not via syslog(), but simply with string and file operations) with the timestamps adjusted by the specified delta. However, that must be done with extreme care, recognizing unacceptable or unexpectedly changing deltas, or maybe flagging them somehow.
If you write your log file monitoring/display widgets, then you can very easily let the user switch between "actual" (syslog) or "derived" (syslog + delta) timestamps, as the delta is trivial to extract from the logged lines if always present; even then, you must be careful to let the user know if a delta is out of bounds or changes unexpectedly, as such a change is most always informative to the user. (If it is not nefarious, it does mean there is something iffy with the machine timekeeping; time should not just jump around. Even NTP adjustments should be quite smooth.)
If you insist on opening that can of worms, just produce your own log files. Many applications do. It's not like syslog() was a magic bullet or a strict requirement for reliable logging, after all.
If your log-receiving application runs as a specific user and group, you can create /var/log/yourlogs/ owned by root user and that group, and save your log files there. Set the directory mode to 02770 (drwxrws--- or u=rwx,g=rwxs,o=), and all files created in that directory will automatically be owned by the same group (that's what the setgid bit, s, does for directories). You just need to make sure your service sets umask to 002 (and uses 0666 or 0660 mode flags when creating log files), so that they stay group-readable and group-writable.
Log rotation (archiving and/or deleting old log files, mailing logs) is usually a separate service, provided by the logrotate package, and configured by dropping a service-specific configuration file in /etc/logrotate.d/ at installation time. In other words, even if you write your own log files, do not rotate them; use the existing service for this. It makes life much easier for your users, us system administrators. (Note: Setting umask 002 at the start of the log rotate scripts is very useful in the above directory case; created files will then be group-writable. umask 022 will make them group-read-only.)
Ok've solved this, by enabling networking support (TCP) and micro seconds timer in rsyslog configuration.
Accroding to RFC 5424 my application build raw syslog messages and sends them via TCP (port 514) to the deamon.
Thanks to Nominal Animal, but i've no choice...
You can write a raw log message to the /dev/log file. This is a Unix domain socket from where the syslog server reads the messages, as they are written with the syslog() function.
I'm not sure about portability since the message format written by syslog() does not seem to follow the RFC 5424. I can only share my findings with busybox and its syslogd and nc utilities.
syslog() function writes messages as datagrams in the form <PRI>Mon DD HH:MM:SS message, where PRI is a priority, i.e. a decimal number computed as facility | severity, followed by a timestamp and a message.
With nc -u local:/dev/log, you can write UDP datagrams to the domain socket directly. For example, writing <84>Apr 3 07:27:20 hello world results in a Apr 3 07:27:20 hostname authpriv.warn hello world line in /var/log/messages.
Then you are free to extend the timestamp with the microseconds precision. Anyway, you need to make sure your syslog server implementation accepts such form. In case of busybox, I had to modify the source code.
Note: Busybox needs to be configured with enabled CONFIG_NC_EXTRA, CONFIG_NC_110_COMPAT and CONFIG_FEATURE_UNIX_LOCAL options to allow for opening /dev/log with nc.
I am attempting to run a fastcgi app written in C language behind the Nginx web server. The web browser never finishes loading and the response never completes. I am not sure how to approach it and debug. Any insight would be appreciated.
The hello world application was taken from fastcgi.com and simplified to look like this:
#include "fcgi_stdio.h"
#include <stdlib.h>
int main(void)
{
while(FCGI_Accept >= 0)
{
printf("Content-type: text/html\r\nStatus: 200 OK\r\n\r\n");
}
return 0;
}
Output executable is executed with either one of:
cgi-fcgi -connect 127.0.0.1:9000 a.out
or
spawn-fcgi -a120.0.0.1 -p9000 -n ./a.out
Nginx configuration is:
server {
listen 80;
server_name _;
location / {
# host and port to fastcgi server
root /home/user/www;
index index.html;
fastcgi_pass 127.0.0.1:9000;
}
}
You need to call FCGI_Accept in the while loop:
while(FCGI_Accept() >= 0)
You have FCGI_Accept >= 0 in your code. I think that results in the address of the FCGI_Accept function being compared to 0. Since the function exists, the comparison is never false, but the function is not being invoked.
Here's a great example of nginx, ubuntu, c++ and fastcgi.
http://chriswu.me/blog/writing-hello-world-in-fcgi-with-c-plus-plus/
If you want to run his code, I've put it into a git repo with instructions. You can check it out and run it for yourself. I've only tested it on Ubuntu.
https://github.com/homer6/fastcgi
After your application handles fastcgi-requests correctly, you need to take care of starting the application. nginx will never spawn fcgi processes itself, so you need a daemon taking care of that.
I recommend using uwsgi for managing fcgi processes. It is capable of spawning worker-processes that are ready for input, and restarting them when they die. It is highly configurable and easy to install and use.
http://uwsgi-docs.readthedocs.org/en/latest/
Here is my config:
[uwsgi]
fastcgi-socket = /var/run/apc.sock
protocol = fastcgi
worker-exec = /home/app/src/apc.bin
spooler = /home/app/spooler/
processes = 15
enable-threads = true
master = true
chdir = /home/app/
chmod-socket = 777
This integrates nicely as systemd service, but can also run without.
Try with:
$ cgi-fcgi -start -connect localhost:9000 ./hello
It works for me.
I'm using archlinux and following the instructions at:
https://wiki.archlinux.org/index.php/Nginx
You can try this
https://github.com/Taymindis/ngx-c-handler
It is built on top on fastcgi, It handle multiple request, and there are some core feature as well. It can handler function mapping with nginx.
To startup a nginx with c/c++ language
https://github.com/Taymindis/ngx-c-handler/wiki/How-to-build-a-cpp-service-as-c-service-interface