Libcurl with digest authorization in C: send http requests periodically - c

I'm developing a system that tracks objects with a P(an)T(ilt)Z(oom) camera which can be controlled via HTTP requests. The C application I develop is supposed to receive position data of the tracked object and to send commands to the camera to control the pan and tilt angle. In addition to these commands the camera has to receive a session refresh command every 5 seconds. HTTP Digest Authorization has to be used for the connection.
I'm sending the HTTP request with libcurl. I figured already out that for digest auth one needs to use on and the same curl handle for all requests in this stackoverflow post.
For sending the session refresh command periodically I tried to use a thread which is just doing this:
while(1)
{
usleep(5000000);
sessionContinue(g_Config.cam_ip);
}
With sessionContinue looking like this:
CURLcode sessionContinue(char* url)
{
CURLcode res;
char requestURL[40];
char referer[47];
struct curl_slist *headers=NULL;
strcpy(requestURL , url);
strcat(requestURL, CAM_SESSION_CONTINUE);
strcpy(referer , "Referer: http://");
strcat(referer , url);
strcat(referer , CAM_MONITOR);
headers = curl_slist_append(headers,"Connection:keep-alive");
headers = curl_slist_append(headers, camCookie);
// In windows, this will init the winsock stuff
curl_global_init(CURL_GLOBAL_ALL);
curl_easy_reset(curl);
if(curl)
{
// First set the URL that is about to receive our POST. This URL can
//just as well be a https:// URL if that is what should receive the
//data.
curl_easy_setopt( curl , CURLOPT_URL , requestURL );
curl_easy_setopt( curl , CURLOPT_HTTPHEADER , headers );
curl_easy_setopt( curl , CURLOPT_HTTPGET , 1 );
curl_easy_setopt( curl , CURLOPT_USERNAME , "root" );
curl_easy_setopt( curl , CURLOPT_PASSWORD , "password" );
curl_easy_setopt( curl , CURLOPT_HTTPAUTH , CURLAUTH_BASIC | CURLAUTH_DIGEST );
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed # %s:%d : %s\n", curl_easy_strerror(res) , __FILE__ , __LINE__ );
}
return res;
}
The application always crashed with segmentation fault after executing curl_easy_perform(curl). So I read the libcurl tutorial again and now I know that using one curl handle in multiple threads is a no go.
What I tried then was to use a timer with SIGALRM to implement the periodic session refresh. This didn't change the problem with the crash at curl_easy_perform(curl). The strange thing is that the application doesn't crash when sending the normal command to control the pan and tilt position which uses the same curl handle. The only difference between session refresh and pan/tilt command is that session refresh uses GET and pan/tilt uses POST.
Are there any other possibilities to send pan/tilt commands continuously with a short pause every 5 seconds used to send the session refresh?

You have a long range of problems in one small program. Here's a few:
You might overflow one of those small fixed-size buffers with the dangerous unbounded C functions you use. Quite likely one of them is the reason for the segfault.
curl_global_init() is documented to be called once, you call it over and over again - this even without calling curl_global_cleanup() in between. You obviously call curl_easy_init() somewhere out of the function and you should move the global init there.
'referer' gets filled with data but is never used otherwise
Another advice is to use CURLOPT_ERRORBUFFER to get error messages in rather than curl_easy_strerror() as you may get some extra details then. And of course to set CURLOPT_VERBOSE while debugging the request to see that things look the way you want it.

Thanks for your comment Daniel Stenberg. I'm now calling curl_global_init() just once when the handle has been set up. referer wasn't really needed here, but I had forgotten to remove it before pasting the code here.
The reason for the segmentation fault was that the session refresh command and the commands for pan and tilt tried to use one and the same curl handle at the same time, which obviously can't really work. So the solution with the timer and SIGALRM wasn't the problem. The segmentation faults have been solved by adding a mutex lock to avoid concurrent accesses to the curl handle.

Related

How to receive large HTML data using SSL_read

while(byte_count != 0){
byte_count = SSL_read(conn,get_buffer,sizeof(get_buffer));
printf("%s",get_buffer);
write_to_file(get_buffer,html,byte_count); // func to write to file
}
I've been trying to write a http/https client using sockets and SSL in C. The task is to get the HTML file of the landing page of a given website into a file on my system. I've handled the HTTP redirections and I was able to read only a portion of the HTTP payload since I've only called recv/SSL_read once. When I put this in a while loop it reads a few more 16kb segments and the connection times out. Is there any other way I can obtain whole of the HTML file ? (Sorry if this question seems vague, I'll be glad to make edits according to your responses)

Autonomically sending a message from kernel-module to user-space application without relying on the invoke of input. from user-space

I will give a detailed exp of the program and lead to the issue regarding the use of netlink socket communication.
The last paragraph asks the actual question I need an answer for, so you might wanna start by peeking it first.
Disclaimer before I start:
- I have made an earlier search before asking here and did not find complete solution / alternative to my issue.
- I know how to initialize a module and insert it to kernel.
- I know to handle communication between module and user-space without using netlink sockets. Meaning using struct file_operations func pointers assignments to later be invoked by the module program whenever a user attempts to read/write etc. and answer to the user using copy_to_user / copy_from_user.
- This topic refers to Linux OS, Mint 17 dist.
- Language is C
Okay, so I am building a system with 3 components:
1. user.c : user application (user types commands here)
2. storage.c : storage device ('virtual' disk-on-key)
3. device.ko : kernel module (used as proxy between 1. and 2.)
The purpose of this system is to be able (as a user) to:
- Copy files to the virtual disk-on-key device (2) - like an "upload" from local directory that belongs to the user.
- Save files from the virtual device on local directory - like "download" from the device storage to the user directory.
Design:
Assuming programs (1),(2) are compiled and running + (3) has successfully inserted using the bash command ' sudo insmod device.ko ' , the following should work like this (simulation ofc):
Step 1 (in user.c) -> user types 'download file.txt'
Step 2 (in device.ko) -> the device recognizes the user have tried to 'write' to it (actually user just passing the string "download file.txt") and invokes the 'write' implementation of the method we set on struct file_operation earlier on module_init().
The device (kernel module) now passes the data (string with a command) to the storage.c application, expecting an answer to later be retrieved to the user.c application.
Step 3 (in storage.c) -> now, lets say this program performs a busy-wait loop of 'readmsg()' and that's how a request from module event is triggered and recognized, the storage device now recognizes that the module has sent a request (string with a command \ data). Now, the storage programs shall perform an implementation of some function 'X' to send the data requested using sendmsg() somewhere inside the function.
Now, here comes the issue.
Usually, on all of the examples I've looked on web, the communication between the kernel-module and a user-space (or the storage.c program in our case) using netlink is triggered by the user-space and not vice versa. Meaning that the sendmsg() function from the user-space invokes the 'request(struct sk_buff *skb)' method (which is set on the module_init() part as following:
struct netlink_kernel_cfg cfg = {
.input = request // when storage.c sends something, it invokes the request function
};
so when the storage.c performs something like:
sendmsg(sock_fd,&msg,0); // send a msg to the module
the module invokes and runs the:
static void request(struct sk_buff *skb) {
char *msg ="Hello from kernel";
msg_size=strlen(msg);
netlink_holder=(struct nlmsghdr*)skb->data;
printk(KERN_INFO "Netlink received msg payload:%s\n",(char*)nlmsg_data(netlink_holder));
pid = netlink_holder->nlmsg_pid; // pid of sending process
skb_out = nlmsg_new(msg_size,0);
if(!skb_out){
printk(KERN_ERR "Failed to allocate new skb\n");
return;
}
netlink_holder=nlmsg_put(skb_out,0,0,NLMSG_DONE,msg_size,0); // add a new netlink message to an skb. more info: http://elixir.free-electrons.com/linux/v3.2/source/include/net/netlink.h#L491
NETLINK_CB(skb_out).dst_group = 0; // not in multicast group
strncpy(nlmsg_data(netlink_holder),msg,msg_size); // assign data as char* (variable msg)
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage. more info: http://elixir.free-electrons.com/linux/latest/source/include/net/netlink.h#L598
if(result<0)
printk(KERN_INFO "Error while sending bak to user\n");
}
and from all that big chunk, the only thing that im interesting in is actually doing this:
result=nlmsg_unicast(sock_netlink,skb_out,pid); // send data to storage.
BUT I can't use nlmsg_unicast() without having the strcut sk_buff* which is provided automatically for me whenever there's an invoke from storage.c !
To sum up everything:
How do I send a msg from the device.ko (kernel module) to the user-space withtout having to wait for request to invoke / rely on the provided strcut sk_buff parameter from the earlier shown 'request()' method ?
Hope this sums up the point.
Thanks.
The only question here is that you need the user-space program connected to kernel-space first to get the pid of your user-program.
After get the pid, you can manually construct the skb_out and send it out through netlink_unicast or nlmsg_unicast.
The pid is always needed, you can set it as static and let your user-space program connect to your device.ko to make a long-maintained link.
Although this question is asked at 2017, I believe OP has already found the answer :D

POST JSON to url using CURL and C

I'm tring to POST JSON data to a url from bash using:
$ curl -v -d '{xxx:200}&apikey=xxxxx' -X POST http://localhost/xxxx/input/post.json -H "Accept: application/json" -H "Content-Type:application/json"
And in C using the following:
int main(void)
{
CURL *easyhandle;
curl_global_init(CURL_GLOBAL_ALL);
easyhandle = curl_easy_init();
if(easyhandle) {
char *data="json={xxx:200}&apikey=xxxxx";
curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, data);
curl_easy_setopt(easyhandle, CURLOPT_URL, "http://localhost/xxxx/input/post.json");
curl_easy_perform(easyhandle);
curl_easy_cleanup (easyhandle);
}
curl_global_cleanup();
return 0;
}
This is what i'm trying to achieve actually:
http://localhost/xxxx/input/post.json?json={xxx:200}&apikey=xxxxx
It doesn't seem to work. :(
I'm a complete novice to curl. Please help.
Thanks!
Fortunately the server I was sending the data handled POST and GET requests, so the code in question would suffice.
Others with a similar problem can use a simple workaround (If your code doesn't have real time constraints and is not performance intensive). You may fork a bash process using system() from C. This avoids you the trouble of encoding.
What you are trying to do is not performing a POST request but performing a GET request. However, I'm not sure this is a good idea, since GET parameters are limited in length (to something like 2 kB or so), and - as others have already mentioned - they need to be encoded and decoded and all the funky stuff which is a pain in the neck.
The URL pointing to localhost suggests me that you have control over the server code. If you used the POST parameters instead of the GET ones, you could use your current code as-is (which sets the POST body of the request, which is probably the right thing to do -- so you don't have to change your client code, you only have to change the server code.)

libcurl clean CURLOPT_POSTFIELDS?

Hello what I am trying to do is send post method twice, however when I send it a second time the information from the first time is also being included and I do not want that.
To illustrate what I mean, this is the code that sends using post method. (the handle curl was already created)
void process(char* transferBuffer) {
curl_easy_setopt(curl, CURLOPT_URL, "http://localhost/cpp.php");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, transferBuffer);
res = curl_easy_perform(curl);
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
If I do something like:
process("name=John"); - webserver receives name=John
process("name=El"); - webserver receives name=John AND name=El
What I want to do is somehow clean previously used data;
the curl handle was already created ... What I want to do is somehow clean previously used data
All I can say is that if you want to reuse your curl handle - which is a best practice, you should reset it with curl_easy_reset before re-setting your options and re-performing the transfer.
Note that without the complete sample code (including the creation of your curl handle, etc) it is quite hard to provide a detailed answer.

Does the libcurl library give any way to determine which response header came from which command?

Background:
I'm working on my first C program with the library and I need to gather responses from each command sent to a SMTP server.
I've gotten as far as sending commands to the SMTP server and printing the response headers using curl_easy_setopt(curl_handle, CURLOPT_HEADERFUNCTION, parse_head), but I'm using multi threaded options. It is not at all clear when I get a response which command it was caused by. I am assuming that they will not necessarily be received in the same order sent. Is that correct?
Making it more difficult, since the library handles some calls (like setting up the initial connection) without my explicit request, I would need to handle more headers than explicit requests. That would be predictable and repeatable, but definitely adds an extra level of complexity.
Question:
Is there a "good" way to determine exactly which command resulted in which response header using multi thread?
Also, moderately related, does the library support returning the numeric return code or do I have to manually parse that out? Looking through the library, it seems that it doesn't. I just want to be sure.
I am assuming that they will not necessarily be received in the same order sent. Is that correct?
Yes, it is. That's how multithreading works.
Is there a "good" way to determine exactly which command resulted in which response header using multi thread?
Yes. You can set user data (context info, whatever you call it) using the CURLOPT_HEADERDATA option - this will be passed in as the 4th argument of your header function. So you can write code like this:
CURL *hndl = curl_easy_init();
// ...
curl_easy_setopt(hndl, CURLOPT_HEADERFUNCTION, parse_head);
curl_easy_setopt(hndl, CURLOPT_HEADERDATA, some_pointer_that_identifies_the_thread);
// ...
size_t parse_head(void *buf, void *size_t sz, size_t nmemb, void *context)
{
// context will be the pointer identifying the thread
}
does the library support returning the numeric return code or do I have to manually parse that out?
Yes, it does:
long httpStatus;
curl_easy_getinfo(hndl, CURLINFO_RESPONSE_CODE, &httpStatus);
if (200 <= httpStatus && httpStatus < 300) {
// HTTP 2XX OK
} else {
// Error (4XX, 5XX) or redirect (3XX)
}

Resources