How to store curl responds in a variable in c - c

I am trying to generate curl get request using c program .Here I need to store the response in a variable and I tried with the following code.
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
function_pt(void *ptr, size_t size, size_t nmemb, void *stream){
char **response_ptr = (char**)stream;
*response_ptr = strndup(ptr, (size_t)(size *nmemb));
}
int main(void)
{
CURL *curl;
CURLcode res;
char *response =calloc(1,sizeof(char));
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, function_pt);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
res=curl_easy_perform(curl);
curl_easy_cleanup(curl);
printf("%s\n",response);
}
return 0;
}
The data I get form http get request is real time,so i need to continue with the get request and store value in a variable in an iterative manner,only so that I can use the data in all other parts of the program.But the following code works only once and then exit.
How can I do it? Are there any other methods to generate http get request?

You don't need to calloc() the pointer if you are going to strndup() the original string, assuming that the response is a string is not good because that is not necessarily true.
I would suggest a structure where you can also store the length of the response, so if it's not text but for example a jpeg file nothing bad will happen, and you should not call printf() unless you check from the response headers that the response is indeed text, and it will be nul terminated afaik.

Related

how do i POST an empty array with curl in c?

When trying to send an empty array using curl the array get's received as so emptyArr['']
with quote instead of being posted as an empty emptyArr[]
how do i post the emptyArr without the quote?
#include <curl/curl.h>
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if(curl) {
/* First set the URL that is about to receive our POST. This URL can
just as well be a https:// URL if that is what should receive the
data. */
curl_easy_setopt(curl, CURLOPT_URL, "http://someaddress.com");
/* Now specify the POST data */
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "emptyArr[]");
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
curl_global_cleanup();
Empty arrays don't really exist in URL-encoded parameters. When you send an array, it's sent as:
name[]=firstElement&name[]=secondElement&name[]=thirdElement
An empty array means you don't send any of these, but then there's no parameter at all.
It's the responsibility of the server code to handle the nonexisting parameter and treat it as an empty array.
When you write
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "emptyArr[]");
it's being treated as
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "emptyArr[]=");
so you're creating an array with one element whose value is an empty string.
You should simply leave out the parameter entirely, and the server should treat it as empty.

Getting 'access denied' when trying to access an airport RSS feed

I'm trying to access an airport RSS feed using the curl library in C. Whenever I try to access it, though, I get access denied error. The following is the code I'm using. It's almost identical to the example code at https://curl.haxx.se/libcurl/c/simple.html:
#include <stdio.h>
#include <curl/curl.h>
int main(int argc, char **argv)
{
CURL *curl;
CURLcode res;
char *feed_addr = "http://w1.weather.gov/xml/current_obs/KUCP.rss";
//airport not in the state I live
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, feed_addr);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if(res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",curl_easy_strerror(res));
}
// always cleanup
curl_easy_cleanup(curl);
}
return 0;
}
I've tried this with other websites besides weather.gov (google, youtube, mit.edu) and they all work just fine. But when I tried this, even with other airport RSS feeds (which can be found at http://w1.weather.gov/xml/current_obs/seek.php?state=pa&Find=Find), I get the same access denied error. I also get the same error when I set feed_addr to any weather.gov page.
To make this stranger, when I tried to access the feed using Python3's urllib.request module, it worked just fine. I can also access it just fine with Google Chrome. So I can rule out the idea that it doesn't want me accessing the feed.
Is there something that I'm missing? Is there a way to get the feed via the curl library? Or is there a way using a different library?
Answering my own question (I did not originally plan to, but I solved the issue before someone else answered):
So here is the code that I got to work
#include <stdio.h>
#include <curl/curl.h>
int main(int argc, char **argv)
{
CURL *curl;
CURLcode res;
char *feed_addr = "http://w1.weather.gov/xml/current_obs/KUCP.rss";
//airport not in the state I live
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, feed_addr);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
//line that solved my issue
curl_easy_setopt(curl, CURLOPT_USERAGENT, <string>);
/**
* The line as written will not work.
* I'm only using <string> as a stand-in for some personal information
* If you're having the same issue as I did, I explain how to solve it under the code
*/
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if(res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() failed: %s\n",curl_easy_strerror(res));
}
// always cleanup
curl_easy_cleanup(curl);
}
return 0;
}
The way I solved this was finding a site that printed out my HTTP request. In this case, it was http://rve.org.uk/dumprequest. I visited the site like normal in one tab, and I used the program to get the source, then launched that in another tab. That was when I saw that several fields were present when I manually opened the page, but absent when I used the code.
So, I looked at curl_easy_setopt to see if there was a way to set those fields. It turns out, curl_easy_setopt has documentation for that and many more at the website https://curl.haxx.se/libcurl/c/curl_easy_setopt.html. At the suggestion of one of the comments, I looked at CURLOPT_USERAGENT first.
Since that one took a char *, I found the line in the HTTP request that started with User-Agent:, copy-and-pasted the rest, and that was <string> in the above line.
So if the request included the line:
User-Agent: Lord Voldemort (Tom Marvolo Riddle)
The line I included would be:
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Lord Voldemort (Tom Marvolo Riddle)");

Libcurl C - Save information Array

I am new using libcurl and I need some help for simple procedures.
I want my program to read information from a website, which will return the information in the following format:
text | xxxxx | xxxxxxxxx
text | xxxxx | xxxxxxxxx
text | xxxxx | xxxxxxxxx
and I want to save that information into an array.
Then read the information from the array, but only the second and third column, just the numbers to use them with another function.
I know how to get the information using libcurl in C from the website and I get it printed in the terminal window, but I do not know how to save it into an array. I am stuck there...
For example with this code:
How would you save the information it returns into an array?
#include <stdio.h>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http:www.example.com");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}
I would appreciate any help I can get.
Thank you.
Take a look at CURLOPT_WRITEFUNCTION option of curl_easy_setopt(), and follow their examples.
size_t write_callback(char *ptr, size_t size, size_t nmemb, void *userdata);
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_WRITEFUNCTION, write_callback);

cURL getting a response

Im Finding the cURL Documentation to read/Understand. what im trying to do here is post a postcode and return whether or not it is valid.
This is my function so far (what i managed to pull together from tutorials)
int checkPostCode(char postCode[5])
{
printf("checkPostCode\n");
char uidPath[200] = "http://zim.cs.uow.edu.au/~dfs/cgi-bin/postcodes.cgi?pcode=";
//Concatinate poth strings
strcat (uidPath,postCode);
//Output URl
printf("%s \n ", uidPath);
CURL *curl;
CURLcode res;
// Create our curl handle
curl = curl_easy_init();
if (curl)
{
curl_easy_setopt(curl, CURLOPT_URL, uidPath);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}
the script returns either a "Yes" or a "No".
I just don't know what to enter into that if statement to get my result.
thanks in advance :).
It sounds like you want to read a response from the server. For that you'll need to enable the option CURLOPT_READFUNCTION and specify a callback that will receive the data from the network and store it in a array of char.

hide curl_easy_perform

How can I hide curl_easy_perform output (in a shell)?
This is in regards to a C application.
Use CURLOPT_NOBODY in curl_easy_setopt().
Example:
...
CURL *curl;
CURLcode statusCode;
curl = curl_easy_init();
if(curl){
curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com/");
//CURLOPT_NOBODY does the trick
curl_easy_setopt(curl, CURLOPT_NOBODY, 1);
curl_easy_perform(curl);
...
Link to docs: http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTNOBODY
Set the CURLOPT_WRITEFUNCTION and/or CURLOPT_WRITEDATA options:
FILE *f = fopen("target.txt", "wb");
curl_easy_setopt(handle, CURLOPT_WRITEDATA, f);
By default, libcurl writes output to stdout. When you override this (which is what almost any application will do), it will write to another file or to pass chunks of output to a callback. See the documentation for CURLOPT_WRITEFUNCTION for more details.
As Joey said, CURLOPT_WRITEFUNCTION will allow you to completely disregard all output. Just set up a callback that does absolutely nothing if you want the data to just go away, without being written to any file descriptor.
For instance,
/* Never writes anything, just returns the size presented */
size_t my_dummy_write(char *ptr, size_t size, size_t nmemb, void *userdata)
{
return size * nmemb;
}
Then in your options:
curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, &my_dummy_write);
Or, point the file handle at a NULL device (a lot easier).

Resources