I'm expanding from perl to C and I'm trying to use curl's library to simply save a file from a remote url but I'm having a hard time finding a good example to work from.
Also, I'm not sure if I should be using curl_easy_recv or curl_easy_perform
I find this resource very developer friendly.
I compiled the source code below with:
gcc demo.c -o demo -I/usr/local/include -L/usr/local/lib -lcurl
Basically, it will download a file and save it on your hard disk.
File demo.c
#include <curl/curl.h>
#include <stdio.h>
void get_page(const char* url, const char* file_name)
{
CURL* easyhandle = curl_easy_init();
curl_easy_setopt( easyhandle, CURLOPT_URL, url ) ;
FILE* file = fopen( file_name, "w");
curl_easy_setopt( easyhandle, CURLOPT_WRITEDATA, file) ;
curl_easy_perform( easyhandle );
curl_easy_cleanup( easyhandle );
fclose(file);
}
int main()
{
get_page( "http://blog.stackoverflow.com/wp-content/themes/zimpleza/style.css", "style.css" ) ;
return 0;
}
Also, I believe your question is similar to this one:
Download file using libcurl in C/C++
Related
I am trying to compile a simple c program on windows 10 using gcc from the libcurl website I cloned vcpkg and then ran the .bat file , next I installed curl with the command vcpkg install curl and got this output
Computing installation plan...
The following packages are already installed:
curl[core,non-http,openssl,schannel,ssl,sspi]:x86-windows -> 7.80.0
Package curl:x86-windows is already installed
Restored 0 packages from C:\Users\<me>\AppData\Local\vcpkg\archives in 138.1 us. Use --debug to see more details.
Total elapsed time: 386.9 ms
The package curl provides CMake targets:
find_package(CURL CONFIG REQUIRED)
target_link_libraries(main PRIVATE CURL::libcurl)
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
int report()
{
CURL* curl;
CURLcode res;
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if (curl) {
/* First set the URL that is about to receive our POST. This URL can
just as well be a https:// URL if that is what should receive the
data. */
curl_easy_setopt(curl, CURLOPT_URL, "http://192.168.1.12:8000");
const char* c = const_cast<char*>(output.c_str());
printf ("%s", c);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, c );
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return 0;
}
int main()
{
report();
return 0;
}
Now when I try to compile the code above I get the error
c_example_curl.c:3:23: fatal error: curl/curl.h: No such file or directory
#include <curl/curl.h>
^
compilation terminated.
I am compiling using the command
gcc -lcurl file.c
I also tried using the command
gcc -lcurl -I F:\_C_\vcpkg\installed\x86-windows\include\curl file.c
doesn't seem to change anything
I even tried to copy the curl header file to the working directory of the above code but I was not lucky
How do you install libcurl for windows isn't there any command equivalent to the linux one
download curl for windows here.
unpack the zip and place it anywhere on the system (i personally prefer C:\curl).
then compile your script with: gcc file.c -I<path-to-curl> -L<path-to-curl>\lib -lcurl
NOTE: replace <path-to-curl> with the curl location on the system, for example C:\curl
I have downloaded a compiled curl for Windows (the .a files) and linked it to my project. The problem is, the compiled .exe is very large (2,41MB) just for this simple program:
#include <stdio.h>
#include <curl/curl.h>
size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream) {
size_t written = fwrite(ptr, size, nmemb, stream);
return written;
}
int main(void)
{
CURL *curl;
FILE *fp;
CURLcode res;
char *url = "https://website/app.exe";
char outfilename[FILENAME_MAX] = "file.exe";
curl = curl_easy_init();
if (curl) {
fp = fopen(outfilename,"wb");
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
fclose(fp);
}
return 0;
}
Now, I assume that is because curl I have linked to my project has many functions enabled in itself/contains many functions which I don't need. Is there any way to enable only HTTPS (and HTTP by that?) and with that make the program size smaller? I have found something like that on the curl website (https://curl.haxx.se/docs/install.html - Disabling Specific Protocols in Windows builds), but I didn't quite understand what should I do. I tried putting the "HTTP_ONLY" in "defines" in CodeBlocks but it did nothing. Probably curl has to be compiled somehow with that option. What can I do?
Any help is appreciated, thank you!
Compile your application in release mode and turn on all strips, compiling an application in release mode will reduce its size due to removal of debugging symbols
You have to compile libcurl yourself and configure it. configuring a library could be a little tricky, but it worth, you can fully customize it so disabling unused features let you reduce library size, also don't remember to compile curl without debugging informations.
compile all staffs with different optimization flags, say -Os is preferable but you should check with other optimization levels to verify.
use shared libraries as much as possible
I am using libcurl for one of my projects. I know that curl is not used to make multiple concurrent requests but does libcurl support it?
I know there are other tools like ab but that there are many features that libcurl provides.
Again I know I can use curl within script to run multiple requests,but that's not what I am looking for.
I could not find a satisfactory answer for this expect this one. Although, It's not conclusive.
I should be able to use multiple handles for multiple connections.
Has anyone tried this? Are there any gotchas I need to look out for?
I should be able to do something like this:
my_app --total_connections 1000 --concurrency 100 <Other libcurl options> url
To test what you are looking for, i wrote a little C program. It executes 10 http-get requests using libcurl in a loop. The loop is parallelized using openmp (if available).
To run it, just save it in a file called for example parallel_curl_test.cpp and compile it two times. First using g++ parallel_curl_test.cpp -fopenmp $(pkg-config --libs --cflags libcurl) -o parallel_curl for the parallel version and a second time using g++ parallel_curl_test.cpp $(pkg-config --libs --cflags libcurl) -o sequential_curl without openmp for the sequential version.
Here is the code:
#include <cmath>
#include <stdio.h>
#include <curl/curl.h>
#include <time.h>
void curl_request();
size_t write_data(void *, size_t, size_t, void *);
static struct timeval tm1;
static int num_requests = 10;
static inline void start()
{
gettimeofday(&tm1, NULL);
}
static inline void stop()
{
struct timeval tm2;
gettimeofday(&tm2, NULL);
unsigned long long t = 1000 * (tm2.tv_sec - tm1.tv_sec) + (tm2.tv_usec - tm1.tv_usec) / 1000;
printf("%d requests in %llu ms\n",num_requests , t);
}
int main()
{
start();
#pragma omp parallel for
for(int n=0; n<num_requests; ++n){
curl_request();
}
stop();
return 0;
}
void curl_request()
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
res = curl_easy_perform(curl);
if(res != CURLE_OK)
fprintf(stderr, "curl_request() failed: %s\n",
curl_easy_strerror(res));
curl_easy_cleanup(curl);
}
}
size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp)
{
return size * nmemb;
}
The output for ./parallel_curl will look like this:
10 requests in 657 ms
the output for ./sequential_curl will look something like:
10 requests in 13794 ms
As you can see, the parallel_curl which uses concurrency finished significantly faster than sequential_curl which ran sequential.
Thus the answer to your question is : Yes!
Of course, sequential execution could be done much more efficient using pipelining, keep-alives and reusage of resources. But this is another question.
I'm trying to build curl library sample program that downloads file from ftp. I'm using Eclipse IDE , OS -Ubuntu.
I have installed curl using command:
apt-get install libcurl4-gnutls-dev
I see heades files in /usr/include/curl (I don't know where are c files)
Looks Eclipse is happy with #include <curl/curl.h> , bu all curl functions used in program are maked with 'undefined reference'.
Compilation failes with linking errors:
**** Build of configuration Debug for project updDown ****
make all
Building target: updDown
Invoking: GCC C Linker
gcc -o "updDown" ./src/updDown.o
./src/updDown.o: In function `main':
/home/g/proj/updDown/Debug/../src/updDown.c:45: undefined reference to `curl_global_init'
/home/g/proj/updDown/Debug/../src/updDown.c:47: undefined reference to `curl_easy_init'
/home/g/proj/updDown/Debug/../src/updDown.c:52: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:55: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:57: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:60: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:62: undefined reference to `curl_easy_perform'
/home/g/proj/updDown/Debug/../src/updDown.c:65: undefined reference to `curl_easy_cleanup'
/home/g/proj/updDown/Debug/../src/updDown.c:76: undefined reference to `curl_global_cleanup'
collect2: error: ld returned 1 exit status
make: *** [updDown] Error 1
**** Build Finished ****
How to solve this problem?
Hwole code:
/*
============================================================================
Name : updDown.c
Author :
Version :
Copyright : Your copyright notice
Description : Hello World in C, Ansi-style
============================================================================
*/
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
struct FtpFile {
const char *filename;
FILE *stream;
};
static size_t my_fwrite(void *buffer, size_t size, size_t nmemb, void *stream)
{
struct FtpFile *out=(struct FtpFile *)stream;
if(out && !out->stream) {
/* open file for writing */
out->stream=fopen(out->filename, "wb");
if(!out->stream)
return -1; /* failure, can't open file to write */
}
return fwrite(buffer, size, nmemb, out->stream);
}
int main(void)
{
puts("starting");
CURL *curl;
CURLcode res;
struct FtpFile ftpfile={
"curl.tar.gz", /* name to store the file as if succesful */
NULLs
};
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
if(curl) {
/*
* You better replace the URL with one that works!
*/
curl_easy_setopt(curl, CURLOPT_URL,
"ftp://ftp.example.com/pub/www/utilities/curl/curl-7.9.2.tar.gz");
/* Define our callback to get called when there's data to be written */
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, my_fwrite);
/* Set a pointer to our struct to pass to the callback */
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ftpfile);
/* Switch on full protocol/debug output */
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
if(CURLE_OK != res) {
/* we failed */
fprintf(stderr, "curl told us %d\n", res);
}
}
if(ftpfile.stream)
fclose(ftpfile.stream); /* close the local file */
curl_global_cleanup();
return 0;
}
You shall specify the linker option of -lcurl, at the location of C/C++ build -> Settings -> GCC C Linker -> Libraries in project properties.
EDITED:
Let's be more detailed.
I see heades files in /usr/include/curl (I don't know where are c files)
Typically such packages don't consist of source codes, instead they provide pre-compiled object files called libraries.bu all curl functions used in program are maked with 'undefined reference'.
Looks Eclipse is happy with #include
That's true since the path /usr/include/ is typically inside the include path (where the compiler tries to find the header files), so you don't need to setup anything for this.
bu all curl functions used in program are maked with 'undefined reference'.
In the code you use functions like curl_global_init without implementing them yourself, which means those functions are treated as external functions that are expected to be imported. You shall "tell" the linker where to find these functions (to be more exact, these symbols). That's done by using the option -l followed by the library name. And for specifying the paths of libraries, use -L.
For further information, you could see the section of linker options in Option Sammary of GCC
In PHP I can do it as simple as :
file_get_contents('http://stackoverflow.com/questions/ask');
What's the shortest code to do the same in C?
UPDATE
When I compile the sample with curl, got errors like this:
unresolved external symbol __imp__curl_easy_cleanup referenced in function _main
Use libcurl, refer to their example C snippets
#include <stdio.h>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "curl.haxx.se");
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}
Try the libcurl C interface
I should have commented the Richard Harrison good answer, but I have not 50 reputations points yet, so I put here as an answer my hint to ieplugin for compiling the code:
On Ubuntu 10.04 (and supposing you named the source file getpage.c):
sudo apt-get install libcurl4-dev
gcc getpage.c -lcurl -o getpage