What's the most efficient way to get source code of web page in C? - c

In PHP I can do it as simple as :
file_get_contents('http://stackoverflow.com/questions/ask');
What's the shortest code to do the same in C?
UPDATE
When I compile the sample with curl, got errors like this:
unresolved external symbol __imp__curl_easy_cleanup referenced in function _main

Use libcurl, refer to their example C snippets
#include <stdio.h>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "curl.haxx.se");
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}

Try the libcurl C interface

I should have commented the Richard Harrison good answer, but I have not 50 reputations points yet, so I put here as an answer my hint to ieplugin for compiling the code:
On Ubuntu 10.04 (and supposing you named the source file getpage.c):
sudo apt-get install libcurl4-dev
gcc getpage.c -lcurl -o getpage

Related

compiling c language program with libcurl on windows

I am trying to compile a simple c program on windows 10 using gcc from the libcurl website I cloned vcpkg and then ran the .bat file , next I installed curl with the command vcpkg install curl and got this output
Computing installation plan...
The following packages are already installed:
curl[core,non-http,openssl,schannel,ssl,sspi]:x86-windows -> 7.80.0
Package curl:x86-windows is already installed
Restored 0 packages from C:\Users\<me>\AppData\Local\vcpkg\archives in 138.1 us. Use --debug to see more details.
Total elapsed time: 386.9 ms
The package curl provides CMake targets:
find_package(CURL CONFIG REQUIRED)
target_link_libraries(main PRIVATE CURL::libcurl)
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
int report()
{
CURL* curl;
CURLcode res;
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if (curl) {
/* First set the URL that is about to receive our POST. This URL can
just as well be a https:// URL if that is what should receive the
data. */
curl_easy_setopt(curl, CURLOPT_URL, "http://192.168.1.12:8000");
const char* c = const_cast<char*>(output.c_str());
printf ("%s", c);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, c );
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return 0;
}
int main()
{
report();
return 0;
}
Now when I try to compile the code above I get the error
c_example_curl.c:3:23: fatal error: curl/curl.h: No such file or directory
#include <curl/curl.h>
^
compilation terminated.
I am compiling using the command
gcc -lcurl file.c
I also tried using the command
gcc -lcurl -I F:\_C_\vcpkg\installed\x86-windows\include\curl file.c
doesn't seem to change anything
I even tried to copy the curl header file to the working directory of the above code but I was not lucky
How do you install libcurl for windows isn't there any command equivalent to the linux one
download curl for windows here.
unpack the zip and place it anywhere on the system (i personally prefer C:\curl).
then compile your script with: gcc file.c -I<path-to-curl> -L<path-to-curl>\lib -lcurl
NOTE: replace <path-to-curl> with the curl location on the system, for example C:\curl

Using curl in C on Windows

I have now followed multiple "guides" on how to use curl in a C project on windows with little to no success. I have installed and build a static build of curl using vs-code and I end up in with the following problem, when I build the following code:
#define CURL_STATICLIB
#include <stdio.h>
#include "curl/curl.h"
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "https://example.com");
/* example.com is redirected, so we tell libcurl to follow redirection */
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
return 0;
}
I get:
>gcc libcurl_test.c
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0xc3): undefined reference to `curl_easy_init'
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0xf6): undefined reference to `curl_easy_setopt'
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0x11a): undefined reference to `curl_easy_setopt'
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0x126): undefined reference to `curl_easy_perform'
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0x13d): undefined reference to `curl_easy_strerror'
C:\Users\JAKOBV~1.SOF\AppData\Local\Temp\ccsyNorR.o:libcurl_test.c:(.text+0x165): undefined reference to `curl_easy_cleanup'
collect2.exe: error: ld returned 1 exit status
I saw some that solved the problem using gcc libcurl_test.c -lcurl however then all I get is c:/mingw/bin/../lib/gcc/mingw32/6.3.0/../../../../mingw32/bin/ld.exe: cannot find -lcurl
I am quite unexperienced when it comes to using C and especially libraries, what am I doing wrong? My file structure is as follows:
And inside the curl folder is the following files:
Please leave a comment if I should add more information, I am not sure what is of interest and what is not....
Merely including curl/curl.h lets the compiler know what functions exist, but it doesn't actually add those functions to the executable. For that, you also need to link with the library that contains the actual implementation of these functions, namely curl/libcurl_a.lib.
To tell gcc to link with libcurl_a.lib, because it is a static library, just put its path on the command line:
gcc libcurl_test.c curl/libcurl_a.lib

Linker problems with libcurl in eclipse C\Cpp

im trying to use libcurl in Eclipse C\Cpp and cross compiling it to a Raspberry Pi for a university project. I have included the header files and added the -lcurl option in the compiler.
#include <stdio.h>
#include <unistd.h> //Used for UART
#include <fcntl.h> //Used for UART
#include <termios.h> //Used for UART
#include "anybus.h"
#include <string.h>
#include <time.h>
#include <wiringPi.h>
#include "curl.h"
int main()
{
CURL *curl;
CURLcode res;
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, "charsets: utf-8");
char* string = "{\"name\":\"JohnDoe\"}";
printf(string);
// In windows, this will init the winsock stuff
res = curl_global_init(CURL_GLOBAL_ALL);
// get a curl handle
curl = curl_easy_init();
if(curl) {
// First set the URL that is about to receive our POST. This URL can
// just as well be a https:// URL if that is what should receive the
// data.
curl_easy_setopt(curl,CURLOPT_HTTPHEADER, headers );
curl_easy_setopt(curl, CURLOPT_URL, "http://134.61.41.2:8080/Cascade/rest/maschinen");
// Now specify the POST data
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, string);
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if(res != CURLE_OK){
printf("Curl Easy Perform failed\n");
}
// always cleanup
curl_easy_cleanup(curl);
}
curl_global_cleanup();
}
However when I compile it I get undefined references to most of the curl functions, but not the curl objects CURL *curl and CURLcode res.
Building target: RaspAnybusDriver_2007.a
Invoking: Cygwin C++ Linker
arm-unknown-linux-gnueabi-g++ -L"c:\cygwin\opt\cross\x-tools\arm-unknown-linux- gnueabi\arm-unknown-linux-gnueabi\sysroot\usr\lib" -L"C:\cygwin\raspberry\usr\lib" -L"C:\cygwin\lib\python2.7" -L"C:\cygwin\lib" -o "RaspAnybusDriver_2007.a" ./anybus.o ./bytefloat.o ./main.o -lrt -lcurl.dll -lwiringPi
./main.o: In function `main':
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:17: undefined reference to `curl_slist_append'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:18: undefined reference to `curl_slist_append'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:19: undefined reference to `curl_slist_append'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:26: undefined reference to `curl_global_init'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:29: undefined reference to `curl_easy_init'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:34: undefined reference to `curl_easy_setopt'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:35: undefined reference to `curl_easy_setopt'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:37: undefined reference to `curl_easy_setopt'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:40: undefined reference to `curl_easy_perform'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:47: undefined reference to `curl_easy_cleanup'
/cygdrive/c/Users/Bjorn/workspace/RaspAnybusDriver_2007/Debug/../main.c:49: undefined reference to `curl_global_cleanup'
collect2: ld returned 1 exit status
make: *** [RaspAnybusDriver_2007.a] Error 1
Does anyone have a clue to whats going on?
After looking at compilation log it seems that the linker is not able to find the curl.. functions. So verify at the build setting in eclipse.

Linking curl library problems

I'm trying to build curl library sample program that downloads file from ftp. I'm using Eclipse IDE , OS -Ubuntu.
I have installed curl using command:
apt-get install libcurl4-gnutls-dev
I see heades files in /usr/include/curl (I don't know where are c files)
Looks Eclipse is happy with #include <curl/curl.h> , bu all curl functions used in program are maked with 'undefined reference'.
Compilation failes with linking errors:
**** Build of configuration Debug for project updDown ****
make all
Building target: updDown
Invoking: GCC C Linker
gcc -o "updDown" ./src/updDown.o
./src/updDown.o: In function `main':
/home/g/proj/updDown/Debug/../src/updDown.c:45: undefined reference to `curl_global_init'
/home/g/proj/updDown/Debug/../src/updDown.c:47: undefined reference to `curl_easy_init'
/home/g/proj/updDown/Debug/../src/updDown.c:52: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:55: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:57: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:60: undefined reference to `curl_easy_setopt'
/home/g/proj/updDown/Debug/../src/updDown.c:62: undefined reference to `curl_easy_perform'
/home/g/proj/updDown/Debug/../src/updDown.c:65: undefined reference to `curl_easy_cleanup'
/home/g/proj/updDown/Debug/../src/updDown.c:76: undefined reference to `curl_global_cleanup'
collect2: error: ld returned 1 exit status
make: *** [updDown] Error 1
**** Build Finished ****
How to solve this problem?
Hwole code:
/*
============================================================================
Name : updDown.c
Author :
Version :
Copyright : Your copyright notice
Description : Hello World in C, Ansi-style
============================================================================
*/
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
struct FtpFile {
const char *filename;
FILE *stream;
};
static size_t my_fwrite(void *buffer, size_t size, size_t nmemb, void *stream)
{
struct FtpFile *out=(struct FtpFile *)stream;
if(out && !out->stream) {
/* open file for writing */
out->stream=fopen(out->filename, "wb");
if(!out->stream)
return -1; /* failure, can't open file to write */
}
return fwrite(buffer, size, nmemb, out->stream);
}
int main(void)
{
puts("starting");
CURL *curl;
CURLcode res;
struct FtpFile ftpfile={
"curl.tar.gz", /* name to store the file as if succesful */
NULLs
};
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
if(curl) {
/*
* You better replace the URL with one that works!
*/
curl_easy_setopt(curl, CURLOPT_URL,
"ftp://ftp.example.com/pub/www/utilities/curl/curl-7.9.2.tar.gz");
/* Define our callback to get called when there's data to be written */
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, my_fwrite);
/* Set a pointer to our struct to pass to the callback */
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &ftpfile);
/* Switch on full protocol/debug output */
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
if(CURLE_OK != res) {
/* we failed */
fprintf(stderr, "curl told us %d\n", res);
}
}
if(ftpfile.stream)
fclose(ftpfile.stream); /* close the local file */
curl_global_cleanup();
return 0;
}
You shall specify the linker option of -lcurl, at the location of C/C++ build -> Settings -> GCC C Linker -> Libraries in project properties.
EDITED:
Let's be more detailed.
I see heades files in /usr/include/curl (I don't know where are c files)
Typically such packages don't consist of source codes, instead they provide pre-compiled object files called libraries.bu all curl functions used in program are maked with 'undefined reference'.
Looks Eclipse is happy with #include
That's true since the path /usr/include/ is typically inside the include path (where the compiler tries to find the header files), so you don't need to setup anything for this.
bu all curl functions used in program are maked with 'undefined reference'.
In the code you use functions like curl_global_init without implementing them yourself, which means those functions are treated as external functions that are expected to be imported. You shall "tell" the linker where to find these functions (to be more exact, these symbols). That's done by using the option -l followed by the library name. And for specifying the paths of libraries, use -L.
For further information, you could see the section of linker options in Option Sammary of GCC

Saving a file using libcurl in C

I'm expanding from perl to C and I'm trying to use curl's library to simply save a file from a remote url but I'm having a hard time finding a good example to work from.
Also, I'm not sure if I should be using curl_easy_recv or curl_easy_perform
I find this resource very developer friendly.
I compiled the source code below with:
gcc demo.c -o demo -I/usr/local/include -L/usr/local/lib -lcurl
Basically, it will download a file and save it on your hard disk.
File demo.c
#include <curl/curl.h>
#include <stdio.h>
void get_page(const char* url, const char* file_name)
{
CURL* easyhandle = curl_easy_init();
curl_easy_setopt( easyhandle, CURLOPT_URL, url ) ;
FILE* file = fopen( file_name, "w");
curl_easy_setopt( easyhandle, CURLOPT_WRITEDATA, file) ;
curl_easy_perform( easyhandle );
curl_easy_cleanup( easyhandle );
fclose(file);
}
int main()
{
get_page( "http://blog.stackoverflow.com/wp-content/themes/zimpleza/style.css", "style.css" ) ;
return 0;
}
Also, I believe your question is similar to this one:
Download file using libcurl in C/C++

Resources