I am trying to upload in C/C++ code to S3 using libcurl. I am uploading binary data to be generated from a callback.
I've been staring at this for a while, so I may be way off :( but here is what my code looks like right now:
First, I build my form like so:
struct curl_httppost *buildForm(void *streamData, long contentLength) {
struct curl_httppost *formpost=NULL;
struct curl_httppost *lastptr=NULL;
for( int i=0; i<form.size(); ++i ) {
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, form[i].first.c_str(),
CURLFORM_COPYCONTENTS, form[i].second.c_str(),
CURLFORM_END);
}
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "file",
CURLFORM_STREAM, streamData,
CURLFORM_CONTENTSLENGTH, (long) contentLength,
CURLFORM_FILENAME, "chunk",
//CURLFORM_CONTENTTYPE, "application/octet-stream",
CURLFORM_END);
return formpost;
}
Then I setup the request like this:
curl_easy_reset( handle );
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Expect:");
curl_easy_setopt( handle, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt( handle, CURLOPT_VERBOSE, 1L );
curl_easy_setopt( handle, CURLOPT_DEBUGFUNCTION, curlDebugger );
curl_easy_setopt( handle, CURLOPT_URL, postInstructions.url.c_str() );
curl_easy_setopt( handle, CURLOPT_POST, 1L);
// set the data and header read callbacks:
curl_easy_setopt( handle, CURLOPT_READFUNCTION, &(UploadCallback::writeDataStatic));
curl_easy_setopt( handle, CURLOPT_HTTPPOST, formpost);
//I have tried with and without this line
curl_easy_setopt( handle, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)contentLength);
curl_easy_setopt( handle, CURLOPT_WRITEFUNCTION, &(XonamiResponse::readDataStatic));
curl_easy_setopt( handle, CURLOPT_WRITEDATA, &response);
curl_easy_setopt( handle, CURLOPT_HEADERFUNCTION, &(XonamiResponse::readHeaderStatic));
curl_easy_setopt( handle, CURLOPT_WRITEHEADER, &response);
// do it!
int success = curl_easy_perform( handle );
However, curl complains:
operation aborted by callback
even though I am generating the same number of bytes as contentLength. When I setup my callback to just fill the buffers with zero as long as data is requested, S3 responds with:
The body of your POST request is not well-formed multipart/form-data.
and looking at the raw form it looks to me like the last (closing) boundary is missing.
curl --version says:
curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz
It looks like the issue is in my callback which was inadvertently returning abort instead of CURL_READFUNC_ABORT when it was done. Oops.
Related
I am trying to generate curl get request using c program .Here I need to store the response in a variable and I tried with the following code.
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
function_pt(void *ptr, size_t size, size_t nmemb, void *stream){
char **response_ptr = (char**)stream;
*response_ptr = strndup(ptr, (size_t)(size *nmemb));
}
int main(void)
{
CURL *curl;
CURLcode res;
char *response =calloc(1,sizeof(char));
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, function_pt);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
res=curl_easy_perform(curl);
curl_easy_cleanup(curl);
printf("%s\n",response);
}
return 0;
}
The data I get form http get request is real time,so i need to continue with the get request and store value in a variable in an iterative manner,only so that I can use the data in all other parts of the program.But the following code works only once and then exit.
How can I do it? Are there any other methods to generate http get request?
You don't need to calloc() the pointer if you are going to strndup() the original string, assuming that the response is a string is not good because that is not necessarily true.
I would suggest a structure where you can also store the length of the response, so if it's not text but for example a jpeg file nothing bad will happen, and you should not call printf() unless you check from the response headers that the response is indeed text, and it will be nul terminated afaik.
I'm writing my server code in C on my Mac, and I need it to access Facebook's Graph API at the URL "https://graph.facebook.com/app?access_token=[insert access token here]". I'm getting a CURLE_COULDNT_RESOLVE_HOST error, maybe because I've never done this before and am probably doing something totally wrong. I'm setting up a CURL* and connecting like this:
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, true);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 2);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8");
char* httpURLFormat = "https://graph.facebook.com/app?access_token=%s";
char* httpURL = emalloc(sizeof(char)*(strlen(httpURLFormat)+MAX_TOKEN_LENGTH)); //emalloc is just malloc that makes sure there is free memory
sprintf(httpURL, httpURLFormat, data->password); //data->password is the access token
curl_easy_setopt(curl, CURLOPT_URL, &httpURL);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
struct url_data response; //this struct contains int size and char* data
response.size = 0;
response.data = emalloc(4096);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
CURLcode curlCode = curl_easy_perform(curl);
if (curlCode!=CURLE_OK){
printf("curl failed!!!\n"); //this ends up printing
free(response.data);
free(httpURL);
free(httpURLFormat);
return false;
}
//Yes, I free everything left over afterwards...
I tested the URL it's connecting to in my web browser using the same user agent my code uses, and it connects without trouble. I tried adding a '\n' character to the end of the URL, and my code still won't work.
I found the problem: The httpURLFormat should be "https://graph.facebook.com:443/app?access_token=%s". You need to specify the port in the URL. CURL apparently doesn't use the default 443 otherwise.
I have developed a ‘C’ application on a Linux box using the libcurl example http://curl.askapache.com/c/ftpupload.html I work fine. I have been asked to use SSL for both “data encryption for both the control channel and data channel.” I am not been able to find an example of adding SSL to the example I followed. Here is the core of the FTP program:
// get a FILE * of the same file
hd_src = fopen(local_file, "rb");
// curl init
curl_global_init(CURL_GLOBAL_ALL);
// get a curl handle
curl = curl_easy_init();
if (curl) { // build a list of commands to pass to libcurl
headerlist = curl_slist_append(headerlist, buf_1);
#ifdef DEBUG
// we want to use our own read function
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);
#endif
// enable uploading
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
// specify target
curl_easy_setopt(curl, CURLOPT_URL, ftp_url);
curl_easy_setopt(curl, CURLOPT_USERPWD, user_password);
curl_easy_setopt(curl, CURLOPT_PORT, 21);
// pass in that last of FTP commands to run after the transfer
curl_easy_setopt(curl, CURLOPT_POSTQUOTE, headerlist);
// now specify which file to upload
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
// Set the size of the file to upload
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t) fsize);
// Now run off and do what you've been told!
res = curl_easy_perform(curl);
// Check for errors
if (res != CURLE_OK) {
char *s;
s = malloc((sizeof(char) * 100) + 1);
sprintf(s, "curl_easy_perform() failed: %s - Error Number: %d\n",
curl_easy_strerror(res), res);
returnResults->error = true;
returnResults->errorMessage = s;
return returnResults;
}
// clean up the FTP commands list
curl_slist_free_all(headerlist);
// always cleanup
curl_easy_cleanup(curl);
}
fclose(hd_src); // close the local file
curl_global_cleanup();
There's however an existing example code on the libcurl site showing how you do FTP-SSL to download a file: ftpsget.c - it shows the very little SSL magic you need to add. That magic is the same for uploads:
/* We activate SSL and we require it for both control and data */
curl_easy_setopt(curl, CURLOPT_USE_SSL, CURLUSESSL_ALL);
In fact, you can add this single line into the FTP upload example. and it'll do FTPS uploads for you.
When I post to a certain URL via libcURL most of the time the status code will be a 401, but sometimes it will return the 200 I expect even though the credentials I pass are the same. The weird thing is that if I use the same code ported to python and using urllib2 I will get a 200 ok and I will get the data I need. Here is the relevant code that I am using to post to the website. libcURL also works fine for me to get the needed credentials for this function such as the xut_sid key and cookie, it is just this url that sometimes 401s and sometimes 200s.
CURL *curl_handle;
CURLcode res;
struct curl_slist *header = NULL;
MemoryStruct chunk;
...
header = curl_slist_append(header, l->cookie);// l just contains the data I need
header = curl_slist_append(header, l->xut_sid);
header = curl_slist_append(header, "Content-Type: application/json;charset=utf-8");
header = curl_slist_append(header, "x-http-method-override: GET");
curl_handle = curl_easy_init();
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl_handle, CURLOPT_URL, searchString);
curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &chunk);
res = curl_easy_perform(curl_handle);
EDIT: I fixed it, twas a dumb mistake by me where the xut_sid would get overwritten(I'm new to C lol and didn't use strdup to copy the string) and not be sent. I don't know why it showed up in the verbose log as there, but it wasn't being sent.
I'm trying to connect to a google api.
This works fine in my terminal, there I'm doing:
curl https://www.googleapis.com/tasks/v1/users/#me/lists --header "Authorization: Bearer myAccessCode".
This works fine, but now I want to make this inside a c program.
For this I have:
CURL *curl;
char *header = "Authorization: Bearer myAccessCode";
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, header);
curl = curl_easy_init();
char *response = NULL;
curl_easy_setopt(curl, CURLOPT_URL, "https://www.googleapis.com/tasks/v1/users/#me/lists");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, httpsCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
curl_easy_perform(curl);
curl_easy_cleanup(curl);
But here I'm just getting a message that a login is required.
I have no idea what I'm doing wrong, is there someone who sees my failure?
Like I wrote in the comment above:
I just realized that I made: curl_slist_append(headers, header);
instead of: headers = curl_slist_append(headers, header);
So headers was always NULL and I made the get request without a header.
(I edited it in my question above, so the code works, if some)