i tried to calling DynamoDB low-level API's from C code. This is my code
#include <stdio.h>
#include <curl/curl.h>
int main(void)
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
struct curl_slist *chunk = NULL;
chunk = curl_slist_append(chunk, "Host: dynamodb.us-east-1.amazonaws.com;");
chunk = curl_slist_append(chunk, "Accept-Encoding: identity;");
chunk = curl_slist_append(chunk, "Content-Length: 53;");
chunk = curl_slist_append(chunk, "User-Agent: CustomApp42;");
chunk = curl_slist_append(chunk, "Content-Type: application/x-amz-json-1.0;");
chunk = curl_slist_append(chunk, "Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=<Headers>, Signature=<signature>;");
chunk = curl_slist_append(chunk, "X-Amz-Date: 4.4.2016 ;");
chunk = curl_slist_append(chunk, "X-Amz-Target: DynamoDB_20120810.GetItem;");
res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
curl_easy_setopt(curl, CURLOPT_URL, "dynamodb.us-east-1.amazonaws.com");
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "{\"TableName\":\"Pets\",\"Key\":{\"AnimalType\":{\"S\": \"Dog\"},\"Name\": {\"S\": \"Fido\"}}}");
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
/* free the custom headers */
curl_slist_free_all(chunk);
}
return 0;
}
But it produce error while running like
HTTP/1.1 400 Bad Request
I faced mainly two issues.
i have aws_access_key_id and aws_secret_access_key. How to create Authorization (SignedHeaders & Signature) using these two credentials?
How to modify "X-Amz-Target" for dynamo query method?
Are you able to use the AWS SDK for C++? It will do all this heavy lifting and more for you. Actually, even if you are making a C program, you can expose an extern C library structure to the AWS SDK calls that you need. So, either way, I recommend coding against the AWS SDK for C++, and create an extern C library wrapper in case you need to compile your main program as a C program.
Related
I am trying to generate curl get request using c program .Here I need to store the response in a variable and I tried with the following code.
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
function_pt(void *ptr, size_t size, size_t nmemb, void *stream){
char **response_ptr = (char**)stream;
*response_ptr = strndup(ptr, (size_t)(size *nmemb));
}
int main(void)
{
CURL *curl;
CURLcode res;
char *response =calloc(1,sizeof(char));
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, function_pt);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
res=curl_easy_perform(curl);
curl_easy_cleanup(curl);
printf("%s\n",response);
}
return 0;
}
The data I get form http get request is real time,so i need to continue with the get request and store value in a variable in an iterative manner,only so that I can use the data in all other parts of the program.But the following code works only once and then exit.
How can I do it? Are there any other methods to generate http get request?
You don't need to calloc() the pointer if you are going to strndup() the original string, assuming that the response is a string is not good because that is not necessarily true.
I would suggest a structure where you can also store the length of the response, so if it's not text but for example a jpeg file nothing bad will happen, and you should not call printf() unless you check from the response headers that the response is indeed text, and it will be nul terminated afaik.
I am currently writing a REST-like client that only is required to do PUT requests.
Problem:
Running the program is not giving me the correct results on the URL's API and I do not know why.
Using curl_easy_perform(curl) does not throw an error when called. But the expected result is not generated on the URL's API.
Using curl_easy_send(curl,..,..,..) throws a : unsupported protocol error
Assumption:
I am assuming the order in which I am using the curl_easy_opts is a problem? And I am even missing a couple of key lines?
I have been reading on here of how other people do PUT requests and have been using their methods.
Summary of Program:
My program prompts the user for some string/character data, and from that, I construct the strings myself such as the header and the payload. The header and payload are both in JSON format but the payload is simply a string ( in this case, a char *str = (char *)mallo.. etc). How the header is constructed is shown below.
My header is being constructed using
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
//there is more content being appended to the header
The CURL function calls :
//init winsock stuff
curl_global_init(CURL_GLOBAL_ALL);
//get a curl handle
curl = curl_easy_init();
if(curl){
//append the headers
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
//specify the target URL
curl_easy_setopt(curl, CURLOPT_URL, url);
//connect ( //i added this here since curl_easy_send() says it requires it. )
curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY,1L);
//specify the request (PUT in our case)
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
//append the payload
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, payload);
res = curl_easy_perform(curl);
//res = curl_easy_send(curl, payload, strlen(payload),&iolen);
//check for errors
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
curl_easy_cleanup(curl);
}
You should not be using the CURLOPT_CONNECT_ONLY option or curl_easy_send() function, those are intended to be used for custom, non-HTTP protocols.
See this page for an example of how to do a PUT request with libcurl. Basically, you want to enable the CURLOPT_UPLOAD and CURLOPT_PUT options to say that you're doing a PUT request and to enable uploading a body with the request, and then you set the CURLOPT_READDATA and CURLOPT_INFILESIZE_LARGE options to tell libcurl how to read the data you're uploading and how big the data is.
In your case, if you already have the data in memory, then you don't need to read it out of a file, and you can just memcpy() it inside your read callback.
Example code copied below:
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) 1998 - 2012, Daniel Stenberg, <daniel#haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at http://curl.haxx.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
***************************************************************************/
#include <stdio.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <curl/curl.h>
/*
* This example shows a HTTP PUT operation. PUTs a file given as a command
* line argument to the URL also given on the command line.
*
* This example also uses its own read callback.
*
* Here's an article on how to setup a PUT handler for Apache:
* http://www.apacheweek.com/features/put
*/
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t retcode;
curl_off_t nread;
/* in real-world cases, this would probably get this data differently
as this fread() stuff is exactly what the library already would do
by default internally */
retcode = fread(ptr, size, nmemb, stream);
nread = (curl_off_t)retcode;
fprintf(stderr, "*** We read %" CURL_FORMAT_CURL_OFF_T
" bytes from file\n", nread);
return retcode;
}
int main(int argc, char **argv)
{
CURL *curl;
CURLcode res;
FILE * hd_src ;
struct stat file_info;
char *file;
char *url;
if(argc < 3)
return 1;
file= argv[1];
url = argv[2];
/* get the file size of the local file */
stat(file, &file_info);
/* get a FILE * of the same file, could also be made with
fdopen() from the previous descriptor, but hey this is just
an example! */
hd_src = fopen(file, "rb");
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if(curl) {
/* we want to use our own read function */
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);
/* enable uploading */
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
/* HTTP PUT please */
curl_easy_setopt(curl, CURLOPT_PUT, 1L);
/* specify target URL, and note that this URL should include a file
name, not only a directory */
curl_easy_setopt(curl, CURLOPT_URL, url);
/* now specify which file to upload */
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
/* provide the size of the upload, we specicially typecast the value
to curl_off_t since we must be sure to use the correct data size */
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE,
(curl_off_t)file_info.st_size);
/* Now run off and do what you've been told! */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
fclose(hd_src); /* close the local file */
curl_global_cleanup();
return 0;
}
I agree, don't use CUSTOMREQUEST. One detail that is being missed on every related to PUT and CURL I've seen here is that you NEED to set the file size, otherwise you'll get HTTP error 411.
Use CURLOPT_INFILESIZE or CURLOPT_INFILESIZE_LARGE for that.
See more details here:
How do I send long PUT data in libcurl without using file pointers?
I know this is a very old question, but in case somebody want to use libcurl with GLib and json-glib to send a JSON with PUT request.
Code below works for me:
#include <curl/curl.h>
#include <json-glib/json-glib.h>
//this is callback function for CURLOPT_READFUNCTION:
static size_t
curlPutJson ( void *ptr, size_t size, size_t nmemb, void *_putData )
{
GString *putData = ( GString * ) _putData;
size_t realsize = ( size_t ) putData->len;
memcpy ( ptr, putData->str, realsize );
return realsize;
}
/*now inside main or other function*/
//json_to_string ( jsonNode, FALSE ) is from json-glib to stringify JSON
//created in jsonNode
GString *putData = g_string_new ( json_to_string ( mainNode, FALSE ) );
//now goes curl as usual: headers, url, other options and so on
//and 4 most important lines
curl_easy_setopt ( curl, CURLOPT_READFUNCTION, curlPutJson );
curl_easy_setopt ( curl, CURLOPT_UPLOAD, 1L );
curl_easy_setopt ( curl, CURLOPT_READDATA, putData ); //GString
curl_easy_setopt ( curl, CURLOPT_INFILESIZE, putData->len ); //type long
I am currently working on a project that requires using C to make an http get request. I am trying to do this using curl. However, I get a response that says
error: unable to request data from https://coinex.pw/api/v2/currencies:
Unsupported protocol
I am not sure if the error is coming from curl or from the server. Here is my code, borrowed from example code:
#include <curl/curl.h>
static char *request(const char *url)
{
CURL *curl = NULL;
CURLcode status;
struct curl_slist *headers = NULL;
char *data = NULL;
long code;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if(!curl)
goto error;
data = malloc(BUFFER_SIZE);
if(!data)
goto error;
struct write_result write_result = {
.data = data,
.pos = 0
};
curl_easy_setopt(curl, CURLOPT_URL, url);
headers = curl_slist_append(headers, "Content-type: application/json");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_response);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &write_result);
status = curl_easy_perform(curl);
if(status != 0)
{
fprintf(stderr, "error: unable to request data from %s:\n", url);
fprintf(stderr, "%s\n", curl_easy_strerror(status));
goto error;
}
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &code);
if(code != 200)
{
fprintf(stderr, "error: server responded with code %ld\n", code);
goto error;
}
curl_easy_cleanup(curl);
curl_slist_free_all(headers);
curl_global_cleanup();
/* zero-terminate the result */
data[write_result.pos] = '\0';
return data;
error:
if(data)
free(data);
if(curl)
curl_easy_cleanup(curl);
if(headers)
curl_slist_free_all(headers);
curl_global_cleanup();
return NULL;
}
Any tips / hints are welcome.
curl (and libcurl) gives an unsupported protocol error when they can't interpret the protocol part of the URL. In your case that means https:, which is a bit odd.
First check you can you use the curl tool from the command line to retrieve the URL.
curl -V will give you a list of the protocols curl (and thus libcurl) will support:
$ curl -V
curl 7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP
Check that https is there. It may be that your libcurl is not built against an SSL library or if it is that the SSL library is not installed.
Finally on this page: http://curl.haxx.se/libcurl/c/example.html you will note that is a simple https example (second one down). Please confirm that works with your libcurl. If so, I'd find out what your program is doing different. If not, I would find out what's wrong with your libcurl installation.
try to use "http" but "https" in your URL.
I have developed a ‘C’ application on a Linux box using the libcurl example http://curl.askapache.com/c/ftpupload.html I work fine. I have been asked to use SSL for both “data encryption for both the control channel and data channel.” I am not been able to find an example of adding SSL to the example I followed. Here is the core of the FTP program:
// get a FILE * of the same file
hd_src = fopen(local_file, "rb");
// curl init
curl_global_init(CURL_GLOBAL_ALL);
// get a curl handle
curl = curl_easy_init();
if (curl) { // build a list of commands to pass to libcurl
headerlist = curl_slist_append(headerlist, buf_1);
#ifdef DEBUG
// we want to use our own read function
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);
#endif
// enable uploading
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
// specify target
curl_easy_setopt(curl, CURLOPT_URL, ftp_url);
curl_easy_setopt(curl, CURLOPT_USERPWD, user_password);
curl_easy_setopt(curl, CURLOPT_PORT, 21);
// pass in that last of FTP commands to run after the transfer
curl_easy_setopt(curl, CURLOPT_POSTQUOTE, headerlist);
// now specify which file to upload
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
// Set the size of the file to upload
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t) fsize);
// Now run off and do what you've been told!
res = curl_easy_perform(curl);
// Check for errors
if (res != CURLE_OK) {
char *s;
s = malloc((sizeof(char) * 100) + 1);
sprintf(s, "curl_easy_perform() failed: %s - Error Number: %d\n",
curl_easy_strerror(res), res);
returnResults->error = true;
returnResults->errorMessage = s;
return returnResults;
}
// clean up the FTP commands list
curl_slist_free_all(headerlist);
// always cleanup
curl_easy_cleanup(curl);
}
fclose(hd_src); // close the local file
curl_global_cleanup();
There's however an existing example code on the libcurl site showing how you do FTP-SSL to download a file: ftpsget.c - it shows the very little SSL magic you need to add. That magic is the same for uploads:
/* We activate SSL and we require it for both control and data */
curl_easy_setopt(curl, CURLOPT_USE_SSL, CURLUSESSL_ALL);
In fact, you can add this single line into the FTP upload example. and it'll do FTPS uploads for you.
When I post to a certain URL via libcURL most of the time the status code will be a 401, but sometimes it will return the 200 I expect even though the credentials I pass are the same. The weird thing is that if I use the same code ported to python and using urllib2 I will get a 200 ok and I will get the data I need. Here is the relevant code that I am using to post to the website. libcURL also works fine for me to get the needed credentials for this function such as the xut_sid key and cookie, it is just this url that sometimes 401s and sometimes 200s.
CURL *curl_handle;
CURLcode res;
struct curl_slist *header = NULL;
MemoryStruct chunk;
...
header = curl_slist_append(header, l->cookie);// l just contains the data I need
header = curl_slist_append(header, l->xut_sid);
header = curl_slist_append(header, "Content-Type: application/json;charset=utf-8");
header = curl_slist_append(header, "x-http-method-override: GET");
curl_handle = curl_easy_init();
curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl_handle, CURLOPT_URL, searchString);
curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &chunk);
res = curl_easy_perform(curl_handle);
EDIT: I fixed it, twas a dumb mistake by me where the xut_sid would get overwritten(I'm new to C lol and didn't use strdup to copy the string) and not be sent. I don't know why it showed up in the verbose log as there, but it wasn't being sent.