Is it possible to use libcurl without file descriptor based socket? - c

I wrote the following libcurl code with my custom write and read socket functions:
//...
CURL *curl;
FILE *fp;
CURLcode res;
char outfilename[FILENAME_MAX] = "file";
curl = curl_easy_init();
/* no progress meter please */
//curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_data);
curl_easy_setopt(curl, CURLOPT_READDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_OPENSOCKETFUNCTION, opensocket);
curl_easy_setopt(curl, CURLOPT_OPENSOCKETDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, closecb);
curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_URL, hostname.c_str());
curl_easy_setopt(curl, CURLOPT_SOCKOPTFUNCTION, sockopt_callback);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
I didn't know what to do with CURLOPT_SOCKOPTFUNCTION so I did sockopt_callback which is
static curl_socket_t opensocket(void *clientp,
curlsocktype purpose,
struct curl_sockaddr *address)
{
return 0;
}
But I get
* Trying 212.183.159.230:80...
* Could not set TCP_NODELAY: Socket operation on non-socket
* connect to 212.183.159.230 port 80 failed: Socket operation on non-socket
libcurl wants to close 0 now
because probably it's trying to write to the socket 0. Apparently I can't prevent it from calling options on the socket, for example setting TCP_NODELAY and most important, connect.
Is it possible to make libcurl do not call the socket file descriptor ever?
One idea I had was to pass an existing socket to be a dummy for libcurl and then still use my write and read functions. However, libcurl simply send things through the socket, not the write and read functions

I thought write_data was a way to redirect the socket calls but it has nothing to do with it, it actually is a function that writes what the client receives.
However, by passing a socketpair, as sugested by #domen, I made it work!
Sketch:
//...
static curl_socket_t opensocket(void *clientp,
curlsocktype purpose,
struct curl_sockaddr *address)
{
auto *sharedValue = static_cast<SharedValue *>(clientp);
return sharedValue->curl_socket;
}
int socket_vector[2];
if (0 != socketpair(AF_UNIX, SOCK_STREAM, 0, socket_vector))
{
std::cout << "problem creating socketpair" << std::endl;
std::exit(2);
}
sharedValue.curl_socket = socket_vector[0];
sharedValue.vpn_socket = socket_vector[1];
//writes the result to a file
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data_file);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, destFile);
curl_easy_setopt(curl, CURLOPT_OPENSOCKETFUNCTION, opensocket);
curl_easy_setopt(curl, CURLOPT_OPENSOCKETDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_CLOSESOCKETFUNCTION, closecb);
curl_easy_setopt(curl, CURLOPT_CLOSESOCKETDATA, &sharedValue);
curl_easy_setopt(curl, CURLOPT_SOCKOPTFUNCTION, sockopt_callback);
curl_easy_setopt(curl, CURLOPT_URL, hostname.c_str());
//curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
//Start a thread here that will read/write to the socket pair on the other endpoint (in my case vpn_socket)
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
if (res)
{
std::cout << "libcurl error code: " << res << ", libcurl error: " << curl_easy_strerror(res) << std::endl;
return 1;
}

Related

c libcurl CURLOPT_POSTFIELDS strange error

i'm a quite new c developer and I have a little trouble with libcurl. I searched a lot in the net before but i didn't managed to find an answer working for me.
This is pure c, no ++.
I have this code and the request is failing
char * write_url(MyData data)
{
char *field = malloc(20 * sizeof(char));
if (strcmp(data.sensorName,"Temperature")==0){
sprintf(field,"&temperature=%.2f",data.measure);
}
CURL *curl;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_URL, "http://127.0.0.1/");
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, field);
curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
But if I change
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, field);
for
char *data="&temperature=20";
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
It's working.
But I want to pass the data through that funcion!
Thanks!
malloc() will not initialize the memory, therefore you have to do it. But if the if statement isn't taken the memory remains uninitialized. Do this in the else part of the code:
if (strcmp(data.sensorName,"Temperature")==0)
{
sprintf(field,"&temperature=%.2f",data.measure);
}
else
{
strcmp(field,"Unknown") ;
}
And free the memory when you are done:
free(field) ;
Also note that the libcurl calls always return an error value, use it.
curl = curl_easy_init(); //no error checking is perform in your code
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_URL, "http://127.0.0.1/");
curl_easy_setopt(curl, CURLOPT_POST, 1);

cURL Returns Error Due to Too Many Threads (or Sockets?) [C]

I'm writing a server that uses a thread pool. Each thread connects to an SQL database then calls accept on the same server socket once it starts and waits for a client to connect. I've found that if and only if I spawn too many threads, the curl_easy_perform function fails the very first time it is called, returning error code 56 (CURL_RECV_ERROR). The limit is somewhere between 950 and 970 threads, but my computer allows me to create up to 2047 threads for the program, so this limit seems arbitrary to me. I also am not low on RAM.
Just one thread calls the cURL function once before it gives me the error, so I'm not doing multiple cURL requests simultaneously in this test. Am I doing anything wrong, or do I have to accept a thread limit for cURL to work?
These are my functions for sending HTTPS requests and getting responses with cURL:
size_t write_data(void *ptr, size_t size, size_t nmemb, struct url_data *s){ // callback function I have to use with Curl
size_t new_len = s->len + size*nmemb;
s->ptr = erealloc(s->ptr, new_len+1);
memcpy(s->ptr+s->len, ptr, size*nmemb);
s->ptr[new_len] = '\0';
s->len = new_len;
return size*nmemb;
}
char* curlHTTPS(char* url){ // returns the response as a string or NULL upon error
url = strdup(url);
CURL* curl = curl_easy_init();
struct url_data* response = emalloc(sizeof(struct url_data));
response->len = 0;
response->ptr = emalloc(4096*sizeof(char));
curl_easy_setopt(curl, CURLOPT_URL, url); // Define target site
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, true);
curl_easy_setopt(curl, CURLOPT_USERAGENT, userAgent);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 2);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_HEADER, false);
curl_easy_setopt(curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, response);
CURLcode curlCode = curl_easy_perform(curl);
if (curlCode!=CURLE_OK){
if (logLevel >= LOG_ERROR) printf("Curl failed! Error: %d\n", curlCode);
curl_easy_cleanup(curl);
efree(response->ptr);
efree(response);
return NULL;
}
char* ret = strdup(response->ptr);
curl_easy_cleanup(curl);
efree(response);
return ret;
}
Sounds like you are opening lots of file handles. Have you checked your open file limit?
Socket accept - "Too many open files"

curl wsdl soap error - mismatched Actions between sender and receiver

I am trying to post the curl based soap request to url using c API's. My libcurl version is curl 7.30.0 (i686-pc-linux-gnu) libcurl/7.30.0 OpenSSL/1.0.0 zlib/1.2.5.I am receiving the below error.Also below code i am using to post the request.i am using correct soap action aswell "GetAuthToken", Why this error is happening while posting the request.Thanks in advance.
Error:
The message with Action '' cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and receiver) or a binding/security mismatch between the sender and the receiver. Check that sender and receiver have the same contract and the same binding (including security requirements, e.g. Message, Transport, None)
Request:
123456789
url:
http://x.xx.xx.xxx:20003/HIMS/SecurityService/?wsdl
Sample code:
int main()
{
CURL *curl;
CURLcode res;
struct curl_slist *headers = NULL;
FILE *out_fd = (FILE *) 0;
char errorbuf[300]="",filename[32]="Response.txt";
char errmsg[256];
int Timeout=120;
int buffer_size = 0;
char urlbuff[100]="";
char buff[128] = "http://x.xx.xx.xxx:20003/HIMS/SecurityService/?wsdl";
memset(urlbuff,0,sizeof(urlbuff));
curl = curl_easy_init();
buffer_size = strlen(buffer);
if(curl)
{
out_fd = fopen (filename, "w");
curl_easy_setopt(curl, CURLOPT_FILE, out_fd);
headers = curl_slist_append(headers, "Content-type:text/xml;charset=utf-8; SOAPAction=GetAuthToken");
sprintf(urlbuff,"%s",buff);
curl_easy_setopt(curl, CURLOPT_URL, urlbuff);
Timeout=2000;
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, buffer_size);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, buffer);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, Timeout);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER,errmsg);
res = curl_easy_perform(curl);
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
fclose(out_fd);
if(CURLE_OK != res)
{
printf("\nerrorbuf:%s:%d\n",errorbuf,res);
return -1;
}
return 0;
}
}
Assuming you have created a valid soap envelope for this request. In your code, you missed to define a POST request with your curl.
curl_easy_setopt(curl, CURLOPT_POST, 1L);

Submitting HTTP POST only works when CURLOPT_NOBODY is set to 1

I am trying to automate logging into a site via an HTTP form (using POST). This is the code I have so far. When a login is successful, the user should get a 302 redirect to another page. The code below works only if I have CURLOPT_NOBODY set to false. This code outputs https://mytestsite.com/success.jsp, but if I set CURLOPT_NOBODY to 1L, I simply get the url before the redirect (https://mytestsite.com/status.html). Why is this?
int login()
{
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL, "https://mytestsite.com/login.do");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_POST, 1L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/4.0");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "/home/cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "/home/cookies.txt");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "loginId=person1#mytestsite.com&password=mypassword");
// Disable output from curl_easy_perform
curl_easy_setopt(curl, CURLOPT_NOBODY, 0L);
res = curl_easy_perform(curl);
char *url;
curl_easy_getinfo(curl, CURLINFO_EFFECTIVE_URL, &url);
fprintf(stdout, "#####\n");
fprintf(stdout, "%s\n", url);
fprintf(stdout, "#####\n");
curl_easy_cleanup(curl);
return 0;
}

c libcurl piping HTTP GET and PUT

I want to make some kind of migrations for virtual hosts in XCP.
At the moment I do it by downloading the file with HTTP GET and then uploading it to another host with HTTP PUT, here is example of downloading
curl = curl_easy_init();
//Auto escape user password
password = curl_easy_escape(curl, XEN.user_passwd, 0);
snprintf(api_req, sizeof(api_req), "http://%s:%s#%s/export?uuid=%s",
XEN.user_name, password, XEN.host_ip, uuid);
curl_free(password);
puts(api_req);
TASK_LOG(api_req);
curl_download=TRUE;
curl_easy_setopt(curl, CURLOPT_URL, api_req);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_write_func);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, xva_export);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, FALSE);
// Install the callback function
curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, curl_progress_func);
//And....go!
run_progress_bar = TRUE;
//g_timeout_add_seconds(1, (GSourceFunc)update_progress_bar, 0);
res = curl_easy_perform(curl);
And uploading, if needed
curl = curl_easy_init();
//Auto escape user password
char *password = curl_easy_escape(curl, XEN.user_passwd, 0);
snprintf(api_req, sizeof(api_req), "http://%s:%s#%s/import",XEN.user_name, password, XEN.host_ip);
curl_free(password);
puts(api_req);
TASK_LOG(api_req);
curl_upload=TRUE;
g_timeout_add_seconds(1, (GSourceFunc) update_progress_bar, 0);
abort_flag = 0;
curl_easy_setopt(curl, CURLOPT_URL, api_req);
curl_easy_setopt(curl, CURLOPT_READFUNCTION, export_read_callback);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_PUT, 1L);
curl_easy_setopt(curl, CURLOPT_READDATA, xva);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE,(curl_off_t)file_info.st_size);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, FALSE);
// Install the callback function
curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, curl_progress_func);
puts("Upload started");
TASK_LOG("Upload started");
CURLcode res = curl_easy_perform(curl);
I just wonder if it possible to pipe traffic directly from one machine to another without downloading it?
You are using curl to download the file locally and then upload it.
Why not directly copy the file from source to destination using scp?

Resources