libcur post file to web server asynchronously doesn't work? - file

I want to upload file to web server like IIS or Apache through http post.
The file is larger than 1GB, so I want to upload it asynchronously .
But the code can't work and never executed to. the callback "readfunc" function?
Is there anything wrong or what I should do?
Thanks very much!
size_t readfunc(void *ptr, size_t size, size_t nmemb, void *stream)
{
FILE *f = (FILE *)stream;
size_t n;
if (ferror(f))
return CURL_READFUNC_ABORT;
n = fread(ptr, size, nmemb, f) * size;
return n;
}
void curl_post_form()
{
curl_global_init(CURL_GLOBAL_ALL);
CURL* hCurl = curl_easy_init();
if (hCurl != NULL)
{
curl_httppost* pFormPost = NULL;
curl_httppost* pLastElem = NULL;
curl_formadd(&pFormPost, &pLastElem,
CURLFORM_COPYNAME, "ufile01",
CURLFORM_FILE, "D:\\123.iso",
CURLFORM_CONTENTTYPE, "application/octet-stream",
CURLFORM_END);
curl_easy_setopt(hCurl, CURLOPT_HTTPPOST, pFormPost);
curl_easy_setopt(hCurl, CURLOPT_URL,"http://192.168.138.1:8800/Record/");
/* I want to use my own read function */
curl_easy_setopt(curlhandle, CURLOPT_READFUNCTION, readfunc);
/* now specify which file to upload */
curl_easy_setopt(curlhandle, CURLOPT_READDATA, f);
curl_easy_setopt(hCurl, CURLOPT_VERBOSE, 1L);
CURLcode res = curl_easy_perform(hCurl);
if (res != CURLE_OK)
{
printf("Error");
}
curl_formfree(pFormPost);
curl_easy_cleanup(hCurl);
}
curl_global_cleanup();
}

Related

libcurl inserts additional data to download files

I use libcurl in my C code to download files given their urls. My code looks similar to this:
#include <stdio.h>
#include <curl.h>
#include <pthread.h>
static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t written = fwrite(ptr, size, nmemb, (FILE *)stream);
return written;
}
int progress_func(void *ptr, double TotalToDownload, double NowDownloaded,
double TotalToUpload, double NowUploaded)
{
struct my_custom_struct *my_dummy_data = (struct my_custom_struct *) data;
//do some stuffs here
return 0;
}
void *download_with_curl(void *data)
{
char *url = (char *) data;
int res = 0;
// My custom struct to store data
struct my_custom_struct my_dummy_data;
char errbuff[CURL_ERROR_SIZE] = {0};
CURL *curl_handle;
/* init the curl session */
curl_handle = curl_easy_init();
/* set URL to get here */
curl_easy_setopt(curl_handle, CURLOPT_URL, url);
/* disable progress meter, set to 0L to enable*/
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 0L);
/* send all data to this function*/
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl_handle, CURLOPT_LOW_SPEED_TIME, RESPOND_TIME);
curl_easy_setopt(curl_handle, CURLOPT_LOW_SPEED_LIMIT, 30L);
/* set the progress function */
curl_easy_setopt(curl_handle, CURLOPT_PROGRESSFUNCTION, progress_func);
/* set the progress data */
curl_easy_setopt(curl_handle, CURLOPT_PROGRESSDATA, &my_dummy_data);
/* provide a buffer to store errors in */
curl_easy_setopt(curl_handle, CURLOPT_ERRORBUFFER, errbuff);
FILE *pagefile = fopen(path_to_where_I_want_to_store_the_file, "wb");
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, pagefile);
/* get the file*/
int status = curl_easy_perform(curl_handle);
res = 0;
int response_code;
curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &response_code);
fclose(pagefile);
if (status != 0) {
log_warn("CURL ERROR %d: %s", status, errbuff);
response_code = -status;
}
/* cleanup curl stuff */
curl_easy_cleanup(curl_handle);
return NULL;
}
int main()
{
// sockfd = create a sockfd
// bind, listen
do {
// accept new connection
char *url;
// receive the url from client
pthread_t tid;
pthread_create(&tid, NULL, download_with_curl, url);
} while (1);
}
When I send a single download request, the code works fine. "Works fine" means that the md5sum values of the original file and the downloaded file are equal. However, when I send multiple requests to download multiple files, only the first file that is downloaded has the correct md5sum value. To be clear, if I send requests to download files A (200MB), B (5MB) and C (50MB) in that order, only file B is correctly downloaded because it is finished first. Files A and C will have incorrect md5sum values. Moreover, when I check the content of files A and C, it looks like curl just inserts random segments of data into them. If the original file content is
This is the content of a file
then the downloaded file is like
This is the #$%!##%#% content of $%(#(!)$()$%||##$%*&) a file
After spending two days of debugging, I finally solved the problem (I hope so). All I did was just flushing the data after calling fwrite. The function write_data now looks like this:
static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t written = fwrite(ptr, size, nmemb, (FILE *)stream);
fflush((FILE *) stream);
return written;
}
I do not know if it completely solves the problem or not. Could anyone explain why it behaves that way and give my a solution to this?
UPDATE 1
It seems that there is something to do with fwrite()'s internal buffer. Changing from fwrite(ptr, size, nmemb, stream) to write(fileno(stream), ptr, size * nmemb) seems to give the same result as using fflush().
UPDATE 2
Using the default write function (remove the option CURLOPT_WRITEFUNCTION) of libcurl gives the same problem.

Curl c example for webdav put with digest authentication on Apache server

I am trying to write curl c code to using the http webdav put method to upload a file.
Using wireshark I have tried to capture the packets, there is 301 response from the server.
When I try to put the file from the PC to webserver it works fine
Below is the code:
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *stream)
{
size_t retcode;
curl_off_t nread;
/* in real-world cases, this would probably get this data differently
as this fread() stuff is exactly what the library already would do
by default internally */
retcode = fread(ptr, size, nmemb, stream);
nread = (curl_off_t)retcode;
fprintf(stderr, "*** We read %" CURL_FORMAT_CURL_OFF_T
" bytes from file\n", nread);
return retcode;
}
int curlApache ()
{
CURL *curl;
CURLcode res;
FILE * hd_src;
struct stat file_info;
char *file;
char *url;
char error;
file = "/bd0/filecreate.txt";
url = "http://10.1.21.14/webdav/test.txt";
curl_slist *slist = NULL;
slist = curl_slist_append(slist, "Accept: text/xml");
slist = curl_slist_append(slist, "Depth: infinity");
slist = curl_slist_append(slist, "Connection: Keep-Alive");
slist = curl_slist_append(slist, "Content-Type: text/xml");
slist = curl_slist_append(slist, "Expect:");
/* get the file size of the local file */
stat(file, &file_info);
hd_src = fopen(file, "a+");
if (hd_src == NULL)
printf("Disc full or no permission\n");
const char *str = "This is the file content";
const char read[24];
if (hd_src != NULL)
if (fputs (str, hd_src) != EOF);
if( fgets (read, 24, hd_src)!=NULL )
{
/* writing content to stdout */
puts(read);
}
/* In windows, this will init the winsock stuff */
curl_global_init(CURL_GLOBAL_ALL);
/* get a curl handle */
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_VERBOSE, 3L);
/* we want to use our own read function */
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);
/* enable uploading */
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
/* HTTP PUT please */
curl_easy_setopt(curl, CURLOPT_PUT, 1L);
/* tell libcurl we can use "any" auth, which lets the lib pick one, but it also costs one extra round-trip and possibly sending of all the PUT data twice!!! */
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, (long)CURLAUTH_DIGEST);
curl_easy_setopt(curl, CURLOPT_USERPWD, "admin:nimo0630");
fseek(hd_src, 0L, SEEK_END);
int file_size;
file_size = ftell(hd_src);
Curl_setopt(curl, CURLOPT_INFILE, hd_src);
Curl_setopt(curl, CURLOPT_INFILESIZE, file_size);
/* specify target URL, and note that this URL should include a file
name, not only a directory */
curl_easy_setopt(curl, CURLOPT_URL, url);
/* now specify which file to upload */
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
/* provide the size of the upload, we specicially typecast the value
to curl_off_t since we must be sure to use the correct data size */
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, curl_off_t)file_info.st_size);
/* Now run off and do what you've been told! */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
if(!res) {
/* extract the available authentication types */
long auth;
res = curl_easy_getinfo(curl, CURLINFO_HTTPAUTH_AVAIL, &auth);
if(!res)
{
if(!auth)
printf("No auth available, perhaps no 401?\n");
else
{
printf("%s%s%s%s\n", \
auth & CURLAUTH_BASIC ? "Basic ":"", \
auth & CURLAUTH_DIGEST ? "Digest ":"", \
auth & CURLAUTH_NEGOTIATE ? "Negotiate ":"", \
auth % CURLAUTH_NTLM ? "NTLM ":"");
}
}
}
/* always cleanup */
curl_easy_cleanup(curl);
}
fclose(hd_src); /* close the local file */
curl_global_cleanup();
return 0;
}
There is status code 301 returned from the server
I believe you want to understand how to debug this issue. If you paste your code, a more specific answer can be provided.
First check if you are able to upload file using command line curl. This will tell you if server is working fine
Check if the options you use are provided in C API
Check the API's are not returning any error
You could use tcpdump / wireshark to capture packets on the client or server to see if packets went out and what was the http content. You may not see any packet if API failed

Play mp3 file in C

I want to play mp3 files on the internet without downloading them. So, I use libcurl to get it as a stream in memory, like this:
static size_t use_data(void *ptr, size_t size, size_t nmemb, FILE *stream) {
/* stream is NULL */
/* What to do with the stream of data ? */
}
CURLcode download_file(const char *url, const char *path, curl_progress_callback progress) {
CURL *curl;
CURLcode res = 0;
FILE *fp;
if ((curl = curl_easy_init())) {
if (progress) {
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0);
curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, progress);
}
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, use_data);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
fclose(fp);
}
return res;
}
How can I parse the stream in memory to play sounds ?
The easiest way for you IMHO would be using lightweight MP3 decoding library. For example, minimp3 does it's job and consists of only 2 files.
http://keyj.emphy.de/minimp3
The API is very simple and a usage example can be found here: https://github.com/corporateshark/PortAMP/tree/master/src/Decoders/MP3

Libcurl how to not show all this information

How to not show all this info?
All I do is using little edited FTP example and i dont want that info to be shown.
edit: added full code from main.c
image in link: http://www.mediafire.com/convkey/71e9/oyhctzcdjxakzxzfg.jpg
#include <stdio.h>
#include <stdlib.h>
#include <curl/curl.h>
struct FtpFile {
const char *filename;
FILE *stream;
};
static size_t my_fwrite(void *buffer, size_t size, size_t nmemb, void *stream)
{
struct FtpFile *out=(struct FtpFile *)stream;
if(out && !out->stream) {
/* open file for writing */
out->stream=fopen(out->filename, "wb");
if(!out->stream)
return -1; /* failure, can't open file to write */
}
return fwrite(buffer, size, nmemb, out->stream);
}
int main()
{
CURL *curl;
CURLcode res;
struct FtpFile version={"version.txt", /* name to store the file as if succesful */NULL};
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
FILE *file_verzija;
int trenutna_verzija;
int nova_verzija;
char pitanje_za_update;
file_verzija=fopen("version.txt","r");
fscanf(file_verzija,"%i",&trenutna_verzija);
fclose(file_verzija);
printf("Current version %i",trenutna_verzija);
printf("\nChecking for updates...\n");
if(curl)
{
/*You better replace the URL with one that works!*/curl_easy_setopt(curl, CURLOPT_URL,"http://elektro-srb.com/site/muffin_finder_files/version.txt");
/* Define our callback to get called when there's data to be written */
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, my_fwrite);
/* Set a pointer to our struct to pass to the callback */
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &version);
/* Switch on full protocol/debug output */
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
if(CURLE_OK != res)
{
/* we failed */
printf("\nerror");
}
}
if(version.stream)
fclose(version.stream); /* close the local file */
file_verzija=fopen("version.txt","r");
fscanf(file_verzija,"%i",&nova_verzija);
fclose(file_verzija);
if(trenutna_verzija != nova_verzija)
{
printf("\nUpdate found! New version is %i",nova_verzija);
}
else
{
printf("You are running latest version of Muffin Finder!");
}
if(trenutna_verzija != nova_verzija)
{
printf("\nUpdate? y/n");
scanf("\n%c",&pitanje_za_update);
if((pitanje_za_update == 'y') || (pitanje_za_update == 'Y'))
{
//UPDATE
}
else if((pitanje_za_update == 'n') || (pitanje_za_update == 'N'))
{
//pokretanje stare
}
}
curl_global_cleanup();
return 0;
}
You should construct a WRITEFUNCTION option, to prevent it from using stdout for printing.
See here: http://curl.haxx.se/libcurl/c/curl_easy_setopt.html.
Search for "WRITEFUNCTION". You should implement the function (and I assume you would like to leave it empty).
EDIT: As the manual states, you should do the following:
Implement a function to replace the default stdout:
size_t outputFunction(char *ptr, size_t size, size_t nmemb, void *userdata) {}
When you initialize the CURL structure, use this option:
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, outputFunction);
comment following line:
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

C libcurl - measure download speed and time remaining

I am using the following code to download files from the internet:
size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream)
{
size_t written;
written = fwrite(ptr, size, nmemb, stream);
return written;
}
int main(int argc, char** argv)
{
FILE *downloaded_file;
if ( (downloaded_file = fopen (download_path , "w" ) ) != NULL )
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "www.asd.com/files/file_to_download.rar");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, downloaded_file);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
if (res == CURLE_OK)
{
printf("Download complete!\n");
}
}
fclose(downloaded_file);
}
}
How can I measure the current download speed (e.g. every second) and the remaining time to complete the download?
You can use CURLOPT_PROGRESSFUNCTION. curl will pass 5 arguments to your callback function, clientp, dltotal, dlnow, ultotal, and ulnow. clientp is a pointer you provide with CURLOPT_PROGRESSDATA. The total parameters are the total amounts that need to be downloaded; the now ones are the amounts so far. Unknown values are 0.
To use this, you must set CURLOPT_NOPROGRESS to 0.
CURLOPT_PROGRESSFUNCTION has been deprecated since v7.32.0, instead you can use CURLOPT_XFERINFOFUNCTION, the usage and callback structure (all the arguments dltotal, dlnow ...etc) are almost the same as CURLOPT_PROGRESSFUNCTION

Resources