I compiled the following code and run:
$ ./run_curl 'http://ftp.gnu.org/gnu/grep/grep-2.4.tar.gz'.
But I cannot get full bytes of the file I want to download. Why?
I use curl --silent 'http://ftp.gnu.org/gnu/grep/grep-2.4.tar.gz' > a.tgz would get the full bytes.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define SIZE 512
void download_pkg(const char* url)
{
char content[SIZE] = {0};
FILE *fp = NULL;
char cmd[255] = {0};
FILE *fp_download = fopen("a.tgz", "w");
sprintf(cmd, "curl --silent '%s'", url);
fp = popen(cmd, "r");
while(fgets(content, SIZE - 1, fp) != NULL) {
fputs(content, fp_download);
}
fclose(fp_download);
pclose(fp);
return;
}
int main(int argc, char *argv[])
{
download_pkg(argv[1]);
return 0;
}
EDIT:
According Jonathan Leffler's answer, problem solved by the following code:
int size_read = 0;
while((size_read = fread(content, sizeof(char), SIZE - 1, fp)) > 0) {
fwrite(content, sizeof(char), size_read, fp_download);
}
A gzipped tar file will contain numerous null bytes. These null bytes will screw up the processing of data read via fgets(). You will miss the null bytes and any other bytes after them. fgets() does not tell you how many bytes it read, so you cannot safely read past the null bytes.
You need to use fread() and fwrite(), most probably. At least, those will not be confused by embedded nulls in the data.
Related
I written a simple copy program that copies a file and generates an MD5, It runs and generates the MD5 correctly.
However when verifying the file generated by the copy function it does not match the source MD5. I can't see any reason for this in my code, can anyone help?
#include <stdio.h>
#include <openssl/md5.h>
#include <assert.h>
#define BUFFER_SIZE 512
int secure_copy(char *filepath, char *destpath);
int main(int argc, char * argv[]) {
secure_copy(argv[1], argv[2]);
return 0;
}
int secure_copy(char *filepath, char *destpath) {
FILE *src, *dest;
src = fopen(filepath, "r");
assert(src != NULL);
dest = fopen(destpath, "w");
assert(dest != 0);
MD5_CTX c;
char buf[BUFFER_SIZE];
ssize_t bytes, out_writer;
unsigned char out[MD5_DIGEST_LENGTH];
MD5_Init(&c);
while((bytes = fread(buf, 1, BUFFER_SIZE, src)) != 0) {
MD5_Update(&c, buf, bytes);
out_writer = fwrite(buf, 1, BUFFER_SIZE, dest);
assert(out_writer != 0);
}
MD5_Final(out, &c);
printf("MD5: ");
for (int i=0; i < MD5_DIGEST_LENGTH; i++)
{
printf("%02x", out[i]);
}
printf("\n");
fclose(src);
fclose(dest);
return 0;
}
Output
$ ./md5speed doc.txt /home/doc.txt
MD5: 4c55e4b9185eece3cc000c4023f8f6fe
when verifying the copied file with md5sum I get a completely different hash.
md5sum doc.txt
29cb4da30c3e28fdb81463b5f0a76894 doc.txt
Though the file still opens and content is uncorrupted.
regarding:
while((bytes = fread(buf, 1, BUFFER_SIZE, src)) != 0)
and
out_writer = fwrite(buf, 1, BUFFER_SIZE, dest);
on the last read, the amount read can be less than BUFFER_SIZE so should always use bytes variable for the number of bytes to write.
Also, certain errors can occur when calling fread() and/or fwrite() Such errors are indicated by negative values (and/or values less than the 3rd parameter to those functions) in the returned variables (bytes, outwriter). The code, to be robust, must be checking those values and handling any errors that occur, including EOF
As stated in comments, changing the fwrite function to use bytes as opposed to BUFFER_SIZE combined with changing file operations mode "rb" and "wb" to binary.
I'm trying to read and base64-decode a file. For some reason OpenSSL reads only a part of it, then on all subsequent calls it simply returns 0. Here is the code I'm using:
#include <stdio.h>
#include <stdlib.h>
#include <openssl/bio.h>
#include <openssl/buffer.h>
#include <openssl/evp.h>
size_t b64_get_datalen(const char* b64, size_t b64len) {
size_t actual_len = 0;
int padding = 0;
for (int i = 0; i < b64len; i++) {
if (b64[i] != '\n') actual_len++;
}
int last = b64len-1;
if (b64[last] == '\n') {
last--;
}
if (b64[last] == '=') {
padding = b64[last-1] == '=' ? 2 : 1;
}
return (actual_len*3)/4 - padding;
}
int main(int argc, char** argv) {
FILE* fp = fopen("7.txt", "r");
if (fp == NULL) {
perror("fopen");
exit(EXIT_FAILURE);
}
fseek(fp, 0, SEEK_END);
size_t b64_len = ftell(fp);
rewind(fp);
char b64[b64_len];
fread(b64, b64_len, 1, fp);
fclose(fp);
size_t data_len = b64_get_datalen(b64, b64_len);
char data[data_len];
BIO *bio, *base64;
base64 = BIO_new(BIO_f_base64());
bio = BIO_new_mem_buf(b64, -1);
bio = BIO_push(base64, bio);
int read = BIO_read(bio, data, b64_len);
printf("expected %d bytes, got %d bytes\n", (int)data_len, (int)read);
BIO_free_all(bio);
}
The program outputs: expected 2880 bytes, got 2244 bytes
The file I'm trying to read is from cryptopals challenges (http://cryptopals.com/static/challenge-data/7.txt). I solved that challenge in Python, there were no issues with reading and base64-decoding the file.
What can be a reason of such behavior? And a more general question, how one should debug such issues with OpenSSL?
The problem is in this line:
bio = BIO_new_mem_buf(b64, -1);
From the man page for BIO_new_mem_buf:
https://www.openssl.org/docs/man1.1.0/crypto/BIO_s_mem.html
BIO_new_mem_buf() creates a memory BIO using len bytes of data at buf,
if len is -1 then the buf is assumed to be nul terminated and its
length is determined by strlen.
But you have passed -1 and your buffer is not nul terminated! Therefore the strlen will overrun the buffer and keep going until it does find a nul terminator. This will likely include lots of values which are not valid base64, and hence the base64 decode fails.
It appears that the behaviour of the BIO_f_base64() BIO is to read and return what it has successfully read, but give up trying to read any more as soon as it encounters an error - as opposed to failing the entire read. It's not clear to me whether that is intended behaviour or a bug.
And a more general question, how one should debug such issues with OpenSSL?
In general you should inspect the OpenSSL error stack which usually gives good information about the source of a problem. For example by using ERR_print_errors_fp():
https://www.openssl.org/docs/man1.1.0/crypto/ERR_print_errors_fp.html
Not that it would have helped in this particular issue because it seems no error is placed on the error stack for this problem.
I am trying to find out a way to select the last line of a text file using C (not c++ or c#, just C) and I am having a difficult time finding a way to do this, if anyone could assist me with this problem I would be very grateful, thanks! (btw for a good example of what i am trying to do, this would be similar what to tail -n 1 would be doing in bash)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void main(int argc, char *argv[])
{
FILE *fd; // File pointer
char filename[] = "./Makefile"; // file to read
char buff[1024];
if ((fd = fopen(filename, "r")) != NULL) // open file
{
fseek(fd, 0, SEEK_SET); // make sure start from 0
while(!feof(fd))
{
memset(buff, 0x00, 1024); // clean buffer
fscanf(fd, "%[^\n]\n", buff); // read file *prefer using fscanf
}
printf("Last Line :: %s\n", buff);
}
}
I'm using Linux.
CMIIW
No direct way, but my preferred method is:
Go to the end of the file
Read last X bytes
If they contain '\n' - you got your line - read from that offset to the end of the file
Read X bytes before them
back to 3 until match found
If reached the beginning of the file - the whole file is the last line
E.g.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef max
#define max(a, b) ((a)>(b))? (a) : (b)
#endif
long GetFileSize(FILE *fp){
long fsize = 0;
fseek(fp,0,SEEK_END);
fsize = ftell(fp);
fseek(fp,0,SEEK_SET);//reset stream position!!
return fsize;
}
char *lastline(char *filepath){
FILE *fp;
char buff[4096+1];
int size,i;
long fsize;
if(NULL==(fp=fopen(filepath, "r"))){
perror("file cannot open at lastline");
return NULL;
}
fsize= -1L*GetFileSize(fp);
if(size=fseek(fp, max(fsize, -4096L), SEEK_END)){
perror("cannot seek");
exit(1);
}
size=fread(buff, sizeof(char), 4096, fp);
fclose(fp);
buff[size] = '\0';
i=size-1;
if(buff[i]=='\n'){
buff[i] = '\0';
}
while(i >=0 && buff[i] != '\n')
--i;
++i;
return strdup(&buff[i]);
}
int main(void){
char *last;
last = lastline("data.txt");
printf("\"%s\"\n", last);
free(last);
return 0;
}
If you are using *nix operating system, you can use the command 'last'. See 'last' man page for details.
If you want integrate the functionality inside another program, you can use 'system' call to execute 'last' and get it's result.
A simple and inefficient way to do it is to read each line into a buffer.
When the last read gives you EOF, you have the last line in the buffer.
Binyamin Sharet's suggestion is more efficient, but just a bit harder to implement.
I have a text file named test.txt
I want to write a C program that can read this file and print the content to the console (assume the file contains only ASCII text).
I don't know how to get the size of my string variable. Like this:
char str[999];
FILE * file;
file = fopen( "test.txt" , "r");
if (file) {
while (fscanf(file, "%s", str)!=EOF)
printf("%s",str);
fclose(file);
}
The size 999 doesn't work because the string returned by fscanf can be larger than that. How can I solve this?
The simplest way is to read a character, and print it right after reading:
int c;
FILE *file;
file = fopen("test.txt", "r");
if (file) {
while ((c = getc(file)) != EOF)
putchar(c);
fclose(file);
}
c is int above, since EOF is a negative number, and a plain char may be unsigned.
If you want to read the file in chunks, but without dynamic memory allocation, you can do:
#define CHUNK 1024 /* read 1024 bytes at a time */
char buf[CHUNK];
FILE *file;
size_t nread;
file = fopen("test.txt", "r");
if (file) {
while ((nread = fread(buf, 1, sizeof buf, file)) > 0)
fwrite(buf, 1, nread, stdout);
if (ferror(file)) {
/* deal with error */
}
fclose(file);
}
The second method above is essentially how you will read a file with a dynamically allocated array:
char *buf = malloc(chunk);
if (buf == NULL) {
/* deal with malloc() failure */
}
/* otherwise do this. Note 'chunk' instead of 'sizeof buf' */
while ((nread = fread(buf, 1, chunk, file)) > 0) {
/* as above */
}
Your method of fscanf() with %s as format loses information about whitespace in the file, so it is not exactly copying a file to stdout.
There are plenty of good answers here about reading it in chunks, I'm just gonna show you a little trick that reads all the content at once to a buffer and prints it.
I'm not saying it's better. It's not, and as Ricardo sometimes it can be bad, but I find it's a nice solution for the simple cases.
I sprinkled it with comments because there's a lot going on.
#include <stdio.h>
#include <stdlib.h>
char* ReadFile(char *filename)
{
char *buffer = NULL;
int string_size, read_size;
FILE *handler = fopen(filename, "r");
if (handler)
{
// Seek the last byte of the file
fseek(handler, 0, SEEK_END);
// Offset from the first to the last byte, or in other words, filesize
string_size = ftell(handler);
// go back to the start of the file
rewind(handler);
// Allocate a string that can hold it all
buffer = (char*) malloc(sizeof(char) * (string_size + 1) );
// Read it all in one operation
read_size = fread(buffer, sizeof(char), string_size, handler);
// fread doesn't set it so put a \0 in the last position
// and buffer is now officially a string
buffer[string_size] = '\0';
if (string_size != read_size)
{
// Something went wrong, throw away the memory and set
// the buffer to NULL
free(buffer);
buffer = NULL;
}
// Always remember to close the file.
fclose(handler);
}
return buffer;
}
int main()
{
char *string = ReadFile("yourfile.txt");
if (string)
{
puts(string);
free(string);
}
return 0;
}
Let me know if it's useful or you could learn something from it :)
Instead just directly print the characters onto the console because the text file maybe very large and you may require a lot of memory.
#include <stdio.h>
#include <stdlib.h>
int main() {
FILE *f;
char c;
f=fopen("test.txt","rt");
while((c=fgetc(f))!=EOF){
printf("%c",c);
}
fclose(f);
return 0;
}
Use "read()" instead o fscanf:
ssize_t read(int fildes, void *buf, size_t nbyte);
DESCRIPTION
The read() function shall attempt to read nbyte bytes from the file associated with the open file descriptor, fildes, into the buffer pointed to by buf.
Here is an example:
http://cmagical.blogspot.com/2010/01/c-programming-on-unix-implementing-cat.html
Working part from that example:
f=open(argv[1],O_RDONLY);
while ((n=read(f,l,80)) > 0)
write(1,l,n);
An alternate approach is to use getc/putc to read/write 1 char at a time. A lot less efficient. A good example: http://www.eskimo.com/~scs/cclass/notes/sx13.html
You can use fgets and limit the size of the read string.
char *fgets(char *str, int num, FILE *stream);
You can change the while in your code to:
while (fgets(str, 100, file)) /* printf("%s", str) */;
Two approaches leap to mind.
First, don't use scanf. Use fgets() which takes a parameter to specify the buffer size, and which leaves any newline characters intact. A simple loop over the file that prints the buffer content should naturally copy the file intact.
Second, use fread() or the common C idiom with fgetc(). These would process the file in fixed-size chunks or a single character at a time.
If you must process the file over white-space delimited strings, then use either fgets or fread to read the file, and something like strtok to split the buffer at whitespace. Don't forget to handle the transition from one buffer to the next, since your target strings are likely to span the buffer boundary.
If there is an external requirement to use scanf to do the reading, then limit the length of the string it might read with a precision field in the format specifier. In your case with a 999 byte buffer, then say scanf("%998s", str); which will write at most 998 characters to the buffer leaving room for the nul terminator. If single strings longer than your buffer are allowed, then you would have to process them in two pieces. If not, you have an opportunity to tell the user about an error politely without creating a buffer overflow security hole.
Regardless, always validate the return values and think about how to handle bad, malicious, or just malformed input.
You can use getline() to read your text file without worrying about large lines:
getline() reads an entire line from stream, storing the address of the buffer containing the text into *lineptr. The buffer is null-terminated and includes the newline character, if one was found.
If *lineptr is set to NULL before the call, then getline() will allocate a buffer for storing the line. This buffer should be freed by the user program even if getline() failed.
bool read_file(const char *filename)
{
FILE *file = fopen(filename, "r");
if (!file)
return false;
char *line = NULL;
size_t linesize = 0;
while (getline(&line, &linesize, file) != -1) {
printf("%s", line);
free(line);
}
free(line);
fclose(file);
return true;
}
You can use it like this:
int main(void)
{
if (!read_file("test.txt")) {
printf("Error reading file\n");
exit(EXIT_FAILURE);
}
}
I use this version
char* read(const char* filename){
FILE* f = fopen(filename, "rb");
if (f == NULL){
exit(1);
}
fseek(f, 0L, SEEK_END);
long size = ftell(f)+1;
fclose(f);
f = fopen(filename, "r");
void* content = memset(malloc(size), '\0', size);
fread(content, 1, size-1, f);
fclose(f);
return (char*) content;
}
You could read the entire file with dynamic memory allocation, but isn't a good idea because if the file is too big, you could have memory problems.
So is better read short parts of the file and print it.
#include <stdio.h>
#define BLOCK 1000
int main() {
FILE *f=fopen("teste.txt","r");
int size;
char buffer[BLOCK];
// ...
while((size=fread(buffer,BLOCK,sizeof(char),f)>0))
fwrite(buffer,size,sizeof(char),stdout);
fclose(f);
// ...
return 0;
}
I'm totally new in C, and I'm trying to do a little application that searches for a string in a file. My problem is that I need to open a big file (more than 1GB) with just one line inside and fgets return me the entire file (I'm doing test with a 10KB file).
Actually this is my code:
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[]) {
char *search = argv[argc-1];
int retro = strlen(search);
int pun = 0;
int sortida;
int limit = 10;
char ara[20];
FILE *fp;
if ((fp = fopen ("SEARCHFILE", "r")) == NULL){
sortida = -1;
exit (1);
}
while(!feof(fp)){
if (fgets(ara, 20, fp) == NULL){
break;
}
//this must be a 20 bytes line, but it gets the entyre 10Kb file
printf("%s",ara);
}
sortida = 1;
if(fclose(fp) != 0){
sortida = -2;
exit (1);
}
return 0;
}
What can I do to find an string into a file?
I've tried with GREP but it don't helps, because it returns the position:ENTIRE_STRING.
I'm open to ideas.
Try
printf("%s\n",ara);
Also consider initializing variables before you use them:
char ara[20]={0x0};
You only allocated 20 bytes for the input buffer, but told the fgets to read 20 bytes.
Make this change:
if (fgets(ara, sizeof(ara)-1, fp) == NULL){
remember, if you want 20 characters PLUS the trailing '\0' that marks the end of the string you have to allocate 21 bytes.