How to write a file to a buffer memory in C - c

So i got my function here that works to write back any file
int write_file(FILE *f_write) {
// Temp variables
FILE *img = fopen("test.pdf", "wb");
unsigned char buffer[255];
while ( (bytes_read = fread(buffer, 1, sizeof(buffer), f_write) ) > 0) {
fwrite(buffer, 1, bytes_read, img);
}
fclose(img);
return 1;
}
So this works perfecly ive tried with pnj / pdf / jpg etc..
But now i want to stock what ive writen in the memory so i can use it later and not write right away
like an array of uint8_t (maybe) that will contain all the bytes ive writen and that i can send later with sockets to my server and store the file
no idea how to do it
Or maybe i'm making it too complicated and i can just
send(client_socket, FILE, sizeof(FILE), 0); ?

One way to do it would be to create a buffer that exactly fits the size of the file.
In order to do so, you can write a function to get the size of an openned file like so:
size_t get_file_size(FILE *f)
{
size_t pos = ftell(f); // store the cursor position
size_t size;
// go to the end of the file and get the cursor position
fseek(f, 0L, SEEK_END);
size = ftell(f);
// go back to the old position
fseek(f, pos, SEEK_SET);
return size;
}
Then create and fill your buffer:
FILE *f = fopen("your_file", "r");
size_t size = get_file_size(f);
char *buffer = malloc(size);
if (fread(buffer, 1, size, f) != size) { // bytes read != computed file size
// error handling
}
// use your buffer...
// don't forget to free and fclose
free(buffer);
fclose(f);
It is worth mentioning that you should check if the file was opened correctly, and to check if you have enough memory to store the buffer (the one created with malloc).
Edit:
As Andrew Henle said, fseek()/ftell() to get the size of a file is non-portable. Instead, to get the size of your file, you should use one of these techniques depending on your OS (assuming you are trying to open a 'normal' file):
On Linux / MacOS:
#include <sys/stat.h>
struct stat st;
size_t size;
if (stat("your_file", &st) != 0) {
// error handling...
}
size = st.st_size;
On Windows (as answered here) :
__int64 FileSize(const wchar_t* name)
{
HANDLE hFile = CreateFile(name, GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE)
return -1; // error condition, could call GetLastError to find out more
LARGE_INTEGER size;
if (!GetFileSizeEx(hFile, &size)) {
CloseHandle(hFile);
return -1; // error condition, could call GetLastError to find out more
}
CloseHandle(hFile);
return size.QuadPart;
}

Related

FFmpeg: unspecified pixel format when opening video with custom context

I am trying to decode a video with a custom context. The purpose is that I want to decode the video directly from memory. In the following code, I am reading from file in the read function passed to avio_alloc_context - but this is just for testing purposes.
I think I've read any post there is on Stackoverflow or on any other website related to this topic. At least I definitely tried my best to do so. While there is much in common, the details differ: people set different flags, some say av_probe_input_format is required, some say it isn't, etc. And for some reason nothing works for me.
My problem is that the pixel format is unspecified (see output below), which is why I run into problems later when calling sws_getContext. I checked pFormatContext->streams[videoStreamIndex]->codec->pix_fmt, and it is -1.
Please note my comments // things I tried and // seems not to help in the code. I think, the answer might be hidden somehwere there. I tried many combinations of hints that I've read so far, but I am missing a detail I guess.
The problem is not the video file, because when I go the standard way and just call avformat_open_input(&pFormatContext, pFilePath, NULL, NULL) without a custom context, everything runs fine.
The code compiles and runs as is.
#include <libavformat/avformat.h>
#include <string.h>
#include <stdio.h>
FILE *f;
static int read(void *opaque, uint8_t *buf, int buf_size) {
if (feof(f)) return -1;
return fread(buf, 1, buf_size, f);
}
int openVideo(const char *pFilePath) {
const int bufferSize = 32768;
int ret;
av_register_all();
f = fopen(pFilePath, "rb");
uint8_t *pBuffer = (uint8_t *) av_malloc(bufferSize + AVPROBE_PADDING_SIZE);
AVIOContext *pAVIOContext = avio_alloc_context(pBuffer, bufferSize, 0, NULL,
&read, NULL, NULL);
if (!f || !pBuffer || !pAVIOContext) {
printf("error: open / alloc failed\n");
// cleanup...
return 1;
}
AVFormatContext *pFormatContext = avformat_alloc_context();
pFormatContext->pb = pAVIOContext;
const int readBytes = read(NULL, pBuffer, bufferSize);
printf("readBytes = %i\n", readBytes);
if (readBytes <= 0) {
printf("error: read failed\n");
// cleanup...
return 2;
}
if (fseek(f, 0, SEEK_SET) != 0) {
printf("error: fseek failed\n");
// cleanup...
return 3;
}
// required for av_probe_input_format
memset(pBuffer + readBytes, 0, AVPROBE_PADDING_SIZE);
AVProbeData probeData;
probeData.buf = pBuffer;
probeData.buf_size = readBytes;
probeData.filename = "";
probeData.mime_type = NULL;
pFormatContext->iformat = av_probe_input_format(&probeData, 1);
// things I tried:
//pFormatContext->flags = AVFMT_FLAG_CUSTOM_IO;
//pFormatContext->iformat->flags |= AVFMT_NOFILE;
//pFormatContext->iformat->read_header = NULL;
// seems not to help (therefore commented out here):
AVDictionary *pDictionary = NULL;
//av_dict_set(&pDictionary, "analyzeduration", "8000000", 0);
//av_dict_set(&pDictionary, "probesize", "8000000", 0);
if ((ret = avformat_open_input(&pFormatContext, "", NULL, &pDictionary)) < 0) {
char buffer[4096];
av_strerror(ret, buffer, sizeof(buffer));
printf("error: avformat_open_input failed: %s\n", buffer);
// cleanup...
return 4;
}
printf("retrieving stream information...\n");
if ((ret = avformat_find_stream_info(pFormatContext, NULL)) < 0) {
char buffer[4096];
av_strerror(ret, buffer, sizeof(buffer));
printf("error: avformat_find_stream_info failed: %s\n", buffer);
// cleanup...
return 5;
}
printf("nb_streams = %i\n", pFormatContext->nb_streams);
// further code...
// cleanup...
return 0;
}
int main() {
openVideo("video.mp4");
return 0;
}
This is the output that I get:
readBytes = 32768
retrieving stream information...
[mov,mp4,m4a,3gp,3g2,mj2 # 0xdf8d20] stream 0, offset 0x30: partial file
[mov,mp4,m4a,3gp,3g2,mj2 # 0xdf8d20] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 640x360, 351 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
nb_streams = 2
UPDATE:
Thanks to WLGfx, here is the solution: The only thing that was missing was the seek function. Apparently, implementing it is mandatory for decoding. It is important to return the new offset - and not 0 in case of success (some solutions found in the web just return the return value of fseek, and that is wrong). Here is the minimal solution that made it work:
static int64_t seek(void *opaque, int64_t offset, int whence) {
if (whence == SEEK_SET && fseek(f, offset, SEEK_SET) == 0) {
return offset;
}
// handling AVSEEK_SIZE doesn't seem mandatory
return -1;
}
Of course, the call to avio_alloc_context needs to be adapted accordingly:
AVIOContext *pAVIOContext = avio_alloc_context(pBuffer, bufferSize, 0, NULL,
&read, NULL, &seek);
Seeing as yours is a file based stream then it is seekable so you can provide the AVIO seek when creating the AVIOContext:
avioContext = avio_alloc_context((uint8_t *)avio_buffer, AVIO_QUEUE_SIZE * PKT_SIZE7,
0,
this, // *** This is your data pointer to a class or other data passed to the callbacks
avio_ReadFunc,
NULL,
avio_SeekFunc);
Handle the seeking with this callback: (You can cast ptr to your class or other data structure)
int64_t FFIOBufferManager::avio_SeekFunc(void *ptr, int64_t pos64, int whence) {
// SEEK_SET(0), SEEK_CUR(1), SEEK_END(2), AVSEEK_SIZE
// ptr is cast to your data or class
switch (whence) {
case 0 : // SEEK_SET
... etc
case (AVSEEK_SIZE) : // get size
return -1; // if you're unable to get the size
break;
}
// set new position in the file
return (int64_t)new_pos; // new position
}
You can also define the codec and the probesize when attaching the AVIOContext to the AVFormatContext. This allows ffmpeg to seek in the stream to better determine the format.
context->pb = ffio->avioContext;
context->flags = AVFMT_FLAG_CUSTOM_IO;
context->iformat = av_find_input_format("mpegts"); // not necessary
context->probesize = 1200000;
So far I haven't had the need for av_probe_input_format, but then again my streams are mpegts.
Hope this helps.
EDIT: Added a comment to the avio_alloc_context function to mention how the ptr is used in the callbacks.
Although the seek was the right answer in your situation, the fact is in my case it's not possible because I have to stream the data and in that situation a seek is just not possible.
So I had to look into: why is a seek required?
From what the ffmpeg docs say, they will cache some data so that way they can seek back if required by the current encoder/decoder. But that buffer is relatively small (you probably don't want to cache 100's of Mb of data).
The fact is that MP4 saves some metadata at the end of the file (once it's known). When reading that format, the decoder wants to seek to a position really very far in the file (near the end) and read what is called the moov atom. Without that info, the system doesn't want to decompress your data.
What I had to do to fix this issue is move that moov atom with the following command:
ffmpeg -i far.mp4 -c copy -map 0 -movflags +faststart close.mp4
faststart means you do not have to stream the entire file to start playing (decoding) the file.

Why does allocating a lot of memory give worse results?

So in my assignment I am testing the times it takes for different copying functions to copy. One of them I am a bit curious on the results. In one my copy functions it involves allocating memory like so:
int copyfile3(char* infilename, char* outfilename, int size) {
int infile; //File handles for source and destination.
int outfile;
infile = open(infilename, O_RDONLY); // Open the input and output files.
if (infile < 0) {
open_file_error(infilename);
return 1;
}
outfile = open(outfilename, O_WRONLY | O_CREAT, S_IRUSR | S_IWUSR);
if (outfile < 0) {
open_file_error(outfilename);
return 1;
}
int intch; // Character read from input file. must be an int to catch EOF.
char *ch = malloc(sizeof(char) * (size + 1));
gettimeofday(&start, NULL);
// Read each character from the file, checking for EOF.
while ((intch = read(infile, ch, size)) > 0) {
write(outfile, ch, intch); // Write out.
}
gettimeofday(&end, NULL);
// All done--close the files and return success code.
close(infile);
close(outfile);
free(ch);
return 0; // Success!
}
The main program allows the user to input the infile outfile copyFunctionNumber. If 3 is chosen the user can input a specific buffer size. So I was testing copying a file (6.3 MB) with different buffer sizes. When I choose 1024 it gives a difference of 42,000 microseconds, for 2000 it gives 26,000 microseconds, but for 3000 it gives 34,000 microseconds. My question is why does it go back up? And how could you tell what the perfect buffer size will be for the copying to take the least amount of time.

How to Calculate MD5 of xls file in C language

I have made many researches about MD5 of an xls file but my effort seems be in vain
I tried to used lirary and recommendation in this link "https://stackoverflow.com/questions/27858288/calculate-md5-for-a-file-in-c-language"
but , still give wrong result ,
can you help me ??
Well I used to answer the link you gave but then the question was closed.
The idea is as follows. First read the file into a buffer. You can do this using following function:
unsigned char * readFile(const char *path)
{
FILE * pFile;
long lSize;
unsigned char * buffer;
size_t result;
pFile = fopen (path , "rb" );
if (pFile==NULL) {fputs ("File error",stderr); exit (1);}
// obtain file size:
fseek (pFile , 0 , SEEK_END);
lSize = ftell (pFile);
rewind (pFile);
// allocate memory to contain the whole file:
buffer = malloc (sizeof(char)*lSize);
if (buffer == NULL) {fputs ("Memory error",stderr); exit (2);}
// copy the file into the buffer:
result = fread (buffer,1,lSize,pFile);
if (result != lSize) {fputs ("Reading error",stderr); exit (3);}
// terminate
fclose (pFile);
return buffer;
}
Read the file
unsigned char * data = readFile("c:\\file.xls");
Then you must apply MD5 on this buffer of data. You can use code similar
to the one in that question (though I am not sure which library/implementation
of md5 author of that question used). e.g.,
char hash[64] = {0};
md5_byte_t digest[16] = {0};
md5_init(&state);
md5_append(&state, (const md5_byte_t *)data, filesize);
md5_finish(&state,digest);
int i=0;
for(i; i<16; i++)
{
snprintf(hash + i*2,sizeof(hash),"%02x",digest[i]);
}
Now hash should store the hash of the file, encoded in hexadecimal string. ps. Indeed that sample is incorrectly using strlen with binary file. That is why I suggested the readFile method above; that function also contains code to get file size - you can use that code to get file size and then pass the file size to md5_append method.
ps. also don't forget to free data when you are done with it.
MD5 of xls file is very same of MD5 of any other kind of file since it operates on bytes. See by example openssl implementation openssl/crypto/md5/md5.c and md5test.c ( code is in git://git.openssl.org/openssl.git ).
The problem is that your example uses strlen to determine the file size. But .xls format is binary, so strlen will not work properly.
Adapt the function to return the total data read from the file, and it should work.
Edit. Try something like this code:
void *addr;
struct stat s;
int ret, fd;
ret = stat(filename, &s);
if (ret) {
fprintf(stderr, "Error while stat()ing file: %m\n");
return -1;
}
fd = open(filename, O_RDONLY);;
if (fd < 0) {
fprintf(stderr, "Error while opening file: %m\n");
return -1;
}
addr = mmap(NULL, s.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (addr == MAP_FAILED) {
fprintf(stderr, "Error while mapping file: %m\n");
close(fd);
return -1;
}
md5_init(&state);
md5_append(&state,addr, s.st_size);
md5_finish(&state,digest);

Zero'ing out a file

What is the most efficient quickest way to write all zeros to a file? including error checking. Would it just be fwrite? or is fseek involved?
I've looked elsewhere and saw code similar to this:
off_t size = fseek(pFile,0,SEEK_END);
fseek(pFile,0,SEEK_SET);
while (size>sizeof zeros)
size -= fwrite(&address, 1, sizeof zeros, pFile);
while (size)
size -= fwrite(&address, 1, size, pFile);
where zeros is an array of file size I suspect. Not sure exactly what off_t was because it wasn't directly intuitive to me anyways
Do you want to replace the contents of the file with a stream of binary zeroes of the same length, or do you want to simply empty the file? (make it have length zero)
Either way, this is best done with the OS file I/O primitives. Option one:
char buf[4096];
struct stat st;
int fd;
off_t pos;
ssize_t written;
memset(buf, 0, 4096);
fd = open(file_to_overwrite, O_WRONLY);
fstat(fd, &st);
for (pos = 0; pos < st.st_size; pos += written)
if ((written = write(fd, buf, min(st.st_size - pos, 4096))) <= 0)
break;
fsync(fd);
close(fd);
Option two:
int fd = open(file_to_truncate, O_WRONLY);
ftruncate(fd, 0);
fsync(fd);
close(fd);
Error handling left as an exercise.
mmap() and memset()

how to send an image in winsock2, using c

I am writing a very simple webserver in c (winsock2).
I am able to return the contents of my html pages.
Currently, what I am doing is writing the contents of a file into a char* buffer and sending it using "send()"
Although when I try to read an image (jpg, bmp), I can't write the characters into a buffer a some characters are "null" (0).
How can I send a whole image file ?
Thanks.
You can store null character in a char* buffer. You just have to use a counter to remember how many characters were written, instead of recomputing it by counting number of non-null characters (this can either be an integer or a pointer to the next point of insertion in the buffer).
To send a file, you'll do something like that:
int sendFile(int sock, const char* filename) {
FILE* file = fopen(filename, "rb");
if (file == NULL)
return -1;
if (fseek(file, 0, SEEK_END) != 0) {
fclose(file);
return -1;
}
off_t size = ftello(file);
if (fseek(file, 0, SEEK_SET) != 0) {
fclose(file);
return -1;
}
if (SendBinaryFileHeaderAndSize(sock, size) < 0) {
fclose(file);
return -1;
}
char buffer[4096];
for (;;) {
size_t read = fread(buffer, 1, sizeof(buffer), file);
if (read == 0) {
int retcode = 0;
if (ferror(file))
retcode = -1;
fclose(file);
return retcode;
}
for (size_t sent = 0; sent < read;) {
int ret = send(sock, buffer + sent, read - sent, 0);
if (ret < 0) {
fclose(file);
return -1;
}
assert(ret <= read - sent);
sent += ret;
}
}
}
You need to understand how send() and fread() work. 0s in the buffer are not a problem for send or fread - they do not interpret their buffers as null-terminated strings.
Depending on how you load the image into your webserver, you would need to use either Winsock:TransmitPackets or Winsock:TransmitFile, also also wrapping the image in the appropriate HTTP headers
Note that these are MS specific extensions.
Also see c++ - Bitmap transfer using winsock getdibits and setdibits

Resources