I'm new in the world of C programming language and I was trying to read an image as a binary file. Unfortunately, when I tried to read the file I got an incomplete sequence ("\377\330\377", <incomplete sequence \340>).
I've already tried to use fread reading only the size of a byte and now I'm trying to use sizeof(unsigned char), but I always get the same incomplete sequence. Here is a part of the code:
caratteri **createStruct(unsigned char c[], caratteri car[], caratteri *ptr[], long size) {
for (long i = 0; i < size; i++) {
ptr[i] = &car[i];
car[i].first = &c[i];
car[i].last = &c[(size - 1 + i) % size];
car[i].index=i;
}
return ptr;
}
int compare(const void *a, const void *b) {
caratteri *ca = *(caratteri **) a;
caratteri *cb = *(caratteri **) b;
unsigned char *c1;
unsigned char *c2;
c1 = ca->first;
c2 = cb->first;
while (*c1 - *c2 == 0) {
c1++;
c2++;
}
return (*c1 - *c2);
}
caratteri **bwt(long size, FILE *file) {
FILE *risultato;
unsigned char *c = malloc(sizeof(unsigned char) * size);
fread(c, sizeof(unsigned char), size, file);
caratteri *car = malloc(sizeof(caratteri) * size);
caratteri **pCaratteri = malloc(sizeof(caratteri *) * size);
pCaratteri = createStruct(c, car, pCaratteri, size);
qsort(pCaratteri, size, sizeof(pCaratteri), compare);
risultato=fopen("risultato","wb");
for(long i = 0; i < size; i++)
fputc(*pCaratteri[i]->last,risultato);
fclose(risultato);
return pCaratteri;
}
int main() {
FILE *file;
file = fopen("thumbnail.jpg","rb");
if (file == NULL) {
printf("Errore di apertura file!");
exit(2);
}
fseek(file, SEEK_SET, SEEK_END);
long size = ftell(file)+1;
rewind(file);
caratteri **car = bwt(size, file);
FILE *risultato;
decryptbwt(risultato);
return 0;
}
This is not the full code but I need this part to work properly.
Also, I need this code to work with every kind of file (.txt, .jpg, .png, etc) and I need the \0 character in case of a txt file, that's why I used ftell(file)+1 but I'm not sure this is appropriate. Anyway, how can I read a file in binary mode avoiding the problem of incomplete sequences?
Thank you for your answers and sorry for the dumb question but I'm new in this world.
I need to take an extra \0 because I'm using an algorithm that needs an end of string character, that's why I'm using ftell with a +1. This algorithm should work with every kind of file so I need to read it correctly and then use the burrows wheeler transform in order to sort the file that I need to compress. Also, I'm not sure that I have to use the fseek in that way but I think there is not another way to get the size of the file.
Related
The program only works for the first time. What was supposed to happen the second time was to add the same data to the binary file but that doesn't happen.
First run: It runs normal and it shows that it writed to the file.
Secound run: It writes to the file but doesnt read.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
char *name, *role, *course;
int year, id;
} StudentFile;
void saveBin(StudentFile *studentsFile, int lines){
FILE *file = fopen("studentsx.bin","ab");
if (!file) {
printf("\n\n\tImposible to open file. \n\n");
exit(1);
}
for (int i = 0; i < lines; i++){
fwrite(&studentsFile[i], sizeof(StudentFile), 1, file);
}
fclose(file);
}
void readBin(){
StudentFile *studentsFile = malloc(sizeof(StudentFile)*5000);
FILE *file = fopen("studentsx.bin","rb");
if (!file) {
printf("\n\n\tImposible to open file. \n\n");
exit(1);
}
int j = 0;
while (fread(&studentsFile[j], sizeof(StudentFile), 1, file)){
printf("\nLine read %d: %s\t%s\t%d\t%d\t%s", j+1, studentsFile[j].name, studentsFile[j].role, studentsFile[j].year, studentsFile[j].id, studentsFile[j].course);
j++;
}
fclose(file);
}
void main(){
StudentFile *studentsFile = malloc(sizeof(StudentFile)*2);
int lines = 0;
studentsFile[0].name = "John";
studentsFile[0].role = "Gamer";
studentsFile[0].year = 1999;
studentsFile[0].id = 1;
studentsFile[0].course = "IOT";
studentsFile[1].name = "Piter";
studentsFile[1].role = "GamerXL";
studentsFile[1].year = 1991;
studentsFile[1].id = 2;
studentsFile[1].course = "IOTXL";
lines = 2;
saveBin(studentsFile, lines);
readBin();
}
You are writing pointers, not strings. fwrite writes single contiguous array of memory. In your case the StudentFiles and actual strings are scattered all over the static memory and heap memory.
Consider your struct:
typedef struct {
char *name, *role, *course;
int year, id;
} StudentFile;
it looks something like this in memory:
[<pointer to name><pointer to role><pointer to course><year><id>]
somewhere else in a different block of memory:
[John\0Gamer\0\OIT\o.......]
You wrote the first block above and left out the second one.
There are multiple approaches to this problem and we usually name them "serialization" - take your complex data structure and serialize it into a linear file.
One of the approaches is to allocate fixed size block within your structure StudentFile:
#define MAX_NAME 100
#define MAX_ROLE 100
#define MAX_COURSE 100
typedef struct {
char name[MAX_NAME];
char role[MAX_ROLE];
char course[MAX_COURSE];
int year, id;
} StudentFile;
then strings name, role and course will be inside of StudentFile:
[<100 bytes for name><100 bytes for role><100 bytes for course><year><id>]
this is contiguous block of memory and if can be written using single call to fwrite like you did.
But you won't be able to assign strings like you did with
studentsFile[i].name = "John";
C has strncpy for that:
strcpy(studentsFile[0].name, "John", MAX_NAME);
Another approach is to have several calls to fwrite. For every string, you write length first, then the string itself. For primitive types like int you just write that int.
First you gather strings from different locations pointed by the pointers:
size_t nameLen = strlen(studentsFile[i].name) + 1;/* +1 for the final zero*/
fwrite(&nameLen, sizeof(size_t), 1, file);
fwrite(studentsFile[i].name, nameLen, 1, file);
size_t roleLen = strlen(studentsFile[i].role) + 1;
fwrite(&roleLen, sizeof(size_t), 1, file);
fwrite(studentsFile[i].role, roleLen, 1, file);
size_t courseLen = strlen(studentsFile[i].course) + 1;
fwrite(&courseLen, sizeof(size_t), 1, file);
fwrite(studentsFile[i].course, courseLen, 1, file);
Then you write primitive types:
fwrite(&studentsFile[i].year, sizeof(int), 1, file);
fwrite(&studentsFile[i].id, sizeof(int), 1, file);
Next time when you read the file, you rely on the order of writes and read the fields back in the same order:
size_t nameLen;
fread(&nameLen, sizeof(size_t), 1, file);
char *name = malloc(nameLen);
fread(name, nameLen, 1, file);
size_t roleLen;
fread(&roleLen, sizeof(size_t), 1, file);
char *role = malloc(roleLen);
fread(role, roleLen, 1, file);
size_t courseLen;
fread(&courseLen, sizeof(size_t), 1, file);
char *course = malloc(courseLen);
fread(course, courseLen, 1, file);
int year;
fread(&year, sizeof(int), 1, file);
int id;
fread(&id, sizeof(int), 1, file);
printf("\nLine read %d: %s\t%s\t%d\t%d\t%s", j+1, name, role, year, id, course);
The problem lies somewhere else: Think carefully, what is your code doing with fwrite() here?
typedef struct {
char *name, *role, *course;
int year, id;
} StudentFile;
fwrite(&studentsFile[i], sizeof(StudentFile), 1, file);
What does the file content look like after writing a single element from studentFile?
Three strings and two integers (in their binary form)
Three pointers to somewhere and two integers (all in their binary forms)
As the title states, I'm trying to read a JPEG file using libjpeg-turbo. I tried this code on a mac at home and it worked, but now I'm on Windows and it's giving me a Empty input file error on calling jpeg_read_header. I have verified that the file is not empty by doing a fseek/ftell, and the size I get corresponds to what I expect it to be.
My initial thoughts were that I might not have been opening the file in binary mode, so I tried that as well using _setmode, but that didn't seem to help. Here is my code for reference.
int decodeJpegFile(char* filename)
{
FILE *file = fopen(filename, "rb");
if (file == NULL)
{
return NULL;
}
_setmode(_fileno(file), _O_BINARY);
fseek(file, 0L, SEEK_END);
int sz = ftell(file);
fseek(file, 0L, SEEK_SET);
struct jpeg_decompress_struct info; //for our jpeg info
struct jpeg_error_mgr err; //the error handler
info.err = jpeg_std_error(&err);
jpeg_create_decompress(&info); //fills info structure
jpeg_stdio_src(&info, file);
jpeg_read_header(&info, true); // ****This is where it fails*****
jpeg_start_decompress(&info);
int w = info.output_width;
int h = info.output_height;
int numChannels = info.num_components; // 3 = RGB, 4 = RGBA
unsigned long dataSize = w * h * numChannels;
unsigned char *data = (unsigned char *)malloc(dataSize);
unsigned char* rowptr;
while (info.output_scanline < h)
{
rowptr = data + info.output_scanline * w * numChannels;
jpeg_read_scanlines(&info, &rowptr, 1);
}
jpeg_finish_decompress(&info);
fclose(file);
FILE* outfile = fopen("outFile.raw", "wb");
size_t data_out = fwrite(data, dataSize, sizeof(unsigned char), outfile);
}`
Any help is much appreciated!
The core of the issue is a dll mismatch. The libjpeg is built agains msvcrt.dll, whereas the app is built against whatever runtime provided by MSVS2015. They are incompatible, and the file pointers opened in one runtime make no sense to another.
The solution, as per this discussion, is to avoid jpeg_stdio_src API.
You are passing C++ true value to jpeg_read_header -- that could also be the reason for failure. You should pass TRUE constant instead.
Consider the following abstracted code that reads some bytes from a file:
typedef struct A{
int size;
char * dataArray;
}A
A load(char* filename, int inSize)
{
A newA;
newA.size = inSize;
FILE *filePtr;
filePtr = fopen(filename,"rb");
char buff[1];
int i = 0;
newA.dataArray = ( char*)malloc(sizeof(char) * newA.size);
for (i = 0; i < newA.size; i++)
{
fread(buff, sizeof(char), 1, filePtr);
newA.dataArray[i] = buff[0];
}
char* copyOfDataArray = (char*)malloc(sizeof(char) * newA.size);
for (i = 0; i < newA.size; i++)
{
fread(buff, sizeof(char), 1, filePtr);
copyOfDataArray[i] = newA.dataArray[i];
}
newA.dataArray = copyOfDataArray;
return newA
}
void Initialize()
{
A first = load("file1", 100);
A second = load("file2", 20);
}
Both calls to function load return the expected result (data array has the same bytes as the file). Variables first and second are never used again.
However after a couple of hundreds lines of code the program always crashes with:
*malloc.c:2451: sYSMALLOC: Assertion '(old_top == (..... failed.*
The crash always occurs on the same line of code, but that line has nothing to do with variables first, second or even with struct A whatsoever.
My question is: is my way of instancing and loading 'first' and 'second' wrong? Can it cause some kind of memory leak / memory overflow that crashes the program long after the load function has finished?
Bonus: The crash does not occur if I only load "file1", as soon as i load both "file1" and "file2" the crash reappears.
Sorry for the long question.
You have memory leaks there. You have to free the previously allocated memory in newA.dataArray, before you assign there a new memory.
As stated by Joachim, read operation is very time consuming and you shall read data in blocks to minimize overhead.
Additionally, you have to close file descriptors, otherwise they will be depleted soon.
There are many issue on the code as already given by others.
Please checks bellow
typedef struct A{
int size;
char * dataArray;
}A
A load(char* filename, int inSize)
{
A newA;
newA.size = inSize;
FILE *filePtr = NULL ; //Use NULL
char buff[1]; //Size of buffer is only 1 ,If needed increase that to copy more at a time
int i = 0;
filePtr = fopen(filename,"rb");
//Try to check for the filePtr == NULL or not
newA.dataArray = ( char*)malloc(sizeof(char) * newA.size);
//Same checking should be done here
for (i = 0; i < size; i++) //What is size
{
fread(buff, sizeof(char), 1, filePtr);
newA.dataArray[i] = char[0]; //What is char[0]
}
//instead this you can read the bytes in a single call, use that.
// fread(buff, sizeof(char), <size to read >, filePtr);
char* copyOfDataArray = (char*)malloc(sizeof(char) * newA.size);
for (i = 0; i < size; i++)
{
fread(buff, sizeof(char), 1, filePtr);
copyOfDataArray[i] = newA.dataArray[i];
}
//why reading again once you done above.
newA.dataArray = copyOfDataArray;
return newA; //Please check: How you can return a auto variable.
}
void Initialize()
{
A first = load("file1", 100);
A second = load("file2", 20);
}
Firstly, i'm not very familiarized with C, i come from Java, C#, C++... and possibly i inherited defects from this languages in order to realize this practice, well i have the follows question, here is my code:
#include <stdio.h>
#include <stdlib.h>
void decrypt(unsigned long* v, unsigned long* k);
const int MAX = 32;
const long delta = 0x9e3779b9;
long sum=0xC6EF3720;
int main() {
FILE *fp;
FILE *destino;
unsigned long v[2];
unsigned long k[4] = { 128, 129, 130, 131 };
unsigned long tam=0;
char* buffer;
char* aux[sizeof(unsigned long)];
int i;
if ((fp = fopen("image.png", "rb")) == NULL) {
printf ("Error! \n ");
return 0;
}
else {
fread(&aux,sizeof(unsigned long),1,fp);
memcpy(&tam,&aux,sizeof(unsigned long));
buffer = (char*)malloc(tam);
//fread(&buffer,1,tam,fp);
char *buffer2[28568];
fread(&buffer2,1,28568,fp);
/*for(i = 0;i < tam;++i) {
printf("%c", ((char *)buffer2)[i]);
}*/
for(i=4;i<tam;i+=8) {
memcpy(&v,&buffer2[i],8);
decrypt(&v,&k);
}
if ((result= fopen("image2.png", "rb")) == NULL) {
printf ("Error! \n ");
return 0;
}
else {
fwrite(v,sizeof(unsigned long)*2,1,result);
fclose (result);
fclose(fp);
}
}
return 0;
}
void decrypt(unsigned long* v, unsigned long* k) {
int i=0;
while(i<MAX) {
v[1] = v[1] -((4 << v[0])+(k[2]^v[0])+(sum^(5 >> v[0]))+k[3]);
v[0] = v[0] -((4 << v[1])+(k[0]^v[1])+(sum^(5 >> v[1]))+k[1]);
sum = sum-delta;
i++;
}
}
Where tam is the size of my binary file (image in this case) where i store first 4 bytes (unsigned long) where is located the size in my png file (28568)
When i create my char* buffer i have to assign dynamically with malloc but when i make a new fread from my file i get a "No source available for "msvrct!memcpy() at 0xrandom_memory_address" from Eclipse when i debug, well, i comment this line and i try to make it manually set a new buffer2 with 28568 as size of my array, apparently works, making a iteration of buffer2 prints ascii characters values but when i call decrypt for make the decryption of my image, the final result is stored in v array which i have to copy in a new file, i tried to search how to make a empty image png in C but i didn't find anything, so i created a copy of my encrypt image calling it "image2.png" but i suppose this not the "clean solution" for that, because for the other hand is not working at all.
For more explanation about this exercise just say that the decrypt funcion work with blocks of 8 bytes (64 bits) that through a key (array k) make a series of operation where they store in v array itself, crossing through the loop 8 in 8 and retrieve the value of buffer in v in each one, after the loop execution we have the result in v and only left to copy in a new file where finally show up the image decrypt.
It's a very complex practice for all of one newbies in C, it's driving my crazy trying to figure out what i doing wrong.
I hope anyone can see what i'm not able to for now.
I think you are having problems with the declarations of the buffers. I think the correct should be:
FILE *fp;
FILE *destino;
unsigned long v[2];
unsigned long k[4] = { 128, 129, 130, 131 };
unsigned long tam=0;
char* buffer;
char aux[sizeof(unsigned long)]; // without the "*"
int i;
if ((fp = fopen("image.png", "rb")) == NULL) {
printf ("Error! \n ");
return 0;
}
else {
fread(aux,sizeof(unsigned long),1,fp);
memcpy(&tam,aux,sizeof(unsigned long));
buffer = (char*)malloc(tam);
//fread(buffer,1,tam,fp); // without the "&" in this case
char buffer2[28568]; // without the "*"
fread(buffer2,1,28568,fp); // or fread(buffer,1,tam,fp);
/*for(i = 0;i < tam;++i) {
printf("%c", buffer2[i]); // or buufer[i] if you change to use it again
}*/
for(i=4;i<tam;i+=8) {
memcpy(v,&buffer2[i],8);
decrypt(v,k);
}
...
I don't fully understand what you are trying to accomplish, but one problem is here:
char* aux[sizeof(unsigned long)];
// ... some code ...
fread(&aux,sizeof(unsigned long),1,fp);
Understand that char* aux[sizeof(unsigned long)]; means that you are declaring a double pointer, but fread() prototype states that the destination is a single pointer:
size_t fread(void *ptr, size_t size, size_t nmemb, FILE *stream);
so what you should be doing instead is:
char aux[sizeof(unsigned long)];
// ... some code ...
fread(aux,sizeof(unsigned long),1,fp);
Don't complicate things that are not complicated!
You also do this mistake in other parts of your code, you need to re-check everything, ok? Again:
char *buffer2[28568];
fread(&buffer2,1,28568,fp);
should be:
char buffer2[28568];
fread(buffer2, 1, 28568, fp);
// or: fread(buffer2, 1, sizeof(buffer2), fp);
There are some interesting tutorials on pointers and arrays, I suggest you read some.
I'm having an infuriating issue here where I'm crashing on malloc/calloc/strdup and I'm assuming currently that it's because of a buffer over run somewhere.
I'm finding this very difficult to find and I was wondering if any of you can offer me a hand. I'll post code snippets here, and link to full source.
File reading and array operations: (common.c)
Pastebin
char * S6_ReadFileBytes(const char* path)
FILE * file;
long length;
char * bytes = NULL;
file = fopen(path, "r");
fseek(file, 0, SEEK_END)
length = ftell(file);
fseek(file, 0, 0);
bytes = (char*)calloc(1, (size_t)length + 1);
fread(bytes, 1, (size_t)length, file);
return bytes;
S6_Array * S6_ArrayNew(size_t count, size_t typeSize)
S6_Array * a = (S6_Array*)malloc(sizeof(S6_Array));
a->typeSize = typeSize;
a->Length = count;
void * S6_ArrayGet(S6_Array * a, int idx)
return &((char*)a->Data)[idx * a->typeSize];
void S6_ArraySet(S6_Array * a, int idx, void * val)
memcpy(&((char*)a->Data)[idx * a->typeSize], val, a->typeSize);
void S6_ArrayGrow(S6_Array * a, int amount)
void * data;
data = realloc(a->Data, (a->Length + amount) * a->typeSize);
a->Data = data;
a->Length += amount;
void S6_ArrayPushBack(S6_Array * a, void* val)
S6_ArrayGrow(a, 1);
S6_ArraySet(a, a->Length - 1, val);
CSV Reading: (CSV.c)
Pastebin
void S6_CSV_PushRect(S6_Array ** rectangles, S6_Rectangle * rect)
if( !*rectangles )
*rectangles = S6_ArrayNew(1, sizeof(S6_Rectangle*));
S6_ArraySet(*rectangles, 0, &rect);
else
S6_ArrayPushBack(*rectangles, &rect);
int S6_CSV_ReadRects(const char* file, S6_Array ** rectangles)
char * bytes = S6_ReadFileBytes(file);
char * line;
char * nameIndex;
size_t nameLength;
S6_Rectangle * tempRect;
line = strtok( bytes , "\n");
while( line )
nameIndex = strstr(line, ",");
tempRect = (S6_Rectangle*)calloc(1, sizeof(S6_Rectangle));
nameLength = (size_t)(nameIndex - line) + 1;
strncpy(tempRect->name, line, nameLength-1);
tempRect->name[nameLength-1] = '\0';
sscanf(nameIndex, "%*[,]%d%*[,]%d%*[,]%d%*[,]%d", &tempRect->x, &tempRect->y, &tempRect->w, &tempRect->h)
S6_CSV_PushRect(rectangles , tempRect);
strtok(NULL, "\n");
free(bytes);
A function where I modify the array: (BinPacker.c)
Pastebin
int S6_BinPacker_Pack(S6_Array * rectangles, int binSize)
// This sort appears to be working fine. View pastebin for test.
qsort(rectangles->Data, rectangles->Length, sizeof(S6_Rectangle*), S6_BinPacker_CompareRects);
CSV Writing [CRASH]
: (CSV.c)
Pastebin
void S6_CSV_WriteRects(const char* file, S6_Array * rectangles)
char * bytes = NULL;
char buffer[128];
S6_Rectangle * tempRect;
size_t i;
for( i = 0; i < rectangles->Length; ++i)
tempRect = *(S6_Rectangle**)S6_ArrayGet(rectangles, i);
memset(buffer, '\0', sizeof(buffer));
sprintf(buffer,
"%s,%d,%d,%d,%d\n",
tempRect->name,
temprect->x,
temprect->y,
temprect->w,
temprect->h);
if( bytes )
bytes = strcat(bytes, _strdup(buffer));
else
bytes = _strdup(buffer);
So I'm crashing here on the strcat(bytes, _strdup(buffer)) line. When I separate it out It's still the string duplication or any sort of allocation I've tried.
I get the following break dialog from visual studio:
Windows has triggered a breakpoint in myapp.exe.
This may be due to a corruption of the heap, which indicates a bug in Slant6.Debug.exe or any of the DLLs it has loaded.
This may also be due to the user pressing F12 while Slant6.Debug.exe has focus.
The output window may have more diagnostic information.
And the break point it triggers is in tidtable.c on
PFLS_GETVALUE_FUNCTION flsGetValue = FLS_GETVALUE;
SOLUTION
strdup doesn't do any allocations, and even if it did I would be leaking like crazy. So instead of:
bytes = strcat(bytes, _strdup(buffer));
in CSV.c, I replaced it with some manual string concatenation that's easier for me to read (and remember).
size_t oldSize = strlen(bytes);
size_t bufferSize = strlen(buffer);
size_t newSize = oldSize + bufferSize ;
char * newMem = (char*)calloc(newSize + 1, 1);
memcpy(newMem, bytes, newSize);
memcpy(&newMem[oldSize], buffer, bufferSize);
free(bytes);
bytes = newMem;
/SOLUTION
I'm thinking that this line:
bytes = strcat(bytes, _strdup(buffer));
Does not do what you think it does.
You are making a copy of a string (buffer), and then concatenating that onto bytes. The duplicated string is never freed and
bytes is only as big as the last _strdup, thus doing a strcat will overflow the buffer.
You need to allocate (or reallocate) strlen(bytes) + strlen(buffer), etc. etc. for the strcat.