Memory problems with C - c

Hello and have a good day, I've come here after days of trial and error so forgive me if I'm beign silly.
I have the following code. The idea of this code is first of all read all the files I have and store all the data into a matrix NsitesxNxxNy and then use the data for other unrelated things.
The amount of data is not very much, I mean i have 800 files of data which occupe no more than 80MB but anyway if I try to use a number for DataFiles higher than 134 I get a Segmentation Fault error.
I think it's weird because if it works with a number of DataFiles=100 why it should'nt work for higher?
I thought it was because for a reason my program does not get enough memory allocated for the process or because I'm having an issue when allocating the memory. But I always have the same amount of data and my data files have exactly 88*44 values and working only until 134 files it's... I don't have experience with "high amount" of data/memory usage but I think that 1000*88*44 which is about 10^6 double digits it's not too much.
I'm using GCC compiler and Ubuntu (14.02 I think), when I try to compile and execute this program in Windows using Codeblocks it just crashes (another mistery).
Oh I also had a terminal open with RAM memory usage and with 134 files it was nothing big to handle for the computer.
EDIT: I also tried making several [100][Nx][Ny] arrays and use them one by one but that also lead to the Segmentation Fault error.
EDIT2: minor erratas text and code
Also, I'm following this way because I need all that data simultaneously... I'm thinking of new ways of avoiding this but last couple days did'nt find any alternative.
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
const int Nx=88; //
const int Ny=44; //
const int DataFiles=100; // How many data files are we going to read
int main() {
int i, j, ki , kj ,index;
double fun[DataFiles][Nx][Ny], Res[DataFiles][Nx][Ny],mean[Nx][Ny];
FILE * conf;
char file[100];
for (index=0; index<DataFiles; index++){
sprintf(file,"data//file%i00.txt",5000+index*25);
conf=fopen(file,"r");
for(ki=0;ki<Nx;ki++){
for(kj=0;kj<Ny;kj++){
fscanf(conf,"%i %i %lf", &i, &j, &fun[index][ki][kj]);
mean[ki][kj] = mean[ki][kj] + fun[index][ki][kj] ;
}}
fclose (conf);
}
// do things with my loaded data
}

You ran out of stack. Generally speaking, don't allocate more than 8k at once on the stack. Oops.
Change
double fun[DataFiles][Nx][Ny], Res[DataFiles][Nx][Ny],mean[Nx][Ny];
To
double (*fun)[Nx][Ny] = malloc(sizeof(fun[0]) * DataFiles), (*Res)[Nx][Ny] = malloc(sizeof(Res[0]) * DataFiles), mean[Nx][Ny];
if (!fun || !Res) {
/* handle OOM */
}

Related

char** : Parse the entire memory properly

Im am trying to get some words from a txt file and store them into a char** . Although I think my array allocates corectly room for 100 words , my last one fails to be printed in a human-readable way.Also,if you look carefully there are 4 characters that are being printed (so this indicates that the correct size of memory refers to the lenght of the word,which is 4). Can you please point out for what am I missing ? I am running in linux Ubuntu 20.04 (VirtualBox).
#include<stdio.h>
#include<stdlib.h>
#include<sys/types.h>
#include<sys/stat.h>
#include <fcntl.h>
#define N 100
#define STRLIM 60
char** cat;
int fd;
int main(int argc, char const *argv[])
{ int i;
cat=(char**)malloc(sizeof(char)*N);
fd=open("dictionary.txt",O_RDONLY);
if(fd==-1){
printf("fd error %d" , fd);
perror("Error:");
}
for(i=0;i<N;i++){
cat[i]=(char*)malloc(sizeof(char)*STRLIM);
read(fd,cat[i],40);
printf("%s", cat[i]);
}
printf("the value of i is: %d",i);
return 0;
}//end of main
PS: I am getting a warning that which goes like this :
v1.c:25:9: warning: implicit declaration of function ‘read’; did you mean ‘fread’? [-Wimplicit-function-declaration]
Is it really necessary for "read" to have an unsigned int as a third arguement?
The words of the "dictionary.txt" are these (there a hundred words down bellow) :
shocking
boundary
post
dapper
zoom
bashful
damaging
sore
unadvised
fresh
birthday
wrathful
hook
nose
wonder
doubt
important
synonymous
bell
dare
selective
raspy
unused
heavy
wiggly
land
coal
humorous
humdrum
plausible
languid
depressed
imminent
helpless
parsimonious
verse
deep
tricky
window
sedate
torpid
earsplitting
protect
breath
drawer
pear
bomb
drum
can
superficial
crook
stimulating
majestic
innocent
steep
robin
weak
tumble
geese
bulb
channel
frantic
obtain
shave
nerve
boiling
picture
sand
measly
tasteful
steadfast
hallowed
rabid
fax
aspiring
utter
wave
confused
zephyr
absent
lamentable
idea
oatmeal
comfortable
cars
reduce
colossal
heat
lettuce
simple
homeless
decision
cellar
ruthless
time
railway
possible
silly
chance
food
Sorry, wasn't able to add a comment...
you won't get a warning if you include unistd.h.
I don't know why you want to use the malloc function in this code (maybe for another part of the code).
It is better to have a buffer and do something like this:
char buffer[STRLIM];
int x;
while ((x = read(fd, buffer, STRLIM)) > 0) {
write(1, buffer, x);
}
where x is the number of bytes that the read function was able to read from the file, and then we write x bytes to the output.

libopcodes: get the size of a instruction

I have to find out the size of a instruction which I have in memory (actually, I have a small code segment in memory and want to get the size of the first instruction).
It took me some time to find libopcodes and libbfd. I red the headers and tried to come up with a simple solution but it seems like I missunderstood something since the program always crashes:
int main(int argc, char **argv) {
disassemble_info *dis = malloc(sizeof(*dis));
assert(dis != NULL);
dis->arch = bfd_arch_i386;
dis->read_memory_func = buffer_read_memory;
dis->buffer_length = 64;
dis->buffer = malloc(dis->buffer_length);
memset(dis->buffer, 0x90, dis->buffer_length);
disassemble_init_for_target(dis);
int instr_size = print_insn_i386(0, dis);
printf("instruction size is %d\n", instr_size);
return 0;
}
The expected result would be an instruction size of 1 (nop).
EDIT:
sorry guys, I'm a stupid person.
memset(dis, 0, sizeof(*dis));
There is some code in the Linux kernel you can steal. It should work well if copied into a user mode program.
Take a look at arch/x86/lib and arch/x86/tools
There's an opcode map file there, and an awk script that reads it to produce a table in a file named innat.c. There are some other files there that use the table to implement a decoder.
It is sufficient to determine instruction sizes.
This assumes you are ok with GPL, of course.
It looks like the disassemble_info data structure requires more initialization than you have provided. From examples I have been studying, the correct way to initialize is to call init_disassemble_info().
See if that helps. Failing that, compile your program with debug info ('-g') and run gdb to diagnose where the crash occurs.

OpenCL atomics do nothing

I try to program an example histogram tool using OpenCL. To start, I was just interessted to atomicly increment each bin. I came up with the following kernel code:
__kernel void Histogram(
__global const int* input,
__global int* histogram,
int numElements) {
// get index into global data array
int iGID = get_global_id(0);
// bound check, equivalent to the limit on a 'for' loop
if (iGID >= numElements) {
return;
}
if( iGID < 100 ) {
// initialize histogram
histogram[iGID] = 0;
}
barrier(CLK_GLOBAL_MEM_FENCE);
int bin = input[iGID];
atomic_inc(&histogram[bin]);
}
But the output histogram is zero in every bin. Why is that? Further more, the real strange things happen if a put a printf(" ") in the last line. Suddenly, it works. I am completely lost, has someone an idea why this happens?
P.S.
I enabled all extensions
I solved to problem by my self.
After nothing fixed the problem, I tried to change the CLDevice to the CPU. Everything went as it was supposed to be (unfortunately very slow :D). But this gave me the idea that it might not be a code problem but a OpenCL infrastructure problem.
I updated the OpenCL platform of AMD and now everything works.
Thank you, in case you thought about my problem.

How to write self modifying code in C?

I want to write a piece of code that changes itself continuously, even if the change is insignificant.
For example maybe something like
for i in 1 to 100, do
begin
x := 200
for j in 200 downto 1, do
begin
do something
end
end
Suppose I want that my code should after first iteration change the line x := 200 to some other line x := 199 and then after next iteration change it to x := 198 and so on.
Is writing such a code possible ? Would I need to use inline assembly for that ?
EDIT :
Here is why I want to do it in C:
This program will be run on an experimental operating system and I can't / don't know how to use programs compiled from other languages. The real reason I need such a code is because this code is being run on a guest operating system on a virtual machine. The hypervisor is a binary translator that is translating chunks of code. The translator does some optimizations. It only translates the chunks of code once. The next time the same chunk is used in the guest, the translator will use the previously translated result. Now, if the code gets modified on the fly, then the translator notices that, and marks its previous translation as stale. Thus forcing a re-translation of the same code. This is what I want to achieve, to force the translator to do many translations. Typically these chunks are instructions between to branch instructions (such as jump instructions). I just think that self modifying code would be fantastic way to achieve this.
You might want to consider writing a virtual machine in C, where you can build your own self-modifying code.
If you wish to write self-modifying executables, much depends on the operating system you are targeting. You might approach your desired solution by modifying the in-memory program image. To do so, you would obtain the in-memory address of your program's code bytes. Then, you might manipulate the operating system protection on this memory range, allowing you to modify the bytes without encountering an Access Violation or '''SIG_SEGV'''. Finally, you would use pointers (perhaps '''unsigned char *''' pointers, possibly '''unsigned long *''' as on RISC machines) to modify the opcodes of the compiled program.
A key point is that you will be modifying machine code of the target architecture. There is no canonical format for C code while it is running -- C is a specification of a textual input file to a compiler.
Sorry, I am answering a bit late, but I think I found exactly what you are looking for : https://shanetully.com/2013/12/writing-a-self-mutating-x86_64-c-program/
In this article, they change the value of a constant by injecting assembly in the stack. Then they execute a shellcode by modifying the memory of a function on the stack.
Below is the first code :
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <sys/mman.h>
void foo(void);
int change_page_permissions_of_address(void *addr);
int main(void) {
void *foo_addr = (void*)foo;
// Change the permissions of the page that contains foo() to read, write, and execute
// This assumes that foo() is fully contained by a single page
if(change_page_permissions_of_address(foo_addr) == -1) {
fprintf(stderr, "Error while changing page permissions of foo(): %s\n", strerror(errno));
return 1;
}
// Call the unmodified foo()
puts("Calling foo...");
foo();
// Change the immediate value in the addl instruction in foo() to 42
unsigned char *instruction = (unsigned char*)foo_addr + 18;
*instruction = 0x2A;
// Call the modified foo()
puts("Calling foo...");
foo();
return 0;
}
void foo(void) {
int i=0;
i++;
printf("i: %d\n", i);
}
int change_page_permissions_of_address(void *addr) {
// Move the pointer to the page boundary
int page_size = getpagesize();
addr -= (unsigned long)addr % page_size;
if(mprotect(addr, page_size, PROT_READ | PROT_WRITE | PROT_EXEC) == -1) {
return -1;
}
return 0;
}
It is possible, but it's most probably not portably possible and you may have to contend with read-only memory segments for the running code and other obstacles put in place by your OS.
This would be a good start. Essentially Lisp functionality in C:
http://nakkaya.com/2010/08/24/a-micro-manual-for-lisp-implemented-in-c/
Depending on how much freedom you need, you may be able to accomplish what you want by using function pointers. Using your pseudocode as a jumping-off point, consider the case where we want to modify that variable x in different ways as the loop index i changes. We could do something like this:
#include <stdio.h>
void multiply_x (int * x, int multiplier)
{
*x *= multiplier;
}
void add_to_x (int * x, int increment)
{
*x += increment;
}
int main (void)
{
int x = 0;
int i;
void (*fp)(int *, int);
for (i = 1; i < 6; ++i) {
fp = (i % 2) ? add_to_x : multiply_x;
fp(&x, i);
printf("%d\n", x);
}
return 0;
}
The output, when we compile and run the program, is:
1
2
5
20
25
Obviously, this will only work if you have finite number of things you want to do with x on each run through. In order to make the changes persistent (which is part of what you want from "self-modification"), you would want to make the function-pointer variable either global or static. I'm not sure I really can recommend this approach, because there are often simpler and clearer ways of accomplishing this sort of thing.
A self-interpreting language (not hard-compiled and linked like C) might be better for that. Perl, javascript, PHP have the evil eval() function that might be suited to your purpose. By it, you could have a string of code that you constantly modify and then execute via eval().
The suggestion about implementing LISP in C and then using that is solid, due to portability concerns. But if you really wanted to, this could also be implemented in the other direction on many systems, by loading your program's bytecode into memory and then returning to it.
There's a couple of ways you could attempt to do that. One way is via a buffer overflow exploit. Another would be to use mprotect() to make the code section writable, and then modify compiler-created functions.
Techniques like this are fun for programming challenges and obfuscated competitions, but given how unreadable your code would be combined with the fact you're exploiting what C considers undefined behavior, they're best avoided in production environments.
In standard C11 (read n1570), you cannot write self modifying code (at least without undefined behavior). Conceptually at least, the code segment is read-only.
You might consider extending the code of your program with plugins using your dynamic linker. This require operating system specific functions. On POSIX, use dlopen (and probably dlsym to get newly loaded function pointers). You could then overwrite function pointers with the address of new ones.
Perhaps you could use some JIT-compiling library (like libgccjit or asmjit) to achieve your goals. You'll get fresh function addresses and put them in your function pointers.
Remember that a C compiler can generate code of various size for a given function call or jump, so even overwriting that in a machine specific way is brittle.
My friend and I encountered this problem while working on a game that self-modifies its code. We allow the user to rewrite code snippets in x86 assembly.
This just requires leveraging two libraries -- an assembler, and a disassembler:
FASM assembler: https://github.com/ZenLulz/Fasm.NET
Udis86 disassembler: https://github.com/vmt/udis86
We read instructions using the disassembler, let the user edit them, convert the new instructions to bytes with the assembler, and write them back to memory. The write-back requires using VirtualProtect on windows to change page permissions to allow editing the code. On Unix you have to use mprotect instead.
I posted an article on how we did it, as well as the sample code.
These examples are on Windows using C++, but it should be very easy to make cross-platform and C only.
This is how to do it on windows with c++. You'll have to VirtualAlloc a byte array with read/write protections, copy your code there, and VirtualProtect it with read/execute protections. Here's how you dynamically create a function that does nothing and returns.
#include <cstdio>
#include <Memoryapi.h>
#include <windows.h>
using namespace std;
typedef unsigned char byte;
int main(int argc, char** argv){
byte bytes [] = { 0x48, 0x31, 0xC0, 0x48, 0x83, 0xC0, 0x0F, 0xC3 }; //put code here
//xor %rax, %rax
//add %rax, 15
//ret
int size = sizeof(bytes);
DWORD protect = PAGE_READWRITE;
void* meth = VirtualAlloc(NULL, size, MEM_COMMIT, protect);
byte* write = (byte*) meth;
for(int i = 0; i < size; i++){
write[i] = bytes[i];
}
if(VirtualProtect(meth, size, PAGE_EXECUTE_READ, &protect)){
typedef int (*fptr)();
fptr my_fptr = reinterpret_cast<fptr>(reinterpret_cast<long>(meth));
int number = my_fptr();
for(int i = 0; i < number; i++){
printf("I will say this 15 times!\n");
}
return 0;
} else{
printf("Unable to VirtualProtect code with execute protection!\n");
return 1;
}
}
You assemble the code using this tool.
While "true" self modifying code in C is impossible (the assembly way feels like slight cheat, because at this point, we're writing self modifying code in assembly and not in C, which was the original question), there might be a pure C way to make the similar effect of statements paradoxically not doing what you think are supposed do to. I say paradoxically, because both the ASM self modifying code and the following C snippet might not superficially/intuitively make sense, but are logical if you put intuition aside and do a logical analysis, which is the discrepancy which makes paradox a paradox.
#include <stdio.h>
#include <string.h>
int main()
{
struct Foo
{
char a;
char b[4];
} foo;
foo.a = 42;
strncpy(foo.b, "foo", 3);
printf("foo.a=%i, foo.b=\"%s\"\n", foo.a, foo.b);
*(int*)&foo.a = 1918984746;
printf("foo.a=%i, foo.b=\"%s\"\n", foo.a, foo.b);
return 0;
}
$ gcc -o foo foo.c && ./foo
foo.a=42, foo.b="foo"
foo.a=42, foo.b="bar"
First, we change the value of foo.a and foo.b and print the struct. Then we change only the value of foo.a, but observe the output.

Heap corruption while using OpenCV datastructure

I am using OpenCV 2.1 with codeblocks (gcc under mingw). Within my code I am trying (for some sane reason) to access the imagedata within IplImage datastructure directly. Kindly refer the code snippet for more details:
int main(void)
{
IplImage* test_image = cvLoadImage("test_image.bmp",CV_LOAD_IMAGE_GRAYSCALE);
int mysize = test_image->height * test_image->widthStep;
char* imagedata_ptr = NULL;
int i = 0;
imagedata_ptr = test_image->imageData;
char* temp_buff = (char *)malloc(sizeof(mysize));
memcpy(temp_buff,imagedata_ptr,mysize);
free(temp_buff);
}
When I run this code it crashes. On running it in the debug mode it generates a SIGTRAP is due to heap corruption. At first I suspected that this might be a compiler related issue and hence tried running the same code in Visual Studio. But it still crashes. Thats the reason I feel it could be an OpenCV related issue.
NOTE: There are no other instances of program open, this is the only code that I am running, no threading etc is being done here.
Awaiting your comments on the same.
Regards,
Saurabh Gandhi
You're not allocating enough memory, this:
char* temp_buff = (char *)malloc(sizeof(mysize))
only allocates sizeof(int) bytes (probably 4) and that's probably a lot less than you need. Then the memcpy right after that will copy test_image->height * test_image->widthStep bytes of data into somewhere that only has space for sizeof(int) bytes and you have now scribbled all over your memory and corrupted your heap.
I'd guess that you really want to say this:
char *temp_buff = malloc(mysize);
And don't cast the return value from malloc, you don't need it and it can hide problems.

Resources