How to work with large integer, do I need GMP libraries or something?
I want an array that has elements starting from 0 to 2^32
How to get this to work:
#include <stdio.h>
int main(){
unsigned int i,j=0,sz=4294967295;
unsigned int A[sz];
A[j]=0;
for(i=1;i<=sz;i++){
A[i]=A[j]+1 ;
j++;
printf("%u\n",A[i]);
}
return 0;
}
error: process exited with return value 3221225725
is it that the array is too big or something??
According to Google, your A array is approximately 17 gigabytes. That's a lot. You're probably overflowing the stack.
If you really need this much memory, you may be able to malloc() it instead, but on older 32-bit architectures, you're basically out of luck (address space has a hard upper limit of 4 GB, minus kernel space).
You are allocating an array of 16-17GB which overflows the stack.
As haccks said you can try allocating on heap.
unsigned int *A = malloc(sizeof(int)*sz);
if(A == NULL) {
printf("Unable to allocate memory for array.\n");
exit(1);
}
Don't forget to free afterwards:
...
free(A);
return 0;
}
And you also have a bug in your code. Array is indexed from 0 to size - 1.
This will when i becomes sz write to invalid memory.
for(i=1;i<=sz;i++) { // Will cause invalid memory write
A[i]=A[j]+1 ;
j++;
printf("%u\n",A[i]);
}
Change to:
for(i=1; i < sz; i++) {
A[i] = A[j] + 1;
j++;
printf("%u\n", A[i]);
}
Memory for arrays is allocated on stack and its size is generally small and will result in stack overflow. You need to allocate memory on heap for such a large array. Either place
unsigned int A[429496729];
out side the main or use dynamic memory allocation
unsigned int *A = malloc(sizeof(int)*sz);
if(A == NULL)
exit(0);
Use free(A) to free the allocated memory once you are done with A.
Better use define constants from limits.h, such as UINT_MAX or ULONG_MAX, and check what type is used for indexing of arrays (perhaps your unsigned int transformed to int)
Related
I have a char array, we know that that a char size is 1 byte. Now I have to collect some char -> getchar() of course and simultaneously increase the array by 1 byte (without malloc, only library: stdio.h)
My suggestion would be, pointing to the array and somehow increase that array by 1 till there are no more chars to get OR you run out of Memory...
Is it possible to increase char array while using it, WITHOUT malloc?
No.
You cannot increase the size of a fixed size array.
For that you need realloc() from <stdlib.h>, which it seems you are not "allowed" to use.
Is it possible to increase char array while using it, WITHOUT malloc?
Quick answer: No it is not possible to increase the size of an array without reallocating it.
Fun answer: Don't use malloc(), use realloc().
Long answer:
If the char array has static or automatic storage class, it is most likely impossible to increase its size at runtime because keeping it at the same address that would require objects that are present at higher addresses to be moved or reallocated elsewhere.
If the array was obtained by malloc, it might be possible to extend its size if no other objects have been allocated after it in memory. Indeed realloc() to a larger size might return the same address. The problem is it is impossible to predict and if realloc returns a different address, the current space has been freed so pointers to it are now invalid.
The efficient way to proceed with this reallocation is to increase the size geometrically, by a factor at a time, 2x, 1.5x, 1.625x ... to minimize the number of reallocations and keep linear time as the size of the array grows linearly. You would a different variable for the allocated size of the array and the number of characters that you have stored into it.
Here is an example:
#include <stdio.h>
#include <stdlib.h>
int main(void) {
char *a = NULL;
size_t size = 0;
size_t count = 0;
int c;
while ((c = getchar()) != EOF && c != '\n') {
if (count >= size) {
/* reallocate the buffer to 1.5x size */
size_t newsize = size + size / 2 + 16;
char *new_a = realloc(a, new_size);
if (new_a == NULL) {
fprintf("out of memory for %zu bytes\n", new_size);
free(a);
return 1;
}
a = new_a;
size = new_size;
}
a[count++] = c;
}
for (i = 0; i < count; i++) {
putchar(a[i]);
}
free(a);
return 0;
}
There are two ways to create space for the string without using dynamic memory allocation(malloc...). You can use a static array or an array with automatic storage duration, you need to specify a maximum amount, you might never reach. But always check against it.
#define BUFFER_SIZE 0x10000
Static
static char buffer[BUFFER_SIZE];
Or automatic (You need to ensure BUFFER_SIZE is smaller than the stack size)
int main() {
char buffer[BUFFER_SIZE];
...
};
There are also optimizations done by the operating system. It might lazily allocate the whole (static/automatic) buffer, so that only the used part is in the physical memory. (This also applies to the dynamic memory allocation functions.) I found out that calloc (for big chunks) just allocates the virtual memory for the program; memory pages are cleared only, when they are accessed (probably through some interrupts raised by the cpu). I compared it to an allocation with malloc and memset. The memset does unnessecary work, if not all bytes/pages of the buffer are accessed by the program.
If you cannot allocate a buffer with malloc..., create a static/automatic array with enough size and let the operating system allocate it for you. It does not occupy the same space in the binary, because it is just stored as a size.
How can I let the user choose a number say n and then create an array with the size of n?
Can I just say int a[]=malloc (n*sizeof(int))?
There are two ways to do that. If the array size is small then you can use variable length array
/* Valid in C99 and later */
int n;
scanf("%d", &n);
int a[n];
This will allocate memory on stack. Other way is you can use dynamic memory allocation which will allocate memory on the heap
int *a = malloc(n*sizeof(int));
Your idea is nearly correct:
int a[] = malloc(n*sizeof(int));
Using malloc is the correct way.
But you cannot assign the returned address to an array.
You must use a pointer variable instead:
int *a = malloc(n*sizeof(int));
Yes if u want to set the size of the array at run-time.
Then u should go for dynamic memory allocation(malloc/calloc).
int a[]=malloc (n*sizeof(int));//this not possible.
int *a =malloc (n*sizeof(int)); // this is possible.
There are two basic ways for allocating the memory to create an array where the size to the array is determined as input:
The first one is,
allocating the memory for array in the 'stack' segment of memory where the size of array is taken as input ant then the array of that particular size is defined and granted memory accordingly.
int n;
scanf("%d",&n); //scanning the size
int arr[n]; //declaring the array of that particular size here
The second one is,
allocating the required memory in the 'heap' segment of memory.It is the memory allocated during runtime (execution of the program)
So,another way of declaring an array where size is defined by user is
int n,*arr;
scanf("%d",&n);
arr=malloc(n*sizeof(int)); //malloc function provides a contiguous space
or
arr=calloc(n,sizeof(int)); //calloc function is similar,initializes as 0
to use both these functions make sure to include stdlib.h.
Variable length arrays (VLAs) were added to C with C99, but made optional with C11. They are still widely supported, though. This is the simplest way to define an array with user-selected size at runtime.
Other than that VLAs may not be available on all platforms, they also may fail silently when there is an allocation failure. This is a disadvantage that malloc() avoids when used correctly.
You can't assign to an array in C, and instead you need to store the value returned by malloc() in a pointer. Note that malloc() returns NULL when there is an allocation failure, allowing code to check for failure and proceed accordingly. The actual allocation might look like this:
int *a_dyn = malloc(sizeof *a_dyn * arr_sz);
This is an idiomatic way of calling malloc(). Note that there is no need to cast the result of malloc(), and note that the operand to sizeof is not an explicit type, but rather an expression involving a_dyn. The sizeof operator uses the type of the expression *a_dyn, which is in fact int (there is no dereference made). This is less error-prone and easier to maintain when types change during the life of a program than coding with explicit types. Also note that the sizeof expression comes before arr_sz. This is a good practice to follow: sometimes you might have a call like:
int *arr = malloc(sizeof *arr * nrows * ncols);
Placing sizeof first forces the multiplication to be done using size_t values, helping to avoid overflow issues in the multiplication.
Don't forget to free any memory allocated with malloc() when it is no longer needed, avoiding memory leaks.
Whether you use a VLA or malloc(), you must validate user input before using it to avoid undefined behavior. Attempting to allocate an array of non-positive size leads to undefined behavior, and attempting to allocate too much memory will lead to an allocation failure.
Here is an example program that illustrates all of this:
#include <stdio.h>
#include <stdlib.h>
#define ARR_MAX 1024 // some sensible maximum array size
int main(void)
{
int arr_sz;
int ret_val;
/* validate user input */
do {
printf("Enter array size: ");
ret_val = scanf("%d", &arr_sz);
} while (ret_val != 1 || arr_sz < 1 || arr_sz > ARR_MAX);
/* use a VLA */
int a_vla[arr_sz];
for (int i = 0; i < arr_sz; i++) {
a_vla[i] = i;
printf("%d ", a_vla[i]);
}
putchar('\n');
/* use malloc() */
int *a_dyn = malloc(sizeof *a_dyn * arr_sz);
if (a_dyn == NULL) { // malloc failure?
fprintf(stderr, "Unable to allocate memory\n");
} else { // malloc success
for (int i = 0; i < arr_sz; i++) {
a_dyn[i] = i;
printf("%d ", a_dyn[i]);
}
putchar('\n');
}
/* avoid memory leaks */
free(a_dyn);
return 0;
}
I run in a problem with a program and I'm not sure how to solve it. I'm processing a file and to do so I get the size with ftell and store it in M_size. After that I declare a unsigned char pointer array with N. The array is then used in two functions a() and b().
...
unsigned long N = (M_size/ x);
int LstElemSize = M_size % x;
if(LstElemSize != 0){
N += 1;
}
unsigned char *ptr_M[N]
a(ptr_M)
b(ptr_M)
...
Function a() actually initializes each element of ptr_M in a for loop:
a(){
int i;
for(i = 0; i < N-1; i ++){
ptr_M[i] = malloc(sizeof(unsigned char) * x);
}
}
Function b() iterates then over each element and calculates stuff and at the end each element is freed.
My problem is now that when I try to process a file e.g. 1 GB the array size will be around 4 000 000 and a Segmentation error occurs (In the line i declare my array). If I calculated it correctly that is 8 byte (char pointer) times 4 000 000 = 32MB. The server running the program has enough memory to hold the file, but i guess as mentioned in Response 1 the stack space is not enough.
What can I do to solve my problem? Increase my stack space? Thanks!
The problem is that you're defining ptr_M in the stack, which has a small size limit. The heap does not have such a small size limit and is able to use more of your system's memory. You need to use malloc() to allocate ptr_M just like you allocate the subarrays. (Make sure to free it at some point too along with all those subarrays!)
unsigned char **ptr_M = malloc(sizeof(unsigned char*) * N);
Also, your a() has an off-by-one error. It ignores the last item of the array. Use this:
for(i = 0; i < N; i ++){
unsigned char *ptr_M[N] is a variable-length array declaring N number of unsigned char on the stack in your case. You should dynamically allocate the space for the array as well.
unsigned char **ptr_M = malloc(sizeof(unsigned char*) * N);
a(ptr_M);
b(ptr_M);
...
//After you free each element in ptr_M
free(ptr_M);
malloc allocates space from heap, not from stack. You may try increasing your heapsize looking at the compiler option. Check the upper limit of heapsize that is supported there.
I recently submitted a small program for an assignment that had the following two functions and a main method inside of it:
/**
* Counts the number of bits it
* takes to represent an integer a
*/
int num_bits(int a)
{
int bitCount = 0;
while(a > 0)
{
bitCount++;
a = a >> 1; //shift to the right 1 bit
}
return bitCount;
}
/**
* Converts an integer into binary representation
* stored in an array
*/
void int2bin_array(int a, int *b)
{
//stopping point in search for bits
int upper_bound = num_bits(a);
int i;
for(i = 0; i < upper_bound; i++)
{
*(b+i) = (a >> i) & 1; //store the ith bit in b[i]
}
}
int main()
{
int numBits = num_bits(exponent);
int arr[numBits]; //<- QUESTION IS ABOUT THIS LINE
int2bin_array(exponent, arr);
//do some operations on array arr
}
When my instructor returned the program he wrote a comment about the line I marked above saying that since the value of numBits isn't known until run-time, initializing an array to size numBits is a dangerous operation because the compiler won't know how much memory to allocate to array arr.
I was wondering if someone could:
1) Verify that this is a dangerous operation
2) Explain what is going on memory wise when I initialize an array like that, how does the compiler know what memory to allocate? Is there any way to determine how much memory was allocated?
Any inputs would be appreciated.
That's a C99 variable length array. It is allocated at runtime (not by the compiler) on the stack, and is basically equivalent to
char *arr = alloca(num_bits);
In this case, since you can know the upper bound of the function, and it is relatively small, you'd be best off with
char arr[sizeof(int)*CHAR_BIT];
This array has a size known at compile time, will always fit everything you need, and works on platforms without C99 support.
It should be ok, it will just go on the stack.
The only danger is blowing out the stack.
malloc would be the normal way, then you know if you have enough memory or not and can make informed decisions on what to do next. But in many cases its ok to assume you can put not too big objects on the stack.
But strictly speaking, if you don't have enough space, this will fail badly.
I started to learn C recently. I use Code::Blocks with MinGW and Cygwin GCC.
I made a very simple prime sieve for Project Euler problem 10, which prints primes below a certain limit to stdout. It works fine until roughly 500000 as limit, but above that my minGW-compiled .exe crashes and the GCC-compiled one throws a "STATUS_STACK_OVERFLOW" exception.
I'm puzzled as to why, since the code is totally non-recursive, consisting of simple for loops.
#include <stdio.h>
#include <math.h>
#define LIMIT 550000
int main()
{
int sieve[LIMIT+1] = {0};
int i, n;
for (i = 2; i <= (int)floor(sqrt(LIMIT)); i++){
if (!sieve[i]){
printf("%d\n", i);
for (n = 2; n <= LIMIT/i; n++){
sieve[n*i] = 1;
}
}
}
for (i; i <= LIMIT; i++){
if (!sieve[i]){
printf("%d\n", i);
}
}
return 0;
}
Seems like you cannot allocate 550000 ints on the stack, allocate them dynamically instead.
int * sieve;
sieve = malloc(sizeof(int) * (LIMIT+1));
Your basic options are to store variables in data segment when your memory chunk is bigger than stack:
allocating memory for array in heap with malloc (as #Binyamin explained)
storing array in Data/BSS segments by declaring array as static int sieve[SIZE_MACRO]
All the memory in that program is allocated on the stack. When you increase the size of the array you increase the amount of space required on the stack. Eventually the method cannot be called as there isn't enough space on the stack to accomodate it.
Either experiement with mallocing the array (so it's allocated on the heap). Or learn how to tell the compiler to allocate a larger stack.