Is there any limit as to how many elements a 2D integer array can contain in C?
PS : I was expecting there would be some space limitations in declaring an array but could not find any such reference in the internet.
It depends on your RAM or the memory available for you.
i:e: My program used to crash when I declared a global array a[100000][10000], but this declaration is fine with the system now I have.
The size_t type is defined to be large enough to contain the size of any object in the program, including arrays. So the largest possible array size can be described as 2^(8*sizeof(size_t) bytes.
For convenience, this value can be obtained through the SIZE_MAX constant in limits.h. It is guaranteed to be at least 65535 but is realistically a larger value, most likely 2^32 on 32 bit systems and 2^64 on 64 bit systems.
Maximum imposed by the C/C++ standard: x * y * z <= SIZE_MAX, where SIZE_MAX is implementation defined, x is one dimension of the array, y is the other dimension, and z is the size of the element in bytes. e.g. element_t A[x][y], z = sizeof(element_t).
Related
This question already has answers here:
size_t vs. uintptr_t
(8 answers)
Closed 1 year ago.
There was a question here about equality of sizeof(size_t) and sizeof(void*) and the accepted answer was that they are not guaranteed to be equal.
But at least, must it be that:
sizeof(void*) >= sizeof(size_t)
I think so. Because, take the largest stored object possible in a given C implementation, of size S. Now, the storage area can be thought of as array of bytes of size S. Therefore, there must be a pointer to each byte, and all these pointers are comparable and different. Therefore, the number of distinct elements of type void*, must be at least, the largest number of type size_t, which is unsigned integer type. Thus sizeof(void*) >= sizeof(size_t) .
Is my reasoning making sense or not?
Is my reasoning making sense or not?
The problem with your reeasoning is that you assume that the size of the largest object possible equals SIZE_MAX. But that's not true. If you do
void* p = malloc(SIZE_MAX);
you will (most likely) get a NULL pointer back.
You may also get warnings like:
main.cpp:48:15: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
48 | void* p = malloc(SIZE_MAX);
| ^~~~~~~~~~~~~~~~
Since the maximum object size isn't (always) SIZE_MAX you can't use the value of SIZE_MAX to argue about the size of pointers.
BTW: Some CPU implementation that uses 64 bit pointers at the SW level may not have 64 bit at the HW level. Instead some bits are just treated as all-ones/all-zeros.
This question already has answers here:
size of int variable
(6 answers)
Closed 5 years ago.
This is a bit of a general question and not completely related to the c programming language but it's what I'm on studying at the moment.
Why does an integer take up 4 bytes or How ever many bytes dependant on the system?
Why does it not take up 1 byte per integer?
For example why does the following take up 8 bytes:
int a = 1;
int b = 1;
Thanks
I am not sure whether you are asking why int objects have fixed sizes instead of variable sizes or whether you are asking why int objects have the fixed sizes they do. This answers the former.
We do not want the basic types to have variable lengths. That makes it very complicated to work with them.
We want them to have fixed lengths, because then it is much easier to generate instructions to operate on them. Also, the operations will be faster.
If the size of an int were variable, consider what happens when you do:
b = 3;
b += 100000;
scanf("%d", &b);
When b is first assigned, only one byte is needed. Then, when the addition is performed, the compiler needs more space. But b might have neighbors in memory, so the compiler cannot just grow it in place. It has to release the old memory and allocate new memory somewhere.
Then, when we do the scanf, the compiler does not know how much data is coming. scanf will have to do some very complicated work to grow b over and over again as it reads more digits. And, when it is done, how does it let you know where the new b is? The compiler has to have some mechanism to update the location for b. This is hard and complicated and will cause additional problems.
In contrast, if b has a fixed size of four bytes, this is easy. For the assignment, write 3 to b. For the addition, add 100000 to the value in b and write the result to b. For the scanf, pass the address of b to scanf and let it write the new value to b. This is easy.
The basic integral type int is guaranteed to have at least 16 bits; At least means that compilers/architectures may also provide more bits, and on 32/64 bit systems int will most likely comprise 32 bits or 64 bits (i.e. 4 bytes or 8 bytes), respectively (cf, for example, cppreference.com):
Integer types
... int (also accessible as signed int): This is the most optimal
integer type for the platform, and is guaranteed to be at least 16
bits. Most current systems use 32 bits (see Data models below).
If you want an integral type with exactly 8 bits, use the int8_t or uint8_t.
It doesn't. It's implementation-defined. A signed int in gcc on an Atmel 8-bit microcontroller, for example, is a 16-bit integer. An unsigned int is also 16-bits, but from 0-65535 since it's unsigned.
The fact that an int uses a fixed number of bytes (such as 4) is a compiler/CPU efficiency and limitation, designed to make common integer operations fast and efficient.
There are types (such as BigInteger in Java) that take a variable amount of space. These types would have 2 fields, the first being the number of words being used to represent the integer, and the second being the array of words. You could define your own VarInt type, something like:
struct VarInt {
char length;
char bytes[]; // Variable length
}
VarInt one = {1, {1}}; // 2 bytes
VarInt v257 = {2, {1,1}}; // 3 bytes
VarInt v65537 = {4, {1,0,0,1}}; // 5 bytes
and so on, but this would not be very fast to perform arithmetic on. You'd have to decide how you would want to treat overflow; resizing the storage would require dynamic memory allocation.
The C99 standard suggests that the type size_t is large enough to store the size of any object, as it is the resulting type of the sizeof operator.
The sizeof operator yields the size (in bytes) of its operand, which may be an expression or the parenthesized name of a type. ...
The value of the result ... is implementation-defined, and its type (an unsigned integer type) is size_t, defined in (and other headers).
Since SIZE_MAX (<limits.h>, if I recall correctly) is defined to be the largest value that a size_t can store, it should follow that the largest object would have a size equal to SIZE_MAX. That would in fact be helpful, but alas it seems as though we'd be asking quite a lot to allocate anything even one quarter that size.
Are there any implementations where you can actually declare (or otherwise allocate) an object as large as SIZE_MAX?
It certainly doesn't seem to be the common case... In C11 the optional rsize_t type and its corresponding RSIZE_MAX macro were introduced. It's supposed to be a runtime constraint if any standard C function is used with a value greater than RSIZE_MAX as an rsize_t argument. This seems to imply that the largest object might be RSIZE_MAX bytes. However, this doesn't seem to be widely supported, either!
Are there any implementations where RSIZE_MAX exists and you can actually declare (or otherwise allocate) an object as large as RSIZE_MAX?
I think all C implementations will allow you to declare an object of that size. The OS may refuse to load the executable, as there isn't enough memory.
Likewise, all C runtime libraries will allow you to attempt to allocate that size of memory, however, it will probably fail as there isn't that much memory, neither virtual nor real.
Just think: if size_t is a type equal to the machine word size (32 bits, 64 bits), then the highest adressable memory cell (byte) is 2^32 (or 2^64). Given there are lower memory interrupt vectors, BIOS and OS, not to mention the code and data of your program, there is never this amount of memory available.
Oh, now I see the issue. Assuming realloc factor of ~1.5, there are two integer limits. First is SIZE_MAX/3, because above that size*3/2 will overflow. But at that large low bits are insignificant and we can inverse the operator order to size/2*3 to still grow by ~1.5 (that imposes second limit of SIZE_MAX/3*2). At last, resort to SIZE_MAX.
After that, we just have to binary-search the amount that system can actually allocate (in range from the result of growing to minimal required size).
int
grow(char **data_p, size_t *size_p, size_t min)
{
size_t size = *size_p;
while (size < min)
size = (size <= SIZE_MAX / 3 ? size * 3 / 2 :
size <= SIZE_MAX / 3 * 2 ? size / 2 * 3 : SIZE_MAX);
if (size != *size_p) {
size_t ext = size - min;
char *data;
for (;; ext /= 2)
if ((data = realloc(*data_p, size + ext)) || ext == 0)
break;
if (data == NULL) return -1; // ENOMEM
*data_p = data;
*size_p = size + ext;
}
return 0;
}
And none of os-dependent or manual limits!
As you can see, the original question is a consequence of [probably] imperfect implementation that doesn't treat edge cases with respect. It doesn't matter if any questioned systems do or do not exist now – any algorithm that is supposed to work near to the integer limits should take proper care of them.
(Please note that above code assumes char data and has no accounting for other element sizes; implementing that may add more complex checks.)
Sorry if this is not the correct place to do this question, this is not about programation instead is a technical question.
I need to work with enormous size arrays of 2D vectors in double precision, 10 million of them aproximately. But, in other programs I had memory problem in deal with this kind of arrays. My question is if there is some kind of limit for the array size in double precision.
I work in Linux, Intel two core, 32-bits.
thanks
Ok, I will explain by why the number of bytes is limited, not only the element count. During array indexing, the address of the element must be calculated. Of course it must fit to the intptr_t C variable. Also, the size of the array in bytes must fit in a size_t C variable. These are both 32-bit or 64-bit on 32-bit and 64-bit programs on modern machines. The same also holds for the virtual memory addressable by program! And also the memory addressable by the OS and CPU, though they can be 64-bit even if the program is 32-bit.
This is the fundamental reason why 32-bit programs and operating systems cannot address more than 4 GB of memory. Even if you could somehow compute the address using a Fortran variable wider than the chosen CPU word size, the CPU simply cannot access it.
Finally I mad an experiment in Intel Fortran in 32-bit mode with array with 32 byte elements:
complex(16), allocatable :: a(:)
do i=1,100
allocate(a(2**i))
a(size(a)) = 1
deallocate(a)
write(*,*) i
end do
end
ifort arraysize.f90 -m32 -check -traceback -g
The output is as expected:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
forrtl: severe (179): Cannot allocate array - overflow on array size calculation.
As expected the size of the array in bytes overflowed and the program crashed long before the indexing variable overflowed. This is not a compiler specific feature, but there is a fundamental reason for this.
The Fortran language standards don't define a limit to the size of arrays that a program can (attempt to) declare or allocate. In practice you may find that your compiler limits the total number of elements in an array to 2^31-1 or 2^63-1, depending on whether your default integer size is 32- or 64-bits. You may find that the maximum size of any dimension of an array is also limited to the same values.
In practice the maximum size of array that you can declare will be limited by the RAM available on your computer. Since a double precision value occupies 8 bytes it's relatively easy for you to calculate the maximum bounds of arrays that you are likely to be able to work with. Any storage overhead required for an array is tiny compared with the volumes of data you seem to want to work with.
In response to VladimirF's comments
I meant, and still mean, the number of elements, not the number of bytes. It is the number of elements which determines the maximum index value required to access an element of an array.
It may be that some compilers impose a limit to the number of bytes used in a single array, but that is not a point I am making.
Fortran arrays can, of course, be indexed from 0, indeed from any positive or negative integer within range, but that is really just a convenience for the programmer.
Suppose I'm given a function with the following signature:
void SendBytesAsync(unsigned char* data, T length)
and I need a buffer large enough to hold a byte array of the maximum length that can be specified by type T. How do I declare that buffer? I can't just use sizeof as it will return the size (in bytes) of type T and not the maximum value that the type could contain. I don't want to use limits.h as the underlying type could change and my buffer be too small. I can't use pow from math.h because I need a constant expression. So how do I get a constant expression for the maximum size of a type at compile time in C?
Edit
The type will be unsigned. Since everyone seems to be appalled at the idea of a statically allocated buffer determined at compile time, I'll provide a little background. This is for an embedded application (on a microcontroller) where reliability and speed are the priorities. As such, I'm perfectly OK with wasting statically assigned memory for the sake of run time integrity (no malloc issues) and performance (no overhead for memory allocation each time I need the buffer). I understand the risk that if the max size of T is too large my linker will not be able to allocate a buffer that big, but that will be a compile-time failure, which can be accommodated, rather than a run-time failure, which cannot be tolerated. If, for example I use size_t for the size of the payload and allocate the memory dynamically, there is a very real possibility that the system will not have that much memory available. I would much rather know this at compile time, than at run-time where this will result in packet loss, data corruption, etc. Looking at the function signature I provided, it is ridiculous to provide a type as a size parameter for a dynamically allocated buffer and not expect the possibility that a caller will use the max value of the type. So I'm not sure why there seems to be so much consternation about allocating that memory once, for good. I can see this being a huge problem in the Windows world where multiple processes are fighting for the same memory resources, but in the embedded world, there's only 1 task to be done and if you can't do that effectively, then it doesn't matter how much memory you saved.
Use _Generic:
#define MAX_SIZE(X) _Generic((X),
long: LONG_MAX,
unsigned long: ULONG_MAX,
/* ... */)
Prior to C11 there isn't a portable way to find an exact maximum value of an object of type T (all calculations with CHAR_BIT, for example, may yield overestimates due to padding bits).
Edit: Do note that under certain conditions (think segmented memory of real-life situations) you might not be able to allocate a buffer large enough to equal the maximum value of any given type T.
if T is unsigned, then would ((T) -1) work?
(This is probably really bad, and if so, please let me know why :-) )
Is there a reason why you are allocating the maximum possible buffer size instead of a buffer that is only as large as you need? Why not have the caller simply specify the amount of memory needed?
Recall that the malloc() function takes an argument of type size_t. That means that (size_t)(-1) (which is SIZE_MAX in C99 and later) will represent the largest value that can be passed to malloc. If you are using malloc as your allocator, then this will be your absolute upper limit.
Maybe try using a bit shift?
let's see:
unsigned long max_size = (1 << (8 * sizeof(T))) - 1
sizeof(T) gives you the number of bytes T occupies in memory. (not technically true. usually the compiler will align the structure with memory... so if T is one byte, it will actually allocate 4, or something.)
Breaking it down:
8 * sizeof(T) gives you the number of bits that size represents
1 << x is the same as saying 2 to the x power. Because every time you shift to the left, you're multiplying by two. Just as every time you shift to the left in base 10, you are multiplying by 10.
- 1 an 8-bit number can hold 256 values. 0..255.
Interesting question. I would start by looking in the 'limits' header for the max value of a numeric type T. I have not tried it but I would do something that uses T::max