I'm preparing for a Computer Architecture exam, and I can't seem to answer this question:
The following code is useful in checking a memory-related ISA feature. What can you determine using this function?
#define X 0
#define Y 1
int mystery_test(){
int i = 1;
char *p = (char *) &i;
if(p[0] == 1) return X;
else return Y;
}
I was thinking that it would check that pointers and arrays are basically the same, but that isn't a memory-related feature so I'm pretty sure my answer is wrong.
Could someone help, please? And also, what are the memory-related ISA features?
Thank you!
The answer from Retired Ninja is way more than you ever want to know, but the shorter version is that the mystery code is testing the endian-ness of the CPU.
We all know that memory in modern CPUs is byte oriented, but when it's storing a larger item - say, a 4-byte integer - what order does it lay down the components?
Imagine the integer value 0x11223344, which is four separate bytes (0x11 .. 0x44). They can't all fit at a single byte memory address, so the CPU has to put them in some kind of order.
Little-endian means the low part - the 0x44 - is in the lowest memory address, while big-endian puts the most significant byte first; here we're pretending that it's stored at memory location 0x9000 (chosen at random):
Little Big -endian
0x9000: x44 x11
0x9001: x33 x22
0x9002: x22 x33
0x9003: x11 x44
It has to pick something, right?
The code you're considering is storing an integer 1 value into a 4 (or 8) byte chunk of memory, so it's going to be in one of these two organizations:
Little Big
0x9000: x01 x00
0x9001: x00 x00
0x9002: x00 x00
0x9003: x00 x01
But by turning an integer pointer into a char pointer, it's looking at only the low byte, and the 0/1 value tells you if this is big-endian or little-endian.
Return is X/0 for little-endian or Y/1 for big-endian.
Good luck on your exam.
Related
I'm sorting the "children" of prefixes for IP address space. For example, 8.8.8.0/24 is the child of 8.8.8.0/23 in IP address space. I'm confused as to why the following two operations provide different results on my x86 little endian system
A little background information:
A /24 means that the first 24 bits of a 32 bit IPv4 address are "defined". That means that 8.8.8.0/24 encompasses 8.8.8.0 - 8.8.8.255. Similarly, for every bit that's not defined, the amount of address space doubles. 8.8.8.0/23 would have only the first 23 bits defined, so the actual address space goes from 8.8.8.0 - 8.8.9.255, or twice the size of a /24.
Now the confusion I'm having is with the following bitshifts
inet_addr("8.8.8.0") << (32 - 23) produces 269488128
inet_addr("8.8.9.0") << (32 - 23) produces 303042560
inet_addr produces a big endian number. However, when converting it to little endian -
htonl(inet_addr("8.8.8.0")) >> 9 produces 263172
htonl(inet_addr("8.8.9.0")) >> 9 produces 263172
Which is the expected result. Dropping the last 9 bits would mean that 8.8.9.0 would be equal to 8.8.8.0 in theory.
What am I missing here? Shouldn't it work the same for big endian?
Edit: Not a duplicate because I do understand the difference in how endianness affects the way numbers are stored, but I'm clearly missing something with these bitwise operators. The question is more to do with bitwise than endianness - the endianness is just there to foster an example
x86 is little endian. The number 1 in binary in little endian is
|10000000|00000000|00000000|00000000
If you bit shift this left by 9 bits it becomes...
|00000000|01000000|00000000|00000000
In a little endian machine 0xDEADBEEF printed out as a series of bytes from low to high address would actually print EFBEADDE, see
https://www.codeproject.com/Articles/4804/Basic-concepts-on-Endianness
and
https://www.gnu-pascal.de/gpc/Endianness.html.
Most people when thinking in binary think the number 1 is represented as follows (me included) and some people think this is big endian but it's not...
|00000000|00000000|00000000|00000001
In the code below I've printed out 0xDEADBEEF in little endian because my machine is an x86 and I've used the htonl function to convert it to network byte order. Note network byte order is defined as Big Endian.
So when I print out the big endian value for 1 ie htonl(1). The big endian representation of 1 is
|00000000|00000000|00000000|10000000
Try this code
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
void print_deadbeef(void *p, size_t bytes) {
size_t i = 0;
for (i = 0; i < bytes; ++i) {
printf("%02X", ((unsigned char*)p)[i]);
}
printf("\n");
}
void print_bin(uint64_t num, size_t bytes) {
int i = 0;
for(i = bytes * 8; i > 0; i--) {
(i % 8 == 0) ? printf("|") : 1;
(num & 1) ? printf("1") : printf("0");
num >>= 1;
}
printf("\n");
}
int main(void) {
in_addr_t left = inet_addr("8.8.8.0");
in_addr_t right = inet_addr("8.8.9.0");
in_addr_t left_h = htonl(left);
in_addr_t right_h = htonl(right);
in_addr_t left_s = left << 9;
in_addr_t right_s = right >> 9;
assert(left != right);
printf("left != right\n");
print_bin(left, 4);
print_bin(right, 4);
printf("Big Endian if on x86\n");
print_bin(left_s, 4);
print_bin(right_s, 4);
printf("Little Endian if on x86\n");
print_bin(left_h, 4);
print_bin(right_h, 4);
printf("\n\nSome notes\n\n");
printf("0xDEADBEEF printed on a little endian machine\n");
uint32_t deadbeef = 0xDEADBEEF;
print_deadbeef(&deadbeef, 4);
uint32_t deadbeefBig = htonl(deadbeef);
printf("\n0xDEADBEEF printed in network byte order (big endian)\n");
print_deadbeef(&deadbeefBig, 4);
printf("\n1 printed on a little endian machine\n");
print_bin(1, 4);
printf("\nhtonl(1) ie network byte order (big endian) on a little endian machine\n");
print_bin(htonl(1), 4);
return 0;
}
This is the output
left != right
|00010000|00010000|00010000|00000000
|00010000|00010000|10010000|00000000
Big Endian if on x86
|00000000|00001000|00001000|00001000
|00100001|00100000|00000000|00000000
Little Endian if on x86
|00000000|00010000|00010000|00010000
|00000000|10010000|00010000|00010000
Some notes
0xDEADBEEF printed on a little endian machine
EFBEADDE
0xDEADBEEF printed in network byte order (big endian)
DEADBEEF
1 printed on a little endian machine
|10000000|00000000|00000000|00000000
htonl(1) ie network byte order on a little endian machine
|00000000|00000000|00000000|10000000
The question of Big Endian and Little Endian isn't really known to the machine.
The types in C don't contain such information since it's a Hardware issue, not a type related one.
The machine assumes that all multi-byte numbers are ordered according to it's local endian (on x86, this is usually little endian).
For this reason, bit shifting is always performed using the local endian assumption.
You can't correctly apply bit-shifting to a Big Endian number on a Little Endian machine.
You can't even print a Big Endian number to the screen on a Little Endian machine without getting a funny result.
This is why #Harry's answer was so cool, it prints out each bit, circumventing the issue.
Wikipedia has an article about Endianness with more details.
It should be noted that Endianness actually refers to the way a machine stores it's bytes in the memory.
For example, if the number were as String, Endianness would refer to the question: which "letter" (byte) would come first?
Some machine would store "Hello" and some would store "olleH" (for numbers only, in actual strings, the bytes are always ordered correctly).
Notice that although the order of bytes is reversed, each byte has all the bits ordered the same way, so each byte retains it's value.
When a bit-shift occurs, it always occurs according to the machine's byte ordering system, since this is how it's CPU and memory store are designed.
The accepted answer provides a good sample program. However, I think this sample is a little bit misleading.
The bit string of 1 in little-endian is printed as:
10000000|00000000|00000000|00000000
I ran this code on my x86 pc and I think the results are reliable. But that doesn't mean that the value 1 is stored as printed above in a little-endian machine.
According to the code of print_bin, the num right shifts one bit each time and the least significant bit is printed.
Also, the right shift operator always shifts from the most significant bit (MSB) to the least significant bit (LSB).
In the end, no matter the bit order, the result of print_bin(1, 4) is always the reverse of the human-writing bit representation of 1, which is:
00000000|00000000|00000000|00000001
For example, the bit string may be:
byte significance increase -->
byte
/-------\
00000001|00000000|00000000|00000000
|
bit
<-- bit significance increase
In this example, the bit-order is different from the byte-order. But the results of print_bin(1,4) would be the same.
In other words, the printed bit string doesn't necessarily mean reverse bit-order in the little-endian machine.
I talked about this further in this blog.
I am curious about little-endian
and I know that computers almost have little-endian method.
So, I praticed through a program and the source is below.
int main(){
int flag = 31337;
char c[10] = "abcde";
int flag2 = 31337;
return 0;
}
when I saw the stack via gdb,
I noticed that there were 0x00007a69 0x00007a69 .... ... ... .. .... ...
0x62610000 0x00656463 .. ...
So, I have two questions.
For one thing,
how can the value of char c[10] be under the flag?
I expected there were the value of flag2 in the top of stack and the value of char c[10] under the flag2 and the value of flag under the char c[10].
like this
7a69
"abcde"
7a69
Second,
I expected the value were stored in the way of little-endian.
As a result, the value of "abcde" was stored '6564636261'
However, the value of 31337 wasn't stored via little-endian.
It was just '7a69'.
I thought it should be '697a'
why doesn't integer type conform little-endian?
There is some confusion in your understanding of endianness, stack and compilers.
First, the locations of variables in the stack may not have anything to do with the code written. The compiler is free to move them around how it wants, unless it is a part of a struct, for example. Usually they try to make as efficient use of memory as possible, so this is needed. For example having char, int, char, int would require 16 bytes (on a 32bit machine), whereas int, int, char, char would require only 12 bytes.
Second, there is no "endianness" in char arrays. They are just that: arrays of values. If you put "abcde" there, the values have to be in that order. If you would use for example UTF16 then endianness would come into play, since then one part of the codeword (not necessarily one character) would require two bytes (on a normal 8-bit machine). These would be stored depending on endianness.
Decimal value 31337 is 0x007a69 in 32bit hexadecimal. If you ask a debugger to show it, it will show it as such whatever the endianness. The only way to see how it is in memory is to dump it as bytes. Then it would be 0x69 0x7a 0x00 0x00 in little endian.
Also, even though little endian is very popular, it's mainly because x86 hardware is popular. Many processors have used big endian (SPARC, PowerPC, MIPS amongst others) order and some (like older ARM processors) could run in either one, depending on the requirements.
There is also a term "network byte order", which actually is big endian. This relates to times before little endian machines became most popular.
Integer byte order is an arbitrary processor design decision. Why for example do you appear to be uncomfortable with little-endian? What makes big-endian a better choice?
Well probably because you are a human used to reading numbers from left-to-right; but the machine hardly cares.
There is in fact a reasonable argument that it is intuitive for the least-significant-byte to be placed in the lowest order address; but again, only from a human intuition point-of-view.
GDB shows you 0x62610000 0x00656463 because it is interpreting data (...abcde...) as 32bit words on a little endian system.
It could be either way, but the reasonable default is to use native endianness.
Data in memory is just a sequence of bytes. If you tell it to show it as a sequence (array) of short ints, it changes what it displays. Many debuggers have advanced memory view features to show memory content in various interpretations, including string, int (hex), int (decimal), float, and many more.
You got a few excellent answers already.
Here is a little code to help you understand how variables are laid out in memory, either using little-endian or big-endian:
#include <stdio.h>
void show_var(char* varname, unsigned char *ptr, size_t size) {
int i;
printf ("%s:\n", varname);
for (i=0; i<size; i++) {
printf("pos %d = %2.2x\n", i, *ptr++);
}
printf("--------\n");
}
int main() {
int flag = 31337;
char c[10] = "abcde";
show_var("flag", (unsigned char*)&flag, sizeof(flag));
show_var("c", (unsigned char*)c, sizeof(c));
}
On my Intel i5 Linux machine it produces:
flag:
pos 0 = 69
pos 1 = 7a
pos 2 = 00
pos 3 = 00
--------
c:
pos 0 = 61
pos 1 = 62
pos 2 = 63
pos 3 = 64
pos 4 = 65
pos 5 = 00
pos 6 = 00
pos 7 = 00
pos 8 = 00
pos 9 = 00
--------
I have some confusion regarding reading a word from a byte array. The background context is that I'm working on a MIPS simulator written in C for an intro computer architecture class, but while debugging my code I ran into a surprising result that I simply don't understand from a C programming standpoint.
I have a byte array called mem defined as follows:
uint8_t *mem;
//...
mem = calloc(MEM_SIZE, sizeof(uint8_t)); // MEM_SIZE is pre defined as 1024x1024
During some of my testing I manually stored a uint32_t value into four of the blocks of memory at an address called mipsaddr, one byte at a time, as follows:
for(int i = 3; i >=0; i--) {
*(mem+mipsaddr+i) = value;
value = value >> 8;
// in my test, value = 0x1084
}
Finally, I tested trying to read a word from the array in one of two ways. In the first way, I basically tried to read the entire word into a variable at once:
uint32_t foo = *(uint32_t*)(mem+mipsaddr);
printf("foo = 0x%08x\n", foo);
In the second way, I read each byte from each cell manually, and then added them together with bit shifts:
uint8_t test0 = mem[mipsaddr];
uint8_t test1 = mem[mipsaddr+1];
uint8_t test2 = mem[mipsaddr+2];
uint8_t test3 = mem[mipsaddr+3];
uint32_t test4 = (mem[mipsaddr]<<24) + (mem[mipsaddr+1]<<16) +
(mem[mipsaddr+2]<<8) + mem[mipsaddr+3];
printf("test4= 0x%08x\n", test4);
The output of the code above came out as this:
foo= 0x84100000
test4= 0x00001084
The value of test4 is exactly as I expect it to be, but foo seems to have reversed the order of the bytes. Why would this be the case? In the case of foo, I expected the uint32_t* pointer to point to mem[mipsaddr], and since it's 32-bits long, it would just read in all 32 bits in the order they exist in the array (which would be 00001084). Clearly, my understanding isn't correct.
I'm new here, and I did search for the answer to this question but couldn't find it. If it's already been posted, I apologize! But if not, I hope someone can enlighten me here.
It is (among others) explained here: http://en.wikipedia.org/wiki/Endianness
When storing data larger than one byte into memory, it depends on the architecture (means, the CPU) in which order the bytes are stored. Either, the most significant byte is stored first and the least significant byte last, or vice versa. When you read back the individual bytes through byte access operations, and then merge them to form the original value again, you need to consider the endianess of your particular system.
In your for-loop, you are storing your value byte-wise, starting with the most significant byte (counting down the index is a bit misleading ;-). Your memory looks like this afterwards: 0x00 0x00 0x10 0x84.
You are then reading the word back with a single 32 bit (four byte) access. Depending on our architecture, this will either become 0x00001084 (big endian) or 0x84100000 (little endian). Since you get the latter, you are working on a little endian system.
In your second approach, you are using the same order in which you stored the individual bytes (most significant first), so you get back the same value which you stored earlier.
It seems to be a problem of endianness, maybe comes from casting (uint8_t *) to (uint32_t *)
If I understand correctly, the DCPU-16 specification for 0x10c describes a 16-bit address space where each offset addresses a 16-bit word, instead of a byte as in most other memory architectures. This has some curious consequences, e.g. I imagine that sizeof(char) and sizeof(short) would both return 1.
Is it feasible to keep C code portable between such different memory addressing schemes? What would be the gotchas to keep in mind?
edit: perhaps I should have given a more specific example. Let's say you have some networking code that deals with byte streams. Do you throw away half of your memory by putting only one byte at each address so that the code can stay the same, or do you generalize everything with bitshifts to deal with N bytes per offset?
edit2: The answers seem to focus on the issue of data type sizes, which wasn't the point - I shouldn't even have mentioned it. The question is about how to cope with losing the ability to address any byte in memory with a pointer. Is it reasonable to expect code to be agnostic about this?
It's totally feasible. Roughly speaking, C's basic integer data types have sizes that uphold:
sizeof (char) <= sizeof (short) <= sizeof (int) <= sizeof (long)
The above is not exactly what the spec says, but it's close.
As pointed out by awoodland in a comment, you'd also expect a C compiler for the DCPU-16 to have CHAR_BIT == 16.
Bonus for not assuming that the DCPU-16 would have sizeof (char) == 2, that's a common fallacy.
When you say, 'losing the ability to address a byte', I assume you mean 'bit-octet', rather than 'char'. Portable code should only assume CHAR_BIT >= 8. In practice, architectures that don't have byte addressing often define CHAR_BIT == 8, and let the compiler generate instructions for accessing the byte.
I actually disagree with the answers suggesting: CHAR_BIT == 16 as a good choice. I'd prefer: CHAR_BIT == 8, with sizeof(short) == 2. The compiler can handle the shifting / masking, just as it does for many RISC architectures, for byte access in this case.
I imagine Notch will revise and clarify the DCPU-16 spec further; there are already requests for an interrupt mechanism, and further instructions. It's an aesthetic backdrop for a game, so I doubt there will be an official ABI spec any time soon. That said, someone will be working on it!
Edit:
Consider an array of char in C. The compiler packs 2 bytes in each native 16-bit word of DCPU memory. So if we access, say, the 10th element (index 9), fetch the word # [9 / 2] = 4, and extract the byte # [9 % 2] = 1.
Let 'X' be the start address of the array, and 'I' be the index:
SET J, I
SHR J, 1 ; J = I / 2
ADD J, X ; J holds word address
SET A, [J] ; A holds word
AND I, 0x1 ; I = I % 2 {0 or 1}
MUL I, 8 ; I = {0 or 8} ; could use: SHL I, 3
SHR A, I ; right shift by I bits for hi or lo byte.
The register A holds the 'byte' - it's a 16 bit register, so the top half can be ignored.
Alternatively, the top half can be zeroed:
AND A, 0xff ; mask lo byte.
This is not optimized, but it conveys the idea.
The equality goes rather like this:
1 == sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long)
The short type can be 1, and as a matter of fact maybe you'll even want the int type to be 1 too actually (I didn't read the spec, but I'm supposing the normal data type is 16 bit). This stuff is defined by the compiler.
For practicity, the compiler may want to set long to something larger than int even if it requires the compiler doing some extra work (like implementing addition/multiplication etc in software).
This isn't a memory addressing issue, but rather a granularity question.
yes it is entirely possible to port C code
in terms of data transfer it would be advisable to either pack the bits (or use a compression) or send in 16 bit bytes
because the CPU will almost entirely communicate only with (game) internal devices that will likely also be all 16 bit this should be no real problem
BTW I agree that CHAR_BIT should be 16 as (IIRC) each char must be addressable so making CHAR_BIT ==8 will REQUIRE sizeof(char*) ==2 which will make everything else overcomplicated
I read about Endianness and understood squat...
so I wrote this
main()
{
int k = 0xA5B9BF9F;
BYTE *b = (BYTE*)&k; //value at *b is 9f
b++; //value at *b is BF
b++; //value at *b is B9
b++; //value at *b is A5
}
k was equal to A5 B9 BF 9F
and (byte)pointer "walk" o/p was 9F BF b9 A5
so I get it bytes are stored backwards...ok.
~
so now I thought how is it stored at BIT level...
I means is "9f"(1001 1111) stored as "f9"(1111 1001)?
so I wrote this
int _tmain(int argc, _TCHAR* argv[])
{
int k = 0xA5B9BF9F;
void *ptr = &k;
bool temp= TRUE;
cout<<"ready or not here I come \n"<<endl;
for(int i=0;i<32;i++)
{
temp = *( (bool*)ptr + i );
if( temp )
cout<<"1 ";
if( !temp)
cout<<"0 ";
if(i==7||i==15||i==23)
cout<<" - ";
}
}
I get some random output
even for nos. like "32" I dont get anything sensible.
why ?
Just for completeness, machines are described in terms of both byte order and bit order.
The intel x86 is called Consistent Little Endian because it stores multi-byte values in LSB to MSB order as memory address increases. Its bit numbering convention is b0 = 2^0 and b31 = 2^31.
The Motorola 68000 is called Inconsistent Big Endian because it stores multi-byte values in MSB to LSB order as memory address increases. Its bit numbering convention is b0 = 2^0 and b31 = 2^31 (same as intel, which is why it is called 'Inconsistent' Big Endian).
The 32-bit IBM/Motorola PowerPC is called Consistent Big Endian because it stores multi-byte values in MSB to LSB order as memory address increases. Its bit numbering convention is b0 = 2^31 and b31 = 2^0.
Under normal high level language use the bit order is generally transparent to the developer. When writing in assembly language or working with the hardware, the bit numbering does come into play.
Endianness, as you discovered by your experiment refers to the order that bytes are stored in an object.
Bits do not get stored differently, they're always 8 bits, and always "human readable" (high->low).
Now that we've discussed that you don't need your code... About your code:
for(int i=0;i<32;i++)
{
temp = *( (bool*)ptr + i );
...
}
This isn't doing what you think it's doing. You're iterating over 0-32, the number of bits in a word - good. But your temp assignment is all wrong :)
It's important to note that a bool* is the same size as an int* is the same size as a BigStruct*. All pointers on the same machine are the same size - 32bits on a 32bit machine, 64bits on a 64bit machine.
ptr + i is adding i bytes to the ptr address. When i>3, you're reading a whole new word... this could possibly cause a segfault.
What you want to use is bit-masks. Something like this should work:
for (int i = 0; i < 32; i++) {
unsigned int mask = 1 << i;
bool bit_is_one = static_cast<unsigned int>(ptr) & mask;
...
}
Your machine almost certainly can't address individual bits of memory, so the layout of bits inside a byte is meaningless. Endianness refers only to the ordering of bytes inside multibyte objects.
To make your second program make sense (though there isn't really any reason to, since it won't give you any meaningful results) you need to learn about the bitwise operators - particularly & for this application.
Byte Endianness
On different machines this code may give different results:
union endian_example {
unsigned long u;
unsigned char a[sizeof(unsigned long)];
} x;
x.u = 0x0a0b0c0d;
int i;
for (i = 0; i< sizeof(unsigned long); i++) {
printf("%u\n", (unsigned)x.a[i]);
}
This is because different machines are free to store values in any byte order they wish. This is fairly arbitrary. There is no backwards or forwards in the grand scheme of things.
Bit Endianness
Usually you don't have to ever worry about bit endianness. The most common way to access individual bits is with shifts ( >>, << ) but those are really tied to values, not bytes or bits. They preform an arithmatic operation on a value. That value is stored in bits (which are in bytes).
Where you may run into a problem in C with bit endianness is if you ever use a bit field. This is a rarely used (for this reason and a few others) "feature" of C that allows you to tell the compiler how many bits a member of a struct will use.
struct thing {
unsigned y:1; // y will be one bit and can have the values 0 and 1
signed z:1; // z can only have the values 0 and -1
unsigned a:2; // a can be 0, 1, 2, or 3
unsigned b:4; // b is just here to take up the rest of the a byte
};
In this the bit endianness is compiler dependant. Should y be the most or least significant bit in a thing? Who knows? If you care about the bit ordering (describing things like the layout of a IPv4 packet header, control registers of device, or just a storage formate in a file) then you probably don't want to worry about some different compiler doing this the wrong way. Also, compilers aren't always as smart about how they work with bit fields as one would hope.
This line here:
temp = *( (bool*)ptr + i );
... when you do pointer arithmetic like this, the compiler moves the pointer on by the number you added times the sizeof the thing you are pointing to. Because you are casting your void* to a bool*, the compiler will be moving the pointer along by the size of one "bool", which is probably just an int under the covers, so you'll be printing out memory from further along than you thought.
You can't address the individual bits in a byte, so it's almost meaningless to ask which way round they are stored. (Your machine can store them whichever way it wants and you won't be able to tell). The only time you might care about it is when you come to actually spit bits out over a physical interface like I2C or RS232 or similar, where you have to actually spit the bits out one-by-one. Even then, though, the protocol would define which order to spit the bits out in, and the device driver code would have to translate between "an int with value 0xAABBCCDD" and "a bit sequence 11100011... [whatever] in protocol order".