Bitwise operation on big endian and little endian differences - c

Question

I'm sorting the "children" of prefixes for IP address space. For example, 8.8.8.0/24 is the child of 8.8.8.0/23 in IP address space. I'm confused as to why the following two operations provide different results on my x86 little endian system
A little background information:
A /24 means that the first 24 bits of a 32 bit IPv4 address are "defined". That means that 8.8.8.0/24 encompasses 8.8.8.0 - 8.8.8.255. Similarly, for every bit that's not defined, the amount of address space doubles. 8.8.8.0/23 would have only the first 23 bits defined, so the actual address space goes from 8.8.8.0 - 8.8.9.255, or twice the size of a /24.
Now the confusion I'm having is with the following bitshifts
inet_addr("8.8.8.0") << (32 - 23) produces 269488128
inet_addr("8.8.9.0") << (32 - 23) produces 303042560
inet_addr produces a big endian number. However, when converting it to little endian -
htonl(inet_addr("8.8.8.0")) >> 9 produces 263172
htonl(inet_addr("8.8.9.0")) >> 9 produces 263172
Which is the expected result. Dropping the last 9 bits would mean that 8.8.9.0 would be equal to 8.8.8.0 in theory.
What am I missing here? Shouldn't it work the same for big endian?
Edit: Not a duplicate because I do understand the difference in how endianness affects the way numbers are stored, but I'm clearly missing something with these bitwise operators. The question is more to do with bitwise than endianness - the endianness is just there to foster an example

Answer 1

x86 is little endian. The number 1 in binary in little endian is
|10000000|00000000|00000000|00000000
If you bit shift this left by 9 bits it becomes...
|00000000|01000000|00000000|00000000
In a little endian machine 0xDEADBEEF printed out as a series of bytes from low to high address would actually print EFBEADDE, see
https://www.codeproject.com/Articles/4804/Basic-concepts-on-Endianness
and
https://www.gnu-pascal.de/gpc/Endianness.html.
Most people when thinking in binary think the number 1 is represented as follows (me included) and some people think this is big endian but it's not...
|00000000|00000000|00000000|00000001
In the code below I've printed out 0xDEADBEEF in little endian because my machine is an x86 and I've used the htonl function to convert it to network byte order. Note network byte order is defined as Big Endian.
So when I print out the big endian value for 1 ie htonl(1). The big endian representation of 1 is
|00000000|00000000|00000000|10000000
Try this code
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <arpa/inet.h>
void print_deadbeef(void *p, size_t bytes) {
size_t i = 0;
for (i = 0; i < bytes; ++i) {
printf("%02X", ((unsigned char*)p)[i]);
}
printf("\n");
}
void print_bin(uint64_t num, size_t bytes) {
int i = 0;
for(i = bytes * 8; i > 0; i--) {
(i % 8 == 0) ? printf("|") : 1;
(num & 1) ? printf("1") : printf("0");
num >>= 1;
}
printf("\n");
}
int main(void) {
in_addr_t left = inet_addr("8.8.8.0");
in_addr_t right = inet_addr("8.8.9.0");
in_addr_t left_h = htonl(left);
in_addr_t right_h = htonl(right);
in_addr_t left_s = left << 9;
in_addr_t right_s = right >> 9;
assert(left != right);
printf("left != right\n");
print_bin(left, 4);
print_bin(right, 4);
printf("Big Endian if on x86\n");
print_bin(left_s, 4);
print_bin(right_s, 4);
printf("Little Endian if on x86\n");
print_bin(left_h, 4);
print_bin(right_h, 4);
printf("\n\nSome notes\n\n");
printf("0xDEADBEEF printed on a little endian machine\n");
uint32_t deadbeef = 0xDEADBEEF;
print_deadbeef(&deadbeef, 4);
uint32_t deadbeefBig = htonl(deadbeef);
printf("\n0xDEADBEEF printed in network byte order (big endian)\n");
print_deadbeef(&deadbeefBig, 4);
printf("\n1 printed on a little endian machine\n");
print_bin(1, 4);
printf("\nhtonl(1) ie network byte order (big endian) on a little endian machine\n");
print_bin(htonl(1), 4);
return 0;
}
This is the output
left != right
|00010000|00010000|00010000|00000000
|00010000|00010000|10010000|00000000
Big Endian if on x86
|00000000|00001000|00001000|00001000
|00100001|00100000|00000000|00000000
Little Endian if on x86
|00000000|00010000|00010000|00010000
|00000000|10010000|00010000|00010000
Some notes
0xDEADBEEF printed on a little endian machine
EFBEADDE
0xDEADBEEF printed in network byte order (big endian)
DEADBEEF
1 printed on a little endian machine
|10000000|00000000|00000000|00000000
htonl(1) ie network byte order on a little endian machine
|00000000|00000000|00000000|10000000

Answer 2

The question of Big Endian and Little Endian isn't really known to the machine.
The types in C don't contain such information since it's a Hardware issue, not a type related one.
The machine assumes that all multi-byte numbers are ordered according to it's local endian (on x86, this is usually little endian).
For this reason, bit shifting is always performed using the local endian assumption.
You can't correctly apply bit-shifting to a Big Endian number on a Little Endian machine.
You can't even print a Big Endian number to the screen on a Little Endian machine without getting a funny result.
This is why #Harry's answer was so cool, it prints out each bit, circumventing the issue.
Wikipedia has an article about Endianness with more details.
It should be noted that Endianness actually refers to the way a machine stores it's bytes in the memory.
For example, if the number were as String, Endianness would refer to the question: which "letter" (byte) would come first?
Some machine would store "Hello" and some would store "olleH" (for numbers only, in actual strings, the bytes are always ordered correctly).
Notice that although the order of bytes is reversed, each byte has all the bits ordered the same way, so each byte retains it's value.
When a bit-shift occurs, it always occurs according to the machine's byte ordering system, since this is how it's CPU and memory store are designed.

Answer 3

The accepted answer provides a good sample program. However, I think this sample is a little bit misleading.
The bit string of 1 in little-endian is printed as:
10000000|00000000|00000000|00000000
I ran this code on my x86 pc and I think the results are reliable. But that doesn't mean that the value 1 is stored as printed above in a little-endian machine.
According to the code of print_bin, the num right shifts one bit each time and the least significant bit is printed.
Also, the right shift operator always shifts from the most significant bit (MSB) to the least significant bit (LSB).
In the end, no matter the bit order, the result of print_bin(1, 4) is always the reverse of the human-writing bit representation of 1, which is:
00000000|00000000|00000000|00000001
For example, the bit string may be:
byte significance increase -->
byte
/-------\
00000001|00000000|00000000|00000000
|
bit
<-- bit significance increase
In this example, the bit-order is different from the byte-order. But the results of print_bin(1,4) would be the same.
In other words, the printed bit string doesn't necessarily mean reverse bit-order in the little-endian machine.
I talked about this further in this blog.