I'm reading in HEX values from a file into an array.
The part of the buffer I'm using is contains 4 bytes in Hex -> CE EE 00 00
unsigned int fileLocationOffset = 64;
unsigned char fileSize[4]; //This is actually in a struct.
//Putting here for purposes of this question
unsigned char buff[sizeOfRoot];
fseek(fp, startOfRoot, SEEK_SET); //Seek to point in file fp
fread(buff, 1, sizeOfRoot, fp); //Save contents to a buffer
//Read in 4 Bytes backwards to put as Big-Endian
for(int z = 31; z > 27; z--){
fileSize[31 - z] = buff[fileLocationOffset + z];
}
//TEST: Print Values at each element to see if correct
for(int z = 0; z < 4; z++){
printf("%X ", fileSize[z]);
}
// Output: 0 0 EE CE <- Correct
So, I know that my fileSize array contains the correct values, but now I need to convert 0x00EECE to a decimal.
Could somebody please advise how I should go about doing this?
Ok, so you have the bytes of an int value in big endian order and want to build the value. You could simply place the bytes into an uint32_t variable:
uint32_t bytes_to_int(unsigned char *bytes) {
uint32_t val = 0;
for(int i=0; i<4; i++) {
val <<= 8; //shift 8 positions to the right
val |= *bytes++;
}
return val;
}
With it, the following test program:
int main() {
unsigned char foo[] = { 0, 0, 0xee, 0xce };
unsigned val = bytes_to_int(foo);
printf("%d - 0x%x\n", val, val);
return 0;
}
output as expected:
61134 - 0xeece
What do you do with the data?
Concepts like hex and decimal only apply when you print the data.
What is the output of
printf("%d", *(int *)filesize);
Your previous comment suggests you are on a little endian machine.
The value in your file is also little endian.
Why do you change your endianess?
See following test program:
// big endian
unsigned char fileSize[4] = { 0x0, 0x0, 0xEE, 0xCE };
// little endian
unsigned char fileSize2[4] = { 0xCE, 0xEE, 0x00, 0x00 };
int main(void)
{
int i;
// Output: 0 0 EE CE <- Correct
for (i = 0; i < sizeof fileSize; i++)
printf("%02x ", fileSize[i]);
printf("\n");
for (i = 0; i < sizeof fileSize2; i++)
printf("%02x ", fileSize2[i]);
printf("\n");
printf("%u\n", *(int *)fileSize);
printf("%u\n", *(int *)fileSize2);
}
Output (on pc):
00 00 ee ce
ce ee 00 00
3471704064
61134
You could extract the data as a single unsigned 32-bit integer and use that almost directly (depending on endianness issues of course).
Perhaps something like this:
uint32_t fileSize;
memcpy(&fileSize, buff[fileLocationOffset + 28], 4);
Then for the endianness issue, if you're on Linux you could use be32toh (see e.g. this endian manual page) to convert big-endian to host encoding (and it does nothing if your host system is big-endian):
fileSize = be32toh(fileSize);
The closest function to this in the Windows API is ntohl, which could be used in a similar way.
It's not hard to implement absolute byte-swapping functions, or even macros, for this:
inline uint16_t byteswap16(uint16_t value)
{
return (value & 0xff) << 8 | (value >> 8);
}
inline uint32_t byteswap32(uint32_t value)
{
return byteswap16(value & 0xffff) << 16 | byteswap16(value >> 16);
}
...
fileSize = byteswap32(fileSize);
...
Answer given by Serge Ballesta is correct, however there is one other dirty but short trick to do this.
int main() {
char data[]={0xce,0xee,0,0};
int *ptr;
ptr=data;
printf("%d\n",*ptr);
//if you want to store this result
int x=*ptr;
printf("%d\n",x);
return 0;
}
This way you won't even have to reverse your bytes.
Output:
61134
61134
Here compiler itself takes care of the endianess
Related
I did test a crc-16/ibm implementation i found on the net. when I test it with hex byte array it works fine but if I include some 0x00 values, then it doesn't give the proper result.
here is its code
unsigned short ComputeCRC16(const unsigned char* buf, unsigned int len) {
unsigned short crc = 0;
for (unsigned int j = 0; j < len; j++)
{
unsigned char b = buf[j];
for (unsigned char i = 0; i < 8; i++)
{
crc = ((b ^ (unsigned char)crc) & 1) ? ((crc >> 1) ^ 0xA001) : (crc >> 1);
b >>= 1;
}
}
return crc;
}
I tested it with this code:
int main() {
//fe b5 5f f7
unsigned char buf1[4096] = { 0xfe, 0xb5, 0x5f, 0xf7 };
//fe b5 00 5f f7 00
unsigned char buf2[4096] = { 0xfe, 0xb5, 0x00, 0x5f, 0xf7, 0x00 };
int a = strlen(buf1);
unsigned short res = ComputeCRC16(buf1, a);
printf("res = %04x\n", res); //res : 7858, the result is correct
int b = strlen(buf2);
unsigned short res = ComputeCRC16(buf2, b);
printf("res = %04x\n", res); //res : d781, the result is not correct
return 0; //the correct result : 26EE
}
to verify the result I use this website:
https://www.lammertbies.nl/comm/info/crc-calculation
Your CRC routine gives correct results. It is your test that's wrong. strlen(p) returns how many bytes there are before the first zero byte at p. For buf2, that's four, not five as you intended. For buf1 it's not even defined, since there can be anything in memory after that array. You might be getting four, if the compiler happened to put zeros after the array.
For testing, you should simply provide len manually. (buf1, 4), (buf2, 5).
By the way, that code could be more efficient. It doesn't have to test with b every time. Just exclusive-oring with b to start has the same effect:
crc ^= buf[j];
for (unsigned char i = 0; i < 8; i++)
crc = crc & 1 ? (crc >> 1) ^ 0xa001 : crc >> 1;
I know that to get the number of bytes used by a variable type, you use sizeof(int) for instance. How do you get the value of the individual bytes used when you store a number with that variable type? (i.e. int x = 125.)
You have to know the number of bits (often 8) in each "byte". Then you can extract each byte in turn by ANDing the int with the appropriate mask. Imagine that an int is 32 bits, then to get 4 bytes out of the_int:
int a = (the_int >> 24) & 0xff; // high-order (leftmost) byte: bits 24-31
int b = (the_int >> 16) & 0xff; // next byte, counting from left: bits 16-23
int c = (the_int >> 8) & 0xff; // next byte, bits 8-15
int d = the_int & 0xff; // low-order byte: bits 0-7
And there you have it: each byte is in the low-order 8 bits of a, b, c, and d.
You can get the bytes by using some pointer arithmetic:
int x = 12578329; // 0xBFEE19
for (size_t i = 0; i < sizeof(x); ++i) {
// Convert to unsigned char* because a char is 1 byte in size.
// That is guaranteed by the standard.
// Note that is it NOT required to be 8 bits in size.
unsigned char byte = *((unsigned char *)&x + i);
printf("Byte %d = %u\n", i, (unsigned)byte);
}
On my machine (Intel x86-64), the output is:
Byte 0 = 25 // 0x19
Byte 1 = 238 // 0xEE
Byte 2 = 191 // 0xBF
Byte 3 = 0 // 0x00
You could make use of a union but keep in mind that the byte ordering is processor dependent and is called Endianness http://en.wikipedia.org/wiki/Endianness
#include <stdio.h>
#include <stdint.h>
union my_int {
int val;
uint8_t bytes[sizeof(int)];
};
int main(int argc, char** argv) {
union my_int mi;
int idx;
mi.val = 128;
for (idx = 0; idx < sizeof(int); idx++)
printf("byte %d = %hhu\n", idx, mi.bytes[idx]);
return 0;
}
If you want to get that information, say for:
int value = -278;
(I selected that value because it isn't very interesting for 125 - the least significant byte is 125 and the other bytes are all 0!)
You first need a pointer to that value:
int* pointer = &value;
You can now typecast that to a 'char' pointer which is only one byte, and get the individual bytes by indexing.
for (int i = 0; i < sizeof(value); i++) {
char thisbyte = *( ((char*) pointer) + i );
// do whatever processing you want.
}
Note that the order of bytes for ints and other data types depends on your system - look up 'big-endian' vs 'little-endian'.
This should work:
int x = 125;
unsigned char *bytes = (unsigned char *) (&x);
unsigned char byte0 = bytes[0];
unsigned char byte1 = bytes[1];
...
unsigned char byteN = bytes[sizeof(int) - 1];
But be aware that the byte order of integers is platform dependent.
Say I have a two dimensional array where each entry contains a length and a value:
int array[4][2] = { /* {length, value}, */
{5, 3},
{6, 7},
{1, 0},
{8, 15},
};
I want to store them sequentially into memory with leading zeros to make each field the appropriate length. The example above would be:
00011 000111 0 00001111
The first block is five bits long and stores decimal 3. The second block is six bits long and stores decimal seven. The third block is one bit long and stores decimal 0, and the last block is eight bits long and stores decimal 15.
I can do it with some bitwise manipulation but I thought I would ask to see if there is an easier way.
I am coding in C for a Tensilica 32-bit RISC processor.
The purpose is to write a sequence of Exponential-Golomb codes.
EDIT: SOLUTION:
int main(int argc, char *argv[])
{
unsigned int i = 0, j = 0;
unsigned char bit = 0;
unsigned int bit_num = 0;
unsigned int field_length_bits = 0;
unsigned int field_length_bytes = 0;
unsigned int field_array_length = 0;
unsigned int field_list[NUM_FIELDS][2] = {
/*{Length, Value},*/
{4, 3},
{5, 5},
{6, 9},
{7, 11},
{8, 13},
{9, 15},
{10, 17},
};
unsigned char *seq_array;
// Find total length of field list in bits
for (i = 0; i < NUM_FIELDS; i++)
field_length_bits += field_list[i][LENGTH];
// Number of bytes needed to store FIELD parameters
for (i = 0; i < (field_length_bits + i) % 8 != 0; i++) ;
field_length_bytes = (field_length_bits + i) / 8;
// Size of array we need to allocate (multiple of 4 bytes)
for (i = 0; (field_length_bytes + i) % 4 != 0; i++) ;
field_array_length = (field_length_bytes + i);
// Allocate memory
seq_array = (unsigned char *) calloc(field_array_length, sizeof(unsigned char));
// Traverse source and set destination
for(i = 0; i < NUM_FIELDS; i++)
{
for(j = 0; j < field_list[i][LENGTH]; j++)
{
bit = 0x01 & (field_list[i][VALUE] >> (field_list[i][LENGTH] - j - 1));
if (bit)
setBit(seq_array, field_array_length, bit_num, 1);
else
setBit(seq_array, field_array_length, bit_num, 0);
bit_num++;
}
}
return 0;
}
void setBit(unsigned char *array, unsigned int array_len, unsigned int bit_num, unsigned int bit_value)
{
unsigned int byte_location = 0;
unsigned int bit_location = 0;
byte_location = bit_num / 8;
if(byte_location > array_len - 1)
{
printf("setBit(): Unauthorized memory access");
return;
}
bit_location = bit_num % 8;
if(bit_value)
array[byte_location] |= (1 << (7-bit_location));
else
array[byte_location] &= ~(1 << (7-bit_location));
return;
}
You can use a bitstream library:
Highly recommended bitstream library:
http://cpansearch.perl.org/src/KURIHARA/Imager-QRCode-0.033/src/bitstream.c
http://cpansearch.perl.org/src/KURIHARA/Imager-QRCode-0.033/src/bitstream.h
Because this bitstream library seems to be very self-contained, and doesn't seem to require external includes.
http://www.codeproject.com/Articles/32783/CBitStream-A-simple-C-class-for-reading-and-writin - C library, but using windows WORD, DWORD types (you can still typedef to use this library)
http://code.google.com/p/youtube-mobile-ffmpeg/source/browse/trunk/libavcodec/bitstream.c?r=8 - includes quite a few other include files to use the bitstream library
If you just want exponential golomb codes, there are open-source C implementations:
http://www.koders.com/c/fid8A317DF502A7D61CC96EC4DA07021850B6AD97ED.aspx?s=gcd
Or you can use bit manipulation techniques.
For example:
unsigned int array[4][2] = ???
unsigned int mem[100] = {};
int index=0,bit=0;
for (int i=0;i<4;i++) {
int shift = (32 - array[i][0] - bit);
if (shift>0) mem[index] &= array[i][1] << shift;
else {
mem[index] &= array[i][1] >> -shift;
mem[index+1] &= array[i][1] << (32+shift);
}
bit += array[i][1];
if (bit>=32) {
bit-=32;
index++;
}
}
Disclaimer:
The code only works if your computer byte-order is little endian, and the result will actually be little-endian within each 4-byte boundary, and big-endian across 4-byte boundaries. If you convert mem from int type to char, and replace the constants 32 to 8, you will get a big-endian representation of your bit-array.
It also assumes that the length is less than 32. Obviously, the code you actually want will depend on the bounds of valid input, and what you want in terms of byte-ordering.
Do you mean something like a bit field?
struct myBF
{
unsigned int v1 : 5;
unsigned int v2 : 5;
unsigned int v3 : 1;
unsigned int v4 : 8;
};
struct myBF b = { 3, 7, 0, 15 };
I may be misunderstanding your requirements entirely. Please comment if that's the case.
Update: Suppose you want to do this dynamically. Let's make a function that accepts an array of pairs, like in your example, and an output buffer:
/* Fill dst with bits.
* Returns one plus the number of bytes used or 0 on error.
*/
size_t bitstream(int (*arr)[2], size_t arrlen,
unsigned char * dst, size_t dstlen)
{
size_t total_bits = 0, bits_so_far = 0;
/* Check if there's enough space */
for (size_t i = 0; i != arrlen; ++i) { total_bits += arr[i][0]; }
if (dst == NULL || total_bits > CHAR_BIT * dstlen) { return 0; }
/* Set the output range to all zero */
memset(dst, 0, dstlen);
/* Populate the output range */
for (size_t i = 0; i != arrlen; ++i)
{
for (size_t bits_to_spend = arr[i][0], value = arr[i][1];
bits_to_spend != 0; /* no increment */ )
{
size_t const bit_offset = bits_so_far % CHAR_BIT;
size_t const byte_index = bits_so_far / CHAR_BIT;
size_t const cur_byte_capacity = CHAR_BIT - bit_offset;
/* Debug: Watch it work! */
printf("Need to store %zu, %zu bits to spend, capacity %zu.\n",
value, bits_to_spend, cur_byte_capacity);
dst[byte_index] |= (value << bit_offset);
if (cur_byte_capacity < bits_to_spend)
{
value >>= cur_byte_capacity;
bits_so_far += cur_byte_capacity;
bits_to_spend -= cur_byte_capacity;
}
else
{
bits_so_far += bits_to_spend;
bits_to_spend = 0;
}
}
}
return (bits_so_far + CHAR_BIT - 1) / CHAR_BIT;
}
Notes:
If the number arr[i][1] does not fit into arr[i][0] bits, only the residue modulo 2arr[i][0] is stored.
To be perfectly correct, the array type should be unsigned as well, otherwise the initialization size_t value = arr[i][1] may be undefined behaviour.
You can modify the error handling behaviour. For example, you could forgo transactionality and move the length check into the main loop. Also, instead of returning 0, you could return the numĀber of required bytes, so that the user can figure out how big the destination array needs to be (like snptrintf does).
Usage:
unsigned char dst[N];
size_t n = bitstream(array, sizeof array / sizeof *array, dst, sizeof dst);
for (size_t i = 0; i != n; ++i) { printf("0x%02X ", dst[n - i - 1]); }
For your example, this will produce 0x00 0xF0 0xE3, which is:
0x00 0xF0 0xE3
00000000 11110000 11100011
0000 00001111 0 000111 00011
padd 15 0 7 3
In standard C there's no way to access anything smaller than a char by any way other than the 'bitwise manipulation` you mention. I'm afraid you're out of luck, unless you come across a library somewhere out there that can help you.
I have a buffer of bits with 8 bits of data followed by 1 parity bit. This pattern repeats itself. The buffer is currently stored as an array of octets.
Example (p are parity bits):
0001 0001 p000 0100 0p00 0001 00p01 1100 ...
should become
0001 0001 0000 1000 0000 0100 0111 00 ...
Basically, I need to strip of every ninth bit to just obtain the data bits. How can I achieve this?
This is related to another question asked here sometime back.
This is on a 32 bit machine so the solution to the related question may not be applicable. The maximum possible number of bits is 45 i.e. 5 data octets
This is what I have tried so far. I have created a "boolean" array and added the bits into the array based on the the bitset of the octet. I then look at every ninth index of the array and through it away. Then move the remaining array down one index. Then I've got only the data bits left. I was thinking there may be better ways of doing this.
Your idea of having an array of bits is good. Just implement the array of bits by a 32-bit number (buffer).
To remove a bit from the middle of the buffer:
void remove_bit(uint32_t* buffer, int* occupancy, int pos)
{
assert(*occupancy > 0);
uint32_t high_half = *buffer >> pos >> 1;
uint32_t low_half = *buffer << (32 - pos) >> (32 - pos);
*buffer = high_half | low_half;
--*occupancy;
}
To add a byte to the buffer:
void add_byte(uint32_t* buffer, int* occupancy, uint8_t byte)
{
assert(*occupancy <= 24);
*buffer = (*buffer << 8) | byte;
*occupancy += 8;
}
To remove a byte from the buffer:
uint8_t remove_byte(uint32_t* buffer, int* occupancy)
{
uint8_t result = *buffer >> (*occupancy - 8);
assert(*occupancy >= 8);
*occupancy -= 8;
return result;
}
You will have to arrange the calls so that the buffer never overflows. For example:
buffer = 0;
occupancy = 0;
add_byte(buffer, occupancy, *input++);
add_byte(buffer, occupancy, *input++);
remove_bit(buffer, occupancy, 7);
*output++ = remove_byte(buffer, occupancy);
add_byte(buffer, occupancy, *input++);
remove_bit(buffer, occupancy, 6);
*output++ = remove_byte(buffer, occupancy);
... (there are only 6 input bytes, so this should be easy)
In pseudo-code (since you're not providing any proof you've tried something), I would probably do it like this, for simplicity:
View the data (with parity bits included) as a stream of bits
While there are bits left to read:
Read the next 8 bits
Write to the output
Read one more bit, and discard it
This "lifts you up" from worrying about reading bytes, which no longer is a useful operation since your bytes are interleaved with bits you want to discard.
I have written helper functions to read unaligned bit buffers (this was for AVC streams, see original source here). The code itself is GPL, I'm pasting interesting (modified) bits here.
typedef struct bit_buffer_ {
uint8_t * start;
size_t size;
uint8_t * current;
uint8_t read_bits;
} bit_buffer;
/* reads one bit and returns its value as a 8-bit integer */
uint8_t get_bit(bit_buffer * bb) {
uint8_t ret;
ret = (*(bb->current) >> (7 - bb->read_bits)) & 0x1;
if (bb->read_bits == 7) {
bb->read_bits = 0;
bb->current++;
}
else {
bb->read_bits++;
}
return ret;
}
/* reads up to 32 bits and returns the value as a 32-bit integer */
uint32_t get_bits(bit_buffer * bb, size_t nbits) {
uint32_t i, ret;
ret = 0;
for (i = 0; i < nbits; i++) {
ret = (ret << 1) + get_bit(bb);
}
return ret;
}
You can use the structure like this:
uint_8 * buffer;
size_t buffer_size;
/* assumes buffer points to your data */
bit_buffer bb;
bb.start = buffer;
bb.size = buffer_size;
bb.current = buffer;
bb.read_bits = 0;
uint32_t value = get_bits(&bb, 8);
uint8_t parity = get_bit(&bb);
uint32_t value2 = get_bits(&bb, 8);
uint8_t parity2 = get_bit(&bb);
/* etc */
I must stress that this code is quite perfectible, proper bound checking must be implemented, but it works fine in my use-case.
I leave it as an exercise to you to implement a proper bit buffer reader using this for inspiration.
This also works
void RemoveParity(unsigned char buffer[], int size)
{
int offset = 0;
int j = 0;
for(int i = 1; i + j < size; i++)
{
if (offset == 0)
{
printf("%u\n", buffer[i + j - 1]);
}
else
{
unsigned char left = buffer[i + j - 1] << offset;
unsigned char right = buffer[i + j] >> (8 - offset);
printf("%u\n", (unsigned char)(left | right));
}
offset++;
if (offset == 8)
{
offset = 0;
j++; // advance buffer (8 parity bit consumed)
}
}
}
I know that to get the number of bytes used by a variable type, you use sizeof(int) for instance. How do you get the value of the individual bytes used when you store a number with that variable type? (i.e. int x = 125.)
You have to know the number of bits (often 8) in each "byte". Then you can extract each byte in turn by ANDing the int with the appropriate mask. Imagine that an int is 32 bits, then to get 4 bytes out of the_int:
int a = (the_int >> 24) & 0xff; // high-order (leftmost) byte: bits 24-31
int b = (the_int >> 16) & 0xff; // next byte, counting from left: bits 16-23
int c = (the_int >> 8) & 0xff; // next byte, bits 8-15
int d = the_int & 0xff; // low-order byte: bits 0-7
And there you have it: each byte is in the low-order 8 bits of a, b, c, and d.
You can get the bytes by using some pointer arithmetic:
int x = 12578329; // 0xBFEE19
for (size_t i = 0; i < sizeof(x); ++i) {
// Convert to unsigned char* because a char is 1 byte in size.
// That is guaranteed by the standard.
// Note that is it NOT required to be 8 bits in size.
unsigned char byte = *((unsigned char *)&x + i);
printf("Byte %d = %u\n", i, (unsigned)byte);
}
On my machine (Intel x86-64), the output is:
Byte 0 = 25 // 0x19
Byte 1 = 238 // 0xEE
Byte 2 = 191 // 0xBF
Byte 3 = 0 // 0x00
You could make use of a union but keep in mind that the byte ordering is processor dependent and is called Endianness http://en.wikipedia.org/wiki/Endianness
#include <stdio.h>
#include <stdint.h>
union my_int {
int val;
uint8_t bytes[sizeof(int)];
};
int main(int argc, char** argv) {
union my_int mi;
int idx;
mi.val = 128;
for (idx = 0; idx < sizeof(int); idx++)
printf("byte %d = %hhu\n", idx, mi.bytes[idx]);
return 0;
}
If you want to get that information, say for:
int value = -278;
(I selected that value because it isn't very interesting for 125 - the least significant byte is 125 and the other bytes are all 0!)
You first need a pointer to that value:
int* pointer = &value;
You can now typecast that to a 'char' pointer which is only one byte, and get the individual bytes by indexing.
for (int i = 0; i < sizeof(value); i++) {
char thisbyte = *( ((char*) pointer) + i );
// do whatever processing you want.
}
Note that the order of bytes for ints and other data types depends on your system - look up 'big-endian' vs 'little-endian'.
This should work:
int x = 125;
unsigned char *bytes = (unsigned char *) (&x);
unsigned char byte0 = bytes[0];
unsigned char byte1 = bytes[1];
...
unsigned char byteN = bytes[sizeof(int) - 1];
But be aware that the byte order of integers is platform dependent.